fresh start on makefile cleanup, esp. tests and cleanup

[turbomam]

I would like to help clean up the EnvO Makefiles, so I know that i can thoroughly clean, thoroughly rebuild and thoroughly test. I am not sure what I can deliver or whether this seems valuable to others. I certainly don't want to disrupt the workflow others are used to.

If I pursue this, it will replace

• clean makefile target. make all works after using it. #1603
• Seems like src/envo/Makefile would benefit from a clean target #1602

[turbomam]

I didn't make much progress yesterday. I probably haven't defined the current problems well enough, nor identified the return on investment. May even want to give up on

• Claude's solutions for cleanup and testing #1626

Here are the EnvO Makefiles:

• ./Makefile
• ./mappings/Makefile
• ./src/envo/diffs/Makefile
• ./src/envo/envo.Makefile
• ./src/envo/Makefile
• ./src/envo/mappings/Makefile
• ./src/envo/sources/Makefile

[turbomam]

to run validate_mappings in the root ./Makefile:

docker run -m 14g -v $PWD:/work -w /work --rm -ti obolibrary/odkfull make validate_mappings

that is also run by .github/workflows/sssom-qc.yml

It only takes mappings/*.sssom.tsv as input, but there are several other files in mappings/

[turbomam]

there's also mappings/Makefile and src/envo/mappings/Makefile which run different processes on a different subset of files

Filename	Root Makefile	mappings/Makefile
geonames-mapping.tsv	No	Yes
geonames-unmapped.tsv	No	Yes
gold-mapping.tsv	No	No
lter-mapping.tsv	No	Yes
lter-unmapped.tsv	No	Yes
spire-mapping.tsv	No	Yes
spire-mapping-by-type.tsv	No	Yes
spire-unmapped.tsv	No	Yes
sweet-envo_cryosphere.sssom.tsv	Yes	No
sweetrealm-mapping.tsv	No	Yes
sweetrealm-unmapped.tsv	No	Yes

I haven't been able to get the target in those to run. They both depend on a relative ../envo.obo and ../sources/ (which isn't available to mappings/Makefile. They also depend on blip-findall from https://github.com/cmungall/blipkit, which doesn't appear to be installed into the obolibrary/odkfull Docker image... at least on the default path

[turbomam]

I can run all of this, with no errors from src/envo in the current master branch.

./run.sh make -B all_assets all_imports all_main all_modules all_reports all_subsets \
continuous_integration_test copy-release debug.owl envo-base.owl envo-edit-module-merged.owl \
envo-edit.csv envo-edit.owl envo-full.owl envo.owl matches matches_annotations modules/tags.owl release \
reports/envo-base-robot-report.tsv reports/envo-edges.tsv reports/envo-edit-robot-report.tsv \
reports/envo-src-robot-report.tsv reports/envo-stats.tsv reports/envo-subsetTable.tsv \
reports/envo-synonyms.tsv reports/envo-tags.tsv reports/envo-termTable.tsv show_assets sparql_test \
test_python tmp/tags-dosdp.owl tmp/tags-dosdp.tsv travis_test validate-dl-profile

Many of those targets are meta-targets that call one or more of the targets on that list, so it's kinda redundant

I haven't been able to run these

• fill
• is_utf
• mappings/gold-mapping.txt
• obo-check
• release-diffs
• sparql_test2

I think that accounts for all non-pattern targets in ./src/envo/envo.Makefile and ./src/envo/Makefile

sparql_test2 may be the only one I really care about working on

[turbomam]

I would like to add a cleanup target at some point, like below, but I think rerunning the make command above can't recreate all of these without some inter-dependencies

.PHONY: clean

clean:
	# Remove common release artifacts
	rm -f ../../envo.{owl,obo,json}
	rm -f ../../envo-base.owl
	rm -f ../../imports/*_import.{owl,obo}
	rm -f ../../subsets/*.{owl,obo,json,tsv}
	rm -f imports/*_import.{owl,obo}
	rm -f envo.{owl,obo,json}
	rm -f envo-base.owl
	rm -f envo-simple.owl
	rm -f envo-edit.csv
	rm -f debug.owl

	# Remove module outputs
	rm -f modules/*.owl

	# Remove DOSDP outputs
	rm -f patterns/matches/logical/*.tsv
	rm -rf patterns/matches/annotations
	rm -f tmp/tags-dosdp.{tsv,owl}

	# Remove reports
	rm -f reports/envo-*.tsv

[turbomam]

First error below. No, I am not going to down the rabbit hole now

./run.sh make -B all_assets all_imports all_main all_modules all_reports all_subsets continuous_integration_test copy-release debug.owl envo-base.owl envo-edit-module-merged.owl envo-edit.csv envo-edit.owl envo-full.owl envo.owl matches matches_annotations modules/tags.owl release reports/envo-base-robot-report.tsv reports/envo-edges.tsv reports/envo-edit-robot-report.tsv reports/envo-src-robot-report.tsv reports/envo-stats.tsv reports/envo-subsetTable.tsv reports/envo-synonyms.tsv reports/envo-tags.tsv reports/envo-termTable.tsv show_assets sparql_test test_python tmp/tags-dosdp.owl tmp/tags-dosdp.tsv travis_test validate-dl-profile

owltools http://purl.obolibrary.org/obo/pato.owl --remove-annotation-assertions -l -s -d --remove-axiom-annotations --remove-dangling-annotations --make-subset-by-properties -f BFO:0000050 BFO:0000051 RO:0002202 immediate_transformation_of RO:0002176 IAO:0000136 --set-ontology-id http://purl.obolibrary.org/obo/pato.owl -o mirror/pato.owl
2025-05-21 17:01:43,831 WARN  (CommandRunnerBase:233) Could not find an OWLObjectProperty for id: 'immediate_transformation_of'
2025-05-21 17:01:43,858 WARN  (CommandRunnerBase:233) Could not find an OWLObjectProperty for id: 'IAO:0000136'
owltools --catalog-xml catalog-v001.xml envo-edit.owl   --remove-import-declaration http://purl.obolibrary.org/obo/envo/modules/entity_attribute_location.owl  --remove-import-declaration http://purl.obolibrary.org/obo/envo/modules/entity_quality_location.owl  --remove-import-declaration http://purl.obolibrary.org/obo/envo/modules/entity_attribute.owl  --remove-import-declaration http://purl.obolibrary.org/obo/envo/modules/process_attribute.owl  --remove-import-declaration http://purl.obolibrary.org/obo/envo/modules/chemical_concentration.owl -o envo-edit-module-merged.owl.tmp
2025-05-21 17:01:46,824 ERROR (CommandRunner:4815) could not parse:envo-edit.owl
org.semanticweb.owlapi.model.UnloadableImportException: Could not load imported ontology: <http://purl.obolibrary.org/obo/envo/modules/chemical_concentration.owl> Cause: /work/src/envo/modules/chemical_concentration.owl (No such file or directory)
        at uk.ac.manchester.cs.owl.owlapi.OWLOntologyManagerImpl.makeLoadImportRequest(OWLOntologyManagerImpl.java:1870)
        at org.semanticweb.owlapi.functional.parser.OWLFunctionalSyntaxParser.Ontology(OWLFunctionalSyntaxParser.java:231)
        at org.semanticweb.owlapi.functional.parser.OWLFunctionalSyntaxParser.parse(OWLFunctionalSyntaxParser.java:171)
        at org.semanticweb.owlapi.functional.parser.OWLFunctionalSyntaxOWLParser.parse(OWLFunctionalSyntaxOWLParser.java:92)
        at uk.ac.manchester.cs.owl.owlapi.OWLOntologyFactoryImpl.loadOWLOntology(OWLOntologyFactoryImpl.java:220)
        at uk.ac.manchester.cs.owl.owlapi.OWLOntologyManagerImpl.actualParse(OWLOntologyManagerImpl.java:1254)
        at uk.ac.manchester.cs.owl.owlapi.OWLOntologyManagerImpl.loadOntology(OWLOntologyManagerImpl.java:1208)
        at uk.ac.manchester.cs.owl.owlapi.OWLOntologyManagerImpl.loadOntology(OWLOntologyManagerImpl.java:1108)
        at uk.ac.manchester.cs.owl.owlapi.OWLOntologyManagerImpl.loadOntology(OWLOntologyManagerImpl.java:1064)
        at owltools.io.ParserWrapper.parseOWL(ParserWrapper.java:163)
        at owltools.io.ParserWrapper.parseOWL(ParserWrapper.java:150)
        at owltools.io.ParserWrapper.parse(ParserWrapper.java:132)
        at owltools.cli.CommandRunner.runSingleIteration(CommandRunner.java:4803)
        at owltools.cli.CommandRunnerBase.run(CommandRunnerBase.java:76)
        at owltools.cli.CommandRunnerBase.run(CommandRunnerBase.java:68)
        at owltools.cli.CommandLineInterface.main(CommandLineInterface.java:12)
Caused by: org.semanticweb.owlapi.io.OWLOntologyCreationIOException: /work/src/envo/modules/chemical_concentration.owl (No such file or directory)
        at uk.ac.manchester.cs.owl.owlapi.OWLOntologyFactoryImpl.loadOWLOntology(OWLOntologyFactoryImpl.java:230)
        at uk.ac.manchester.cs.owl.owlapi.OWLOntologyManagerImpl.actualParse(OWLOntologyManagerImpl.java:1254)
        at uk.ac.manchester.cs.owl.owlapi.OWLOntologyManagerImpl.loadOntology(OWLOntologyManagerImpl.java:1208)
        at uk.ac.manchester.cs.owl.owlapi.OWLOntologyManagerImpl.loadOntology(OWLOntologyManagerImpl.java:1108)
        at uk.ac.manchester.cs.owl.owlapi.OWLOntologyManagerImpl.loadImports(OWLOntologyManagerImpl.java:1825)
        at uk.ac.manchester.cs.owl.owlapi.OWLOntologyManagerImpl.makeLoadImportRequest(OWLOntologyManagerImpl.java:1863)
        ... 15 more
Caused by: java.io.FileNotFoundException: /work/src/envo/modules/chemical_concentration.owl (No such file or directory)
        at java.base/java.io.FileInputStream.open0(Native Method)
        at java.base/java.io.FileInputStream.open(FileInputStream.java:219)
        at java.base/java.io.FileInputStream.<init>(FileInputStream.java:157)
        at java.base/java.io.FileInputStream.<init>(FileInputStream.java:112)
        at java.base/sun.net.www.protocol.file.FileURLConnection.connect(FileURLConnection.java:86)
        at java.base/sun.net.www.protocol.file.FileURLConnection.getInputStream(FileURLConnection.java:184)
        at org.semanticweb.owlapi.io.AbstractOWLParser.getInputStreamFromContentEncoding(AbstractOWLParser.java:179)
        at org.semanticweb.owlapi.io.AbstractOWLParser.getInputStream(AbstractOWLParser.java:141)
        at org.semanticweb.owlapi.io.AbstractOWLParser.getInputSource(AbstractOWLParser.java:264)
        at org.semanticweb.owlapi.rdf.rdfxml.parser.RDFXMLParser.parse(RDFXMLParser.java:72)
        at uk.ac.manchester.cs.owl.owlapi.OWLOntologyFactoryImpl.loadOWLOntology(OWLOntologyFactoryImpl.java:220)
        ... 20 more
make: *** [Makefile:121: envo-edit-module-merged.owl] Error 1