Skip to content

Commit 634a372

Browse files
000specificclaude
andcommitted
annotations: reformat Annotation_Definitions to 'definition ==accession'
Change the annogroup definition format from 'accession=definition' to 'definition ==accession' (literal ' ==' separator: space + two equals signs), so the column reads as human text with the Pfam accession appended for provenance. E.g. 'PF00001=7 transmembrane receptor (rhodopsin family)' becomes '7 transmembrane receptor (rhodopsin family) ==PF00001'. Accessions also still stand alone in the Annotation_Accessions column. BLOCK_annotations_X_ocl (the formatter / source of truth): - utils_run_summary.format_annotation_definitions now emits 'def ==acc' (missing definition -> '==acc'); docstrings + the self-documenting column headers in scripts 001-004 updated to match. - Existing RUN_02 outputs (the canonical, server-published run) were backfilled in place by a one-off idempotent migration: 945 of 1155 definition-bearing TSVs rewritten across all 105 structures (the 210 untouched are the empty 'zero'-subtype definition columns); full re-scan confirms 0 untransformed pairs. A pipeline re-run would reproduce the identical text. integrator/BLOCK_annotations_X_orthogroups: - Script 003 no longer strips the accession; it passes the upstream OCL definitions through verbatim (so Table 1 inherits 'def ==acc'). RUN_1 re-run, validation PASS, re-published. Also fixes a §39 canonical-RUN violation surfaced while investigating: the OCL data server was being fed the stale RUN_01 (2026-05-05, pre-definitions) because RUN_01 kept a live upload_manifest.tsv while the canonical RUN_02's manifest sat at template defaults (all 'no'). RUN_02's manifest is now switched on (21 yes rows), the stale RUN_01 manifest removed, and the server re-published from RUN_02 (32,130 files; 16,695 stale RUN_01 symlinks cleaned). The server now serves the canonical run with definitions in the new format. (RUN manifests + run outputs are gitignored runtime; only the COPYME source scripts are committed here.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 583fdad commit 634a372

7 files changed

Lines changed: 22 additions & 40 deletions

File tree

gigantic_project-COPYME/subprojects/integrator/BLOCK_annotations_X_orthogroups/workflow-COPYME-annotations_X_orthogroups/ai/scripts/003_ai-python-build_annogroup_X_orthogroups.py

Lines changed: 4 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -71,27 +71,6 @@ def load_protein_to_orthogroup( orthogroups_path: Path ) -> dict:
7171
return proteins___orthogroups
7272

7373

74-
def strip_accession_prefixes( definitions: str ) -> str:
75-
"""
76-
Drop the leading 'ACCESSION=' from each semicolon-delimited definition pair,
77-
keeping only the human-readable definition text (the accessions remain in the
78-
separate Annotation_Accessions column).
79-
80-
'PF00001=7 transmembrane receptor (rhodopsin family); PF13927=Immunoglobulin domain'
81-
-> '7 transmembrane receptor (rhodopsin family); Immunoglobulin domain'
82-
"""
83-
if not definitions:
84-
return definitions
85-
cleaned_definitions = []
86-
for pair in definitions.split( ';' ):
87-
pair = pair.strip()
88-
if not pair:
89-
continue
90-
# split on the FIRST '=' only — definition text may itself contain '='
91-
cleaned_definitions.append( pair.split( '=', 1 )[ 1 ] if '=' in pair else pair )
92-
return '; '.join( cleaned_definitions )
93-
94-
9574
def load_annogroup_annotations( summary_path: Path ) -> dict:
9675
"""
9776
annogroup_id -> ( subtype, accessions, definitions ) from the all-types OCL
@@ -182,7 +161,7 @@ def main():
182161
"Annogroup_ID (annogroup identifier format annogroup_pfam_N)",
183162
"Annogroup_Subtype (single or combo)",
184163
"Annotation_Accessions (comma delimited pfam accessions for this annogroup)",
185-
"Annotation_Definitions (semicolon delimited pfam definition text for this annogroup; accession prefixes removed, accessions are in Annotation_Accessions)",
164+
"Annotation_Definitions (semicolon delimited definition ==accession pairs for this annogroup passed through from the upstream annogroup OCL summary)",
186165
"Annogroup_Species_Count (number of unique species in the annogroup)",
187166
"Annogroup_Member_Protein_Count (number of member proteins in the annogroup)",
188167
"Members_With_Orthogroup_Count (count of member proteins that map to an orthogroup)",
@@ -255,9 +234,9 @@ def main():
255234
subtype_from_summary, accessions, definitions = annogroups___annotations.get(
256235
annogroup_id, ( subtype, "", "" )
257236
)
258-
# Definitions carry only the human-readable text — the pfam
259-
# accessions live in the Annotation_Accessions column.
260-
definitions = strip_accession_prefixes( definitions )
237+
# Definitions are passed through verbatim from the upstream
238+
# annogroup OCL summary, which formats them as 'definition ==accession'
239+
# (semicolon delimited); accessions also stand alone in Annotation_Accessions.
261240

262241
all_ogs = sorted( orthogroup_ids )
263242
output = '\t'.join( [

gigantic_project-COPYME/subprojects/integrator/BLOCK_annotations_X_orthogroups/workflow-RUN_1-annotations_X_orthogroups-run_summary.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Workflow Run Summary: species70_pfam_X_OrthoHMM
22

3-
**Status**: ✅ **SUCCESS (completed 2026-06-17 20:05:41)**
3+
**Status**: ✅ **SUCCESS (completed 2026-06-17 22:49:34)**
44

55
**Run label**: `species70_pfam_X_OrthoHMM`
66
**Species set**: `species70`

gigantic_project-COPYME/subprojects/ocl_phylogenetic_structures/BLOCK_annotations_X_ocl/workflow-COPYME-ocl_analysis/ai/scripts/001_ai-python-create_annogroups.py

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
# AI: Claude Code | Opus 4.6 | 2026 April 18 | Purpose: Create annotation groups (annogroups) from annotation files and prepare inputs for OCL analysis
2-
# AI: Claude Code | Opus 4.8 | 2026 June 05 | Purpose: Capture per-accession descriptions and emit Annotation_Definitions (acc=def) alongside Annotation_Accessions on annogroup outputs
2+
# AI: Claude Code | Opus 4.8 | 2026 June 05 | Purpose: Capture per-accession descriptions and emit Annotation_Definitions (def ==acc) alongside Annotation_Accessions on annogroup outputs
33
# Human: Eric Edsinger
44

55
"""
@@ -926,7 +926,7 @@ def write_annogroup_map( annogroup_map_entries ):
926926
output += 'Annogroup_Subtype (single or combo or zero)\t'
927927
output += 'Annotation_Database (name of annotation database)\t'
928928
output += 'Annotation_Accessions (comma delimited annotation accessions from the database e.g. Pfam PF00069 or unannotated identifier for zero subtype)\t'
929-
output += 'Annotation_Definitions (semicolon delimited accession=definition pairs where definition is the InterProScan signature description e.g. PF00069=Protein kinase domain)\t'
929+
output += 'Annotation_Definitions (semicolon delimited definition ==accession pairs where definition is the InterProScan signature description e.g. Protein kinase domain ==PF00069)\t'
930930
output += 'Species_Count (number of unique species with at least one member sequence)\t'
931931
output += 'Sequence_Count (total number of member sequences)\t'
932932
output += 'Species_List (comma delimited list of species names as Genus_species)\t'
@@ -968,7 +968,7 @@ def write_annogroups_standardized( annogroup_map_entries ):
968968
output += 'Species_Count (number of unique species in annogroup)\t'
969969
output += 'Species_List (comma delimited list of species names as Genus_species)\t'
970970
output += 'Annotation_Accessions (comma delimited annotation accessions from the database e.g. Pfam PF00069 or unannotated identifier for zero subtype)\t'
971-
output += 'Annotation_Definitions (semicolon delimited accession=definition pairs where definition is the InterProScan signature description e.g. PF00069=Protein kinase domain)\n'
971+
output += 'Annotation_Definitions (semicolon delimited definition ==accession pairs where definition is the InterProScan signature description e.g. Protein kinase domain ==PF00069)\n'
972972
output_file.write( output )
973973

974974
for entry in annogroup_map_entries:
@@ -998,7 +998,7 @@ def write_per_subtype_annogroup_files( subtypes___annogroup_data ):
998998
output += 'Species_List (comma delimited list of species names as Genus_species)\t'
999999
output += 'Sequence_IDs (comma delimited list of sequence identifiers)\t'
10001000
output += 'Annotation_Accessions (comma delimited annotation accessions from the database e.g. Pfam PF00069 or unannotated identifier for zero subtype)\t'
1001-
output += 'Annotation_Definitions (semicolon delimited accession=definition pairs where definition is the InterProScan signature description e.g. PF00069=Protein kinase domain)\n'
1001+
output += 'Annotation_Definitions (semicolon delimited definition ==accession pairs where definition is the InterProScan signature description e.g. Protein kinase domain ==PF00069)\n'
10021002
output_file.write( output )
10031003

10041004
for entry in annogroup_entries:

gigantic_project-COPYME/subprojects/ocl_phylogenetic_structures/BLOCK_annotations_X_ocl/workflow-COPYME-ocl_analysis/ai/scripts/002_ai-python-determine_origins.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -777,7 +777,7 @@ def write_annogroup_origins( annogroup_origins, clade_id_names___phylogenetic_bl
777777
'Species_Count (total unique species in annogroup)',
778778
'Species_List (comma delimited list of species in annogroup)',
779779
'Annotation_Accessions (comma delimited annotation accessions from the database e.g. Pfam PF00069 or unannotated identifier for zero subtype)',
780-
'Annotation_Definitions (semicolon delimited accession=definition pairs where definition is the InterProScan signature description e.g. PF00069=Protein kinase domain)'
780+
'Annotation_Definitions (semicolon delimited definition ==accession pairs where definition is the InterProScan signature description e.g. Protein kinase domain ==PF00069)'
781781
]
782782

783783
# Single-row header

gigantic_project-COPYME/subprojects/ocl_phylogenetic_structures/BLOCK_annotations_X_ocl/workflow-COPYME-ocl_analysis/ai/scripts/003_ai-python-quantify_conservation_loss.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -814,7 +814,7 @@ def write_annogroup_patterns( annogroup_patterns, annogroup_ids___annotation_col
814814
'Continued_Absence_Events (count of phylogenetic blocks in block-state X where annogroup is absent at both parent and child after an upstream loss)',
815815
'Species_List (comma delimited list of all species containing this annogroup)',
816816
'Annotation_Accessions (comma delimited annotation accessions from the database e.g. Pfam PF00069 or unannotated identifier for zero subtype)',
817-
'Annotation_Definitions (semicolon delimited accession=definition pairs where definition is the InterProScan signature description e.g. PF00069=Protein kinase domain)'
817+
'Annotation_Definitions (semicolon delimited definition ==accession pairs where definition is the InterProScan signature description e.g. Protein kinase domain ==PF00069)'
818818
]
819819

820820
# Write single-row header

gigantic_project-COPYME/subprojects/ocl_phylogenetic_structures/BLOCK_annotations_X_ocl/workflow-COPYME-ocl_analysis/ai/scripts/004_ai-python-comprehensive_ocl_analysis.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -903,7 +903,7 @@ def write_annogroup_summaries( annogroup_summaries, output_file_path, annogroup_
903903
'Continued_Absence_Events (count of phylogenetic blocks in block-state X where annogroup is absent at both parent and child after an upstream loss)',
904904
'Species_List (comma delimited list of all species containing this annogroup)',
905905
'Annotation_Accessions (comma delimited annotation accessions from the database e.g. Pfam PF00069 or unannotated identifier for zero subtype)',
906-
'Annotation_Definitions (semicolon delimited accession=definition pairs where definition is the InterProScan signature description e.g. PF00069=Protein kinase domain)'
906+
'Annotation_Definitions (semicolon delimited definition ==accession pairs where definition is the InterProScan signature description e.g. Protein kinase domain ==PF00069)'
907907
]
908908

909909
csv_writer.writerow( header_columns )

gigantic_project-COPYME/subprojects/ocl_phylogenetic_structures/BLOCK_annotations_X_ocl/workflow-COPYME-ocl_analysis/ai/scripts/utils_run_summary.py

Lines changed: 10 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -139,9 +139,9 @@ def clear_fragments_directory( workflow_directory ):
139139
# Script 001 writes the authoritative annogroup map with two annotation columns:
140140
# Annotation_Accessions -- comma delimited database accessions (e.g. Pfam PF00069)
141141
# or the unannotated identifier for the zero subtype
142-
# Annotation_Definitions -- semicolon delimited accession=definition pairs, where
142+
# Annotation_Definitions -- semicolon delimited "definition ==accession" pairs, where
143143
# definition is the InterProScan signature description
144-
# (e.g. "PF00069=Protein kinase domain; PF00400=WD40 repeat")
144+
# (e.g. "Protein kinase domain ==PF00069; WD40 repeat ==PF00400")
145145
#
146146
# Downstream scripts (002, 003, 004) carry these two columns onto every output
147147
# that bears an Annogroup_ID by looking them up here by Annogroup_ID. They are
@@ -155,7 +155,7 @@ def sanitize_annotation_text( text ):
155155
156156
Tabs and newlines would break the column/row structure, so they are
157157
collapsed to spaces. Commas, semicolons and '=' are left intact (the
158-
accession=definition; ... format relies on them and TSV only splits on
158+
"definition ==accession" ... format relies on them and TSV only splits on
159159
tabs). Returns the cleaned string.
160160
"""
161161
if text is None:
@@ -179,9 +179,12 @@ def format_annotation_definitions( accessions, accessions___descriptions ):
179179
accessions___descriptions: dict mapping accession -> definition text.
180180
181181
Returns:
182-
str: semicolon delimited 'accession=definition' pairs over the UNIQUE
183-
accessions, e.g. 'PF00069=Protein kinase domain; PF00400=WD40 repeat'.
184-
Missing definitions render as 'accession=' (empty definition).
182+
str: semicolon delimited 'definition ==accession' pairs over the UNIQUE
183+
accessions, e.g. 'Protein kinase domain ==PF00069; WD40 repeat ==PF00400'.
184+
Note the literal ' ==' separator (space + two equals signs); the
185+
definition comes first so the column reads as human text with the
186+
accession appended for provenance. Missing definitions render as
187+
'==accession' (no leading text).
185188
"""
186189
seen_accessions = set()
187190
pairs = []
@@ -190,7 +193,7 @@ def format_annotation_definitions( accessions, accessions___descriptions ):
190193
continue
191194
seen_accessions.add( accession )
192195
definition = sanitize_annotation_text( accessions___descriptions.get( accession, '' ) )
193-
pairs.append( f"{accession}={definition}" )
196+
pairs.append( f"{definition} =={accession}" if definition else f"=={accession}" )
194197
return '; '.join( pairs )
195198

196199

0 commit comments

Comments
 (0)