Add detailed documentation for updating GTNet

[ajtritt]

• Add documentation on the implicit dependencies on deep-taxon code.
  • How confidence features are calculated in deep-taxon and how they are used in GTNet
    • https://github.com/exabiome/gtnet/blob/d25bd39027980b8ec3de20963790ff745fd79a88/src/gtnet/predict.py#L172
    • https://github.com/exabiome/deep-taxon/blob/a42fc6a02bca25b36ce8d04746b464d344d822a0/src/deep_taxon/nn/summarize.py#L1168
    • https://github.com/exabiome/deep-taxon/blob/a42fc6a02bca25b36ce8d04746b464d344d822a0/src/deep_taxon/nn/infer.py#L422
• Steps for reproducing model
  • Preparing training data
    • Files needed from GTDB
      • Metadata files i.e. *_metadata*.tar.gz from e.g. https://data.ace.uq.edu.au/public/gtdb/data/releases/release207/207.0/
    • Data retrieval (deep-taxon ncbi-fetch): https://github.com/exabiome/deep-taxon/blob/a42fc6a02bca25b36ce8d04746b464d344d822a0/src/deep_taxon/gtdb/download.py#L93
    • Formatting data (deep-taxon prepare-data): https://github.com/exabiome/deep-taxon/blob/a42fc6a02bca25b36ce8d04746b464d344d822a0/src/deep_taxon/gtdb/prepare_data.py#L109
      • Also include details on formatting for representative data and calibration data
  • Running training:
    • Training command (deep-taxon train): https://github.com/exabiome/deep-taxon/blob/a42fc6a02bca25b36ce8d04746b464d344d822a0/src/deep_taxon/nn/train.py#L473
    • Submitting training jobs (deep taxon train-job): https://github.com/exabiome/deep-taxon/blob/a42fc6a02bca25b36ce8d04746b464d344d822a0/src/deep_taxon/run/run_job.py#L14
  • Running inference:
• Building confidence model:
  • Inference command (deep-taxon infer): https://github.com/exabiome/deep-taxon/blob/a42fc6a02bca25b36ce8d04746b464d344d822a0/src/deep_taxon/nn/infer.py#L446
  • Submitting inference jobs (deep-taxon infer-job): https://github.com/exabiome/deep-taxon/blob/a42fc6a02bca25b36ce8d04746b464d344d822a0/src/deep_taxon/run/run_infer.py#L12
  • Building confidence model with inference output (deep-taxon conf-model): https://github.com/exabiome/deep-taxon/blob/a42fc6a02bca25b36ce8d04746b464d344d822a0/src/deep_taxon/nn/summarize.py#L1130
• Building deployment package:
  • Aggregating models and metadata for GTNet deployment (deep-taxon deploy-pkg): https://github.com/exabiome/deep-taxon/blob/a42fc6a02bca25b36ce8d04746b464d344d822a0/src/deep_taxon/nn/deploy.py#L163
  • Adding deployment package to GTNet
    • Uploading to OSF
    • GTNet lines that need to be updated:
      • OSF link: https://github.com/exabiome/gtnet/blob/d25bd39027980b8ec3de20963790ff745fd79a88/src/gtnet/utils.py#L37
      • checksum: https://github.com/exabiome/gtnet/blob/d25bd39027980b8ec3de20963790ff745fd79a88/src/gtnet/utils.py#L39