[chore] Update TODO list with completed tasks and new considerations

Author: sroussey

TODO.md

@@ -9,10 +9,10 @@ TODO.md
 - [ ] Datasets Package
   - [x] Documents dataset (mabye rename to DocumentDataset)
   - [ ] Chunks Package (or part of DocumentDataset?)
-  - [ ] Move Model repository to datasets package.
-- [ ] Chunks and nodes are not always the same.
-  - [ ] And we may need to save the chunk's node path. Or paths? or document range? Standard metadata?
-- [ ] Instead of passing doc_id around, pass a document key that is unknonwn (string or object)
+  - [x] Move Model repository to datasets package.
+- [x] Chunks and nodes are not always the same.
+  - [x] And we may need to save the chunk's node path. Or paths? or document range? Standard metadata?
+- [ ] Instead of passing doc_id around, pass a document key that is of type unknonwn (string or object)
 
 - [ ] Get a better model for question answering.
 - [ ] Get a better model for named entity recognition, the current one recognized everything as a token, not helpful.
@@ -25,16 +25,11 @@ TODO.md
 - [ ] rename the registration stuff to not look ugly: registerHuggingfaceTransformers() and registerHuggingfaceTransformersUsingWorkers() and registerHuggingfaceTransformersInsideWorker()
 - [ ] fix image transferables
 
-onnx-community/ModernBERT-finetuned-squad-ONNX - summarization
+- [ ] Consider different ways to connect tasks to queues. What is a task? What is a job?
+
+- [ ] Input and outputs are all scalar, arrays, or unions. But what about streams? Stream of items in an array, stream of content for a scalar like a string, etc.
 
-- [x] Auto-generated primary keys for TabularStorage
-  - [x] Schema annotation with `x-auto-generated: true`
-  - [x] Type system with `InsertEntity` for optional auto-generated keys
-  - [x] Support for autoincrement (integer) and UUID (string) strategies
-  - [x] Configurable client-provided keys: "never", "if-missing", "always"
-  - [x] Implementations: InMemory, SQLite, Postgres, Supabase, IndexedDB, FsFolder
-  - [x] Comprehensive test suite (342 tests pass)
-  - [x] Documentation updated
+onnx-community/ModernBERT-finetuned-squad-ONNX - summarization
 
 Rework the Document Dataset. Currently there is a Document storage of tabular storage type, and that should be registered as a "dataset:document:source" meaning the source material in node format. And there is already a "dataset:document-chunk" for the chunk/vector storage which should be registered as a "dataset:document:chunk" with a well defined metadata schema. The two combined should be registered as a "dataset:document" which is the complete document with its source and all its chunks and metadata. This is for convenience but not used by tasks or ai tasks.