[BUG] version 3 todos

[davidkoski]

TODOs for the version 3 API:

Download jinja files

calls to downloader should also download jinja files, e.g. ModelAdapterFactory:

            adapterDirectory = try await downloader.download(
                id: id, revision: revision,
                matching: ["*.safetensors", "*.json"],

should be:

            ["*.safetensors", "*.json", "*.jinja"]

Note: some already do

package let tokenizerDownloadPatterns = ["*.json", "*.jinja"]
package let modelDownloadPatterns = ["*.safetensors"] + tokenizerDownloadPatterns

Macros need fully qualified names to avoid collisions

fully qualify the type names in the macros, e.g. Progress -> Foundation.Progress

The LoRATrainingExample has a local "Progress" type that collides inside the macro.

MLXEmbedders has colliding ModelConfiguration, etc.

In embedder-tool there are uses of:

• ModelConfiguration
• ModelContainer

I think we got away with this before because the code was only importing MLXEmbedders. Now, in order to provide Downloader and Tokenizer we have to import MLXLMCommon, which has a different definition.

In order to compile, the code would need fully qualified names for these types to disambiguate.

Ideas:

• can we just use the MLXLMCommon form? I know we looked at that earlier and it wasn't trivial, but it might be the best way
• use a different name in MLXEmbedders -- more breaking change, so not ideal
• do we need to import MLXLMCommon -- maybe re-export types? this might be sweeping it under the rug

[davidkoski]

See also ml-explore/mlx-swift-examples#468

FYI @DePasqualeOrg I can work on these.

[davidkoski]

Honor ModelFactory

The loadModel*() code no longer takes a ModelFactory -- this used to be a method on ModelFactory (I think). Anyway, the load() function can end up loading a VLM model for an LLM (many models can run in either). We need a way to control that I think.

public func loadModelContainer(
    from downloader: any Downloader,
    using tokenizerLoader: any TokenizerLoader,
    configuration: ModelConfiguration,
    useLatest: Bool = false,
    progressHandler: @Sendable @escaping (Progress) -> Void = { _ in }
) async throws -> sending ModelContainer {
    try await load {
        try await $0.loadContainer(
            from: downloader, using: tokenizerLoader, configuration: configuration,
            useLatest: useLatest, progressHandler: progressHandler)
    }
}


private func load<R>(loader: (ModelFactory) async throws -> sending R) async throws -> sending R {
    let factories = ModelFactoryRegistry.shared.modelFactories()
    var lastError: Error?
    for factory in factories {

[DePasqualeOrg]

loadModel and loadModelContainer didn't take a ModelFactory before #118.

Before:
loadModel
loadModelContainer

After:
loadModel
loadModelContainer

In llm-tool/LLMTool.swift you can do this:

// Current (broken, ignores modelFactory)
func load(defaultModel: String, modelFactory: ModelFactory) async throws -> ModelContainer {
    ...
    return try await loadModelContainer(
        from: self.downloader,
        using: #huggingFaceTokenizerLoader(),
        configuration: modelConfiguration)
}

// Fixed: use the factory directly
func load(defaultModel: String, modelFactory: ModelFactory) async throws -> ModelContainer {
    ...
    return try await modelFactory.loadContainer(
        from: self.downloader,
        using: #huggingFaceTokenizerLoader(),
        configuration: modelConfiguration)
}

[davidkoski]

Perfect, that works and is roughly what it did before -- I didn't see the function on ModelFactory for some reason.