Add Hub-free TokenizersCore product

[aleroot]

Summary

Adds a lightweight TokenizersCore product for applications that provide their own downloader and tokenizer configuration files.

Currently, using Tokenizers also compiles the Hub integration and its transitive networking dependencies, including swift-huggingface, EventSource, Crypto, and potentially SwiftNIO—even when the application never uses them.

With this change, applications can depend on:

.product(name: "TokenizersCore", package: "swift-transformers")

and import:

import TokenizersCore

TokenizersCore provides tokenizer construction from tokenizer_config.json and tokenizer.json data without compiling or linking the Hub and networking targets.

The existing Tokenizers product remains source-compatible and continues to provide Hub-backed APIs such as:

AutoTokenizer.from(pretrained:)
AutoTokenizer.from(modelFolder:)

Implementation

• Adds the TokenizersCore product.
• Moves shared configuration types into TokenizerConfig.
• Keeps Hub-dependent loading APIs in the existing Tokenizers compatibility product.
• Adds a focused test verifying that Config and tokenizer APIs are exposed through TokenizersCore.
• Documents how clients with their own downloader can select the lightweight product.

SwiftPM limitation

SwiftPM still resolves package-level dependencies declared by swift-transformers. However, when only TokenizersCore is selected, Hub, HuggingFace, EventSource, Crypto, yyjson, and SwiftNIO are not compiled or linked into the application.