Welcome to Project Tapestry

Project Tapestry is creating a global frontier foundation model with a new "consortium training" platform harnessing data, compute and contributors from around the world to enable sovereign AI.

This repo contains the code and technical documentation for the project. Check out the Project Tapestry website for more information about partnering, events, and more.

The rest of this README provides information for contributors, developers, and users of this repository.

Quick Paths

Note

Make sure to read Getting Involved below for information on contribution guidelines, etc.

Tasks Happening Now

Please join us!

• Help us build a PoC exploring cultural alignment based on the Ingelhart-Wenzel Cultural Map as a metric. This task will implement a corresponding evaluation and do a small model tuning exercise to see if improvement can be observed. Prior expertise in evaluation and tuning technologies are especially welcome.
• Help us flesh out the details of consortium training, the adaptation of federated learning that meets the unique requirements and constraints of Tapestry. Prior expertise in LLM training and federated learning technologies are especially welcome.
• Tell us about your data sets. What unique datasets do you have that could contribute to Tapestry model training? They don't have to be fully open; we will work with you to define and enforce appropriate handling.

Working with the Source Code

The source code is under the src directory.

• Use the Makefile targets, e.g., make help. More details are in Development below.
• Runnable demos in examples/ (try make consortium-demo).
• Consortium training prototype in src/tapestry/training/consortium/ (try make consortium-demo and also make consortium-tests).

Working with the Technical Documentation

The technical documentation lives under tech-docs:

• Architecture
  • The TVA methodology: phased outputs (stakeholder map through design goals), architectural options and core thesis, plus:
    • Architecture Decision Records
    • Diagrams
• Governance
• Work Groups
• Strategic Plan
• Reference Materials (e.g. training paradigms)

For repo layout, conventions, and where to find implementation code, see AGENTS.md.

Development

Setup

This project uses uv for Python package management.

Install uv

On macOS/Linux:

curl -LsSf https://astral.sh/uv/install.sh | sh

On Windows:

powershell -c "irm https://astral.sh/uv/install.ps1 | iex"

The rest of the steps discussed next are automated using make. Try the following:

make one-time-setup

Create a Virtual Environment

The one-time-setup target runs the following command (but it only works on macOS or Linux). You can also do this manually:

On macOS/Linux:

uv venv
source .venv/bin/activate

On Windows:

uv venv
.venv\Scripts\activate

Install Dependencies

The one-time-setup target runs the first of the following commands (but it only works on macOS or Linux). You can also run either command manually:

uv pip install -e ".[dev]"  # full development dependencies
uv pip install -e .         # minimum dependencies

Running Tests

We use unittest and hypothesis for testing. The easiest way to run the test suite is using make:

make unit-tests # or just tests; they are currently the same.

This runs the following commands, which you can run yourself if you prefer:

cd src
uv run python -m unittest discover \
    --pattern 'test_*.py' \
    --start-directory tests \
    --top-level-directory .

Code Formatting

Use either of the following commands to format the Python code with black:

make format
# or
uv run black src

Linting

Use either of the following commands to lint the Python code with ruff and pylint:

make lint
# or
uv run ruff check src
uv pylint src

Type Checking

Use either of the following commands to type check the Python code with ty:

make type-check
# or
uv run ty src

There is also a "watch" option that keeps ty running as you fix mistakes and save the files:

make type-check-watch
# or
uv run ty --watch src

Before You Submit a PR...

Before submitting a PR, please run the format, lint, and type checking commands, then run the tests. Make sure everything passes cleanly! Use the convenient make target before-pr, or run the individual commands above:

make before-pr               # Equivalent to 'make format lint type-check tests'
make format-lint-type-check  # Equivalent to 'make format lint type-check'

Note

Make sure to read Getting Involved below before submitting a PR.

Project Code Structure

In addition to the top-level directories tech-docs, discussed above, and docs, discussed below, the code structure is as follows. At this time, there are three major subsystems:

• data for all data governance and management capabilities.
• training for all distributed training and tuning capabilities.
• infrastructure for all underlying infrastructure.

tapestry/
├── src/
│   └── tapestry/
│       └── data/
│       └── infrastructure/
│       └── training/
│   └── tests
│       └── tapestry/
│           └── data/
│           └── infrastructure/
│           └── training/

Getting Involved

We welcome contributions as pull requests, issues, and discussions.

You can also join one or more work groups that are being organized to identify requirements in several areas and to start the engineering work to prototype and test ideas, followed by the initial implementation iterations. Details are are being documented in tech-docs/work-groups/.

See the AI Alliance CONTRIBUTING guidelines. You will need to agree with the AI Alliance Code of Conduct.

Licenses

All code contributions are licensed under the Apache 2.0 LICENSE (which is also in this repo, LICENSE.Apache-2.0).

All documentation contributions are licensed under the Creative Commons Attribution 4.0 International (which is also in this repo, LICENSE.CC-BY-4.0).

All data contributions are licensed under the Community Data License Agreement - Permissive - Version 2.0 (which is also in this repo, LICENSE.CDLA-2.0).

We use the "Developer Certificate of Origin" (DCO).

Warning

Before you make any git commits with changes, understand what's required for DCO.

See the Alliance contributing guide section on DCO for details. In practical terms, supporting this requirement means you must use the -s flag with your git commit commands.

About the Technical Website (GitHub Pages)

The website for this repository provides another way to discover and navigate the technical documentation content in tech-docs. However, at this time, the site mostly just points to the content in tech-docs. The website sources are in the docs directory.

The website is published using GitHub Pages, where the pages are written in Markdown and served using Jekyll. See GITHUB_PAGES.md for all the details.