diff --git a/content/community/roadmap-and-release-notes/986-v1.9.2-release-notes.md b/content/community/roadmap-and-release-notes/986-v1.9.2-release-notes.md new file mode 100644 index 0000000..6eaf5f1 --- /dev/null +++ b/content/community/roadmap-and-release-notes/986-v1.9.2-release-notes.md @@ -0,0 +1,28 @@ +--- +title: Bacalhau v1.9.2 Release Notes +description: Bacalhau v1.9.2 propagates compute execution errors, hardens release gates, and consolidates dependency maintenance. +--- + +# Bacalhau v1.9.2 Release Notes + +Bacalhau v1.9.2 was released on July 19, 2026. This maintenance release improves execution error reporting and release reliability while updating core Go and Python dependencies. + +## Compute execution error propagation + +Errors reported by a compute execution are now propagated through the job execution path instead of being lost at the compute boundary. This gives operators and callers a clearer failure signal when a workload cannot complete successfully. + +## Release and dependency maintenance + +The release also: + +- hardens weekly release gates and tag creation; +- consolidates weekly dependency updates; +- updates the NATS Go client and Python client dependencies; and +- improves compatibility for CI gateway tests. + +## Release artifacts and complete change history + +- [Bacalhau v1.9.2 release](https://github.com/bacalhau-project/bacalhau/releases/tag/v1.9.2) +- [Complete v1.9.1 to v1.9.2 comparison](https://github.com/bacalhau-project/bacalhau/compare/v1.9.1...v1.9.2) + +Install or upgrade Bacalhau with the instructions in the [installation guide](/docs/getting-started/installation/). diff --git a/content/community/roadmap-and-release-notes/987-v1.7.0-release-notes.md b/content/community/roadmap-and-release-notes/987-v1.7.0-release-notes.md index f7122d8..dc69679 100644 --- a/content/community/roadmap-and-release-notes/987-v1.7.0-release-notes.md +++ b/content/community/roadmap-and-release-notes/987-v1.7.0-release-notes.md @@ -6,11 +6,11 @@ description: Enterprise-Ready Distributed Computing We are thrilled to announce the release of Bacalhau v1.7.0, marking a significant milestone in our journey to make distributed computing accessible and enterprise-ready. This release introduces enterprise support options, enhanced job management capabilities, significant performance improvements, and a host of new features that make it easier than ever for organizations to deploy and manage distributed compute networks at scale. -### Enterprise Support through Expanso +### Optional Commercial Support -Bacalhau v1.7.0 introduces enterprise-grade support options through a flexible licensing model. Organizations can now purchase licenses for their on-premises Bacalhau networks. This offering ensures that enterprises have access to professional support, enabling them to confidently deploy and maintain their distributed compute infrastructure. +Bacalhau v1.7.0 introduced license-handling features for optional commercial support offerings. The Bacalhau project and software remain open source under the Apache 2.0 license; any commercial license applies only to separately contracted support terms. -Our tiered licensing model provides flexibility for organizations of all sizes, from growing startups to large enterprises, ensuring they receive the level of support that matches their needs. +Optional support terms can be sized for organizations ranging from small teams to large enterprises without changing the open-source license for Bacalhau itself. ### Enhanced Job Execution with Partitioned Workloads @@ -85,11 +85,11 @@ We've completely rebuilt our documentation with a clear, task-based organization #### Enterprise Support -**Q: What license tiers are available?** A: Bacalhau offers a free tier supporting up to 5 nodes, making it easy for teams to get started. Beyond that, organizations can purchase licenses based on their node count requirements. +**Q: What license tiers are available?** A: Bacalhau itself is open-source software under Apache 2.0. Optional commercial support terms are provided separately and may vary by node count. **Q: What happens if my network exceeds the licensed node count?** A: Your network will continue to operate normally even if you exceed your licensed node count. The system will display a friendly notice about the overage, giving you time to adjust your license as needed. -**Q: How do I purchase a license?** A: Licenses can be purchased by contacting our sales team at [support@expanso.io](mailto:support@expanso.io). +**Q: How do I purchase commercial support?** A: Use the commercial support link on the [Bacalhau home page](/) for the current contact path. **Q: How are licenses deployed?** A: Once a user receives their license, they can deploy it to their network. Only the orchestrator needs the license deployed, eliminating the need to manage licenses on compute nodes or clients. A new configuration option allows you to specify the local license path on the orchestrator. diff --git a/content/community/social-media.md b/content/community/social-media.md index 70473c7..7060a7b 100644 --- a/content/community/social-media.md +++ b/content/community/social-media.md @@ -8,7 +8,7 @@ Bacalhau has a presence on all major social media networks, so you can follow us ## YouTube -The [Bacalhau](https://www.youtube.com/@bacalhauproject) and [Expanso](https://www.youtube.com/@ExpansoIO) YouTube channels are home to a wealth of information about the Bacalhau project — everything from developer demos, educational videos, events, etc. You can explore and learn more about Bacalhau. +The [Bacalhau YouTube channel](https://www.youtube.com/@bacalhauproject) hosts developer demos, educational videos, event recordings, and other information about the project. ## Blog @@ -16,11 +16,11 @@ Our blog contains product updates, company news, and educational content on how ## X (Formerly Twitter) -You can follow our X accounts [@BacalhauProject](https://x.com/BacalhauProject) and [@ExpansoIO](https://x.com/ExpansoIO) to get Bacalhau news, product updates, etc in tweet-sized bites. +Follow [@BacalhauProject](https://x.com/BacalhauProject) for project news and updates. ## LinkedIn -Connect with us on [LinkedIn](https://www.linkedin.com/company/expanso-io) to stay informed about the latest news, industry trends, and exclusive job opportunities! +Follow the [Bacalhau Project on LinkedIn](https://www.linkedin.com/showcase/bacalhau-project/posts) for project news and community updates. ## Need Support? diff --git a/content/use-cases/use-cases/distributed-data-warehousing.mdx b/content/use-cases/use-cases/distributed-data-warehousing.mdx index 4b05f0b..4dd001f 100644 --- a/content/use-cases/use-cases/distributed-data-warehousing.mdx +++ b/content/use-cases/use-cases/distributed-data-warehousing.mdx @@ -1,16 +1,11 @@ --- +title: Distributed Data Warehousing description: >- Efficiently query and analyze data across multiple regions by deploying compute tasks directly where your data resides, reducing latency, enhancing performance and ensuring compliance. --- -import Head from '@docusaurus/Head'; - - - - - # Distributed Data Warehousing ## Overview diff --git a/content/use-cases/use-cases/distributed-machine-learning.mdx b/content/use-cases/use-cases/distributed-machine-learning.mdx index 4a60e72..06b039c 100644 --- a/content/use-cases/use-cases/distributed-machine-learning.mdx +++ b/content/use-cases/use-cases/distributed-machine-learning.mdx @@ -1,16 +1,11 @@ --- +title: Distributed Machine Learning description: >- Train and deploy machine learning models across a distributed compute fleet, optimizing performance, reducing data movement, and enabling large-scale parallelism. --- -import Head from '@docusaurus/Head'; - - - - - # Distributed Machine Learning ## Overview diff --git a/content/use-cases/use-cases/fleet-management.mdx b/content/use-cases/use-cases/fleet-management.mdx index a6e33e1..3537535 100644 --- a/content/use-cases/use-cases/fleet-management.mdx +++ b/content/use-cases/use-cases/fleet-management.mdx @@ -1,16 +1,11 @@ --- +title: Fleet Management description: >- Efficiently manage and operate a large fleet of distributed nodes with remote execution, real-time monitoring, targeted job execution, and automated software updates. --- -import Head from '@docusaurus/Head'; - - - - - # Fleet Management ## Overview diff --git a/content/use-cases/use-cases/log-processing.mdx b/content/use-cases/use-cases/log-processing.mdx index 43dfbce..1137f76 100644 --- a/content/use-cases/use-cases/log-processing.mdx +++ b/content/use-cases/use-cases/log-processing.mdx @@ -1,15 +1,10 @@ --- +title: Log Processing description: >- Process logs efficiently at scale by running distributed jobs directly at the source, reducing costs, improving real-time insights, and enhancing security. --- -import Head from '@docusaurus/Head'; - - - - - # Log Processing ## Overview diff --git a/docs/api/README.md b/docs/api/README.md index f7c478e..d4ad608 100644 --- a/docs/api/README.md +++ b/docs/api/README.md @@ -1,5 +1,7 @@ --- sidebar_position: 10 +title: Bacalhau HTTP API Reference +description: Use the versioned Bacalhau HTTP API to submit and inspect jobs and manage nodes programmatically. --- # Overview diff --git a/docs/api/agent.md b/docs/api/agent.md index 955db59..0926dee 100644 --- a/docs/api/agent.md +++ b/docs/api/agent.md @@ -1,3 +1,8 @@ +--- +title: Agent API +description: Query Bacalhau agent health, capabilities, node identity, and version through the HTTP API. +--- + # Agent The Bacalhau Agent APIs provide a convenient means to retrieve information about the Bacalhau node you are communicating with, whether it serves as the orchestrator or functions as a compute node. These APIs offer valuable insights into the node's health, capabilities, and deployed Bacalhau version. diff --git a/docs/cli/README.md b/docs/cli/README.md index 5c62df5..ca3ff7f 100644 --- a/docs/cli/README.md +++ b/docs/cli/README.md @@ -1,5 +1,7 @@ --- sidebar_position: 01 +title: Bacalhau CLI Reference +description: Learn the Bacalhau command structure, global flags, output formats, configuration paths, and connection settings. --- # Overview diff --git a/docs/cli/agent/README.md b/docs/cli/agent/README.md index 3d6922e..fea9d68 100644 --- a/docs/cli/agent/README.md +++ b/docs/cli/agent/README.md @@ -1,3 +1,8 @@ +--- +title: Agent Commands +description: Use the bacalhau agent command group to inspect agent health, version, and node information. +--- + # Agent The `bacalhau agent` command is a parent command that offers sub-commands to query information about the Bacalhau agent. This can be useful for debugging, monitoring, or managing the agent's behavior and health. diff --git a/docs/cli/agent/alive.md b/docs/cli/agent/alive.md index 0af4120..f8e5088 100644 --- a/docs/cli/agent/alive.md +++ b/docs/cli/agent/alive.md @@ -1,3 +1,7 @@ +--- +description: Check whether a Bacalhau agent is responding and retrieve its current liveness and health information. +--- + # Alive ## Description diff --git a/docs/cli/agent/node.md b/docs/cli/agent/node.md index 7baaf8a..dff1c55 100644 --- a/docs/cli/agent/node.md +++ b/docs/cli/agent/node.md @@ -1,3 +1,8 @@ +--- +title: Agent Node Command +description: Retrieve the local Bacalhau agent node information in table, JSON, or YAML format. +--- + # Node ## Description diff --git a/docs/cli/config/list.md b/docs/cli/config/list.md index 0f074fc..853aa2e 100644 --- a/docs/cli/config/list.md +++ b/docs/cli/config/list.md @@ -1,3 +1,7 @@ +--- +description: List the configuration keys and values that a Bacalhau node will apply on its next restart. +--- + # Config List ## Description diff --git a/docs/cli/job/describe.md b/docs/cli/job/describe.md index 4de2550..5a1acb5 100644 --- a/docs/cli/job/describe.md +++ b/docs/cli/job/describe.md @@ -1,3 +1,8 @@ +--- +title: Describe a Job +description: Retrieve the complete specification and current status of a Bacalhau job by its identifier. +--- + # Describe ## Description diff --git a/docs/cli/job/executions.md b/docs/cli/job/executions.md index 21f7fc2..fb5878e 100644 --- a/docs/cli/job/executions.md +++ b/docs/cli/job/executions.md @@ -1,3 +1,7 @@ +--- +description: List the executions and states associated with a Bacalhau job. +--- + # Executions ## Description diff --git a/docs/cli/job/get.md b/docs/cli/job/get.md index a23f64f..aaa15b6 100644 --- a/docs/cli/job/get.md +++ b/docs/cli/job/get.md @@ -1,3 +1,7 @@ +--- +description: Download a Bacalhau job's published results and retrieve its standard output and standard error streams. +--- + # Get ## Description @@ -44,4 +48,4 @@ bacalhau job get [id] [flags] 2. **Get the results of a job, using a short ID**: - `bacalhau job get 51225160` \ No newline at end of file + `bacalhau job get 51225160` diff --git a/docs/cli/job/history.md b/docs/cli/job/history.md index 0f89be9..7a15e10 100644 --- a/docs/cli/job/history.md +++ b/docs/cli/job/history.md @@ -1,3 +1,7 @@ +--- +description: Inspect the ordered history of state changes and execution events for a Bacalhau job. +--- + # History ## Description diff --git a/docs/cli/job/list.md b/docs/cli/job/list.md index 06713ec..b17a009 100644 --- a/docs/cli/job/list.md +++ b/docs/cli/job/list.md @@ -1,3 +1,8 @@ +--- +title: List Jobs +description: List submitted Bacalhau jobs and control filtering, sorting, pagination, and output formatting. +--- + # List ## Description diff --git a/docs/cli/job/logs.md b/docs/cli/job/logs.md index 06afe1c..0fa7c29 100644 --- a/docs/cli/job/logs.md +++ b/docs/cli/job/logs.md @@ -1,3 +1,7 @@ +--- +description: Retrieve logs from a running or completed Bacalhau job for monitoring and debugging. +--- + # Logs ## Description diff --git a/docs/cli/job/run.md b/docs/cli/job/run.md index d1329fa..0d57749 100644 --- a/docs/cli/job/run.md +++ b/docs/cli/job/run.md @@ -1,3 +1,7 @@ +--- +description: Submit a Bacalhau job from YAML, JSON, standard input, or command-line arguments and control template values. +--- + # Run ## Description @@ -310,7 +314,7 @@ Tasks: Engine: Type: docker Params: - Image: expanso/nginx-access-log-processor:1.0.0 + Image: ghcr.io/bacalhau-project/nginx-access-log-query:1.0.0 Parameters: - --query - { { .query } } @@ -347,7 +351,7 @@ Tasks: Engine: Type: docker Params: - Image: expanso/nginx-access-log-processor:1.0.0 + Image: ghcr.io/bacalhau-project/nginx-access-log-query:1.0.0 Parameters: - --query - { { .query } } diff --git a/docs/cli/job/stop.md b/docs/cli/job/stop.md index ae2e260..c7004c0 100644 --- a/docs/cli/job/stop.md +++ b/docs/cli/job/stop.md @@ -1,3 +1,7 @@ +--- +description: Stop a previously submitted Bacalhau job by its identifier. +--- + # Stop ## Description diff --git a/docs/cli/node/README.md b/docs/cli/node/README.md index 366b174..6b1216f 100644 --- a/docs/cli/node/README.md +++ b/docs/cli/node/README.md @@ -1,3 +1,8 @@ +--- +title: Node Commands +description: Use the bacalhau node command group to list, inspect, approve, reject, and delete compute nodes. +--- + # Node The `bacalhau node` command provides a set of sub-commands to query and manage node-related information within Bacalhau. With these tools, users can access specific details about nodes, list all network nodes, and more. diff --git a/docs/cli/node/describe.md b/docs/cli/node/describe.md index 439acdd..2c48d19 100644 --- a/docs/cli/node/describe.md +++ b/docs/cli/node/describe.md @@ -1,3 +1,8 @@ +--- +title: Describe a Node +description: Retrieve detailed state, capabilities, and configuration information for a Bacalhau node by its identifier. +--- + # Describe The `bacalhau node describe` command offers users the ability to retrieve detailed information about a specific node using its unique identifier. diff --git a/docs/cli/node/list.md b/docs/cli/node/list.md index 1868d47..0b84588 100644 --- a/docs/cli/node/list.md +++ b/docs/cli/node/list.md @@ -1,3 +1,8 @@ +--- +title: List Nodes +description: List Bacalhau network nodes with filters, ordering, and machine-readable output formats. +--- + # List The `bacalhau node list` command is designed to provide users with a comprehensive list of network nodes along with details based on specified flags. diff --git a/docs/components/engines/README.mdx b/docs/components/engines/README.mdx index d72d493..6dae2ce 100644 --- a/docs/components/engines/README.mdx +++ b/docs/components/engines/README.mdx @@ -1,5 +1,7 @@ --- slug: /engines +title: Execution Engines +description: Configure the Docker and WebAssembly execution engines that Bacalhau compute nodes use to run distributed workloads. --- # Engines diff --git a/docs/components/publishers/README.mdx b/docs/components/publishers/README.mdx index 439478a..7c1d6b4 100644 --- a/docs/components/publishers/README.mdx +++ b/docs/components/publishers/README.mdx @@ -1,5 +1,7 @@ --- slug: /publishers +title: Result Publishers +description: Publish Bacalhau job results to local storage, Amazon S3 or compatible object storage, and IPFS. --- # Publishers diff --git a/docs/components/publishers/ipfs.md b/docs/components/publishers/ipfs.md index 64c3866..44b780b 100644 --- a/docs/components/publishers/ipfs.md +++ b/docs/components/publishers/ipfs.md @@ -1,5 +1,7 @@ --- slug: /publishers/ipfs +title: IPFS Result Publisher +description: Publish Bacalhau task results to IPFS for content-addressed, decentralized retrieval. --- # IPFS diff --git a/docs/components/publishers/local.md b/docs/components/publishers/local.md index 7322eeb..675b6c1 100644 --- a/docs/components/publishers/local.md +++ b/docs/components/publishers/local.md @@ -1,5 +1,7 @@ --- slug: /publishers/local +title: Local Result Publisher +description: Store Bacalhau task results on the compute node for local development and testing. --- # Local diff --git a/docs/components/publishers/s3.md b/docs/components/publishers/s3.md index 2d5867b..4e962a8 100644 --- a/docs/components/publishers/s3.md +++ b/docs/components/publishers/s3.md @@ -1,5 +1,7 @@ --- slug: /publishers/s3 +title: S3 Result Publisher +description: Publish Bacalhau task results to Amazon S3 or another S3-compatible object storage service. --- # S3 diff --git a/docs/components/sources/README.mdx b/docs/components/sources/README.mdx index 61bf7d2..0a6fbc0 100644 --- a/docs/components/sources/README.mdx +++ b/docs/components/sources/README.mdx @@ -1,5 +1,7 @@ --- slug: /sources +title: Input Sources +description: Mount local files, HTTP URLs, S3-compatible object storage, and IPFS data as inputs to Bacalhau jobs. --- # Sources diff --git a/docs/components/sources/ipfs.md b/docs/components/sources/ipfs.md index 25fd7ef..3f280a9 100644 --- a/docs/components/sources/ipfs.md +++ b/docs/components/sources/ipfs.md @@ -1,5 +1,7 @@ --- slug: /sources/ipfs +title: IPFS Input Source +description: Fetch files and directories by IPFS content identifier and mount them as inputs to a Bacalhau task. --- # IPFS diff --git a/docs/components/sources/local.md b/docs/components/sources/local.md index 603f825..009d18e 100644 --- a/docs/components/sources/local.md +++ b/docs/components/sources/local.md @@ -1,5 +1,7 @@ --- slug: /sources/local +title: Local Input Source +description: Mount files and directories already present on a Bacalhau compute node as task inputs. --- # Local diff --git a/docs/components/sources/s3.md b/docs/components/sources/s3.md index 06291c3..2e8874d 100644 --- a/docs/components/sources/s3.md +++ b/docs/components/sources/s3.md @@ -1,5 +1,7 @@ --- slug: /sources/s3 +title: S3 Input Source +description: Mount objects and prefixes from Amazon S3 or an S3-compatible service as inputs to Bacalhau jobs. --- # S3 diff --git a/docs/guides/queueing.md b/docs/guides/queueing.md index 5d50dd9..359cc20 100644 --- a/docs/guides/queueing.md +++ b/docs/guides/queueing.md @@ -1,3 +1,7 @@ +--- +description: Configure how long Bacalhau jobs wait for compute nodes that satisfy capacity and placement constraints. +--- + # Queueing ## Introduction diff --git a/docs/guides/timeouts.md b/docs/guides/timeouts.md index 8ae5dcb..f1e0c75 100644 --- a/docs/guides/timeouts.md +++ b/docs/guides/timeouts.md @@ -1,3 +1,8 @@ +--- +title: Configure Job Timeouts +description: Configure Bacalhau job timeouts to stop stalled workloads, protect compute resources, and control execution duration. +--- + # Timeouts This guide explains how Bacalhau uses timeouts to manage job execution duration, protecting both users and compute resources from problematic jobs. diff --git a/docs/guides/webui.md b/docs/guides/webui.md index 3f90286..274d1f5 100644 --- a/docs/guides/webui.md +++ b/docs/guides/webui.md @@ -1,3 +1,7 @@ +--- +description: Enable and use the Bacalhau web interface to monitor jobs, executions, and nodes in a deployment. +--- + # Web UI ## Overview diff --git a/docs/references/help-and-faq/faqs.md b/docs/references/help-and-faq/faqs.md index 4a4fcd6..a078ba6 100644 --- a/docs/references/help-and-faq/faqs.md +++ b/docs/references/help-and-faq/faqs.md @@ -1,9 +1,35 @@ --- slug: /references/faq +title: Bacalhau Frequently Asked Questions +description: Direct answers about compute over data, Bacalhau architecture, workloads, data locality, execution engines, nodes, and job operations. --- # FAQs +## What is compute over data? + +Compute over data is an architecture that runs processing near the systems where data is generated or stored. It reduces unnecessary data transfer by sending workload instructions to the data location and returning only the required results. + +## What is Bacalhau? + +Bacalhau is an open-source distributed compute orchestration framework. An orchestrator schedules jobs, and compute nodes execute those jobs close to their input data across edge, on-premises, and cloud environments. + +## When should I use Bacalhau? + +Use Bacalhau when moving raw data to a central compute system is slow, expensive, unreliable, or restricted. Common workloads include distributed log processing, fleet operations, regional analytics, machine learning, and processing data at the edge. + +## Does Bacalhau require data to move to a central cluster? + +No. Bacalhau can schedule a workload on compute nodes near local or remote data sources. A job can process data in place and publish only its output, although the exact data movement depends on the input source, job specification, and selected compute node. + +## Is Bacalhau open source? + +Yes. The Bacalhau source code is available on [GitHub](https://github.com/bacalhau-project/bacalhau) under the [Apache 2.0 license](https://github.com/bacalhau-project/bacalhau/blob/main/LICENSE). + +## Which workload formats can Bacalhau run? + +Bacalhau supports Docker containers and WebAssembly modules. Existing containerized tools can usually run without being rewritten, while WebAssembly provides a lightweight execution option for compatible workloads. + ## How do I restrict what nodes my jobs run on? You can describe each node with labels in a `key=value` format, which can later be used as conditions for choosing nodes to run your jobs on: diff --git a/docs/references/operators/gpu.md b/docs/references/operators/gpu.md index b8127c7..b824800 100644 --- a/docs/references/operators/gpu.md +++ b/docs/references/operators/gpu.md @@ -1,3 +1,7 @@ +--- +description: Configure Bacalhau compute nodes to run Docker workloads on NVIDIA, AMD, or Intel GPUs. +--- + # GPU Workloads ## Overview diff --git a/docs/references/operators/node-management.md b/docs/references/operators/node-management.md index 3999fd3..0fc375d 100644 --- a/docs/references/operators/node-management.md +++ b/docs/references/operators/node-management.md @@ -1,3 +1,7 @@ +--- +description: Manage Bacalhau orchestrator and compute node registration, approval, health, and cluster membership. +--- + # Node Management ## Overview diff --git a/docs/references/operators/security/auth.md b/docs/references/operators/security/auth.md index 2321b45..20ec00a 100644 --- a/docs/references/operators/security/auth.md +++ b/docs/references/operators/security/auth.md @@ -1,3 +1,7 @@ +--- +description: Configure HTTP basic authentication, API tokens, OAuth 2.0 single sign-on, and authorization for Bacalhau APIs. +--- + # Authentication & Authorization ## Introduction diff --git a/docs/specifications/job/timeouts.md b/docs/specifications/job/timeouts.md index bc34c8a..374527b 100644 --- a/docs/specifications/job/timeouts.md +++ b/docs/specifications/job/timeouts.md @@ -1,3 +1,8 @@ +--- +title: Job Timeout Specification +description: Define execution time limits for Bacalhau tasks with the Timeouts object in a job specification. +--- + # Timeouts The `Timeouts` object provides a mechanism to impose timing constraints on specific task operations, particularly execution. By setting these timeouts, users can ensure tasks don't run indefinitely and align them with intended durations. diff --git a/docs/specifications/job/type.md b/docs/specifications/job/type.md index 26c96c0..dbd1613 100644 --- a/docs/specifications/job/type.md +++ b/docs/specifications/job/type.md @@ -291,7 +291,7 @@ Tasks: Params: # Image: docker image to be used in the task. - Image: expanso/nginx-access-log-processor:1.0.0 + Image: ghcr.io/bacalhau-project/nginx-access-log-query:1.0.0 # Entrypoint defines a command that will be executed when container starts. @@ -483,7 +483,7 @@ Tasks: Params: # Image: docker image to be used in the task. - Image: expanso/nginx-access-log-agent:1.0.0 + Image: ghcr.io/bacalhau-project/nginx-access-log-agent:1.0.0 # Entrypoint defines a command that will be executed when container starts. diff --git a/docs/specifications/other/README.md b/docs/specifications/other/README.md index d07f55f..885debf 100644 --- a/docs/specifications/other/README.md +++ b/docs/specifications/other/README.md @@ -1,2 +1,6 @@ -# Other Specifications +--- +title: Supporting Specifications +description: Reference the shared state and configuration types used across Bacalhau job and node specifications. +--- +# Other Specifications diff --git a/docusaurus.config.ts b/docusaurus.config.ts index c463640..f338115 100644 --- a/docusaurus.config.ts +++ b/docusaurus.config.ts @@ -93,6 +93,30 @@ const config: Config = { ], headTags: [ + { + tagName: 'script', + attributes: { + type: 'application/ld+json', + }, + innerHTML: JSON.stringify({ + '@context': 'https://schema.org', + '@type': 'SoftwareSourceCode', + name: 'Bacalhau', + url: 'https://bacalhau.org/', + codeRepository: 'https://github.com/bacalhau-project/bacalhau', + license: 'https://www.apache.org/licenses/LICENSE-2.0', + programmingLanguage: 'Go', + description: + 'Open-source distributed compute orchestration software that runs workloads near the data they process.', + keywords: [ + 'compute over data', + 'distributed computing', + 'data locality', + 'edge computing', + 'workload orchestration', + ], + }), + }, { tagName: 'link', attributes: { @@ -178,11 +202,6 @@ const config: Config = { label: 'Blog', position: 'right', }, - { - href: 'https://expanso.io/?_gl=1*sdzh2w*_gcl_au*ODM0MTE4NTkyLjE3NDIyOTQ5MDQ.*_ga*ODgxNjg0Mjg3LjE3NDIyOTQ5MDQ.*_ga_X1RJ0QGN3Z*czE3NDY1OTkyNDkkbzI5JGcxJHQxNzQ2NjAzMzIxJGoxMCRsMCRoMA..', - label: 'Enterprise', - position: 'right', - }, { href: 'https://github.com/bacalhau-project/bacalhau', position: 'right', @@ -257,19 +276,6 @@ const config: Config = { }, ], }, - { - title: 'Expanso', - items: [ - { - label: 'Expanso.io', - href: 'https://expanso.io', - }, - { - label: 'Enterprise Support', - href: 'https://expanso.io/contact', - }, - ], - }, ], copyright: `Copyright © ${new Date().getFullYear()} Bacalhau Project.`, }, diff --git a/package.json b/package.json index 4d8b72c..4ba8826 100644 --- a/package.json +++ b/package.json @@ -5,7 +5,7 @@ "scripts": { "docusaurus": "docusaurus", "start": "docusaurus start", - "build": "docusaurus build", + "build": "docusaurus build && node scripts/copy-legacy-machine-routes.mjs", "swizzle": "docusaurus swizzle", "clear": "docusaurus clear", "serve": "docusaurus serve", diff --git a/redirects.ts b/redirects.ts index 3a1e7a8..7c00a1e 100644 --- a/redirects.ts +++ b/redirects.ts @@ -1,6 +1,69 @@ import type { RedirectItem } from '@docusaurus/plugin-client-redirects/lib/types' -const redirects: RedirectItem[] = [] +// docs.bacalhau.org currently forwards every legacy path to the same path under +// bacalhau.org/docs. These redirect pages resolve the legacy paths that moved +// to a different section of the current documentation. +const redirects: RedirectItem[] = [ + { from: '/docs/getting-started/cli/overview', to: '/docs/cli' }, + { from: '/docs/getting-started/cli', to: '/docs' }, + { from: '/docs/getting-started/architecture', to: '/docs/overview/architecture' }, + { from: '/docs/common-workflows/publishing-and-retrieving-results', to: '/docs/guides/publishing-results' }, + { from: '/docs/common-workflows/resource-constraints', to: '/docs/guides/resource-requirements' }, + { from: '/docs/common-workflows/', to: '/docs' }, + { from: '/docs/common-pitfalls', to: '/docs/references/troubleshooting' }, + { from: '/docs/guides/using-labels-and-constraints', to: '/docs/guides/labels-and-constraints' }, + { from: '/docs/use-cases/readme/log-processing', to: '/use-cases/log-processing' }, + { from: '/docs/use-cases/readme/distributed-data-warehousing', to: '/use-cases/distributed-data-warehousing' }, + { from: '/docs/use-cases/readme/fleet-management', to: '/use-cases/fleet-management' }, + { from: '/docs/use-cases/readme/distributed-machine-learning', to: '/use-cases/distributed-machine-learning' }, + { from: '/docs/use-cases/readme/edge-computing', to: '/use-cases/edge-computing' }, + { from: '/docs/cli-api', to: '/docs/cli' }, + { from: '/docs/cli-api/api/overview', to: '/docs/api' }, + { from: '/docs/community', to: '/community/social-media' }, + { from: '/docs/references/running-locally', to: '/docs/references/developers/running-locally' }, + { from: '/docs/references/setting-up/running-node/gpu', to: '/docs/references/operators/gpu' }, + { from: '/docs/references/setting-up/gpu', to: '/docs/references/operators/gpu' }, + { from: '/docs/references/setting-up/node_management', to: '/docs/references/operators/node-management' }, + { from: '/docs/guides/configuration-management', to: '/docs/references/operators/configuration-reference' }, + { from: '/docs/references/setting-up/auth_flow', to: '/docs/references/operators/security/auth' }, + { from: '/docs/references/setting-up/inter-nodes-tls', to: '/docs/references/operators/security/node-tls' }, + { from: '/docs/references/setting-up/running-node/auth', to: '/docs/references/operators/security/auth' }, + { from: '/docs/references/setting-up/running-node/configuring-tls', to: '/docs/references/operators/security/node-tls' }, + { from: '/docs/references/help-and-faq/faqs', to: '/docs/references/faq' }, + { from: '/docs/references/help-and-faq/glossary', to: '/docs/references/glossary' }, + { from: '/docs/references/setting-up/running-node/job-selection', to: '/docs/guides/selection-policy' }, + { from: '/docs/references/setting-up/running-node/storage-providers', to: '/docs/guides/mounting-input-data' }, + { from: '/docs/references/setting-up/running-node/resource-limits', to: '/docs/guides/resource-requirements' }, + { from: '/docs/references/setting-up/running-node/webui', to: '/docs/guides/webui' }, + { from: '/docs/references/setting-up/hardware-setup', to: '/docs/guides/resource-requirements' }, + { from: '/docs/references/guides/jobs/job-queuing', to: '/docs/guides/queueing' }, + { from: '/docs/references/guides/jobs/using-labels-and-constraints', to: '/docs/guides/labels-and-constraints' }, + { from: '/docs/references/jobs/job/job-types', to: '/docs/specifications/job/type' }, + { from: '/docs/references/setting-up', to: '/docs/getting-started/network-setup' }, + { from: '/docs/references/setting-up/running-node', to: '/docs/getting-started/network-setup' }, + { from: '/docs/references/setting-up/running-node/quick-start-docker', to: '/docs/getting-started/network-setup' }, + { from: '/docs/examples/data-engineering/duckdb', to: '/integrations/duckdb' }, + { from: '/docs/examples/case-studies/duckdb-log-processing', to: '/integrations/duckdb' }, + { from: '/docs/integrations/lilypad', to: '/integrations/lilypad' }, + { from: '/docs/references/api/index', to: '/docs/api' }, + { from: '/docs/setting-up/networking-instructions/networking', to: '/docs/guides/networking' }, + { from: '/docs/setting-up/running-node/private-ipfs-network-setup', to: '/docs/sources/ipfs' }, + { from: '/docs/next-steps/private-cluster', to: '/docs/getting-started/network-setup' }, + { from: '/docs/setting-up/running-node/quick-start-docker', to: '/docs/getting-started/network-setup' }, + { from: '/docs/setting-up/inter-nodes-tls', to: '/docs/references/operators/security/node-tls' }, + { from: '/docs/setting-up/running-node/gpu', to: '/docs/references/operators/gpu' }, + { from: '/docs/references/node_management', to: '/docs/references/operators/node-management' }, + { from: '/docs/v.1.3.2-1/setting-up/jobs/job-types', to: '/docs/specifications/job/type' }, + { from: '/docs/v.1.3.2-1/setting-up/running-node/storage-providers', to: '/docs/guides/mounting-input-data' }, + { from: '/docs/references/jobs/job/task/engines/docker', to: '/docs/engines/docker' }, + { from: '/docs/references/jobs/job/task/sources/ipfs', to: '/docs/sources/ipfs' }, + { from: '/docs/v.1.3.0/references/other-specifications/engines/docker', to: '/docs/engines/docker' }, + { from: '/docs/getting-started/workload-onboarding/docker-workload-onboarding', to: '/docs/references/developers/workload-onboarding/docker' }, + { from: '/docs/v.1.3.0/community/social-media', to: '/community/social-media' }, + { from: '/docs/integrations/python-sdk', to: '/docs' }, + { from: '/docs/references/databases', to: '/docs' }, + { from: '/docs/references/setting-up/update-checks', to: '/docs' }, +] const createRedirects = (existingPath: string): string[] | undefined => { return undefined diff --git a/scripts/copy-legacy-machine-routes.mjs b/scripts/copy-legacy-machine-routes.mjs new file mode 100644 index 0000000..8cb7966 --- /dev/null +++ b/scripts/copy-legacy-machine-routes.mjs @@ -0,0 +1,21 @@ +import { copyFile, mkdir } from 'node:fs/promises' +import { join } from 'node:path' + +const buildDirectory = new URL('../build/', import.meta.url) +const legacyDocsDirectory = new URL('../build/docs/', import.meta.url) +const machineFiles = ['robots.txt', 'sitemap.xml', 'llms.txt'] + +await mkdir(legacyDocsDirectory, { recursive: true }) + +await Promise.all( + machineFiles.map((filename) => + copyFile( + new URL(filename, buildDirectory), + new URL(filename, legacyDocsDirectory), + ), + ), +) + +console.log( + `Copied ${machineFiles.map((filename) => join('docs', filename)).join(', ')}`, +) diff --git a/src/components/HomepagePartners/index.tsx b/src/components/HomepagePartners/index.tsx index 6b3c225..79a371a 100644 --- a/src/components/HomepagePartners/index.tsx +++ b/src/components/HomepagePartners/index.tsx @@ -9,10 +9,6 @@ type PartnerItem = { }; const PartnerList: PartnerItem[] = [ - { - name: 'Expanso', - image: require('@site/static/img/partners/expanso.png').default, - }, { name: 'University of Maryland', image: require('@site/static/img/partners/maryland.png').default, @@ -78,4 +74,4 @@ export function HomepagePartners(): ReactNode { ); -} \ No newline at end of file +} diff --git a/src/pages/index.tsx b/src/pages/index.tsx index bba599e..f18693f 100644 --- a/src/pages/index.tsx +++ b/src/pages/index.tsx @@ -55,6 +55,8 @@ function HomepageHeader() {

With the open-source software Bacalhau, you can streamline your existing workflows without rewriting by running Docker containers and WebAssembly (WASM) images as tasks. This architecture is also referred to as Compute Over Data (or CoD).

+ Commercial support for Bacalhau is available through Expanso. +

The name "Bacalhau" comes from the Portuguese word for cod.

@@ -103,4 +105,4 @@ export default function Home(): ReactNode { ); -} \ No newline at end of file +} diff --git a/static/.well-known/mcp.json b/static/.well-known/mcp.json index 615d0cd..52ab835 100644 --- a/static/.well-known/mcp.json +++ b/static/.well-known/mcp.json @@ -1,33 +1,17 @@ { "name": "Bacalhau Documentation", - "description": "MCP server for Bacalhau distributed compute documentation - CLI reference, API docs, examples, and tutorials", + "description": "Discovery metadata for the Bacalhau open-source distributed compute documentation", "version": "1.0.0", - "server": { - "url": "https://mcp.expanso.io", - "transport": "https" - }, - "capabilities": { - "tools": true, - "resources": true - }, - "tools": [ - { - "name": "search_bacalhau_docs", - "description": "Search Bacalhau documentation for answers about distributed compute, CLI commands, and API usage" - }, - { - "name": "get_cli_help", - "description": "Get detailed help for Bacalhau CLI commands" - }, - { - "name": "get_example", - "description": "Get code examples for running jobs with Bacalhau" - } - ], + "mcp_server": null, + "note": "Bacalhau does not currently publish an MCP server. Use the documentation and LLM-readable text resources below.", "documentation": { "main": "https://bacalhau.org/docs", "github": "https://github.com/bacalhau-project/bacalhau", - "llms_txt": "https://bacalhau.org/llms.txt" + "llms_txt": "https://bacalhau.org/llms.txt", + "getting_started": "https://bacalhau.org/llms/getting-started.txt", + "cli": "https://bacalhau.org/llms/cli.txt", + "api": "https://bacalhau.org/llms/api.txt", + "concepts": "https://bacalhau.org/llms/concepts.txt" }, "contact": { "discord": "https://bit.ly/bacalhau-project-slack", diff --git a/static/img/partners/expanso.png b/static/img/partners/expanso.png deleted file mode 100644 index 48a646c..0000000 Binary files a/static/img/partners/expanso.png and /dev/null differ diff --git a/static/llms.txt b/static/llms.txt index c37e595..9d12359 100644 --- a/static/llms.txt +++ b/static/llms.txt @@ -20,8 +20,8 @@ bacalhau job list - [Getting Started](/llms/getting-started.txt): Installation, quickstart, first jobs - [CLI Reference](/llms/cli.txt): All bacalhau commands and options - [API Reference](/llms/api.txt): REST API documentation -- [Examples](/llms/examples.txt): Code examples and tutorials - [Concepts](/llms/concepts.txt): Architecture and core concepts +- [Guides and FAQs](/llms/other.txt): Data sources, result publishing, configuration, operations, and troubleshooting ## Key Concepts @@ -31,13 +31,22 @@ bacalhau job list - **Distributed**: Scale from laptop to thousands of nodes - **Open Source**: Apache 2.0 licensed, community-driven +## Common Questions + +### What is compute over data? + +Compute over data runs processing near the systems where data is generated or stored, reducing unnecessary transfers of raw data to a central compute environment. + +### What is Bacalhau? + +Bacalhau is an open-source distributed compute orchestration framework. Orchestrator nodes schedule jobs, and compute nodes run Docker containers or WebAssembly modules near their input data. + +### When is Bacalhau useful? + +Bacalhau is useful when moving raw data is slow, expensive, unreliable, or restricted, including distributed log processing, fleet operations, regional analytics, machine learning, and edge workloads. + ## Resources - GitHub: https://github.com/bacalhau-project/bacalhau - Discord: https://bit.ly/bacalhau-project-slack - Website: https://bacalhau.org - -## Related - -- [Expanso Platform](https://docs.expanso.io/llms.txt): Enterprise features -- [Examples](https://examples.expanso.io/llms.txt): Code examples diff --git a/static/llms/getting-started.txt b/static/llms/getting-started.txt index 5f9e648..709cc7e 100644 --- a/static/llms/getting-started.txt +++ b/static/llms/getting-started.txt @@ -9,9 +9,6 @@ This section explains how to install Bacalhau on your machine, verify it's worki ## Install the Bacalhau Binary To install the CLI, choose your environment, and run the command(s) below. - - - ### Verify the Installation ```bash @@ -37,7 +34,7 @@ To upgrade Bacalhau to the latest version, run the installation script. If Bacal - **AWS Credentials** (if you're using S3): - For S3 inputs or outputs, the node needs valid AWS credentials (e.g., environment variables). - **Running an Orchestrator & Compute**: - - See [Quick Start](./quick-start.mdx) for how to run a local or hybrid node with `bacalhau serve --orchestrator --compute`. +- See [Quick Start](https://bacalhau.org/docs/getting-started/quick-start/) for how to run a local or hybrid node with `bacalhau serve --orchestrator --compute`. - **`bacalhau devstack`**: - Perfect for local development or running tests. @@ -214,7 +211,7 @@ This Quick Start guide shows how to run your first Bacalhau job with minimal set ## 1. Installation -1. Install Bacalhau using the one-liner below (Linux/macOS) or see the [installation guide](./installation.mdx) for Windows and Docker options. +1. Install Bacalhau using the one-liner below (Linux/macOS) or see the [installation guide](https://bacalhau.org/docs/getting-started/installation/) for Windows and Docker options. ```bash curl -sL https://get.bacalhau.org/install.sh | bash @@ -240,9 +237,6 @@ bacalhau serve --orchestrator --compute ## 3. Submit a Data Analysis Job Bacalhau supports two primary methods of job submission: Imperative (CLI) and Declarative (YAML). We'll demonstrate a word count job on the classic novel Moby Dick. - - - - The job downloads a sample dataset and processes it locally - Bacalhau will display job progress until completion - You'll receive a Job ID once the job is submitted @@ -280,4 +274,3 @@ You've just: 2. Submitted a job using both imperative and declarative methods 3. Tracked job progress with detailed descriptions 4. Retrieved and viewed job results - diff --git a/static/llms/other.txt b/static/llms/other.txt index b738275..03c5fe2 100644 --- a/static/llms/other.txt +++ b/static/llms/other.txt @@ -19,9 +19,6 @@ In Bacalhau, you need to configure two key components for handling outputs: 1. A **Publisher** defines where your job's output files are stored after execution 2. **Result Paths** specify which directories should be captured as job results. - - - ### Retrieving Local Outputs After your job completes, retrieve outputs using the `bacalhau job get` command: @@ -46,21 +43,12 @@ Bacalhau supports multiple publisher types to accommodate different needs and in ### S3 Publisher The S3 Publisher uploads outputs to an Amazon S3 bucket or any S3-compatible storage service, such as MinIO. The compute node must have permission to write to the bucket, and the orchestrator must have permission to provide pre-signed URLs to download the results. - - - ### IPFS Publisher The IPFS Publisher uploads outputs to the InterPlanetary File System. Both the client (downloading the result) and the compute node must be connected to an IPFS daemon. - - - ### Local Publisher The Local Publisher saves outputs to the local filesystem of the compute node that ran your job. This is intended for **local testing only**, as it requires the client downloading the results to be on the same network as the compute node. - - - If you are using the local publish, make SURE you have set the path to be available to your job. For example, in your config file for your node, you probably want to mount in the local file system: @@ -72,7 +60,7 @@ AllowListedLocalPaths: - /outputs:rw - /etc/\*.conf:ro ``` -You can read more about that here: [Understanding Publishers and Results](publishing-results.mdx#understanding-publishers-and-result-paths) +You can read more about that here: [Understanding Publishers and Results](https://bacalhau.org/docs/guides/publishing-results/#understanding-publishers-and-result-paths) ## Troubleshooting @@ -1005,7 +993,7 @@ This guide explains how to use existing Docker containers and create custom ones ## Prerequisites -1. [Install the Bacalhau client](/getting-started/installation.mdx) +1. [Install the Bacalhau client](https://bacalhau.org/docs/getting-started/installation/) 2. [Docker](https://docs.docker.com/install/) (for both building custom containers and running on compute nodes) ## Understanding Docker Workloads in Bacalhau @@ -1691,4 +1679,3 @@ Utilizing the `Timeouts` judiciously helps in managing resource utilization and ## working-with-nodes [Content truncated - see full docs] -