static: update llms.txt for new navigation (#682)

Author: Oreoxmt

static/ai/llms.txt

@@ -0,0 +1,85 @@
+# TiDB for AI
+
+> TiDB is a distributed SQL database designed for modern AI applications, offering integrated vector search, full-text search, and hybrid search capabilities. This document provides an overview of the AI features and tools available for building AI-powered applications with TiDB.
+
+## QUICK START
+
+- [Get Started via Python](https://docs.pingcap.com/ai/quickstart-via-python.md): Learn how to get started with vector search in TiDB using Python SDK.
+- [Get Started via SQL](https://docs.pingcap.com/ai/quickstart-via-sql.md): Learn how to quickly get started with Vector Search in TiDB using SQL statements to power your generative AI applications.
+
+## CONCEPTS
+
+- [Vector Search](https://docs.pingcap.com/ai/vector-search-overview.md): Learn about Vector Search in TiDB. This feature provides an advanced search solution for performing semantic similarity searches across various data types, including documents, images, audio, and video.
+
+## GUIDES
+
+- [Connect to TiDB](https://docs.pingcap.com/ai/connect.md): Learn how to connect to a TiDB database using the `pytidb` client.
+- [Working with Tables](https://docs.pingcap.com/ai/tables.md): Learn how to work with tables in TiDB.
+- Search Features
+  - [Vector Search](https://docs.pingcap.com/ai/vector-search.md): Learn how to use vector search in your application.
+  - Full-Text Search
+    - [Full-Text Search via Python](https://docs.pingcap.com/ai/vector-search-full-text-search-python.md): Full-text search lets you retrieve documents for exact keywords. In Retrieval-Augmented Generation (RAG) scenarios, you can use full-text search together with vector search to improve the retrieval quality.
+    - [Full-Text Search via SQL](https://docs.pingcap.com/ai/vector-search-full-text-search-sql.md): Full-text search lets you retrieve documents for exact keywords. In Retrieval-Augmented Generation (RAG) scenarios, you can use full-text search together with vector search to improve the retrieval quality.
+  - [Hybrid Search](https://docs.pingcap.com/ai/vector-search-hybrid-search.md): Use full-text search and vector search together to improve the retrieval quality.
+  - [Image Search](https://docs.pingcap.com/ai/image-search.md): Learn how to use image search in your application.
+- Advanced Features
+  - [Auto Embedding](https://docs.pingcap.com/ai/auto-embedding.md): Learn how to use auto embedding in your application.
+  - [Filtering](https://docs.pingcap.com/ai/filtering.md): Learn how to use filtering in your application.
+  - [Reranking](https://docs.pingcap.com/ai/reranking.md): Learn how to use reranking in your application.
+  - [Join Queries](https://docs.pingcap.com/ai/join-queries.md): Learn how to use multiple table joins in your application.
+  - [Raw SQL Queries](https://docs.pingcap.com/ai/raw-queries.md): Learn how to use raw queries in your application.
+  - [Transactions](https://docs.pingcap.com/ai/transactions.md): Learn how to use transactions in your application.
+
+## EXAMPLES
+
+- [Basic CRUD Operations](https://docs.pingcap.com/ai/basic-with-pytidb.md): Learn fundamental `pytidb` operations including database connection, table creation, and data manipulation.
+- [Auto Embedding](https://docs.pingcap.com/ai/auto-embedding-with-pytidb.md): Automatically generate embeddings for your text data using built-in embedding models.
+- Search & Retrieval
+  - [Vector Search](https://docs.pingcap.com/ai/vector-search-with-pytidb.md): Implement semantic search using vector embeddings to find similar content.
+  - [Full-Text Search](https://docs.pingcap.com/ai/fulltext-search-with-pytidb.md): Perform traditional text search using TiDB full-text search.
+  - [Hybrid Search](https://docs.pingcap.com/ai/hybrid-search-with-pytidb.md): Combine vector search and full-text search for more comprehensive results.
+  - [Image Search](https://docs.pingcap.com/ai/image-search-with-pytidb.md): Build an image search application using multimodal embeddings for both text-to-image and image-to-image search.
+- AI Applications
+  - [RAG Application](https://docs.pingcap.com/ai/rag-with-pytidb.md): Build a RAG application that combines document retrieval with language generation.
+  - [Conversational Memory](https://docs.pingcap.com/ai/memory-with-pytidb.md): Implement conversation memory for chatbots and conversational AI applications.
+  - [Text-to-SQL](https://docs.pingcap.com/ai/text2sql-with-pytidb.md): Convert natural language queries into SQL statements using AI models.
+
+## INTEGRATIONS
+
+- [Integration Overview](https://docs.pingcap.com/ai/vector-search-integration-overview.md): An overview of TiDB vector search integration, including supported AI frameworks, embedding models, and ORM libraries.
+- Auto Embedding
+  - [Overview](https://docs.pingcap.com/ai/vector-search-auto-embedding-overview.md): Learn how to use Auto Embedding to perform semantic searches with plain text instead of vectors.
+  - [OpenAI](https://docs.pingcap.com/ai/vector-search-auto-embedding-openai.md): Learn how to use OpenAI embedding models in TiDB Cloud.
+  - [OpenAI Compatible](https://docs.pingcap.com/ai/embedding-openai-compatible.md): Learn how to integrate TiDB Vector Search with an OpenAI-compatible embedding model to store embeddings and perform semantic search.
+  - [Jina AI](https://docs.pingcap.com/ai/vector-search-auto-embedding-jina-ai.md): Learn how to use Jina AI embedding models in TiDB Cloud.
+  - [Cohere](https://docs.pingcap.com/ai/vector-search-auto-embedding-cohere.md): Learn how to use Cohere embedding models in TiDB Cloud.
+  - [Google Gemini](https://docs.pingcap.com/ai/vector-search-auto-embedding-gemini.md): Learn how to use Google Gemini embedding models in TiDB Cloud.
+  - [Hugging Face](https://docs.pingcap.com/ai/vector-search-auto-embedding-huggingface.md): Learn how to use Hugging Face embedding models in TiDB Cloud.
+  - [NVIDIA NIM](https://docs.pingcap.com/ai/vector-search-auto-embedding-nvidia-nim.md): Learn how to use NVIDIA NIM embedding models in TiDB Cloud.
+  - [Amazon Titan](https://docs.pingcap.com/ai/vector-search-auto-embedding-amazon-titan.md): Learn how to use Amazon Titan embedding models in TiDB Cloud.
+- AI Frameworks
+  - [LangChain](https://docs.pingcap.com/ai/vector-search-integrate-with-langchain.md): Learn how to integrate TiDB Vector Search with LangChain.
+  - [LlamaIndex](https://docs.pingcap.com/ai/vector-search-integrate-with-llamaindex.md): Learn how to integrate TiDB Vector Search with LlamaIndex.
+- ORM Libraries
+  - [SQLAlchemy](https://docs.pingcap.com/ai/vector-search-integrate-with-sqlalchemy.md): Learn how to integrate TiDB Vector Search with SQLAlchemy to store embeddings and perform semantic searches.
+  - [Django ORM](https://docs.pingcap.com/ai/vector-search-integrate-with-django-orm.md): Learn how to integrate TiDB Vector Search with Django ORM to store embeddings and perform semantic search.
+  - [Peewee](https://docs.pingcap.com/ai/vector-search-integrate-with-peewee.md): Learn how to integrate TiDB Vector Search with peewee to store embeddings and perform semantic searches.
+- Cloud Services
+  - [Jina AI Embedding](https://docs.pingcap.com/ai/vector-search-integrate-with-jinaai-embedding.md): Learn how to integrate TiDB Vector Search with Jina AI Embeddings API to store embeddings and perform semantic search.
+  - [Amazon Bedrock](https://docs.pingcap.com/ai/vector-search-integrate-with-amazon-bedrock.md): Learn how to integrate TiDB Vector Search with Amazon Bedrock to build a Retrieval-Augmented Generation (RAG) Q&A bot.
+- MCP Server
+  - [Overview](https://docs.pingcap.com/ai/tidb-mcp-server.md): Manage your TiDB databases using natural language instructions with the TiDB MCP Server.
+  - [Claude Code](https://docs.pingcap.com/ai/tidb-mcp-claude-code.md): This guide shows you how to configure the TiDB MCP Server in Claude Code.
+  - [Claude Desktop](https://docs.pingcap.com/ai/tidb-mcp-claude-desktop.md): This guide shows you how to configure the TiDB MCP Server in Claude Desktop.
+  - [Cursor](https://docs.pingcap.com/ai/tidb-mcp-cursor.md): This guide shows you how to configure the TiDB MCP Server in the Cursor editor.
+  - [VS Code](https://docs.pingcap.com/ai/tidb-mcp-vscode.md): This guide shows you how to configure the TiDB MCP Server in Visual Studio Code.
+  - [Windsurf](https://docs.pingcap.com/ai/tidb-mcp-windsurf.md): This guide shows you how to configure the TiDB MCP Server in Windsurf.
+
+## REFERENCE
+
+- [Vector Data Types](https://docs.pingcap.com/ai/vector-search-data-types.md): Learn about the Vector data types in TiDB.
+- [Functions and Operators](https://docs.pingcap.com/ai/vector-search-functions-and-operators.md): Learn about functions and operators available for Vector data types.
+- [Vector Search Index](https://docs.pingcap.com/ai/vector-search-index.md): Learn how to build and use the vector search index to accelerate K-Nearest neighbors (KNN) queries in TiDB.
+- [Performance Tuning](https://docs.pingcap.com/ai/vector-search-improve-performance.md): Learn best practices for improving the performance of TiDB Vector Search.
+- [Limitations](https://docs.pingcap.com/ai/vector-search-limitations.md): Learn the limitations of the TiDB vector search.
+- [Changelogs](https://docs.pingcap.com/ai/vector-search-changelogs.md): Learn about the new features, compatibility changes, improvements, and bug fixes for the TiDB vector search feature.

static/api/llms.txt

@@ -0,0 +1,17 @@
+# TiDB API
+
+> TiDB provides various APIs for querying and operating clusters, managing data replication, monitoring system status, and more. This document provides an overview of the available APIs for both TiDB Cloud and TiDB Self-Managed.
+
+## TIDB CLOUD
+
+- [API Overview](https://docs.pingcap.com/api/tidb-cloud-api-overview.md): Learn about what TiDB Cloud API is, its features, and how to use the API to manage your TiDB Cloud clusters.
+- [API v1beta1](https://docs.pingcap.com/api/tidb-cloud-api-v1beta1.md): Learn about the v1beta1 API of TiDB Cloud.
+- [API v1beta](https://docs.pingcap.com/api/tidb-cloud-api-v1beta.md): Learn about the v1beta API of TiDB Cloud.
+
+## TIDB SELF-MANAGED
+
+- [TiProxy API](https://docs.pingcap.com/api/tiproxy-api-overview.md): Learn about the API for TiProxy.
+- [Data Migration API](https://docs.pingcap.com/api/dm-api-overview.md): Learn the API of Data Migration (DM).
+- [Monitoring API](https://docs.pingcap.com/api/monitoring-api-overview.md): Learn the API of TiDB monitoring services.
+- [TiCDC API](https://docs.pingcap.com/api/ticdc-api-overview.md): Learn the API of TiCDC.
+- [TiDB Operator API](https://docs.pingcap.com/api/tidb-operator-api-overview.md): Learn the API of TiDB Operator.

static/best-practices/llms.txt

@@ -0,0 +1,34 @@
+# TiDB Best Practices
+
+> By following best practices for deploying, configuring, and using TiDB, you can optimize the performance, reliability, and scalability of your TiDB deployments. This document provides an overview of the best practices for using TiDB.
+
+## Overview
+
+- [Use TiDB](https://docs.pingcap.com/best-practices/tidb-best-practices.md): This document summarizes best practices for using TiDB, covering SQL use and optimization tips for OLAP and OLTP scenarios, with a focus on TiDB-specific optimization options. It also recommends reading three blog posts introducing TiDB's technical principles before diving into the best practices.
+
+## Schema Design
+
+- [Manage DDL](https://docs.pingcap.com/best-practices/ddl-introduction.md): Learn about how DDL statements are implemented in TiDB, the online change process, and best practices.
+- [Use UUIDs as Primary Keys](https://docs.pingcap.com/best-practices/uuid.md): UUIDs, when used as primary keys, offer benefits such as reduced network trips, support in most programming languages and databases, and protection against enumeration attacks. Storing UUIDs as binary in a `BINARY(16)` column is recommended. It's also advised to avoid setting the `swap_flag` with TiDB to prevent hotspots. MySQL compatibility is available for UUIDs.
+- [Use TiDB Partitioned Tables](https://docs.pingcap.com/best-practices/tidb-partitioned-tables-best-practices.md): Learn best practices for using TiDB partitioned tables to improve performance, simplify data management, and handle large-scale datasets efficiently.
+- [Optimize Multi-Column Indexes](https://docs.pingcap.com/best-practices/multi-column-index-best-practices.md): Learn how to use multi-column indexes effectively in TiDB and apply advanced optimization techniques.
+- [Manage Indexes and Identify Unused Indexes](https://docs.pingcap.com/best-practices/index-management-best-practices.md): Learn the best practices for managing and optimizing indexes, identifying and removing unused indexes in TiDB.
+
+## Deployment
+
+- [Deploy TiDB on Public Cloud](https://docs.pingcap.com/best-practices/best-practices-on-public-cloud.md): Learn about the best practices for deploying TiDB on public cloud.
+- [Three-Node Hybrid Deployment](https://docs.pingcap.com/best-practices/three-nodes-hybrid-deployment.md): TiDB cluster can be deployed in a cost-effective way on three machines. Best practices for this hybrid deployment include adjusting parameters for stability and performance. Limiting resource consumption and adjusting thread pool sizes are key to optimizing the cluster. Adjusting parameters for TiKV background tasks and TiDB execution operators is also important.
+- [Local Reads in Three-Data-Center Deployments](https://docs.pingcap.com/best-practices/three-dc-local-read.md): TiDB's three data center deployment model can cause increased access latency due to cross-center data reads. To mitigate this, the Stale Read feature allows for local historical data access, reducing latency at the expense of real-time data availability. When using Stale Read in geo-distributed scenarios, TiDB accesses local replicas to avoid cross-center network latency. This is achieved by configuring the `zone` label and setting `tidb_replica_read` to `closest-replicas`. For more information on performing Stale Read, refer to the documentation.
+
+## Operations
+
+- [Use HAProxy for Load Balancing](https://docs.pingcap.com/best-practices/haproxy-best-practices.md): HAProxy is a free, open-source load balancer and proxy server for TCP and HTTP-based applications. It provides high availability, load balancing, health checks, sticky sessions, SSL support, and monitoring. To deploy HAProxy, ensure hardware and software requirements are met, then install and configure it. Use the latest stable version for best results.
+- [Use Read-Only Storage Nodes](https://docs.pingcap.com/best-practices/readonly-nodes.md): This document introduces configuring read-only storage nodes for isolating high-tolerance delay loads from online services. Steps include marking TiKV nodes as read-only, using Placement Rules to store data on read-only nodes as learners, and using Follower Read to read data from read-only nodes.
+- [Monitor TiDB Using Grafana](https://docs.pingcap.com/best-practices/grafana-monitor-best-practices.md): Best Practices for Monitoring TiDB Using Grafana. Deploy a TiDB cluster using TiUP and add Grafana and Prometheus for monitoring. Use metrics to analyze cluster status and diagnose problems. Prometheus collects metrics from TiDB components, and Grafana displays them. Tips for efficient Grafana use include modifying query expressions, switching Y-axis scale, and using API for query results. The platform is powerful for analyzing and diagnosing TiDB cluster status.
+
+## Performance Tuning
+
+- [Handle Millions of Tables in SaaS Multi-Tenant Scenarios](https://docs.pingcap.com/best-practices/saas-best-practices.md): Learn best practices for TiDB in SaaS (Software as a Service) multi-tenant scenarios, especially for environments where the number of tables in a single cluster exceeds one million.
+- [Handle High-Concurrency Writes](https://docs.pingcap.com/best-practices/high-concurrency-best-practices.md): This document provides best practices for handling highly-concurrent write-heavy workloads in TiDB. It addresses challenges and solutions for data distribution, hotspot cases, and complex hotspot problems. The article also discusses parameter configuration for optimizing performance.
+- [Tune TiKV Performance with Massive Regions](https://docs.pingcap.com/best-practices/massive-regions-best-practices.md): TiKV performance tuning involves reducing the number of Regions and messages, increasing Raftstore concurrency, enabling Hibernate Region and Region Merge, adjusting Raft base tick interval, increasing TiKV instances, and adjusting Region size. Other issues include slow PD leader switching and outdated PD routing information.
+- [Tune PD Scheduling](https://docs.pingcap.com/best-practices/pd-scheduling-best-practices.md): This document summarizes PD scheduling best practices, including scheduling process, load balancing, hot regions scheduling, cluster topology awareness, scale-in and failure recovery, region merge, query scheduling status, and control scheduling strategy. It also covers common scenarios such as uneven distribution of leaders/regions, slow node recovery, and troubleshooting TiKV nodes.