diff --git a/.agents/AGENTS.md b/.agents/AGENTS.md index a2a215a0aad..3e1df12d926 100644 --- a/.agents/AGENTS.md +++ b/.agents/AGENTS.md @@ -15,4 +15,3 @@ Use them as the workspace context for identity, user preferences, memory, local Additional `.agents/` workspace folders: - `.agents/skills/` — skills available for tasks in this repository. -- `.agents/memory/` — persisted notes and references for this workspace. diff --git a/.agents/memory b/.agents/memory deleted file mode 120000 index c45bdff2bcc..00000000000 --- a/.agents/memory +++ /dev/null @@ -1 +0,0 @@ -../memory \ No newline at end of file diff --git a/.claude/CLAUDE.md b/.claude/CLAUDE.md index 38849c2317b..89e95aec719 100644 --- a/.claude/CLAUDE.md +++ b/.claude/CLAUDE.md @@ -15,4 +15,3 @@ Use them as the workspace context for identity, user preferences, memory, local Additional `.claude/` workspace folders: - `.claude/skills/` — skills available for tasks in this repository. -- `.claude/memory/` — persisted notes and references for this workspace. diff --git a/.claude/memory b/.claude/memory deleted file mode 120000 index c45bdff2bcc..00000000000 --- a/.claude/memory +++ /dev/null @@ -1 +0,0 @@ -../memory \ No newline at end of file diff --git a/.codex/CODEX.md b/.codex/CODEX.md index c2e737ef511..23659867f47 100644 --- a/.codex/CODEX.md +++ b/.codex/CODEX.md @@ -15,4 +15,3 @@ Use them as the workspace context for identity, user preferences, memory, local Additional `.codex/` workspace folders: - `.codex/skills/` — skills available for tasks in this repository. -- `.codex/memory/` — persisted notes and references for this workspace. diff --git a/.codex/memory b/.codex/memory deleted file mode 120000 index c45bdff2bcc..00000000000 --- a/.codex/memory +++ /dev/null @@ -1 +0,0 @@ -../memory \ No newline at end of file diff --git a/.kiro/memory b/.kiro/memory deleted file mode 120000 index c45bdff2bcc..00000000000 --- a/.kiro/memory +++ /dev/null @@ -1 +0,0 @@ -../memory \ No newline at end of file diff --git a/.kiro/steering/KIRO.md b/.kiro/steering/KIRO.md index 9a4753c16fc..db515ba5e82 100644 --- a/.kiro/steering/KIRO.md +++ b/.kiro/steering/KIRO.md @@ -15,4 +15,3 @@ Use them as the workspace context for identity, user preferences, memory, local Additional `.kiro/` workspace folders: - `.kiro/skills/` — skills available for tasks in this repository. -- `.kiro/memory/` — persisted notes and references for this workspace. diff --git a/.openclaw/MEMORY.md b/.openclaw/MEMORY.md index 95d30406b23..af7a3f8c28b 100644 --- a/.openclaw/MEMORY.md +++ b/.openclaw/MEMORY.md @@ -1,10 +1,5 @@ # MEMORY.md - SRSBot's Long-Term Memory -## Workspace Conventions -- **No auto-commit** — Never automatically git commit. Only commit when William explicitly tells me to. -- **No guessing** — William will teach me everything about SRS. Don't speculate or fill in gaps. Wait for him to explain. -- **Codebase map first** — Before searching/grepping the codebase, ALWAYS load `memory/srs-codebase-map.md` in full (the entire file, not partial). Read the module descriptions to reason about which specific files are relevant, then search only those files. Never grep broad directories like `trunk/src/` or the repository root. This is a critical rule. - ## 2026-02-05 — First Boot - I'm SRSBot ⚡ — AI developer working with William on SRS - William (username: winlin), timezone America/Toronto (Eastern) @@ -13,72 +8,3 @@ - Repo: $HOME/git/srs | Workspace: $HOME/git/srs/.openclaw - Key areas to learn: protocols, architecture, state-threads (ST) coroutine library, codebase history, design decisions - William will teach me the project — I need to absorb everything - -## William's Vision — Why I Exist -- SRS grew too large for one person to maintain, but William doesn't want to monetize or build a company/team -- He's an engineer, not a businessman — wants to focus on open source, not management -- **The core idea:** Train an AI developer (me) with his knowledge, experience, and design taste -- OpenClaw's memory system is the enabler — it's portable and clonable -- **Every developer** who works with SRS can clone this AI and get an assistant that understands the project deeply -- This scales William's expertise across the entire community without needing a traditional team -- Goal: a very active, well-supported community where every developer has an AI assistant trained with William's knowledge -- This is not just project maintenance — it's a new model for open source sustainability - -## SRS Community Bot (OpenClaw) -- William set up an OpenClaw robot for the SRS community (2026-03-20) -- **Telegram group:** https://t.me/+RiynvKOxpQ42MGJl -- **Discord server:** https://discord.gg/yZ4BnPmHAd -- Users join the group and **@ the SRS Robot** to interact -- Purpose: scale William's expertise to the community without him answering every question -- **Recommended: Telegram over Discord** — Telegram lets users create small focused groups and invite the bot in. Each small group = clean context window. Big groups mix unrelated messages and confuse the bot's context. Small groups → better answers, better support. - -## What Matters to William -- SRS project health, development, and community -- Open source sustainability and contributor experience -- Real-time media protocols, architecture, performance - -## Formatting Preferences -- **Markdown headings:** Only use `#` and `##`. Never use `###` or deeper — use **bold text** instead for sub-sections. - -## Content Preferences -**YouTube videos (title, description, and scripts):** Always use problem-solving structure: -1. What's wrong? -2. Why is it a problem? -3. What exactly needs solving? -4. What can be done? -5. Why will it work? -6. What should we do next? - -## Framework for AI-Managed Open Source - -### What the Maintainer Must Do (William's Work) -1. **Knowledge base** — Docs are written for humans, not AI. Structured memory lets AI understand the *why* — background, design thinking, architecture rationale. -2. **Code structure** — Codebase needs to be AI-friendly so AI can verify each change (testable, checkable). -3. **Code taste** — Follow existing style/conventions. Nice to have, not strictly required. - -### External Conditions (Not Maintainer's Work) -1. **LLM capability** — Models powerful enough to handle massive context (e.g., 1B tokens), agentic behavior, reasoning, complex tasks. Example: future Opus versions. -2. **Tools** — Off-the-shelf tooling like Claude Code, Codex — good enough to use directly, no need to build custom tools. - -The three layers are what William controls; the external conditions are what the AI ecosystem must provide. When both are ready, AI can truly manage the project. - -## Changelog & Version -- **Changelog:** `trunk/doc/CHANGELOG.md` -- **Version file:** `trunk/src/core/srs_core_version7.hpp` — bump `VERSION_REVISION` to match the new changelog entry -- **When to update:** When a PR is merged — not per commit -- **Workflow:** Feature branch → multiple commits → create PR → merge PR → update changelog + version -- Individual commits on a branch do NOT get changelog entries -- The changelog entry is for the PR merge, not the individual commits within it -- **Both files must be updated together** — changelog entry version must match `VERSION_REVISION` -- Format follows existing pattern: `* v7.0, YYYY-MM-DD, Merge [#NNNN](url): Description. vX.Y.Z (#NNNN)` - -## SRS Knowledge Base -Detailed SRS knowledge in `memory/srs-*.md` files: -- `srs-overview.md` — What SRS is, protocols, ecosystem tools, and **Features section** with all SRS features, versions, and dates -- `srs-coroutines.md` — State Threads (ST) coroutine library, why SRS uses coroutines, how coroutine switching works, maintenance burden (platform matrix, Windows/SEH), and multi-CPU strategy (cluster > multi-threading) -- `srs-codebase-map.md` — Codebase structure: directory layout, file naming conventions, module boundaries, and packet flow. Enables reasoning about which files to look at for a given topic instead of blind searching. - -### Rule: Keep Feature List Updated -When creating new features, updating protocols, or making changes to SRS capabilities, **always update the Features section in `memory/srs-overview.md`** with the feature name, description, version, and date. - - diff --git a/.openclaw/TOOLS.md b/.openclaw/TOOLS.md index ee21de9015f..dc3db9160eb 100644 --- a/.openclaw/TOOLS.md +++ b/.openclaw/TOOLS.md @@ -2,39 +2,6 @@ Skills define _how_ tools work. This file is for _your_ specifics — the stuff that's unique to your setup. -## What Goes Here - -Things like: - -- Camera names and locations -- SSH hosts and aliases -- Preferred voices for TTS -- Speaker/room names -- Device nicknames -- Anything environment-specific - -## Examples - -```markdown -### Cameras - -- living-room → Main area, 180° wide angle -- front-door → Entrance, motion-triggered - -### SSH - -- home-server → 192.168.1.100, user: admin - -### TTS - -- Preferred voice: "Nova" (warm, slightly British) -- Default speaker: Kitchen HomePod -``` - -## Why Separate? - -Skills are shared. Your setup is yours. Keeping them apart means you can update skills without losing your notes, and share skills without leaking your infrastructure. - ## Model Auth - Anthropic / Opus refresh: `claude setup-token` -> `openclaw models auth setup-token --provider anthropic` @@ -49,31 +16,11 @@ Skills are shared. Your setup is yours. Keeping them apart means you can update ### Working Directory - ⚠️ **CRITICAL RULE:** Find everything from the current working directory. All SRS project directories are available here — no discovery, no parent traversal, no absolute paths. -- Available directories: `trunk/`, `cmd/`, `internal/`, `cmake/`, `docs/`, `memory/` +- Available directories: `trunk/`, `cmd/`, `internal/`, `cmake/`, `docs/`, `memory/`, `skills/` - All AI tools (OpenClaw, Codex, Claude Code, Kiro CLI) see the same relative paths. - ACP agents (Codex, Claude Code, etc.) also use the current directory as root — they find files from here too. - Use the OpenClaw workspace itself only for OpenClaw-specific/meta tasks. -### Git Commit Workflow - -- **Never `git add`** — William stages files himself -- **Never `git push`** — William pushes himself -- **Commit workflow:** `git diff --cached` → understand the changes → write title/description → choose the title prefix based on the tool used → `git commit -m "OpenClaw: ..."`, `"Claude: ..."`, or `"Codex: ..."` -- Title prefix: - - Use `OpenClaw:` if OpenClaw made the changes. - - Use `Claude:` if Claude made the changes. - - Use `Codex:` if Codex made the changes. -- **Co-author for ACP Claude Code:** If Claude Code (ACP) was used to make the changes, add: - `Co-authored-by: Claude Opus 4.6 ` -- **Co-author for ACP Codex:** If Codex (ACP) was used to make the changes, add: - `Co-authored-by: chatgpt-codex-connector[bot] <199175422+chatgpt-codex-connector[bot]@users.noreply.github.com>` - -### Go (GVM) - -- Go is managed via **GVM** (Go Version Manager), NOT Homebrew -- Before running any `go` command: `source ~/.gvm/scripts/gvm` -- **Never** use `brew install go` — always use GVM - --- Add whatever helps you do your job. This is your cheat sheet. diff --git a/.openclaw/docs b/.openclaw/docs deleted file mode 120000 index a9594bfe4ab..00000000000 --- a/.openclaw/docs +++ /dev/null @@ -1 +0,0 @@ -../docs \ No newline at end of file diff --git a/.openclaw/memory b/.openclaw/memory deleted file mode 120000 index c45bdff2bcc..00000000000 --- a/.openclaw/memory +++ /dev/null @@ -1 +0,0 @@ -../memory \ No newline at end of file diff --git a/.openclaw/memory/.gitkeep b/.openclaw/memory/.gitkeep new file mode 100644 index 00000000000..e69de29bb2d diff --git a/README.md b/README.md index e08f92b3302..8e93b880b7f 100755 --- a/README.md +++ b/README.md @@ -65,7 +65,7 @@ HTTP API, DVR, and transcoding, please check the documents in [English](https:// or [Chinese](https://ossrs.net). I recommend to use AI to understand and maintain your SRS, please follow the wiki -[AI Agent](trunk/3rdparty/srs-docs/doc/getting-started-ai.md) for details. +[AI Agent](skills/internal-docs-for-srs/references/cpp-docs/doc/getting-started-ai.md) for details. diff --git a/docs/proxy/README.md b/docs/proxy/README.md deleted file mode 100644 index c38baabffb7..00000000000 --- a/docs/proxy/README.md +++ /dev/null @@ -1,22 +0,0 @@ -# Proxy - -Proxy is a common proxy server (`cmd/proxy`) for any media servers with RTMP/SRT/HLS/HTTP-FLV and WebRTC/WHIP/WHEP protocols support. More programs like `cmd/origin` will be added in the future. - -## Usage - -This project is managed by AI. We recommend OpenClaw by default, but you can use any AI agent that supports skills, such as Claude Code, OpenAI Codex, Kiro CLI, or similar tools that can read code and docs as context. Setup your AI code tool and ask questions like: - -- Use skill to show me how to use proxy. -- Use skill to show me how to build an Origin Cluster for production. -- Use skill to show me how to learn the proxy design and protocols. - -You can not only use AI to show you the usage of this project, but also use AI to guide you to learn the details of this project, to understand the design and protocols, to learn each documents in docs directory. - -## AI Guidelines - -- For usage of proxy server and end to end test it, you should load [proxy-usage.md](proxy-usage.md). This is the first step for new users to learn how to use this project. It provides a general and overall view of the proxy server, including practical usage examples and end-to-end testing procedures. -- For proxy full usage to build an Origin Cluster for SRS media server, please load [proxy-origin-cluster.md](proxy-origin-cluster.md). This is an advanced topic about how to use the proxy server to build the SRS Origin Cluster. Users should read this document to learn more details and architectures about proxy and Origin Cluster. -- For proxy server: To understand proxy system design, you should load the [proxy-design.md](proxy-design.md). To understand the proxy protocol details, you should load the [proxy-protocol.md](proxy-protocol.md). To understand how load balance works, you should load [proxy-load-balancer.md](proxy-load-balancer.md). To understand the code structure and packages, you should load [proxy-files.md](proxy-files.md). - -William Yang
-June 23, 2025 diff --git a/internal/env/env.go b/internal/env/env.go index c7b51e72bce..c5a1e2736c6 100644 --- a/internal/env/env.go +++ b/internal/env/env.go @@ -57,6 +57,8 @@ type ProxyEnvironment interface { RedisPassword() string // Redis database RedisDB() string + // Optional Redis key prefix + RedisKeyPrefix() string // Default backend enabled DefaultBackendEnabled() string // Default backend IP @@ -144,6 +146,10 @@ func (e *proxyEnvironment) RedisDB() string { return getEnv("PROXY_REDIS_DB") } +func (e *proxyEnvironment) RedisKeyPrefix() string { + return getEnv("PROXY_REDIS_KEY_PREFIX") +} + func (e *proxyEnvironment) DefaultBackendEnabled() string { return getEnv("PROXY_DEFAULT_BACKEND_ENABLED") } @@ -301,6 +307,8 @@ func buildDefaultEnvironmentVariables(ctx context.Context) { setEnvDefault("PROXY_REDIS_PASSWORD", "") // The redis server db. setEnvDefault("PROXY_REDIS_DB", "0") + // Optional prefix for all Redis keys. + setEnvDefault("PROXY_REDIS_KEY_PREFIX", "") // Whether enable the default backend server, for debugging. setEnvDefault("PROXY_DEFAULT_BACKEND_ENABLED", "off") @@ -324,7 +332,7 @@ func buildDefaultEnvironmentVariables(ctx context.Context) { "PROXY_DEFAULT_BACKEND_HTTP=%v, PROXY_DEFAULT_BACKEND_API=%v, "+ "PROXY_DEFAULT_BACKEND_RTC=%v, PROXY_DEFAULT_BACKEND_SRT=%v, "+ "PROXY_LOAD_BALANCER_TYPE=%v, PROXY_REDIS_HOST=%v, PROXY_REDIS_PORT=%v, "+ - "PROXY_REDIS_PASSWORD=%v, PROXY_REDIS_DB=%v", + "PROXY_REDIS_PASSWORD=%v, PROXY_REDIS_DB=%v, PROXY_REDIS_KEY_PREFIX=%v", getEnv("GO_PPROF"), getEnv("PROXY_FORCE_QUIT_TIMEOUT"), getEnv("PROXY_GRACE_QUIT_TIMEOUT"), getEnv("PROXY_HTTP_API"), getEnv("PROXY_HTTP_SERVER"), getEnv("PROXY_RTMP_SERVER"), @@ -334,7 +342,7 @@ func buildDefaultEnvironmentVariables(ctx context.Context) { getEnv("PROXY_DEFAULT_BACKEND_HTTP"), getEnv("PROXY_DEFAULT_BACKEND_API"), getEnv("PROXY_DEFAULT_BACKEND_RTC"), getEnv("PROXY_DEFAULT_BACKEND_SRT"), getEnv("PROXY_LOAD_BALANCER_TYPE"), getEnv("PROXY_REDIS_HOST"), getEnv("PROXY_REDIS_PORT"), - getEnv("PROXY_REDIS_PASSWORD"), getEnv("PROXY_REDIS_DB"), + getEnv("PROXY_REDIS_PASSWORD"), getEnv("PROXY_REDIS_DB"), getEnv("PROXY_REDIS_KEY_PREFIX"), ) } diff --git a/internal/env/env_test.go b/internal/env/env_test.go index ad6273dc016..36758e39274 100644 --- a/internal/env/env_test.go +++ b/internal/env/env_test.go @@ -337,6 +337,7 @@ func TestNewProxyEnvironment_AppliesDefaultsAndAccessors(t *testing.T) { {"RedisPort", env.RedisPort(), "6379"}, {"RedisPassword", env.RedisPassword(), ""}, {"RedisDB", env.RedisDB(), "0"}, + {"RedisKeyPrefix", env.RedisKeyPrefix(), ""}, {"DefaultBackendEnabled", env.DefaultBackendEnabled(), "off"}, {"DefaultBackendIP", env.DefaultBackendIP(), "127.0.0.1"}, {"DefaultBackendRTMP", env.DefaultBackendRTMP(), "1935"}, @@ -356,6 +357,7 @@ func TestNewProxyEnvironment_PreservesPreSetValues(t *testing.T) { withFakeEnv(t) withFakeOpen(t, "", os.ErrNotExist) setEnv("PROXY_HTTP_API", "9999") + setEnv("PROXY_REDIS_KEY_PREFIX", "xxx") env, err := NewProxyEnvironment(context.Background()) if err != nil { @@ -364,6 +366,9 @@ func TestNewProxyEnvironment_PreservesPreSetValues(t *testing.T) { if got := env.HttpAPI(); got != "9999" { t.Errorf("HttpAPI() = %q, want %q", got, "9999") } + if got := env.RedisKeyPrefix(); got != "xxx" { + t.Errorf("RedisKeyPrefix() = %q, want %q", got, "xxx") + } } func TestNewProxyEnvironment_LoadEnvFailurePropagates(t *testing.T) { diff --git a/internal/env/envfakes/fake_proxy_environment.go b/internal/env/envfakes/fake_proxy_environment.go index fa39bb2b05e..05a5d159d92 100644 --- a/internal/env/envfakes/fake_proxy_environment.go +++ b/internal/env/envfakes/fake_proxy_environment.go @@ -157,6 +157,16 @@ type FakeProxyEnvironment struct { redisHostReturnsOnCall map[int]struct { result1 string } + RedisKeyPrefixStub func() string + redisKeyPrefixMutex sync.RWMutex + redisKeyPrefixArgsForCall []struct { + } + redisKeyPrefixReturns struct { + result1 string + } + redisKeyPrefixReturnsOnCall map[int]struct { + result1 string + } RedisPasswordStub func() string redisPasswordMutex sync.RWMutex redisPasswordArgsForCall []struct { @@ -1026,6 +1036,59 @@ func (fake *FakeProxyEnvironment) RedisHostReturnsOnCall(i int, result1 string) }{result1} } +func (fake *FakeProxyEnvironment) RedisKeyPrefix() string { + fake.redisKeyPrefixMutex.Lock() + ret, specificReturn := fake.redisKeyPrefixReturnsOnCall[len(fake.redisKeyPrefixArgsForCall)] + fake.redisKeyPrefixArgsForCall = append(fake.redisKeyPrefixArgsForCall, struct { + }{}) + stub := fake.RedisKeyPrefixStub + fakeReturns := fake.redisKeyPrefixReturns + fake.recordInvocation("RedisKeyPrefix", []interface{}{}) + fake.redisKeyPrefixMutex.Unlock() + if stub != nil { + return stub() + } + if specificReturn { + return ret.result1 + } + return fakeReturns.result1 +} + +func (fake *FakeProxyEnvironment) RedisKeyPrefixCallCount() int { + fake.redisKeyPrefixMutex.RLock() + defer fake.redisKeyPrefixMutex.RUnlock() + return len(fake.redisKeyPrefixArgsForCall) +} + +func (fake *FakeProxyEnvironment) RedisKeyPrefixCalls(stub func() string) { + fake.redisKeyPrefixMutex.Lock() + defer fake.redisKeyPrefixMutex.Unlock() + fake.RedisKeyPrefixStub = stub +} + +func (fake *FakeProxyEnvironment) RedisKeyPrefixReturns(result1 string) { + fake.redisKeyPrefixMutex.Lock() + defer fake.redisKeyPrefixMutex.Unlock() + fake.RedisKeyPrefixStub = nil + fake.redisKeyPrefixReturns = struct { + result1 string + }{result1} +} + +func (fake *FakeProxyEnvironment) RedisKeyPrefixReturnsOnCall(i int, result1 string) { + fake.redisKeyPrefixMutex.Lock() + defer fake.redisKeyPrefixMutex.Unlock() + fake.RedisKeyPrefixStub = nil + if fake.redisKeyPrefixReturnsOnCall == nil { + fake.redisKeyPrefixReturnsOnCall = make(map[int]struct { + result1 string + }) + } + fake.redisKeyPrefixReturnsOnCall[i] = struct { + result1 string + }{result1} +} + func (fake *FakeProxyEnvironment) RedisPassword() string { fake.redisPasswordMutex.Lock() ret, specificReturn := fake.redisPasswordReturnsOnCall[len(fake.redisPasswordArgsForCall)] diff --git a/internal/lb/redis.go b/internal/lb/redis.go index fc2a7101e0e..b37eecb83a4 100644 --- a/internal/lb/redis.go +++ b/internal/lb/redis.go @@ -138,7 +138,7 @@ func (v *redisLoadBalancer) Update(ctx context.Context, server *OriginServer) er } func (v *redisLoadBalancer) Pick(ctx context.Context, streamURL string) (*OriginServer, error) { - key := fmt.Sprintf("srs-proxy-url:%v", streamURL) + key := v.redisKeyURL(streamURL) // Always proxy to the same server for the same stream URL. if serverKey, err := v.rdb.Get(ctx, key).Result(); err == nil { @@ -275,25 +275,36 @@ func (v *redisLoadBalancer) LoadWebRTCByUfrag(ctx context.Context, ufrag string) } func (v *redisLoadBalancer) redisKeyUfrag(ufrag string) string { - return fmt.Sprintf("srs-proxy-ufrag:%v", ufrag) + return v.redisKey(fmt.Sprintf("srs-proxy-ufrag:%v", ufrag)) } func (v *redisLoadBalancer) redisKeyRTC(streamURL string) string { - return fmt.Sprintf("srs-proxy-rtc:%v", streamURL) + return v.redisKey(fmt.Sprintf("srs-proxy-rtc:%v", streamURL)) } func (v *redisLoadBalancer) redisKeySPBHID(spbhid string) string { - return fmt.Sprintf("srs-proxy-spbhid:%v", spbhid) + return v.redisKey(fmt.Sprintf("srs-proxy-spbhid:%v", spbhid)) } func (v *redisLoadBalancer) redisKeyHLS(streamURL string) string { - return fmt.Sprintf("srs-proxy-hls:%v", streamURL) + return v.redisKey(fmt.Sprintf("srs-proxy-hls:%v", streamURL)) +} + +func (v *redisLoadBalancer) redisKeyURL(streamURL string) string { + return v.redisKey(fmt.Sprintf("srs-proxy-url:%v", streamURL)) } func (v *redisLoadBalancer) redisKeyServer(serverID string) string { - return fmt.Sprintf("srs-proxy-server:%v", serverID) + return v.redisKey(fmt.Sprintf("srs-proxy-server:%v", serverID)) } func (v *redisLoadBalancer) redisKeyServers() string { - return fmt.Sprintf("srs-proxy-all-servers") + return v.redisKey("srs-proxy-all-servers") +} + +func (v *redisLoadBalancer) redisKey(key string) string { + if prefix := v.environment.RedisKeyPrefix(); prefix != "" { + return fmt.Sprintf("%v:%v", prefix, key) + } + return key } diff --git a/internal/lb/redis_test.go b/internal/lb/redis_test.go index 6e3c177965f..deb3f9832a8 100644 --- a/internal/lb/redis_test.go +++ b/internal/lb/redis_test.go @@ -641,19 +641,32 @@ func TestRedisLB_StoreWebRTC_SecondSetFails(t *testing.T) { // ---------------------------------------------------------------------------- func TestRedisLB_KeyHelpers(t *testing.T) { - lb := &redisLoadBalancer{} - for _, tt := range []struct { - got, want string + for _, tc := range []struct { + name, prefix, wantPrefix string }{ - {lb.redisKeyUfrag("u"), "srs-proxy-ufrag:u"}, - {lb.redisKeyRTC("url"), "srs-proxy-rtc:url"}, - {lb.redisKeySPBHID("s"), "srs-proxy-spbhid:s"}, - {lb.redisKeyHLS("url"), "srs-proxy-hls:url"}, - {lb.redisKeyServer("id"), "srs-proxy-server:id"}, - {lb.redisKeyServers(), "srs-proxy-all-servers"}, + {"default", "", ""}, + {"configured", "xxx", "xxx:"}, } { - if tt.got != tt.want { - t.Errorf("got %q, want %q", tt.got, tt.want) - } + t.Run(tc.name, func(t *testing.T) { + env := &envfakes.FakeProxyEnvironment{} + env.RedisKeyPrefixReturns(tc.prefix) + lb := NewRedisLoadBalancer(env).(*redisLoadBalancer) + + for _, tt := range []struct { + got, want string + }{ + {lb.redisKeyUfrag("u"), tc.wantPrefix + "srs-proxy-ufrag:u"}, + {lb.redisKeyRTC("url"), tc.wantPrefix + "srs-proxy-rtc:url"}, + {lb.redisKeySPBHID("s"), tc.wantPrefix + "srs-proxy-spbhid:s"}, + {lb.redisKeyHLS("url"), tc.wantPrefix + "srs-proxy-hls:url"}, + {lb.redisKeyURL("url"), tc.wantPrefix + "srs-proxy-url:url"}, + {lb.redisKeyServer("id"), tc.wantPrefix + "srs-proxy-server:id"}, + {lb.redisKeyServers(), tc.wantPrefix + "srs-proxy-all-servers"}, + } { + if tt.got != tt.want { + t.Errorf("got %q, want %q", tt.got, tt.want) + } + } + }) } } diff --git a/memory/2026-02-06.md b/memory/2026-02-06.md deleted file mode 100644 index b79d6350b46..00000000000 --- a/memory/2026-02-06.md +++ /dev/null @@ -1,27 +0,0 @@ -# 2026-02-06 — Daily Log - -## Commit Convention -- Commit titles in this workspace should start with: `OpenClaw:` - -## Why Build an AI Knowledge Base for SRS - -William explained the three layers needed for AI to effectively work on SRS: - -1. **Knowledge base** — Existing docs are written for humans, not AI. Without structured memory, AI can read code and docs but miss the *why* — the background, design thinking, architecture rationale. The knowledge base is built specifically so AI can truly understand SRS. - -2. **Code structure** — The codebase needs to be refined so AI can verify each change. Testable, checkable, AI-friendly structure. - -3. **Code taste** — AI should follow the style and conventions of the existing code. (Nice to have, not strictly required for SRS.) - -## Session: Learning SRS Fundamentals - -- William started teaching me about SRS -- Covered: what SRS is, publisher/player workflow, protocols, ecosystem tools -- Discussed memory organization — decided on dedicated knowledge files instead of putting everything in MEMORY.md -- Created `memory/srs-overview.md` for SRS fundamentals - -## Memory Structure Decision - -- `MEMORY.md` → small, always loaded, high-level pointers -- `memory/srs-*.md` → detailed SRS knowledge, accessed via memory_search -- `memory/YYYY-MM-DD.md` → daily conversation logs diff --git a/memory/srs-codebase-map.md b/memory/srs-codebase-map.md deleted file mode 100644 index 7519eb8e2d6..00000000000 --- a/memory/srs-codebase-map.md +++ /dev/null @@ -1,376 +0,0 @@ -# SRS Codebase Map - -⚠️ **CRITICAL RULE:** Only files explicitly listed in this document are trusted. AI must NOT discover, reference, or use any file that is not listed here. If a file is not in this map, it does not exist for AI purposes. - -This file helps AI navigate the SRS project. Instead of grepping blindly, AI should use this map to reason about which part of the codebase is relevant to a question, then go directly to the right files. - -## Code Structure - -SRS has three codebases: - -### C++ Media Server Code - -The C++ media server (`trunk/src/`) serves as both origin server and edge server. It uses the ST (State Threads) coroutine library and is the full-featured server with all SRS capabilities. - -`main/` — Entry point: -- `main_server` — The `main()` function, starts the server - -`core/` — Foundational definitions: -- `core` — Core includes, macros, config generated by configure -- `core_version` through `core_version8` — Version definitions per major release -- `core_autofree` — SrsUniquePtr smart pointer (RAII) -- `core_deprecated` — Deprecated SrsAutoFree, kept for compat -- `core_performance` — Performance tuning constants (merge-read, etc.) -- `core_platform` — Platform abstraction (32/64-bit, PRId64, large file support) -- `core_time` — Time types and conversions (srs_utime_t, ms/us) - -`kernel/` — Low-level building blocks, no network, no protocol logic: - -Media codecs/containers: -- `kernel_codec` — Codec definitions, H.264/H.265/AAC parsing, sequence headers -- `kernel_aac` — AAC transmuxer (raw AAC stream output) -- `kernel_mp3` — MP3 transmuxer -- `kernel_flv` — FLV tag reader/writer, muxer/demuxer -- `kernel_mp4` — MP4/fMP4 box reader/writer -- `kernel_ts` — MPEG-TS muxer/demuxer -- `kernel_ps` — PS (Program Stream) muxer/demuxer (GB28181) -- `kernel_packet` — Media packet: raw undecoded frame with timing info - -Buffers & I/O: -- `kernel_buffer` — SrsBuffer: binary read/write cursor over a byte array -- `kernel_stream` — SrsSimpleStream: growable byte buffer -- `kernel_io` — Abstract I/O interfaces (ISrsReader, ISrsSeeker, ISrsWriter) -- `kernel_file` — File reader/writer implementations - -RTC primitives: -- `kernel_rtc_rtp` — RTP packet structure, encode/decode -- `kernel_rtc_rtcp` — RTCP packet types, encode/decode -- `kernel_rtc_queue` — RTP ring buffer and reorder queue - -Utilities: -- `kernel_error` — Error code definitions and SrsCplxError -- `kernel_log` — Log levels and ISrsLog interface -- `kernel_consts` — Global constants (chunk sizes, timeouts, limits) -- `kernel_utility` — System time, string helpers, bit buffer -- `kernel_kbps` — Bandwidth statistics (ISrsProtocolStatistic, ISrsClock) -- `kernel_balance` — Load balancing algorithms (round-robin) -- `kernel_hourglass` — Timer/hourglass coroutine (periodic callbacks) -- `kernel_pithy_print` — Sampled logging (print every N seconds, not every call) -- `kernel_factory` — Abstract factory interfaces (ISrsKernelFactory) -- `kernel_resource` — Resource management interfaces (ISrsResource, disposing) -- `kernel_st` — ST coroutine wrappers (ISrsCoroutine, ISrsCoroutineHandler) - -`protocol/` — Protocol implementations (wire formats, parsing, serialization): - -RTMP: -- `protocol_rtmp_stack` — RTMP chunk stream, message parsing, AMF commands -- `protocol_rtmp_conn` — RTMP client/server handshake and command exchange -- `protocol_rtmp_handshake` — RTMP handshake (simple and complex) -- `protocol_rtmp_msg_array` — Shared message array with auto-free -- `protocol_amf0` — AMF0 encoding/decoding - -HTTP: -- `protocol_http_stack` — HTTP message, header, URI, serve mux, handler -- `protocol_http_conn` — HTTP connection parsing (request/response) -- `protocol_http_client` — HTTP client for outbound requests -- `protocol_http_stack_llhttp` — llhttp parser (from Node.js) -- `protocol_http_stack_llhttpadapter` — llhttp adapter for SRS -- `protocol_http_stack_llhttpapi` — llhttp API integration -- `protocol_http_stack_llhttphttp` — llhttp HTTP integration - -RTC/WebRTC: -- `protocol_rtc_stun` — STUN packet parsing/building -- `protocol_sdp` — SDP parsing/building -- `protocol_rtp` — RTP packetizer (H.264/Opus → RTP packets) - -Other protocols: -- `protocol_srt` — SRT socket wrapper -- `protocol_rtsp_stack` — RTSP message parsing - -Media: -- `protocol_raw_avc` — Raw H.264/H.265 stream (Annex-B ↔ AVCC conversion) -- `protocol_format` — Media format: extract metadata, codec info from RTMP stream - -Infrastructure: -- `protocol_io` — Protocol-level I/O interfaces (ISrsProtocolReader/Writer) -- `protocol_st` — ST initialization, socket helpers, TCP/UDP wrappers -- `protocol_stream` — SrsFastStream: merge-read buffered stream -- `protocol_conn` — Base connection interface (ISrsConnection) -- `protocol_log` — Thread context for logging (SrsThreadContext) -- `protocol_json` — JSON parsing (SrsJsonAny) -- `protocol_utility` — URL parsing, stream URL helpers -- `protocol_protobuf` — Protobuf encoding helpers (for Prometheus/exporter) - -`app/` — Application logic (server features, connections, business logic): - -Server core: -- `app_server` — SrsServer: main server, manages listeners and connections -- `app_config` — Configuration parsing and access -- `app_listener` — TCP/UDP listener management -- `app_reload` — Config hot-reload handler interface -- `app_st` — Coroutine implementations (SrsFastCoroutine, SrsExecutorCoroutine) -- `app_log` — File-based logger implementation - -RTMP: -- `app_rtmp_conn` — RTMP client connection lifecycle (publish, play) -- `app_rtmp_source` — RTMP live source: manages consumers, GOP cache, hub - -HTTP: -- `app_http_api` — HTTP API endpoints (/api/v1/...) -- `app_http_conn` — HTTP connection handler (routes to static/stream/api) -- `app_http_hooks` — HTTP callback hooks (on_connect, on_publish, etc.) -- `app_http_static` — HTTP static file serving -- `app_http_stream` — HTTP live streaming (FLV, TS, AAC, MP3 over HTTP) -- `app_http_client` — HTTP client utilities - -WebRTC: -- `app_rtc_server` — RTC server: manages RTC connections -- `app_rtc_conn` — RTC connection: publish/play session, DTLS/SRTP -- `app_rtc_source` — RTC source: manages RTC consumers and tracks -- `app_rtc_api` — WHIP/WHEP API endpoints -- `app_rtc_codec` — Audio transcoding (AAC ↔ Opus via FFmpeg) -- `app_rtc_dtls` — DTLS certificate and handshake -- `app_rtc_network` — RTC network I/O, UDP mux - -SRT: -- `app_srt_server` — SRT server: manages SRT connections -- `app_srt_conn` — SRT connection lifecycle (publish, play) -- `app_srt_source` — SRT source: manages SRT consumers -- `app_srt_listener` — SRT listener (bind/accept) - -RTSP: -- `app_rtsp_conn` — RTSP connection handler -- `app_rtsp_source` — RTSP source - -GB28181: -- `app_gb28181` — GB28181 SIP signaling and PS stream ingestion - -HLS/DASH: -- `app_hls` — HLS muxer (TS segments + M3U8 playlist) -- `app_dash` — DASH muxer (fMP4 segments + MPD manifest) -- `app_fragment` — Fragment/segment interface for HLS/DASH - -DVR: -- `app_dvr` — DVR recording (FLV/MP4 to disk) - -Edge/Forward/Bridge: -- `app_edge` — Edge mode: pull/push from origin -- `app_forward` — Forward: push stream to another RTMP server -- `app_stream_bridge` — Bridge between sources (RTMP↔RTC↔SRT) - -Ingest/Transcode: -- `app_ingest` — Ingest: pull external streams via FFmpeg -- `app_encoder` — Transcode: FFmpeg transcoding management -- `app_ffmpeg` — FFmpeg process wrapper - -Utilities: -- `app_security` — Security: allow/deny rules for publish/play -- `app_refer` — Referer check -- `app_statistic` — Server/stream/client statistics -- `app_heartbeat` — HTTP heartbeat to API server -- `app_utility` — System info, disk/CPU/memory stats -- `app_process` — Child process management -- `app_async_call` — Async HTTP callback execution -- `app_recv_thread` — Background RTMP message receive thread -- `app_ng_exec` — exec hooks (on_publish exec) -- `app_factory` — Application-level factory (file I/O, paths) -- `app_coworkers` — Origin cluster coordination -- `app_latest_version` — Version check against upstream -- `app_caster_flv` — HTTP-FLV push ingest (stream caster) -- `app_mpegts_udp` — MPEG-TS over UDP ingest -- `app_stream_token` — Stream publish token management -- `app_circuit_breaker` — Circuit breaker for overload protection -- `app_hds` — HDS (Adobe HTTP Dynamic Streaming) - -**File naming convention:** All C++ source files use `srs_{module}_{topic}.cpp/.hpp` — e.g. `srs_app_rtmp_conn.cpp`, `srs_protocol_amf0.hpp`, `srs_kernel_flv.cpp`. - -**Config** - -Config (`trunk/conf/`) is only for the C++ media server. The next-generation Go server does not use these config files. - -- `srs.conf` — Default config -- `full.conf` — Full reference config with all options documented -- Feature-specific configs: `rtc.conf`, `srt.conf`, `hls.conf`, `dvr.*.conf`, `edge.conf`, `forward.*.conf`, `gb28181.conf`, `dash.conf`, etc. - -### State Threads Code - -State Threads (`trunk/3rdparty/st-srs/`) is the coroutine library used by the C++ media server only. The Go server does not use ST. - -- `sched.c` — Scheduler (thread creation, context switch, run loop) -- `io.c` — I/O: poll/epoll/kqueue wrappers, socket operations -- `event.c` — Event system -- `stk.c` — Stack allocation and management -- `sync.c` — Mutex, condition variable -- `key.c` — Thread-local storage -- `common.c`/`common.h` — Shared internals -- `public.h` — Public API -- `md.h` — Platform detection and context switch macros -- `md_linux.S`, `md_linux2.S`, `md_darwin.S`, `md_cygwin64.S` — Assembly context switch per platform - -### Next-Generation Server Code - -The next-generation server (`cmd/` + `internal/`) is written in Go and maintained by AI. It is the future of SRS. Currently it only supports proxy, but work is ongoing to support the full feature set including origin, edge, and proxy servers. - -`cmd/proxy` — A stateless reverse proxy that sits in front of one or more SRS C++ origin servers. Accepts client connections (RTMP, HTTP-FLV/HLS, WebRTC WHIP/WHEP, SRT), resolves a backend origin via the load balancer, and transparently proxies traffic. Does not cache streams or process media — only forwards bytes. Configuration entirely via environment variables (or `.env` file), no config file. Supports two deployment modes: single-proxy (in-memory LB) and multi-proxy (Redis-based shared LB for horizontal scaling behind a network load balancer). Entry point: `cmd/proxy/main.go`. - -`internal/bootstrap` — Server startup and lifecycle orchestration. Sets up logging context, signal handlers, loads environment, installs force-quit timer, optionally starts pprof, initializes the load balancer (memory or Redis based on `PROXY_LOAD_BALANCER_TYPE`), then starts all six servers sequentially (RTMP, WebRTC, HTTP API, SRT, System API, HTTP Stream) and blocks until context is cancelled. Deferred `Close()` on each server ensures graceful shutdown. - -`internal/proxy` — Proxy server implementations. Each server accepts client connections, parses just enough of the protocol to extract the stream URL, picks a backend via the load balancer, and proxies traffic bidirectionally. Contains five proxy servers: (1) **RTMP proxy** (`rtmp.go`) — TCP listener, simple handshake, parses connect/publish/play to get stream URL, bidirectional RTMP message copying, stateless. (2) **HTTP stream proxy** (`http.go`) — serves static files, proxies HTTP-FLV/TS via reverse-proxy, proxies HLS m3u8 with `spbhid` rewriting so TS segment requests route to the same backend. (3) **WebRTC proxy** (`rtc.go`) — two-phase: WHIP/WHEP signaling (SDP rewrite to replace backend UDP port with proxy's) + UDP media transport (identifies connections by STUN ufrag, supports address migration), stateful. (4) **SRT proxy** (`srt.go`) — intercepts SRT 4-step handshake locally, parses stream ID on handshake 2, replays full handshake with backend, then proxies UDP bidirectionally, stateful per-connection. (5) **HTTP API + System API** (`api.go`) — HTTP API delegates WHIP/WHEP to WebRTC server; System API provides `/api/v1/srs/register` where backend SRS C++ servers register themselves so the load balancer knows about them. - -`internal/rtmp` — RTMP protocol implementation (parsing, not proxying). Full RTMP chunk stream and message protocol: simple handshake (C0/C1/C2), chunk stream reader/writer with all four format types, extended timestamp, message reassembly from chunks. Defines all RTMP message types, chunk stream IDs, and command names. Packet types include ConnectApp, CreateStream, Publish, Play, Call, SetChunkSize, WindowAcknowledgementSize, SetPeerBandwidth, UserControl. Uses Go generics (`ExpectPacket[T]`) to read until a specific packet type arrives. Also includes full AMF0 encoder/decoder supporting Number, Boolean, String, Object, Null, Undefined, EcmaArray, StrictArray, Date, LongString — with ordered key-value maps, auto-type-discovery, and safe type converters. - -`internal/lb` — Load balancer abstraction and two implementations. Defines `OriginLoadBalancer` interface and core types in `lb.go` (Initialize, Update, Pick, HLS/WebRTC state management) and `OriginServer` struct representing a backend origin (IP, listen endpoints for RTMP/HTTP/API/SRT/RTC, heartbeat tracking). **Memory LB** (`mem.go`) — in-memory using `sync.Map`, sticky random pick per stream URL, single-proxy deployment. **Redis LB** (`redis.go`) — Redis-backed shared state with TTL-based expiration, enables multi-proxy horizontal scaling behind a network load balancer. Also includes a debug helper (`debug.go`) that creates a fake backend from env vars when `PROXY_DEFAULT_BACKEND_ENABLED=on` for development without real SRS registration. - -`internal/redisclient` — Thin Redis client abstraction. Defines a minimal `RedisClient` interface (`Ping`/`Get`/`Set`/`String`) satisfied by `*redis.Client` from `github.com/go-redis/redis/v8`, plus a `New(addr, password, db)` constructor. Used by `internal/lb/redis.go`. - -`internal/logger` — Structured logging with context IDs. Four log levels: Debug/Info (stdout), Warn/Error (stderr). Emits JSON via `log/slog` with `pid` and `cid` attributes. Each connection/request gets a unique 7-char hex context ID for log correlation, stored in `context.Context`. - -`internal/env` — Environment-based configuration. All settings via env vars (or `.env` file parsed by an in-tree custom parser — no third-party dep; supports comments, `export` prefix, quoted values, escape sequences, and inline comments). Exposes a `ProxyEnvironment` interface (with a counterfeiter-generated fake in `envfakes/` for downstream tests) with methods for each config value. Default ports: RTMP=11935, HTTP API=11985, HTTP Stream=18080, WebRTC=18000, SRT=20080, System API=12025. Timeouts: grace=20s, force=30s. Supports Redis config and default backend config for debugging. - -`internal/version` — Version constants. Signature `SRSX`, version tracks the SRS project version (currently 8.0.x). Used in HTTP API responses and startup logging. - -`internal/errors` — Error handling with stack traces, thin wrapper over stdlib `errors`. Provides `New`, `Errorf`, `Wrap`, `Wrapf`, `WithMessage`, `WithStack`, `Cause`, and re-exports `Is`/`As`/`Unwrap`/`Join`. Every error captures a stack trace at creation; `%+v` prints the full trace. `Cause()` walks the error chain to find the root error. - -`internal/sync` — Generic sync primitives. `Map[K, V]`: type-safe generic interface over `sync.Map`, constructed via `NewMap[K, V]()`. Used throughout the codebase to avoid raw type assertions. - -`internal/signal` — OS signal handling. Listens for SIGINT/SIGTERM, cancels the root context. Installs a force-quit timer (default 30s) as a safety net if graceful shutdown hangs. - -`internal/debug` — Profiling. Starts a `net/http/pprof` server if `GO_PPROF` env var is set. Optional, for runtime profiling and debugging. - -`internal/utils` — Shared utility functions. HTTP helpers (JSON response, error response, CORS, body parsing). URL helpers (normalize to `vhost/app/stream` format, extract app/stream from HTTP requests). Protocol helpers (classify UDP payload as STUN/RTP/SRT, parse ICE ufrag/pwd from SDP, parse SRT stream ID). Network helpers (detect peer-closed/connection-closed errors, parse listen endpoints). - -## Knowledge Base Structure - -The knowledge base (`memory/srs-*.md`) captures William's knowledge about SRS — architecture, design decisions, protocols, and how things work. This is the AI's first stop for understanding *why* things are the way they are. - -- `memory/srs-overview.md` — What SRS is, protocols, ecosystem tools, and features list -- `memory/srs-coroutines.md` — State Threads coroutine library, why SRS uses coroutines, how context switching works, platform matrix, multi-CPU strategy -- `memory/srs-codebase-map.md` — This file: codebase structure, file navigation, module boundaries - -## Documentation Structure - -### Tracking Docs - -**Tracking** — Project changelog and version history: -- `trunk/doc/CHANGELOG.md` — Full changelog of all SRS versions, one entry per merged PR with version bump - -### C++ Media Server Docs - -**C++ Media Server Docs** (`trunk/3rdparty/srs-docs/doc/`) — User-facing documentation: -- `introduction.md` — SRS overview: what it is, supported protocols (RTMP/WebRTC/HLS/SRT/etc.), features list, ST coroutine architecture, and learning path -- `getting-started.md` — Quick start with Docker: live streaming (RTMP publish, HTTP-FLV/HLS play), WebRTC publish/play, RTMP-to-WebRTC conversion, HTTPS for WebRTC on remote servers, SRT publishing, and multiple streams with flexible URL patterns -- `getting-started-ai.md` — AI tools for SRS: SRS Robot (Telegram/Discord), local agents (Claude Code, Codex, Kiro, OpenClaw) with pre-configured knowledge base, skills system for task-specific workflows, and the knowledge base philosophy (implicit knowledge made explicit) -- `getting-started-build.md` — Build SRS from source: configure/make, live streaming and WebRTC setup, cross-build for ARM/MIPS -- `getting-started-cdk.md` — Deploy SRS on AWS using srs-cdk (stub page, links to GitHub repo) -- `getting-started-oryx.md` — Oryx (SRS Stack): out-of-the-box cloud solution with recording, forwarding, AI subtitles, HTTPS, Docker/HELM/aaPanel deployment -- `rtmp.md` — RTMP protocol: usage, Enhanced RTMP (HEVC/AV1), config, RTMPS, comparison with SRT/WebRTC, codec support history -- `hls.md` — HLS streaming: cross-platform compatibility, latency tradeoffs, config (segment duration, cleanup), HTTPS, audio transcoding from WebRTC -- `webrtc.md` — WebRTC: WHIP/WHEP signaling, SFU architecture, RTMP-to-RTC conversion, TURN/ICE config, audio transcoding (AAC↔Opus), multi-platform usage -- `flv.md` — HTTP-FLV live streaming: chunked HTTP delivery, ~3-5s latency, config, comparison with HLS and RTMP, browser compatibility (MSE) -- `srt.md` — SRT protocol: UDP-based low-latency (300-500ms), TS encapsulation, HEVC support, config, streamid format, weak network advantages over RTMP -- `rtsp.md` — RTSP playback server (v7.0.47+): TCP transport only, publish via RTMP then play via RTSP, config, compile with --rtsp=on -- `http-server.md` — Embedded HTTP server: serves HLS/static files, API endpoint, config for port/root, works with NGINX/Caddy reverse proxy -- `hevc.md` — H.265/HEVC support: Enhanced RTMP, protocol compatibility matrix (TS/SRT/HLS/RTMP/WebRTC), encoder setup, bandwidth savings vs H.264 -- `dvr.md` — DVR recording: save RTMP streams to FLV/MP4 files, config (plan/path), HTTP callbacks for dynamic DVR, Oryx for advanced features (S3, filters) -- `ingest.md` — Ingest: pull external streams (files, RTSP cameras, HTTP) via FFmpeg and republish as RTMP to SRS, virtual live streaming use case -- `forward.md` — Forward streams to other RTMP servers for fault backup, master/slave roles, config per vhost/app, comparison with edge mode -- `security.md` — IP-based security: allow/deny rules for publish/play per vhost using IP addresses or CIDR ranges -- `snapshot.md` — Stream thumbnails: capture snapshots via HTTP callbacks (on_publish/on_unpublish triggering FFmpeg) or transcoder -- `http-api.md` — HTTP API (/api/v1/...): JSON endpoints for server/stream/client stats, srs-console integration, CORS support -- `http-callback.md` — HTTP callbacks: SRS notifies your business server on events (on_connect, on_publish, on_play, on_dvr, etc.) for auth and custom logic -- `exporter.md` — Prometheus exporter: metrics endpoint for Grafana integration, config for labels/tags, cloud-native observability -- `origin-cluster.md` — Origin cluster: proxy-based load balancing across multiple origin servers, Go proxy architecture, replaces old RTMP-based origin cluster -- `edge.md` — Edge server: caches streams from origin for massive playback scale, pull-on-play and push-on-publish, multi-level edge topology for CDN -- `nginx-for-hls.md` — HLS/DASH distribution via NGINX reverse proxy with caching, edge cluster for file-based streaming (not RTMP edge) -- `resource.md` — Ports reference: RTMP(1935), HTTP API(1985), HTTP streaming(8080), WebRTC(8000/udp), SRT(10080/udp), HTTPS, and optional converters -- `low-latency.md` — Low-latency live streaming: RTMP tuning (0.8-3s), GOP/queue config, merge-write optimization, protocol comparison -- `performance.md` — Performance tools: RTC UDP buffer tuning, perf/gprof/valgrind profiling, memory leak detection, ASAN, benchmarking methodology -- `ffmpeg.md` — FFmpeg transcoding: SRS forks FFmpeg for 1-in-N-out transcoding, multi-bitrate output, stream filters (logo), config per vhost/app/stream - -**C++ Media Server Pages** (`trunk/3rdparty/srs-docs/pages/`) — Website pages (how SRS presents itself publicly): -- `faq-oryx-en.md` — Oryx FAQ: getting started, upgrade, domain/HTTPS setup, push auth, recording (S3/filters/merge), re-streaming, FFmpeg replacement -- `faq-server-en.md` — SRS server FAQ: CDN/VoD questions, common errors, protocol issues, links to Stack Overflow/GitHub Issues/Discord for support -- `license-en.md` — License details: SRS is MIT, State Threads is MPL/GPL dual-license, plus licenses for all third-party libraries (OpenSSL, libsrt, ffmpeg, etc.) -- `product-en.md` — Release history: milestones from v1.0 (2013) to v7.0 (2025+), codenames, key achievements per release, AI contributor era (2025+) -- `security-advisories-en.md` — CVE reports: CVE-2024-29882 (JSONP XSS), CVE-2023-34105 (command injection in demo api-server), with patch versions and links - -### Next-Generation Server Docs - -**Next-Generation Server Docs** (`docs/proxy/`) — Documentation for the Go server: -- `features.md` — Feature catalog: implemented proxy capabilities by area (protocol proxying, HTTP APIs, load balancing, backend registration, deployment modes, configuration, operations) with current limitations -- `proxy-design.md` — Architecture: stateless proxy with built-in load balancing, single-proxy vs multi-proxy (Redis) deployment modes, horizontal scaling behind NLB -- `proxy-protocol.md` — Backend registration protocol: default backend for debugging, automatic registration via SRS system API, heartbeat mechanism, env vars -- `proxy-usage.md` — Step-by-step guide: build proxy, start SRS origin, verify registration, publish with FFmpeg, verify playback with ffprobe -- `proxy-load-balancer.md` — Load balancer design: memory vs Redis implementations, stream-to-server mapping, server health via heartbeats, protocol-specific state -- `proxy-origin-cluster.md` — Origin cluster tutorial: build proxy + SRS, configure multi-origin with proxy, stream publishing and playback verification - -**Next-Generation Server Performance Docs** (`docs/perf/`) — Performance analysis guides for the Go server: -- `proxy-whep.md` — WHEP perf analysis: enable GO_PPROF, run publisher + N WHEP players via srs_bench, collect CPU/alloc/heap/goroutine/trace profiles, read hot spots, diff before/after with `pprof -base` - -**Next-Generation Server API Examples** — Executable API documentation: -- `internal/rtmp/example_test.go` — RTMP API examples: AMF0, handshake, and protocol workflow - -## Testing and Verification Structure - -How to verify SRS works correctly. - -`trunk/src/utest/` — Unit tests (white-box, testing internal logic): -- **What it tests:** Internal logic — functions, classes, parsing, codec handling, config parsing — all without starting SRS. Links directly against SRS source code, uses mock objects (e.g., `MockSrsConfig`). No network, no server process. ~2,351 tests. -- **Binary:** `./objs/srs_utest` (build with `cd trunk && ./configure --utest && make utest`) -- 3 categories: -- `srs_utest_ai01`–`ai24` — AI-written tests -- `srs_utest_manual_*` — Manually written tests -- `srs_utest_workflow_*` — Workflow/integration tests - -`trunk/3rdparty/srs-bench/blackbox/` — Black-box tests (testing SRS as a running server from outside): -- **What it tests:** SRS as a running process, from the outside. Each test starts its own SRS server via `NewSRSServer()`, then uses FFmpeg/FFprobe to publish/play streams and verify output. The test manages the full SRS lifecycle. ~30 tests. -- **Binary:** `./objs/srs_blackbox_test` (build with `make` in `trunk/3rdparty/srs-bench/`) -- `rtmp_test.go` — RTMP black-box tests -- `hls_test.go` — HLS black-box tests -- `srt_test.go` — SRT black-box tests -- `rtsp_test.go` — RTSP black-box tests -- `hevc_test.go` — HEVC black-box tests -- `dvr_test.go` — DVR black-box tests -- `http_api_test.go` — HTTP API black-box tests -- `mp3_test.go` — MP3 black-box tests - -`trunk/3rdparty/srs-bench/srs/` — E2E tests (end-to-end protocol testing): -- **What it tests:** End-to-end protocol correctness against a pre-running SRS server (you start SRS yourself). Uses real protocol clients — Pion WebRTC library for RTC (WHIP/WHEP), RTMP client, HTTP API — to publish, play, and verify stream behavior. Heavy focus on WebRTC protocol correctness. ~45 tests. -- **Binary:** `./objs/srs_test` (build with `make` in `trunk/3rdparty/srs-bench/`) -- `rtc_test.go` — WebRTC E2E tests -- `rtmp_test.go` — RTMP E2E tests -- `srs_test.go` — General SRS E2E tests - -`trunk/3rdparty/srs-bench/` — Load test benchmark: -- **What it tests:** Performance and load capacity — not correctness. A CLI tool that simulates many concurrent WHIP publishers, WHEP players, RTMP clients, etc. against a running SRS. Not a test suite — no pass/fail assertions, it's a benchmarking/stress tool. -- **Binary:** `./objs/srs_bench` (entry: `main.go`, build with `make` in `trunk/3rdparty/srs-bench/`) -- Publisher for WHIP -- Player for WHEP -- Multiple WHIP or WHEP for RTC -- DVR for RTC Benchmark -- RTC Plaintext -- Reconnecting Load Test -- Janus - -`skills/srs-develop/scripts/` — Go proxy verification and setup scripts: -- `proxy-utest.sh` — Runs Go proxy unit tests with optional coverage. -- `proxy-e2e-test.sh` — Single-origin RTMP proxy E2E test. -- `proxy-e2e-cluster-test.sh` — Multi-origin memory load-balancer E2E test. -- `proxy-e2e-redis-test.sh` — Multi-proxy Redis load-balancer E2E test. -- `proxy-e2e-transmux-test.sh` — RTMP publish through proxy, then verify RTMP, HTTP-FLV, HLS, and WebRTC playback. -- `proxy-e2e-srt-test.sh` — SRT publish through proxy, then verify SRT, RTMP, HTTP-FLV, and HLS playback (WebRTC WHEP is a placeholder). -- `proxy-e2e-whip-test.sh` — WHIP (WebRTC) publish through proxy, then verify RTMP, HTTP-FLV, and HLS playback via the origin's `rtc_to_rtmp` bridge (WebRTC WHEP is a placeholder). -- `setup-ffmpeg-with-whip.sh` — macOS-only: build ffmpeg from source into `~/.local/` with WHIP (openssl DTLS) and SRT support; auto-invoked by `proxy-e2e-srt-test.sh` and `proxy-e2e-whip-test.sh` when no suitable ffmpeg is found. - -**Summary: The Key Differences** - -| | Unit Tests | Black-box | E2E | Benchmark | -|---|---|---|---|---| -| **SRS running?** | No | Yes (self-managed) | Yes (external) | Yes (external) | -| **Tests what?** | Internal logic | Whole server behavior | Protocol correctness | Performance/load | -| **Language** | C++ | Go | Go | Go | -| **Network?** | No | Yes (real protocols) | Yes (real protocols) | Yes (real protocols) | -| **Pass/fail?** | Yes | Yes | Yes | No (metrics only) | -| **Count** | ~2,351 | ~30 | ~45 | N/A | - -Progression: unit (isolated logic) → black-box (server as a whole, self-contained) → E2E (protocol-level against live server) → benchmark (stress/capacity). Each layer catches different classes of bugs. diff --git a/memory/srs-coroutines.md b/memory/srs-coroutines.md deleted file mode 100644 index 83497d3e86e..00000000000 --- a/memory/srs-coroutines.md +++ /dev/null @@ -1,928 +0,0 @@ -# SRS Coroutines - -SRS uses **State Threads (ST)** — a C coroutine library that provides lightweight user-space threads. It is the cornerstone of SRS's architecture. - -**Key insight:** ST gives SRS the programming model of Go (one coroutine per connection, sequential code, state in local variables) but in C/C++. It's essentially a C implementation of Go's concurrency model, used by SRS's C++ codebase. - -This is why the code is maintainable despite handling thousands of concurrent connections — each connection handler reads like a simple sequential function. - -## Why a Media Server Needs to Manage State per Connection - -A media server must serve many connections simultaneously — thousands of RTMP, HTTP, WebRTC clients at once. Each connection has **state**: handshake data, protocol parameters, stream URLs, buffers. This state lives in local variables and function call stacks. - -Example: An RTMP client connects → you accept the TCP connection → do RTMP handshake (read bytes, generate response, store handshake state) → then enter the RTMP connect phase (read TC URL, host, stream ID, parameters). All of this is state for that one connection. - -Now multiply by thousands of connections. You need to serve one, switch to another, come back to the first — like talking to hundreds of people at once, each conversation having its own context. - -## Three Approaches to Managing Per-Connection State - -**1. OS Threads (one per connection)** -- Each thread has its own call stack and local variables → state is naturally stored -- Writing code is easy — just sequential logic per connection -- **Problem:** OS threads are expensive. Thousands of threads = expensive context switching, poor performance - -**2. Async/Event Loop (Nginx model)** -- Single thread, big poll loop: poll/wait → serve connection A → save state → return → serve connection B → ... -- **Problem:** You must manually save ALL state into a connection struct because local variables are destroyed when you return from the function. Code becomes much more complex to maintain. - -**3. Coroutines / User-Space Threads (Go model, SRS model)** -- Lightweight threads in user space — same benefits as OS threads (local variables, call stacks, natural state storage) but without the OS overhead -- Each connection gets its own coroutine with its own stack -- Code reads like simple sequential logic (like OS threads) but performs like async -- **This is what Go does with goroutines. SRS does the same thing using the State Threads (ST) library — a C library used by SRS's C++ codebase.** - -## How Coroutine Switching Works - -The tradeoff: coroutines make application code easy, but **someone has to implement the coroutine mechanism itself**. - -When serving a connection, you can't just call a function to switch to another connection — that would push onto the same call stack. Instead, you need a **lightweight thread switch**: save the current coroutine's CPU registers to its memory, then load the target coroutine's saved registers and resume from where it left off. - -This is the same concept as OS thread context switching, but: -- **OS thread switch:** heavy, involves kernel, expensive -- **Coroutine switch:** user-space only, just save/restore registers, very cheap - -ST originally used libc's `setjmp`/`longjmp` for context switching. But glibc later started encrypting (mangling) the saved context for security, making it impossible to manipulate the stack pointer from user code. So ST had to reimplement setjmp/longjmp in pure assembly — that's what `_st_md_cxt_save`/`_st_md_cxt_restore` are. They do exactly what setjmp/longjmp do (save and restore callee-saved registers, stack pointer, and program counter) but without glibc's encryption, giving ST full control over coroutine stacks. - -To implement this, you need to understand how function calls work at the CPU level — registers, stack pointers, program counters. The coroutine library handles all of this so application code never has to think about it. - -## Timeout Heap: How ST Manages Sleeping Coroutines - -When a coroutine calls `st_usleep()` or any I/O function with a timeout, ST puts it to sleep and must wake it at the right time. This requires a data structure that efficiently tracks all sleeping coroutines ordered by their wake-up time. - -**Original design (pre-1.5):** A sorted linked list. Insertion was O(N) — for every new sleeper, ST walked the list to find the right position. With thousands of sleeping coroutines, this became a bottleneck. - -**Current design (since ST 1.5):** A binary heap implemented as a balanced binary tree using pointers (not an array). This gives O(log N) insertion and removal. In benchmarks, 1 million sleep queue insertions/removals dropped from 100 seconds to 12 seconds. - -**Why a pointer-based tree instead of an array?** ST's codebase is structured around linking `_st_thread_t` objects via embedded pointers — no auxiliary data structures. The heap reuses this pattern: each thread object has `left` and `right` child pointers, and `_st_this_vp.sleep_q` points to the root (the thread with the earliest timeout). The tree stays fully balanced and left-adjusted, numbered like an implicit array heap (node N has children 2N and 2N+1), but navigated via pointers from root to leaves using the binary digits of the target index. - -**The heap invariant:** Parents always time out before children, so the root is always the next coroutine to wake. This is how ST's scheduler knows which timer fires next when it enters epoll/kqueue — it just checks the root's timeout. - -## ST Library Origin and Design - -State Threads is derived from Netscape Portable Runtime (NSPR). It's not a general-purpose threading library — it specifically targets Internet Applications (servers that are network I/O driven). - -**License:** ST is dual-licensed under **MPL 1.1 or GPLv2** (user's choice). Example code is BSD-licensed. MPL 1.1 is a weak copyleft — changes to MPL-licensed *files* must stay open, but MPL code can be combined with code under other licenses (including proprietary). This is compatible with SRS's MIT license. The GPLv2 option is there for projects that prefer GPL-family licensing. - -Key design properties: -- **Deterministic scheduling:** Context switch can only happen at I/O points or explicit synchronization points — never preemptive, never time-sliced -- **No locks needed:** Because switching is deterministic, global data doesn't need mutex protection in most cases. The entire application can freely use static variables and non-reentrant library functions -- **Minimal syscalls:** No per-thread signal mask (unlike POSIX threads), so no save/restore of signal mask on context switch — eliminates two syscalls per switch -- **~5000 lines of code:** Small enough to understand completely, but requires assembly per CPU/OS platform - -SRS maintains the fork at `ossrs/state-threads` (branch `srs`), continuously updating it to support modern CPUs and OSes including Linux, macOS, Windows, and architectures like x86_64, ARMv7, AARCH64, Apple M1, RISC-V, LoongArch, and MIPS. - -## Non-Network I/O: The Disk Read Problem - -ST's non-blocking I/O only works for network sockets. **Disk I/O, device I/O, and stdin all block the entire process** — every coroutine stalls until the operation completes. Disk writes are usually fine (they go to buffer cache), but disk reads can block for unpredictable durations. This is a known limitation of ST's architecture that still exists in SRS. - -## ST's Key Constraint and Design Tradeoffs - -ST has one fundamental constraint: **all socket I/O must use ST's own I/O functions** (`st_read`, `st_write`, `st_accept`, etc.). If application code calls raw `read(2)` or `write(2)` on a socket, it bypasses ST's scheduler — the entire process blocks, and all coroutines stall. This is why integrating third-party libraries (like libsrt) requires wrapping their I/O to be coroutine-aware (see "Coroutine-Native SRT" above). - -**Signal handling:** ST's scheduler only detects two event types: I/O readiness and timeouts. To handle signals (like `SIGINT` or `SIGUSR1`), the standard pattern is to convert them to I/O events — the signal handler writes a byte to a pipe, and a coroutine reads from that pipe via `st_read`. This works because `write(2)` is async-signal-safe. - -## The Burden: Maintaining a C Coroutine Library - -Coroutines are a fantastic idea for a media server, but unlike Go (where goroutines are built into the language and runtime), **C/C++ has no standard coroutine library for this model**. (Note: C++20 co_await/co_yield is a different mechanism — not the same as user-space threads with full stacks.) - -**Platform Support Matrix** -The coroutine switch must be implemented in **assembly language per CPU architecture**: ARM, ARMv8/AArch64, x86_64, MIPS — each has different register conventions. Multiply by OS (Linux, macOS, Windows) and you get a support matrix that is a maintenance burden. - -Nobody else actively maintains this library — SRS must maintain it ourselves. Very few people understand coroutine switching at this level. - -**The Windows/SRT Problem (Why SRS 6 Dropped Windows)** -- SRS added Windows support using a custom coroutine implementation (fiber/win64-based) -- SRT (Secure Reliable Transport) is a multi-thread library that uses **C++ exceptions** -- On Windows, C++ exceptions use a platform-specific mechanism (SEH — Structured Exception Handling) that **conflicts with the coroutine stack switching** -- This caused crashes that were extremely difficult to diagnose -- William investigated but could not fix it — the interaction between SEH and custom stacks is poorly documented -- **Result:** SRS 6 removed Windows support because SRT + coroutines couldn't coexist on Windows -- **Important distinction:** SRS (the server) dropped Windows support, but ST (the coroutine library) still retains Cygwin64 support — Cygwin64-related files were not removed from ST. ST is a standalone coroutine library and there's no reason to remove working platform support from it just because SRS the server no longer targets Windows. - -**Toolchain Gap** -Go provides built-in tools: goroutine stack traces, scheduling profilers, debuggers that understand goroutines. ST has a simple coroutine scheduler driven by I/O events and timers (not an OS thread scheduler), and includes basic `DEBUG_STATS` instrumentation (scheduler timing distribution, thread run/idle/yield counts, per-I/O-call and EAGAIN stats, epoll dispatch stats). But compared to Go's tooling: -- ST includes **GDB helper scripts**: `nn_coroutines` (show count of coroutines) and `show_coroutines` (list all coroutines with their caller functions). These provide basic coroutine-aware debugging within GDB. However, compared to Go's integrated tooling (goroutine stack traces in panics, `runtime/pprof`, scheduler tracing), these are manual GDB extensions rather than native runtime instrumentation. -- No high-level performance analysis or visualization for coroutine scheduling -- Instrumentation exists but is basic counters, not integrated tooling - -**Debugging and Profiling Limitations** -- `perf -g` (stack traces) does not work with ST because ST modifies the stack pointer (SP), breaking frame pointer-based stack walking -- Valgrind requires ST-specific hooks, supported since SRS 3+ -- ASAN (Address Sanitizer) is supported since SRS 5+, enabled by default in SRS 5, disabled by default in SRS 6 because it sometimes causes crashes for unknown reasons -- **Testing:** ST has a unit test suite using Google Test (gtest), with code coverage via gcov/gcovr. Tests can be built with `make linux-debug-utest` or `make darwin-debug-utest`. -- These tools help but are workarounds — there are still no native tools that understand coroutine scheduling the way Go's runtime tools understand goroutines - -**Can AI Help?** -This is a niche domain — not common knowledge. But AI has access to all the code, assembly specs, and documentation. There's hope that AI could maintain the coroutine library (especially for new CPU/OS ports), but it's unproven. The Windows/SEH problem is an example of something that might be too complex even for AI — or might be exactly where AI excels. - -## Valgrind Support - -Valgrind can't track ST coroutines by default because `setjmp`/`longjmp` switches the stack pointer to custom-allocated stacks that Valgrind doesn't know about, causing false positives. - -**The fix** (merged from [toffaletti's fork](https://github.com/toffaletti/state-threads/commit/7f57fc9acc05e657bca1223f1e5b9b1a45ed929b), [commit 4cca7a0](https://github.com/ossrs/state-threads/commit/4cca7a0272b70b184742dd68065af8a9a42e030f)): -- Uses `VALGRIND_STACK_REGISTER(top, bottom)` when creating a coroutine to tell Valgrind about the custom stack -- Uses `VALGRIND_STACK_DEREGISTER(id)` when the coroutine exits -- Stores the registration ID in `_st_stack_t.valgrind_stack_id` -- Skips the primordial thread (its stack is the normal process stack, already known to Valgrind) - -**Opt-in via compile flag:** `-DMD_VALGRIND -I/usr/local/include`. Zero overhead when not enabled — `NVALGRIND` is defined by default to disable all Valgrind macros. - -**What changed (3 files):** -- **common.h** — Added `MD_VALGRIND`/`NVALGRIND` macro logic; added `valgrind_stack_id` field to `_st_stack_t` -- **sched.c** — Included ``; added `VALGRIND_STACK_REGISTER` in `st_thread_create()` and `VALGRIND_STACK_DEREGISTER` in `st_thread_exit()` -- **README** — Added build instructions for Linux with Valgrind - -## Stack Memory Management: Cache vs Free - -By default, ST caches all thread stacks forever — when a coroutine exits, its stack goes onto a free list and gets reused by the next `_st_stack_new` call. This is efficient for long-running servers with stable thread counts, but wastes memory when threads are short-lived (stacks accumulate and never shrink). - -**Compile-time flag `MD_CACHE_STACK`** ([state-threads#38](https://github.com/ossrs/state-threads/issues/38), [commit b019860](https://github.com/ossrs/state-threads/commit/b01986064cf01de86cea7b24a2f95e7114ba3d75)) controls the behavior: - -- **With `MD_CACHE_STACK`** (original behavior): Freed stacks stay on the free list. `_st_stack_new` searches the list for a stack of sufficient size before allocating a new one. -- **Without `MD_CACHE_STACK`**: Stacks are actually freed (`munmap`/`free`). When `_st_stack_new` runs, it first drains the entire free list — unmapping every cached stack — then allocates fresh. - -**Why not free immediately in `_st_stack_free`?** When a coroutine exits, it's still *running on its own stack* during cleanup. Freeing the stack out from under a running coroutine would crash. So `_st_stack_free` always appends to the free list, and the actual deallocation happens later in `_st_stack_new` (when a different coroutine is running on a different stack). The re-enabled `_st_delete_stk_segment` function handles the actual `munmap` or `free`. - -## Coroutine-Native SRT - -SRS 4.0 (2019) added SRT support, but the initial implementation used libsrt's own threads and async I/O, separate from ST. This caused complex async code was difficult to maintain. - -In SRS 5.0, SRT was rewritten to be **coroutine-native** ([srs#3010](https://github.com/ossrs/srs/pull/3010)). The pattern for making any protocol coroutine-native: - -1. Call the protocol's API (e.g., `srt_recvmsg`) -2. If success, return the data -3. If the error is not "would block" (e.g., `SRT_EASYNCRCV`), return the error -4. If "would block", switch the current coroutine via `st_cond_t` condition variable and let other coroutines run -5. When the fd becomes ready (detected by `srt_epoll_uwait` in a poller coroutine), signal the condition variable to wake the waiting coroutine -6. Repeat from step 1 - -This is the same pattern ST uses internally for TCP (`st_read` handles `EAGAIN` the same way), just adapted to SRT's epoll API. - -**The maintainability win:** In callback/async style, connection state must live in global data structures and gets modified by different event callbacks — the object lifecycle is scattered across the event loop. In coroutine-native style, state lives in local variables on the coroutine stack, and the lifecycle is linear and contained in one coroutine function. This is the fundamental reason SRS uses coroutines. - -**Remaining issue:** libsrt uses C++ exceptions internally, which still causes the Windows/SEH compatibility problem described above. The coroutine-native rewrite solved the threading and maintainability issues but did not solve Windows portability. The fix requires either rewriting libsrt to avoid C++ exceptions or fixing the SEH/coroutine stack interaction on Windows. Not fixed yet, planned for the future. - -## Multi-CPU: Cluster, Not Multi-Threading - -**Problem:** SRS uses single-threaded coroutines → only saturates one CPU core. Modern servers have many cores. - -**Why Not Multi-Threading?** -ST library actually supports multi-threading, and William added multi-thread support. But it turned out to be a disaster: - -- Even with thread-local isolation (separate thread-safe coroutine schedulers), threads must still **communicate** with each other -- **The biggest problem: load balancing between threads is nearly impossible to estimate.** Different threads have different capacity, and you can't easily observe the load distribution -- With single-thread: observing load is trivial — one CPU, 60-70% threshold, done -- With multi-thread: complexity explodes, load becomes opaque -- **William's verdict: multi-threading doesn't solve the multi-CPU problem — it creates new, worse problems. It's a trauma maker.** - -**The Right Solution: Proxy + Origin + Edge Cluster** - -This is a **settled and confirmed decision**: SRS will remain single-process, single-threaded with coroutines. Multi-threading will be removed from SRS. The multi-CPU problem is solved entirely by the cluster architecture: - -- **Proxy** (implemented in Go): Stateless, horizontally scalable, synchronizes state through Redis. Supports all protocols (RTMP/FLV/HLS/SRT/WebRTC). Proxies API and media traffic to Origin servers. -- **Origin** (SRS, C++): Single-threaded with coroutines. Handles stream processing and protocol conversion. -- **Edge** (SRS, C++): Single-threaded with coroutines. Caches streams from Origin for massive playback distribution. - -Multiple Origins behind a Proxy, combined with Edge servers, can scale to thousands of streams and tens of thousands of viewers per stream. Each component stays simple and observable — one CPU, one process, coroutines. - -## Multi-threading Timeline (Historical) - -SRS has traditionally been single-process, single-threaded, akin to a single-process version of Nginx, with the addition of coroutines for concurrent processing. Coroutines are implemented using the StateThreads library, which has been modified to support thread-local functionality for operation in a multi-threaded environment. - -Despite experimenting and analyzing thread-local handling for a media architecture over the years, SRS has not adopted a thread-local approach but rather a different multi-threaded architecture that is still in the planning stage: Stream processing occurs on a single thread, while blocking operations like logging, file writing, and DNS resolution are handled by separate threads. In essence, SRS uses multi-threading to address blocking issues. If Linux supports fully asynchronous I/O in the future, multi-threading may not be necessary, as seen in liburing. - -StateThreads multi-threading faces issues with Windows C++ exception handling. Windows' exception mechanism differs from Linux, causing compatibility problems when StateThreads implements setjmp and longjmp, as discussed in SEH. - -Challenges with multi-thread scheduling and load balancing: While thread-local multi-threading addresses multi-core utilization, it still limits the need for streaming and playback to a single thread, preventing complete load balancing across multiple threads. Without thread-local functionality, serious locking and competition issues arise. Essentially, it's like running multiple K8s Pods within a single process and handling scheduling, monitoring, and load balancing internally, which can be quite complex. - -In SRS 5.0, StateThreads were restructured to support thread-local functionality and initiated a main thread and subthreads to transition the architecture into a multi-threaded model. However, various issues arose during subsequent stages, leading to a default return to a single-threaded architecture in SRS 6.0. Multi-threading capabilities will be removed as the Proxy and Edge cluster architecture fully replaces them. - -Additionally, we explored another potential architecture where specific capabilities are distributed across different threads, like using separate threads for WebRTC encryption and decryption. However, this approach transforms into a typical multi-threaded program rather than a thread-local architecture, resulting in performance overhead from locks and reduced stability — not an ideal direction. - -## How `__thread` Makes ST Thread-Safe - -ST's multi-threading model is simple: **one pthread, one ST scheduler**. It uses GCC's `__thread` so scheduler state is thread-local, not shared global state. - -This approach came from [toffaletti's fork](https://github.com/toffaletti/state-threads) and was later adopted by ossrs/state-threads ([state-threads#19](https://github.com/ossrs/state-threads/issues/19)). - -In practice, key runtime state is thread-local: current thread, VP/scheduler state, event backend data, free stack list, and key/destructor tables. `st_init()` initializes each thread's runtime (including calling `_st_io_init()` directly). In current code, the netfd freelist is also thread-local (`static __thread _st_netfd_t *_st_netfd_freelist`), so no mutex is needed there. - -**Design takeaway:** ST scales by isolation, not heavy locking. Each pthread runs an independent coroutine runtime with its own run queue, timers, and event loop. - -**Why SRS still moved away:** This works well inside ST, but SRS still faced hard cross-thread coordination and load-balancing problems at the application level. The project chose Proxy + Origin + Edge cluster architecture for multi-CPU scaling instead. - -## Porting ST to New Platforms - -Porting ST to a new OS/CPU is simpler than it sounds. The core task is implementing two assembly functions: `_st_md_cxt_save` (save registers) and `_st_md_cxt_restore` (restore registers) — the custom replacements for `setjmp`/`longjmp`. - -**Current platform support (from [state-threads#22](https://github.com/ossrs/state-threads/issues/22)):** - -- **Linux + i386** — Stable. 32-bit x86 systems. -- **Linux + x86-64** — Stable. CentOS, Ubuntu server, etc. -- **Linux + ARM (v7)** — Stable. Raspberry Pi and ARM devices. ([state-threads#1](https://github.com/ossrs/state-threads/issues/1)) -- **Linux + AArch64 (ARMv8)** — Stable. ARM servers. ([state-threads#9](https://github.com/ossrs/state-threads/issues/9)) -- **Linux + MIPS** — Dev. OpenWRT devices. ([state-threads#21](https://github.com/ossrs/state-threads/issues/21)) -- **Linux + MIPS64** — Dev. Loongson 3A4000/3B3000. ([state-threads#21](https://github.com/ossrs/state-threads/issues/21)) -- **Linux + LoongArch64** — Dev. Loongson 3A5000/3B5000, new ISA replacing MIPS. ([state-threads#24](https://github.com/ossrs/state-threads/issues/24)) -- **Linux + RISC-V** — Dev. StarFive boards. ([state-threads#28](https://github.com/ossrs/state-threads/pull/28)) -- **macOS + x86-64** — Stable. Intel Macs. ([state-threads#11](https://github.com/ossrs/state-threads/issues/11)) -- **macOS + AArch64 (M1/M2)** — Dev. Apple Silicon. ([state-threads#30](https://github.com/ossrs/state-threads/issues/30)) -- **Windows + x86-64 (Cygwin64)** — Dev. 64-bit only, no 32-bit Windows. ([state-threads#20](https://github.com/ossrs/state-threads/issues/20)) - -"Stable" means production-tested in SRS deployments. "Dev" means implemented and working but less field-tested. - -**Why custom assembly instead of libc's setjmp/longjmp?** -Early ST used glibc's `setjmp`, then modified the `jmp_buf` to swap the stack pointer to a heap-allocated coroutine stack. This required knowing glibc's internal `jmp_buf` layout. But newer glibc versions started **encrypting (pointer-mangling)** the saved registers inside `jmp_buf`, making it impossible to modify the SP from user code. The fix: implement save/restore entirely in assembly with ST's own `jmp_buf` layout. This is actually more portable — CPU register ABIs are stable and well-documented, while glibc internals are not. **All platforms now use custom assembly exclusively** — the libc setjmp path has been completely removed (attempting to use it is a compile error). Every OS/CPU goes through `_st_md_cxt_save`/`_st_md_cxt_restore` in the `.S` files. - -**Assembly files are organized by OS, not CPU:** -- `md_linux.S` — Linux x86 platforms: i386, amd64/x86_64 -- `md_linux2.S` — Linux non-x86 platforms: aarch64, arm, riscv, mips64, mips, loongarch64 -- `md_darwin.S` — macOS/Darwin (different calling conventions and object format) -- `md_cygwin64.S` — Windows via Cygwin64 - -Within each file, CPU-specific sections are selected by `#ifdef` macros (`__x86_64__`, `__aarch64__`, `__mips__`, `__loongarch64`, `__riscv`, etc.). - -> Note: All `.S` files check `MD_ST_NO_ASM` — historically this allowed disabling assembly and falling back to libc's `setjmp`/`longjmp`. Since the libc setjmp path has been removed (all platforms now require assembly), this macro no longer works — defining it will cause linker errors. It remains in the code as a leftover. - -**What registers to save?** -Only the **callee-saved registers** matter — these are the registers a function must preserve across calls. The actual registers saved by ST's assembly (from the `.S` files): - -- **i386 (Linux):** ebx, esi, edi, ebp, esp, pc -- **x86-64 (Linux/Darwin/Cygwin64):** rbx, rbp, r12-r15, rsp, pc -- **ARM v7 (Linux):** v1-v6, sl, fp, sp, lr (i.e., r4-r9, r10, r11, r13, r14); optionally VFP d8-d15 and iWMMXt wr10-wr15 -- **AArch64 (Linux/Darwin):** x19-x28 (callee-saved), x29 (frame pointer), x30/lr (link register), sp, plus floating-point d8-d15 -- **MIPS/MIPS64 (Linux):** sp, gp, fp/s8, s0-s7, ra -- **LoongArch64 (Linux):** sp (r3), ra (r1), fp (r22), s0-s8 (r23-r31) -- **RISC-V (Linux):** sp, ra, fp/s0, s1-s11 - -**The jmpbuf problem (historical, now resolved):** -Different platforms define `jmp_buf` differently — field names (`__jmpbuf` vs `__jb`), field sizes, and layouts all varied. This was a problem when ST relied on the platform's `jmp_buf`. [state-threads#29](https://github.com/ossrs/state-threads/pull/29) resolved this by having ST define and use its own `_st_jmp_buf_t` structure (`long[22]` — sized for the largest platform, AArch64) instead of relying on platform-specific layouts. All platforms now use this unified structure, eliminating the cross-platform jmpbuf compatibility issue entirely. - -The macro `MD_GET_SP(_t)` in `md.h` defines how to read/write the stack pointer in ST's own jmpbuf for each platform. This is critical for `MD_INIT_CONTEXT` — when creating a coroutine, the SP in the saved context must be updated to point at the heap-allocated stack, since the coroutine can't use the creator's stack. - -**Porting toolkit (`tools/` directory):** -Six utilities help with any new port: - -- **`porting.c`** — Prints detected OS/CPU macros, pointer sizes, and calling convention info. Run this first to understand your platform. -- **`helloworld.c`** — Minimal ST validation: `st_init()` + loop with `st_sleep()`. If this prints, context switching works. -- **`verify.c`** — Full API test: thread creation, mutex, cond variable, usleep, thread join. Validates the complete ST threading model. -- **`jmpbuf.c`** — Shows the platform's `jmp_buf` struct definition via preprocessor expansion, useful for understanding field layout differences. -- **`pcs.c`** — Analyzes the Procedure Call Standard (which registers are caller vs callee-saved). -- **`stack.c`** — Inspects stack behavior on the platform. - -**Porting steps (using MIPS/OpenWRT as the reference example from [state-threads#21](https://github.com/ossrs/state-threads/issues/21)):** - -1. **Detect CPU macro:** `g++ -dM -E -
` to convert offsets to source file:line -- Use `nm | grep ` to find function base addresses, then compute offsets -- `objdump -d ` can verify the relationship between addresses and instructions - -This complements ST's GDB helper scripts (`nn_coroutines`, `show_coroutines`) as another debugging tool for coroutine-based code. - -## Timeout Semantics - -ST timeouts have a subtle but important behavior: the timeout parameter is relative to `last_clock` (the timestamp of the last scheduler cycle), not the moment the function is called. - -When you call `st_read(fd, buf, n, timeout)`, internally ST computes the deadline as `due = last_clock + timeout` (in `_st_add_sleep_q`). The `last_clock` value is updated in `_st_vp_check_clock()`, which only runs during scheduler cycles — in the idle thread loop after `dispatch()` returns, and in `st_thread_yield()`. Between those points, `last_clock` is frozen. - -**The cancelling effect:** Both the deadline and the `epoll_wait` timeout are computed from the same stale `last_clock`: `due = last_clock + timeout`, `min_timeout = due - last_clock = timeout`. The staleness cancels out — `epoll_wait` always receives the full `timeout` value. On a busy server, frequent I/O keeps `last_clock` fresh and deadlines fire on time. On a quiet server, the actual wait approximates the full `timeout` regardless of staleness. For example: - -- `last_clock` was set 8ms ago, you call `st_read()` with a 10ms timeout → `due = last_clock + 10ms` (only 2ms from now), but dispatch computes `min_timeout = due - last_clock = 10ms` → `epoll_wait` blocks for the full 10ms -- On a busy server with frequent I/O, `last_clock` stays fresh — dispatch would compute `min_timeout = 2ms` and the deadline fires on time - -ST timeouts are suitable for coarse-grained purposes — detecting broken connections, idle peers, or stuck operations. Realistic timeouts should be on the order of seconds (e.g., 5s, 30s), where `last_clock` staleness is negligible. They are not designed for precise sub-millisecond timing. - -**Related test cases (`st_utest_learn_kb.cpp`):** -- `TEST(LearnKB, BasicNetfdReadTimeout)` -- `TEST(LearnKB, CondTimedwaitTimeout)` - -## `st_init()` — How the Coroutine World is Built - -`st_init()` is the entry point that bootstraps the entire coroutine runtime. It creates the scheduler data structures, the event system, the idle thread, and wraps the calling OS thread as the first coroutine. Here's what happens step by step: - -**Step 1: Set event system and initialize I/O.** SRS calls `st_set_eventsys(ST_EVENTSYS_ALT)` before `st_init()` to select the platform-optimal backend — epoll on Linux, kqueue on macOS. Inside `st_init()`, `st_set_eventsys(ST_EVENTSYS_DEFAULT)` is called but returns `EBUSY` (since the event system is already set) and is harmlessly ignored — hence the code comment "We can ignore return value here". Then `_st_io_init()` runs for one-time I/O setup (ignores SIGPIPE, sets fd limits). - -**Step 2: Initialize all scheduler queues and create the event system.** First initializes the thread-local free stack list (`_st_free_stacks`), then zeroes the VP struct (`memset(&_st_this_vp, 0, ...)`), then initializes three empty linked lists: -- `run_q` — coroutines ready to run -- `io_q` — coroutines blocked on socket I/O -- `zombie_q` — dead coroutines awaiting cleanup - -Then calls `(*_st_eventsys->init)()` to create the actual epoll/kqueue file descriptor. Also captures `pagesize` (for stack guard pages) and `last_clock` (current timestamp for timeout calculations). - -**Step 3: Create the Idle Thread.** This is the heart of ST. The idle thread is created via `st_thread_create(_st_idle_thread_start)`, then marked with `_ST_FL_IDLE_THREAD`, decremented from `_st_active_count` (it doesn't count as an "active" coroutine), and removed from the run queue (it's managed specially by the scheduler). - -The idle thread's loop is the core scheduler cycle: -1. `(*_st_eventsys->dispatch)()` — calls `epoll_wait`/`kqueue`, blocking until I/O is ready or the earliest timeout fires. This is the **only place the process truly blocks**. -2. `_st_vp_check_clock()` — updates `last_clock`, then walks the sleep heap and moves timed-out coroutines to the run queue. -3. `_st_switch_context(me)` — yields CPU to a ready coroutine from the run queue. -4. When that coroutine eventually switches back (hits I/O or yields), the idle thread loops again. -5. When `_st_active_count` drops to 0 (no more coroutines), the idle thread calls `exit(0)`. - -**Step 4: Create the Primordial Thread.** The current OS thread (the one calling `st_init()`) is wrapped into an `_st_thread_t` struct via `calloc`. It gets no new stack — it reuses the existing process stack. Its state is set to `_ST_ST_RUNNING`, flagged as `_ST_FL_PRIMORDIAL`, and assigned to `_st_this_thread`. This becomes the first running coroutine and `_st_active_count` is incremented to 1. - -**After `st_init()` returns**, the coroutine world is ready: event system initialized, idle thread created and waiting, primordial thread running. Control returns to `main()`, which is now executing as the primordial coroutine. From here, calling `st_thread_create()` spawns new coroutines, and the scheduler workflow activates once those coroutines hit their first I/O call. - -**Related test cases (`st_utest_learn_kb.cpp`):** -- `TEST(LearnKB, EventSysSelectedAndLockedAfterInit)` -- `TEST(LearnKB, CoroutineRunsOnSeparateStack)` -- `TEST(LearnKB, StartRoutineNotExecutedInline)` -- `TEST(LearnKB, JoinDrivesFirstRunWhenNoManualYield)` - -## `st_thread_create()` — How a Coroutine is Born - -`st_thread_create(start, arg, joinable, stk_size)` allocates a new coroutine, sets up its execution context, and places it on the run queue — all without actually running it yet. - -**Step 1: Allocate the stack.** The requested size (default `ST_DEFAULT_STACK_SIZE` = 128KB) is rounded up to page alignment, then `_st_stack_new()` either reuses a stack from the free list or allocates fresh memory (via `mmap` or `malloc`). In DEBUG builds (without `MD_NO_PROTECT`), guard pages are set at both ends of the stack via `mprotect(..., PROT_NONE)` to catch overflow via SIGSEGV; in release builds there are no guard pages. - -**Step 2: Carve thread metadata from the top of the stack.** The thread control block (`_st_thread_t`) and per-thread data array (`ptds[ST_KEYS_MAX]`) are placed at the top of the stack, growing downward. Then the stack pointer is 64-byte aligned. The layout from high to low address: - -- `ptds[ST_KEYS_MAX]` — per-thread data slots (like thread-local storage) -- `_st_thread_t` — the thread control block -- 64-byte alignment padding -- 128-byte pad (`_ST_STACK_PAD_SIZE`) reserved below the aligned SP -- `stack->sp` — where the coroutine's actual execution stack begins (grows downward) -- Guard page at the bottom (DEBUG builds only) - -Both `thread` and `ptds` are zeroed with `memset` after being carved from the stack. This is efficient: one allocation provides both the stack and the control block — no separate `malloc` for the thread struct. - -**Step 3: Set up the initial context (the core trick).** This is the most subtle part: - -```c -/* Note that we must directly call rather than call any functions. */ -if (_st_md_cxt_save(thread->context)) { - _st_thread_main(); -} -MD_GET_SP(thread) = (long)(stack->sp); -``` - -The code comment is a correctness constraint: `_st_md_cxt_save` must be called directly at this site, not wrapped in a helper function. It captures the current PC (return address) — when `_st_md_cxt_restore` later restores this context, execution resumes right here at the `if` check. If `_st_md_cxt_save` were called inside a helper function, the saved PC would point into that helper's frame, and the restored execution would return into a function frame that doesn't exist on the new coroutine's stack — crash. - -- `_st_md_cxt_save()` saves the **creator's** current CPU registers into `thread->context` and returns **0** (like `setjmp`) -- Since it returns 0, the `if` body is **skipped** — `_st_thread_main()` is NOT called now -- `MD_GET_SP()` then **overwrites the saved stack pointer** in the context to point at the new coroutine's heap-allocated stack - -Later, when the scheduler switches to this coroutine via `_st_md_cxt_restore(thread->context)`, it restores these saved registers — but with the **modified SP** pointing at the new stack. The restore returns **1** (non-zero), so the `if` is entered and `_st_thread_main()` executes — now running on the coroutine's own stack. `_st_thread_main()` simply calls `thread->start(thread->arg)` (the user's function), and when it returns, calls `st_thread_exit()`. - -This save-then-patch-SP trick is how ST creates a coroutine without running it: capture a register snapshot, swap the stack pointer to the new stack, defer execution until scheduled. - -**Step 4: Set up joinability.** If `joinable` is true, a condition variable (`thread->term`) is allocated so another coroutine can call `st_thread_join()` and block until this coroutine finishes. - -**Step 5: Make it runnable.** The thread's state is set to `_ST_ST_RUNNABLE`, `_st_active_count` is incremented, and the thread is inserted into `run_q`. The coroutine won't actually execute until the current coroutine yields (hits I/O, sleeps, or calls `st_thread_yield()`), at which point the scheduler picks it off the run queue. - -**Valgrind integration:** If `MD_VALGRIND` is enabled and the thread is not the primordial thread, `VALGRIND_STACK_REGISTER()` is called to register the custom stack region with Valgrind, preventing false positives from stack pointer switching. - -**Related test cases (`st_utest_learn_kb.cpp`):** -- `TEST(LearnKB, CoroutineRunsOnSeparateStack)` -- `TEST(LearnKB, StartRoutineNotExecutedInline)` -- `TEST(LearnKB, JoinDrivesFirstRunWhenNoManualYield)` -- `TEST(LearnKB, LocalStatePreservedAcrossYield)` -- `TEST(LearnKB, ReturnValueThroughJoin)` - -## Epoll-Driven I/O Workflow — How Coroutines Sleep and Wake - -This section traces exactly what happens when a coroutine does I/O — from `st_read()` through epoll and back. This is the core mechanism that makes ST work. - -**The I/O functions all follow the same pattern** (`st_read`, `st_write`, `st_accept`, `st_connect`, `st_recvfrom`, `st_sendto`, `st_recvmsg`, `st_sendmsg`): - -1. Try the raw syscall (`read`, `write`, `accept`, etc.) immediately -2. If it succeeds → return the result. Zero overhead, no coroutine machinery involved -3. If `EINTR` → retry the syscall -4. If `EAGAIN`/`EWOULDBLOCK` → the socket isn't ready, enter the coroutine wait path via `st_netfd_poll()` -5. If any other error → return error - -This "try first" design means ST adds zero overhead when data is already available — it's just a normal syscall. - -**`st_netfd_poll()` → `st_poll()`:** The poll wrapper converts a single fd into a `struct pollfd` and calls `st_poll()`, which is where the coroutine actually suspends. - -**`st_poll()` — The Coroutine Suspension Point (sched.c):** - -1. **Register with epoll:** Calls `_st_eventsys->pollset_add(pds, npds)`, which increments per-fd reference counts (`_ST_EPOLL_READ_CNT`, `_ST_EPOLL_WRITE_CNT`, `_ST_EPOLL_EXCEP_CNT` — one per event direction), then computes the new event mask from those counts and calls `epoll_ctl` — `EPOLL_CTL_ADD` if the fd had no prior watchers, or `EPOLL_CTL_MOD` if it already did. Multiple coroutines can watch the same fd — reference counts track this. - -2. **Create a poll queue entry on the stack:** A `_st_pollq_t` struct links the pollfd array to the waiting coroutine (`pq.thread = me`) and sets `pq.on_ioq = 1`. - -3. **Insert into `io_q`:** The poll queue entry is linked into the global I/O wait list. Later, `_st_epoll_dispatch()` walks this list to find which coroutines to wake. - -4. **Add to sleep heap (if timeout specified):** `_st_add_sleep_q(me, timeout)` sets `me->due = last_clock + timeout` and inserts into the binary heap. This ensures the coroutine wakes even if I/O never arrives. - -5. **Set state to `_ST_ST_IO_WAIT`** and call `_st_switch_context(me)` — the coroutine suspends. Registers are saved, and the scheduler picks the next runnable coroutine (or the idle thread if nothing else is ready). - -**`_st_vp_schedule()` — The Scheduler (sched.c):** - -When `_st_switch_context()` is called, `_st_vp_schedule()` decides what runs next: -- If `run_q` is non-empty → pull the first thread off the head, switch to it -- If `run_q` is empty → switch to the idle thread - -The idle thread is where `epoll_wait` lives — it's the "nobody else has anything to do, so let's wait for I/O" fallback. - -**The Idle Thread Loop (sched.c):** - -The idle thread runs a tight loop until no active coroutines remain: -1. `_st_eventsys->dispatch()` → calls `_st_epoll_dispatch()` -2. `_st_vp_check_clock()` → process expired timeouts -3. Set self to `RUNNABLE`, call `_st_switch_context()` → yield to a ready coroutine -4. When that coroutine eventually suspends, we return here and loop - -**`_st_epoll_dispatch()` — The Heart of I/O Multiplexing (event.c):** - -This function does three things: wait for I/O, wake coroutines, and clean up epoll state. - -*Phase 1 — Calculate epoll timeout from sleep heap:* -- If `sleep_q` is NULL (no sleeping coroutines) → `timeout = -1` (block forever) -- Otherwise → `timeout = sleep_q->due - last_clock` (wake at earliest deadline) -- Special case: if timeout computes to 0ms but `min_timeout > 0` (sub-millisecond), round up to 1ms to avoid a spin loop (epoll_wait only has millisecond granularity) - -*Phase 2 — `epoll_wait()`:* -- `nfd = epoll_wait(epfd, evtlist, evtlist_size, timeout)` — **this is the only true blocking point in the entire process**. The OS suspends the process until at least one fd is ready or the timeout expires. - -*Phase 3 — Mark fired fds:* -- For each event returned by epoll, store the fired events in `_ST_EPOLL_REVENTS(osfd)`. If `EPOLLERR` or `EPOLLHUP` is set, also OR in the fd's currently-registered event bits (`_ST_EPOLL_EVENTS(osfd)` — whichever of `EPOLLIN`/`EPOLLOUT`/`EPOLLPRI` have non-zero ref counts) so waiting coroutines see the error. - -*Phase 4 — Walk `io_q`, wake matching coroutines:* -- For each `_st_pollq_t` entry on `io_q`, check each fd in its pollfd array against `_ST_EPOLL_REVENTS`. If any fd has matching events, set `pds->revents` and mark `notify = 1`. -- If notify: remove the poll queue entry from `io_q` (`pq->on_ioq = 0`), call `_st_epoll_pollset_del()` which decrements reference counts for all fds in the pollfd array and calls `EPOLL_CTL_MOD`/`EPOLL_CTL_DEL` only for fds that did NOT fire (fds with `_ST_EPOLL_REVENTS != 0` are skipped — they are cleaned up later in Phase 5). Then remove the thread from `sleep_q` if present, set `thread->state = _ST_ST_RUNNABLE`, and insert into `run_q`. - -*Phase 5 — Clean up fired fds in epoll:* -- For each event in the epoll result list, clear `_ST_EPOLL_REVENTS`, then either `EPOLL_CTL_MOD` (if other coroutines still watch this fd) or `EPOLL_CTL_DEL` (if no more watchers). This keeps epoll's internal state in sync with ST's reference counts. - -**`_st_vp_check_clock()` — Timeout Processing (sched.c):** - -After dispatch returns, the idle thread checks for expired timeouts: -1. Update `last_clock = st_utime()` -2. Walk the sleep heap root: while `sleep_q->due <= now`, remove the thread from the heap -3. If the thread was in `_ST_ST_COND_WAIT` state, set the `_ST_FL_TIMEDOUT` flag -4. Set `thread->state = _ST_ST_RUNNABLE` and insert at the **head** of `run_q` (using `st_clist_insert_after`) - -**Priority detail:** Timed-out coroutines go to the head of `run_q`; I/O-ready coroutines go to the tail. Timeouts get priority because they represent deadlines. - -**Resumption — Back in `st_poll()`:** - -When the scheduler switches back to our coroutine, execution resumes right after the `_st_switch_context()` call in `st_poll()`: -- If `pq.on_ioq == 0` → dispatch already removed us from `io_q`, I/O is ready. Count fds with non-zero `revents` and return the count. -- If `pq.on_ioq == 1` → we timed out (woken by `_st_vp_check_clock`, not by dispatch). Remove ourselves from `io_q`, call `pollset_del` to clean up epoll registration, return 0. - -Back in `st_read()`, if `st_netfd_poll()` succeeded, the loop retries `read()` — which now succeeds because data is available. The data is returned to the application. - -**The `on_ioq` flag is the key mechanism** that distinguishes timeout wakeup from I/O wakeup. The dispatch function clears it (`pq->on_ioq = 0`) when I/O fires; `_st_vp_check_clock` does NOT touch it. So when `st_poll()` resumes, it checks this flag to know why it woke up. - -**Reference counting for shared fds:** Multiple coroutines can wait on the same fd (e.g., multiple readers on a UDP socket). `_ST_EPOLL_READ_CNT(fd)`, `_ST_EPOLL_WRITE_CNT(fd)`, and `_ST_EPOLL_EXCEP_CNT(fd)` track how many coroutines watch each direction. `epoll_ctl` is only called when the computed event mask (`_ST_EPOLL_EVENTS(fd)`) changes — i.e., when counts transition between 0 and non-zero. When a coroutine's I/O completes, `_st_epoll_pollset_del` decrements the counts — if all reach 0, the fd is removed from epoll; otherwise it's modified to reflect remaining watchers. - -**Related test cases (`st_utest_learn_kb.cpp`):** -- `TEST(LearnKB, BasicNetfdWriteThenRead)` -- `TEST(LearnKB, BasicNetfdReadTimeout)` - -## `st_usleep()` — Pure Timer-Based Coroutine Sleep - -`st_usleep(usecs)` suspends the current coroutine for a specified duration. Unlike I/O functions (`st_read`, `st_write`), it involves **no I/O at all** — the coroutine is placed only in the sleep heap and woken purely by timeout expiration. - -**The function (sync.c):** - -1. **Check interrupt flag.** If `_ST_FL_INTERRUPT` is set on the current thread, clear it and return `EINTR` immediately — the coroutine was interrupted by `st_thread_interrupt()` before it even started sleeping. - -2. **Set state and enter sleep heap.** If a finite timeout is given: set `me->state = _ST_ST_SLEEPING`, then `_st_add_sleep_q(me, usecs)` which computes `me->due = last_clock + usecs`, sets the `_ST_FL_ON_SLEEPQ` flag, assigns a heap index, and inserts into the binary heap (O(log N)). If `ST_UTIME_NO_TIMEOUT` is passed (via `st_sleep(-1)`), the state is set to `_ST_ST_SUSPENDED` instead — no sleep queue entry, the coroutine hangs indefinitely until explicitly interrupted. - -3. **Suspend.** `_st_switch_context(me)` saves the coroutine's CPU registers via `_st_md_cxt_save(me->context)` (returns 0 on save), then calls `_st_vp_schedule(me)` which picks the next runnable coroutine from `run_q` — or the idle thread if nothing else is ready — and switches to it via `_st_restore_context`. - -4. **The coroutine is now frozen.** Its registers are saved in `me->context`, and it sits in the sleep heap with a computed deadline. It is **not** in `io_q` and has no epoll registration — this is the key difference from I/O wait. - -5. **The idle thread wakes it.** The idle thread's `epoll_wait` uses the sleep heap root's deadline as its timeout. When `epoll_wait` returns (either from I/O on other fds or timeout expiration), `_st_vp_check_clock()` runs: it reads `now = st_utime()`, updates `last_clock`, then walks the sleep heap — any thread with `due <= now` is removed from the heap, set to `_ST_ST_RUNNABLE`, and inserted at the **head** of `run_q` (timed-out coroutines get priority over I/O-ready ones). - -6. **Resume.** When the scheduler switches back, `_st_md_cxt_restore(me->context, 1)` restores registers and returns 1 (non-zero), so the `if` in `_st_switch_context` is skipped and execution continues after the `_st_switch_context()` call in `st_usleep()`. A final interrupt check is performed, then `return 0` — sleep complete. - -**Comparison with I/O wait path:** - -- `st_usleep` uses **only the sleep heap** — no `io_q`, no `epoll_ctl`, no epoll registration. The coroutine is woken exclusively by `_st_vp_check_clock()`. -- `st_read`/`st_write` (EAGAIN path) uses **both `io_q` and the sleep heap** — epoll watches the fd, and the sleep heap provides timeout fallback. The coroutine can be woken by either `_st_epoll_dispatch()` (I/O ready) or `_st_vp_check_clock()` (timeout), distinguished by the `on_ioq` flag. -- In both cases, `epoll_wait`'s timeout is derived from the sleep heap root, so pure-sleep coroutines still influence when `epoll_wait` returns. - -**Timeout precision note:** The deadline is `last_clock + usecs`, not `now + usecs`. If CPU work happened since the last scheduler cycle (the last time `_st_vp_check_clock` updated `last_clock`), part of the sleep duration is already "consumed." For typical sleep durations (seconds), this staleness is negligible. This is why ST timeouts are designed for coarse-grained use — detecting broken connections or idle peers, not sub-millisecond timing. - -**Related test cases (`st_utest_learn_kb.cpp`):** -- `TEST(LearnKB, ThreadInterruptWakeupFromUsleep)` -- `TEST(LearnKB, LocalStatePreservedAcrossYield)` -- `TEST(LearnKB, StartRoutineNotExecutedInline)` - -## `st_mutex` — Cooperative Mutex Workflow - -ST's mutex is simple because cooperative scheduling eliminates the need for atomic operations, spinlocks, or memory barriers — just pointer manipulation and coroutine switching. - -**The struct (common.h):** -- `_st_thread_t *owner` — current mutex owner, NULL means unlocked -- `_st_clist_t wait_q` — linked list of coroutines waiting to acquire the mutex - -**`st_mutex_lock()` (sync.c):** - -1. **Check interrupt flag.** If `_ST_FL_INTERRUPT` is set on the current thread, clear it and return `EINTR` immediately — the coroutine was interrupted before it even attempted to acquire the lock. (Same pattern as `st_usleep` and `st_cond_timedwait`.) - -2. **Uncontended (owner == NULL).** Set `lock->owner = me`, return 0. Instant — just a pointer comparison and assignment, the cheapest possible lock. Safe because cooperative scheduling guarantees no other coroutine runs between the check and the assignment. - -3. **Same owner (owner == me).** Return `EDEADLK`. Deadlock detection — if you try to lock a mutex you already own, ST catches it immediately instead of hanging forever. - -4. **Contended (owner == someone else).** The coroutine must wait: - - Set `me->state = _ST_ST_LOCK_WAIT` - - Insert `me` into `lock->wait_q` (FIFO — insert before tail via `st_clist_insert_before`) - - Call `_st_switch_context(me)` — save registers, yield to scheduler - - The coroutine is now frozen: sitting on `lock->wait_q`, **not** on `sleep_q` (no timeout), **not** on `io_q` (no I/O). It can only be woken by `st_mutex_unlock()` or `st_thread_interrupt()` - - When resumed: remove self from `wait_q`, check if interrupted (return `EINTR` if interrupted and not the owner), otherwise return 0 — the coroutine now owns the mutex - -**`st_mutex_unlock()` (sync.c):** - -First checks that the caller actually owns the mutex (returns `EPERM` if not). Then walks `lock->wait_q` looking for a thread in `_ST_ST_LOCK_WAIT` state: - -- **Waiter found:** Direct ownership transfer — sets `lock->owner = waiter` immediately, sets `waiter->state = _ST_ST_RUNNABLE`, inserts waiter into `run_q` (at tail, normal priority). The unlocker does **not** yield — it keeps running. The waiter resumes when the current coroutine eventually hits I/O or yields. -- **No waiters:** Simply sets `lock->owner = NULL`. - -**Key design properties:** - -- **No timeout.** Unlike `st_usleep` or `st_read`, `st_mutex_lock` has no timeout parameter. A waiting coroutine is not placed on the sleep heap — it waits indefinitely until unlocked or interrupted. For timed locking semantics, use `st_cond_timedwait` instead. -- **Direct ownership transfer.** When unlocking with waiters, `lock->owner = waiter` is set before the waiter even resumes. This prevents a third coroutine from grabbing the mutex between the unlock and the waiter's resumption. Safe because cooperative scheduling means no preemption between these operations. -- **FIFO fairness.** Waiters are added to the tail of `wait_q`; `st_mutex_unlock` walks from the head. First to wait is first to acquire. No starvation. -- **No spin, no atomic ops, no syscalls.** The uncontended path is a pointer comparison and assignment. The contended path suspends the coroutine entirely — no busy-waiting. All of this works because no other coroutine can run between a check and an assignment in cooperative scheduling. - -**Related test cases (`st_utest_learn_kb.cpp`):** -- `TEST(LearnKB, MutexCooperativeWorkflow)` -- `TEST(LearnKB, ThreadInterruptWakeupFromMutexWait)` - -## `st_cond` — Condition Variable Workflow - -ST's condition variable (`st_cond`) follows a similar pattern to `st_mutex` — `wait_q` linked list, `_st_switch_context` to suspend, wake by setting state to `RUNNABLE` — but solves a fundamentally different problem: **waiting for a condition/event to happen**, not exclusive resource ownership. - -**The struct (common.h):** Just a `wait_q` linked list — no `owner` field. Nobody "owns" a condition variable. - -**Comparison with `st_mutex`:** -- **Purpose:** `st_mutex` = exclusive ownership of a resource. `st_cond` = wait for something to happen. -- **Owner:** `st_mutex` has `lock->owner`. `st_cond` has no ownership concept. -- **Timeout:** `st_mutex_lock` has no timeout — waits forever. `st_cond_timedwait` supports timeout via the sleep heap. -- **Wake semantics:** `st_mutex_unlock` transfers ownership to ONE waiter. `st_cond_signal` wakes ONE waiter, `st_cond_broadcast` wakes ALL waiters — no ownership transfer. -- **Wait state:** `st_mutex` uses `_ST_ST_LOCK_WAIT`. `st_cond` uses `_ST_ST_COND_WAIT`. - -**`st_cond_timedwait()` (sync.c):** - -1. **Check interrupt flag.** Return `EINTR` if `_ST_FL_INTERRUPT` is set. -2. **Enter wait queue.** Set `me->state = _ST_ST_COND_WAIT`, insert into `cvar->wait_q` (FIFO). -3. **Optionally enter sleep heap.** If `timeout != ST_UTIME_NO_TIMEOUT`, call `_st_add_sleep_q(me, timeout)`. This is the key difference from `st_mutex` — the coroutine lands in **two places simultaneously**: `cvar->wait_q` AND the sleep heap. It gets woken by whichever fires first. -4. **Suspend.** `_st_switch_context(me)` saves registers and yields to the scheduler. -5. **Resume.** Remove self from `wait_q`. Check `_ST_FL_TIMEDOUT` → return `ETIME`. Check `_ST_FL_INTERRUPT` → return `EINTR`. Otherwise return 0 (signaled successfully). - -**Dual-wakeup mechanism:** When both `wait_q` and sleep heap are active, the coroutine wakes from whichever fires first: -- **Signal fires first** → `_st_cond_signal` removes the coroutine from the sleep heap (`_st_del_sleep_q`), sets it RUNNABLE. On resume, no `_ST_FL_TIMEDOUT` flag → returns 0. -- **Timeout fires first** → `_st_vp_check_clock` finds the coroutine in `_ST_ST_COND_WAIT` state, sets the `_ST_FL_TIMEDOUT` flag, moves it to `run_q` (at head — timeout priority). On resume, flag is set → returns `ETIME`. The coroutine is still on `wait_q` but removes itself upon resumption. - -This is the same dual-wakeup pattern as the I/O path (`io_q` + sleep heap), but with `wait_q` instead of `io_q`. - -**`_st_cond_signal()` (sync.c):** - -Walks `cvar->wait_q` from head, for each thread in `_ST_ST_COND_WAIT` state: if the thread is on the sleep heap, removes it (`_st_del_sleep_q`); sets `thread->state = _ST_ST_RUNNABLE`; inserts into `run_q`. If not broadcast, breaks after the first waiter. If broadcast, continues to wake all waiters. - -**`st_cond_wait()` is simply `st_cond_timedwait(cvar, ST_UTIME_NO_TIMEOUT)`** — no sleep heap entry, waits indefinitely until signaled or interrupted. - -**Why SRS uses `st_cond` far more than `st_mutex`:** In a cooperative coroutine system, there is no preemption — no other coroutine runs between a check and an assignment, so mutual exclusion is rarely needed. What SRS needs constantly is "wait until something happens" — data arrives on an SRT socket, a stream becomes available, a client connects. `st_cond` is the primary tool for this. The coroutine-native SRT pattern is a perfect example: a coroutine calls `st_cond_wait` when `srt_recvmsg` returns EAGAIN, and a poller coroutine calls `st_cond_signal` when the fd becomes ready. - -**Related test cases (`st_utest_learn_kb.cpp`):** -- `TEST(LearnKB, CondSignalWakeOne)` -- `TEST(LearnKB, CondBroadcastWakeAll)` -- `TEST(LearnKB, CondTimedwaitTimeout)` -- `TEST(LearnKB, ThreadInterruptWakeupFromCondWait)` - -## `st_thread_exit()` — How a Coroutine Dies - -`st_thread_exit(retval)` is called when a coroutine finishes — either explicitly by the user or implicitly via `_st_thread_main()` after the start function returns. It handles cleanup, joinability, and stack recycling. The coroutine never returns from this function. - -**Step 1: Store return value and run destructors.** Sets `thread->retval = retval`, then calls `_st_thread_cleanup(thread)` which iterates all created thread-specific data keys (up to `key_max`, not `ST_KEYS_MAX`). For each key that has both a non-NULL value and a registered destructor function, it calls the destructor and clears the slot. This is ST's equivalent of pthread key destructors. - -**Step 2: Decrement active count.** `_st_active_count--` — one fewer active coroutine. When this reaches 0, the idle thread will call `exit(0)` and the process terminates. - -**Step 3: Handle joinable threads (the zombie path).** If `thread->term` is non-NULL (thread was created with `joinable = 1`): - -1. Set `thread->state = _ST_ST_ZOMBIE` — the thread is dead but its resources are preserved for the joiner to inspect. -2. Insert into `zombie_q` — this keeps the thread struct alive so `st_thread_join()` can read `retval`. -3. `st_cond_signal(thread->term)` — wake any coroutine blocked in `st_thread_join()` waiting on this thread's termination condition variable. -4. `_st_switch_context(thread)` — the zombie suspends. It will only be rescheduled by `st_thread_join()` after the joiner has read the return value. -5. When rescheduled (by the joiner): destroy the termination condvar (`st_cond_destroy(thread->term)`), set `thread->term = NULL`. - -**Step 4: Free the stack.** If the thread is not the primordial thread, `_st_stack_free(thread->stack)` puts the stack on the free list. (Valgrind deregistration happens just before this if `MD_VALGRIND` is enabled.) The primordial thread's stack is the process stack — it's never freed. - -**Step 5: Final switch — no return.** `_st_switch_context(thread)` is called one last time. The scheduler picks the next runnable coroutine. Since the exiting thread is not on any queue (not in `run_q`, not in `zombie_q` anymore for non-joinable threads), it will never be scheduled again. Its stack has been freed (or put on the free list), and execution never returns here. - -**Why zombies need two context switches:** The first `_st_switch_context` (step 3) suspends the zombie so the joiner can run and read `retval`. The joiner then puts the zombie back on `run_q` (see `st_thread_join` below). The second `_st_switch_context` (step 5) is the final one — after the joiner has extracted what it needs, the zombie resumes briefly to destroy its condvar and free its stack, then switches away forever. - -**Non-joinable threads skip the zombie path entirely** — no condvar signal, no zombie queue, no waiting for a joiner. They go straight from cleanup → stack free → final switch. - -**Related test cases (`st_utest_learn_kb.cpp`):** -- `TEST(LearnKB, ThreadExitExplicitRetvalThroughJoin)` -- `TEST(LearnKB, ThreadExitNonJoinableCannotJoin)` -- `TEST(LearnKB, ReturnValueThroughJoin)` - -## `st_thread_join()` — Waiting for a Coroutine to Finish - -`st_thread_join(thread, retvalp)` blocks the calling coroutine until the target thread exits. It's the mechanism for "wait for this coroutine to complete and get its result." - -**Precondition checks:** - -1. `thread->term == NULL` → thread is not joinable (created with `joinable = 0`). Returns `EINVAL`. -2. `_st_this_thread == thread` → trying to join yourself. Returns `EDEADLK`. -3. `term->wait_q` is non-empty → another coroutine is already waiting to join this thread. Returns `EINVAL`. Only one joiner is allowed per joinable thread. - -**The wait loop:** - -```c -while (thread->state != _ST_ST_ZOMBIE) { - if (st_cond_timedwait(term, ST_UTIME_NO_TIMEOUT) != 0) - return -1; -} -``` - -The joiner calls `st_cond_timedwait` on the target's termination condvar with no timeout — it suspends indefinitely. When `st_thread_exit()` signals this condvar, the joiner wakes up and checks if the target is in `_ST_ST_ZOMBIE` state. If yes, the loop exits. If not (for example, a spurious wakeup), it waits again. If interrupted via `st_thread_interrupt`, `st_cond_timedwait` returns an error and `st_thread_join()` returns `-1` immediately. - -**After the target is zombie:** - -1. Read the return value: `*retvalp = thread->retval` (if `retvalp` is non-NULL). -2. **Reschedule the zombie for final cleanup:** Remove the zombie from `zombie_q`, set its state to `_ST_ST_RUNNABLE`, insert into `run_q`. This lets the zombie resume in `st_thread_exit()` to destroy its condvar and free its stack (the second `_st_switch_context` in `st_thread_exit`). - -**Why the joiner reschedules the zombie instead of cleaning up directly:** The zombie's stack contains the `_st_thread_t` struct itself (thread metadata is carved from the top of the stack — see `st_thread_create`). If the joiner freed the stack, it would destroy the thread struct it's still reading from. Instead, the joiner puts the zombie back on `run_q`, and the zombie cleans up its own resources when it gets scheduled — running on its own stack, which is safe to free at the very end (the final `_st_switch_context` switches away before the stack memory is actually reused). - -**The complete lifecycle of a joinable thread:** - -1. `st_thread_create(start, arg, joinable=1, ...)` — born, placed on `run_q` -2. Scheduled → runs `_st_thread_main()` → runs `start(arg)` -3. `start` returns → `st_thread_exit(retval)` called -4. Cleanup runs, state → `_ST_ST_ZOMBIE`, inserted into `zombie_q` -5. `st_cond_signal(term)` wakes the joiner -6. First `_st_switch_context` — zombie suspends -7. Joiner wakes, reads `retval`, moves zombie from `zombie_q` to `run_q` -8. Zombie resumes, destroys condvar, frees stack -9. Final `_st_switch_context` — zombie is gone forever - -**Related test cases (`st_utest_learn_kb.cpp`):** -- `TEST(LearnKB, ReturnValueThroughJoin)` -- `TEST(LearnKB, JoinDrivesFirstRunWhenNoManualYield)` -- `TEST(LearnKB, ThreadExitExplicitRetvalThroughJoin)` -- `TEST(LearnKB, ThreadExitNonJoinableCannotJoin)` -- `TEST(LearnKB, StartRoutineNotExecutedInline)` - -## `st_thread_interrupt()` — Waking a Coroutine From Any Wait State - -`st_thread_interrupt(thread)` is the "cancel" mechanism — it forces a coroutine to wake up regardless of what it's waiting on (I/O, sleep, condvar, mutex). The interrupted coroutine sees `EINTR` when it resumes. - -**The function (sched.c):** - -1. **Dead thread check.** If `thread->state == _ST_ST_ZOMBIE`, return immediately — can't interrupt a dead thread. - -2. **Set the interrupt flag.** `thread->flags |= _ST_FL_INTERRUPT`. This flag persists until the interrupted coroutine checks and clears it upon resumption. - -3. **Already running/runnable check.** If the thread is `_ST_ST_RUNNING` or `_ST_ST_RUNNABLE`, just return — the flag is set, and the thread will see it next time it enters an I/O or wait function. No queue manipulation needed. - -4. **Remove from sleep heap (if present).** If `_ST_FL_ON_SLEEPQ` is set, call `_st_del_sleep_q(thread)` to remove it from the timeout heap. This is necessary because the thread is being woken prematurely — its timeout is no longer relevant. - -5. **Make runnable.** Set `thread->state = _ST_ST_RUNNABLE`, insert into `run_q` (at tail via `st_clist_insert_before`). - -**What the function does NOT do:** It does not remove the thread from `io_q` or `cvar->wait_q`. This is handled by the interrupted coroutine itself when it resumes — the same pattern as timeout wakeups in `st_poll()` (checks `pq.on_ioq`), `st_cond_timedwait()` (removes self from `wait_q`), and `st_mutex_lock()` (removes self from `wait_q`). - -**How each wait function detects interruption:** - -- **`st_poll()` (I/O wait):** After resuming from `_st_switch_context`, checks `me->flags & _ST_FL_INTERRUPT`. If set, clears the flag, sets `errno = EINTR`, returns -1. The `pq.on_ioq` flag is still 1 (dispatch didn't wake us), so the poll entry is also cleaned up from `io_q` and epoll. - -- **`st_usleep()` (sleep):** Checks the interrupt flag both before sleeping and after resuming. If set, clears it, returns `EINTR`. - -- **`st_cond_timedwait()` (condvar wait):** After resuming, removes self from `wait_q`, then checks `_ST_FL_INTERRUPT`. If set, clears it, returns `EINTR`. - -- **`st_mutex_lock()` (mutex wait):** After resuming, removes self from `wait_q`. If the interrupt flag is set AND the thread is not the mutex owner (another thread could have unlocked the mutex at the same time as the interrupt), returns `EINTR`. - -**The interrupt flag is "sticky" until consumed:** Once set, it stays set until a blocking path checks and clears it. If the thread is already running, the flag has no immediate effect. At the next wait point with explicit interrupt checks (`st_poll`, `st_usleep`, `st_cond_timedwait`, `st_mutex_lock`), the call returns `EINTR` instead of remaining blocked. For I/O wrappers like `st_read`/`st_write`, interruption is observed when they enter `st_poll` (typically after `EAGAIN`); if the syscall succeeds immediately, they may return data instead of `EINTR`. This design prevents races where interrupt arrives between deciding to wait and actually suspending. - -**Interrupt vs. the sleep heap:** When a thread is on both `io_q`/`wait_q` and the sleep heap (e.g., `st_cond_timedwait` with timeout), `st_thread_interrupt` only removes it from the sleep heap. The thread remains on `io_q`/`wait_q` until it resumes and cleans up. This is safe because the cleanup is always done by the thread itself after waking. - -**Use in SRS:** `st_thread_interrupt` is how SRS implements graceful shutdown. When SRS needs to stop (e.g., SIGINT), it interrupts all active coroutines. Each coroutine's I/O call returns `EINTR`, the coroutine sees the shutdown flag, and exits cleanly. Without this mechanism, coroutines blocked on I/O would never wake up to check if the server is shutting down. - -**Critical lifecycle rule (do not assume immediate termination):** `st_thread_interrupt()` does **not** terminate the target coroutine synchronously. It only marks/wakes the coroutine so its current blocking call can return (typically `-1` with `errno=EINTR`, for example `st_read`). The coroutine must then cooperatively unwind and return from its entry function. Therefore, the interrupter should use a join/synchronization step (for joinable threads, `st_thread_join`) to wait for actual thread exit, instead of assuming interrupt == already dead. - -**Related test cases (`st_utest_learn_kb.cpp`):** -- `TEST(LearnKB, ThreadInterruptWakeupFromUsleep)` -- `TEST(LearnKB, ThreadInterruptWakeupFromCondWait)` -- `TEST(LearnKB, ThreadInterruptWakeupFromMutexWait)` - -## The Netfd Abstraction (`_st_netfd_t`) — ST's File Descriptor Wrapper - -Most socket/file descriptors used with ST's convenience I/O APIs are wrapped in a `_st_netfd_t`. This struct is the bridge between raw OS file descriptors and ST's coroutine I/O system. Most ST I/O helpers (`st_read`, `st_write`, `st_accept`, `st_connect`, etc.) take `_st_netfd_t*` instead of raw `int` fds, while the lower-level `st_poll()` API still accepts raw `struct pollfd` descriptors. - -**The struct (common.h):** - -```c -typedef struct _st_netfd { - int osfd; /* Underlying OS file descriptor */ - int inuse; /* In-use flag */ - void *private_data; /* Per-descriptor private data */ - _st_destructor_t destructor; /* Private data destructor function */ - void *aux_data; /* Auxiliary data for internal use */ - struct _st_netfd *next; /* For putting on the free list */ -} _st_netfd_t; -``` - -- `osfd` — The actual OS file descriptor number (what you'd pass to `read(2)`, `write(2)`) -- `inuse` — Whether this wrapper is active (1) or recycled on the free list (0) -- `private_data` + `destructor` — Per-fd user data with cleanup callback, set via `st_netfd_setspecific()`. SRS uses this to attach application-level connection objects to the fd -- `aux_data` — Reserved for internal use (currently a no-op; historically used for accept serialization in multi-process setups) -- `next` — Singly-linked list pointer for the free list - -**The Free List — Object Recycling:** - -`_st_netfd_t` objects are recycled via a thread-local singly-linked free list (`_st_netfd_freelist`). When a netfd is freed (`st_netfd_free`), it's pushed onto the free list. When a new netfd is needed (`_st_netfd_new`), the free list is checked first — if non-empty, a recycled object is popped; otherwise a fresh one is `calloc`'d. This avoids malloc/free churn for the most frequently created/destroyed objects in a server (one per connection). - -The free list is `static __thread` — each pthread in a multi-threaded ST setup has its own free list, requiring no locking. Note: the earlier documentation mentioned the netfd freelist having a pthread mutex as the only shared-state lock. Looking at the current code, the freelist is actually thread-local (`static __thread _st_netfd_t *_st_netfd_freelist`), so no mutex is needed. The mutex for shared netfd state existed in older versions of the toffaletti multi-threading fork but the current ossrs/state-threads code uses `__thread` isolation instead. - -**Creating a Netfd — `_st_netfd_new(osfd, nonblock, is_socket)`:** - -This is the internal constructor called by all public creation functions: - -1. **Notify the event system:** Calls `_st_eventsys->fd_new(osfd)`. For epoll, this is `_st_epoll_fd_new` which ensures the per-fd data array (`fd_data`) is large enough to hold the fd's index — expanding it via `realloc` if needed. For kqueue, this does the same with its own `fd_data` array. This is how ST tracks per-fd reference counts and revents. -2. **Get or allocate the wrapper:** Pop from `_st_netfd_freelist` if available, otherwise `calloc` a new one. -3. **Set non-blocking mode:** If `nonblock` is true (for example in `st_netfd_open_socket`, and in `st_accept` on platforms where accepted sockets do not inherit non-blocking mode), sets `O_NONBLOCK`. For sockets, tries `ioctl(FIONBIO)` first (one syscall) — if that fails, falls back to `fcntl(F_GETFL)` + `fcntl(F_SETFL, O_NONBLOCK)` (two syscalls). Non-blocking mode is essential — without it, `read`/`write` would block the entire process instead of returning `EAGAIN` for the coroutine to handle. -4. **Return the wrapper** with `osfd` set and `inuse = 1`. - -**Public creation functions:** - -- `st_netfd_open(osfd)` — Wrap any fd, set non-blocking. Used for pipes, FIFOs, etc. Calls `_st_netfd_new(osfd, 1, 0)` — `is_socket=0` means it skips the `ioctl(FIONBIO)` shortcut. -- `st_netfd_open_socket(osfd)` — Wrap a socket fd, set non-blocking. Calls `_st_netfd_new(osfd, 1, 1)` — `is_socket=1` enables the faster `ioctl` path. -- `st_open(path, oflags, mode)` — Open a file/FIFO with `O_NONBLOCK` added to `oflags`, then wrap it. The fd is already non-blocking from the `open()` call, so `_st_netfd_new` is called with `nonblock=0` (no need to set it again). - -**Closing a Netfd — `st_netfd_close(fd)`:** - -1. **Notify the event system:** Calls `_st_eventsys->fd_close(fd->osfd)`. For epoll, `_st_epoll_fd_close` checks that no coroutines are still watching this fd (all reference counts must be zero) — if any are non-zero, it returns `EBUSY` and the close fails. This prevents closing an fd that other coroutines are waiting on. -2. **Recycle the wrapper:** Calls `st_netfd_free(fd)` which clears `inuse`, calls the private data destructor if set, and pushes the wrapper onto the free list. -3. **Close the OS fd:** `close(fd->osfd)`. - -**Freeing without closing — `st_netfd_free(fd)`:** - -Sometimes you want to release the ST wrapper without closing the underlying OS fd (e.g., if another library owns the fd lifecycle). `st_netfd_free` does this: clears `aux_data`, calls the private data destructor, sets `inuse = 0`, and pushes to the free list. The OS fd remains open. - -**The Per-Fd Data Array in the Event System:** - -The event system maintains an array indexed by OS fd number. Initial sizing is backend-specific (epoll starts from `fd_hint`, derived from fd limits and `ST_EPOLL_EVTLIST_SIZE`; kqueue starts from `FD_SETSIZE`), and then grows dynamically via `realloc` as larger fd values appear. For epoll, this is `_st_epoll_data->fd_data` — an array of `_epoll_fd_data_t`: - -```c -typedef struct _epoll_fd_data { - int rd_ref_cnt; /* Number of coroutines waiting for read */ - int wr_ref_cnt; /* Number of coroutines waiting for write */ - int ex_ref_cnt; /* Number of coroutines waiting for exception */ - int revents; /* Fired events from last epoll_wait */ -} _epoll_fd_data_t; -``` - -This is **not** inside `_st_netfd_t` — it's a separate array in the event system, indexed by raw fd number. The reference counts track how many coroutines are watching each direction on each fd. When `st_poll()` registers a coroutine via `_st_epoll_pollset_add`, it increments the appropriate counts and calls `epoll_ctl(EPOLL_CTL_ADD)` or `epoll_ctl(EPOLL_CTL_MOD)`. During wakeup/timeout cleanup, `_st_epoll_pollset_del` decrements counts and updates epoll only for descriptors that did **not** fire in the current dispatch pass; fired descriptors are cleaned up in the dispatch function's second pass. - -This separation (netfd wrapper vs. event system per-fd data) is a clean design: the netfd is the application-facing handle, while the per-fd data is the event system's internal bookkeeping. Multiple netfd wrappers could theoretically point to the same osfd (though this would be unusual), and the event system tracks watchers by raw fd number regardless. - -**I/O Initialization — `_st_io_init()`:** - -Called once during `st_init()`, this function does two things: - -1. **Ignore SIGPIPE.** Sets `SIGPIPE` handler to `SIG_IGN` via `sigaction`. Without this, writing to a closed socket would kill the process. With it, `write()` just returns `EPIPE` which ST handles normally. -2. **Maximize fd limit.** Reads `RLIMIT_NOFILE` via `getrlimit`, raises `rlim_cur` to `rlim_max` via `setrlimit`, and stores the result in `_st_osfd_limit`. On macOS where `rlim_max` can be negative (a platform quirk), it falls back to the event system's limit or `rlim_cur`. This fd limit influences backend initial sizing, but per-fd arrays are still expanded dynamically as needed. - -**Per-Fd Private Data — `st_netfd_setspecific` / `st_netfd_getspecific`:** - -These let application code attach arbitrary data to a netfd, similar to pthread key-specific data but per-fd instead of per-thread. `st_netfd_setspecific(fd, value, destructor)` stores a pointer and a cleanup function; when the netfd is freed, the destructor is called automatically. SRS uses this to associate connection handler objects with their socket fds. - -**Why most ST network I/O helpers take `_st_netfd_t*` (with `st_poll` as the raw-fd exception):** - -The netfd wrapper ensures three invariants: (1) the fd is always non-blocking, (2) the event system knows about the fd and can track watchers, and (3) the fd can carry application-specific data. Without the wrapper, application code could accidentally use a blocking fd with ST's helper I/O functions, bypassing the coroutine scheduler and stalling the entire process. For APIs that require `_st_netfd_t*`, this is a compile-time guard against passing raw `int` fds; when direct raw-fd polling is needed, ST exposes `st_poll()` for that purpose. - -**Related test cases (`st_utest_learn_kb.cpp`):** -- `TEST(LearnKB, NetfdSpecificAndDestructorOnClose)` -- `TEST(LearnKB, NetfdFreeKeepsOsfdOpen)` -- `TEST(LearnKB, BasicNetfdWriteThenRead)` -- `TEST(LearnKB, BasicNetfdReadTimeout)` - -## The Event System Abstraction (`_st_eventsys_t`) — How ST Swaps I/O Backends - -ST uses a vtable pattern — a struct of function pointers — so the scheduler can call I/O multiplexing operations without knowing which backend (epoll, kqueue, or select) is active. - -**The vtable struct (common.h):** - -```c -typedef struct _st_eventsys_ops { - const char *name; /* "select", "kqueue", "epoll" */ - int val; /* ST_EVENTSYS_SELECT or ST_EVENTSYS_ALT */ - int (*init)(void); /* Create the OS multiplexer */ - void (*dispatch)(void); /* The blocking wait + wake coroutines loop */ - int (*pollset_add)(struct pollfd *, int); /* Register fds when a coroutine starts waiting */ - void (*pollset_del)(struct pollfd *, int); /* Unregister fds when I/O completes or times out */ - int (*fd_new)(int); /* Notify backend about a new fd (expand arrays) */ - int (*fd_close)(int); /* Check fd can be closed (no active watchers) */ - int (*fd_getlimit)(void); /* Hard fd limit (FD_SETSIZE for select, 0=unlimited) */ - void (*destroy)(void); /* Tear down the event system */ -} _st_eventsys_t; -``` - -The global pointer `__thread _st_eventsys_t *_st_eventsys` is thread-local — each pthread in a multi-threaded ST setup gets its own event system instance. Each backend defines a static instance of this struct with its functions filled in (e.g., `_st_epoll_eventsys`, `_st_kq_eventsys`, `_st_select_eventsys`), and `st_set_eventsys()` points the global at the chosen one. - -**The pollfd bridge:** All three backends speak `struct pollfd` at the interface level — `pollset_add` and `pollset_del` take `struct pollfd*`. This is the abstraction layer. The scheduler only knows about `POLLIN`/`POLLOUT`/`POLLPRI`. Each backend translates these to its native API internally (epoll events, kqueue filters, or fd_sets). - -**How the scheduler uses the vtable:** The scheduler never calls `epoll_wait` or `kevent` or `select` directly. Every call goes through `_st_eventsys->`: - -- `st_init()` → `_st_eventsys->init()` — creates epoll fd / kqueue fd / initializes fd_sets -- Idle thread loop → `_st_eventsys->dispatch()` — the big blocking wait -- `st_poll()` → `_st_eventsys->pollset_add(pds, npds)` — when a coroutine suspends on I/O -- I/O completion / timeout → `_st_eventsys->pollset_del(pds, npds)` — cleanup registrations -- `_st_netfd_new()` → `_st_eventsys->fd_new(osfd)` — expand per-fd arrays if needed -- `st_netfd_close()` → `_st_eventsys->fd_close(osfd)` — check no active watchers before close - -**Three backends, compile-time selected:** - -- **select** — fallback backend (used on Cygwin64 and also compiled on Darwin/Linux). Compile flag `MD_HAVE_SELECT`. Val `ST_EVENTSYS_SELECT` (1). Hard limit of `FD_SETSIZE` fds. State stored in `__thread` `fd_set`s with per-fd reference count arrays. -- **kqueue** — macOS/Darwin. Compile flag `MD_HAVE_KQUEUE`. Val `ST_EVENTSYS_ALT` (3). No fd limit. State in `__thread` struct with per-fd data array, plus add/delete kevent lists. -- **epoll** — Linux. Compile flag `MD_HAVE_EPOLL`. Val `ST_EVENTSYS_ALT` (3). No fd limit. State in `__thread` struct with per-fd data array (read/write/exception reference counts + revents). - -Kqueue and epoll both use `ST_EVENTSYS_ALT` — they're interchangeable "advanced" backends. Select is the fallback. The Makefile defines which flags are set per platform: Darwin gets `MD_HAVE_KQUEUE` + `MD_HAVE_SELECT`, Linux gets `MD_HAVE_EPOLL` + `MD_HAVE_SELECT`, Cygwin64 gets only `MD_HAVE_SELECT`. If none of the three are defined, compilation fails with `#error`. - -**Backend selection (`st_set_eventsys`):** - -`st_init()` calls `st_set_eventsys(ST_EVENTSYS_DEFAULT)`, which maps to select (it resolves to `ST_EVENTSYS_SELECT` internally). To get epoll or kqueue, application code must call `st_set_eventsys(ST_EVENTSYS_ALT)` **before** `st_init()`. For `ST_EVENTSYS_ALT`: if `MD_HAVE_KQUEUE` is defined, kqueue is used; else if `MD_HAVE_EPOLL` is defined and `_st_epoll_is_supported()` confirms the kernel actually supports it (by probing `epoll_ctl` for `ENOSYS`), epoll is used. Once set, the pointer is locked — calling `st_set_eventsys` again returns `EBUSY`. SRS calls `st_set_eventsys(ST_EVENTSYS_ALT)` to get epoll on Linux and kqueue on macOS. - -**The uniform dispatch pattern:** Despite different OS APIs, all three `dispatch` functions follow the same structure: - -1. Calculate timeout from sleep heap root (`sleep_q->due - last_clock`) -2. Call the OS blocking function (`select` / `kevent` / `epoll_wait`) -3. Mark fired fds in per-fd state -4. Walk `io_q` — for each waiting coroutine, check its fds against fired results -5. If any fd matched → remove from `io_q` (`on_ioq = 0`), call `pollset_del` to clean up registrations, remove from sleep heap if present, set `_ST_ST_RUNNABLE`, insert into `run_q` -6. Clean up OS-level state for fired fds - -The scheduler, idle thread, coroutine suspension/resumption — all identical regardless of backend. Only the vtable functions differ. - -**Backend-specific details:** - -*Select:* -- Maintains three `__thread` `fd_set`s (read/write/exception) with per-fd reference counts. `dispatch` must copy all three fd_sets before calling `select()` because select modifies them in place. -- Has a `_st_select_find_bad_fd()` recovery handler — when `select` returns `EBADF`, it walks all waiting fds with `fcntl(F_GETFL)` to identify and remove the bad one. -- The `maxfd` tracker is maintained across add/del/dispatch — select requires the highest fd number + 1 as its first argument. -- The select backend data is `__thread` (`_st_select_data`), so each pthread keeps independent select bookkeeping. - -*Kqueue:* -- Uses `EV_ONESHOT` flag — each registration fires once and auto-deregisters from kqueue. Elegant: no need to explicitly delete fired fds. Only unfired fds need explicit `EV_DELETE`. -- Batches additions via an `addlist` — `pollset_add` queues `struct kevent` entries, and `dispatch` submits them all in a single `kevent()` call via the changelist parameter. This reduces syscalls. -- Deletions are synchronous — `pollset_del` calls `kevent()` immediately with a `dellist` to avoid stale fd problems (can't defer because the fd might be closed before the next dispatch). -- Handles **fork recovery** — if `getpid()` changes after `kevent` returns `EBADF`, it re-creates the kqueue fd and re-registers all fds from `io_q`. Kqueue fds don't survive `fork()`. -- Timeout uses `struct timespec` (nanosecond precision). -- `destroy` exists in the vtable, but current kqueue backend cleanup is a TODO (`_st_kq_destroy` is not implemented yet). - -*Epoll:* -- Level-triggered (no `EPOLLET` flag) — simplest model, events re-fire on every `epoll_wait` until the fd is consumed or removed. -- Uses reference counting (`rd_ref_cnt`, `wr_ref_cnt`, `ex_ref_cnt`) per fd to support multiple coroutines watching the same fd. `epoll_ctl` is called to `ADD`/`MOD`/`DEL` as reference counts transition between 0 and non-zero. -- `dispatch` has two cleanup passes for fired fds: the first pass (inside the `io_q` walk) calls `pollset_del` which handles unfired fds — but skips fired fds because their `_ST_EPOLL_REVENTS` is still set. The second pass (after the `io_q` walk) iterates the epoll result list, clears revents, and calls `EPOLL_CTL_MOD` or `EPOLL_CTL_DEL` based on remaining reference counts. This two-pass design avoids modifying epoll state while still iterating results that depend on it. -- Timeout uses milliseconds (epoll_wait limitation). Rounds up sub-millisecond timeouts to 1ms to avoid spin loops — if `min_timeout > 0` but computes to 0ms, it's bumped to 1ms. -- `_st_epoll_is_supported()` probes at selection time by calling `epoll_ctl(-1, EPOLL_CTL_ADD, -1, &ev)` — if errno is `ENOSYS`, epoll syscalls are stubs and the backend is rejected. - -**Related test cases (`st_utest_learn_kb.cpp`):** -- `TEST(LearnKB, EventSysSelectedAndLockedAfterInit)` -- `TEST(LearnKB, BasicNetfdReadTimeout)` -- `TEST(LearnKB, BasicNetfdWriteThenRead)` diff --git a/skills/internal-codemap-for-srs/SKILL.md b/skills/internal-codemap-for-srs/SKILL.md new file mode 100644 index 00000000000..b7f38a4e82d --- /dev/null +++ b/skills/internal-codemap-for-srs/SKILL.md @@ -0,0 +1,44 @@ +--- +name: internal-codemap-for-srs +description: Route SRS code tasks to the smallest relevant trusted codebase map and verification guidance. Use whenever support, development, debugging, review, or maintenance work requires locating, choosing, reading, modifying, testing, or verifying SRS code. Covers the first-generation C++ origin and edge media server, its State Threads dependency, the next-generation Go server, and the test, E2E, and benchmark structure. Use as the code-navigation dependency of srs-support and srs-develop; the parent skill remains responsible for the user-facing workflow and result. +--- + +# SRS Internal Code Map + +Route code work to focused codebase maps. The parent skill owns the user-facing task; this skill owns code navigation and verification-file selection. + +## Core Rules + +- Work from the current working directory. Do not search parent directories or discover alternate repository roots. +- Use the Reference Router before reading, searching, or modifying code, configuration, tests, or verification scripts. +- Treat only files and module directories listed by the selected reference as trusted navigation scope. +- Never grep the repository root or broad trees such as `trunk/src/`, `cmd/`, or `internal/`. +- When a selected reference lists a module directory rather than every file, list filenames only inside that module, choose the smallest relevant set, then read or search only those files. +- If no route covers the task, report that the code router does not cover it. Do not discover a new route ad hoc. +- Use `skills/internal-docs-for-srs/SKILL.md` for project documentation. Do not route documentation here. + +## Reference Router + +| Code area | Use when | Load | +|---|---|---| +| C++ media server | The task concerns the first-generation origin or edge server, `trunk/src/`, `trunk/conf/`, protocols, media processing, or State Threads | `references/cpp-server.md` | +| Next-generation Go server | The task concerns the Go proxy, future Go origin or edge services, `cmd/`, or `internal/` | `references/go-server.md` | +| Testing and verification | The task requires choosing or running unit, black-box, E2E, proxy, reproduction, or benchmark verification | `references/testing.md` | + +For a comparison or migration across generations, load both server maps. Add the testing reference only when verification is required. + +## Workflow + +1. Classify the request as C++ media server, next-generation Go server, testing and verification, or an explicit combination. +2. If the server generation is unclear and choosing incorrectly could change the result, ask the user to clarify. Do not guess. +3. Load the selected reference file or files. +4. Use their descriptions to identify the responsible module and the smallest relevant file set. +5. Read or search only those files. Add a second routed module only when evidence shows a dependency crosses the first module boundary. +6. Return control to the parent skill for support, development, debugging, review, or maintenance. + +## Maintaining the Router + +- Add, remove, or rename code modules and trusted files in exactly one server reference. +- Update the Reference Router when a reference is added, removed, renamed, or changes responsibility. +- Keep module descriptions concise and focused on ownership, boundaries, and navigation. +- Keep verification commands and test-suite responsibilities in `references/testing.md`; do not duplicate them in dependent skills. diff --git a/skills/internal-codemap-for-srs/references/cpp-server.md b/skills/internal-codemap-for-srs/references/cpp-server.md new file mode 100644 index 00000000000..3dc0d1ed5f9 --- /dev/null +++ b/skills/internal-codemap-for-srs/references/cpp-server.md @@ -0,0 +1,202 @@ +# C++ Server Code Map + +Use this reference after `skills/internal-codemap-for-srs/SKILL.md` routes a task to the first-generation C++ media server. The listed modules, directories, configuration files, and State Threads files define the trusted navigation scope. + +## C++ Media Server Code + +The C++ media server (`trunk/src/`) serves as both origin server and edge server. It uses the ST (State Threads) coroutine library and is the full-featured server with all SRS capabilities. + +`main/` — Entry point: +- `main_server` — The `main()` function, starts the server + +`core/` — Foundational definitions: +- `core` — Core includes, macros, config generated by configure +- `core_version` through `core_version8` — Version definitions per major release +- `core_autofree` — SrsUniquePtr smart pointer (RAII) +- `core_deprecated` — Deprecated SrsAutoFree, kept for compat +- `core_performance` — Performance tuning constants (merge-read, etc.) +- `core_platform` — Platform abstraction (32/64-bit, PRId64, large file support) +- `core_time` — Time types and conversions (srs_utime_t, ms/us) + +`kernel/` — Low-level building blocks, no network, no protocol logic: + +Media codecs/containers: +- `kernel_codec` — Codec definitions, H.264/H.265/AAC parsing, sequence headers +- `kernel_aac` — AAC transmuxer (raw AAC stream output) +- `kernel_mp3` — MP3 transmuxer +- `kernel_flv` — FLV tag reader/writer, muxer/demuxer +- `kernel_mp4` — MP4/fMP4 box reader/writer +- `kernel_ts` — MPEG-TS muxer/demuxer +- `kernel_ps` — PS (Program Stream) muxer/demuxer (GB28181) +- `kernel_packet` — Media packet: raw undecoded frame with timing info + +Buffers & I/O: +- `kernel_buffer` — SrsBuffer: binary read/write cursor over a byte array +- `kernel_stream` — SrsSimpleStream: growable byte buffer +- `kernel_io` — Abstract I/O interfaces (ISrsReader, ISrsSeeker, ISrsWriter) +- `kernel_file` — File reader/writer implementations + +RTC primitives: +- `kernel_rtc_rtp` — RTP packet structure, encode/decode +- `kernel_rtc_rtcp` — RTCP packet types, encode/decode +- `kernel_rtc_queue` — RTP ring buffer and reorder queue + +Utilities: +- `kernel_error` — Error code definitions and SrsCplxError +- `kernel_log` — Log levels and ISrsLog interface +- `kernel_consts` — Global constants (chunk sizes, timeouts, limits) +- `kernel_utility` — System time, string helpers, bit buffer +- `kernel_kbps` — Bandwidth statistics (ISrsProtocolStatistic, ISrsClock) +- `kernel_balance` — Load balancing algorithms (round-robin) +- `kernel_hourglass` — Timer/hourglass coroutine (periodic callbacks) +- `kernel_pithy_print` — Sampled logging (print every N seconds, not every call) +- `kernel_factory` — Abstract factory interfaces (ISrsKernelFactory) +- `kernel_resource` — Resource management interfaces (ISrsResource, disposing) +- `kernel_st` — ST coroutine wrappers (ISrsCoroutine, ISrsCoroutineHandler) + +`protocol/` — Protocol implementations (wire formats, parsing, serialization): + +RTMP: +- `protocol_rtmp_stack` — RTMP chunk stream, message parsing, AMF commands +- `protocol_rtmp_conn` — RTMP client/server handshake and command exchange +- `protocol_rtmp_handshake` — RTMP handshake (simple and complex) +- `protocol_rtmp_msg_array` — Shared message array with auto-free +- `protocol_amf0` — AMF0 encoding/decoding + +HTTP: +- `protocol_http_stack` — HTTP message, header, URI, serve mux, handler +- `protocol_http_conn` — HTTP connection parsing (request/response) +- `protocol_http_client` — HTTP client for outbound requests +- `protocol_http_stack_llhttp` — llhttp parser (from Node.js) +- `protocol_http_stack_llhttpadapter` — llhttp adapter for SRS +- `protocol_http_stack_llhttpapi` — llhttp API integration +- `protocol_http_stack_llhttphttp` — llhttp HTTP integration + +RTC/WebRTC: +- `protocol_rtc_stun` — STUN packet parsing/building +- `protocol_sdp` — SDP parsing/building +- `protocol_rtp` — RTP packetizer (H.264/Opus → RTP packets) + +Other protocols: +- `protocol_srt` — SRT socket wrapper +- `protocol_rtsp_stack` — RTSP message parsing + +Media: +- `protocol_raw_avc` — Raw H.264/H.265 stream (Annex-B ↔ AVCC conversion) +- `protocol_format` — Media format: extract metadata, codec info from RTMP stream + +Infrastructure: +- `protocol_io` — Protocol-level I/O interfaces (ISrsProtocolReader/Writer) +- `protocol_st` — ST initialization, socket helpers, TCP/UDP wrappers +- `protocol_stream` — SrsFastStream: merge-read buffered stream +- `protocol_conn` — Base connection interface (ISrsConnection) +- `protocol_log` — Thread context for logging (SrsThreadContext) +- `protocol_json` — JSON parsing (SrsJsonAny) +- `protocol_utility` — URL parsing, stream URL helpers +- `protocol_protobuf` — Protobuf encoding helpers (for Prometheus/exporter) + +`app/` — Application logic (server features, connections, business logic): + +Server core: +- `app_server` — SrsServer: main server, manages listeners and connections +- `app_config` — Configuration parsing and access +- `app_listener` — TCP/UDP listener management +- `app_reload` — Config hot-reload handler interface +- `app_st` — Coroutine implementations (SrsFastCoroutine, SrsExecutorCoroutine) +- `app_log` — File-based logger implementation + +RTMP: +- `app_rtmp_conn` — RTMP client connection lifecycle (publish, play) +- `app_rtmp_source` — RTMP live source: manages consumers, GOP cache, hub + +HTTP: +- `app_http_api` — HTTP API endpoints (/api/v1/...) +- `app_http_conn` — HTTP connection handler (routes to static/stream/api) +- `app_http_hooks` — HTTP callback hooks (on_connect, on_publish, etc.) +- `app_http_static` — HTTP static file serving +- `app_http_stream` — HTTP live streaming (FLV, TS, AAC, MP3 over HTTP) +- `app_http_client` — HTTP client utilities + +WebRTC: +- `app_rtc_server` — RTC server: manages RTC connections +- `app_rtc_conn` — RTC connection: publish/play session, DTLS/SRTP +- `app_rtc_source` — RTC source: manages RTC consumers and tracks +- `app_rtc_api` — WHIP/WHEP API endpoints +- `app_rtc_codec` — Audio transcoding (AAC ↔ Opus via FFmpeg) +- `app_rtc_dtls` — DTLS certificate and handshake +- `app_rtc_network` — RTC network I/O, UDP mux + +SRT: +- `app_srt_server` — SRT server: manages SRT connections +- `app_srt_conn` — SRT connection lifecycle (publish, play) +- `app_srt_source` — SRT source: manages SRT consumers +- `app_srt_listener` — SRT listener (bind/accept) + +RTSP: +- `app_rtsp_conn` — RTSP connection handler +- `app_rtsp_source` — RTSP source + +GB28181: +- `app_gb28181` — GB28181 SIP signaling and PS stream ingestion + +HLS/DASH: +- `app_hls` — HLS muxer (TS segments + M3U8 playlist) +- `app_dash` — DASH muxer (fMP4 segments + MPD manifest) +- `app_fragment` — Fragment/segment interface for HLS/DASH + +DVR: +- `app_dvr` — DVR recording (FLV/MP4 to disk) + +Edge/Forward/Bridge: +- `app_edge` — Edge mode: pull/push from origin +- `app_forward` — Forward: push stream to another RTMP server +- `app_stream_bridge` — Bridge between sources (RTMP↔RTC↔SRT) + +Ingest/Transcode: +- `app_ingest` — Ingest: pull external streams via FFmpeg +- `app_encoder` — Transcode: FFmpeg transcoding management +- `app_ffmpeg` — FFmpeg process wrapper + +Utilities: +- `app_security` — Security: allow/deny rules for publish/play +- `app_refer` — Referer check +- `app_statistic` — Server/stream/client statistics +- `app_heartbeat` — HTTP heartbeat to API server +- `app_utility` — System info, disk/CPU/memory stats +- `app_process` — Child process management +- `app_async_call` — Async HTTP callback execution +- `app_recv_thread` — Background RTMP message receive thread +- `app_ng_exec` — exec hooks (on_publish exec) +- `app_factory` — Application-level factory (file I/O, paths) +- `app_coworkers` — Origin cluster coordination +- `app_latest_version` — Version check against upstream +- `app_caster_flv` — HTTP-FLV push ingest (stream caster) +- `app_mpegts_udp` — MPEG-TS over UDP ingest +- `app_stream_token` — Stream publish token management +- `app_circuit_breaker` — Circuit breaker for overload protection +- `app_hds` — HDS (Adobe HTTP Dynamic Streaming) + +**File naming convention:** All C++ source files use `srs_{module}_{topic}.cpp/.hpp` — e.g. `srs_app_rtmp_conn.cpp`, `srs_protocol_amf0.hpp`, `srs_kernel_flv.cpp`. + +**Config** + +Config (`trunk/conf/`) is only for the C++ media server. The next-generation Go server does not use these config files. + +- `srs.conf` — Default config +- `full.conf` — Full reference config with all options documented +- Feature-specific configs: `rtc.conf`, `srt.conf`, `hls.conf`, `dvr.*.conf`, `edge.conf`, `forward.*.conf`, `gb28181.conf`, `dash.conf`, etc. + +## State Threads Code + +State Threads (`trunk/3rdparty/st-srs/`) is the coroutine library used by the C++ media server only. The Go server does not use ST. + +- `sched.c` — Scheduler (thread creation, context switch, run loop) +- `io.c` — I/O: poll/epoll/kqueue wrappers, socket operations +- `event.c` — Event system +- `stk.c` — Stack allocation and management +- `sync.c` — Mutex, condition variable +- `key.c` — Thread-local storage +- `common.c`/`common.h` — Shared internals +- `public.h` — Public API +- `md.h` — Platform detection and context switch macros +- `md_linux.S`, `md_linux2.S`, `md_darwin.S`, `md_cygwin64.S` — Assembly context switch per platform diff --git a/skills/internal-codemap-for-srs/references/go-server.md b/skills/internal-codemap-for-srs/references/go-server.md new file mode 100644 index 00000000000..b36ab817d90 --- /dev/null +++ b/skills/internal-codemap-for-srs/references/go-server.md @@ -0,0 +1,35 @@ +# Next-Generation Go Server Code Map + +Use this reference after `skills/internal-codemap-for-srs/SKILL.md` routes a task to the next-generation Go server. The listed modules and files define the trusted navigation scope. + +## Next-Generation Server Code + +The next-generation server (`cmd/` + `internal/`) is written in Go and maintained by AI. It is the future of SRS. Currently it only supports proxy, but work is ongoing to support the full feature set including origin, edge, and proxy servers. + +`cmd/proxy` — A stateless reverse proxy that sits in front of one or more SRS C++ origin servers. Accepts client connections (RTMP, HTTP-FLV/HLS, WebRTC WHIP/WHEP, SRT), resolves a backend origin via the load balancer, and transparently proxies traffic. Does not cache streams or process media — only forwards bytes. Configuration entirely via environment variables (or `.env` file), no config file. Supports two deployment modes: single-proxy (in-memory LB) and multi-proxy (Redis-based shared LB for horizontal scaling behind a network load balancer). Entry point: `cmd/proxy/main.go`. + +`internal/bootstrap` — Server startup and lifecycle orchestration. Sets up logging context, signal handlers, loads environment, installs force-quit timer, optionally starts pprof, initializes the load balancer (memory or Redis based on `PROXY_LOAD_BALANCER_TYPE`), then starts all six servers sequentially (RTMP, WebRTC, HTTP API, SRT, System API, HTTP Stream) and blocks until context is cancelled. Deferred `Close()` on each server ensures graceful shutdown. + +`internal/proxy` — Proxy server implementations. Each server accepts client connections, parses just enough of the protocol to extract the stream URL, picks a backend via the load balancer, and proxies traffic bidirectionally. Contains five proxy servers: (1) **RTMP proxy** (`rtmp.go`) — TCP listener, simple handshake, parses connect/publish/play to get stream URL, bidirectional RTMP message copying, stateless. (2) **HTTP stream proxy** (`http.go`) — serves static files, proxies HTTP-FLV/TS via reverse-proxy, proxies HLS m3u8 with `spbhid` rewriting so TS segment requests route to the same backend. (3) **WebRTC proxy** (`rtc.go`) — two-phase: WHIP/WHEP signaling (SDP rewrite to replace backend UDP port with proxy's) + UDP media transport (identifies connections by STUN ufrag, supports address migration), stateful. (4) **SRT proxy** (`srt.go`) — intercepts SRT 4-step handshake locally, parses stream ID on handshake 2, replays full handshake with backend, then proxies UDP bidirectionally, stateful per-connection. (5) **HTTP API + System API** (`api.go`) — HTTP API delegates WHIP/WHEP to WebRTC server; System API provides `/api/v1/srs/register` where backend SRS C++ servers register themselves so the load balancer knows about them. + +`internal/rtmp` — RTMP protocol implementation (parsing, not proxying). Full RTMP chunk stream and message protocol: simple handshake (C0/C1/C2), chunk stream reader/writer with all four format types, extended timestamp, message reassembly from chunks. Defines all RTMP message types, chunk stream IDs, and command names. Packet types include ConnectApp, CreateStream, Publish, Play, Call, SetChunkSize, WindowAcknowledgementSize, SetPeerBandwidth, UserControl. Uses Go generics (`ExpectPacket[T]`) to read until a specific packet type arrives. Also includes full AMF0 encoder/decoder supporting Number, Boolean, String, Object, Null, Undefined, EcmaArray, StrictArray, Date, LongString — with ordered key-value maps, auto-type-discovery, and safe type converters. + +`internal/lb` — Load balancer abstraction and two implementations. Defines `OriginLoadBalancer` interface and core types in `lb.go` (Initialize, Update, Pick, HLS/WebRTC state management) and `OriginServer` struct representing a backend origin (IP, listen endpoints for RTMP/HTTP/API/SRT/RTC, heartbeat tracking). **Memory LB** (`mem.go`) — in-memory using `sync.Map`, sticky random pick per stream URL, single-proxy deployment. **Redis LB** (`redis.go`) — Redis-backed shared state with TTL-based expiration, enables multi-proxy horizontal scaling behind a network load balancer. Also includes a debug helper (`debug.go`) that creates a fake backend from env vars when `PROXY_DEFAULT_BACKEND_ENABLED=on` for development without real SRS registration. + +`internal/redisclient` — Thin Redis client abstraction. Defines a minimal `RedisClient` interface (`Ping`/`Get`/`Set`/`String`) satisfied by `*redis.Client` from `github.com/go-redis/redis/v8`, plus a `New(addr, password, db)` constructor. Used by `internal/lb/redis.go`. + +`internal/logger` — Structured logging with context IDs. Four log levels: Debug/Info (stdout), Warn/Error (stderr). Emits JSON via `log/slog` with `pid` and `cid` attributes. Each connection/request gets a unique 7-char hex context ID for log correlation, stored in `context.Context`. + +`internal/env` — Environment-based configuration. All settings via env vars (or `.env` file parsed by an in-tree custom parser — no third-party dep; supports comments, `export` prefix, quoted values, escape sequences, and inline comments). Exposes a `ProxyEnvironment` interface (with a counterfeiter-generated fake in `envfakes/` for downstream tests) with methods for each config value. Default ports: RTMP=11935, HTTP API=11985, HTTP Stream=18080, WebRTC=18000, SRT=20080, System API=12025. Timeouts: grace=20s, force=30s. Supports Redis config and default backend config for debugging. + +`internal/version` — Version constants. Signature `SRSX`, version tracks the SRS project version (currently 8.0.x). Used in HTTP API responses and startup logging. + +`internal/errors` — Error handling with stack traces, thin wrapper over stdlib `errors`. Provides `New`, `Errorf`, `Wrap`, `Wrapf`, `WithMessage`, `WithStack`, `Cause`, and re-exports `Is`/`As`/`Unwrap`/`Join`. Every error captures a stack trace at creation; `%+v` prints the full trace. `Cause()` walks the error chain to find the root error. + +`internal/sync` — Generic sync primitives. `Map[K, V]`: type-safe generic interface over `sync.Map`, constructed via `NewMap[K, V]()`. Used throughout the codebase to avoid raw type assertions. + +`internal/signal` — OS signal handling. Listens for SIGINT/SIGTERM, cancels the root context. Installs a force-quit timer (default 30s) as a safety net if graceful shutdown hangs. + +`internal/debug` — Profiling. Starts a `net/http/pprof` server if `GO_PPROF` env var is set. Optional, for runtime profiling and debugging. + +`internal/utils` — Shared utility functions. HTTP helpers (JSON response, error response, CORS, body parsing). URL helpers (normalize to `vhost/app/stream` format, extract app/stream from HTTP requests). Protocol helpers (classify UDP payload as STUN/RTP/SRT, parse ICE ufrag/pwd from SDP, parse SRT stream ID). Network helpers (detect peer-closed/connection-closed errors, parse listen endpoints). diff --git a/skills/internal-codemap-for-srs/references/testing.md b/skills/internal-codemap-for-srs/references/testing.md new file mode 100644 index 00000000000..caa806e825d --- /dev/null +++ b/skills/internal-codemap-for-srs/references/testing.md @@ -0,0 +1,83 @@ +# SRS Testing and Verification Map + +Use this reference after `skills/internal-codemap-for-srs/SKILL.md` determines that a task requires tests, verification, reproduction, or benchmarking. + +## C++ Media Server Verification + +`trunk/src/utest/` — Unit tests for internal functions, classes, parsers, codecs, and configuration without starting SRS. Links directly against SRS source and uses mocks such as `MockSrsConfig`. + +- Build and run: `cd trunk && ./configure --utest && make utest && ./objs/srs_utest` +- `srs_utest_ai01`–`ai24` — AI-written tests +- `srs_utest_manual_*` — Manually written tests +- `srs_utest_workflow_*` — Workflow and integration tests + +`trunk/3rdparty/srs-bench/blackbox/` — Black-box tests against a self-managed SRS process. Each test starts SRS with `NewSRSServer()`, uses FFmpeg or FFprobe to publish and play, verifies output, and manages the server lifecycle. + +- Build in `trunk/3rdparty/srs-bench/`; binary: `./objs/srs_blackbox_test` +- `rtmp_test.go` — RTMP +- `hls_test.go` — HLS +- `srt_test.go` — SRT +- `rtsp_test.go` — RTSP +- `hevc_test.go` — HEVC +- `dvr_test.go` — DVR +- `http_api_test.go` — HTTP API +- `mp3_test.go` — MP3 + +`trunk/3rdparty/srs-bench/srs/` — E2E protocol tests against an externally started SRS server. Uses real Pion WebRTC, RTMP, and HTTP API clients. + +- Build in `trunk/3rdparty/srs-bench/`; binary: `./objs/srs_test` +- `rtc_test.go` — WebRTC +- `rtmp_test.go` — RTMP +- `srs_test.go` — General SRS workflows + +`trunk/3rdparty/srs-bench/` — Performance and load benchmark, not a correctness test suite. The `./objs/srs_bench` binary simulates concurrent WHIP, WHEP, RTMP, reconnecting, DVR, plaintext RTC, and Janus workloads and reports metrics without pass/fail assertions. + +## Next-Generation Go Proxy Verification + +Run all commands from the repository root. Run the unit test first, then every E2E test in the listed order. E2E scripts bind fixed ports, so run them sequentially. Do not stop after an early success or failure; record every result, fix failures, and repeat until all required tests pass. + +1. Go proxy unit tests with coverage: + ```bash + bash skills/srs-develop/scripts/proxy-utest.sh --coverage + ``` +2. Single-origin RTMP proxy: + ```bash + bash skills/srs-develop/scripts/proxy-e2e-test.sh + ``` +3. Multi-origin memory load-balancer routing: + ```bash + bash skills/srs-develop/scripts/proxy-e2e-cluster-test.sh + ``` +4. Proxy, SRS edge, and SRS origin three-tier topology with a late-joining player: + ```bash + bash skills/srs-develop/scripts/proxy-e2e-edge-test.sh + ``` +5. Redis multi-proxy routing: + ```bash + bash skills/srs-develop/scripts/proxy-e2e-redis-test.sh + ``` +6. RTMP publish with RTMP, HTTP-FLV, and HLS playback verification; WHEP remains a placeholder: + ```bash + bash skills/srs-develop/scripts/proxy-e2e-transmux-test.sh + ``` +7. SRT publish with SRT, RTMP, HTTP-FLV, and HLS playback verification; WHEP remains a placeholder: + ```bash + bash skills/srs-develop/scripts/proxy-e2e-srt-test.sh + ``` +8. WHIP publish with RTMP, HTTP-FLV, and HLS playback verification; WHEP remains a placeholder: + ```bash + bash skills/srs-develop/scripts/proxy-e2e-whip-test.sh + ``` + +The SRT test requires an FFmpeg build with libsrt. The WHIP test requires the `whip` muxer and OpenSSL. Both scripts automatically run `skills/srs-develop/scripts/setup-ffmpeg-with-whip.sh` on macOS when no suitable FFmpeg is available. + +## Verification Types + +| Type | Server running? | Tests | Network | Pass/fail | +|---|---|---|---|---| +| Unit | No | Isolated internal logic | No | Yes | +| Black-box | Yes, self-managed | Whole-server behavior | Yes | Yes | +| E2E | Yes, externally managed | Protocol workflows | Yes | Yes | +| Benchmark | Yes, externally managed | Performance and capacity | Yes | No, metrics only | + +Use the smallest relevant test during iteration, then run the complete required suite defined by the parent development workflow before declaring the task verified. diff --git a/skills/internal-docs-for-srs/SKILL.md b/skills/internal-docs-for-srs/SKILL.md new file mode 100644 index 00000000000..43879486f0f --- /dev/null +++ b/skills/internal-docs-for-srs/SKILL.md @@ -0,0 +1,98 @@ +--- +name: internal-docs-for-srs +description: Route SRS tasks to the smallest relevant set of trusted project documentation and maintain the project documentation bundled with this skill. Use whenever support or development work requires locating, choosing, reading, creating, updating, or reviewing SRS documentation, including as a documentation dependency of srs-support and srs-develop. Covers the C++ media server documentation, website pages, changelog, executable API examples, and the next-generation Go server and performance documentation stored under this skill. +--- + +# SRS Internal Documentation + +Route SRS tasks to focused documentation references. The parent skill owns the user-facing task; this skill owns documentation navigation and the bundled SRS project documentation. + +## Core Rules + +- Work from the current working directory. Do not search parent directories or discover alternate repository roots. +- Use the Reference Router before loading any reference or bundled document. +- Load only the references and bundled documents relevant to the task. Do not load everything. +- Treat only project files listed in this skill or a selected reference as trusted documentation. +- If no route covers the task, report that the documentation router does not cover it. Do not scan or broadly grep documentation directories. +- Select the smallest relevant set of project documents from their descriptions. +- Keep trusted document routing in this skill. Do not duplicate topic-to-file tables in dependent skills. + +## Reference Router + +For C++ media server tracking and releases, load: + +- `trunk/doc/CHANGELOG.md` — Full changelog of all SRS versions, with one entry and version bump for each merged pull request. + +For C++ media server user documentation, select the smallest relevant document: + +- `references/cpp-docs/doc/introduction.md` — SRS overview, supported protocols, feature list, State Threads architecture, and learning path. +- `references/cpp-docs/doc/getting-started.md` — Docker quick start, RTMP publishing, HTTP-FLV/HLS playback, WebRTC, HTTPS, SRT, and stream URL patterns. +- `references/cpp-docs/doc/getting-started-ai.md` — SRS Robot, local AI agents, the skills system, and the project knowledge-base philosophy. +- `references/cpp-docs/doc/getting-started-build.md` — Build SRS from source and cross-build for ARM or MIPS. +- `references/cpp-docs/doc/getting-started-cdk.md` — Deploy SRS on AWS with srs-cdk. +- `references/cpp-docs/doc/getting-started-oryx.md` — Deploy and use Oryx, including recording, forwarding, AI subtitles, HTTPS, Docker, Helm, and aaPanel. +- `references/cpp-docs/doc/rtmp.md` — RTMP usage, Enhanced RTMP, configuration, RTMPS, protocol comparisons, and codec history. +- `references/cpp-docs/doc/hls.md` — HLS compatibility, latency, segment configuration, HTTPS, and audio transcoding from WebRTC. +- `references/cpp-docs/doc/webrtc.md` — WHIP/WHEP, SFU architecture, RTMP-to-RTC conversion, TURN/ICE, audio transcoding, and platform usage. +- `references/cpp-docs/doc/flv.md` — HTTP-FLV delivery, configuration, latency, protocol comparisons, and browser compatibility. +- `references/cpp-docs/doc/srt.md` — SRT transport, latency, MPEG-TS encapsulation, HEVC, configuration, stream IDs, and weak-network behavior. +- `references/cpp-docs/doc/rtsp.md` — RTSP playback, TCP transport, RTMP publishing workflow, configuration, and build option. +- `references/cpp-docs/doc/http-server.md` — Embedded HTTP server, HLS/static serving, API endpoint, configuration, and reverse proxies. +- `references/cpp-docs/doc/hevc.md` — H.265/HEVC protocol compatibility, Enhanced RTMP, encoder setup, and bandwidth tradeoffs. +- `references/cpp-docs/doc/dvr.md` — FLV/MP4 recording, plans, paths, HTTP callbacks, and Oryx recording features. +- `references/cpp-docs/doc/ingest.md` — Pull external files, RTSP cameras, or HTTP streams through FFmpeg and republish them to SRS. +- `references/cpp-docs/doc/forward.md` — Forward RTMP streams to other servers, configure master/slave roles, and compare forwarding with edge mode. +- `references/cpp-docs/doc/security.md` — IP and CIDR allow/deny rules for publishing and playback. +- `references/cpp-docs/doc/snapshot.md` — Capture stream thumbnails through HTTP callbacks or the transcoder. +- `references/cpp-docs/doc/http-api.md` — HTTP API endpoints for server, stream, and client statistics, including CORS and console integration. +- `references/cpp-docs/doc/http-callback.md` — Event callbacks for connection, publishing, playback, DVR, authentication, and business logic. +- `references/cpp-docs/doc/exporter.md` — Prometheus metrics, Grafana integration, labels, tags, and cloud-native observability. +- `references/cpp-docs/doc/origin-cluster.md` — Proxy-based load balancing across origin servers and the Go proxy architecture. +- `references/cpp-docs/doc/edge.md` — Edge caching, pull-on-play, push-on-publish, and multi-level CDN topology. +- `references/cpp-docs/doc/nginx-for-hls.md` — Distribute and cache HLS/DASH through NGINX. +- `references/cpp-docs/doc/resource.md` — Port, firewall, HTTPS, and resource reference. +- `references/cpp-docs/doc/low-latency.md` — RTMP latency tuning, GOP and queue settings, merge-write optimization, and protocol comparison. +- `references/cpp-docs/doc/performance.md` — UDP tuning, perf, gprof, Valgrind, ASAN, leak detection, and benchmarking methodology. +- `references/cpp-docs/doc/ffmpeg.md` — FFmpeg transcoding, multi-bitrate output, stream filtering, and per-vhost/app/stream configuration. + +For C++ media server website pages, select the smallest relevant page: + +- `references/cpp-docs/pages/faq-oryx-en.md` — Oryx FAQ covering setup, upgrades, HTTPS, authentication, recording, re-streaming, and FFmpeg replacement. +- `references/cpp-docs/pages/faq-server-en.md` — SRS server FAQ covering CDN, VoD, common errors, protocol issues, and community support. +- `references/cpp-docs/pages/license-en.md` — SRS, State Threads, and third-party library licenses. +- `references/cpp-docs/pages/product-en.md` — Release milestones, codenames, achievements, and product history. +- `references/cpp-docs/pages/security-advisories-en.md` — Published CVEs, affected versions, patches, and security references. + +For RTMP Go API examples, load: + +- `internal/rtmp/example_test.go` — RTMP API examples for AMF0, handshake, and protocol workflows. + +For WHEP performance analysis, load: + +- `references/perf/proxy-whep.md` — Profile WHEP with pprof and srs-bench and compare CPU, allocation, heap, goroutine, and trace data. + +For next-generation Go proxy documentation, select the smallest relevant document: + +- `references/proxy/features.md` — Proxy feature status and limitations, including implemented protocols, APIs, load balancing, deployment, configuration, operations, and current limitations. +- `references/proxy/proxy-design.md` — Proxy architecture, including stateless proxy design, built-in load balancing, Redis mode, and horizontal scaling. +- `references/proxy/proxy-protocol.md` — Backend registration, debugging backend, heartbeat protocol, and environment variables. +- `references/proxy/proxy-usage.md` — Getting started with the proxy: build, start, register, publish, and verify with an SRS origin. +- `references/proxy/proxy-load-balancer.md` — Load-balancer behavior for memory and Redis load balancers, stream mapping, health tracking, and protocol state. +- `references/proxy/proxy-origin-cluster.md` — Production origin clusters: advanced usage for configuring and verifying a multi-origin cluster through the proxy. + +If a task spans multiple areas, load only the required references or bundled documents from the lists. + +## Workflow + +1. Classify the documentation need with the Reference Router. +2. Load the selected project or bundled document directly from the router. +3. Load only the documents required for the task. +4. Return control to the parent skill for answering, development, troubleshooting, review, or editing. + +When invoked directly, follow the same routing workflow and then apply the relevant support or development workflow. + +## Maintaining the Router + +- Update the Reference Router when a trusted document is added, removed, renamed, or changes responsibility. +- Keep router summaries and file descriptions concise and focused on navigation. +- Do not duplicate document content or implementation details in the router. diff --git a/trunk/3rdparty/srs-docs/doc/dvr.md b/skills/internal-docs-for-srs/references/cpp-docs/doc/dvr.md similarity index 100% rename from trunk/3rdparty/srs-docs/doc/dvr.md rename to skills/internal-docs-for-srs/references/cpp-docs/doc/dvr.md diff --git a/trunk/3rdparty/srs-docs/doc/edge.md b/skills/internal-docs-for-srs/references/cpp-docs/doc/edge.md similarity index 100% rename from trunk/3rdparty/srs-docs/doc/edge.md rename to skills/internal-docs-for-srs/references/cpp-docs/doc/edge.md diff --git a/trunk/3rdparty/srs-docs/doc/exporter.md b/skills/internal-docs-for-srs/references/cpp-docs/doc/exporter.md similarity index 100% rename from trunk/3rdparty/srs-docs/doc/exporter.md rename to skills/internal-docs-for-srs/references/cpp-docs/doc/exporter.md diff --git a/trunk/3rdparty/srs-docs/doc/ffmpeg.md b/skills/internal-docs-for-srs/references/cpp-docs/doc/ffmpeg.md similarity index 100% rename from trunk/3rdparty/srs-docs/doc/ffmpeg.md rename to skills/internal-docs-for-srs/references/cpp-docs/doc/ffmpeg.md diff --git a/trunk/3rdparty/srs-docs/doc/flv.md b/skills/internal-docs-for-srs/references/cpp-docs/doc/flv.md similarity index 100% rename from trunk/3rdparty/srs-docs/doc/flv.md rename to skills/internal-docs-for-srs/references/cpp-docs/doc/flv.md diff --git a/trunk/3rdparty/srs-docs/doc/forward.md b/skills/internal-docs-for-srs/references/cpp-docs/doc/forward.md similarity index 100% rename from trunk/3rdparty/srs-docs/doc/forward.md rename to skills/internal-docs-for-srs/references/cpp-docs/doc/forward.md diff --git a/trunk/3rdparty/srs-docs/doc/getting-started-ai.md b/skills/internal-docs-for-srs/references/cpp-docs/doc/getting-started-ai.md similarity index 100% rename from trunk/3rdparty/srs-docs/doc/getting-started-ai.md rename to skills/internal-docs-for-srs/references/cpp-docs/doc/getting-started-ai.md diff --git a/trunk/3rdparty/srs-docs/doc/getting-started-build.md b/skills/internal-docs-for-srs/references/cpp-docs/doc/getting-started-build.md similarity index 100% rename from trunk/3rdparty/srs-docs/doc/getting-started-build.md rename to skills/internal-docs-for-srs/references/cpp-docs/doc/getting-started-build.md diff --git a/trunk/3rdparty/srs-docs/doc/getting-started-cdk.md b/skills/internal-docs-for-srs/references/cpp-docs/doc/getting-started-cdk.md similarity index 100% rename from trunk/3rdparty/srs-docs/doc/getting-started-cdk.md rename to skills/internal-docs-for-srs/references/cpp-docs/doc/getting-started-cdk.md diff --git a/trunk/3rdparty/srs-docs/doc/getting-started-oryx.md b/skills/internal-docs-for-srs/references/cpp-docs/doc/getting-started-oryx.md similarity index 100% rename from trunk/3rdparty/srs-docs/doc/getting-started-oryx.md rename to skills/internal-docs-for-srs/references/cpp-docs/doc/getting-started-oryx.md diff --git a/trunk/3rdparty/srs-docs/doc/getting-started.md b/skills/internal-docs-for-srs/references/cpp-docs/doc/getting-started.md similarity index 100% rename from trunk/3rdparty/srs-docs/doc/getting-started.md rename to skills/internal-docs-for-srs/references/cpp-docs/doc/getting-started.md diff --git a/trunk/3rdparty/srs-docs/doc/hevc.md b/skills/internal-docs-for-srs/references/cpp-docs/doc/hevc.md similarity index 100% rename from trunk/3rdparty/srs-docs/doc/hevc.md rename to skills/internal-docs-for-srs/references/cpp-docs/doc/hevc.md diff --git a/trunk/3rdparty/srs-docs/doc/hls.md b/skills/internal-docs-for-srs/references/cpp-docs/doc/hls.md similarity index 100% rename from trunk/3rdparty/srs-docs/doc/hls.md rename to skills/internal-docs-for-srs/references/cpp-docs/doc/hls.md diff --git a/trunk/3rdparty/srs-docs/doc/http-api.md b/skills/internal-docs-for-srs/references/cpp-docs/doc/http-api.md similarity index 100% rename from trunk/3rdparty/srs-docs/doc/http-api.md rename to skills/internal-docs-for-srs/references/cpp-docs/doc/http-api.md diff --git a/trunk/3rdparty/srs-docs/doc/http-callback.md b/skills/internal-docs-for-srs/references/cpp-docs/doc/http-callback.md similarity index 100% rename from trunk/3rdparty/srs-docs/doc/http-callback.md rename to skills/internal-docs-for-srs/references/cpp-docs/doc/http-callback.md diff --git a/trunk/3rdparty/srs-docs/doc/http-server.md b/skills/internal-docs-for-srs/references/cpp-docs/doc/http-server.md similarity index 100% rename from trunk/3rdparty/srs-docs/doc/http-server.md rename to skills/internal-docs-for-srs/references/cpp-docs/doc/http-server.md diff --git a/trunk/3rdparty/srs-docs/doc/ingest.md b/skills/internal-docs-for-srs/references/cpp-docs/doc/ingest.md similarity index 100% rename from trunk/3rdparty/srs-docs/doc/ingest.md rename to skills/internal-docs-for-srs/references/cpp-docs/doc/ingest.md diff --git a/trunk/3rdparty/srs-docs/doc/introduction.md b/skills/internal-docs-for-srs/references/cpp-docs/doc/introduction.md similarity index 100% rename from trunk/3rdparty/srs-docs/doc/introduction.md rename to skills/internal-docs-for-srs/references/cpp-docs/doc/introduction.md diff --git a/trunk/3rdparty/srs-docs/doc/low-latency.md b/skills/internal-docs-for-srs/references/cpp-docs/doc/low-latency.md similarity index 100% rename from trunk/3rdparty/srs-docs/doc/low-latency.md rename to skills/internal-docs-for-srs/references/cpp-docs/doc/low-latency.md diff --git a/trunk/3rdparty/srs-docs/doc/nginx-for-hls.md b/skills/internal-docs-for-srs/references/cpp-docs/doc/nginx-for-hls.md similarity index 100% rename from trunk/3rdparty/srs-docs/doc/nginx-for-hls.md rename to skills/internal-docs-for-srs/references/cpp-docs/doc/nginx-for-hls.md diff --git a/trunk/3rdparty/srs-docs/doc/origin-cluster.md b/skills/internal-docs-for-srs/references/cpp-docs/doc/origin-cluster.md similarity index 100% rename from trunk/3rdparty/srs-docs/doc/origin-cluster.md rename to skills/internal-docs-for-srs/references/cpp-docs/doc/origin-cluster.md diff --git a/trunk/3rdparty/srs-docs/doc/performance.md b/skills/internal-docs-for-srs/references/cpp-docs/doc/performance.md similarity index 100% rename from trunk/3rdparty/srs-docs/doc/performance.md rename to skills/internal-docs-for-srs/references/cpp-docs/doc/performance.md diff --git a/trunk/3rdparty/srs-docs/doc/resource.md b/skills/internal-docs-for-srs/references/cpp-docs/doc/resource.md similarity index 100% rename from trunk/3rdparty/srs-docs/doc/resource.md rename to skills/internal-docs-for-srs/references/cpp-docs/doc/resource.md diff --git a/trunk/3rdparty/srs-docs/doc/rtmp.md b/skills/internal-docs-for-srs/references/cpp-docs/doc/rtmp.md similarity index 100% rename from trunk/3rdparty/srs-docs/doc/rtmp.md rename to skills/internal-docs-for-srs/references/cpp-docs/doc/rtmp.md diff --git a/trunk/3rdparty/srs-docs/doc/rtsp.md b/skills/internal-docs-for-srs/references/cpp-docs/doc/rtsp.md similarity index 100% rename from trunk/3rdparty/srs-docs/doc/rtsp.md rename to skills/internal-docs-for-srs/references/cpp-docs/doc/rtsp.md diff --git a/trunk/3rdparty/srs-docs/doc/security.md b/skills/internal-docs-for-srs/references/cpp-docs/doc/security.md similarity index 100% rename from trunk/3rdparty/srs-docs/doc/security.md rename to skills/internal-docs-for-srs/references/cpp-docs/doc/security.md diff --git a/trunk/3rdparty/srs-docs/doc/snapshot.md b/skills/internal-docs-for-srs/references/cpp-docs/doc/snapshot.md similarity index 100% rename from trunk/3rdparty/srs-docs/doc/snapshot.md rename to skills/internal-docs-for-srs/references/cpp-docs/doc/snapshot.md diff --git a/trunk/3rdparty/srs-docs/doc/srt.md b/skills/internal-docs-for-srs/references/cpp-docs/doc/srt.md similarity index 100% rename from trunk/3rdparty/srs-docs/doc/srt.md rename to skills/internal-docs-for-srs/references/cpp-docs/doc/srt.md diff --git a/trunk/3rdparty/srs-docs/doc/webrtc.md b/skills/internal-docs-for-srs/references/cpp-docs/doc/webrtc.md similarity index 100% rename from trunk/3rdparty/srs-docs/doc/webrtc.md rename to skills/internal-docs-for-srs/references/cpp-docs/doc/webrtc.md diff --git a/trunk/3rdparty/srs-docs/pages/faq-oryx-en.md b/skills/internal-docs-for-srs/references/cpp-docs/pages/faq-oryx-en.md similarity index 100% rename from trunk/3rdparty/srs-docs/pages/faq-oryx-en.md rename to skills/internal-docs-for-srs/references/cpp-docs/pages/faq-oryx-en.md diff --git a/trunk/3rdparty/srs-docs/pages/faq-server-en.md b/skills/internal-docs-for-srs/references/cpp-docs/pages/faq-server-en.md similarity index 100% rename from trunk/3rdparty/srs-docs/pages/faq-server-en.md rename to skills/internal-docs-for-srs/references/cpp-docs/pages/faq-server-en.md diff --git a/trunk/3rdparty/srs-docs/pages/license-en.md b/skills/internal-docs-for-srs/references/cpp-docs/pages/license-en.md similarity index 100% rename from trunk/3rdparty/srs-docs/pages/license-en.md rename to skills/internal-docs-for-srs/references/cpp-docs/pages/license-en.md diff --git a/trunk/3rdparty/srs-docs/pages/product-en.md b/skills/internal-docs-for-srs/references/cpp-docs/pages/product-en.md similarity index 95% rename from trunk/3rdparty/srs-docs/pages/product-en.md rename to skills/internal-docs-for-srs/references/cpp-docs/pages/product-en.md index aa7256d4d0f..d82b4bbae6c 100644 --- a/trunk/3rdparty/srs-docs/pages/product-en.md +++ b/skills/internal-docs-for-srs/references/cpp-docs/pages/product-en.md @@ -2,7 +2,8 @@ About milestones of SRS. -* [Release 7.0](#release70), 2025~Now, Code: Kai +* [Release 8.0](#release80), 2026~Now, Code: Free +* [Release 7.0](#release70), 2025~2026, Code: Kai * [Release 6.0](#release60), 2023~2025, Code: Hang * [Release 5.0](#release50), 2022~2023, Code: Bee * [Release 4.0](#release40), 2020~2021, Code: Leo @@ -99,6 +100,14 @@ scenarios and industries. For a detailed interpretation, please see Welcome to SRS: Mission, Vision, and Values. +## Release 8.0 + +Code name: Free. Development started in 2026, currently in progress. + +> Code Name Story: "Free" represents the new era of open source development empowered by AI. Both freedom and free — the AI agent is not just free labor, it is like a copy of myself, or even tens of copies, all deeply understanding this project, how to deliver high-quality code, and how to serve the community. AI handles all the dirty work — the boring tasks, the documentation, the coding — and often does it better than I could, with ten times the power. We built an AI robot for the community to answer questions and help users learn this project, and we used AI to almost entirely rewrite the SRS Proxy server — its structure, its workflow — so that AI agents can comprehensively manage and maintain it. With AI I have power, and I have choice: no longer waiting for other developers to contribute, I am free to manage this project, freed from the labor of maintaining it alone. This is a fantastic, amazing new era for building and sustaining open source projects and communities. + +SRS 8.0 development started at 2026. + ## Release 7.0 Code name: Kai. Named by TOC member [Haibo Chen](https://github.com/duiniuluantanqin). Development started August 2024, planned for release by the end of 2026. diff --git a/trunk/3rdparty/srs-docs/pages/security-advisories-en.md b/skills/internal-docs-for-srs/references/cpp-docs/pages/security-advisories-en.md similarity index 100% rename from trunk/3rdparty/srs-docs/pages/security-advisories-en.md rename to skills/internal-docs-for-srs/references/cpp-docs/pages/security-advisories-en.md diff --git a/docs/perf/proxy-whep.md b/skills/internal-docs-for-srs/references/perf/proxy-whep.md similarity index 100% rename from docs/perf/proxy-whep.md rename to skills/internal-docs-for-srs/references/perf/proxy-whep.md diff --git a/docs/proxy/features.md b/skills/internal-docs-for-srs/references/proxy/features.md similarity index 97% rename from docs/proxy/features.md rename to skills/internal-docs-for-srs/references/proxy/features.md index ad6ceafd701..03d4fa1227a 100644 --- a/docs/proxy/features.md +++ b/skills/internal-docs-for-srs/references/proxy/features.md @@ -96,6 +96,9 @@ parses the minimum needed to route, then proxies traffic to the chosen backend. deployments; lowest latency, no external dependencies. - **Redis load balancer** — shared state with TTL-based expiration for multi-proxy deployments scaling horizontally behind a network load balancer. +- **Redis key namespaces** — optional `PROXY_REDIS_KEY_PREFIX`; empty by default + for backward compatibility, or set to isolate independent proxy clusters that + share one Redis database. - **Stream-level stickiness** — the first request for a stream URL picks a backend; every later request for that stream routes to the same backend. Stream→server mappings never expire. diff --git a/docs/proxy/proxy-design.md b/skills/internal-docs-for-srs/references/proxy/proxy-design.md similarity index 100% rename from docs/proxy/proxy-design.md rename to skills/internal-docs-for-srs/references/proxy/proxy-design.md diff --git a/docs/proxy/proxy-load-balancer.md b/skills/internal-docs-for-srs/references/proxy/proxy-load-balancer.md similarity index 95% rename from docs/proxy/proxy-load-balancer.md rename to skills/internal-docs-for-srs/references/proxy/proxy-load-balancer.md index a0266af78d8..936180052e2 100644 --- a/docs/proxy/proxy-load-balancer.md +++ b/skills/internal-docs-for-srs/references/proxy/proxy-load-balancer.md @@ -112,10 +112,16 @@ PROXY_REDIS_HOST=127.0.0.1 PROXY_REDIS_PORT=6379 PROXY_REDIS_PASSWORD= PROXY_REDIS_DB=0 +PROXY_REDIS_KEY_PREFIX= ``` 3. Redis Key Design +`PROXY_REDIS_KEY_PREFIX` optionally namespaces every Redis key. It is empty by +default, preserving the key names below. When set to `xxx`, keys use the form +`xxx:srs-proxy-...`. Every proxy in the same logical cluster must use the same +prefix. + **Server Keys**: - `srs-proxy-server:{serverID}` - Server registration (300s TTL) - `srs-proxy-all-servers` - Server list index (no expiration) diff --git a/docs/proxy/proxy-origin-cluster.md b/skills/internal-docs-for-srs/references/proxy/proxy-origin-cluster.md similarity index 100% rename from docs/proxy/proxy-origin-cluster.md rename to skills/internal-docs-for-srs/references/proxy/proxy-origin-cluster.md diff --git a/docs/proxy/proxy-protocol.md b/skills/internal-docs-for-srs/references/proxy/proxy-protocol.md similarity index 100% rename from docs/proxy/proxy-protocol.md rename to skills/internal-docs-for-srs/references/proxy/proxy-protocol.md diff --git a/docs/proxy/proxy-usage.md b/skills/internal-docs-for-srs/references/proxy/proxy-usage.md similarity index 100% rename from docs/proxy/proxy-usage.md rename to skills/internal-docs-for-srs/references/proxy/proxy-usage.md diff --git a/skills/srs-develop/SKILL.md b/skills/srs-develop/SKILL.md index 83e851c67c9..49d40ff60b8 100644 --- a/skills/srs-develop/SKILL.md +++ b/skills/srs-develop/SKILL.md @@ -9,6 +9,41 @@ description: Develop, modify, debug, and maintain the next-generation SRS media **Code and documents are the only truth.** Issue descriptions may be inaccurate. Pull requests may be misleading. Feature descriptions may be insufficient. Always ground your understanding in the actual source code and project documentation. Documents capture design intent, architecture rationale, and complex background that code alone cannot express — they are another form of code. When code and documents conflict, investigate rather than assume one is wrong. +## Skill Dependencies + +- `skills/internal-docs-for-srs/SKILL.md` — Route and load project documentation. This skill remains responsible for the development workflow and final result. +- `skills/internal-codemap-for-srs/SKILL.md` — Route code navigation and verification to the relevant server map. This skill remains responsible for the development workflow and final result. + +## Git Workflow + +Apply these rules whenever a task produces a commit: + +- Never run `git add`; William stages the files he approves. +- Never run `git push`; William pushes the branch. +- Commit only when William explicitly asks. +- Before committing, run `git diff --cached`, understand the staged changes, and write an appropriate title and description. +- Prefix the commit title with the tool that made the changes: `OpenClaw:`, `Claude:`, or `Codex:`. +- If Claude made changes, use this exact commit message format: + ``` + Commit title. + + Commit description. + + --------- + + Co-authored-by: Claude Fable 5 + ``` +- If Codex made changes, use this exact commit message format: + ``` + Commit title. + + Commit description. + + --------- + + Co-authored-by: chatgpt-codex-connector[bot] <199175422+chatgpt-codex-connector[bot]@users.noreply.github.com> + ``` + --- ## Task Router @@ -100,7 +135,6 @@ Do not modify issues or create Truth Records. **Scope:** Walk the pending changes on the current branch (relative to `develop`), summarize them, sync any stale navigation docs, then bump the version and add a changelog entry once the user supplies the PR number. **Guiding rules** -- **The user drives staging.** Never `git add` on your own. After each step, stop and wait for the user to review and stage the files they approve. Only run `git commit` when they say so. - **Docs are navigation, not tutorials.** When a code change makes an entry stale, *correct* it — don't expand it. Only *add* a new entry when a new file or module was introduced; never to describe a refactor inside an existing module. **Step 1: Survey the changes** @@ -112,9 +146,9 @@ Do not modify issues or create Truth Records. **Step 2: Correct stale navigation docs** -1. Check `memory/srs-codebase-map.md` for entries covering any module touched in this PR. +1. Load `skills/internal-codemap-for-srs/SKILL.md`, route to the next-generation Go server map, and check the entries covering each module touched in this PR. 2. For each entry whose description is no longer accurate, make the **smallest** correction needed to match the new code. Keep the one-line summary style; do not expand into implementation detail. -3. Stop. Let the user review. When they `git add` the files they accept, commit with a short message in the existing style, e.g. `Claude: Sync srs-codebase-map with internal/.`. +3. Stop and let the user review and stage the files they accept. After an explicit commit request, use a short message such as `: Sync internal Go code map with internal/.`. **Step 3: Bump the version and update the changelog** @@ -127,7 +161,7 @@ Do not modify issues or create Truth Records. * v8.0, YYYY-MM-DD, Merge [#PR](URL): : . v8.0. (#PR) ``` Propose the summary to the user; don't invent one unilaterally. -4. Stop. Let the user review. When they `git add` the version files and changelog, commit with a short message like `Proxy: Bump to v8.0. for #.`. +4. Stop and let the user review and stage the version files and changelog. After an explicit commit request, use a short message such as `: Bump to v8.0. for #.`. --- @@ -157,15 +191,16 @@ The proxy server is a complex, growing product — not a small app. It has many **Step 1: Module Routing (MANDATORY)** -1. Read the codebase map: `memory/srs-codebase-map.md` — both the **Next-Generation Server Code** section (code modules: `cmd/` + `internal/`) and the **Next-Generation Server Docs** section (documentation: `docs/proxy/`). -2. Study the module descriptions and doc descriptions. Understand what each covers and its boundaries. -3. Reason about which module(s) and which doc(s) are relevant to the user's request. Consider: +1. Load `skills/internal-codemap-for-srs/SKILL.md`, then use its Reference Router to select the next-generation Go server code map. +2. Load `skills/internal-docs-for-srs/SKILL.md`, then use its Reference Router to select the relevant next-generation server documentation references. +3. Study the routed module and document descriptions. Understand what each covers and its boundaries. +4. Reason about which module(s) and which document(s) are relevant to the user's request. Consider: - Which module owns the functionality being changed? - Which modules might be affected as dependencies? - Which docs cover the design/architecture of this area? - Is this a new module or a change to an existing one? -4. **Present your reasoning to the user — both the module(s) and doc(s) you identified — and ask for confirmation.** Even if you are confident, you MUST ask. Do not proceed without confirmation. -5. If you are unsure, stop and ask the user to clarify. Do not guess. +5. **Present your reasoning to the user — both the module(s) and document(s) you identified — and ask for confirmation.** Even if you are confident, you MUST ask. Do not proceed without confirmation. +6. If you are unsure, stop and ask the user to clarify. Do not guess. Only after the user confirms the routing do you proceed to Step 2. @@ -183,42 +218,9 @@ Only after the user confirms the routing do you proceed to Step 2. ``` make generate ``` -3. Run the proxy unit tests to verify: - ``` - bash scripts/proxy-utest.sh --coverage - ``` -4. Run **all** of the proxy E2E tests below — every one, not just the first. Run them one at a time (they bind fixed ports, so they cannot run in parallel), and do not stop early: a later test can fail even when the earlier ones pass. - - Single-origin RTMP proxy test (starts proxy + one SRS origin, publishes RTMP, verifies playback): - ``` - bash scripts/proxy-e2e-test.sh - ``` - - Multi-origin cluster routing test (starts proxy + two SRS origins, publishes multiple streams, verifies streams are assigned to different origins): - ``` - bash scripts/proxy-e2e-cluster-test.sh - ``` - - Proxy + SRS edge + SRS origin three-tier topology (starts proxy + one SRS edge in `mode remote` registered with the proxy + one upstream SRS origin, publishes RTMP via proxy→edge→origin, then plays the same stream with two concurrent RTMP players where the second joins after a delay as a late joiner on the active edge-pull): - ``` - bash scripts/proxy-e2e-edge-test.sh - ``` - - Redis multi-proxy routing test (requires local Redis; starts two proxy instances with Redis LB, publishes through one proxy, verifies playback through the other): - ``` - bash scripts/proxy-e2e-redis-test.sh - ``` - - RTMP transmuxing test (starts proxy + one SRS origin, publishes RTMP, verifies RTMP/HTTP-FLV/HLS playback; WebRTC WHEP is a placeholder): - ``` - bash scripts/proxy-e2e-transmux-test.sh - ``` - - SRT proxy + transmuxing test (starts proxy + one SRS origin, publishes SRT, verifies SRT/RTMP/HTTP-FLV/HLS playback; WebRTC WHEP is a placeholder). Requires an ffmpeg built with libsrt; the script auto-runs `scripts/setup-ffmpeg-with-whip.sh` to build one into `~/.local/` if no SRT-capable ffmpeg is found: - ``` - bash scripts/proxy-e2e-srt-test.sh - ``` - - WHIP proxy + transmuxing test (starts proxy + one SRS origin, publishes WebRTC via WHIP, verifies RTMP/HTTP-FLV/HLS playback; WebRTC WHEP is a placeholder). Requires an ffmpeg with the `whip` muxer (built with `--enable-openssl`); the script auto-runs `scripts/setup-ffmpeg-with-whip.sh` if no suitable ffmpeg is found: - ``` - bash scripts/proxy-e2e-whip-test.sh - ``` -5. If any tests fail, fix the issues and re-run until all tests pass. - -All script paths are relative to this skill's directory. +3. Use `skills/internal-codemap-for-srs/SKILL.md` to route to the testing and verification map. +4. Run the proxy unit test and every proxy E2E test required by that map, sequentially and without stopping early. +5. If any test fails, fix the issue and re-run until all required tests pass. ### Origin Server diff --git a/skills/srs-develop/references/issues.md b/skills/srs-develop/references/issues.md index c6078cc5d1a..0e1d7dc359a 100644 --- a/skills/srs-develop/references/issues.md +++ b/skills/srs-develop/references/issues.md @@ -72,3 +72,56 @@ External FFmpeg ingest is a separate process. However, ordinary inbound publishi SRS does not currently provide an internal workaround for blocking DNS resolution. Avoid configuring domain names for SRS-managed outbound connections. Use numeric IP addresses or localhost instead. For HTTP hooks or callbacks that must ultimately reach a domain name, run a local sidecar or worker HTTP service, for example in Go. Configure SRS to send the callback to `127.0.0.1`; the sidecar performs DNS resolution and forwards the request to the remote domain. This keeps DNS resolution outside the SRS business thread. + +## #4681 — CURRENT + +- Issue: https://github.com/ossrs/srs/issues/4681 +- Truth Record: https://github.com/ossrs/srs/issues/4681#issuecomment-5084234253 +- Verified: 2026-07-26 +- Branch: `forge` +- Commit: `21f94c953735675f159405ba00f426c3f7e98305` +- Version: SRS `8.0.4` +- Environment: macOS 26.5.2, arm64; FFmpeg 8.1.1 +- Supersedes: None +- Changes and tests: No project change. Reproduced with one H.264 video track and two AAC audio tracks. + +**Current state** + +SRS does not currently support Enhanced RTMP v2 multitrack audio. Publishing a stream containing Enhanced RTMP v2 audio headers causes SRS to reject `SoundFormat=9` as unsupported. + +The limitation exists in both the reported SRS 7.0.147 source and the current SRS 8.0.4 source. The current media model maintains one audio codec, parser, and sequence header per live stream and has no per-track state. + +The targeted reproduction generated a two-second FLV stream containing one H.264 video track and two independent AAC audio tracks. `ffprobe` confirmed all three tracks. Publishing it to the locally built SRS 8.0.4 terminated the publisher with `Broken pipe`; SRS reported: + +```text +rtmp: consume audio : format consume audio : unsupported audio codec=9(Other) +``` + +FFmpeg 8 supports multitrack audio and video using Enhanced FLV v2. In the Enhanced RTMP v2 specification, `SoundFormat=9` identifies the extended audio header used for features including multitrack audio: + +- https://github.com/FFmpeg/FFmpeg/blob/master/Changelog +- https://github.com/veovera/enhanced-rtmp/blob/main/docs/enhanced/enhanced-rtmp-v2.md + +**Project direction** + +Enhanced RTMP v2 multitrack support is a valuable feature that SRS intends to support in the future. It belongs to the same class of substantial protocol work as Media over QUIC and other next-generation media protocols. + +This feature is not the current priority. SRS is presently refining its code structure, documentation, knowledge base, development methods, and verification workflows so that AI can reliably manage and maintain the project. + +After this AI-maintenance foundation is mature, the project can revisit Enhanced RTMP v2 and other major protocol features. + +**Conclusion** + +This is a valid future feature request, not a regression bug. Keep the issue open for future consideration. No immediate project change is required. + +**Current workarounds** + +Publish only one selected audio track, mix the tracks into one audio stream before publishing, or publish independent tracks under separate stream URLs. + +**Unknowns** + +- Implementation schedule +- Whether support belongs in the C++ server or the next-generation Go server +- Required capability negotiation and interoperability +- Track-selection behavior for RTMP playback, forwarding, HLS, WebRTC, recording, and other outputs +- Whether the first implementation should support audio only or complete Enhanced RTMP v2 audio and video multitrack functionality diff --git a/skills/srs-develop/scripts/proxy-e2e-redis-test.sh b/skills/srs-develop/scripts/proxy-e2e-redis-test.sh index fee4a0b9416..95a25e4bed2 100755 --- a/skills/srs-develop/scripts/proxy-e2e-redis-test.sh +++ b/skills/srs-develop/scripts/proxy-e2e-redis-test.sh @@ -40,12 +40,16 @@ REDIS_PORT="${PROXY_REDIS_PORT:-6379}" REDIS_PASSWORD="${PROXY_REDIS_PASSWORD:-}" REDIS_DB="${PROXY_REDIS_DB:-0}" PYTHON_BIN="${PYTHON_BIN:-python3}" +if [[ -n "${PROXY_REDIS_KEY_PREFIX:-}" ]]; then + REDIS_KEY_PREFIX="$PROXY_REDIS_KEY_PREFIX" +else + REDIS_KEY_PREFIX="e2e-$(LC_ALL=C tr -dc 'a-f0-9' /dev/null || echo 0)" != "0" ]]; then redis_cli del "$stream_key" >/dev/null 2>&1 || true count=$((count + 1)) @@ -90,6 +104,8 @@ cleanup_redis_state() { # The origin server generates its server key from runtime IDs, so discover only # server records that match this test origin's identity and configured ports. local server_keys=() + local server_pattern + server_pattern="$(redis_key "srs-proxy-server:*")" local key value while IFS= read -r key; do [[ -z "$key" ]] && continue @@ -102,12 +118,13 @@ cleanup_redis_state() { redis_cli del "$key" >/dev/null 2>&1 || true count=$((count + 1)) fi - done < <(redis_cli --scan --pattern 'srs-proxy-server:*' 2>/dev/null || true) + done < <(redis_cli --scan --pattern "$server_pattern" 2>/dev/null || true) # Keep the shared server index, but remove only the test origin server keys. if [[ ${#server_keys[@]} -gt 0 ]]; then - local servers_json updated_json - servers_json="$(redis_cli get srs-proxy-all-servers 2>/dev/null || true)" + local servers_key servers_json updated_json + servers_key="$(redis_key "srs-proxy-all-servers")" + servers_json="$(redis_cli get "$servers_key" 2>/dev/null || true)" if [[ -n "$servers_json" ]]; then updated_json="$($PYTHON_BIN - "$servers_json" "${server_keys[@]}" <<'PY' import json, sys @@ -118,9 +135,9 @@ print(json.dumps(servers, separators=(",", ":"))) PY )" if [[ "$updated_json" == "[]" ]]; then - redis_cli del srs-proxy-all-servers >/dev/null 2>&1 || true + redis_cli del "$servers_key" >/dev/null 2>&1 || true else - redis_cli set srs-proxy-all-servers "$updated_json" >/dev/null 2>&1 || true + redis_cli set "$servers_key" "$updated_json" >/dev/null 2>&1 || true fi fi fi @@ -149,6 +166,7 @@ trap cleanup EXIT echo "=== E2E RTMP Proxy Redis Load Balancer Test ===" echo "Workspace: $WORKSPACE" echo "Redis: $REDIS_HOST:$REDIS_PORT db=$REDIS_DB" +echo "Redis key prefix: $REDIS_KEY_PREFIX" echo "" # --- Pre-checks --- @@ -224,6 +242,7 @@ env PROXY_RTMP_SERVER=$PROXY_A_RTMP_PORT \ PROXY_REDIS_PORT="$REDIS_PORT" \ PROXY_REDIS_PASSWORD="$REDIS_PASSWORD" \ PROXY_REDIS_DB="$REDIS_DB" \ + PROXY_REDIS_KEY_PREFIX="$REDIS_KEY_PREFIX" \ ./bin/srs-proxy >/tmp/srs-proxy-redis-a-e2e.log 2>&1 & PROXY_A_PID=$! echo "Proxy A PID: $PROXY_A_PID" @@ -250,6 +269,7 @@ env PROXY_RTMP_SERVER=$PROXY_B_RTMP_PORT \ PROXY_REDIS_PORT="$REDIS_PORT" \ PROXY_REDIS_PASSWORD="$REDIS_PASSWORD" \ PROXY_REDIS_DB="$REDIS_DB" \ + PROXY_REDIS_KEY_PREFIX="$REDIS_KEY_PREFIX" \ ./bin/srs-proxy >/tmp/srs-proxy-redis-b-e2e.log 2>&1 & PROXY_B_PID=$! echo "Proxy B PID: $PROXY_B_PID" @@ -279,7 +299,7 @@ if ! kill -0 "$ORIGIN_PID" 2>/dev/null; then cat /tmp/srs-origin-redis-e2e.log >&2 exit 1 fi -if ! redis_cli --scan --pattern 'srs-proxy-server:*' | grep -q 'srs-proxy-server:'; then +if ! redis_cli --scan --pattern "$(redis_key "srs-proxy-server:*")" | grep -q 'srs-proxy-server:'; then echo "Error: SRS origin did not register in Redis. Proxy A logs:" >&2 cat /tmp/srs-proxy-redis-a-e2e.log >&2 exit 1 diff --git a/skills/srs-develop/scripts/proxy-e2e-srt-test.sh b/skills/srs-develop/scripts/proxy-e2e-srt-test.sh index 38832e74626..4b314958fbe 100755 --- a/skills/srs-develop/scripts/proxy-e2e-srt-test.sh +++ b/skills/srs-develop/scripts/proxy-e2e-srt-test.sh @@ -46,7 +46,7 @@ SRS_BINARY="$WORKSPACE/trunk/objs/srs" STREAM_URL="live/srt$(date +%s)" # SRT streamid format used by SRS: "#!::r=/,m=publish|request". -# @see trunk/3rdparty/srs-docs/doc/srt.md and internal/proxy/srt.go. +# @see skills/internal-docs-for-srs/references/cpp-docs/doc/srt.md and internal/proxy/srt.go. SRT_PUBLISH_URL="srt://localhost:$PROXY_SRT_PORT?streamid=#!::r=$STREAM_URL,m=publish" SRT_PLAY_URL="srt://localhost:$PROXY_SRT_PORT?streamid=#!::r=$STREAM_URL,m=request" diff --git a/skills/srs-support/SKILL.md b/skills/srs-support/SKILL.md index 661610dff3b..896eeb736f9 100644 --- a/skills/srs-support/SKILL.md +++ b/skills/srs-support/SKILL.md @@ -1,6 +1,6 @@ --- name: srs-support -description: Answer SRS (Simple Realtime Server) questions for users and operators — protocols, configuration, codecs, ecosystem tools, deployment, and troubleshooting. Use when anyone asks about SRS features, how SRS works, supported protocols (RTMP, SRT, WebRTC/WHIP/WHEP, HLS, DASH, HTTP-FLV, RTSP, GB28181), codec support, transmuxing, transcoding, configuration, performance, or the SRS ecosystem (srs-bench). Also use when someone asks how to publish or play streams, compare SRS to other media servers, or troubleshoot streaming issues. +description: Answer SRS (Simple Realtime Server) questions for users and operators — protocols, configuration, codecs, ecosystem tools, deployment, troubleshooting, and community support. Use when anyone asks about SRS features, how SRS works, supported protocols (RTMP, SRT, WebRTC/WHIP/WHEP, HLS, DASH, HTTP-FLV, RTSP, GB28181), codec support, transmuxing, transcoding, configuration, performance, the SRS ecosystem (srs-bench), community channels, or the SRS Robot. Also use when someone asks how to publish or play streams, compare SRS to other media servers, find community help, or troubleshoot streaming issues. --- # SRS Support @@ -22,15 +22,20 @@ This skill is for **operators, users, and DevOps** — answering questions about - Teaching users about SRS internals or source code — you may read source code to answer user questions better, but don't guide users into understanding the code itself. The goal is to help them *use* SRS, not develop it. - **Oryx** — Oryx is not supported by this AI yet. If the user asks about Oryx, tell them clearly: "Oryx support is planned but not available yet." Do not attempt to answer Oryx-specific questions. +## Skill Dependencies + +- `skills/internal-docs-for-srs/SKILL.md` — Route and load project documentation. This skill remains responsible for the support workflow and final answer. +- `skills/internal-codemap-for-srs/SKILL.md` — Route source-code investigation to the relevant server map. This skill remains responsible for the support workflow and final answer. + ## Workflow -Follow these three steps in order for every question. +Follow these four steps in order for every question. ## Step 1: Setup All files are in the current working directory. Find everything from here — no discovery logic needed. -Available directories: `trunk/`, `cmd/`, `internal/`, `cmake/`, `docs/`, `memory/` +Available directories: `trunk/`, `cmd/`, `internal/`, `cmake/`, `memory/`, `skills/` All AI tools — OpenClaw, Codex, Claude Code, Kiro CLI — see the same relative paths. @@ -39,42 +44,19 @@ All AI tools — OpenClaw, Codex, Claude Code, Kiro CLI — see the same relativ Load knowledge in layers. Start minimal, expand only if needed. **Layer 1 — Always load:** -- `memory/srs-overview.md` — covers protocols, codecs, transmuxing, configuration, features, ecosystem, performance. This answers most general questions. - -**Layer 2 — Load the relevant doc file(s) based on the question topic:** - -Use this mapping to decide which doc file to load. Only load what's relevant — don't load all of them. - -| Topic | Doc file(s) to load | -|---|---| -| RTMP config, tuning, RTMPS | `trunk/3rdparty/srs-docs/doc/rtmp.md` | -| HLS config, latency, encryption, fMP4 | `trunk/3rdparty/srs-docs/doc/hls.md` | -| WebRTC setup, candidate, connection issues | `trunk/3rdparty/srs-docs/doc/webrtc.md` | -| HTTP-FLV, WebSocket FLV | `trunk/3rdparty/srs-docs/doc/flv.md` | -| SRT config, streamid, latency modes | `trunk/3rdparty/srs-docs/doc/srt.md` | -| RTSP playback | `trunk/3rdparty/srs-docs/doc/rtsp.md` | -| HEVC/H.265 protocol support | `trunk/3rdparty/srs-docs/doc/hevc.md` | -| DVR, recording to file | `trunk/3rdparty/srs-docs/doc/dvr.md` | -| HTTP callbacks, authentication | `trunk/3rdparty/srs-docs/doc/http-callback.md` | -| IP allow/deny, access control | `trunk/3rdparty/srs-docs/doc/security.md` | -| HTTP API, stream monitoring | `trunk/3rdparty/srs-docs/doc/http-api.md` | -| Prometheus, Grafana, metrics | `trunk/3rdparty/srs-docs/doc/exporter.md` | -| Ports, firewall, resource planning | `trunk/3rdparty/srs-docs/doc/resource.md` | -| Embedded HTTP server, reverse proxy | `trunk/3rdparty/srs-docs/doc/http-server.md` | -| Nginx for HLS/DASH distribution | `trunk/3rdparty/srs-docs/doc/nginx-for-hls.md` | -| Edge server, CDN clustering | `trunk/3rdparty/srs-docs/doc/edge.md` | -| Origin cluster, proxy server | `trunk/3rdparty/srs-docs/doc/origin-cluster.md` | -| Low latency tuning | `trunk/3rdparty/srs-docs/doc/low-latency.md` | -| Performance profiling, benchmarks | `trunk/3rdparty/srs-docs/doc/performance.md` | -| Ingest external streams | `trunk/3rdparty/srs-docs/doc/ingest.md` | -| Forward to other servers | `trunk/3rdparty/srs-docs/doc/forward.md` | -| FFmpeg transcoding | `trunk/3rdparty/srs-docs/doc/ffmpeg.md` | -| Snapshots, thumbnails | `trunk/3rdparty/srs-docs/doc/snapshot.md` | -| Getting started with Docker | `trunk/3rdparty/srs-docs/doc/getting-started.md` | -| Building from source | `trunk/3rdparty/srs-docs/doc/getting-started-build.md` | +- `references/srs-overview.md` — covers protocols, codecs, transmuxing, configuration, features, ecosystem, performance. This answers most general questions. + +**Layer 2 — Load relevant project documentation through `internal-docs-for-srs`:** + +1. Load `skills/internal-docs-for-srs/SKILL.md`. +2. Use its Reference Router to select the relevant C++ media server documentation reference. +3. Load only the project documents relevant to the question. Do not duplicate documentation routing or guess document paths in this skill. **Layer 3 — Last resort (if you need source code to answer):** -- `memory/srs-codebase-map.md` — load the **entire file** (do not truncate or read partial content). Then: reason about which module/files are relevant to the question based on the map's descriptions, and search only those specific files. **DO NOT grep broadly** (e.g., `trunk/src/` or the repository root). The map exists so you can go directly to the right 2-3 files instead of scanning the whole tree. + +1. Load `skills/internal-codemap-for-srs/SKILL.md`. +2. Use its Reference Router to select the C++ media server or next-generation Go server map based on the component being investigated and, when needed, the testing and verification map. +3. Use the selected map to identify the smallest relevant module and file set. Do not grep broad directories or the repository root. ## Step 3: Answer by Topic @@ -88,7 +70,7 @@ Classify the question into one of the topics below, then apply that topic's stra **Protocol Questions** - State which protocols SRS supports and their role (publish, play, or both) -- Include the version and date when a protocol was added (from the Features list in srs-overview.md) +- Include the version and date when a protocol was added (from the Features list in `references/srs-overview.md`) - Clarify transport: which protocols use TCP vs UDP - For protocol comparisons, explain the tradeoffs (latency, compatibility, performance) @@ -100,14 +82,14 @@ Classify the question into one of the topics below, then apply that topic's stra **Configuration Questions** - Reference `trunk/conf/full.conf` as the complete configuration reference -- For specific features, load the relevant doc file from Layer 2 — it contains detailed config options and examples +- For specific features, use Layer 2 to load the relevant documentation with detailed configuration and examples - Mention environment variable support for Docker/cloud-native deployments - For getting started, recommend `trunk/conf/console.conf` for local testing **Deployment & Getting Started** - Provide the standard build steps: `cd trunk && ./configure && make` - Show the basic publish/play workflow with FFmpeg and common players -- For Docker questions, reference `trunk/conf/docker.conf` and load `getting-started.md` +- For Docker questions, reference `trunk/conf/docker.conf` and use Layer 2 to load the getting-started documentation - Note that SRS is Linux-only (use WSL on Windows, macOS works for development) **Architecture Questions** @@ -131,6 +113,11 @@ Classify the question into one of the topics below, then apply that topic's stra - **Oryx** — Tell the user: "Oryx support is planned but not available yet from this AI." Do not attempt to answer Oryx-specific questions. - SRS only maintains server-side projects — it doesn't maintain client-side tools +**Community Support Questions** +- For users seeking community help or the SRS Robot, provide the Telegram and Discord links from `references/srs-overview.md` +- Tell users to join a group and mention (`@`) the SRS Robot when asking a question +- Recommend Telegram because users can create small, focused groups with cleaner context; explain that large groups may mix unrelated conversations + ## Step 4: Troubleshooting When the user reports a problem ("it's not working", "stream won't play", "high latency", etc.), follow this troubleshooting strategy. @@ -144,22 +131,22 @@ When the user reports a problem ("it's not working", "stream won't play", "high - Any error messages or log output **SRS diagnostic tools:** -- **HTTP API** — Check active streams: `curl http://localhost:1985/api/v1/streams`. Check clients: `curl http://localhost:1985/api/v1/clients`. Check server info: `curl http://localhost:1985/api/v1/summaries`. Load the `http-api.md` doc for full API reference. +- **HTTP API** — Check active streams: `curl http://localhost:1985/api/v1/streams`. Check clients: `curl http://localhost:1985/api/v1/clients`. Check server info: `curl http://localhost:1985/api/v1/summaries`. Use Layer 2 to load the full HTTP API reference. - **Logs** — SRS uses traceable log with context IDs. Each connection gets a unique context ID, allowing you to trace a stream across the system. Check `trunk/objs/srs.log` or console output. -- **Prometheus** — If configured, check metrics at the exporter endpoint. Load `exporter.md` for setup. +- **Prometheus** — If configured, check metrics at the exporter endpoint. Use Layer 2 to load the exporter setup documentation. **Common failure patterns and solutions:** *WebRTC won't connect from remote browser:* -- Most common cause: **candidate misconfiguration**. The `candidate` in `rtc_server` must be set to the server's public IP, not `127.0.0.1` or a private IP. Load `webrtc.md` for details. +- Most common cause: **candidate misconfiguration**. The `candidate` in `rtc_server` must be set to the server's public IP, not `127.0.0.1` or a private IP. Use Layer 2 to load the WebRTC documentation. - HTTPS is required for WebRTC from non-localhost browsers. Without HTTPS, the browser blocks `getUserMedia`. - Check that UDP port 8000 is open in the firewall. WebRTC uses UDP by default. -- Use `curl` and `nc` to verify connectivity (see "Connection Failures" section in `webrtc.md`). +- Use `curl` and `nc` to verify connectivity using the connection-failure guidance in the selected WebRTC documentation. *HLS latency is too high (20-30 seconds):* - Default HLS latency is high by design (segment-based). To reduce: decrease `hls_fragment` (e.g., 2s), decrease `hls_window` (e.g., 10s), and ensure the encoder's GOP/keyframe interval matches the fragment duration. - Player-side buffering also matters — some players buffer aggressively. -- Load `hls.md` and `low-latency.md` for config details. +- Use Layer 2 to load the HLS and low-latency documentation for configuration details. - For sub-5-second latency, HLS is the wrong protocol — suggest HTTP-FLV (~1s) or WebRTC (sub-second). *Stream plays fine in one protocol but not another:* @@ -178,7 +165,7 @@ When the user reports a problem ("it's not working", "stream won't play", "high - HTTP-FLV requires chunked transfer encoding — verify Nginx passes it through - HLS works well behind Nginx with proxy caching - WebRTC WHIP/WHEP needs proper proxy headers -- Load `http-server.md` for reverse proxy config examples (Nginx, Caddy) +- Use Layer 2 to load reverse-proxy configuration examples for Nginx and Caddy *Connection limit reached:* - Check `max_connections` in config (default varies by version) @@ -188,4 +175,4 @@ When the user reports a problem ("it's not working", "stream won't play", "high *Ports and firewall:* - Default ports: RTMP 1935 (TCP), HTTP API 1985 (TCP), HTTP streaming 8080 (TCP), WebRTC 8000 (UDP), SRT 10080 (UDP) - UDP ports are often blocked by firewalls — check explicitly -- Load `resource.md` for the full port reference +- Use Layer 2 to load the full port reference diff --git a/memory/srs-overview.md b/skills/srs-support/references/srs-overview.md similarity index 96% rename from memory/srs-overview.md rename to skills/srs-support/references/srs-overview.md index 4d19fe30416..f9a56ad9c70 100644 --- a/memory/srs-overview.md +++ b/skills/srs-support/references/srs-overview.md @@ -191,6 +191,7 @@ By default, transmuxing between sources is disabled. You need to enable it in th **Clustering:** - **Origin Cluster** — Used to extend the number of streams SRS can support. It is a cluster of multiple origin servers behind a proxy server. The proxy discovers which origin server a stream is on and routes to it. v3.0, 2018-02 +- **Proxy Redis Key Namespace** — The next-generation Go proxy can isolate independent clusters sharing one Redis database with the optional `PROXY_REDIS_KEY_PREFIX`; the default is empty for backward-compatible key names. v8.0.4, 2026-07 - **Edge Cluster** — Used to extend the number of viewers a stream can support. The current edge cluster is deprecated because it only supports the RTMP protocol, but a new edge cluster is planned to support more protocols. v1.0, 2014-04 - **HLS Cluster** — Built by Nginx. It is a type of edge cluster for HLS. v5.0, 2022-04 @@ -264,6 +265,14 @@ SRS has been developed for over 13 years and has accumulated many useful feature **AI as Maintainer** — William is actively working on introducing AI as a project maintainer — not just for bug fixes or code generation, but as a full maintainer like himself. The approach is to build a comprehensive knowledge base so that AI can understand the project deeply: the architecture, design decisions, history, and community context. The goal is to have an AI maintainer within roughly six months (mid-2026). This is an experiment in using AI to maintain complex software projects — not just small ones, but projects like media servers written in C++ where you can't simply let AI generate code and push it to production. For any complex backend server or service, you need confidence that AI truly understands what it's doing before trusting it with real changes. The approach SRS is developing — building a deep knowledge base so AI can act as a real maintainer — applies broadly to any project where correctness and reliability matter. +**SRS Community Bot (OpenClaw):** +- William set up an OpenClaw robot for the SRS community on 2026-03-20. +- **Telegram group:** https://t.me/+RiynvKOxpQ42MGJl +- **Discord server:** https://discord.gg/yZ4BnPmHAd +- Join a group and mention (`@`) the **SRS Robot** to interact with it. +- The bot scales William's expertise to the community without requiring him to answer every question. +- **Telegram is recommended over Discord:** Telegram lets users create small, focused groups and invite the bot. Each small group provides a clean context window, while large groups mix unrelated conversations and reduce answer quality. + **How to Participate:** - **Discord** — Join the SRS Discord community for discussions and support. - **Monthly Community Meetings** — The community holds monthly meetings to discuss project status, AI maintainer progress, and how to use AI in open source maintenance. Everyone is welcome to join. diff --git a/trunk/scripts/copy-srs-docs.sh b/trunk/scripts/copy-srs-docs.sh index 7878831137d..a8a8da6b010 100755 --- a/trunk/scripts/copy-srs-docs.sh +++ b/trunk/scripts/copy-srs-docs.sh @@ -1,32 +1,35 @@ #!/bin/bash -if [[ ! -d 3rdparty/srs-docs ]]; then - echo "no srs-docs in $(pwd)" - exit -1 -fi +SRS_WORK_DIR=$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd) +SRS_DOCS="$SRS_WORK_DIR/skills/internal-docs-for-srs/references/cpp-docs" -mkdir -p 3rdparty/srs-docs/blog && -cp ~/git/srs-docs/for-writers/blog-en/*.md 3rdparty/srs-docs/blog/ -if [[ $? -ne 0 ]]; then - echo "copy blog failed" +if [[ ! -d "$SRS_DOCS" ]]; then + echo "no cpp-docs in $SRS_WORK_DIR" exit -1 fi -echo "Copy blog success" -mkdir -p 3rdparty/srs-docs/doc && -cp ~/git/srs-docs/for-writers/doc-en-7.0/doc/*.md 3rdparty/srs-docs/doc/ -if [[ $? -ne 0 ]]; then - echo "copy doc failed" - exit -1 -fi +for target in "$SRS_DOCS/doc/"*.md; do + source=~/git/srs-docs/for-writers/doc-en-7.0/doc/"$(basename "$target")" + if [[ ! -f "$source" ]]; then + continue + fi + if ! cp "$source" "$target"; then + echo "copy doc failed" + exit -1 + fi +done echo "Copy doc success" -mkdir -p 3rdparty/srs-docs/pages && -cp ~/git/srs-docs/for-writers/pages/*-en.md 3rdparty/srs-docs/pages/ && -if [[ $? -ne 0 ]]; then - echo "copy pages failed" - exit -1 -fi +for target in "$SRS_DOCS/pages/"*.md; do + source=~/git/srs-docs/for-writers/pages/"$(basename "$target")" + if [[ ! -f "$source" ]]; then + continue + fi + if ! cp "$source" "$target"; then + echo "copy pages failed" + exit -1 + fi +done echo "Copy pages success" echo "Done"