Skip to content

Latest commit

 

History

History
431 lines (346 loc) · 18.6 KB

File metadata and controls

431 lines (346 loc) · 18.6 KB

CodeGraph

This project has a CodeGraph MCP server (codegraph_* tools) configured. CodeGraph is a tree-sitter-parsed knowledge graph of every symbol, edge, and file. Reads are sub-millisecond and return structural information grep cannot.

MANDATORY: codegraph first, always

You are FORBIDDEN from using grep / rg / Select-String for code exploration. The ONLY exception is searching for literal text in comments, log messages, documentation, or hardcoded strings. For everything else — finding symbols, tracing call graphs, locating definitions, exploring file structure — use the codegraph tools below.

You want to do this ❌ NEVER use ✅ MUST use
Find where a function is defined grep "function_name" codegraph_search
Find who calls a function grep "function_name" across files codegraph_callers
Find what a function calls Read the source codegraph_callees
Understand a feature's architecture Read multiple files codegraph_context
See source of several related symbols Read each file individually codegraph_explore
Explore directory structure ls / Get-ChildItem codegraph_files
Assess impact of a change Manual grep across codebase codegraph_impact
Get a function's signature grep + read codegraph_node

Tool reference

Question Tool
"Where is X defined?" / "Find symbol named X" codegraph_search
"What calls function Y?" codegraph_callers
"What does Y call?" codegraph_callees
"What would break if I changed Z?" codegraph_impact
"Show me Y's signature / source / docstring" codegraph_node
"Give me focused context for a task/area" codegraph_context
"See several related symbols' source at once" codegraph_explore
"What files exist under path/" codegraph_files
"Is the index healthy?" codegraph_status

Rules of thumb

  • Answer directly — don't delegate exploration. For "how does X work" / architecture / trace questions, answer with 2-3 codegraph calls: codegraph_context first, then ONE codegraph_explore for the source of the symbols it surfaces. Codegraph IS the pre-built index, so spawning a separate file-reading sub-task/agent — or running a grep + read loop — repeats work codegraph already did and costs more for the same answer.
  • Trust codegraph results. They come from a full AST parse. Do NOT re-verify them with grep — that's slower, less accurate, and wastes context.
  • Don't chain codegraph_search + codegraph_node when you just want context — codegraph_context is one call.
  • Don't loop codegraph_node over many symbols — one codegraph_explore call returns several symbols' source grouped in a single capped call, while each separate node/Read call re-reads the whole context and costs far more.
  • Index lag: the file watcher debounces ~500ms behind writes; don't re-query immediately after editing a file in the same turn.

If .codegraph/ doesn't exist

The MCP server returns "not initialized." Ask the user: "I notice this project doesn't have CodeGraph initialized. Want me to run codegraph init -i to build the index?"

Ludo Project Guidance

  1. Do not silently guess. State your assumptions clearly. If anything is ambiguous, ask instead of choosing one interpretation silently. If a simpler solution exists, recommend it before implementing.
  2. Solve the requested problem with the minimum necessary code. Do not add features that were not asked for. Do not introduce abstractions for one-time use. Prefer simple, readable code over clever code.
  3. Only change what the task requires. Do not refactor unrelated code. Do not rewrite comments, formatting, or naming unless necessary. Match the existing style and conventions of the codebase.
  4. Keep edits local, focused, and easy to review. Touch as few files as possible. Change as little code as necessary. Avoid broad rewrites when a targeted fix is enough.
  5. Do not treat 'done' as a guess. Turn requests into clear success criteria. Prefer tests, existing checks, or concrete validation over verbal confidence.
  6. Understand the surrounding code before editing it. Read enough nearby code to understand how the target piece fits in. Identify the local conventions before introducing new patterns.
  7. Do not accidentally erase meaning while making changes. Preserve comments unless they are clearly outdated. Preserve public interfaces unless changing them is necessary.
  8. Do not continue blindly when the risk is high. Pause and ask if: the request is ambiguous, the codebase contains conflicting patterns, or the task requires an architectural decision.
  9. Before considering the task complete, confirm: the request was actually addressed, the change is no larger than necessary, and the final result matches the requested scope.
  10. Leave comment as is. Don't delete it.

1. Project Overview

Ludo is a desktop download manager written in C (C11, MinGW64/GCC 14) on Windows. It embeds Lua 5.2 as a plugin engine and exposes modules to scripts: http (libcurl), ludo (download manager), ui (libui-ng native widgets — GUI build only), and aes128_cbc_decrypt (console and GUI builds). Plugins are .lua files in plugins/ that return a table with validate(url) and process(url).

The project produces two executable targets:

  • ludo — GUI downloader with libui-ng (links libui_static + libuilua_static)
  • ludocon — Pure console CLI (BUILD_CONSOLE), no GUI dependencies

2. Build System

File Purpose
CMakeLists.txt Root CMake build (CMake ≥ 3.16)
build/ Out-of-source build directory
install/ Installed files (all assets, executables, config)
install/ludo.log Runtime log — always check here after test runs

Build Commands

in VSCode open w64devkit terminal (C:\w64devkit\w64devkit.exe) and then follow this command

# Configure + build (full)
cmake -B build . && cmake --build build --parallel

# Incremental build only
cmake --build build --parallel

# Build ludocon only (no GUI dependencies)
cmake -B build . -DBUILD_GUI=OFF
cmake --build build --target ludocon --parallel

# Install to local directory (all assets copied here)
cmake --install build --prefix install

# Create distribution package (ludo-<version>.zip)
cmake --build build --target package

Workflow Summary

Build  →  Install  →  Run / Test  →  Package
 cmake    cmake       ./install/     cmake
 --build  --install   ludo-debug    --build --target package

After every code change:

cmake --build build --parallel          # recompile
cmake --install build --prefix install  # refresh assets

Linux Build Prerequisites

On Linux, the build compiles curl, zlib, brotli, and zstd from bundled source. You need the following system packages:

# Ubuntu / Debian
sudo apt install cmake gcc g++ libgtk-3-dev libssl-dev pkg-config

CMake automatically copies plugins/*.lua, res/, snippets/, and config.ini to the install directory. You do not need to copy plugins manually after a successful install.


3. Plugin Development

3.1 Plugin Skeleton

Every plugin is a file in plugins/ returning a Lua table:

local plugin = { name = "Site", version = "YYYYMMDD", creator = "..." }

local json = json or require("json")  -- global already available

function plugin.validate(url)
    return url:match("https?://www%.example%.com/") ~= nil
end

function plugin.process(url)
    local body, status = http.get(url, { user_agent = ..., timeout = 30 })
    if status ~= 200 then ludo.logError("..."); return end

    local video_url = ...  -- extract from body

    local outdir = ludo.getOutputDirectory()
    -- Optional: pass extra HTTP headers sent with the CDN GET request.
    -- Useful when the server requires Referer or a session cookie (e.g. TikTok).
    local _, dl_status, output = ludo.newDownload(
        video_url, outdir, ludo.DOWNLOAD_NOW, "filename.mp4",
        { ["Referer"] = "https://site.com/" })

    if dl_status == 200 or dl_status == 206 or dl_status == 0 then
        ludo.logSuccess("Queued → " .. (output or "filename.mp4"))
    elseif dl_status == 403 then
        -- CDN may block HEAD probes; queue anyway if URL was just extracted.
        ludo.logSuccess("Queued (CDN blocked HEAD probe) → " .. (output or "filename.mp4"))
    else
        ludo.logError("Preflight HTTP " .. dl_status)
    end
end

return plugin

3.2 Key APIs

-- HTTP
local body, status, headers = http.get(url, { user_agent=..., timeout=N, headers={...} })
local body, status, headers = http.post(url, body_str, { ... })
http.set_cookie("path/to/cookies.txt")   -- Netscape format
http.clear_cookies()
local csrf = http.read_cookie("cookies.txt", "csrftoken")
local b64   = http.base64_encode(str)        -- binary-safe
local raw   = http.base64_decode(b64)        -- binary-safe
local digest = http.sha256(str)              -- 32-byte raw binary digest

-- Ludo
local id, status, path = ludo.newDownload(url [, dir [, mode [, filename [, headers]]]])
-- headers is an optional table: { ["Name"] = "Value", ... } (e.g. Referer, Cookie)
-- Cookies from http.set_cookie() are automatically shared with ludo.newDownload()
ludo.DOWNLOAD_NOW     -- constant
ludo.DOWNLOAD_QUEUE   -- constant
ludo.getOutputDirectory()  -- returns the configured output folder
ludo.logInfo("msg")
ludo.logSuccess("msg")
ludo.logError("msg")

-- Zip
local status [, errmsg] = zip.create(output_path, { file1, file2, ... })
local status [, errmsg] = zip.create(output_path, directory [, glob_filter])
-- status: 0 = success, -1 = failure; glob_filter e.g. "*.mp4" (case-insensitive, * and ? wildcards)

-- JSON (global)
local ok, data = pcall(json.decode, body)
local s = json.encode(table)

-- M3U8 (shared module: dofile("plugins/m3u8.lua"))
local m3u8 = dofile("plugins/m3u8.lua")
local ok, path_or_err = m3u8.download(m3u8_url, output_dir, "file.mp4" [, headers])
-- headers is an optional table: { ["Name"] = "Value", ... } (e.g. Referer)
local is_master = m3u8.is_master(playlist_text)
local variants = m3u8.parse_master(text, base_url)  -- array of {url, bandwidth, resolution}
local best = m3u8.pick_best(variants)  -- highest bandwidth variant

3.3 Common Patterns

Safe filename:

local function safe_name(s)
    return (s or ""):gsub("[^%w%-_]", "_"):sub(1, 80)
end

JSON unescape for embedded URLs (Facebook relay data, etc.):

local function unescape_json_url(s)
    if not s then return nil end
    s = s:gsub("\\/", "/")
    s = s:gsub("\\u(%x%x%x%x)", function(h)
        local cp = tonumber(h, 16)
        if cp < 128 then return string.char(cp) end
        return ""
    end)
    return s
end

Balanced-brace JSON object extraction from HTML:

local function extract_json_object(text, pattern)
    local _, pe = text:find(pattern)
    if not pe then return nil end
    local start = text:find("{", pe + 1)
    if not start then return nil end
    local depth, in_str, escaped = 0, false, false
    for i = start, #text do
        local c = text:sub(i, i)
        if escaped then escaped = false
        elseif c == "\\" and in_str then escaped = true
        elseif c == '"' then in_str = not in_str
        elseif not in_str then
            if c == "{" then depth = depth + 1
            elseif c == "}" then
                depth = depth - 1
                if depth == 0 then
                    local ok, val = pcall(json.decode, text:sub(start, i))
                    if ok then return val end
                    return nil
                end
            end
        end
    end
    return nil
end

Cookie jar setup:

local cookie_path = ludo.getOutputDirectory() .. "/mysite_cookies.txt"
local ck = io.open(cookie_path, "r")
if ck then ck:close(); http.set_cookie(cookie_path) end

Cookie sharing with downloads: After calling http.set_cookie(), any subsequent ludo.newDownload() call automatically uses the same cookie jar. There is no need to manually pass Cookie headers unless you need different cookies per download.

4. Testing

4.1 Test Commands

# Build, install, then run a test script (all output goes to ludo.log, not stdout)
cmake --build build --parallel
cmake --install build --prefix install
./install/ludo-debug.exe -s test_PLUGINNAME.lua

# Check results
grep -E "PASS|FAIL|SUCCESS|ERROR" install/ludo.log | tail -30

# Watch live
tail -f install/ludo.log

After any code change, refresh and test:

cmake --build build --parallel
cmake --install build --prefix install
./install/ludo-debug.exe -s test_PLUGINNAME.lua

4.2 Test Script Template

-- test_myplugin.lua
ludo.logInfo("=== MyPlugin test ===")

local ok, p = pcall(dofile, "plugins/myplugin.lua")
if not ok then ludo.logError("Load failed: " .. tostring(p)); return end

-- validate() cases
local cases = {
    { "https://site.com/valid/url", true  },
    { "https://other.com/bad/url",  false },
}
local passed, failed = 0, 0
for _, c in ipairs(cases) do
    local got = p.validate(c[1])
    if got == c[2] then
        ludo.logInfo("  PASS validate(" .. c[1] .. ") = " .. tostring(got))
        passed = passed + 1
    else
        ludo.logError("  FAIL validate(" .. c[1] .. ") expected=" .. tostring(c[2]))
        failed = failed + 1
    end
end
ludo.logInfo(("validate: %d passed, %d failed"):format(passed, failed))

-- process() live test
local ok2, err = pcall(p.process, "https://site.com/valid/video/123")
if not ok2 then ludo.logError("process() error: " .. tostring(err)) end

ludo.logInfo("=== MyPlugin test done ===")

5. Important Files

Path Role
src/http_module.c C implementation of http global (libcurl); shares DNS/SSL/connection cache via curl_share
src/lua_engine.c Loads and manages Lua state, plugin loader; sorts generic.lua last
src/ludo_module.c C implementation of ludo global
src/download_manager.c Threaded download queue; owns the global CURLSH share handle (DNS, SSL session, connection pool) initialized at startup
src/main.c Entry point; -s scriptfile.lua runs standalone
ludo_scripting.md Full scripting API reference (read this for detailed docs)
install/ludo.log Runtime log — all ludo.log*() calls write here
third_party/curl-8.19.0/ curl 8.19.0 for Windows build (static, Schannel)
third_party/curl-8.18.0/ curl 8.18.0 for Linux build (static, GnuTLS, brotli, zstd)
third_party/curl-8.18.0/lib/curl_config_linux.h curl config for Linux (brotli, zstd, GnuTLS enabled)
third_party/curl-8.18.0/lib/vtls/gtls.c Patched: early-data version check raised to 0x030700 for GnuTLS 3.6.x compat

6. Converting yt-dlp Extractors to Ludo Plugins

See ludo_scripting.md §7 for the full guide. Summary:

  1. Read _VALID_URL regex → write plugin.validate() using Lua patterns (\. not \., %d not \d, no alternation |, use multiple match())
  2. Read _real_extract → write plugin.process()
  3. self._download_webpage(url)http.get(url, { user_agent=..., headers={...} })
  4. self._search_json(regex, html)extract_json_object(html, lua_pattern)
  5. traverse_obj(data, path) → manual Lua table navigation with nil-guards
  6. self._download_json(url)http.get(url) + pcall(json.decode, body)
  7. self._get_cookies().namehttp.read_cookie(path, "name")
  8. No DASH/HLS support in Ludo — pick the most useful direct MP4 URL

Anti-bot considerations

Site Challenge Workaround
TikTok JS challenge, bot fingerprinting Plugin degrades gracefully; works when no challenge is triggered
Facebook Login wall for most content Netscape cookie file
Instagram Session gate for most content Netscape cookie file (sessionid)

7. Lua 5.2 Gotchas in This Codebase

  • %w does not include _; use [%w_] explicitly
  • No | alternation in patterns; use multiple match() calls
  • .- is the lazy quantifier (not *?)
  • json.decode throws on error — always wrap in pcall()
  • Lua doubles lose precision for integers > 2^53; use string arithmetic for large IDs (see bigint_muladd in instagram.lua)
  • string.match returns nil on no-match, not false
  • [%a-] in character classes may not parse correctly — - after %a/%w can be misparsed. Prefer splitting on , and parsing each KEY=VALUE pair individually (see parse_attrs in m3u8.lua).
  • All plugins share the same Lua state (globals persist between plugin calls)

8. Known Build Quirks

  • w64devkit environment: All build commands (cmake, gcc, ninja) must run inside the w64devkit shell (C:\w64devkit\w64devkit.exe). PowerShell or cmd.exe will fail with cannot execute 'cc1' or cannot execute 'as' because gcc's subprocess directory (libexec/gcc/x86_64-w64-mingw32/<version>/) is not on the PATH. The w64devkit terminal sets up PATH correctly.
  • CMake configure step logs a non-fatal Permission denied error for ninja restat — this is benign and can be ignored.
  • ludo-debug.exe --help hangs (it is a GUI app, not a CLI tool).
  • ludocon-debug.exe --help works (it is a console app, use ludocon for CLI tasks).
  • Building outside w64devkit terminal: If you experience cannot execute 'cc1': CreateProcess: No such file or directory, run cmake via cmd.exe with w64devkit bin directory on PATH:
    cmd.exe /c "set PATH=C:\w64devkit\bin;C:\w64devkit\libexec\gcc\x86_64-w64-mingw32\15.2.0;%PATH% && cmake --build build --parallel"
    
  • After editing a plugin, rebuild and reinstall (cmake --build build --parallel && cmake --install build --prefix install); CMake copies all plugins/*.lua to install/plugins/ on install.
  • Linux/Synology: The build uses system libcurl (via find_package(CURL)). Bundled brotli and zstd are built from source as static libraries for potential future use. At runtime, http_module.c and download_manager.c detect which content encodings curl supports via curl_version_info() and only advertise those (dynamic Accept-Encoding). On systems where curl lacks brotli/zstd, the binary falls back gracefully to gzip/deflate.
  • Linux build requirements: cmake, gcc, libgtk-3-dev (for GUI), libcurl4-openssl-dev. The system curl installation is used as-is.

9. Plugin Reference from yt-dlp

We can use yt-dlp extractor source code (C:\Projects\yt-dlp-2026.03.17\yt_dlp\extractor\facebook.py) and yt-dlp.log as reference for making ludo plugin. Run yt-dlp.exe --cookies downloads\facebook_cookies.txt --print-traffic -v https://www.facebook.com/reel/1520832029440197 >yt-dlp.log 2>&1 and check yt-dlp.log