# Staleproof — full documentation > Staleproof keeps a software organisation’s documentation true to its code. For a GitHub pull request it walks a FalkorDB graph of code (Python, TypeScript and JavaScript symbols and call edges across repositories), docs (Markdown, MDX and reStructuredText sections and code examples), maintainers’ decisions and past verdicts; flags the examples whose calls no longer fit the changed code; runs the impacted docs examples before and after the change in a sandbox; and proposes the docs fix with the graph path behind each paragraph. With no pull request it audits the docs (a health score, examples that no longer fit the code, API coverage) and checks a working tree before it is pushed. Open source (MIT), published on npm as `staleproof`. Index: https://staleproof.dev/llms.txt --- # Staleproof documentation Staleproof keeps a software organisation's documentation true to its code. This page explains what it does, the problem it solves, how a check works, and where to find everything else in these docs. Start here if you are new; jump to [Install](install.md) or the [Quickstart](getting-started.md) if you want to run it. ## What it is `staleproof` is an open-source (MIT) command-line tool, GitHub Action, MCP server, HTTP API and Node.js library, also hosted at [staleproof.dev](https://staleproof.dev). It indexes your repositories into a [FalkorDB](https://www.falkordb.com) graph: Python, TypeScript and JavaScript symbols and the calls between them across repositories, every Markdown, MDX and reStructuredText doc section and code example, and the code each one mentions or calls, with every example call checked against the signature it reaches. For a GitHub pull request it walks that graph to find every doc section the change can make false, predicts which examples the change breaks before anything runs, runs the affected Python examples before and after the change in a sandbox, and (in agent mode) proposes the docs fix with the path in the graph behind each paragraph. Everything it concludes is written back to the graph, so the next check starts from what the last one learned. With no pull request, it checks a working tree before it is pushed, lists the docs that depend on a function before you change it, and audits the docs as they are, with a 0-100 health score. ## The problem Docs drift for two reasons that code review does not catch: - **The paragraph is in another repository.** A client library, the SDK built on it and the docs site usually live in separate repositories. A change to a function in the client can falsify a page in the docs site that never names that function: the page calls an SDK method, which calls a client method, which calls the function that changed. Nobody reviewing the client pull request sees that page. - **Nobody re-runs the examples.** Code examples in docs are written once. When the library under them changes, or when they were never quite right, they stay broken until a reader reports it. Searching for the changed names does not solve either problem, because the docs that break are usually the ones that reach the changed code indirectly. ## A worked example [falkordb-py#262](https://github.com/FalkorDB/falkordb-py/pull/262) changed one function, `Is_Cluster`. The graph path is: ```text Is_Cluster ← FalkorDB.__init__ ← README "Asynchronous Example" ``` The README example never names `Is_Cluster`; it calls the constructor, which calls it. Staleproof ran that example (`FalkorDB/falkordb-py:README.md:4`) against falkordb-py installed from source at both commits. It **fails before the pull request** with `TypeError: Redis.__init__() got an unexpected keyword argument 'himport_registry'` and **passes after it**: the pull request fixed a broken README example. A whole-word grep for `Is_Cluster` finds no doc section at all. The same holds at larger scale. Across the four pull requests checked on FalkorDB's repositories, the graph reaches 120 doc sections; a whole-word grep for the changed names finds 30 of them and matches 227 other sections that no path from the change reaches. For the three falkordb-py pull requests, grep finds none of the 47 sections the graph reaches. On [falkordb-py#276](https://github.com/FalkorDB/falkordb-py/pull/276), 4 changed symbols reach 43 doc sections, 41 of them in other repositories. Every number here comes from [Facts from the graph](FACTS.md), which is generated from the graph. ## How a check works 1. **Index.** `staleproof index` (or the first check of a repository) clones your GitHub repositories, or downloads them where there is no git, and writes one graph per organisation: Python, TypeScript and JavaScript symbols and calls (resolved across repositories through imports, types and re-exports), doc sections and code examples, the symbols each one mentions or uses, and the example lines that no longer fit the code they call. 2. **Ingest the pull request.** The diff between the pull request's two commits becomes symbol-level changes (removed, renamed, signature, modified, added, export-removed), attached to the pull request in the graph. 3. **Trace.** Staleproof walks from each changed symbol back through up to 3 callers, in any repository, to the docs that mention or call them. That is the impact. The changed symbols with no such path are the clearance. For each impacted example it compares the call as written with the code after the pull request: a removed name or a call the new signature rejects is a predicted break. 4. **Prove.** For impacted Python examples, Staleproof installs the repositories from source at the commit before the pull request and at the pull request, runs the example in a sandbox against a scratch FalkorDB, and records both runs. 5. **Judge and remember.** In agent mode a model reads each impacted section, checks maintainers' decisions and earlier verdicts, and records a verdict or proposes an edit for every one. The docs patch is built from those edits. Every check, run and verdict stays in the graph. The terms used above, in plain language: - **Impact**: the doc sections and code examples a pull request can make false. A section is impacted when the graph has a path from a changed symbol, through up to 3 callers, to the section (because it mentions one of those symbols, or one of its examples calls one). - **Evidence path**: that path, written out node by node, changed symbol first and doc last, as in the #262 example above. It is the reason a section is listed, and Staleproof refuses to edit a section without one. - **Predicted break**: an example whose call fits the code before the pull request and not after it (a removed or renamed name, a new required argument, a keyword that no longer exists), found statically, before anything runs. - **Mismatch**: an example line that does not fit the code it calls right now, found when the docs are indexed. The [docs audit](audit.md) lists them all, with a health score. - **Clearance**: the changed symbols that no doc reaches within 3 call hops. The graph proves the absence; each one comes with a `git grep` command so you can check the claim yourself. - **Verdict**: the conclusion about one section for one pull request: `stale` (with replacement text), `ok`, `governed` (a maintainer decision says it stays as it is) or `skipped`. Maintainers can accept or reject each verdict. - **Verdict memory**: verdicts, example runs, maintainer decisions and checks stay in the graph. Later checks read them: an identical example run is answered from the graph, a decision is cited instead of re-flagging the section, and earlier verdicts (with whether maintainers accepted them) inform the next judgement. - **Agent mode and graph mode**: in agent mode a model chooses which tools to call and in what order, judges every impacted section and proposes edits. It needs model API credentials (Claude, or an Anthropic-compatible provider). Graph mode runs the same tools in a fixed order with no model and samples rather than judges: impact, clearance, a ranking of the first impact page and at most 3 example runs. It writes no verdicts and no edits, and needs no key. ## Ways to use it | Interface | What it is for | Docs | |---|---|---| | Website | Paste a pull request URL of any public repository on [staleproof.dev](https://staleproof.dev): indexed on the fly, checked in graph mode, nothing installed; or [audit a repository](https://staleproof.dev/#/audit) and get its README badge | [Install](install.md), [Docs audit](audit.md) | | CLI | `npx -y staleproof `: index, check a pull request or your working tree, audit, list the docs for a symbol, run an example, record a decision, build the patch; the first command starts FalkorDB and indexes the repository itself | [CLI](cli.md) | | GitHub Action | `uses: rajkaria/staleproof@v1`, no inputs or secrets: a comment on every pull request with the docs it can make false, the evidence path for each, the examples predicted to break and the examples run before and after; on a schedule, a docs audit kept in one issue | [GitHub Action](github-action.md) | | MCP server | The tools for Claude Code, Claude Desktop, claude.ai, Cursor, VS Code or any MCP client: hosted at `https://staleproof.dev/api/mcp`, or local over stdio | [MCP server](mcp.md) | | HTTP API | `staleproof serve`: checks and audits streamed over server-sent events, the README badge, traces, patches, decisions, and the web UI | [HTTP API](http-api.md) | | Node.js library | Index, check, patch and serve from your own TypeScript code | [Node.js library](library.md) | | Self-host | Vercel with FalkorDB Cloud, Docker Compose on a VM, or a single machine without Docker | [Deploy](DEPLOY.md) | ## Status - **Built and run** against FalkorDB's real repositories on the author's machine: the indexer, cross-repository impact, clearance, PageRank ranking, sandboxed example runs, decisions and verdict memory, the CLI, the MCP server, the web UI and graph-mode checks. - **Agent mode** was written for Claude (default model `claude-sonnet-5-5`) and runs on any Anthropic-compatible API through `ANTHROPIC_BASE_URL`. It has been run with a real model through DeepSeek's Anthropic-compatible endpoint (`deepseek-v4-pro`, also what the live demo uses). The Claude path is tested against a local fake API only. - **No edit proposed on FalkorDB's repositories yet.** The pull requests checked there are fixes, and every verdict recorded on #262, #267 and #276 is `ok`. Proposing edits and building the patch from them are exercised by the test suite on a fixture organisation. - **Run end to end**: the GitHub Action's pull request check, from the one-click workflow, on a private test repository. - **Written, not yet exercised**: the Docker image has not been built. - **Live**: [staleproof.dev](https://staleproof.dev), in hosted mode: any public repository is indexed on first use into a workspace graph of its own, and MCP answers at `/api/mcp`. Examples are not executed there (no sandbox on Vercel); for FalkorDB's repositories, the before/after results come from runs recorded on the graph. ## Limits - Code is read as Python, or TypeScript and JavaScript in published npm packages; docs as Markdown, MDX or reStructuredText. Only Python examples are run and get argument checks; JavaScript and TypeScript examples are linked and checked for names that do not exist. Other languages are indexed as text. - Static checks and predicted breaks judge only calls resolved through imports and types; where the source cannot settle a call, they stay silent. - GitHub repositories only. Private ones need `GITHUB_TOKEN` (the CLI, a server you run, the GitHub Action); the hosted site indexes public repositories only. - Call resolution is static. What types do not reveal falls back to by-name edges (labelled as such) or to no edge. - The example sandbox is a guard, not a security boundary, and example runs use today's third-party dependencies. ## Documentation map **Start here** - [Introduction](README.md): this page. - [Quickstart](getting-started.md): three ways to start (website, GitHub Action, one command), then a local FalkorDB, FalkorDB's three repositories indexed, a pull request checked, an example run, the patch, the web UI and MCP. - [Install](install.md): the website, the GitHub Action, the hosted and local MCP servers and the CLI, with what each needs and does. - [Use cases](use-cases.md): SDK teams with a separate docs site, DevRel after a release, maintainers reviewing pull requests, a weekly docs audit, developers before they push, AI coding agents (with and without a pull request), platform teams with many repositories, and recording maintainer decisions. - [Docs audit](audit.md): how true the docs are with no pull request: the checks, the score, the CLI, the web page and badge, the API, MCP and the scheduled Action. **Integrate** - [CLI](cli.md): every command and option, with examples, output and exit codes. - [GitHub Action](github-action.md): one-click install, what a run does, inputs and outputs, permissions, the comment format, the docs fix pull request, the scheduled audit and keeping a graph between runs. - [MCP server](mcp.md): the hosted endpoint, client configuration, the fourteen tools, result format, worked sessions (with and before a pull request) and security notes. - [HTTP API](http-api.md): every endpoint of `staleproof serve`, including indexing on demand, audits, the badge and MCP over HTTP. - [Node.js library](library.md): using Staleproof from TypeScript. **How it works** - [Core concepts](concepts.md): impact, evidence paths, severity, clearance, static checks, predicted breaks and the audit, verdicts, decisions, runs, checks, the two modes, ids. - [Indexing](indexing.md): which files are read (Python, TypeScript, JavaScript; Markdown, MDX, reStructuredText), how calls are resolved across repositories, how docs link to and are checked against code, example directives, re-index semantics. - [Example runner](example-runner.md): virtualenvs per commit, the sandbox, the scratch FalkorDB, skip rules, run outcomes. - [Decisions and memory](memory.md): maintainer decisions, supersession, verdict review, what every check writes back. - [The agent](agent.md): the loop, its system prompt and stop condition, model providers, budgets, graph mode. **Reference** - [Tools reference](tools.md): the fourteen tools, their parameters and the Cypher each runs. - [Configuration](configuration.md): every environment variable and global option, and the per-repository `.staleproof.toml`. - [Graph model and Cypher](GRAPH-MODEL.md): every Cypher statement Staleproof sends to FalkorDB, generated from the code. - [Facts from the graph](FACTS.md): every number quoted about FalkorDB's repositories, generated from the graph. - [Architecture](ARCHITECTURE.md): source layout and a check end to end. **Operate** - [Deploy](DEPLOY.md): Vercel with FalkorDB Cloud, Docker Compose, or no Docker; hosted mode and its limits. - [Troubleshooting and FAQ](troubleshooting.md): common errors, limits and answers. ## For LLMs and tools The site at [staleproof.dev](https://staleproof.dev) serves these docs as plain Markdown: an index at [/llms.txt](https://staleproof.dev/llms.txt), every page in one file at [/llms-full.txt](https://staleproof.dev/llms-full.txt), and each page at `/docs/.md` (this page is [/docs/introduction.md](https://staleproof.dev/docs/introduction.md)). The repository carries the same index as [llms.txt](../llms.txt), linking to the Markdown files on GitHub. ## Related - [Quickstart](getting-started.md) - [Use cases](use-cases.md) - [Core concepts](concepts.md) - [Facts from the graph](FACTS.md) --- # Quickstart From nothing to a checked pull request. First the three ways to start: no install on the website, one click for the GitHub Action, or one command. Then the full walkthrough: start FalkorDB, index FalkorDB's three public repositories, check a real pull request with and without a model, run one docs example before and after it, build the patch, check your own changes before there is a pull request, open the web UI and connect Claude Code; then your own repositories. Every command here is copy-pasteable; you need no account other than an optional model API key. ## Three ways to start ### No install: the website Open [staleproof.dev](https://staleproof.dev) and paste the URL of a pull request in any public Python, TypeScript or JavaScript repository on GitHub. The first check of a repository downloads and indexes it on the fly, then checks the pull request in graph mode: the doc sections it can make false with the evidence path for each, the changed symbols no doc reaches, and the ranking. Examples are not run on the hosted service, which has no sandbox. A docs site that lives in another repository can be named next to it, so both are indexed together. ### One click: the GitHub Action On [staleproof.dev/#/install](https://staleproof.dev/#/install), type your repository and click **Add to owner/repo**: GitHub opens its "new file" page with `.github/workflows/staleproof.yml` filled in, and you click **Commit changes**. Or copy [examples/staleproof.yml](../examples/staleproof.yml) into the repository yourself. From then on every pull request gets one comment with the docs it can make false and the examples run before and after it. It needs no secrets and no database: the job starts its own FalkorDB. See [GitHub Action](github-action.md). ### One command: the CLI ```bash npx -y staleproof check https://github.com///pull/ --no-agent ``` With no FalkorDB configured, the CLI starts a local one (falkordblite, no Docker), indexes the pull request's repository at its default branch head on first use, checks the pull request in graph mode and runs up to 3 affected examples before and after it. Add `--with /` when the docs live in another repository, and set `GITHUB_TOKEN` for a private repository. Without `--no-agent` the agent runs when model credentials are set ([step 4](#4-check-it-with-the-agent)); without them `check` prints a note and runs graph mode. It needs Node.js, git and uv or `python3` (below). [Install](install.md) compares these with the hosted and local MCP servers. The rest of this page walks through each part step by step. ## Prerequisites | You need | For | Notes | |---|---|---| | Node.js 20 or newer | everything | The package declares `node >= 20`. | | `git` | example runs; indexing, pull request ingestion and `patch` when it is installed | Repositories are cloned over HTTPS, blob-less, into `~/.staleproof/repos//`, sending `GITHUB_TOKEN` when it is set. Without git, indexing downloads a tarball of each repository and pull requests are read through the GitHub API, but examples cannot run. | | `bash` | the local FalkorDB (`db start`, or the one any command starts when none is configured) and the scratch FalkorDB examples run against | Both use the bundled `scripts/falkordb-local.sh`. | | [uv](https://docs.astral.sh/uv/) (recommended), or `python3` 3.12 or newer with `pip` | the local FalkorDB, the scratch FalkorDB, example runs | The local server comes from the `falkordblite` wheel (redis-server plus the FalkorDB module), which needs Python 3.12 or newer; with uv, a Python 3.12 is fetched for you. Example runs build one virtualenv per set of commits: with uv they use Python 3.12; with only `python3` they use whatever version `python3` is. | | `tar` | example runs | Sources are extracted with `git archive \| tar -x`. | | Docker (optional) | running FalkorDB in a container instead of the local server | | | `ANTHROPIC_API_KEY` (optional) | agent mode | Or an Anthropic-compatible provider; see [step 4](#4-check-it-with-the-agent). | | `GITHUB_TOKEN` (optional) | private repositories, pull request metadata, `patch --open-pr` | Without a token, GitHub allows 60 API requests an hour and only public repositories can be read. `GH_TOKEN` is read too. | You can run every command with `npx -y staleproof `, or install it once with `npm install -g staleproof` and drop the `npx -y`. See [CLI](cli.md#install-or-run-with-npx). ## 1. Start FalkorDB Staleproof keeps everything in a FalkorDB graph. The CLI connects to `FALKORDB_URL`, which defaults to `falkor://localhost:6379`. Pick one of the three options below. You can also skip this step: when neither `FALKORDB_URL` nor `--db` is set and nothing answers on `localhost:6379`, any command starts the local server described next and leaves it running. ### Without Docker ```bash npx -y staleproof db start ``` The first start installs `falkordblite` into `~/.staleproof/falkordb/venv`, then runs a FalkorDB server on `127.0.0.1:6379`. Data persists as RDB snapshots in `~/.staleproof/falkordb/data-6379`, so the graph, including verdicts and decisions, survives restarts. Stop it with `npx -y staleproof db stop`; check it with `npx -y staleproof db status`. ### With Docker ```bash docker run -d --name falkordb -p 6379:6379 falkordb/falkordb:latest ``` Examples run against a second, disposable FalkorDB that is flushed before every run. By default Staleproof starts that scratch server itself on port 6390 with the same script as `db start`, so it still needs uv or `python3`. To keep FalkorDB entirely in Docker, start the scratch server as a container too and point Staleproof at it: ```bash docker run -d --name staleproof-scratch -p 6390:6379 falkordb/falkordb:latest export STALEPROOF_SCRATCH_URL=falkor://localhost:6390 ``` Never point `STALEPROOF_SCRATCH_URL` at a server that holds data you need. Staleproof refuses to flush the server named in `FALKORDB_URL`, but any other server is wiped before each example run. The scratch server cannot require a password: docs examples connect to it without one, so a `STALEPROOF_SCRATCH_URL` with credentials is refused. ### FalkorDB Cloud Create an instance, then set its endpoint and password (user `falkordb`). URL-encode the password, and use `falkors://` if the endpoint has TLS: ```bash export FALKORDB_URL='falkor://falkordb:@:' ``` Leave the scratch server local (the default) or point `STALEPROOF_SCRATCH_URL` at a different disposable server. [Deploy](DEPLOY.md) covers FalkorDB Cloud in more detail. ## 2. Index FalkorDB's three repositories ```bash npx -y staleproof index FalkorDB/falkordb-py FalkorDB/GraphRAG-SDK FalkorDB/docs ``` This clones the three repositories (the client library, the SDK built on it, and the docs site), parses their Python code and Markdown/MDX docs, resolves calls across all three, checks every docs example against the code it calls, and writes the graph `falkordb`: a command that names a repository uses the graph of its owner, lowercased, unless `--org` or `STALEPROOF_ORG` names another. Progress goes to stderr; at the end it prints one line per repository with its commit, symbol and call counts, one line per docs repository with sections, examples and links, and the dependencies it found between the repositories (for example `FalkorDB/GraphRAG-SDK → FalkorDB/falkordb-py (declares falkordb)`). Index related repositories in one command: calls and doc links resolve across repositories only within a single `index` run. Re-running the same command later is safe; it updates the graph in place and marks what disappeared as gone instead of deleting it. ## 3. Check a pull request in graph mode Graph mode needs no model. It runs the tools in a fixed order and samples rather than judges: impact, clearance, a PageRank ranking of the first impact page (up to 40 sections), then at most 3 example runs before and after the pull request. It writes no verdicts. ```bash npx -y staleproof check https://github.com/FalkorDB/falkordb-py/pull/262 --no-agent ``` The first check of a pull request reads its metadata from GitHub and diffs its two commits into symbol-level changes. The first example run also builds virtualenvs and starts the scratch FalkorDB, so it takes longer than later ones. ### What the output shows From top to bottom: 1. **The pull request**: its title, URL, state, and the two commits compared (`before..after`). For a merged pull request, `before` is the first parent of the merge commit and `after` is the merge commit. 2. **Changed symbols**: each changed function, method or class with the kind of change (`removed`, `renamed`, `signature`, `modified`, `added`, `export-removed`). #262 changed one function, `Is_Cluster`. 3. **Impacted doc sections**: each section with a severity label (`BREAKING` for removed or renamed symbols and removed exports, `SIGNATURE` for signature changes, `BEHAVIOR` for body changes), its repository, the evidence path (`path: Is_Cluster ← FalkorDB.__init__ ← …`), the number of hops, the confidence (`exact` when every edge was resolved through imports and types, `by-name` or `late` otherwise), how many examples it has, and a link to the section on GitHub at the indexed commit. Sections are ordered by the PageRank of their page in the docs' own link graph, most central first. Only the first 40 sections are listed; a line such as `… 3 more` says how many were left out. For #262 there is one: the README's "Asynchronous Example". 4. **Cleared**: changed symbols that reach no doc within 3 hops, each with its raw text-match count in the docs and the `git grep` command to check it. `Is_Cluster` reaches a doc, so nothing is cleared for #262. 5. **Example runs**: for at most 3 examples on first-page sections whose path is `exact` (a sample, not every example), a `PASS`, `FAIL`, `TIMEOUT`, `SKIPPED` or `ERROR` line for each side with the commits installed, and the traceback of a failure. On the graph behind [Facts from the graph](FACTS.md), the README's Asynchronous Example failed before #262 with `TypeError: Redis.__init__() got an unexpected keyword argument 'himport_registry'` and passed after it: the pull request fixed a broken example. Your run installs today's third-party dependencies, so it can differ. 6. **The summary and the check id**: one paragraph, then `Stored as chk-…`. The check, its full trace and every run are stored in the graph. A whole-word grep for `Is_Cluster` finds no doc section: the README example calls the constructor, which calls the changed function. That is the case the graph exists for. ## 4. Check it with the agent In agent mode a model chooses which tools to call, reads each impacted section, runs examples, checks maintainer decisions, and records a verdict or proposes an edit for every impacted section, on every impact page. ### Claude ```bash export ANTHROPIC_API_KEY='' npx -y staleproof check https://github.com/FalkorDB/falkordb-py/pull/262 ``` The default model is `claude-sonnet-5-5` with effort `medium`; change them with `--model` and `--effort` (or `STALEPROOF_MODEL` and `STALEPROOF_EFFORT`). An `ant auth login` profile works instead of the key. The Claude path is tested against a local fake API only. ### DeepSeek or another Anthropic-compatible provider ```bash export ANTHROPIC_BASE_URL=https://api.deepseek.com/anthropic export ANTHROPIC_API_KEY='' export STALEPROOF_MODEL=deepseek-v4-pro npx -y staleproof check https://github.com/FalkorDB/falkordb-py/pull/262 ``` Any `ANTHROPIC_BASE_URL` whose host is not `api.anthropic.com` switches to compatible mode, which sends only the portable request fields. `STALEPROOF_MODEL` is required there: a `claude-*` model name is refused, because the provider would answer with its own model under that name. This is the path the live demo and the published agent runs use. The output is the trace: one line per tool call (`→ impact(pr_url=…)`), one per result (`← ` with the Cypher queries it ran and their time), the model's summarised reasoning, and a final line with the status (`Done`, `Incomplete`, `refused` or `error`), turns, tool calls, verdicts and tokens, followed by the model's summary. The last line names the patch command and the stored check id. The check is `Done` only when every impacted section (all 43 for #276, not only the first page of 40) has a verdict or an edit; if the model stops early it is reminded of the open sections up to twice, then the check is recorded as `Incomplete` with those sections listed. The tool order differs from one pull request to the next; for #262 the published run called `impact → read_section → decisions_for → run_example → record_verdict`. A large pull request takes longer: a 43-section check takes about 6 minutes. Without credentials, agent mode stops with an error that names the variables to set. Graph mode always works. ## 5. Run one example before and after Every code example has an id, `owner/repo:path:index`, where `index` is the block's position among all fenced blocks in the file (0-based). Impact output and `--json` list them. ```bash npx -y staleproof run "FalkorDB/falkordb-py:README.md:4" --pr https://github.com/FalkorDB/falkordb-py/pull/262 ``` It prints the example id and URL, one line per side (`FAIL before …`, `PASS after …`) with the commits installed, any traceback, and a one-line verdict such as `Fails at and passes at : the PR fixes this example.` If the graph already holds an identical run (same program, same commits), it is answered from the graph and marked as recorded. To execute it again and replace the stored run: ```bash npx -y staleproof run "FalkorDB/falkordb-py:README.md:4" --pr https://github.com/FalkorDB/falkordb-py/pull/262 --fresh ``` ## 6. Get the patch ```bash npx -y staleproof patch https://github.com/FalkorDB/falkordb-py/pull/262 ``` The patch is built from the edits the agent proposed (`stale` verdicts not rejected by a maintainer). It prints a unified diff that `git apply` accepts in each docs repository, then a pull request title and description with the evidence path, the example runs and the cited decisions for every edited section, and a list of the sections checked and left unchanged. Graph mode writes no verdicts, so after a graph-mode check alone the diff is empty and the command prints `(no proposed edits)`. The same is true when the agent concludes that nothing needs editing, which is what has happened on FalkorDB's repositories so far: the pull requests checked there are fixes, and every verdict recorded on #262, #267 and #276 is `ok`. A re-run on #267 considered fixing the GRAPH.EXPLAIN example, which fails before and after the pull request, and declined, because the page assumes a demo graph that already exists; it recommended a separate fix. Proposed edits and the patch built from them are exercised by the test suite on a fixture organisation, not yet on these repositories. The file is read at the commit the graph indexed: from the local clone in `~/.staleproof/repos` when there is one, otherwise from `raw.githubusercontent.com`. A section whose text changed since indexing, or whose file is gone, is listed as not applied instead of being patched. After an agent check that proposed edits, write the diff and the description to files, or open the fix as a pull request in your fork (needs `GITHUB_TOKEN`): ```bash npx -y staleproof patch --out docs-fix.diff npx -y staleproof patch --open-pr ``` ## 7. No pull request yet? Two commands work before anything is pushed, and one with no change at all. In a checkout of a repository the graph holds (or will index on first use): ```bash npx -y staleproof docs-for Graph.query --repo FalkorDB/falkordb-py # which docs depend on it, before you change it npx -y staleproof check # which docs your uncommitted and unpushed changes make false npx -y staleproof audit FalkorDB/falkordb-py FalkorDB/GraphRAG-SDK FalkorDB/docs # how true the docs are right now ``` `check` with no URL lists the impacted sections and the examples predicted to break, each with its line and the reason, without running anything; add `--fail-on-predicted` to use it as a pre-push hook ([CLI](cli.md#check-without-a-pull-request)). `audit` scores the docs from 0 to 100 and lists every example line that no longer fits the code ([Docs audit](audit.md)). ## 8. Open the web UI ```bash npx -y staleproof serve ``` Open `http://127.0.0.1:8080`. Paste a pull request URL, pick agent mode (when the server has model credentials) or graph mode, and watch the trace stream in. The startup line on stderr says which graph is served and whether agent mode is on. Use `--port` and `--host` to change the address. The same process serves the [HTTP API](http-api.md) and MCP over HTTP at `/api/mcp`. With `--hosted`, it indexes any public repository on demand into a graph of its own, as staleproof.dev does. ## 9. Add it to Claude Code ```bash claude mcp add staleproof -- npx -y staleproof mcp ``` Claude Code now has the fourteen tools, on the graph `falkordb` you indexed (the stdio server uses `STALEPROOF_ORG` when set, else the graph of the repository's owner, like the CLI). Ask it, for example: "What docs can https://github.com/FalkorDB/falkordb-py/pull/267 make false? Show the evidence path for each." It answers with `check_pull_request`, or step by step with `impact`, `read_section` and `run_example`; before a pull request, it can call `docs_for_symbol` and `check_changes` on what it is editing. To skip the local setup, add the hosted server instead: `claude mcp add --transport http staleproof https://staleproof.dev/api/mcp` (no example runs there). [MCP server](mcp.md) covers other clients and a full session. ## Index your own repositories ### What works - **Code repositories in Python.** Every `.py` file is parsed, except tests (`test/`, `tests/`, `testing/`, `test_*.py`, `*_test.py`, `conftest.py`), `setup.py`, and files under `scripts/`, `benchmarks/` or `docs/`. Python files under `example/`, `examples/`, `sample/`, `samples/`, `tutorial/`, `tutorials/`, `notebook/` or `notebooks/` become runnable example scripts. - **TypeScript and JavaScript libraries.** The source of every published npm package in the repository (a `package.json` with a `name`, not `private`; monorepos too), except tests and build tooling. JavaScript and TypeScript examples in the docs, and `examples/*.js` or `*.ts` scripts, are linked to it and checked for names that do not exist; they are not run. - **Docs in Markdown, MDX or reStructuredText.** Every `.md`, `.mdx` and `.rst` file in any indexed repository, except housekeeping files (`CHANGELOG`, `CONTRIBUTING`, `LICENSE` and similar). Sphinx docs work: code blocks, doctests, `:func:`/`:meth:`/`:class:` roles and `:doc:`/`:ref:` links. A docs site can be its own repository with no code at all. Notebooks are not read. - **Examples you do not want run** can be marked in the docs (``) or in a `.staleproof.toml` ([Configuration](configuration.md#repository-configuration)). - **GitHub repositories, public or private.** The CLI, a server you run and the GitHub Action read private repositories with `GITHUB_TOKEN` (or `GH_TOKEN`) set to a token that can read them; the Action's default token reads its own repository. The hosted site, staleproof.dev, indexes public repositories only. [Indexing](indexing.md) has the full rules. ### Index everything in one command Each organisation gets its own graph, named after the owner of the first repository (`acme` here) unless `--org` or `STALEPROOF_ORG` names it. Pass all its repositories to one `index` command: ```bash npx -y staleproof index acme/acme-client acme/acme-sdk acme/acme-docs ``` Calls and doc links resolve across repositories only among the repositories of one `index` run. If you index a repository on its own, or re-index it alone later, its calls into the other repositories and its docs' links into their code are re-resolved against that repository only, and those cross-repository edges disappear. Re-index with the same full command. To index a specific branch, tag or commit of one repository, use `--ref` (repeatable). Use `origin/`, a tag or a SHA rather than a bare branch name: ```bash npx -y staleproof --org acme index acme/acme-client acme/acme-docs --ref acme/acme-docs=origin/next ``` ### Check your pull requests `check` uses the graph of the pull request's owner, so a pull request in `acme/acme-client` reads `acme`. If that graph does not have the repository yet, `check` indexes it first, together with the repositories you pass to `--with`; `--no-index` makes it stop with ` is not indexed in the "" graph` instead. ```bash npx -y staleproof check https://github.com/acme/acme-client/pull/123 --no-agent ``` When one graph holds repositories of several owners (a docs site under another account, say), pass the same `--org` to every command, or set `STALEPROOF_ORG`, so they all use that graph. ## Next steps - Compare every way to install it: [Install](install.md). - Comment on every pull request automatically: [GitHub Action](github-action.md). - Learn every command and option: [CLI](cli.md). - Find the scenario closest to yours: [Use cases](use-cases.md). - Record decisions so intentional docs stop being re-flagged: [Decisions and memory](memory.md). - Run it for a team: [Deploy](DEPLOY.md). ## Related - [CLI](cli.md) - [Use cases](use-cases.md) - [MCP server](mcp.md) - [Configuration](configuration.md) - [Troubleshooting and FAQ](troubleshooting.md) --- # Install Four ways to use Staleproof, from nothing installed to one command on your machine: the website, the GitHub Action, an MCP server for your AI assistant, and the CLI. Each section says what it needs, what it does, and what it does not do. The site's install page, [staleproof.dev/#/install](https://staleproof.dev/#/install), has the one-click buttons for the Action and the MCP clients. | Way | Needs | Runs examples | Repositories | |---|---|---|---| | [Website](#website-no-install) | A browser | No | Public | | [GitHub Action](#github-action-one-click) | A workflow file in the repository | Yes, before and after each pull request | Public and private | | [MCP, hosted](#mcp-hosted-no-install) | An MCP client | No | Public | | [MCP, local](#mcp-local) | Node.js 20+, git, uv or `python3` | Yes | Public and private (with `GITHUB_TOKEN`) | | [CLI](#cli-one-command) | Node.js 20+, git, uv or `python3` | Yes | Public and private (with `GITHUB_TOKEN`) | ## Website: no install Open [staleproof.dev](https://staleproof.dev) and paste the URL of a pull request in a public GitHub repository, or open `https://staleproof.dev/#/try?pr=`, which starts the check directly. - **What it does.** The first check of a repository downloads it and indexes it on the fly into the workspace graph of its owner (`gh-`), together with any repositories you name next to it (a docs site, an SDK; up to 4 per workspace), then checks the pull request in graph mode and streams the trace: the impacted doc sections with their evidence paths, the cleared symbols and the ranking. Agent mode is offered when the server has a model key, within a daily budget. - **Audit a repository.** [staleproof.dev/#/audit](https://staleproof.dev/#/audit) audits a public repository with no pull request: every docs example checked against the code it calls, API coverage, a 0-100 health score and the README badge ([Docs audit](audit.md)). - **What it does not do.** It does not run examples: the hosted service has no sandbox. For FalkorDB's own repositories it shows the runs recorded earlier. It reads public repositories only, and leaves nothing on your repository: no comment, no pull request. ## GitHub Action: one click On [staleproof.dev/#/install](https://staleproof.dev/#/install), type the repository and click **Add to owner/repo**. GitHub opens its "new file" page with `.github/workflows/staleproof.yml` filled in; click **Commit changes**. Options on the page add a docs repository to index with it (`also-index`), agent mode (needs an `ANTHROPIC_API_KEY` repository secret), failing the check when an example breaks (`fail-on: breaks`), and a weekly audit of the whole docs kept in one issue. Or copy [examples/staleproof.yml](../examples/staleproof.yml) yourself: ```yaml name: Staleproof on: pull_request: permissions: contents: read pull-requests: write jobs: docs: runs-on: ubuntu-latest steps: - uses: rajkaria/staleproof@v1 ``` - **What it does.** On every pull request the job starts a FalkorDB, indexes the repository, checks the change (including the examples predicted to break), runs up to 3 affected examples at the commit before the pull request and at the pull request, and posts one comment (kept up to date) with the report; the report also goes to the job summary. On a schedule it audits the docs instead ([examples/staleproof-audit.yml](../examples/staleproof-audit.yml)). Private repositories work with the workflow's default token. - **What it does not do.** It changes no files, and opens no pull request unless you set `open-docs-pr` in agent mode. Without `falkordb-url` it keeps nothing between runs. Pull requests from forks get a read-only token: the report lands in the job summary, not as a comment. Inputs, outputs and the comment format: [GitHub Action](github-action.md). ## MCP, hosted: no install The hosted MCP server speaks Streamable HTTP at `https://staleproof.dev/api/mcp`. Add it to your client: | Client | How | |---|---| | Claude Code | `claude mcp add --transport http staleproof https://staleproof.dev/api/mcp` | | Cursor | The **Add to Cursor** button on the install page, or `{"mcpServers": {"staleproof": {"url": "https://staleproof.dev/api/mcp"}}}` in `~/.cursor/mcp.json` | | VS Code | The **Add to VS Code** button on the install page, or `code --add-mcp '{"name":"staleproof","type":"http","url":"https://staleproof.dev/api/mcp"}'` | | Claude Desktop and claude.ai | Add a custom connector with the URL `https://staleproof.dev/api/mcp` | - **What it does.** `check_pull_request` returns the whole report for a pull request in one call; the graph tools behind it (`impact`, `read_section`, `decisions_for`, `rank_sections` and the rest) answer follow-up questions. Before a pull request, `docs_for_symbol` lists the docs that depend on a function and `check_changes` checks edited files; `audit_docs` audits a repository's docs. Any public repository is indexed on first use. - **What it does not do.** `run_example` is not offered and no example is executed. Private repositories cannot be read. ## MCP, local ```bash claude mcp add staleproof -- npx -y staleproof mcp ``` - **What it does.** Serves all fourteen tools over stdio, `run_example` included, from your machine. With no `FALKORDB_URL` it starts a local FalkorDB; the tools index the repository they are about the first time they see it. `check_changes` can read the working tree of your checkout directly. Private repositories work with `GITHUB_TOKEN` in the server's environment (`-e GITHUB_TOKEN=`). - **What it does not do.** It writes nothing to GitHub. The first start installs falkordblite, which takes a minute. Other clients, the tools and security notes: [MCP server](mcp.md). ## CLI: one command ```bash npx -y staleproof check https://github.com///pull/ --no-agent ``` - **What it needs.** Node.js 20 or newer, git, and uv (recommended) or `python3` 3.12 or newer for the local FalkorDB and the example runs. - **What it does.** With no `FALKORDB_URL` it starts a local FalkorDB (falkordblite, no Docker), indexes the pull request's repository at its default branch head into a graph named after the owner, checks the pull request, and runs up to 3 affected examples before and after it in a sandbox. Later checks of the same repository reuse the graph. Add `--with ` for docs that live in another repository, and set `GITHUB_TOKEN` for a private repository. - **Agent mode.** Without `--no-agent`, `check` runs the agent when model credentials are set: `ANTHROPIC_API_KEY`, or `ANTHROPIC_BASE_URL`, `ANTHROPIC_API_KEY` and `STALEPROOF_MODEL` for an Anthropic-compatible provider. Without them it prints a note and runs graph mode. - **Before a pull request, and with none.** `npx -y staleproof check` with no URL checks your uncommitted and unpushed changes (`--fail-on-predicted` for a pre-push hook); `npx -y staleproof audit ` audits the docs as they are. - **What it does not do.** It opens no pull request unless you run `patch --open-pr`. The full walkthrough: [Quickstart](getting-started.md). Every command: [CLI](cli.md). ## Related - [Quickstart](getting-started.md) - [GitHub Action](github-action.md) - [MCP server](mcp.md) - [CLI](cli.md) - [Deploy](DEPLOY.md): run the hosted service yourself --- # Use cases Nine ways teams use `staleproof`, each with the situation it fits, what it does there, the setup, what you get and its limits. Read the one closest to your team; every setup links to the reference page with the full options. The running examples are FalkorDB's client library (`FalkorDB/falkordb-py`), the SDK built on it (`FalkorDB/GraphRAG-SDK`) and its docs site (`FalkorDB/docs`). ## SDK and client-library teams with a separate docs site **The situation.** Your client library, the SDK built on it and the docs site live in different repositories. A change in the client can make a docs page false without naming anything the page mentions: the page calls an SDK method, the SDK method calls the client, and the client function changed. Reviewers of the client pull request never see that page. **What it does.** One graph spans all your repositories. Calls are resolved across them through imports, types and re-exports, and docs are linked to the code their prose mentions and their examples call. For a pull request in the client, [impact](tools.md#impact) follows the call edges into the SDK and on to the docs site, and gives the evidence path for every section it reaches. When it runs an example, it installs every repository the example needs from source at the right commits, using the dependencies the repositories declare on each other. **Setup.** Index the repositories together, then check pull requests from any of them: ```bash npx -y staleproof index FalkorDB/falkordb-py FalkorDB/GraphRAG-SDK FalkorDB/docs npx -y staleproof check https://github.com/FalkorDB/falkordb-py/pull/276 --no-agent ``` To get a comment on every pull request in the client repository, add the [GitHub Action](github-action.md) with the other repositories in `also-index`, so the job indexes them together. **What you get.** On [falkordb-py#276](https://github.com/FalkorDB/falkordb-py/pull/276), 4 changed symbols reach 43 doc sections, 41 of them in the other two repositories, through paths such as: ```text AsyncGraph.query ← FalkorDBConnection.query (GraphRAG-SDK) ← GraphStore.delete_document_chunks ← GraphRAG.delete_document ← GraphRAG-SDK README ``` A whole-word grep for the changed names finds none of those 43 sections and matches 8 others. Of the 21 examples tried before and after #276, 10 pass both times, 6 fail both times (already broken, not by this pull request), 3 cannot be installed together at those commits, and 2 are skipped (placeholders to fill in; not Python). The 3 that cannot be installed show the reason: `dependency conflict: graphrag-sdk==1.4.0 requires falkordb[hiredis]>=1.8.0,<2`, because the SDK pins the client below the version under test. All figures are from [Facts from the graph](FACTS.md). **Limits.** - Code is read as Python, or TypeScript and JavaScript in published npm packages; only Python examples run. - Calls and doc links cross repositories only between repositories indexed in the same `index` command. Indexing or re-indexing one repository alone drops its cross-repository edges. - Example runs install today's third-party dependencies, and version skew between repositories (an SDK pinning an older client) is recorded as an install error, not a result. - Edges resolved by name rather than through types are labelled `by-name`; a path made of them is weaker evidence than an `exact` one. ## DevRel and docs teams after a release **The situation.** A release merged dozens of pull requests across your repositories. You need to know which pages are now wrong, fix the most-read ones first, and send the fixes as reviewable pull requests. **What it does.** Staleproof checks each merged pull request. In agent mode a model judges every impacted section and proposes replacement text for the ones that are wrong; each proposal is refused unless the graph has a path from the change to the section. [rank_sections](tools.md#rank_sections) orders sections by the PageRank of their page in your docs' own link graph, so the most central page comes first. `patch` turns a pull request's proposed edits into one diff across every docs repository it touches, with the evidence path, the example runs and any cited decisions for each edit, and `--open-pr` opens it as a pull request in your fork. **Setup.** List the release's merged pull requests (here with the GitHub CLI), check each one with the agent, then build and open the patches: ```bash prs=$(gh pr list --repo FalkorDB/falkordb-py --state merged --search "merged:>=2026-10-01" --json url --jq '.[].url') for pr in $prs; do npx -y staleproof check "$pr"; done for pr in $prs; do npx -y staleproof patch "$pr" --out "docs-$(basename "$pr").diff"; done ``` Review the diffs, then open the ones you want (needs `GITHUB_TOKEN`): ```bash npx -y staleproof patch https://github.com/FalkorDB/falkordb-py/pull/ --open-pr ``` `--open-pr` forks each docs repository into your account, creates a branch named after the code pull request (for example `staleproof/FalkorDB-falkordb-py-276`) at the commit the graph indexed, commits the edited files and opens the pull request inside your fork. Add `--upstream` to open it against the original repository instead. For ranking, graph mode prints a pull request's sections in PageRank order, and the GitHub Action comment uses the same order. To rank a whole release at once, pass the section ids from several checks to `rank_sections` through an [MCP client](mcp.md) or the [library](library.md). **What you get.** Per pull request: a `git apply`-able diff, a pull request description with the reason and evidence path for each edited section, the sections that were checked and left unchanged with their verdicts, and the sections that could not be applied. **Limits.** - There is no release-level command: you check and patch one pull request at a time, and each gives its own patch and its own docs pull request per docs repository. - Only agent mode proposes edits. Graph mode writes no verdicts, so its patch is empty. - No edit has been proposed on FalkorDB's repositories yet: the pull requests checked there are fixes, and every verdict on #262, #267 and #276 is `ok`. Edits and the patch are exercised by the test suite on a fixture organisation. - An edit is not applied when its section's text changed since the docs repository was indexed, or when the file is gone. Sections longer than 8,000 characters always conflict, because the graph stores only their first 8,000 characters. - `--open-pr` always creates or reuses a fork, even with `--upstream`. - The ranking in graph-mode output and in the comment covers the first impact page (40 sections). ## Open-source maintainers reviewing pull requests **The situation.** Contributors send pull requests that change behaviour. You want to know, during review, which docs pages a change can falsify, without paying for a model on every pull request. **What it does.** The [GitHub Action](github-action.md) runs on every pull request and posts one comment, updated in place on each run. It starts a FalkorDB in the job, indexes the repository, and runs up to 3 affected examples at the commit before the pull request and at the pull request. The comment leads with what broke: the examples run before and after, then the impacted sections with the change type, repository, evidence path and hops, the changed symbols no doc reaches (each with a `git grep` command to check the claim), and any verdicts. Without a model key it runs graph mode; with one, the agent judges each section and the comment adds its verdicts and summary. **Setup.** Click **Add to owner/repo** on [staleproof.dev/#/install](https://staleproof.dev/#/install) and commit the file GitHub opens, or add the workflow as `.github/workflows/staleproof.yml` yourself. No secrets, no database: ```yaml name: Staleproof on: pull_request: permissions: contents: read pull-requests: write jobs: docs: runs-on: ubuntu-latest steps: - uses: rajkaria/staleproof@v1 with: also-index: FalkorDB/docs # docs in another repository (optional) fail-on: breaks # fail the check when an example passes before and fails after (optional) ``` **What you get.** A comment that starts with the hidden marker ``, titled with what broke (`this pull request breaks 1 docs example`, or how many sections it can make false), the examples table, the docs it can make false with the path for each, and the graph name and commits compared at the end. The same report goes to the job summary. [GitHub Action](github-action.md#the-comment) shows the full format. **Limits.** - The action has run end to end on a private test repository with the one-click workflow (no inputs). - Graph mode samples: it runs at most 3 examples (the `runs` input), from exactly linked sections on the first impact page, and writes no verdicts. - With the default FalkorDB nothing carries over between runs: no decisions, verdict history or recorded runs. Set `falkordb-url` to keep them; an open pull request is then diffed again whenever new commits are pushed. - Some rows overstate the change: moving a file reports its definitions as removed, and a module that stops importing a name from its own package reports a removed export, both labelled breaking. - Pull requests from forks get a read-only token and no secrets: the check runs in graph mode and its report goes to the job summary, but no comment is posted. - Both modes install and run code from the pull request on the runner (the examples), with a guarded environment; see the [security notes](github-action.md#security-notes). ## A weekly docs audit **The situation.** Docs drift between releases too, not only in the pull request that changed the code: an SDK renames a parameter, a docs site in another repository keeps the old call, and nobody runs the page again. You want a standing report of what is wrong now, and a number that says whether it is getting better. **What it does.** The [docs audit](audit.md) checks every docs example against the code it calls, across repositories, without running it: unknown keywords, wrong argument counts, names that no longer exist. It adds the runs of up to 20 examples at the default branch and the share of the public API no doc covers, and scores it from 0 to 100. On a schedule the GitHub Action keeps one issue up to date with the report: opened when an example no longer fits or fails, updated on every run, closed once all is clear. **Setup.** Add [examples/staleproof-audit.yml](../examples/staleproof-audit.yml) as `.github/workflows/staleproof-audit.yml` (or tick the weekly audit option on [staleproof.dev/#/install](https://staleproof.dev/#/install)): ```yaml name: Staleproof docs audit on: schedule: - cron: '0 6 * * 1' workflow_dispatch: permissions: contents: read issues: write jobs: audit: runs-on: ubuntu-latest steps: - uses: rajkaria/staleproof@v1 with: also-index: FalkorDB/docs # docs in another repository (optional) ``` To see a first report without CI, audit on [staleproof.dev/#/audit](https://staleproof.dev/#/audit) or run `npx -y staleproof audit `. Put the badge the report prints in your README. **What you get.** Lines like these, each linked to the line in the docs (all found on the current docs of real projects; [more](audit.md#what-it-finds-on-real-repositories)): - the FalkorDB docs page `commands/graph.constraint-create.mdx` calls `client.create_constraint`, which `FalkorDB` does not have; - GraphRAG-SDK's `docs/api-reference.mdx` documents `VectorStore(index_name=..., similarity_function=...)`, parameters the constructor no longer has; - Flask's `docs/views.rst` calls `UserList.as_view()` without its required `name`. Then the failing runs, the most used undocumented API, and the score with each deduction. **Limits.** - Static checks judge only calls resolved through imports and types; a clean audit is not proof that every example works. The runs cover the rest, for Python. - TypeScript and JavaScript examples get name checks only, no argument checks, and are never run. - Coverage counts a symbol as documented when any doc names or calls it, however briefly. - The issue tracks examples that no longer fit or fail; low coverage lowers the score but opens no issue on its own. ## Developers, before they push **The situation.** You are changing a function in a library whose docs, maybe in another repository, call it. You want to know which examples your change breaks before you push, not from a reviewer or a reader later. **What it does.** `staleproof check` with no pull request URL diffs your checkout's uncommitted and unpushed changes against its merge base, walks the graph from the changed symbols to every doc that reaches them, and predicts which examples break: a removed or renamed name, an import that stops existing, a call the new signature rejects. Nothing runs and nothing is written to the graph. `staleproof docs-for ` answers the question before you start: which docs depend on this function. **Setup.** Once, in the checkout (the first run indexes the repository; add `--with ` when the docs live elsewhere): ```bash npx -y staleproof docs-for Graph.query --repo FalkorDB/falkordb-py npx -y staleproof check ``` As a pre-push hook that stops the push when an example is predicted to break: ```bash printf '#!/bin/sh\nnpx -y staleproof check --fail-on-predicted\n' > .git/hooks/pre-push && chmod +x .git/hooks/pre-push ``` **What you get.** The impacted sections with their evidence paths, then the examples predicted to break, each with its line and the reason, for example: ```text Predicted to break (1): the call no longer fits the changed code ✗ docs/getting-started.mdx:0 line 18 g.query `Graph.query` requires `timeout`, which the example does not pass ``` (from the test suite's fixture organisation, after adding a required `timeout` to `Graph.query`), and the impacted lines that were already out of date. [CLI](cli.md#check-without-a-pull-request) has the full output. **Limits.** - It is static: it predicts breaks, it does not run examples. Open the pull request for the before/after runs. - The docs' repository must be in the graph with the code; index them together (`--with`) the first time. - Changes the static check cannot judge (a decorated function, a call with `**kwargs`) are listed as impact without a prediction. ## AI coding agents over MCP **The situation.** You change code with an AI coding agent such as Claude Code, and want the same session to find and fix the docs the change affects before a human reviews it. **What it does.** The MCP server gives the agent the Staleproof tools: `check_pull_request` returns the whole report for a pull request in one call, and the graph tools let it read the sections, run the examples before and after, check maintainer decisions, record verdicts or propose edits, and get the docs patch as a diff to apply locally. Every result includes the Cypher queries it ran. **Setup.** With nothing installed, the hosted server (public repositories, no example runs): ```bash claude mcp add --transport http staleproof https://staleproof.dev/api/mcp ``` Or the local server over stdio, which runs examples and reads private repositories with `GITHUB_TOKEN`; it starts a local FalkorDB and indexes each pull request's repository on first use: ```bash claude mcp add staleproof -- npx -y staleproof mcp ``` Cursor, VS Code and Claude Desktop have one-click buttons or a URL to paste on [staleproof.dev/#/install](https://staleproof.dev/#/install). Prompts to try, after pushing your branch and opening the pull request: - "What docs can https://github.com/FalkorDB/falkordb-py/pull/267 falsify? Show the evidence path for each section." - "Run the examples in those sections before and after that pull request and tell me which ones it breaks." - "Which of those sections is most central in the docs?" - "Check the maintainer decisions for each section, then record a verdict for the ones that still hold and propose an edit for the ones that don't." - "Give me the docs patch for that pull request and apply it to my checkout of FalkorDB/docs." [MCP server](mcp.md) walks through a full session. **What you get.** Answers grounded in the graph, with the evidence path and Cypher behind each, verdicts and proposed edits stored on the graph for the next check, and a diff you can apply. **Limits.** - `check_pull_request` and the graph tools take a GitHub pull request URL; for edits not pushed yet, see [the next use case](#coding-agents-before-the-pull-request). - The hosted server reads public repositories only and does not offer `run_example`. - First-use indexing indexes the pull request's repository alone; index a docs site that lives elsewhere together with it first (`staleproof index `). - `propose_edit` and `record_verdict` need `pr_url` over MCP; there is no pull request "under check" to default to. - One local MCP server serves the graph named by `STALEPROOF_ORG`, else the graph of the owner of the repository each call names. Register a second server with another `STALEPROOF_ORG` to pin another graph. - `run_example` installs repositories and runs docs code on the machine running the local MCP server. See the security notes in [MCP server](mcp.md). ## Coding agents, before the pull request **The situation.** An AI coding agent edits your library. Docs should change in the same commit as the code, but the agent does not know which pages call the function it is changing, and nothing tells it until a human reviews the pull request. **What it does.** Two MCP tools work before there is a pull request. `docs_for_symbol` takes a function, class or method and returns every doc section and example that depends on it, with the evidence path, so the agent knows what to update. `check_changes` takes the files it edited (path and new content) and returns the docs they can make false and the examples predicted to break, compared with the commit the graph indexed; on a local server it can read the checkout's working tree instead. Neither writes anything. **Setup.** Add the MCP server ([hosted or local](mcp.md)), then put these rules in the repository's `AGENTS.md` or `CLAUDE.md` so the agent calls the tools unasked (the install page has them to copy): ```markdown ## Docs stay true to code (Staleproof MCP) - Before changing a public function, class or method, call docs_for_symbol with its name: update every doc section and example it lists in the same change. - Before committing, call check_changes with the files you edited: fix every example it predicts to break (its call no longer fits the new signature). ``` **What you get.** For a change that adds a required `timeout` to `Graph.query` in the fixture organisation: `docs_for_symbol` lists the 3 doc sections that reach it, and `check_changes` names the one example line that breaks (`` `Graph.query` requires `timeout`, which the example does not pass ``) with a link to it. [MCP server](mcp.md#before-a-pull-request) shows the calls. **Limits.** - `check_changes` is static: it predicts breaks and runs nothing. - The hosted server cannot read your disk: send `files` with `repo`. It reads public repositories only. - The rules are instructions to the agent, not enforcement; pair them with the [pre-push hook](#developers-before-they-push) or the GitHub Action. ## Platform teams with many repositories or organisations **The situation.** You run docs tooling for several product lines or several organisations, and want one service rather than one setup per team. **What it does.** Each organisation gets its own graph on the same FalkorDB server; the graph name is the organisation slug. The CLI defaults to the graph of the repository's owner; `--org` or `STALEPROOF_ORG` names another, and the MCP server and the HTTP server use `STALEPROOF_ORG`, else `falkordb`. `serve --hosted` goes further and gives every repository owner a workspace graph of its own on demand, as staleproof.dev does. A second graph built from [encode/httpx](https://github.com/encode/httpx) sits next to `falkordb` on the same server: one repository, 544 symbols, 182 doc sections and 194 code examples ([its facts](FACTS-encode.md)). **Setup.** ```bash npx -y staleproof index encode/httpx npx -y staleproof check https://github.com/encode/httpx/pull/ --no-agent ``` Both use the graph `encode`, the repository's owner. Serve the [HTTP API](http-api.md) once per organisation (without `--hosted` the server answers for one graph), on separate ports: ```bash STALEPROOF_ORG=falkordb npx -y staleproof serve --port 8080 STALEPROOF_ORG=encode npx -y staleproof serve --port 8081 ``` Or embed the [library](library.md), which can open any graph on the server from one process: ```ts import { GraphStore, loadConfig, runGraphCheck } from 'staleproof'; const config = loadConfig(); const store = await GraphStore.connect(config.falkordbUrl); try { console.log(await store.listGraphs()); const graph = store.org('encode'); const result = await runGraphCheck({ graph, config }, 'https://github.com/encode/httpx/pull/', { maxRuns: 0 }); console.log(result.status, result.summary); } finally { await store.close(); } ``` **What you get.** Isolated graphs per organisation on shared infrastructure, each with its own checks, verdicts, decisions and runs. **Limits.** - Graphs do not link to each other: a dependency between organisations is not followed. - One `serve` process answers for one organisation (or, with `--hosted`, for every owner from one FalkorDB server, within the [hosted limits](DEPLOY.md#limits)) and runs one check at a time; further checks queue. - The HTTP API has no authentication. Its POST endpoints (checks, indexing, decisions, verdict reviews) are rate-limited per client, 30 an hour by default, and nothing more; put it behind your own access control. - GitHub repositories only. Private ones need `GITHUB_TOKEN`; a hosted server indexes whatever its token can read, so give it a token limited to public repositories. ## Recording maintainer decisions so docs stop being re-flagged **The situation.** Some docs are intentionally as they are: the async client keeps an old keyword argument until 2.0, or a page documents the previous release on purpose. Without memory, every check flags them again. **What it does.** A decision is a sentence, its source and author, and the doc sections or symbols it governs. Before judging a section, the agent calls [decisions_for](tools.md#decisions_for), which finds decisions that govern the section directly or govern a symbol within 2 call hops of what the section mentions or its examples call. A decision on one function therefore covers the docs of the code that calls it. The agent then records a `governed` verdict that must cite the decision, instead of proposing an edit. A newer decision can supersede an older one; lookups resolve to the active successor. Maintainers can also accept or reject each verdict, and the agent sees earlier verdicts and their review status on the next check. **Setup.** Find the ids to govern: section ids come from impact (`check --json`), and a symbol id is `/:`, the qualified name shown in evidence paths. Then record the decision: ```bash npx -y staleproof decide "The async client keeps positional kwargs until 2.0" \ --governs "FalkorDB/falkordb-py:falkordb.asyncio.falkordb.FalkorDB.__init__" \ --source https://github.com/FalkorDB/falkordb-py/issues/ --author "" ``` It prints `Recorded decision dec-… governing 1 node(s).` To replace it later: ```bash npx -y staleproof decide "The async client drops positional kwargs in 2.0" \ --governs "FalkorDB/falkordb-py:falkordb.asyncio.falkordb.FalkorDB.__init__" \ --supersedes dec- ``` Accept or reject a verdict the agent wrote (rejected edits are left out of the patch): ```bash npx -y staleproof verdict reject ver- --by "" ``` **What you get.** Fewer repeat flags in agent mode, a `governed` verdict that names the decision, and a record of who decided what and where. **Limits.** - Decisions inform the agent's judgement only. Impact still lists a governed section, and graph mode and the graph-mode comment do not consult decisions. - Citing the decision is the model's job, guided by its instructions; the graph enforces only that a `governed` verdict cites at least one decision id. - `decide` refuses when none of the `--governs` ids exists in the graph. An unknown `--supersedes` id is ignored without an error. - Decisions come from people: the agent is never given the tool that writes them. ## Related - [Quickstart](getting-started.md) - [CLI](cli.md) - [GitHub Action](github-action.md) - [MCP server](mcp.md) - [Decisions and memory](memory.md) --- # Docs audit A docs audit answers "how true are these docs right now?" with no pull request: every call in a docs example checked against the code it reaches (in its own repository or another), the latest example runs at the indexed commits, and how much of the public API the docs name or call, summed up as a health score from 0 to 100. This page covers what the audit checks, how the score is computed, the CLI command, the web page, the README badge, the HTTP API and MCP tool, the GitHub Action's scheduled audit, how to read and act on the report, and its limits. It is for maintainers and docs teams who want a baseline before relying on pull request checks, or a weekly report after. ## Run it ```bash npx -y staleproof audit FalkorDB/falkordb-py ``` With no FalkorDB configured the CLI starts a local one, indexes the repository at its default branch head if the graph does not have it, and prints the report. Name every repository that shares the docs, so calls across them resolve: `staleproof audit acme/acme-client acme/acme-docs` indexes the two together on first use. With no repository at all, `staleproof audit` audits every repository in the graph. Other ways to run the same audit: the page [staleproof.dev/#/audit](https://staleproof.dev/#/audit) for any public repository, [`GET /api/audit`](http-api.md#get-apiaudit), the MCP tool [`audit_docs`](tools.md#audit_docs), and the [GitHub Action](github-action.md#audit-the-docs-on-a-schedule) on a schedule. ## What it checks ### Examples that no longer fit the code At index time every docs example is checked statically against the signatures and names its calls reach, and each line that does not fit becomes a `MISMATCH` edge in the graph ([Indexing](indexing.md#static-checks-of-examples) has the rules). The audit lists them, with the doc line, the section, the call as written and what is wrong: | Kind | Label in the report | Example detail | Languages | |---|---|---|---| | `unknown-keyword` | unknown keyword argument | `` `Client` has no parameter `hostname` `` | Python | | `too-many-arguments` | too many positional arguments | `` `Graph.query` takes at most 2 positional arguments; the example passes 3 `` | Python | | `missing-argument` | missing required argument | `` `Graph.query` requires `timeout`, which the example does not pass `` | Python | | `missing-import` | imports a name that does not exist | `` `miniclient` has no `connect` `` | Python, JavaScript, TypeScript | | `missing-attribute` | calls an attribute that does not exist | `` `Graph` has no attribute `read_only` `` | Python, JavaScript, TypeScript | The details above come from the test suite's fixture docs. Nothing runs to find them, so they cost no sandbox, work on the hosted service, and cover examples that could never run (an LLM key, a placeholder host). A check is only made when the source settles it: a call reached by name rather than through imports and types, a decorator that may rewrite the signature, a class with an external base or a `__getattr__`, or a `*args`/`**kwargs` splat leaves the line unjudged. ### Example runs The audit reads the latest recorded run of each example at the indexed commits (the `current` side; runs before and after a pull request are not counted), and only runs whose installed commits are the ones the graph holds now. `staleproof audit --run ` first runs up to `n` Python examples in the [sandbox](example-runner.md): those with a mismatch first, then the ones that call the most code. Examples marked `skip` or `expect-error` ([directives](indexing.md#example-directives)) are never run. Without `--run`, the audit uses only what earlier runs recorded. ### API coverage The public API is what a repository's packages export: - **Python**: names bound in a public package's `__init__.py` (or a top-level module) that resolve to the organisation's definitions, following `from .x import *` and imports from private modules (`httpx._api.get` counts as `httpx.get`), plus the public methods of exported classes and of their base classes in the organisation. - **TypeScript and JavaScript**: what each published package's entry file exports, followed through re-exports, plus the public methods of exported classes. A repository indexed before export information existed, or one whose package has a module-level `__getattr__` (lazy exports the source does not show), falls back to the Python naming rule: no part of the qualified name starts with `_`. Either way, symbols nested in functions and files under `tests/`, `scripts/`, `examples/`, `docs/`, `benchmarks/`, `tools/` and similar are left out. A public symbol is documented when a doc section names it in inline code or an example calls it (a `MENTIONS` or `USES` edge). A class also counts as documented when any of its methods is, so `g.query(...)` in an example documents `Graph`. Coverage is the documented share, to one decimal place. ### Undocumented API, most used first The public symbols no doc names or calls, ordered by how much other code depends on them: by the number of other repositories that call them, then classes before functions and methods, then by the number of symbols that call them. The first 25 are listed (`--limit`). ## The score The score starts at 100 and loses points for three things: | Deduction | Points | Cap | |---|---|---| | Each example with at least one line that no longer fits the code | 8 | 50 | | Each other example that fails when run (`fail` or `timeout`; examples already counted as mismatched are not counted twice) | 6 | 30 | | Each percent of public API that no doc names or calls | 0.3, rounded | none | The score never goes below 0. The grade is `A` from 90, `B` from 80, `C` from 70, `D` from 60, else `F`. Each deduction is listed in the report with its reason. For example, auditing the fixture repository `acme/mini-client` from the test suite: 2 examples call it in ways it no longer accepts (−16), and 4 of its 11 public symbols are named or called by no doc, 63.6% coverage (−11, from 36.4 × 0.3). The score is 73, grade C. ## What an audit covers | Called with | Repositories | Examples | Mismatches | API coverage | |---|---|---|---|---| | No repository | Every repository in the graph | Every live example | Every one | Every repository's symbols | | Repositories | Those | The examples in their docs, and examples in other repositories that call their code | In their docs, or in other docs against their code | Their symbols | So `audit acme/acme-client` also reports a line in `acme/acme-docs` that calls the client wrongly, when both are in the graph. A repository the graph does not have fails with `not indexed in the "" graph: ` (the CLI indexes it first unless `--no-index`). ## The CLI ```text staleproof audit [options] [repos...] ``` | Option | Default | Meaning | |---|---|---| | `--run ` | `0` | First run up to `n` Python examples at the indexed commits (needs `git`, and uv or Python 3.12+) | | `--limit ` | `25` | How many undocumented symbols to list | | `--format ` | `text` | `text`, `markdown` (the Action's job summary and issue body) or `json` | | `--out ` | stdout | Write the report to a file; logs `wrote : ` on stderr | | `--fail-under ` | none | Exit 1 when the score is below this whole number | | `--fail-on-mismatch` | off | Exit 1 when any example line no longer fits the code | | `--no-index` | index on first use | Fail instead of indexing repositories the graph does not have yet | The [global options](cli.md#global-options) apply: without `--org`, the graph is the owner of the first repository named. [CLI](cli.md#audit) has the exit statuses. **Prints, text.** From the fixture `acme/mini-client` (colours off): ```text Docs health 73/100 (C) acme/mini-client@a6214ef Docs health 73/100 (C) for acme/mini-client. 2 of 5 examples that call the code no longer fit it (4 lines). 7 of 11 public symbols (63.6%) are named or called by a doc. Score -16 2 examples call the code in a way it no longer accepts (8 each, at most 50) -11 4 of 11 public symbols are named or called by no doc (0.3 per missing percent) Examples that no longer fit the code (2 examples, 4 lines) ✗ docs/stale.md:0 line 8 · acme/stale-docs `miniclient` has no `connect` (imports a name that does not exist; miniclient.connect) ✗ docs/stale.md:0 line 10 · acme/stale-docs `Client` has no parameter `hostname` (unknown keyword argument; Client) ✗ docs/stale.md:1 line 19 · acme/stale-docs `Graph.query` takes at most 2 positional arguments; the example passes 3 (too many positional arguments; g.query) ✗ docs/stale.md:1 line 20 · acme/stale-docs `Graph` has no attribute `read_only` (calls an attribute that does not exist; g.read_only) API coverage: 7 of 11 public symbols (63.6%) named or called by a doc Undocumented, most used first: · miniclient.asyncio.client.AsyncClient class · 1 callers (1 other repos) · acme/mini-client · miniclient.asyncio.client.AsyncClient.select_graph method · 1 callers (1 other repos) · acme/mini-client · miniclient.asyncio.client.AsyncGraph.query method · 1 callers (1 other repos) · acme/mini-client · miniclient.asyncio.client.AsyncGraph class · 1 callers · acme/mini-client Badge: [![docs health](https://staleproof.dev/api/badge/acme/mini-client.svg)](https://staleproof.dev/#/audit?repo=acme%2Fmini-client) ``` The mismatched examples live in another repository (`acme/stale-docs`) and call `acme/mini-client`'s code, so they count for it. When runs are recorded, a section `Example runs at the indexed commits (

pass, fail, skipped, not installed)` lists the failing ones. The badge line appears when one repository is audited. When a summary sentence `Indexed by an older Staleproof: …` appears, see [Limits](#limits). **Prints, Markdown.** The same content as tables: the deductions, the mismatched lines with links to the exact doc line (`| [docs/stale.md:1 line 20](…#L20) | Upgrading guide > Query | `g.read_only` | `Graph` has no attribute `read_only` |`), the failing runs, coverage with the undocumented symbols in a collapsed `

` block, the badge, a footer with the graph and commits, and a last line the GitHub Action reads: ```text ``` The first line is the marker ``, which the Action uses to find its issue. **JSON.** `{summary, org, repos: [{name, sha, symbols, sections, indexVersion}], outdatedIndex, score, grade, deductions: [{points, why}], examples: {total, python, linked, mismatched}, mismatches: [{example, url, repo, path, heading, line, kind, detail, call, symbol}], coverage: {api, documented, percent}, undocumented: [{qualname, repo, kind, signature, url, callers, crossRepoCallers}], runs: [{example, url, status, error, reason}], runCounts: {pass, fail, skipped, error}, generatedAt}`. `examples.linked` counts the examples that call at least one indexed symbol; `examples.mismatched` the ones with at least one `MISMATCH` line. ## On the web [staleproof.dev/#/audit](https://staleproof.dev/#/audit) audits any public GitHub repository: type `owner/name` (or a repository URL) and press **Audit**. The first audit downloads and indexes the repository into the workspace graph of its owner, showing the progress, then shows the score and grade, the deductions, every example line that no longer fits (linked to the line on GitHub), failing runs recorded earlier, the coverage bar with the undocumented symbols, the badge Markdown (pointing at the host serving the page) and a link to install the GitHub Action. A link to a repository's report is `https://staleproof.dev/#/audit?repo=%2F`; opening it starts the audit. The hosted service runs no examples, so its audit scores the static checks and coverage only (plus runs recorded earlier, for FalkorDB's curated repositories). A server you run with `staleproof serve` serves the same page at `#/audit`; it audits the repositories in its graph, and with `--hosted` indexes any public repository first. ## The README badge ```markdown [![docs health](https://staleproof.dev/api/badge//.svg)](https://staleproof.dev/#/audit?repo=%2F) ``` The report prints this line for one repository. The badge reads `docs health | `, in the flat shields.io style: | Score | Colour | |---|---| | 90 and up | `#2ea44f` (green) | | 80 to 89 | `#97ca00` | | 70 to 79 | `#dfb317` (yellow) | | 60 to 69 | `#fe7d37` (orange) | | below 60 | `#e05d44` (red) | | not indexed on that server | `not audited`, grey | For an indexed repository the server computes the audit when the badge is requested (cached for 10 minutes per repository) and sends `Cache-Control: public, max-age=3600, s-maxage=3600`, so a README shows a score at most about an hour old. A repository the server has not indexed gets the grey badge, cached for 5 minutes: audit it once on the page (or `POST /api/audit`) and the badge fills in. The badge is served by any `staleproof serve`; replace the host to point it at yours. ## HTTP API and MCP | Interface | What it does | |---|---| | [`GET /api/audit?repo=`](http-api.md#get-apiaudit) | The audit of an indexed repository as JSON, cached for 10 minutes (`&fresh=1` recomputes it); 404 when it is not indexed | | [`POST /api/audit`](http-api.md#post-apiaudit) | Hosted mode: index the repository if needed, then audit it, streaming progress as server-sent events | | [`GET /api/badge//.svg`](http-api.md#get-apibadgeownernamesvg) | The badge | | MCP [`audit_docs`](tools.md#audit_docs) | The audit for an assistant: `repos` (default every repository in the graph) and `limit`. The stdio server indexes named repositories on first use; the hosted server indexes the first one and needs it named. It reads recorded runs and runs nothing | ## In CI: audit on a schedule The [GitHub Action](github-action.md#audit-the-docs-on-a-schedule) audits instead of checking a pull request when the workflow runs on a schedule, `workflow_dispatch` or `push`. Copy [examples/staleproof-audit.yml](../examples/staleproof-audit.yml): ```yaml name: Staleproof docs audit on: schedule: - cron: '0 6 * * 1' workflow_dispatch: permissions: contents: read issues: write jobs: audit: runs-on: ubuntu-latest steps: - uses: rajkaria/staleproof@v1 ``` Every Monday at 06:00 UTC it indexes the repository (and `also-index`), runs up to 20 examples at the default branch (`audit-runs`), writes the report to the job summary and keeps one issue up to date: opened when an example no longer fits or fails, updated on every run, and closed with a comment once all is clear. `fail-on: mismatches` and `fail-under: ` fail the job; the outputs `score` and `mismatched` carry the numbers. ## Reading the report Work through the mismatches first: each one is a line a reader will copy and see fail. | Kind | What to do | |---|---| | `unknown-keyword` | The parameter was renamed or removed. Find its new name in the signature the evidence points at (`staleproof docs-for ` shows the signature), or drop the argument | | `too-many-arguments` | A parameter became keyword-only, or was removed. Pass the rest by keyword, as the current signature names them | | `missing-argument` | A parameter became required. Pass it, or check whether the example should call something else | | `missing-import` | The name moved or was removed from the package. Import it from where it lives now, or use its replacement | | `missing-attribute` | The method was renamed or removed, or never existed on that class. Call the current method | Then the failing runs: a run that fails at the indexed commits is broken for readers now, whatever the static checks say (a wrong value, a server the page never sets up). Then coverage: the undocumented list is ordered so the symbols other code relies on most come first. When a finding is intentional, mark the example rather than editing the code: `` above an example that fails on purpose, `` above one that cannot be judged, or `skip-examples` in `.staleproof.toml` for a whole directory of versioned docs ([directives and repository config](configuration.md#repository-configuration)). Marked examples stay linked to the code (so pull request checks still find them) but are never judged or run. Re-index the docs repository after editing, then audit again. ## What it finds on real repositories Lines the static checks report on repositories as they are today (checked on 2026-10-17): - **FalkorDB docs**: `commands/graph.constraint-create.mdx` calls `client.create_constraint`, which `FalkorDB` does not have (`missing-attribute`). Running the example fails with `AttributeError: 'FalkorDB' object has no attribute 'create_constraint'`. - **GraphRAG-SDK**: `docs/api-reference.mdx` documents `VectorStore(index_name=..., similarity_function=...)`, parameters the constructor no longer has (`unknown-keyword`). - **Flask**: `docs/views.rst` calls `UserList.as_view()` without its required `name` (`missing-argument`), found in reStructuredText docs. - **httpx**: the docs pass `extensions=` to `httpx.get`, which no longer accepts it (`unknown-keyword`). ## Limits - **Static checks follow exact resolutions only.** A call whose receiver the types do not reveal (resolved by name, or not at all) is never judged, so a clean audit means "nothing found where the source is certain", not "every example works". Runs (`--run`, or the Action's `audit-runs`) cover the rest for Python. - **Guards leave lines unjudged**: functions with a decorator outside a short allow-list (`staticmethod`, `classmethod`, `functools.cache`, `lru_cache`, `abstractmethod`, `override`, `final`, `deprecated`, `wraps`), several definitions under one name (`@overload`), properties, classes with class decorators, a metaclass, a `__getattr__`, a `__new__` or a base class outside the organisation (other than `object`, `ABC`, `Generic`, `Protocol`), modules with a `__getattr__` or a star import from outside the organisation, calls with `*args` or `**kwargs`, and calls or signatures tree-sitter cannot read. - **TypeScript and JavaScript get no argument checks** (JavaScript accepts any argument count), only names that do not exist; their examples are never run. - **Runs need a sandbox.** `--run` needs `git` and uv or Python 3.12+; the hosted service and `audit_docs` run nothing and read only recorded runs. - **Coverage counts references, not explanations.** A symbol named once in inline code counts as documented. - **An older index** (written before Staleproof 0.3) has no `MISMATCH` edges or export flags. The audit then adds `Indexed by an older Staleproof: examples are not checked against the code until the repositories are indexed again (staleproof index).` to its summary and sets `outdatedIndex`; re-index the repositories. Hosted workspaces indexed by an older version are re-indexed on their next use. - **One graph at a time.** An audit covers repositories in one graph; cross-repository findings need the repositories indexed together. ## Related - [CLI](cli.md#audit): the `audit` command and its exit statuses - [Indexing](indexing.md#static-checks-of-examples): how examples are checked against the code - [GitHub Action](github-action.md#audit-the-docs-on-a-schedule): the scheduled audit and its issue - [Configuration](configuration.md#repository-configuration): directives and `.staleproof.toml` - [Core concepts](concepts.md#static-checks-and-the-audit) --- # CLI Every `staleproof` command and option, with defaults taken from the code, examples, what each command prints or writes, and its exit status. It is for anyone running Staleproof from a terminal, a script or CI. For the environment variables behind the defaults, see [Configuration](configuration.md). ## Common tasks ```bash npx -y staleproof db start npx -y staleproof index FalkorDB/falkordb-py FalkorDB/GraphRAG-SDK FalkorDB/docs npx -y staleproof check https://github.com/FalkorDB/falkordb-py/pull/262 --no-agent npx -y staleproof check https://github.com/FalkorDB/falkordb-py/pull/262 npx -y staleproof run "FalkorDB/falkordb-py:README.md:4" --pr https://github.com/FalkorDB/falkordb-py/pull/262 npx -y staleproof patch https://github.com/FalkorDB/falkordb-py/pull/262 ``` The first line starts a local FalkorDB (optional: any command starts one when none is configured, see [FalkorDB on first use](#falkordb-on-first-use)), the second indexes three repositories into one graph, the third checks a pull request without a model ([graph mode](#check)), the fourth checks it with the agent (needs model credentials), the fifth runs one docs example before and after the pull request, and the last prints the docs patch. For a repository nobody has indexed, `check` alone is enough: it indexes the pull request's repository on first use. ```bash npx -y staleproof check https://github.com///pull/ --no-agent ``` Before there is a pull request, and with none at all: ```bash npx -y staleproof check --fail-on-predicted npx -y staleproof docs-for Graph.query --repo FalkorDB/falkordb-py npx -y staleproof audit FalkorDB/falkordb-py FalkorDB/GraphRAG-SDK FalkorDB/docs ``` The first checks the working tree of the checkout you are in (uncommitted and unpushed changes) and exits 1 when an example is predicted to break, which makes it a pre-push hook ([below](#check-without-a-pull-request)); the second lists the docs that depend on a symbol before you change it; the third audits the docs as they are ([Docs audit](audit.md)). ## Install or run with npx | How | Command | Notes | |---|---|---| | No install | `npx -y staleproof ` | `-y` answers npx's install prompt. Pin a version for repeatable runs: `npx -y staleproof@0.3.0 `. | | Global install | `npm install -g staleproof`, then `staleproof ` | Puts `staleproof` on your `PATH`. | | From a clone | `npm install && npm run build`, then `node dist/cli.js ` | Or `npm run dev -- ` to run the TypeScript source. | All three need Node.js 20 or newer. The package carries the built web UI, the tree-sitter grammars (Python, TypeScript, TSX), the sandbox shim and `scripts/falkordb-local.sh`. Examples on this page use `npx -y staleproof`; with a global install, drop `npx -y`. ## Global options | Option | Default | Meaning | |---|---|---| | `--org ` | `STALEPROOF_ORG`, else the owner of the repository the command is about, else `falkordb` | The organisation, which names the graph (one graph per organisation). The value is turned into a slug: trimmed, lowercased, every run of characters other than `a-z`, `0-9`, `_` and `-` replaced by `-`. An empty result fails with `invalid org name`. | | `--db ` | `FALKORDB_URL`, else `falkor://localhost:6379` | The FalkorDB server holding the graph. `falkor://` and `redis://` connect in plain TCP, `falkors://` and `rediss://` with TLS; credentials go in as `user:password@`. | | `-V`, `--version` | | Print the package version. | | `-h`, `--help` | | Print help. `staleproof --help` or `staleproof help ` shows one command. | Global options work before or after the command: `staleproof --org encode check …` and `staleproof check … --org encode` are the same. Results go to stdout; progress and log lines go to stderr, so you can pipe `--json` output safely. **The default graph.** Without `--org` or `STALEPROOF_ORG`, a command uses the graph of the owner of the repository it is about, slugged: `check https://github.com/encode/httpx/pull/` reads the graph `encode`, and FalkorDB's repositories land in `falkordb`. The repository is the first one given to `index` or `audit`, the pull request's for `check` and `patch`, the checkout's for `check` without a pull request, the first `--repo` (else the symbol id's) for `docs-for`, the example id's for `run`, and the first `--governs` id's for `decide`. `mcp`, `serve`, `verdict` and `facts` name no repository and use `falkordb`, and so do `audit` with no repository and `docs-for` with a bare name and no `--repo`. Pass `--org` when one graph holds repositories of several owners. ### FalkorDB on first use Every command except `db` connects to FalkorDB. When neither `--db` nor `FALKORDB_URL` is set and nothing answers on `localhost:6379`, it first starts a local server there, the same one as `db start` (falkordblite, installed into `$STALEPROOF_HOME/falkordb/venv` on first use, which takes a minute), logs `No FalkorDB on localhost:6379: starting a local one (falkordblite; the first start installs it, which takes a minute).` on stderr, and leaves it running. With `--db` or `FALKORDB_URL` set, an unreachable server is an error: `cannot reach FalkorDB at : …`. ## Exit status | Status | When | |---|---| | `0` | The command finished. | | `1` | Any error. The message is printed to stderr as `staleproof: `. Typical causes: an invalid pull request URL, a repository not indexed in the graph, FalkorDB unreachable (`cannot reach FalkorDB at : …`), missing model credentials, the GitHub API rate limit, an invalid `STALEPROOF_EFFORT`. | | `1` | `check` in agent mode, when the check ends with status `error` or `refused`. The output is printed and the check is stored first. `done` and `incomplete` exit `0`. | | `1` | `check --no-agent --format markdown`, when the stored check ends with status `error`. The comment is written first. | | `1` | `check --fail-on-predicted` without a pull request, when an example is predicted to break. The report is printed first. | | `1` | `audit --fail-under `, when the health score is below it; `audit --fail-on-mismatch`, when any example line no longer fits the code. The report is printed (or written) first, and the reason is logged on stderr. | | `1` | `db status`, when no server answers on the port. | `check --no-agent` without `--format markdown` exits `0` even when the stored check ends with status `error` (for example when the impact query fails after the pull request was read); the summary line says so. ## Ids Commands and tools refer to graph nodes by id: | Node | Format | Example | Where to find it | |---|---|---|---| | Symbol | `/:` | `FalkorDB/falkordb-py:falkordb.asyncio.falkordb.FalkorDB.__init__` | The qualified names in "Changed symbols" and in evidence paths; prefix the repository. | | DocSection | `/:#` | `FalkorDB/docs:commands/graph.explain.mdx#` | Impact results (`check --json`, MCP `impact`). The anchor is the heading's slug; the text before a file's first heading has an empty anchor (the id ends in `#`). | | CodeExample | `/::` | `FalkorDB/docs:commands/graph.explain.mdx:0` | Impact results (`examples` on each section) and `read_section`. `index` is the block's 0-based position among all code blocks in the file, whatever their language (fenced blocks in Markdown and MDX; code directives, `::` literal blocks and `>>>` doctest blocks in reStructuredText). An example script file has index `0`. | | PullRequest | its URL | `https://github.com/FalkorDB/falkordb-py/pull/267` | `/#` is accepted too. | | Decision | `dec-` and 10 hex characters | `dec-` | Printed by `decide`. | | Verdict | `ver-` and 12 hex characters | `ver-` | `check --json` (results of `propose_edit` and `record_verdict`), MCP results, or a Cypher query (below). | | Run | `run-` and 20 hex characters | `run-` | `run --json` (`data.runs[].id`). A run that could not be prepared gets `run--err-`. | | Check | `chk-` and 12 hex characters | `chk-` | The last line of `check`. | List the impacted sections and their examples for a pull request, without running any example: ```bash npx -y staleproof check https://github.com/FalkorDB/falkordb-py/pull/267 --no-agent --runs 0 --json \ | jq -r '.events[] | select(.type == "tool_result" and .name == "impact" and (.isError | not)) | .data.sections[] | "\(.section)\t\(.examples | join(" "))"' ``` This lists the first impact page (up to 40 sections). List the verdicts an agent check recorded: ```bash npx -y staleproof check https://github.com/FalkorDB/falkordb-py/pull/267 --json \ | jq -r '.events[] | select(.type == "tool_result" and (.name == "propose_edit" or .name == "record_verdict") and (.isError | not)) | .data | (.id // .ids[])' ``` Or query the graph directly for every verdict on a pull request, from any check: ```cypher MATCH (v:Verdict)-[:FOR]->(:PullRequest {url: 'https://github.com/FalkorDB/falkordb-py/pull/267'}) MATCH (v)-[:ABOUT]->(d:DocSection) RETURN v.id, v.verdict, v.status, d.id ORDER BY v.created_at DESC ``` ## Commands ### `index` Clone or update GitHub repositories, or download them where there is no git, and index their code and docs into the organisation's graph. ```text staleproof index [--ref ]... [--json] ``` **Arguments.** ``: one or more repositories as `owner/name`, `https://github.com/owner/name` (with or without `.git`) or `git@github.com:owner/name.git`. | Option | Default | Meaning | |---|---|---| | `--ref ` | the remote default branch head | Check out `ref` for that repository. Repeatable. The ref is passed to `git checkout --detach`; use `origin/`, a tag or a commit SHA, because a bare branch name can resolve to a stale local branch. | | `--json` | off | Print the index report as JSON instead of the summary. | **Examples.** ```bash npx -y staleproof index FalkorDB/falkordb-py FalkorDB/GraphRAG-SDK FalkorDB/docs npx -y staleproof --org encode index https://github.com/encode/httpx npx -y staleproof index FalkorDB/falkordb-py FalkorDB/GraphRAG-SDK FalkorDB/docs --ref FalkorDB/docs=origin/main --json > index-report.json ``` **What it does.** Each repository is cloned blob-less over HTTPS into `/repos//` the first time and fetched afterwards, then checked out at the ref; with `GITHUB_TOKEN` (or `GH_TOKEN`) set, git sends the token to github.com, so private repositories the token can read work too. When git is not installed, or with `STALEPROOF_INGEST=github`, each repository is downloaded as a tarball instead and only the files the indexer reads are extracted, into `/tarballs///` ([details](indexing.md#without-git-tarballs)). Python files, and the TypeScript and JavaScript files of published npm packages, are parsed with tree-sitter; calls are resolved across every repository in the command; Markdown, MDX and reStructuredText pages become doc sections and code examples linked to the code they mention or call; every example call is checked against the signature it reaches, and the lines that no longer fit are recorded as `MISMATCH` edges; and dependencies between the repositories are read from `pyproject.toml`, `requirements*.txt` and `package.json` files and from the docs' imports. A repository's `.staleproof.toml` (or `[tool.staleproof]` in `pyproject.toml`) can leave paths out or mark examples not to run ([Configuration](configuration.md#repository-configuration)). [Indexing](indexing.md) has the rules. **Prints.** On stderr, `checkout ` per repository, then one line per phase (`parse`, `write`, `resolve`, `docs`, `done`). On stdout, `Indexed repositories into graph "" in s`, then one line per repository (commit, symbols, calls and how many cross repositories, parse errors if any), one line per repository with docs (sections, examples, uses, mentions, links, and when there are any, ` mismatches (staleproof audit )`: example lines that no longer fit the code) and one line per dependency, for example `FalkorDB/GraphRAG-SDK → FalkorDB/falkordb-py (declares falkordb)`. With `--json`: `{org, run, repos: [{repo, sha, files, symbols, calls: {exact, byName, crossRepo}, gone, parseErrors}], docs: [{repo, pages, sections, examples: {total, python}, mentions, uses: {exact, byName}, links, mismatches, gone}], dependsOn: [{from, to, spec}], ms}`. **Notes.** - Index related repositories in one command. Calls and doc links cross repositories only among the repositories of one run; indexing a repository alone re-resolves its edges against itself and drops the cross-repository ones. - Re-indexing updates nodes in place. Symbols, sections and examples that disappeared are marked `gone`, never deleted, so decisions and verdicts keep pointing at them. Links that an earlier pull request check added for removed symbols (`late` links) are dropped and not recreated. - Without git, `--ref` must be a branch, tag or commit SHA that GitHub knows; `origin/` names a ref in a git clone only. - Private repositories need `GITHUB_TOKEN` (or `GH_TOKEN`) with read access to them. - The graph defaults to the owner of the first repository: `index encode/httpx` writes `encode`. ### `check` Check a pull request against the graph: with the agent (default) or in graph mode (`--no-agent`), as terminal text, JSON or a pull request comment. With no pull request, check the working tree of the checkout you are in, statically ([below](#check-without-a-pull-request)). ```text staleproof check [options] [pr-url] ``` **Arguments.** `[pr-url]`: `https://github.com///pull/` (a trailing `/files`, query or fragment is accepted) or `/#`. When the graph does not have the repository yet, it is indexed first ([below](#check)), unless `--no-index` is given. Owner and name are case-insensitive: the URL is rewritten to the repository's indexed spelling. Leave it out to check your working tree instead. | Option | Default | Meaning | |---|---|---| | `--no-agent` | agent mode | Graph mode: run the tools in a fixed order with no model. | | `--model ` | `STALEPROOF_MODEL`, else `claude-sonnet-5-5` | Agent mode only. The model to call. With an Anthropic-compatible provider, a `claude-*` name is refused. | | `--effort ` | `STALEPROOF_EFFORT`, else `medium` | Agent mode only. `low`, `medium`, `high`, `xhigh` or `max`. The flag is passed to the model as given; only the environment variable is validated. | | `--max-steps ` | `40` | Agent mode only. Model turns before the loop stops; the check is then `incomplete`. | | `--max-hops ` | `3` | Graph mode only. Maximum call hops from a changed symbol to a doc. Clearance always uses 3. | | `--runs ` | `3` | Graph mode only. At most this many examples are run before and after the pull request; `0` runs none. | | `--json` | off | Print the check result as JSON instead of the trace and summary. Ignored with `--no-agent --format markdown`. | | `--format ` | `text` | `text` for the terminal, or `markdown` for the pull request comment the [GitHub Action](github-action.md) posts. | | `--out ` | stdout | With `--format markdown`, write the comment to this file. Ignored otherwise. | | `--with ` | none | Repositories to index together with the pull request's when the graph is missing any of them, for example the docs site: `--with FalkorDB/docs`. Each as `owner/name` or a GitHub URL. | | `--no-index` | index on first use | Fail with ` is not indexed in the "" graph` instead of indexing a repository the graph does not have. | | `--dir ` | the current directory | Without a pull request: the git checkout to check. | | `--base ` | the merge base with `origin/HEAD`, else `origin/main`, else `origin/master`, else `HEAD` | Without a pull request: the commit the working tree is compared with. | | `--repo ` | the checkout's `origin` remote on github.com | Without a pull request: the GitHub repository the checkout is. | | `--fail-on-predicted` | off | Without a pull request: exit 1 when an example is predicted to break (a pre-push hook or a CI gate). | **The four invocations.** | Invocation | What runs | What it stores | |---|---|---| | `check ` | The agent loop: the model calls the tools until every impacted section has a verdict or an edit. | A Check node with the full trace, verdicts, example runs. | | `check --no-agent` | Graph mode: `impact`, `clearance`, `rank_sections` over the first impact page, then `run_example` for at most `--runs` examples: those predicted to break (with an `exact` match) first, then examples on sections whose evidence path is `exact`. | A Check node (model `graph-only (no model)`) with the trace and runs. No verdicts. | | `check --format markdown` | The agent check, then the comment, including the agent's summary. | As the agent check. | | `check --no-agent --format markdown` | Graph mode as above (its trace lines go to stderr), then the comment: the examples run before and after, impact, clearance, ranking and the verdicts already recorded for the pull request. This is what the [GitHub Action](github-action.md) runs without a model key. | As graph mode. | **Indexing on first use.** Before the check, `check` looks for the pull request's repository and every `--with` repository in the graph. If any is missing, it fetches all of them (a git clone, or a tarball without git) at their default branch heads and indexes them together in one run, logging `checkout fetching `, the index phases and `indexed @, … into graph "" in s` on stderr. When all are present it indexes nothing, so later checks start at once; re-index with [`index`](#index) to move the graph to newer commits. **Examples.** ```bash npx -y staleproof check https://github.com/FalkorDB/falkordb-py/pull/262 --no-agent npx -y staleproof check https://github.com/acme/acme-client/pull/12 --no-agent --with acme/acme-docs npx -y staleproof check FalkorDB/falkordb-py#276 --no-agent --runs 0 npx -y staleproof check https://github.com/FalkorDB/falkordb-py/pull/267 --max-steps 60 npx -y staleproof check https://github.com/FalkorDB/falkordb-py/pull/267 --no-agent --format markdown --out comment.md ``` **Prints, graph mode.** The shape of the output, with placeholders in angle brackets: ```text (, ..) Changed symbols () Impacted doc sections () [] (other repo) path: ← ← hops, , example(s) · Predicted to break (

): the call no longer fits the changed code ✗ : line Already out of date before this PR () ! : line Cleared: reach no doc within 3 hops () ✓ (; raw text matches in docs; check: git grep -nw -- '*.md' '*.mdx' '*.rst') FAIL before PASS after

Stored as chk-. ``` Sections are ordered by the PageRank of their page in the docs' link graph. Only the first 40 are listed, followed by `… more` when there are more. **Predicted to break** lists, over every impacted section, the example lines whose call no longer fits the code after the pull request: a removed or renamed symbol, a name the package stops exporting, or a call the new signature rejects that the old one accepted ([Predicted breaks](concepts.md#predicted-breaks)); a line marked `(matched by name)` was linked by name rather than through types. **Already out of date** lists impacted example lines that did not fit the code before the pull request either. Both blocks are left out when empty. A pull request that already edits docs adds a line `This PR already edits: `. A run answered from the graph is marked `(recorded)`. **Prints, agent mode.** The trace, one or more lines per event: ```text Staleproof checking (, chk-) since the last check: → impact(pr_url=) ← Cypher queries: impact, prChanges, … · ms ! impacted section(s) still need a verdict; reminding the model Done turns, tool calls (impact → read_section → …), verdicts, tokens Patch: staleproof patch Trace: stored on chk- ``` The final status is `Done`, `Incomplete`, `refused` or `error`. The check is `Done` only when every impacted section, on every impact page, has a verdict or an edit. When the model stops with sections still open it is reminded up to twice; after that, or after `--max-steps` turns, the check is `Incomplete` and lists the open sections. **Prints, `--format markdown`.** The report the GitHub Action posts as its comment, on stdout or into `--out`: the title (what broke first), the impact summary, the examples predicted to break, a table of the examples run before and after the pull request, the impacted sections with their evidence paths, the lines already out of date, the cleared symbols, the verdicts and, in agent mode, the agent's summary. It ends with a machine-readable line, ``. [GitHub Action](github-action.md#the-comment) shows the format. **JSON.** `{checkId, pr, status, summary, events, impacted, missing, toolOrder, since}`. `events` holds every trace event, including each tool's full result in `tool_result.data`. In graph mode `impacted` and `missing` are empty; read the sections from the `impact` result in `events`. **Notes.** - The first check of a pull request reads its metadata from GitHub and diffs it into symbol-level changes. For a merged pull request the commits compared are the merge commit's first parent and the merge commit; for an open one, the merge base and the head. Before each later check of an open pull request, `check` asks GitHub for its head: when new commits were pushed (or it was merged since), it logs ` has new commits since it was last checked: reading its current diff` and diffs it again. A merged pull request is never re-diffed. When GitHub cannot be reached, the stored diff is used. - Agent mode needs model credentials: `ANTHROPIC_API_KEY` (or another credential the Anthropic SDK finds) for Claude, or `ANTHROPIC_BASE_URL`, `ANTHROPIC_API_KEY` and `STALEPROOF_MODEL` for an Anthropic-compatible provider. Without them it stops with an error naming those variables. See [The agent](agent.md). - The CLI puts no time limit on an agent check. A large pull request takes minutes: a 43-section check takes about 6 minutes. - Graph mode samples: it ranks the first impact page and runs at most `--runs` examples. It never judges sections or writes verdicts. - Two kinds of change are reported more broadly than they happened. A moved or renamed file reports every definition in it as removed and added. A module that stops importing a name from its own package reports that name as a removed export (`BREAKING`), even when the import was not a public re-export. #### Check without a pull request With no pull request URL, `check` checks the working tree of a git checkout against the commit it started from: before you commit, before you push, or from a pre-push hook. ```bash npx -y staleproof check npx -y staleproof check --dir ../acme-client --base origin/release --format markdown --out report.md ``` What it does: 1. Finds the checkout's top directory (from `--dir`, default the current directory), the GitHub repository it is (`--repo`, else its `origin` remote), the base commit (`--base`, else the merge base of `HEAD` with `origin/HEAD`, `origin/main` or `origin/master`, else `HEAD` itself) and the current branch. 2. Indexes the repository (and the `--with` repositories) when the graph does not have it, as a pull request check does, unless `--no-index`. 3. Compares the base commit with the files on disk: staged, unstaged and untracked (not ignored) files, and any commits since the base, so unpushed commits count. Python and TypeScript/JavaScript library files become symbol-level changes, as for a pull request. 4. Walks the graph from the changed symbols: the impacted sections with their evidence paths, the examples [predicted to break](concepts.md#predicted-breaks), the impacted lines already out of date, and the cleared symbols. Nothing runs and nothing about the changes is written to the graph: no pull request, no check, no run. The agent and run options (`--no-agent`, `--model`, `--runs`, …) do not apply. **Prints.** The same blocks as graph mode, without runs. From the test suite's fixture organisation, after adding a required keyword-only `timeout` to `Graph.query`: ```text Changed symbols (1) signature miniclient.graph.Graph.query def query(self, q: str, params=None) → def query(self, q: str, params=None, *, timeout: float) Impacted doc sections (3) SIGNATURE Getting started > Connect [acme/mini-docs] (other repo) path: miniclient.graph.Graph.query ← acme/mini-docs:docs/getting-started.mdx:0 1 hops, exact, 1 example(s) · https://github.com/acme/mini-docs/blob//docs/getting-started.mdx#L14 … Predicted to break (1): the call no longer fits the changed code ✗ docs/getting-started.mdx:0 line 18 g.query `Graph.query` requires `timeout`, which the example does not pass Already out of date before these changes (2) ! docs/stale.md:1 line 19 `Graph.query` takes at most 2 positional arguments; the example passes 3 ! docs/stale.md:1 line 20 `Graph` has no attribute `read_only` Cleared: reach no doc within 3 hops (0) 1 changed symbol reaches 3 doc sections (0 breaking, 3 signature, 0 behavior; 3 in other repositories). 1 example is predicted to break, the call no longer fitting the changed code. Static check of the working tree: nothing was run or written to the graph. Push and open a pull request to run the examples before and after. ``` With `--json`: `{summary, impact, cleared}`, where `impact` has the shape of the [`impact` result](tools.md#impact-result) (`pr.state` is `local`). With `--format markdown`: the comment format, titled `these changes are predicted to break docs example(s)` (or ` doc section(s) these changes can make false`), written to `--out` when given. **As a pre-push hook.** `--fail-on-predicted` exits 1 when any example is predicted to break, so git stops the push: ```bash printf '#!/bin/sh\nnpx -y staleproof check --fail-on-predicted\n' > .git/hooks/pre-push && chmod +x .git/hooks/pre-push ``` `git push --no-verify` skips the hook once. **Errors.** `cannot tell which GitHub repository is (origin: ); pass --repo owner/name` when the `origin` remote is not on github.com (or missing); ` is not indexed in the "" graph: staleproof index ` with `--no-index`; git's own error when the directory is not a checkout or `--base` does not resolve. ### `docs-for` Which docs depend on a symbol, with the evidence path for each: what to update before you change it. ```text staleproof docs-for [--repo ] [--json] ``` **Arguments.** ``: a Symbol id (`/:`), a qualified name (`miniclient.graph.Graph.query`), a dotted suffix (`Graph.query` matches every qualified name ending in `.Graph.query`) or a bare name (`query`). Only live symbols count, never modules. An exact id or qualified name wins over suffix and bare-name matches; otherwise the first 5 matches, in qualified-name order, are looked up together and the summary says the name is ambiguous. | Option | Default | Meaning | |---|---|---| | `--repo ` | any repository | Only symbols defined in these repositories. The first one also picks the graph (its owner). | | `--json` | off | Print `{summary, query, symbols, ambiguous, sections, mismatched}` | **What it does.** The [impact](concepts.md#impact) walk started from the symbol instead of a pull request: symbol ← up to 3 callers in any repository ← the doc sections that name them and the examples that call them, then the lines in those examples that already do not fit the code. It reads the graph and writes nothing. The MCP tool [`docs_for_symbol`](tools.md#docs_for_symbol) and [`GET /api/docs-for`](http-api.md#get-apidocs-for) give the same answer. **Prints.** From the fixture organisation, `docs-for Graph.query --repo acme/mini-client`: ```text miniclient.graph.Graph.query method · acme/mini-client def query(self, q: str, params=None) Getting started > Connect [acme/mini-docs] 1 hops, exact, 1 example(s) path: miniclient.graph.Graph.query ← acme/mini-docs:docs/getting-started.mdx:0 https://github.com/acme/mini-docs/blob//docs/getting-started.mdx#L14 Upgrading guide > Query [acme/stale-docs] 1 hops, exact, 1 example(s) path: miniclient.graph.Graph.query ← acme/stale-docs:docs/stale.md:1 https://github.com/acme/stale-docs/blob/fixture/docs/stale.md#L14 Getting started > Read-only queries [acme/mini-docs] 2 hops, exact, 1 example(s) path: miniclient.graph.Graph.query ← miniclient.graph.Graph.ro_query ← acme/mini-docs:docs/getting-started.mdx:1 https://github.com/acme/mini-docs/blob//docs/getting-started.mdx#L21 ! acme/stale-docs:docs/stale.md:1 line 19: `Graph.query` takes at most 2 positional arguments; the example passes 3 ! acme/stale-docs:docs/stale.md:1 line 20: `Graph` has no attribute `read_only` 3 doc sections reach miniclient.graph.Graph.query (2 name or call it directly; 2 repositories). 2 line(s) in those examples already do not fit the code. ``` No match prints `No live symbol in the "" graph is named .`; a symbol no doc reaches, `No doc section or example reaches within 3 call hops: changing it cannot make a doc false through the code.` Without `--repo` or an id, the graph is `STALEPROOF_ORG`, else `falkordb`: pass `--repo` or `--org` for other organisations. ### `audit` Audit docs with no pull request: a 0-100 health score, the example lines that no longer fit the code, API coverage and the example runs at the indexed commits. [Docs audit](audit.md) explains each part, the score and how to read the report. ```text staleproof audit [options] [repos...] ``` **Arguments.** `[repos...]`: the repositories to audit, as `owner/name` or GitHub URLs. When the graph is missing any of them, all of them are fetched and indexed together at their default branch heads first (as `check` does). With none, every repository in the graph is audited. | Option | Default | Meaning | |---|---|---| | `--run ` | `0` | First run up to `n` Python examples at the indexed commits, examples with a mismatch first (needs `git`, and uv or Python 3.12+) | | `--limit ` | `25` | How many undocumented symbols to list | | `--format ` | `text` | `text`, `markdown` (the GitHub Action's job summary and issue body) or `json` | | `--out ` | stdout | Write the report to a file; logs `wrote : ` on stderr | | `--fail-under ` | none | Exit 1 when the health score is below this (a whole number) | | `--fail-on-mismatch` | off | Exit 1 when any example line no longer fits the code | | `--no-index` | index on first use | Fail with `not indexed in the "" graph: ` instead of indexing | **Examples.** ```bash npx -y staleproof audit FalkorDB/falkordb-py FalkorDB/GraphRAG-SDK FalkorDB/docs npx -y staleproof audit encode/httpx --run 20 --format markdown --out audit.md npx -y staleproof audit acme/acme-client acme/acme-docs --fail-under 80 --fail-on-mismatch ``` **Prints.** The score and grade, the indexed commits, a summary, the deductions, every example line that no longer fits the code (`✗ : line · `, then the detail, its kind and the call), the failing runs, API coverage with the most used undocumented symbols, and for one repository the README badge Markdown. [Docs audit](audit.md#the-cli) shows a full report and the JSON shape. `--format`, `--run`, `--limit` and `--fail-under` are validated before anything runs: `unknown --format (text, markdown or json)`, `--run must be a whole number, got `, `--fail-under must be a score from 0 to 100, got `. **Exit status.** 1 when the score is below `--fail-under` (logs `health score is below --fail-under `) or, with `--fail-on-mismatch`, when any line no longer fits (logs ` example(s) no longer fit the code (--fail-on-mismatch)`); the report is printed or written first. Otherwise 0, whatever the score. ### `run` Run one docs code example in the sandbox, once at the indexed commits or before and after a pull request. ```text staleproof run [--pr ] [--fresh] [--json] ``` **Arguments.** ``: a CodeExample id, `/::`. | Option | Default | Meaning | |---|---|---| | `--pr ` | none | Run twice: with the pull request's repository installed at the commit before the pull request, then at the pull request. Other repositories are installed at their indexed commits. The pull request is read first if it has not been checked. Without `--pr`, the example runs once at the indexed commits. | | `--fresh` | off | Run even if an identical run (same program, same commits) is recorded, and replace the stored run. Only this command offers it. | | `--json` | off | Print the tool result as JSON: `{tool, summary, data: {example, url, verdict, runs}, queries}`. | **Examples.** ```bash npx -y staleproof run "FalkorDB/falkordb-py:README.md:4" --pr https://github.com/FalkorDB/falkordb-py/pull/262 npx -y staleproof run "FalkorDB/docs:commands/graph.explain.mdx:0" --pr https://github.com/FalkorDB/falkordb-py/pull/267 --fresh npx -y staleproof run "FalkorDB/docs:commands/graph.profile.mdx:0" --json ``` **Prints.** The example id and URL; per run, a status line (`PASS`, `FAIL`, `TIMEOUT`, `SKIPPED` or `ERROR`, then `before` or `after`, the `repo@sha` list installed, and `(recorded)` when answered from the graph), the examples stitched in front of it, the reason for a skip or error, and the traceback of a failure; then a one-line summary. The verdict is one of: | Verdict | Meaning | |---|---| | `breaks` | Passes before, fails after: the pull request breaks the example. | | `fixes` | Fails before, passes after. | | `passes` | Passes on every run. | | `already-broken` | Fails before and after with the same error: broken already, not by this pull request. | | `changed-failure` | Fails before and after with different errors. | | `fails` | Fails at the indexed commits (no `--pr`). | | `skipped` | Not run, with the reason. When the run before the pull request is skipped, the run after it is not attempted. | | `error` | The sandbox could not be prepared (for example an install conflict). Such runs are never answered from the graph. | **Notes.** - Examples are skipped, with the reason recorded, when they are not Python, need an LLM provider key, read a local file the docs do not provide, contain placeholders to fill in, wait for input, or call an external service. A fragment that uses names an earlier block on the page defines is run with those blocks stitched in front of it. See [Example runner](example-runner.md). - Each run gets a fresh virtualenv per set of commits (cached), a scratch FalkorDB that is flushed first, no network other than that server, and a timeout (`STALEPROOF_RUN_TIMEOUT_MS`, default 30 seconds). Runs execute one at a time. - Needs `git`, `tar`, and uv or `python3`. On a host without them, runs are reported as skipped and not recorded; runs recorded by other hosts are still answered from the graph. ### `decide` Record a maintainer decision. Future agent checks cite it instead of re-flagging the sections it governs. ```text staleproof decide --governs [--source ] [--author ] [--supersedes ] ``` **Arguments.** ``: the decision, in one sentence. | Option | Default | Meaning | |---|---|---| | `--governs ` | required | DocSection and/or Symbol ids the decision governs. At least one must exist in the graph. | | `--source ` | empty | Where it was decided: an issue, a pull request comment, an ADR. | | `--author ` | `unknown` | Who decided. | | `--supersedes ` | none | The decision this one replaces. The old one is marked `superseded`; an unknown id is ignored without an error. | **Examples.** ```bash npx -y staleproof decide "The async client keeps positional kwargs until 2.0" \ --governs "FalkorDB/falkordb-py:falkordb.asyncio.falkordb.FalkorDB.__init__" \ --source https://github.com/FalkorDB/falkordb-py/issues/ --author "" npx -y staleproof decide "The async client drops positional kwargs in 2.0" \ --governs "FalkorDB/falkordb-py:falkordb.asyncio.falkordb.FalkorDB.__init__" --supersedes dec- ``` **Prints.** `Recorded decision dec- governing node(s).`, with `, superseding ` when it replaces one. Fails with `none of is a DocSection or Symbol id; decision not recorded` when no id matches. **Notes.** Put the decision text before `--governs`: the option takes every value that follows it, up to the next option. [Decisions and memory](memory.md) explains how decisions are matched to sections. ### `verdict` Accept or reject a verdict. Rejected edits are left out of the patch, and later checks see the review. ```text staleproof verdict [--by ] ``` | Option | Default | Meaning | |---|---|---| | `--by ` | `$USER`, else `maintainer` | The reviewer recorded on the verdict. | **Example.** ```bash npx -y staleproof verdict accept ver- --by "" ``` **Prints.** ` accepted` or ` rejected`. Sets the verdict's `status`, `reviewed_by` and `reviewed_at`. Fails with `action must be accept or reject` or `no verdict `. ### `patch` Print the docs patch for a checked pull request: a unified diff and a pull request description with the evidence behind each edit. Optionally open it as a pull request. ```text staleproof patch [--out ] [--open-pr [--upstream | --in-repo [--base ] [--only ]]] ``` | Option | Default | Meaning | |---|---|---| | `--out ` | stdout | Write the diff to `` and the title and description to `.md`. | | `--open-pr` | off | Open the fix as a pull request, one per docs repository. Needs `GITHUB_TOKEN` (or `GH_TOKEN`). | | `--upstream` | off | With `--open-pr`: open it against the original repository instead of inside your fork. | | `--in-repo` | off | With `--open-pr`: push a branch to each docs repository itself and open the pull request there, with no fork: what a CI token can do. Cannot be combined with `--upstream`. | | `--base ` | the repository's default branch | With `--in-repo`: the branch the fix starts from and targets, for example the checked pull request's own branch. | | `--only ` | every edited repository | With `--in-repo`: only these repositories; edits in others are skipped (a CI token can write to its own repository only). | **Examples.** ```bash npx -y staleproof patch https://github.com/FalkorDB/falkordb-py/pull/267 npx -y staleproof patch https://github.com/FalkorDB/falkordb-py/pull/267 --out docs-267.diff npx -y staleproof patch --open-pr npx -y staleproof patch --open-pr --in-repo --base my-feature --only acme/acme-client ``` **Prints.** The diff, or `(no proposed edits)`, then `# docs: update for # ()` and the description: per edited section, the reason, the evidence path, the example runs and the cited decisions; then the sections checked and left unchanged with their verdicts; then any edit that could not be applied. With `--out`, it writes the two files and logs `wrote and .md ( files)` on stderr. With `--open-pr`, it prints `opened (fork)`, `opened (upstream)` or `opened (same-repo)` per docs repository; when `--only` leaves nothing, it logs `no pull request opened: no proposed edit is in a repository this token may write to`. **Notes.** - The patch uses the latest verdict on each section from any check, MCP client or API call. An edit is a `stale` verdict with replacement text that a maintainer has not rejected. Graph mode writes no verdicts, so a graph-mode check alone gives an empty patch. - Each file is read at the commit the graph indexed, from the local clone when there is one, otherwise from `raw.githubusercontent.com`. An edit whose section changed since indexing, or whose file is missing at that commit, is not applied and is listed instead. Sections longer than 8,000 characters always conflict, because the graph stores only their first 8,000 characters. - Diff paths are relative to each docs repository. When the patch spans several repositories, apply each file's part in its own repository with `git apply`, or use `--open-pr`. - `--open-pr` forks each docs repository into your account (always, even with `--upstream`), syncs the fork, creates or resets the branch `staleproof/--` (named after the code pull request) at the indexed commit, commits each edited file, and opens the pull request. It fails with `opening a pull request needs GITHUB_TOKEN` without a token and with `the patch is empty: no proposed edits to open a pull request for` when there is nothing to open. - `--in-repo` creates or resets the branch `staleproof/docs---` in the docs repository at the head of `--base` (default: its default branch), commits each edited file, and opens a pull request into `--base`, or reuses the open one from that branch. The token needs `contents: write` and `pull-requests: write` on that repository. This is how the GitHub Action's `open-docs-pr` puts the docs fix into the checked pull request's branch ([GitHub Action](github-action.md#open-the-docs-fix-as-a-pull-request)). - Fails with ` has not been checked` for a pull request the graph has never read. ### `mcp` Serve the tools over the Model Context Protocol on stdio. ```text staleproof mcp ``` **Example.** ```bash claude mcp add staleproof -- npx -y staleproof mcp ``` **Prints.** On stderr, `staleproof mcp: serving graph "" from ` (or `serving each repository's owner graph`) with any password masked, and `staleproof: ` lines while it indexes. Stdout carries the protocol. It connects to FalkorDB at start ([FalkorDB on first use](#falkordb-on-first-use)) and uses the graph named by `--org` or `STALEPROOF_ORG`, else the graph of the owner of the repository each call is about (as `check` does). `check_pull_request`, `impact` and `clearance` index the pull request's repository into that graph the first time they see it; `audit_docs` and `docs_for_symbol` index the repositories named in `repos`, and `check_changes` the repository it checks (named in `repo`, or the `origin` of the checkout in `dir`, default the directory the server runs in). [MCP server](mcp.md) covers clients, tools, the hosted server and security. ### `serve` Serve the web UI and the HTTP API. ```text staleproof serve [--port ] [--host ] [--preload ] [--hosted] ``` | Option | Default | Meaning | |---|---|---| | `--port ` | `PORT`, else `8080` | Port to listen on. | | `--host ` | `HOST`, else `127.0.0.1` | Interface to bind. `0.0.0.0` accepts connections from other machines. | | `--preload ` | `STALEPROOF_PRELOADED` (comma-separated), else falkordb-py #262, #267 and #276 | Pull requests listed on the home page. | | `--hosted` | off, unless `STALEPROOF_HOSTED=on` | Hosted mode, as on staleproof.dev: a pull request in any public repository the served graph does not hold is downloaded and indexed on demand into a workspace graph of its owner (`gh-`). See [Deploy](DEPLOY.md#hosted-mode). | **Example.** ```bash npx -y staleproof serve --host 0.0.0.0 --port 8080 ``` **Prints.** On stderr, `staleproof serve: http://: (graph "", agent on, ; MCP at /api/mcp)`, with `, hosted: any public repository on demand` after the graph in hosted mode, and `agent off: no model credentials, graph mode only` when no model credentials are found. Agent mode is on when `ANTHROPIC_API_KEY` or another credential the Anthropic SDK finds is present. **Notes.** It connects to FalkorDB before it listens ([FalkorDB on first use](#falkordb-on-first-use)). The server runs one check at a time and queues the rest. It serves MCP over Streamable HTTP at `/api/mcp`. Its POST endpoints other than `/api/mcp` are rate-limited per client (30 an hour) and it has no authentication. An agent check in the server has no time limit unless `STALEPROOF_CHECK_BUDGET_MS` is set (it defaults to 240 seconds on Vercel). See [HTTP API](http-api.md) and [Deploy](DEPLOY.md). ### `facts` Compute every number Staleproof quotes from the graph and write it as Markdown. ```text staleproof facts [--out ] [--json] ``` | Option | Default | Meaning | |---|---|---| | `--out ` | `docs/FACTS.md` | Output file, relative to the current directory. The directory must exist. | | `--json` | off | Print the facts as JSON instead of writing Markdown. | **Example.** ```bash npx -y staleproof facts --out FACTS.md ``` **Prints.** `wrote ` on stderr. The file lists the repositories, node and edge counts, resolution quality, dependencies between repositories, sample cross-repository paths, the pull requests checked, the grep baseline per pull request, the example runs (the latest attempt of each) and the impact query's server-side timing. [Facts from the graph](FACTS.md) is this file for FalkorDB's repositories. ### `db` Run a local FalkorDB without Docker, from the redis-server and FalkorDB module binaries in the `falkordblite` Python wheel. ```text staleproof db [--port ] ``` | Option | Default | Meaning | |---|---|---| | `--port ` | `6379` | Port to start, stop or check. | **Examples.** ```bash npx -y staleproof db start npx -y staleproof db status npx -y staleproof db stop --port 6390 ``` **What it does.** Runs `bash scripts/falkordb-local.sh ` from the package. The first use installs `falkordblite==0.10.0` (override with `FALKORDBLITE_VERSION`) into `$STALEPROOF_HOME/falkordb/venv` with uv, or with `python3 -m venv` and pip when uv is missing; the wheel needs Python 3.12 or newer. `start` launches a daemonised server bound to `127.0.0.1`, keeping RDB snapshots in `$STALEPROOF_HOME/falkordb/data-` (saved every 60 seconds after a change, and on `stop`). **Prints.** `falkordb-local: running on port (data: )`, `falkordb-local: already running on port `, `falkordb-local: stopped port `, `falkordb-local: nothing running on port `, or for `status`, `running on port ` or `not running on port ` (exit status 1). When the script fails, the CLI prints `staleproof: Command failed: bash …` and exits with 1. The undocumented action `bin` prints the directory holding `redis-server` and `redis-cli`. **Notes.** The scratch server that examples run against is started with the same script on port 6390, unless `STALEPROOF_SCRATCH_URL` names another server. ## Environment variables The variables most commands read. [Configuration](configuration.md) lists all of them. | Variable | Default | Used for | |---|---|---| | `FALKORDB_URL` | `falkor://localhost:6379` | The graph's server (`--db`). | | `STALEPROOF_ORG` | the owner of the repository the command is about, else `falkordb` | The graph name (`--org`). | | `STALEPROOF_HOME` | `~/.staleproof` | Clones, downloaded tarballs, the GitHub response cache, virtualenvs, the local FalkorDB. | | `GITHUB_TOKEN`, else `GH_TOKEN` | none | GitHub API requests (60 an hour without a token), clones, tarball downloads and file reads, so private repositories work; required by `patch --open-pr`. | | `ANTHROPIC_API_KEY`, `ANTHROPIC_BASE_URL`, `STALEPROOF_MODEL`, `STALEPROOF_EFFORT` | none, none, `claude-sonnet-5-5`, `medium` | Agent mode. | | `STALEPROOF_SCRATCH_URL`, `STALEPROOF_SCRATCH_PORT` | none, `6390` | The disposable FalkorDB examples run against. A URL with credentials is refused. | | `STALEPROOF_RUN_TIMEOUT_MS` | `30000` | Timeout for each example run. | ## Related - [Quickstart](getting-started.md) - [Configuration](configuration.md) - [Tools reference](tools.md) - [GitHub Action](github-action.md) - [Troubleshooting and FAQ](troubleshooting.md) --- # GitHub Action The composite action in [action.yml](../action.yml), `uses: rajkaria/staleproof@v1`, comments on every pull request with the docs it can make false, the evidence path for each, the examples whose calls no longer fit the changed code, and the result of running the affected docs examples before and after the change. On a schedule it audits the whole docs instead and keeps one issue up to date. It needs no inputs, no secrets and no database: it starts a FalkorDB inside the job and indexes the repository itself. This page is for maintainers adding it to a repository: installing it, what a run does, the inputs and outputs, permissions, agent mode and the docs fix pull request, the scheduled audit, keeping a graph between runs, and the comment format. Status: it has run end to end on a private test repository, from the one-click workflow with no inputs: the comment named a README example the pull request broke, and updated in place when the same pull request fixed the README. ## Install it ### One click from the install page 1. Open [staleproof.dev/#/install](https://staleproof.dev/#/install) and type the repository (`owner/name`, or paste a repository or pull request URL). 2. Optionally, under Options: a docs repository to index with it, agent mode, failing the check when an example breaks, and a weekly audit of the whole docs (adds a `schedule` trigger and `issues: write`). 3. Click **Add to owner/repo**. GitHub opens its "new file" page with `.github/workflows/staleproof.yml` filled in. Click **Commit changes**, to the default branch or as a pull request. The next pull request gets the comment. Agent mode also needs your model key as the repository secret `ANTHROPIC_API_KEY`; the page links to the repository's secrets page when you tick it. ### Or copy the workflow Add [examples/staleproof.yml](../examples/staleproof.yml) as `.github/workflows/staleproof.yml`; without its header comment, it reads: ```yaml name: Staleproof on: pull_request: permissions: contents: read pull-requests: write jobs: docs: runs-on: ubuntu-latest steps: - uses: rajkaria/staleproof@v1 # with: # also-index: your-org/docs # docs in another repository # anthropic-api-key: ${{ secrets.ANTHROPIC_API_KEY }} # the agent judges each section and proposes edits # fail-on: breaks # fail when an example passes before and fails after ``` `pull_request` with no `types` runs on GitHub's default types: `opened`, `synchronize` and `reopened`. The workflow needs no checkout step: Staleproof fetches the repository itself, with the job's token. For a weekly audit of the whole docs as well, add [examples/staleproof-audit.yml](../examples/staleproof-audit.yml) as a second workflow ([Audit the docs on a schedule](#audit-the-docs-on-a-schedule)). ## What a run does The steps of [action.yml](../action.yml), in order, on a pull request (`mode: check`; in [audit mode](#audit-the-docs-on-a-schedule) the audit's steps replace steps 6 to 10): 1. **Set up Node.js and uv**: `actions/setup-node@v7` with Node 22, and `astral-sh/setup-uv@v10.3.0` (the example runner builds its virtualenvs with uv). 2. **Start FalkorDB**, unless `falkordb-url` is set. With Docker on the runner (GitHub's Ubuntu runners have it), two containers from `falkordb/falkordb:latest`: `staleproof-graph` on port 6379 holds the graph, and `staleproof-scratch` on port 6390 is the disposable server the docs examples run against. The step waits up to 90 seconds for both ports, then sets `FALKORDB_URL=falkor://localhost:6379` and `STALEPROOF_SCRATCH_URL=falkor://localhost:6390` for the next steps. Without Docker it starts nothing, and the CLI starts both servers itself from falkordblite. 3. **Install staleproof**: `npm install -g staleproof@` (default `0.3.0`). With `version: source`, the CLI is built from the action's own checkout instead (`npm ci`, then `tsup`). When the version cannot be installed from npm, the step warns `staleproof@ is not on npm; building it from this action's source instead.` and builds it the same way. 4. **Choose check or audit**: with `mode: auto` (the default), `check` when there is a pull request (the `pull_request` event, or `pr-url`), else `audit`. Any other value than `auto`, `check` or `audit` fails with `mode must be auto, check or audit (got )`. 5. **Re-index**, only with `reindex: 'true'` and `falkordb-url` set: `staleproof index `. See [Keep the graph between runs](#keep-the-graph-between-runs). 6. **Check the pull request**: ```bash staleproof check "$PR_URL" --no-agent --runs "$RUNS" --format markdown --out "$RUNNER_TEMP/staleproof-report.md" [--with ] ``` `--no-agent` is left out when `anthropic-api-key` is set. When the graph does not have the pull request's repository yet, the CLI first indexes it together with the `also-index` repositories, at their default branch heads; with the default throwaway FalkorDB that happens on every run. Then it runs the check (graph mode: impact, clearance, ranking, then up to `runs` examples before and after the pull request, those predicted to break first), writes the report, and the step appends the report to the job summary and reads the examples that break and the examples predicted to break from the report's last line (``). 7. **Post or update the comment**, when `comment` is `'true'`: the GitHub CLI lists the pull request's comments (all pages), takes the first whose body starts with `` (or ``, the marker from before the rename) and replaces its body with the report. If there is none, it posts a new comment. This step runs even when an earlier step failed, as long as a report was written. 8. **Open the docs fix as a pull request**, only with `open-docs-pr: 'true'`, an `anthropic-api-key`, and a pull request from this repository (not a fork), when the agent proposed edits: see [Open the docs fix as a pull request](#open-the-docs-fix-as-a-pull-request). 9. **Fail on broken examples**, with `fail-on: breaks` or `predicted`: the step fails with `This pull request breaks docs example(s): they pass before it and fail after. See the report in the job summary.` when the report has any. 10. **Fail on predicted breaks**, with `fail-on: predicted`: the step fails with `A static check predicts this pull request breaks docs example(s): their calls no longer fit the changed code. See the report in the job summary.` when any example is predicted to break. Because the comment is found by its first-line marker, each run edits the same comment instead of adding a new one. When the check command fails without writing a report (for example a pull request URL it cannot read), the check step fails with `staleproof check failed (exit ); see the log above.` When it fails after writing one (an agent check that ends `error` or `refused`), the step warns `staleproof check ended with exit ; the report may be incomplete.`, passes, and the comment is posted. Run outside a pull request event without `pr-url`, the step fails with `No pull request to check: run this on pull_request events, or set pr-url.` ## Inputs Every input is optional. | Input | Default | Meaning | |---|---|---| | `mode` | `auto` | `auto`: check the pull request when there is one (`pull_request` events, or `pr-url`), audit the docs otherwise (`schedule`, `workflow_dispatch`, `push`). `check` or `audit` forces one. | | `also-index` | `''` | Other repositories to index together with this one, space- or comma-separated, for example your docs site `acme/docs`, so calls and doc links across them resolve. Passed to `check --with`, audited with this repository in audit mode, and passed to `index` with `reindex`. | | `anthropic-api-key` | `''` | Selects the mode. Empty: graph mode (`--no-agent`). Set: agent mode, passed as `ANTHROPIC_API_KEY`. | | `model` | `''` | Agent mode's model, passed as `STALEPROOF_MODEL`. Empty: `claude-sonnet-5-5`. Set it whenever you set `base-url`. | | `base-url` | `''` | An Anthropic-compatible API for agent mode, passed as `ANTHROPIC_BASE_URL`, for example `https://api.deepseek.com/anthropic`. | | `runs` | `'3'` | Graph mode runs up to this many affected examples before and after the pull request; `'0'` runs none. Passed as `--runs`. | | `fail-on` | `never` | Check mode: `breaks` fails the job when an example passes before the pull request and fails after it; `predicted` also fails it when a static check predicts an example's call no longer fits the changed code. Audit mode: `mismatches` fails the job when any example no longer fits the code. | | `fail-under` | `''` | Audit mode: fail the job when the docs health score is below this (0-100). | | `audit-runs` | `'20'` | Audit mode: run up to this many Python examples at the default branch before scoring; `'0'` for static checks only. Passed as `audit --run`. | | `issue` | `'true'` | Audit mode: keep one issue up to date with the audit (opened when something needs fixing, updated on each run, closed when all clear). Needs `issues: write`. | | `open-docs-pr` | `'false'` | Agent mode, on a pull request from this repository: `'true'` opens the proposed docs edits as a pull request into the checked pull request's branch. Needs `contents: write` and `pull-requests: write`. | | `comment` | `'true'` | Post one pull request comment and keep updating it. Any other value: the report goes to the job summary only. | | `pr-url` | `''` | The pull request to check. Empty: the one that triggered the workflow (`github.event.pull_request.html_url`). | | `falkordb-url` | `''` | A FalkorDB that keeps the graph between runs (decisions, verdicts, run history). Empty: the throwaway FalkorDB the job starts. | | `org` | `''` | The graph name on `falkordb-url`; ignored without it. Empty: the CLI's default, the owner of the repository lowercased (the input's description in `action.yml` says `falkordb`, which is what that rule gives for FalkorDB's repositories). | | `reindex` | `'false'` | With `falkordb-url`: `'true'` re-indexes this repository and `also-index` at their default branch heads before the check. | | `github-token` | `${{ github.token }}` | Reads the repository (clones, the GitHub API, file contents) and posts the comment. Passed as `GITHUB_TOKEN` to Staleproof and as `GH_TOKEN` to the GitHub CLI. | | `version` | `'0.3.0'` | The `staleproof` npm version to run, or `source` to build the CLI from the action's checkout. When that version is not on npm, the CLI is built from the action's source. | ## Outputs | Output | Meaning | |---|---| | `report` | Check mode: path of the Markdown report, the comment's body: `$RUNNER_TEMP/staleproof-report.md` | | `breaks` | Check mode: how many docs examples pass before the pull request and fail after it | | `predicted` | Check mode: how many docs examples a static check predicts the pull request breaks (their call no longer fits the changed code) | | `score` | Audit mode: the docs health score, 0-100 | | `mismatched` | Audit mode: how many examples no longer fit the code | `breaks` and `predicted` are read from the report's counts line (``); a report from a CLI older than 0.3.0 has none, and `breaks` then counts the rows marked `**breaks**`. `score` and `mismatched` come from the audit report's last line (``). ```yaml - uses: rajkaria/staleproof@v1 id: staleproof - if: steps.staleproof.outputs.breaks != '0' run: echo "${{ steps.staleproof.outputs.breaks }} docs example(s) break" >> "$GITHUB_STEP_SUMMARY" ``` ## Permissions | Need | Why | |---|---| | `contents: read` | Read the repository. | | `pull-requests: write` | Create and update the comment. | | `issues: write` | Audit mode with `issue: 'true'`: open, update and close the audit issue. | | `contents: write` | `open-docs-pr: 'true'`: push the docs fix branch (with `pull-requests: write` to open its pull request). | **Private repositories** work with the default token. Staleproof sends it to `https://github.com/` for clones (through git's environment config, never in the remote URL or on the command line), to the GitHub API and to `raw.githubusercontent.com`. The default token reads only the repository the workflow runs in: a private `also-index` repository elsewhere needs a token that can read it, passed as `github-token` (the comment is then posted as that token's account). **Pull requests from forks** get a read-only token and no repository secrets. The check still runs, in graph mode (an `anthropic-api-key` from a secret is empty there), and the report goes to the job summary; the comment step ends with the warning `Could not comment on the pull request (pull requests from forks get a read-only token). The report is in the job summary.` ## Agent mode and graph mode | | Graph mode (no key, the default) | Agent mode (`anthropic-api-key` set) | |---|---|---| | Command | `check --no-agent --runs --format markdown` | `check --runs --format markdown` (`--runs` applies to graph mode only; the agent chooses its own runs) | | What runs | Impact (with the examples predicted to break), clearance and PageRank ranking, then up to `runs` examples (default 3) before and after the pull request: those predicted to break first, then examples from exactly linked sections. No model. | The full agent check: the model reads sections, runs examples on the runner, checks decisions and records a verdict or an edit for every impacted section. | | What is stored | A Check node (model `graph-only (no model)`) with the trace and the runs | A Check node with the trace, verdicts and runs | | Comment | The examples predicted to break, the examples run before and after, the impacted sections, cleared symbols, and any verdicts recorded earlier | The same, with this check's verdicts and the agent's summary; with `open-docs-pr`, the fix as a pull request | | Cost | Runner minutes only | Model tokens; minutes per pull request (a 43-section check takes about 6 minutes) | What is stored lives in the job's throwaway FalkorDB and disappears with it, unless you set `falkordb-url`. Agent mode with an Anthropic-compatible provider such as DeepSeek: ```yaml - uses: rajkaria/staleproof@v1 with: anthropic-api-key: ${{ secrets.ANTHROPIC_API_KEY }} # the provider's key base-url: https://api.deepseek.com/anthropic model: deepseek-v4-pro ``` Set `model` whenever you set `base-url`: a compatible provider refuses the default `claude-*` name, and the check then fails before it writes a report. ## Open the docs fix as a pull request In agent mode the model proposes replacement text for the sections a pull request makes false. With `open-docs-pr: 'true'`, the action turns those edits into a second pull request into the checked pull request's own branch, so the docs fix lands with the code change: ```yaml permissions: contents: write pull-requests: write jobs: docs: runs-on: ubuntu-latest steps: - uses: rajkaria/staleproof@v1 with: anthropic-api-key: ${{ secrets.ANTHROPIC_API_KEY }} open-docs-pr: 'true' ``` The step runs only when the check proposed edits (the comment then ends its verdicts with `Proposed edits: …`), the pull request comes from this repository (a fork's head branch cannot be written to), and agent mode is on. It runs ```bash staleproof patch "$PR_URL" --open-pr --in-repo --base "" --only "" ``` which creates or resets the branch `staleproof/docs---` at the head of the pull request's branch, commits each edited file in this repository, and opens a pull request from it into the pull request's branch (or reuses the open one). Edits in other repositories (an `also-index` docs site) are left to `staleproof patch` by hand, since the job's token can write only to its own repository. When the step cannot open it, it warns `Could not open the docs fix pull request (the job needs contents: write and pull-requests: write).` and the job continues. GitHub does not start workflows for pull requests a workflow's own `GITHUB_TOKEN` opens, so the docs fix pull request gets no checks of its own unless you pass another token as `github-token`. ## Audit the docs on a schedule With no pull request (a `schedule`, `workflow_dispatch` or `push` event, or `mode: audit`), the action audits the docs instead: every example's calls checked against the code, examples run at the default branch, and API coverage, scored 0 to 100 ([Docs audit](audit.md)). [examples/staleproof-audit.yml](../examples/staleproof-audit.yml), without its header comment: ```yaml name: Staleproof docs audit on: schedule: - cron: '0 6 * * 1' workflow_dispatch: permissions: contents: read issues: write jobs: audit: runs-on: ubuntu-latest steps: - uses: rajkaria/staleproof@v1 # with: # also-index: your-org/docs # docs in another repository # audit-runs: 20 # examples to run (0: static checks only) # fail-on: mismatches # fail the job when an example no longer fits the code ``` The steps after **Choose check or audit**: 1. **Audit the docs**: `staleproof audit --run --format markdown --out "$RUNNER_TEMP/staleproof-audit.md"`. With the default throwaway FalkorDB this indexes the repositories at their default branch heads first. The report goes to the job summary, and the step reads `score`, `mismatched` and the number of failing runs from its last line. When the command writes no report, the step fails with `staleproof audit failed (exit ); see the log above.` 2. **Keep the audit issue up to date**, when `issue` is `'true'`: the GitHub CLI looks for an open issue whose body starts with ``. - Found: its title becomes `Docs health /100: example(s) no longer fit the code` and its body the new report. When nothing needs fixing any more (no mismatched example and no failing run), it adds the comment `All clear: every docs example fits the code again. Closing.` and closes the issue. - Not found, and something needs fixing: it opens one with that title and body. Without `issues: write` it warns `Could not open the audit issue (the job needs issues: write). The report is in the job summary.` - Not found, and nothing to fix: `Nothing to fix: no issue opened.` 3. **Fail on the audit**, with `fail-on: mismatches` when an example no longer fits the code, or with `fail-under` when the score is below it: `Docs health /100; example(s) no longer fit the code. See the report in the job summary.` A low API coverage lowers the score but never opens the issue on its own: the issue tracks examples that a reader would copy and see fail. The issue is matched by its body's first line, so editing its title by hand is fine; after you close it by hand, the next run that finds something opens a new one. ## Keep the graph between runs By default every job starts with an empty FalkorDB, indexes the repository and forgets everything at the end: decisions, verdicts, their review and recorded runs do not carry over. To keep them, point `falkordb-url` at a FalkorDB the runner can reach (FalkorDB Cloud works; see [Deploy](DEPLOY.md)), stored as a secret: ```yaml - uses: rajkaria/staleproof@v1 with: falkordb-url: ${{ secrets.STALEPROOF_FALKORDB_URL }} org: acme also-index: acme/acme-docs ``` With `falkordb-url`: - The Start FalkorDB step does nothing. Examples run against a scratch server the CLI starts on the runner itself (falkordblite on `127.0.0.1:6390`), never against your graph's server. - The repository is indexed on the first check only; later runs reuse the graph. - A pull request is diffed on its first check, and again whenever its head has moved since (on `synchronize`, after new commits are pushed), so the comment follows the pull request. A merged pull request is never re-diffed. - `reindex: 'true'` re-indexes this repository and `also-index` before each check, at their default branch heads (no `--ref` is passed, so not at the pull request's base branch). They are indexed together, but without the graph's other repositories: calls and doc links into repositories not listed are re-resolved away until you index everything together again. Set `org` whenever you set `reindex`: that step passes `org` as `STALEPROOF_ORG`, and an empty value fails with `invalid org name: ""`. For a graph of several repositories, leave `reindex` off and re-index every repository together on a schedule instead, from any one repository, for example as `.github/workflows/staleproof-index.yml`: ```yaml name: Staleproof index on: schedule: - cron: '0 3 * * *' workflow_dispatch: permissions: contents: read jobs: index: runs-on: ubuntu-latest steps: - uses: actions/setup-node@v4 with: node-version: 22 - name: Re-index every repository in one command env: FALKORDB_URL: ${{ secrets.STALEPROOF_FALKORDB_URL }} STALEPROOF_ORG: falkordb run: npx -y staleproof@0.3.0 index FalkorDB/falkordb-py FalkorDB/GraphRAG-SDK FalkorDB/docs ``` This workflow uses only the documented `index` command; unlike the action itself, it has not been run on GitHub. Re-indexing never deletes: what disappeared is marked `gone`, and verdicts and decisions keep pointing at it. ## Pin the version `@v1` follows the action's `v1` tag, and the `version` input pins the CLI it installs (default `0.3.0`). For runs that cannot change under you, pin the action to a commit and the CLI to a version: ```yaml - uses: rajkaria/staleproof@ with: version: 0.3.0 ``` ## The comment The comment is the report that `check --format markdown` renders ([src/format/report.ts](../src/format/report.ts) and [src/format/markdown.ts](../src/format/markdown.ts)); it leads with what broke. The MCP tool `check_pull_request` returns the same report. Its shape, with placeholders in angle brackets: ```markdown ### Staleproof: <n> changed symbols (and <m> added) reach <total> doc sections (<b> breaking, <a> signature, <c> behavior; <x> in other repositories). <p> examples are predicted to break, the call no longer fitting the changed code (<path>:<index>, …). **Predicted to break** — the call as written no longer fits the code after this pull request (a static check, before anything runs): | Example | Call | Why | |---|---|---| | [<path>:<index>](<url of the line>) | `<call>` | `<Class.method>` requires `<param>`, which the example does not pass | | Example | Before (`<before>`) | After (`<after>`) | Result | |---|---|---|---| | [<path>:<index>](<url>) | passes | `<error>` | **breaks**: passes before this PR, fails after | | Change | Doc section | Repository | Evidence path (changed symbol ← … ← doc) | Hops | |---|---|---|---|---| | **breaking** | [<heading>](<url>) (page edited in this PR) | <owner>/<repo> | `<changed symbol>` ← `<caller>` ← `<doc>` | <hops> (<exact|by-name|late>) | _<k> more sections not shown._ <details><summary><n> impacted example line(s) already did not fit the code before this pull request</summary> - [<path>:<index> line <line>](<url>): <what does not fit> </details> <details><summary>Cleared: <n> changed symbol(s) reach no doc within 3 call hops</summary> - `<qualname>` (<change>) — <t> raw text matches in the docs; check with `git grep -nw <name> -- '*.md' '*.mdx' '*.rst'` </details> #### Verdicts | Verdict | Section | Reason | |---|---|---| | <ok|stale|governed|skipped> | [<heading>](<url>) | <reason> | Proposed edits: `npx staleproof patch <pr-url>` This pull request also edits `<path>`, …: examples on those pages are run after the change as the pull request leaves them; the sections above are linked as they are on the default branch. <the agent's summary, in agent mode> <sub>[Staleproof](https://github.com/rajkaria/staleproof) · FalkorDB graph `<org>` · <before>..<after></sub> <!-- staleproof:counts breaks=<n> predicted=<n> sections=<n> mismatched=<n> --> ``` | Part | Content | |---|---| | Marker | `<!-- staleproof -->` on the first line, invisible on GitHub. The post step finds the comment by it. | | Title | `this pull request breaks <n> docs example(s)` when any example breaks; otherwise `this pull request is predicted to break <n> docs example(s)` when a static check predicts any; otherwise, when every impacted section is on a page the pull request edits itself, `the <n> doc sections this pull request reaches are on pages it edits`; otherwise `<total> doc section(s) this pull request can make false`; or `no docs reach the code this pull request changed`. | | Summary | The impact summary: changed symbols, sections reached, the count per severity, how many sections are in other repositories, how many examples are predicted to break and how many impacted examples already did not fit the code. When the pull request changed no symbol of the indexed code, it says so and the sections table is omitted. | | Predicted to break | One row per example line whose call no longer fits the code after the pull request, linked to the line: a call to a removed or renamed symbol, an import of a name the package stops exporting, or a call the new signature rejects that the old one accepted (an unknown keyword, too many positional arguments, a new required argument). `(matched by name)` marks a removed symbol linked by name only. Found without running anything, so it works with `runs: '0'` and on the hosted service. Omitted when there are none. [Predicted breaks](concepts.md#predicted-breaks) has the rules. | | Examples table | One row per example with a run recorded against this pull request, by this check or an earlier one, using the latest attempt on each side. **Before** and **After** are `passes`, the error's last line, `skipped (<reason>)`, `not installed (<reason>)`, or `—` when that side did not run. **Result** is `**breaks**: passes before this PR, fails after`, `fixed by this PR`, `passes before and after`, `already broken (fails before and after)`, `fails before and after, with a different error`, `fails`, `not run` (skipped) or `could not be installed`, followed by `(after: the page as this PR edits it)` for an example on a page the pull request edits. Omitted when no example ran. | | Sections table | One row per impacted section; a section on a page the pull request edits is marked `(page edited in this PR)`. **Change** is `**breaking**` (removed, renamed, or a removed export), `**signature**` or `behavior` (body changed). **Evidence path** lists the nodes from the changed symbol to the doc. **Hops** is the path length and its confidence: `exact` when every edge was resolved through imports and types, `by-name` or `late` otherwise. Rows are in PageRank order of their page. Only the first 40 sections are rendered; the rest are counted below the table. | | Already out of date | Collapsed. Impacted example lines that did not fit the code before the pull request either: drift the pull request did not cause, worth fixing anyway. Omitted when there are none. | | Cleared | Collapsed. The changed symbols with no path to any doc within 3 call hops, each with its raw text-match count and the `git grep` command to check it. Omitted when nothing is cleared. | | Verdicts | The latest verdict per section for this pull request, from any source. Followed by the `patch` command when any verdict is `stale`. Omitted when there are none. | | Edited pages | When the pull request edits Markdown pages of its own repository, a line naming them: their examples are run after the change as the pull request leaves them ([Example runner](example-runner.md#pull-requests-that-edit-the-page)), and the section links point at the default branch. | | Agent summary | The model's closing summary, in agent mode only. | | Footer | A link to Staleproof, the graph name and the two commits compared, then the counts line the action's outputs are read from (invisible on GitHub; `mismatched` counts lines). | ## Known limits - The pull request check has run end to end on a private test repository (top of this page), and audit mode has run on GitHub too. For the rest, the tests in the repository check that the action's commands and flags exist in the CLI, that no input is required, that the default `version` is the package's own, that the comment marker matches, that the counts and audit lines parse into the outputs, and that the audit workflow in `examples/` runs on a schedule with `issues: write` and no secrets. - Code is read as Python, or TypeScript and JavaScript in published npm packages; docs as Markdown, MDX or reStructuredText. Only Python examples are run, and only Python calls get argument checks. - Predicted breaks are static: they follow calls resolved through imports and types, and stay silent where the source cannot settle the call (a decorator that may change the signature, a splat, a dynamic class). A pull request with no predicted break can still break an example; the runs are the proof. - The audit issue is looked up among the first 100 open issues. - With the default FalkorDB, every run indexes the repository again and starts without decisions, verdict history or recorded runs. - Graph mode samples: it runs at most `runs` examples (predicted breaks first, then exactly linked sections on the first impact page) and writes no verdicts. - The sections table shows the first 40 impacted sections; the rest are counted, not listed. - The raw text-match counts in the cleared list cover the docs repositories cloned on the runner: the ones the check indexed there. - A FalkorDB at `falkordb-url` must be reachable from the runner: a public endpoint for GitHub-hosted runners, or a self-hosted runner on your network. ## Security notes - **Both modes run code from the pull request.** Up to `runs` examples are installed from source at the pull request's commits and executed on the runner. The install step, which runs the pull request's build code, gets an allow-listed environment without `GITHUB_TOKEN`, the model key or the FalkorDB URL; the example itself inherits no environment and has no network other than the scratch FalkorDB. That is a guard, not a security boundary ([Example runner](example-runner.md#the-sandbox)). Set `runs: '0'` to run none: graph mode then only reads the diff. - **Stay on `pull_request`.** GitHub gives `pull_request` runs for forks a read-only token and no secrets. Do not switch to `pull_request_target` to get a writable token for forks: it runs with your secrets and a write token, and the job would still install and run the fork's code. - The action writes to GitHub only: the one pull request comment; in audit mode, the one audit issue (with `issue: 'true'`); and with `open-docs-pr: 'true'`, one branch and one pull request into the checked pull request's branch. Without `open-docs-pr`, opening a docs pull request is a manual `patch --open-pr`. - **`open-docs-pr` writes to your repository.** It needs `contents: write`, so the job's token has it while the job installs and runs the pull request's examples. The install step and the examples never receive the token, but the sandbox is a guard, not a security boundary. The step is skipped for pull requests from forks, which get a read-only token anyway. ## Related - [Install](install.md) - [CLI](cli.md) - [Use cases](use-cases.md) - [Indexing](indexing.md) - [Deploy](DEPLOY.md) - [Troubleshooting and FAQ](troubleshooting.md) --- <!-- page: MCP server · Integrate · source: https://staleproof.dev/docs/mcp.md --> # MCP server Staleproof's tools over the Model Context Protocol, so Claude Code, Claude Desktop, claude.ai, Cursor, VS Code or any MCP client can ask what a pull request can make false, run the affected examples, record verdicts and get the docs patch; and, before there is a pull request, which docs depend on a function it is about to change, which docs its edits make false, and how the docs stand overall. Use the hosted server with nothing installed, or run `staleproof mcp` on your machine over stdio. This page covers both, what the server exposes, how results and errors look, a worked session and security notes. ## Hosted: nothing to install The hosted server answers at `https://staleproof.dev/api/mcp` over Streamable HTTP. It is stateless: every request gets a fresh server, there are no sessions, and responses are plain JSON. | Client | How | |---|---| | Claude Code | `claude mcp add --transport http staleproof https://staleproof.dev/api/mcp` | | Cursor | The one-click **Add to Cursor** button on [staleproof.dev/#/install](https://staleproof.dev/#/install), or this in `~/.cursor/mcp.json`: `{"mcpServers": {"staleproof": {"url": "https://staleproof.dev/api/mcp"}}}` | | VS Code | The **Add to VS Code** button on the install page, or `code --add-mcp '{"name":"staleproof","type":"http","url":"https://staleproof.dev/api/mcp"}'` | | Claude Desktop and claude.ai | Add a custom connector (Settings, Connectors) with the URL `https://staleproof.dev/api/mcp` | | Any other client | Its setting for a remote (Streamable HTTP) server, with that URL | Then ask, for example: "Which docs can https://github.com/FalkorDB/falkordb-py/pull/276 make false? Show the evidence path for each." On the hosted server: - **Any public repository is indexed on first use.** `check_pull_request`, `impact` and `clearance` index the pull request's repository when no graph has it, into the workspace graph of its owner (`gh-<owner>`), and so do `check_changes` (its `repo`), `docs_for_symbol` (the first of `repos`, or the `owner/repo:` prefix of the symbol id) and `audit_docs` (the first of `repos`). A workspace is indexed again at its default branch head once it is more than 7 days old, or when an older Staleproof indexed it. FalkorDB's own repositories answer from the curated `falkordb` graph. The limits are in [Deploy](DEPLOY.md#hosted-mode). - **Every other tool goes to the graph of the repository it names**: the `owner/repo:` prefix of its ids. A repository that is not indexed yet fails with `<owner/repo> is not indexed yet: call check_pull_request with one of its pull requests first`. A call that names no repository fails with `name a repository: pass a pull request URL (pr_url), repo or repos ("owner/name"), or ids that start with owner/repo:`, except `decisions_for` and `write_decision`, which then use the curated graph. - **No examples run.** `run_example` is not offered, because the service has no sandbox, and `check_pull_request` with `run_examples` executes nothing there. Run examples with the local server, the [CLI](cli.md#run) or the [GitHub Action](github-action.md). - **It cannot read your disk.** `check_changes` takes the edited files' contents (`files`); `dir` fails with `send the changed files: files: [{path, content}] with repo (this server cannot read your disk)`. ## Local over stdio `staleproof mcp` serves the tools on stdio from your machine, `run_example` included. ### Claude Code ```bash claude mcp add staleproof -- npx -y staleproof mcp ``` With no `FALKORDB_URL`, the server uses the FalkorDB on `localhost:6379`, and starts a local one there (falkordblite) when nothing answers. `check_pull_request`, `impact` and `clearance` index the pull request's repository the first time they see it, at its default branch head, so nothing has to be set up first. `audit_docs` and `docs_for_symbol` index the repositories named in `repos`; `check_changes` indexes the repository it checks: `repo`, or the `origin` remote of the checkout in `dir` (default: the directory the server was started in, usually your project). To use a FalkorDB elsewhere, or another graph: ```bash claude mcp add staleproof -e FALKORDB_URL='falkor://falkordb:<password>@<host>:<port>' -e STALEPROOF_ORG=acme -- npx -y staleproof mcp ``` Use `falkors://` for an endpoint with TLS. Add `-e GITHUB_TOKEN=<token>` for private repositories and to raise GitHub's limit of 60 API requests an hour, which the first look at each pull request uses. Check the server with `claude mcp list`. The stdio server uses the graph named by `--org` or `STALEPROOF_ORG` when one is set, else, like the CLI, the graph named after the owner of the repository each call is about (for `check_changes` on a checkout, the owner of its `origin`), so `check` and the MCP tools read the same graph. Repositories it indexes on first use go into that graph. ### Claude Desktop, Cursor and other clients Any client that launches a stdio server from a command, arguments and environment works. The configuration is the same everywhere: ```json { "mcpServers": { "staleproof": { "command": "npx", "args": ["-y", "staleproof", "mcp"], "env": { "STALEPROOF_ORG": "falkordb" } } } } ``` | Client | Where it goes | |---|---| | Claude Desktop | `claude_desktop_config.json`, opened from Settings, Developer, Edit Config. Restart the app after editing. | | Cursor | `~/.cursor/mcp.json` for every project, or `.cursor/mcp.json` in one project. | | Other clients | Their MCP server settings, as `command`, `args` and `env`. | Desktop apps often start without your shell's `PATH`. If the client reports that `npx` is not found, set `command` to its absolute path (the output of `which npx`). The tools that run examples also need `git`, `tar` and uv or `python3` on the `PATH` the server sees. For a second graph, add a second entry, for example `staleproof-encode` with `"STALEPROOF_ORG": "encode"`. ### Over HTTP from your machine `npx -y staleproof serve` exposes the same endpoint locally, at `http://127.0.0.1:8080/api/mcp`, for clients that only speak HTTP. Started plainly it answers from its one graph and indexes a repository into it the first time an indexing tool names it, as the stdio server does (but `check_changes` takes `files` there, not `dir`); `serve --hosted` indexes any public repository on demand into workspace graphs, as the hosted server does. See [HTTP API](http-api.md#post-apimcp). ## What the server exposes | Property | Value | |---|---| | Server name | `staleproof` | | Version | The package version (`staleproof --version`) | | Capabilities | Tools only | | Transport | stdio (`staleproof mcp`): stdout carries the protocol; the server logs `staleproof mcp: serving graph "<org>" from <url>` (password masked) and indexing progress (`staleproof: <message>`) to stderr. Streamable HTTP (`POST /api/mcp` of `staleproof serve` and of the hosted service): stateless, JSON responses; `GET` and `DELETE` answer 405. | | Startup (stdio) | Connects to FalkorDB first. With neither `FALKORDB_URL` nor `--db` set and nothing on `localhost:6379`, it starts a local FalkorDB; the first start installs falkordblite, which takes about a minute, so a client may report a timeout on the very first launch. With a URL set and the server unreachable, the process exits with `cannot reach FalkorDB at <url>: …` and the client shows the server as failed. | The server gives clients these instructions: ```text Staleproof keeps docs true to code over a FalkorDB graph of an organisation's repositories. For a pull request, call check_pull_request first: it returns every doc section and example the change can make false, with the evidence path for each. To judge sections one by one: impact, read_section and run_example, decisions_for before editing, then propose_edit or record_verdict for every impacted section, and docs_patch for the diff. Before a pull request exists: docs_for_symbol says which docs depend on a function you are about to change, and check_changes says which docs your edits make false and which examples they break. With no change at all, audit_docs reports which examples no longer fit the code and which public API no doc covers. Every result lists the Cypher it ran. ``` The hosted server appends: `This is the hosted server: any public GitHub repository is indexed on first use (into a workspace graph named gh-<owner>); examples are not executed here; run them with the CLI (npx -y staleproof run <example-id> --pr <url>) or the GitHub Action.` ## The tools Fourteen tools over stdio; thirteen on the hosted server, which leaves out `run_example`. Each tool's description, as the client sees it, ends with the Cypher of the first query it runs; its input schema is JSON Schema. [Tools reference](tools.md) lists every parameter. | Tool | What it does | Writes to the graph | |---|---|---| | [`check_pull_request`](tools.md#check_pull_request) | The whole check in one call, as the Markdown report the GitHub Action posts: the examples run before and after, every impacted section with its evidence path, the cleared symbols, and verdicts recorded earlier. `run_examples: true` also runs up to 3 affected examples before and after (local servers only; takes minutes). | A Check node (graph mode) with its trace; on the first call, the pull request and its changes; with `run_examples`, Run nodes | | [`check_changes`](tools.md#check_changes) | Edits that are not a pull request yet: the doc sections they can make false with the evidence path for each, the examples [predicted to break](concepts.md#predicted-breaks), the impacted lines already out of date, and the changed symbols no doc reaches. Send `repo` and `files` (path and full new content, `null` to delete; at most 50), compared with the commit the graph indexed; a local server can instead read a checkout's working tree (`dir`, `base`). | No (it may index the repository first) | | [`docs_for_symbol`](tools.md#docs_for_symbol) | The doc sections and examples that depend on a symbol (an id, a qualified name, `Class.method` or a bare name), with the evidence path for each: what to update in the same change. | No | | [`audit_docs`](tools.md#audit_docs) | The [docs audit](audit.md) with no pull request: a 0-100 health score, every example line that no longer fits the code, API coverage, and the runs recorded at the indexed commits. It runs nothing. | No | | [`impact`](tools.md#impact) | The doc sections and examples a pull request can make false, with severity and the evidence path for each; 40 per call, page with `offset`. It also lists, over every page, the examples predicted to break and the impacted lines already out of date. | On the first call for a pull request: the pull request and its symbol-level changes. | | [`clearance`](tools.md#clearance) | The changed symbols that reach no doc within 3 call hops, each with a `git grep` command to check the claim. | As `impact`, if called first. | | [`read_section`](tools.md#read_section) | The current Markdown of a section and its code examples. | No | | [`run_example`](tools.md#run_example) | Runs an example in the sandbox, before and after a pull request with `pr_url`; identical runs are answered from the graph. Not on the hosted server. | Run nodes | | [`decisions_for`](tools.md#decisions_for) | Maintainer decisions governing a section or symbol, and earlier verdicts on a section. | No | | [`rank_sections`](tools.md#rank_sections) | Orders sections by the PageRank of their page in the docs' link graph. | No | | [`propose_edit`](tools.md#propose_edit) | Records replacement Markdown for a section (a `stale` verdict). Refused without a path from the pull request's changes to the section. | A verdict | | [`record_verdict`](tools.md#record_verdict) | Records `ok`, `governed` (must cite a decision) or `skipped` for one or more sections. | Verdicts | | [`write_decision`](tools.md#write_decision) | Records a maintainer decision and what it governs. Its description tells the model to call it only when a human asks. | A decision | | [`docs_patch`](tools.md#docs_patch) | The docs fix for a pull request as a unified diff, with a pull request title and description. MCP only. | No | ## Results A successful call returns three text blocks: 1. The summary, one or two sentences. 2. The data, as indented JSON. 3. The Cypher the tool ran, omitted when it ran no named query (`docs_patch`, for example): ```text Cypher run on FalkorDB: -- impact (<rows> rows, <ms> ms) MATCH (pr:PullRequest {url: $pr})<-[c:CHANGED_IN]-(s:Symbol) WHERE c.kind <> 'added' … -- prChanges (<rows> rows, <ms> ms) MATCH (pr:PullRequest {url: $pr})<-[c:CHANGED_IN]-(s:Symbol) … ``` The time is FalkorDB's own server-side execution time for that query. Small lookups the tools make with unnamed queries are left out. [Graph model and Cypher](GRAPH-MODEL.md) has every named query in full. `check_pull_request` is different: it returns two text blocks, the Markdown report (its format is in [GitHub Action](github-action.md#the-comment)) and `Stored as check <check-id>. <summary>`. ## Errors Errors come back as a result with `isError: true` and one text block: | Cause | Message | |---|---| | Unknown tool | `unknown tool <name>` | | A parameter the tool does not declare | `<tool> has no parameter <names>; its parameters are <list>` | | Invalid input | `invalid input: <field or (root)>: <problem>; …` | | The tool failed | The tool's message, for example `pr_url is required`, `refused: no path in the graph from <pr>'s changes to <section>. …`, `<pr> has not been checked yet; call impact first`, `<owner/repo> is not indexed in the "<org>" graph. Run: staleproof index …`, or `send the changed files: files: [{path, content}] with repo (this server cannot read your disk)` (`check_changes` with `dir` on an HTTP server) | | Hosted routing | `name a repository: …`, `<owner/repo> is not indexed yet: call check_pull_request with one of its pull requests first`, and the hosted limits ([Troubleshooting](troubleshooting.md#hosted-service)) | Unknown parameters are rejected with the same message the agent loop uses, so a client never gets a silently different answer. `propose_edit` and `record_verdict` need `pr_url` over MCP. In agent mode they default to the pull request under check; an MCP session has none, so without `pr_url` they fail with `pr_url is required`. Verdicts written over MCP belong to no check: they count for the patch and the pull request comment (the latest verdict per section wins), not for any agent check's stop condition. ## A worked session This is the sequence a client would call for [falkordb-py#267](https://github.com/FalkorDB/falkordb-py/pull/267), which fixed the client's execution-plan parser. The prompts are what you type; each block shows the `tools/call` parameters (`name` and `arguments`) the client sends. The client chooses the calls, so its order can differ. `check_pull_request` answers the first prompt in one call; this session takes the step-by-step route, which also yields the section and example ids. **"What docs can https://github.com/FalkorDB/falkordb-py/pull/267 make false? Show the evidence path for each."** ```json { "name": "impact", "arguments": { "pr_url": "https://github.com/FalkorDB/falkordb-py/pull/267" } } ``` The first call reads the pull request from GitHub and stores its changes. The graph reaches 3 doc sections, 2 of them in another repository: the GRAPH.EXPLAIN and GRAPH.PROFILE pages of `FalkorDB/docs`, through paths such as `ExecutionPlan._operation_tree ← ExecutionPlan.__init__ ← Graph.explain`. A grep for the changed names finds none of them. **"Read those sections and run their examples before and after the pull request."** ```json [ { "name": "read_section", "arguments": { "section_id": "FalkorDB/docs:commands/graph.explain.mdx#<anchor>" } }, { "name": "run_example", "arguments": { "example_id": "FalkorDB/docs:commands/graph.explain.mdx:0", "pr_url": "https://github.com/FalkorDB/falkordb-py/pull/267" } }, { "name": "run_example", "arguments": { "example_id": "FalkorDB/docs:commands/graph.profile.mdx:0", "pr_url": "https://github.com/FalkorDB/falkordb-py/pull/267" } } ] ``` Take the section ids from the `impact` result. On the graph behind [Facts from the graph](FACTS.md), the GRAPH.PROFILE example passes before and after the pull request, and the GRAPH.EXPLAIN example fails both times with `Invalid graph operation on empty key`: it explains a query on a graph the page never creates. It was already broken, not by this pull request. When the graph already holds identical runs, the summary ends with `(recorded earlier; not re-run)`. On the hosted server `run_example` is not offered; ask the local server. **"Check for maintainer decisions, then record a verdict for each section."** ```json [ { "name": "decisions_for", "arguments": { "target_id": "FalkorDB/docs:commands/graph.profile.mdx#<anchor>" } }, { "name": "record_verdict", "arguments": { "pr_url": "https://github.com/FalkorDB/falkordb-py/pull/267", "section_ids": ["<section id>", "<section id>"], "verdict": "ok", "reason": "The parser fix does not change what these sections claim; the example passes before and after.", "runs": ["run-<id>", "run-<id>"] } } ] ``` Run ids come from the `run_example` result (`runs[].id`). For the GRAPH.EXPLAIN page you can ask for a fix to its already-broken example with `propose_edit`. The tool accepts it because the graph has a path from the pull request to the section; its description asks the reason to say that the example fails before and after and to cite both runs. When `deepseek-v4-pro` re-checked #267 in agent mode (check `chk-90817863f077`, 12 tool calls), it considered that fix and declined, because the page assumes a demo graph that already exists, and recommended fixing it separately. **"Give me the docs patch for that pull request."** ```json { "name": "docs_patch", "arguments": { "pr_url": "https://github.com/FalkorDB/falkordb-py/pull/267" } } ``` With only `ok` verdicts, the summary is `No proposed edits for this pull request yet.` With proposed edits, the data holds the diff, a pull request title and description, and the changed files as `<owner>/<repo>:<path>`; ask the client to apply the diff in your checkout of the docs repository. ### Before a pull request A coding agent changing the code can ask the same graph before anything is committed. These calls are from the test suite's fixture organisation, where `acme/mini-docs` documents `acme/mini-client`. **"I'm about to add a required `timeout` to `Graph.query`. Which docs depend on it?"** ```json { "name": "docs_for_symbol", "arguments": { "symbol": "Graph.query", "repos": ["acme/mini-client"] } } ``` The summary: `3 doc sections reach miniclient.graph.Graph.query (2 name or call it directly; 2 repositories). 2 line(s) in those examples already do not fit the code.` The data lists each section with its evidence path, for example `miniclient.graph.Graph.query ← miniclient.graph.Graph.ro_query ← acme/mini-docs:docs/getting-started.mdx:1`. **"I've made the change. Which examples does it break?"** ```json { "name": "check_changes", "arguments": { "repo": "acme/mini-client", "files": [{ "path": "miniclient/graph.py", "content": "<the whole new file>" }] } } ``` The file is compared with the commit the graph indexed. The result's `impact.predicted` names the line that breaks: `docs/getting-started.mdx:0` line 18, `` `Graph.query` requires `timeout`, which the example does not pass ``. On a local stdio server, `{ "dir": "/path/to/checkout" }` checks the checkout's working tree instead of the files sent. Nothing is written to the graph. The rules to put in a repository's `AGENTS.md` or `CLAUDE.md`, so an agent does this unasked, are on [staleproof.dev/#/install](https://staleproof.dev/#/install) and in [Use cases](use-cases.md#coding-agents-before-the-pull-request). ## Security notes - **The local server runs code on your machine.** `run_example` (and `check_pull_request` with `run_examples`) clones the repositories, installs them from source, and executes docs code. The install step runs the projects' build steps with an allow-listed environment that leaves out `FALKORDB_URL`, `GITHUB_TOKEN` and model keys; the examples themselves run with a minimal environment and a socket-level block on network access other than the scratch FalkorDB. That is a guard, not a security boundary ([Example runner](example-runner.md#the-sandbox)). - **It flushes the scratch FalkorDB** before every example run. Staleproof refuses only when the scratch server is the one in `FALKORDB_URL`; make sure `STALEPROOF_SCRATCH_URL` (or the local port 6390) holds nothing you need. - **It writes to the graph.** Pull requests and their changes (`check_pull_request`, `impact`, `clearance`), checks (`check_pull_request`), runs (`run_example`), verdicts (`propose_edit`, `record_verdict`) and decisions (`write_decision`, which can also mark an older decision superseded). `check_changes`, `docs_for_symbol` and `audit_docs` write nothing beyond indexing a repository the graph lacks; the files sent to `check_changes` are not stored. Nothing is deleted, but whoever uses the server changes what later checks, the comment and the patch see. On the hosted server those writes go to a shared workspace graph that everyone checking that repository there sees. - **It writes nothing to GitHub.** No MCP tool opens a pull request or posts a comment; opening a docs pull request is the CLI's `patch --open-pr`. - **Credentials sit in the client configuration** in plain text, including any FalkorDB password in `FALKORDB_URL` and a `GITHUB_TOKEN`. The hosted server needs none. - Keep your client's approval prompts on for `run_example`, `propose_edit`, `record_verdict` and `write_decision`. ## Limits - `check_changes` checks edits statically: it predicts which examples break, but runs nothing. Running examples before and after needs a pull request (`run_example` with `pr_url`). The local server indexes the repositories it is asked about on first use (private ones with `GITHUB_TOKEN`); the hosted server reads public repositories only. - First-use indexing indexes the pull request's repository alone. For docs in another repository, index them together first (`staleproof index <code repo> <docs repo>`), or use `staleproof check <pr-url> --with <docs repo>`, or name them next to the repository on the website. - The stdio server serves one graph per call: `STALEPROOF_ORG` when set, else the owner of the repository the call names. The hosted server picks the graph per repository. - No `--fresh` over MCP: an identical recorded run is always answered from the graph. Use `staleproof run --fresh` to execute it again. - Only `check_pull_request` stores a Check node; the other tools store no check and no trace. - `impact` returns 40 sections per call; for a larger pull request the client must page with `offset`, as the summary tells it to. - `/api/mcp` has no authentication, a rate limit of 300 requests per client per hour, and its calls do not wait in the HTTP server's one-check-at-a-time queue. ## Related - [Install](install.md) - [Tools reference](tools.md) - [Use cases](use-cases.md) - [Example runner](example-runner.md) - [Decisions and memory](memory.md) - [CLI](cli.md) --- <!-- page: HTTP API · Integrate · source: https://staleproof.dev/docs/http-api.md --> # HTTP API `staleproof serve` runs one HTTP server with a JSON API, Server-Sent Events (SSE) streams for live checks, audits and indexing, a README badge, MCP over Streamable HTTP, and the web app. This page lists every route with its parameters, response fields, status codes, error bodies and rate limits, shows how to read the check stream from code, and covers hosted mode, where any public repository is indexed on demand. It is for developers who build a client, a dashboard or an integration against a running server. The same `/api` routes answer on a Vercel deployment such as staleproof.dev, through the function in [api/index.js](../api/index.js). ## Start the server and call it ```bash npx -y staleproof serve curl -s http://127.0.0.1:8080/api/health ``` The server binds `127.0.0.1:8080` by default (`--host`, `--port`, or `HOST` and `PORT`; see [Configuration](configuration.md#server)). On startup it prints one line to stderr, either `staleproof serve: http://127.0.0.1:8080 (graph "falkordb", agent on, <model>; MCP at /api/mcp)` or `... agent off: no model credentials, graph mode only; MCP at /api/mcp)`; in hosted mode `, hosted: any public repository on demand` follows the graph name. It connects to FalkorDB before it listens. With neither `FALKORDB_URL` nor `--db` set and nothing on `localhost:6379`, it starts a local FalkorDB first; with either set, an unreachable server stops it with `cannot reach FalkorDB at ...`. Run a graph-mode check and watch the trace arrive: ```bash curl -N -X POST http://127.0.0.1:8080/api/check \ -H 'content-type: application/json' \ -d '{"pr_url": "https://github.com/FalkorDB/falkordb-py/pull/262", "mode": "graph"}' ``` The API has no authentication. Anyone who can reach the port can start checks (which spend your model budget in agent mode), record decisions and review verdicts, and, in hosted mode, index repositories. Keep the default `127.0.0.1` bind, or put an authenticating proxy in front of it. ## Hosted mode Without hosted mode, the server answers from one graph: `STALEPROOF_ORG`, else `falkordb`. In hosted mode (`serve --hosted` or `STALEPROOF_HOSTED=on`; on by default in the Vercel function), that graph is the curated one, and any other public repository is downloaded as a tarball and indexed on demand into the workspace graph of its owner, `gh-<owner>`, together with the repositories named alongside it. The routes then find the right graph themselves: by the repository a pull request URL or an id names, and by searching every graph for a check or verdict id. [Deploy](DEPLOY.md#hosted-mode) has the limits and how workspaces are evicted. ## Routes at a glance | Method and path | Purpose | Rate limited | |---|---|---| | `GET /api/health` | Version, graph, FalkorDB version, whether agent mode is available, whether a check is running, hosted mode | no | | `GET /api/stats` | Every number FACTS.md quotes, computed from the graph | no | | `GET /api/prs` | The preloaded pull requests with their latest check and example runs | no | | `GET /api/checks` | Recent checks, newest first | no | | `GET /api/checks/:id` | The stored trace of one check | no | | `POST /api/check` | Run a check and stream its trace (SSE); hosted, index the repository first | yes, `check` bucket | | `POST /api/index` | Hosted mode: index repositories into one workspace and stream the progress (SSE) | yes, `index` bucket | | `GET /api/repos` | Which graph holds a repository, and whether it is indexed | no | | `GET /api/workspaces` | Hosted mode: every workspace and its repositories | no | | `GET /api/audit` | The docs audit of an indexed repository, as JSON | no | | `POST /api/audit` | Audit a repository and stream the progress (SSE); hosted, index it first | yes, `audit` bucket | | `GET /api/badge/:owner/:name.svg` | The README badge: the repository's docs health score | no | | `GET /api/docs-for` | Which docs depend on a symbol, with the evidence path for each | no | | `POST /api/mcp` | MCP over Streamable HTTP | yes, `mcp` bucket (10 times the others) | | `GET /api/sections` | The text and examples of one doc section | no | | `GET /api/patch` | The docs patch for a checked pull request | no | | `POST /api/decisions` | Record a maintainer decision | yes, `write` bucket | | `POST /api/verdicts/:id/review` | Accept or reject a verdict | yes, `write` bucket | | `GET /api/model` | Every named Cypher query and every tool | no | | `GET /`, `/llms.txt`, `/llms-full.txt`, `/docs/<slug>.md`, `/robots.txt`, `/sitemap.xml`, `/favicon.svg`, `/assets/*`, anything else | The web app and its static files | no | ## Conventions - Responses are JSON. Errors have the body `{"error": "<message>"}` and a 4xx or 5xx status. An error a route does not handle itself becomes `400 {"error": "<message>"}` when it is a hosted limit, a repository too large to index from an archive or a repository that is not indexed, and `500 {"error": "<message>"}` otherwise. - Request bodies are JSON. Send `content-type: application/json`. A body that is not valid JSON is read as `{}`, so it fails validation with the message for a missing field. - Pull request URLs may be `https://github.com/<owner>/<repo>/pull/<n>` (a trailing `/files`, query or fragment is allowed) or `<owner>/<repo>#<n>`. They are normalised to `https://github.com/<owner>/<repo>/pull/<n>`, which is how the graph stores them. - Ids follow the formats in [Core concepts](concepts.md#ids): a section is `owner/repo:path#anchor` (for example `FalkorDB/falkordb-py:README.md#asynchronous-example`), an example is `owner/repo:path:index`, a check is `chk-<12 hex>`, a verdict `ver-<12 hex>`, a decision `dec-<10 hex>`. URL-encode ids in query strings: they contain `/`, `:` and `#`. - Timestamps (`at`, `startedAt`, `finishedAt`, `createdAt`) are milliseconds since the Unix epoch. - No CORS headers are set, so a browser page on another origin cannot call the API directly. Call it from a server, or proxy it under your own origin. - Repository names may be `owner/name` or a GitHub URL (`https://github.com/owner/name`, with or without `.git`). They are case-insensitive, as on GitHub: a pull request URL in any spelling is read with the repository's indexed spelling, and hosted workspaces store names in lower case. ## Rate limits `POST` routes are limited to 30 requests per client per hour, and `POST /api/mcp` to 300 (a client sends a few requests per question). GET routes are not limited. | Bucket | Routes | Limit | |---|---|---| | `check:<client>` | `POST /api/check` | 30 per sliding hour | | `index:<client>` | `POST /api/index` | 30 per sliding hour | | `audit:<client>` | `POST /api/audit` | 30 per sliding hour | | `mcp:<client>` | `POST /api/mcp` | 300 per sliding hour; over it, a JSON-RPC error `rate limited: try again later` with status 429 | | `write:<client>` | `POST /api/decisions` and `POST /api/verdicts/:id/review`, together | 30 per sliding hour | - **The client key** is the `X-Real-IP` header if present (Vercel sets it), else `Fly-Client-IP`, else the last entry of `X-Forwarded-For` (the one the nearest proxy appended), else the constant `local`. Behind no proxy, every caller shares the `local` bucket. - **Counting**: a request counts before it is validated, so a request rejected with 400 still uses one slot. - **Over the limit**: `429 {"error": "rate limited: try again later"}`. No `Retry-After` header is sent; the oldest request in the window drops out an hour after it was made. - **Scope**: counts live in memory, per process. A restart resets them; on Vercel each function instance counts on its own. - **Changing it**: `createApp({ rateLimit })` in the [library](library.md#mount-the-http-api-in-your-own-server). `staleproof serve` and the Vercel function use the default of 30; there is no flag or environment variable. - **Hosted agent checks** also have a daily budget across the service; see [POST /api/check](#post-apicheck). ## One check at a time A server runs one check at a time, agent or graph mode, and in hosted mode one indexing run at a time with them. Further `POST /api/check`, `POST /api/index` and (hosted) `POST /api/audit` requests wait in arrival order; a request that has to wait first receives a `queued` event. `GET /api/health` reports `busy: true` while one runs. Example runs are also serialised, process-wide, because they share the scratch database. Calls to `POST /api/mcp` do not wait in this queue. Closing the connection does not cancel a check. It runs to the end and is stored on its `Check` node; read it later with `GET /api/checks/:id`. ## GET /api/health What the server is connected to and whether agent mode is available. ```bash curl -s http://127.0.0.1:8080/api/health ``` A server without model credentials and without hosted mode answers: ```json {"ok": true, "version": "0.3.0", "org": "falkordb", "falkordb": "4.22.0", "agent": {"available": false, "reason": "the server has no model API credentials (ANTHROPIC_API_KEY)"}, "busy": false, "hosted": {"enabled": false}} ``` | Field | Type | Meaning | |---|---|---| | `ok` | `true` | Always `true` when the route answers | | `version` | string | The `staleproof` package version | | `org` | string | The graph name (the org slug) | | `falkordb` | string | The FalkorDB module version from `MODULE LIST`, or `unknown` if the server does not report it | | `agent` | object | `{available: true, model, effort, provider}` or `{available: false, reason}` | | `agent.provider` | string | The host of `ANTHROPIC_BASE_URL` for an Anthropic-compatible provider (for example `api.deepseek.com`), else `api.anthropic.com` | | `busy` | boolean | A check (or, hosted, an indexing run) is running | | `hosted` | object | `{enabled: false}`, or in hosted mode `{enabled: true, limits: {maxRepos, maxFiles, maxGraphMb}, memory: {used, max}, githubToken}`: the workspace limits, the FalkorDB server's memory in MB (`max` is 0 when uncapped; `memory` is left out when the server does not report it), and whether the server has a GitHub token | `available: true` means credentials were found, not that they work. A `claude-*` model name with a compatible provider still reports `available: true`, and every agent check then fails with an `error` event (see [Troubleshooting](troubleshooting.md#common-errors)). ## GET /api/stats The facts behind [FACTS.md](FACTS.md), as JSON: the same object `staleproof facts --json` prints. It is computed from the graph and cached for 60 seconds per process. ```bash curl -s http://127.0.0.1:8080/api/stats ``` | Field | Shape | |---|---| | `org`, `generatedAt`, `falkordb` | graph name, ISO time, FalkorDB version | | `nodes` | `{<label>: count}`; `Symbol`, `DocSection` and `CodeExample` count live nodes only | | `edges` | `{<relationship type>: count}` | | `repos[]` | `{name, sha, packages, symbols, sections, examples, pythonExamples}` | | `calls`, `uses` | `{exact, byName, crossRepo}`, `{exact, byName}` | | `dependsOn[]` | `{from, to, spec}` | | `crossRepoDocPaths` | number of symbol-to-doc paths that cross a repository boundary | | `samplePaths[]` | `{symbol, doc, hops, path}` | | `impactQueryMs[]` | `{symbol, ms, rows}`: server-side time of the impact query for sample symbols | | `pullRequests[]` | `{url, title, changed, sections, otherRepos}` | | `runs[]` | `{example, url, side, status, error, reason, installed, pr}`, the latest attempt of each run | | `grepBaseline[]` | `{url, names, reached, grepFound, grepOutside}` | ## GET /api/prs The pull requests the home page lists: `serve --preload`, else `STALEPROOF_PRELOADED`, else falkordb-py #262, #267 and #276. ```bash curl -s http://127.0.0.1:8080/api/prs ``` ```text { "prs": [ { url, ref, title, state, latestCheck, runs } ] } ``` | Field | Shape | |---|---| | `url` | canonical PR URL | | `ref` | `{owner, repo, fullName, number, url}` | | `title`, `state` | from the graph; `null` until the PR has been checked once; `state` is `merged`, `open` or `closed` | | `latestCheck` | `{id, model, status, summary, startedAt}` of the newest check in either mode, or `null` | | `runs[]` | `{example, side, status, error}` for every example run linked to the PR, ordered by example and side | A preloaded entry that is not a pull request URL makes this route answer 500. ## GET /api/checks Recent checks, newest first, in both modes. | Query parameter | Default | Meaning | |---|---|---| | `limit` | `20` | Number of checks; values above 100 are capped at 100, fractions are rounded down, and anything that is not a positive number falls back to 20 | ```bash curl -s 'http://127.0.0.1:8080/api/checks?limit=5' ``` Each entry: `{id, pr, title, status, model, startedAt, finishedAt, steps, toolCalls, verdicts, summary, toolOrder, org}`. `org` is the graph that holds the check. In hosted mode the list merges the newest checks of the curated graph and the 7 most recently used workspaces. - `model` is the model name in agent mode (for example `deepseek-v4-pro`) and `graph-only (no model)` in graph mode. - `status` is `running` while a check is in progress (`finishedAt` is then `null`), then `done`, `incomplete`, `refused` or `error` (see [Check statuses](#check-statuses)). A check whose process died stays `running`. - `summary` is the agent's final text in agent mode, which can be several paragraphs of Markdown. ## GET /api/checks/:id The stored trace of one check, for replaying it. ```bash curl -s http://127.0.0.1:8080/api/checks/<check-id> ``` `200 {"id": "<check-id>", "org": "<graph>", "events": [TraceEvent, ...]}`, or `404 {"error": "no such check"}`. In hosted mode the check is looked up in the curated graph first, then in every workspace. The stored trace is the live stream with two differences: `delta` events are dropped, and a `tool_result` whose `data` serialises to more than 60,000 characters has it replaced by `{"__trimmed": true, "preview": "<first 60,000 characters>…"}`. The trace is written when the check ends, so a running check returns `events: []`. ## POST /api/check Run a check of one pull request and stream its trace as Server-Sent Events. | Body field | Type | Required | Meaning | |---|---|---|---| | `pr_url` | string | yes | The pull request (any accepted form, see [Conventions](#conventions)) | | `mode` | `"agent"` \| `"graph"` | no | Default: `agent` when the server has model credentials, else `graph`. Any value other than `"agent"` runs graph mode | | `with` | string[] | no | Hosted mode: repositories to index together with the pull request's, for example its docs site (`["acme/docs"]`). Validated everywhere, used only in hosted mode | What each mode does (definitions in [Agent mode and graph mode](concepts.md#agent-mode-and-graph-mode)): - **Agent mode**: the model chooses every tool call and records a verdict or an edit for each impacted section. The server gives it at most 40 model turns and, if `STALEPROOF_CHECK_BUDGET_MS` is set, a wall-clock budget (240 s by default on Vercel, none elsewhere). Neither can be changed per request. - **Graph mode**: no model. The tools run in a fixed order: `impact` (3 hops), `clearance`, `rank_sections` on the first impact page, then `run_example` before and after the PR for at most 3 exactly-linked examples on that page. In a hosted workspace it runs no examples (the summary says `Example runs not requested.`). It writes no verdicts. The first check of a pull request ingests it (GitHub metadata and the symbol-level diff); later checks reuse what the graph stored, except that an open pull request whose head moved since (new pushes, or merged since) is read again first. A merged pull request is never re-read. In hosted mode, a repository that no graph holds (or whose workspace is more than 7 days old, or that lacks a `with` repository) is indexed first, inside the same stream: `index_started`, `index_progress` and `index_finished` events come before `check_started`. Without hosted mode nothing is indexed, and a repository missing from the graph ends the stream with an `error` event. Indexing comes out of the server's time budget (`STALEPROOF_CHECK_BUDGET_MS`, when set): the agent gets what remains, but at least 45 seconds. Responses before the stream starts: | Status | Body | When | |---|---|---| | 400 | `{"error": "not a GitHub pull request URL: \"<input>\" (expected https://github.com/<owner>/<repo>/pull/<n>)"}` | `pr_url` missing or malformed | | 400 | `{"error": "not a GitHub repository: \"<input>\" (expected owner/name or a github.com URL)"}` | an entry of `with` is not a repository name | | 409 | `{"error": "agent mode needs model API credentials on the server (ANTHROPIC_API_KEY); graph mode works without them", "agentUnavailable": true}` | `mode: "agent"` on a server without model credentials (an Anthropic-compatible provider's key in `ANTHROPIC_API_KEY` counts) | | 429 | `{"error": "rate limited: try again later"}` | over the `check` limit | | 429 | `{"error": "today's hosted agent checks are used up; graph mode still works, and the agent runs anywhere with your own key (npx -y staleproof check <pr-url>)", "agentUnavailable": true}` | hosted mode, `mode: "agent"`, once the service's agent checks for the day (UTC) are used up: `STALEPROOF_AGENT_CHECKS_PER_DAY` on Vercel, default 50. Every agent request counts, including ones that fail later | | 503 | `{"error": "the Staleproof API is not connected to FalkorDB (<message>); set FALKORDB_URL", "offline": true}` | Vercel only, when the function cannot reach FalkorDB | | 200 | `text/event-stream` | the check runs | Once the stream has started the status is 200, so failures after that point (the repository is not indexed, a hosted limit, a GitHub error, a model that cannot be constructed) arrive as an `error` event. ### Stream format Every message has the event name `trace` and one line of JSON data: ```text event: trace data: {"type":"check_started","checkId":"chk-f4062e42d5c6","pr":"https://github.com/FalkorDB/falkordb-py/pull/262","model":"graph-only (no model)","at":1792138484707} event: trace data: {"type":"tool_call","id":"graph_1","name":"impact","input":{"pr_url":"https://github.com/FalkorDB/falkordb-py/pull/262"},"step":1} ``` (The two events are from the graph-mode check of falkordb-py#262 stored in the bundled snapshot.) The response headers are `Content-Type: text/event-stream` and `Cache-Control: no-cache`. Messages carry no `id:` or `retry:` field and there is no heartbeat. The stream closes after `check_finished`, or after an `error` event. ### Trace events The `data` of every message is one of these objects, discriminated by `type`: the eight `TraceEvent` types of [src/agent/trace.ts](../src/agent/trace.ts), plus `queued` and `error`, which only the HTTP stream sends, and in hosted mode the three indexing events of [src/hosted/workspaces.ts](../src/hosted/workspaces.ts). | `type` | Sent | Fields | |---|---|---| | `queued` | once, first, when another check is running | — | | `check_started` | when the `Check` node is created | `checkId`, `pr`, `model`, `at`, `since?` (agent mode: what changed in the graph since the previous check, as one sentence) | | `delta` | agent mode, while the model streams; live only, never stored | `kind` (`text` \| `thinking`), `text` (a fragment), `step` | | `thinking` | agent mode, after each model turn, per thinking block | `text`, `step` | | `text` | agent mode, after each model turn, per text block | `text`, `step` | | `tool_call` | before a tool runs | `id` (the model's tool-use id; `graph_<n>` in graph mode), `name`, `input`, `step` | | `tool_result` | after it returns | `id`, `name`, `summary` (or the error message), `isError`, `queries[]` (`{name, cypher, params, ms, rows}`; `ms` is FalkorDB's server-side time, `name` is `adhoc` for unnamed statements), `data?` (absent on errors), `ms` (wall time of the call), `step` | | `nudge` | agent mode, when the model stops with impacted sections still open (at most twice) | `missing` (section ids), `step` | | `check_finished` | last event of every check | `checkId`, `status`, `summary`, `steps`, `toolCalls`, `toolOrder` (tool names in call order), `verdicts`, `missing`, `usage` (`{input, output, cacheRead}` tokens), `at` | | `error` | when the check could not start or crashed outside the tool loop | `message` | | `index_started` | hosted mode, when a workspace starts indexing | `repos` (every repository of the workspace, indexed together), `graph`, `reason` (`new`, `stale`, `added` or `refresh`), `at` | | `index_progress` | hosted mode, during indexing | `repo?`, `phase` (`download`, `evict`, or an index phase: `checkout`, `parse`, `write`, `resolve`, `docs`, `done`), `message`, `at` | | `index_finished` | hosted mode, when the workspace is written | `graph`, `repos[]` (`{repo, sha, files, symbols, sections, examples}`), `mb?` (the workspace's graph size), `ms`, `at` | - `step` is the model turn in agent mode and the tool call number in graph mode. - `data` on a `tool_result` is the tool's full result (the shapes are in [Tools reference](tools.md)); only the stored trace trims it. - In graph mode `usage` is all zeros, `verdicts` is `0` and `missing` is empty. - In agent mode, `missing` lists the impacted sections with no verdict or edit from this check. After any `impact` call the loop collects the section ids of every impact page, including pages the model has not requested, so `done` means every impacted section was closed. - Typical `error` messages: `<owner/repo> is not indexed in the "<graph>" graph. Run: staleproof index <owner/repo> (plus the repositories that document it).`, the GitHub errors (rate limit, not found), and `"<model>" is a Claude model name, but the model endpoint is an Anthropic-compatible provider ...`. Failures inside the loop, such as the model API rejecting the key, end the check instead: `check_finished` with `status: "error"` and a summary starting `The check failed:`. Each message is explained in [Troubleshooting](troubleshooting.md#common-errors). ### Check statuses | `status` | Meaning | |---|---| | `done` | Agent mode: every impacted section has a verdict or an edit from this check. Graph mode: the fixed steps ran (a failing `clearance`, `rank_sections` or `run_example` is recorded with `isError` and the rest continue; a failing `impact` ends the check with `error`) | | `incomplete` | Agent mode: the model stopped, ran out of turns (40) or ran out of the server's time budget with sections still open; they are listed in `missing` and in the summary | | `refused` | Agent mode: the model declined to continue; the summary gives the category | | `error` | The check failed; the summary starts with `The check failed:` | ## POST /api/index Hosted mode only: index public repositories into one workspace and stream the progress. The web app calls it before a check; `POST /api/check` indexes on its own, so a client rarely needs it. | Body field | Type | Required | Meaning | |---|---|---|---| | `repos` | string[] or string | yes | The repository first, then the repositories to index together with it (for example `["acme/acme-client", "acme/acme-docs"]`) | | `refresh` | boolean | no | `true` re-indexes the workspace at its default branch heads even when it is current | ```bash curl -N -X POST https://staleproof.dev/api/index \ -H 'content-type: application/json' \ -d '{"repos": ["encode/httpx"]}' ``` Responses before the stream starts: | Status | Body | When | |---|---|---| | 404 | `{"error": "this server indexes from the CLI only: staleproof index <owner/repo>..."}` | the server is not in hosted mode | | 429 | `{"error": "rate limited: try again later"}` | over the `index` limit | | 400 | `{"error": "repos required: [\"owner/name\", ...]"}`, or `not a GitHub repository: ...` | `repos` missing, empty, or not repository names | | 200 | `text/event-stream` | the request runs, in the one-at-a-time queue | Every message has the event name `trace`. The stream sends `index_started`, `index_progress` and `index_finished` ([Trace events](#trace-events)) when something is indexed, then `{"type": "indexed", "graph": "<graph>", "curated": <boolean>}`, or `{"type": "error", "message": "<message>"}`. A workspace that is already current sends `indexed` alone. When another request (possibly on another server instance) is indexing the same workspace, the stream sends `index_progress` with `waiting for another request indexing <graph>` every 3 seconds, for up to 150 seconds. The hosted limits arrive as `error` events; [Troubleshooting](troubleshooting.md#hosted-service) lists them. ## GET /api/repos Which graph holds a repository. | Query parameter | Required | Meaning | |---|---|---| | `name` | yes | `owner/name` or a GitHub URL | ```bash curl -s 'https://staleproof.dev/api/repos?name=FalkorDB/falkordb-py' ``` `200 {name, indexed, graph, curated, sha, indexedAt}`, plus `workspace` (the names of every repository in that graph) when a hosted server has it indexed. Without hosted mode, `graph` is the server's graph and `curated` is `true`. In hosted mode, a repository no graph holds answers `indexed: false` with the graph it would be indexed into (`gh-<owner>`). `sha` and `indexedAt` are `null` when it is not indexed. A malformed name answers `400 {"error": "not a GitHub repository: ..."}`. ## GET /api/workspaces Hosted mode: every graph that answers for repositories, the curated one first, then the hosted workspaces, most recently used first. ```bash curl -s https://staleproof.dev/api/workspaces ``` `200 {workspaces: [{graph, curated, repos: [{name, sha, indexedAt, lastUsed}]}]}`. Without hosted mode it answers `{"workspaces": []}`. For the curated graph `lastUsed` is the time it was indexed. ## GET /api/audit The [docs audit](audit.md) of one indexed repository: health score, the example lines that no longer fit the code, API coverage and the runs recorded at the indexed commits. | Query parameter | Required | Meaning | |---|---|---| | `repo` | yes | `owner/name` or a GitHub URL. In hosted mode, the graph that holds it | | `fresh` | no | `1` recomputes the audit instead of answering from the 10-minute cache | ```bash curl -s 'http://127.0.0.1:8080/api/audit?repo=FalkorDB/falkordb-py' ``` `200 {summary, org, repos, outdatedIndex, score, grade, deductions, examples, mismatches, coverage, undocumented, runs, runCounts, generatedAt}`: the summary sentence, then the audit's data ([the JSON shape](audit.md#the-cli)). Audits are cached per graph and repository for 10 minutes, shared with the badge. It runs no examples. | Status | Body | When | |---|---|---| | 400 | `{"error": "not a GitHub repository: ..."}` | `repo` missing or malformed | | 404 | `{"error": "<owner/repo> is not indexed yet: POST /api/audit {\"repo\": \"owner/name\"} indexes and audits it", "indexed": false}` | hosted mode, no graph holds the repository | | 404 | `{"error": "<owner/repo> is not indexed yet: staleproof index <owner/repo>", "indexed": false}` | without hosted mode, the graph does not hold it | ## POST /api/audit Audit one repository and stream the progress as Server-Sent Events. In hosted mode the repository is indexed first when no graph holds it (or its workspace is more than 7 days old, or was indexed by an older Staleproof), together with the `with` repositories. The [audit page](audit.md#on-the-web) calls it. | Body field | Type | Required | Meaning | |---|---|---|---| | `repo` | string | yes | The repository to audit | | `with` | string[] | no | Hosted mode: repositories to index together with it, such as its docs site | ```bash curl -N -X POST https://staleproof.dev/api/audit \ -H 'content-type: application/json' \ -d '{"repo": "encode/httpx"}' ``` Before the stream: `400 {"error": "not a GitHub repository: ..."}` for a bad `repo` or `with` entry, `429 {"error": "rate limited: try again later"}` over the `audit` limit. Then `200 text/event-stream`; every message has the event name `trace`: | `type` | Sent | Fields | |---|---|---| | `queued` | first, when another check or indexing run is in progress (hosted mode then waits for it before indexing) | — | | `index_started`, `index_progress`, `index_finished` | hosted mode, while the repository is indexed ([Trace events](#trace-events)) | as for `POST /api/check` | | `audit` | once, at the end | `graph`, `summary`, `data` (the audit, as `GET /api/audit` returns it without `summary`) | | `error` | instead of `audit` | `message`, for example `<owner/repo> is not indexed in the "<graph>" graph: staleproof index <owner/repo>` (without hosted mode, nothing is indexed) or a [hosted limit](troubleshooting.md#hosted-service) | The audit is always computed fresh and replaces the cached one, so the badge shows it at once. The stream closes after `audit` or `error`. ## GET /api/badge/:owner/:name.svg The README badge: `docs health | <score> <grade>` in the shields.io flat style, coloured by score ([colours](audit.md#the-readme-badge)). The `.svg` suffix is optional. ```markdown [![docs health](https://staleproof.dev/api/badge/<owner>/<name>.svg)](https://staleproof.dev/#/audit?repo=<owner>%2F<name>) ``` It always answers `200` with `content-type: image/svg+xml; charset=utf-8`. For an indexed repository it computes the audit (cached 10 minutes) and sends `cache-control: public, max-age=3600, s-maxage=3600`. For a repository this server has not indexed, or a malformed name, it sends a grey `docs health | not audited` badge with `max-age=300`: audit the repository once (the page, `POST /api/audit` or the CLI against this server's graph) and the next request after the cache expires shows the score. ## GET /api/docs-for Which doc sections and examples depend on a symbol, with the evidence path for each: the CLI's [`docs-for`](cli.md#docs-for) and the MCP tool [`docs_for_symbol`](tools.md#docs_for_symbol) over HTTP. | Query parameter | Required | Meaning | |---|---|---| | `symbol` | yes | A Symbol id (`owner/repo:qualname`), a qualified name, a dotted suffix such as `Graph.query`, or a bare name | | `repo` | hosted mode, unless `symbol` is an id | `owner/name`: picks the graph and only looks at symbols defined in that repository | ```bash curl -sG http://127.0.0.1:8080/api/docs-for --data-urlencode 'symbol=Graph.ro_query' --data-urlencode 'repo=FalkorDB/falkordb-py' ``` `200 {summary, query, symbols, ambiguous, sections, mismatched}` ([the result](tools.md#docs_for_symbol-result)). Errors: `400 {"error": "symbol required, e.g. ?symbol=Graph.query&repo=owner/name"}`; in hosted mode `400 {"error": "repo required (owner/name), or a symbol id that starts with owner/repo:"}` and `404 {"error": "<owner/repo> is not indexed yet", "indexed": false}`; a malformed `repo` answers 400. An unknown symbol is not an error: the summary says no live symbol has that name. ## POST /api/mcp The MCP tools over Streamable HTTP, for any MCP client that takes a URL ([MCP server](mcp.md)). The endpoint is stateless: every request gets a fresh MCP server, no session id is issued, and responses are JSON rather than SSE. `GET` and `DELETE` answer `405` with `Allow: POST` and a JSON-RPC error, `Method not allowed: POST JSON-RPC messages to this endpoint`. ```bash curl -s -X POST http://127.0.0.1:8080/api/mcp \ -H 'content-type: application/json' -H 'accept: application/json, text/event-stream' \ -d '{"jsonrpc": "2.0", "id": 1, "method": "tools/list"}' ``` - **Without hosted mode** every tool runs on the server's one graph; `check_pull_request`, `impact`, `clearance`, `check_changes`, `docs_for_symbol` and `audit_docs` index the repository they name into it when it is missing, like the stdio server. `check_changes` takes `files` here: only the stdio server reads a checkout (`dir`). - **In hosted mode** each call goes to the graph of the repository it names (`pr_url`, `repo`, the first of `repos`, or the `owner/repo:` prefix of an id), and `check_pull_request`, `impact`, `clearance`, `check_changes`, `docs_for_symbol` and `audit_docs` index that repository first when no graph has it. `run_example` is not offered, and the server's instructions say that examples are not executed there. - It is rate limited to 300 requests per client per hour (the `mcp` bucket), needs no authentication, and does not wait in the one-check-at-a-time queue. ## GET /api/sections The current text of one doc section and its code examples, as stored at the last index. | Query parameter | Required | Meaning | |---|---|---| | `id` | yes | Section id, URL-encoded. In hosted mode it is looked up in the graph of the repository its id names | ```bash curl -sG http://127.0.0.1:8080/api/sections --data-urlencode 'id=FalkorDB/falkordb-py:README.md#asynchronous-example' ``` `200 {id, repo, path, heading, body, url, lineno, endLineno, examples: [{id, lang, code, lineno}]}`. `heading` is the heading path (`Parent > Child`), `body` is the section's Markdown including its heading line (stored up to 8,000 characters), and `url` points at the line on GitHub at the indexed commit. Errors: `400 {"error": "id required"}`, `404 {"error": "no such section"}` (also, in hosted mode, when no graph holds the repository). ## GET /api/patch The docs fix for a checked pull request: a unified diff of every stale section's proposed text, and a pull request description with the evidence path behind each edit (see [The docs patch](concepts.md#the-docs-patch)). | Query parameter | Required | Meaning | |---|---|---| | `pr` | yes | The pull request, URL-encoded. In hosted mode the patch comes from the graph that holds its repository | ```bash curl -sG http://127.0.0.1:8080/api/patch --data-urlencode 'pr=https://github.com/FalkorDB/falkordb-py/pull/267' ``` `200 {title, body, diff, files, verdicts, conflicts}`: | Field | Shape | |---|---| | `title` | `docs: update for <repo>#<number> (<PR title>)` | | `body` | Markdown description: one entry per edited section with its path, runs and cited decisions, then the sections left unchanged and the edits not applied | | `diff` | unified diff of every changed file, `git apply`-able in each docs repository; `""` when nothing is edited | | `files[]` | `{repo, path, sections}`: the sections edited in each file | | `verdicts[]` | the latest verdict per section: `{section, repo, path, heading, lineno, endLineno, url, id, verdict, status, reason, newText, oldText, evidence, cites, runs: [{id, status, side, error}]}` | | `conflicts[]` | `{section, reason}` for edits that could not be placed | The patch is built from the latest verdict of each section: a `stale` verdict with new text whose review status is not `rejected`. Each file is read at the commit the graph indexed, from the local clone in `STALEPROOF_HOME` when there is one, otherwise from `raw.githubusercontent.com`, so the route also works on hosts without git such as Vercel. An edit whose section text differs from what the verdict was written against becomes a conflict (`<path> changed since it was indexed at <sha7>`), and so does a file that is missing at that commit (`<path> not found in <repo>@<sha7>`). On the FalkorDB examples the patch is empty: every verdict recorded on falkordb-py #262, #267 and #276 is `ok`. Errors: a missing or malformed `pr` answers `400 {"error": "not a GitHub pull request URL: ..."}`; in hosted mode, a repository no graph holds answers `404 {"error": "<owner/repo> is not indexed"}`. A pull request the graph has never seen (`<url> has not been checked`) and a failed GitHub read (`GitHub raw <path>@<sha7> failed: <status>`) answer 500. ## POST /api/decisions Record a maintainer decision. Later checks find it with `decisions_for` and cite it instead of flagging the section again (see [Decisions and memory](memory.md)). | Body field | Type | Required | Meaning | |---|---|---|---| | `text` | string, 5 to 500 characters | yes | The decision, in one sentence | | `governs` | string[], at least 1 | yes | DocSection and/or Symbol ids it governs | | `source_url` | string | no | Where it was decided (issue, PR comment, ADR) | | `author` | string | no | Who decided; default `unknown` (the web app sends `web demo`) | | `supersedes` | string | no | The decision id this replaces; an unknown id is ignored without an error | ```bash curl -s -X POST http://127.0.0.1:8080/api/decisions \ -H 'content-type: application/json' \ -d '{"text": "The async client keeps positional kwargs until 2.0", "governs": ["FalkorDB/falkordb-py:falkordb.asyncio.falkordb.FalkorDB.__init__"], "author": "maintainer"}' ``` `200 {id, governed, summary}`: the new decision id, how many of the given ids it now governs, and a sentence such as `Recorded decision <id> governing 1 node(s).` | Status | Body | |---|---| | 400 | the validation messages, joined with `; ` | | 400 | `decision text is limited to 500 characters` (the CLI `decide` has no such limit) | | 400 | `none of <ids> is a DocSection or Symbol id; decision not recorded` | | 404 | `<owner/repo> is not indexed`: hosted mode, when no graph holds the repository of the first `governs` id (the decision is written to that repository's graph) | | 429 | `rate limited: try again later` | ## POST /api/verdicts/:id/review A maintainer accepts or rejects a verdict. Reviews are how precision is measured, and a rejected `stale` verdict is left out of the docs patch. The CLI equivalent is `staleproof verdict accept|reject <id>`. The web app has no control for it. | Body field | Type | Required | Meaning | |---|---|---|---| | `status` | `"accepted"` \| `"rejected"` | yes | The review | | `by` | string | no | Reviewer, default `web`, cut to 60 characters | ```bash curl -s -X POST http://127.0.0.1:8080/api/verdicts/<verdict-id>/review \ -H 'content-type: application/json' \ -d '{"status": "accepted", "by": "maintainer"}' ``` `200 {"ok": true}`. Errors: `400 {"error": "status must be accepted or rejected"}`, `404 {"error": "no such verdict"}` (in hosted mode after searching every graph), 429. Verdict ids appear in check traces: the `tool_result` of `propose_edit` carries `data.id` and that of `record_verdict` carries `data.ids`. ## GET /api/model The query catalogue the web app's model page shows. ```bash curl -s http://127.0.0.1:8080/api/model ``` `200 {queries: [{name, group, purpose, cypher}], tools: [{name, description, queries}]}`. `queries` is every named Cypher statement (`group` is `schema`, `index`, `tool`, `memory` or `report`); `tools` lists the nine tools of the agent and MCP (`check_pull_request`, `check_changes`, `docs_for_symbol`, `audit_docs` and `docs_patch`, which only MCP has, are not included), each with the names of the queries it can run. The same statements are in [Graph model and Cypher](GRAPH-MODEL.md). ## Static routes When the package's built web app (`web/dist`) exists, as it does in the npm package, the server also answers: | Path | Answer | |---|---| | `GET /` and any other GET path not listed here | `index.html`: the web app. Its pages are hash routes such as `#/`, `#/check/<id>`, `#/audit?repo=<owner/name>`, `#/model` and `#/docs/<slug>` | | `/assets/*` | the app's JavaScript and CSS | | `/favicon.svg` | the icon | | `/llms.txt` | an index of the documentation for LLMs ([llmstxt.org](https://llmstxt.org) format), `text/plain; charset=utf-8` | | `/llms-full.txt` | every documentation page in one file, `text/plain; charset=utf-8` | | `/docs/<slug>.md` | one documentation page as Markdown, `text/markdown; charset=utf-8`. `<slug>` is lowercase letters, digits and `-` (for example `/docs/http-api.md`); an unknown page is 404, and any other shape falls through to `index.html` | | `/robots.txt`, `/sitemap.xml` | crawler files, `text/plain` and `application/xml` | `/snapshot.json` is served as JSON when the built web app includes one; see [Snapshot mode](#snapshot-mode). Without a built web app, `GET /` answers the text ``Staleproof API is running. The web UI is not built (run `npm run build:web`).`` and other non-API paths are 404. On Vercel these files are served by Vercel from `web/dist`, not by the function, and `/snapshot.json` is among them. ## Consuming the SSE stream `EventSource` only sends GET requests, so read the `POST /api/check` response body with `fetch`. This works in Node.js 20 or later and in browsers (same origin): ```ts /** POST a check and hand each trace event to `onEvent` as it arrives. */ type Event = { type: string; [key: string]: unknown }; async function streamCheck(base: string, prUrl: string, mode: 'agent' | 'graph', onEvent: (e: Event) => void): Promise<void> { const res = await fetch(`${base}/api/check`, { method: 'POST', headers: { 'content-type': 'application/json' }, body: JSON.stringify({ pr_url: prUrl, mode }), }); if (!res.ok || !res.body) { // 400 malformed URL, 409 agent mode unavailable, 429 rate limited, 503 no FalkorDB (Vercel) const body = (await res.json().catch(() => ({}))) as { error?: string }; throw new Error(body.error ?? `HTTP ${res.status}`); } const reader = res.body.getReader(); const decoder = new TextDecoder(); let buffer = ''; for (;;) { const { done, value } = await reader.read(); if (done) break; buffer += decoder.decode(value, { stream: true }); let end: number; while ((end = buffer.indexOf('\n\n')) >= 0) { const message = buffer.slice(0, end); buffer = buffer.slice(end + 2); const data = message .split('\n') .filter((line) => line.startsWith('data:')) .map((line) => line.slice(5).trimStart()) .join('\n'); if (data) onEvent(JSON.parse(data) as Event); } } } await streamCheck('http://127.0.0.1:8080', 'https://github.com/FalkorDB/falkordb-py/pull/262', 'graph', (e) => { if (e.type === 'tool_result') console.log(`← ${String(e.summary)}`); else if (e.type === 'check_finished') console.log(`${String(e.status)}: ${String(e.summary)}`); else if (e.type === 'error') console.error(`error: ${String(e.message)}`); else if (e.type === 'queued') console.log('queued behind another check'); }); ``` The web app's own client, `streamCheck` in [web/src/api.ts](../web/src/api.ts), parses the stream the same way. From a shell, `curl -N` disables buffering so events print as they arrive; keep only the JSON with `sed`: ```bash curl -sN -X POST http://127.0.0.1:8080/api/check \ -H 'content-type: application/json' \ -d '{"pr_url": "https://github.com/FalkorDB/falkordb-py/pull/267", "mode": "graph"}' \ | sed -n 's/^data: //p' ``` Each output line is one event. Pipe it to `jq -c 'select(.type == "tool_result") | {name, summary}'` to follow the tools. ## Snapshot mode A static deployment can show the demo without a database. When an API call fails with no HTTP status (network error), with a status of 500 or above, or with a response that is not JSON, the web app loads `/snapshot.json` once and, if the file has data for that page, switches to snapshot mode for the rest of the page session. A banner shows when the snapshot was taken. | In snapshot mode | Behaviour | |---|---| | Health, stats, preloaded PRs, recent checks, stored traces, the model page | answered from the snapshot; health reports `agent: {available: false, reason: "snapshot mode"}` | | Section text and the docs patch | fail with `the Staleproof API is unreachable and no snapshot is bundled` | | Running a check, recording a decision | fail with `Live checks need the Staleproof API connected to FalkorDB; this deployment is showing a snapshot. Run it locally: npx staleproof serve` | `npm run snapshot` (in a clone of the repository; [scripts/snapshot.ts](../scripts/snapshot.ts)) writes `web/public/snapshot.json` from the graph `FALKORDB_URL` and `STALEPROOF_ORG` point at: health, stats, the preloaded PRs (`STALEPROOF_PRELOADED`), the model page, and stored traces for the latest check of each preloaded PR and for every agent check among the 20 most recent. The snapshot's list of recent checks keeps only the checks whose trace it carries. The next `npm run build` copies it into `web/dist`. ```bash FALKORDB_URL='falkor://localhost:6379' npm run snapshot npm run build ``` Snapshot mode serves static hosting, where Vercel serves `web/dist/snapshot.json` as a file and the function answers 503 without FalkorDB. `staleproof serve` cannot be in this state at startup (it refuses to start without FalkorDB), and it does not serve `/snapshot.json`. ## On Vercel The function in [api/index.js](../api/index.js) wraps the same app, so every `/api` route above behaves the same, with these differences: - **Hosted mode is on** unless `STALEPROOF_HOSTED=off`: any public repository is indexed on demand into its own workspace graph, and MCP answers at `/api/mcp`. - **No database**: it connects to FalkorDB on the first request. If that fails, every `/api` route answers `503 {"error": "the Staleproof API is not connected to FalkorDB (<message>); set FALKORDB_URL", "offline": true}`, and the next request tries again. - **No git**: repositories are downloaded as tarballs, pull requests are ingested through the GitHub API, examples are not executed (runs recorded on the graph by another host are still answered), and `clearance` reports 0 raw text matches because there are no clones to grep. - **Methods**: the function answers `GET` and `POST` only. - **Time**: the function may run 300 s (`maxDuration` in [vercel.json](../vercel.json)); agent checks stop at 240 s by default and end `incomplete`. - **Per instance**: rate limits, the one-check-at-a-time queue and the stats cache are per function instance. The hosted workspaces' locks and daily budgets are shared, because they are stored in FalkorDB. [Deploy](DEPLOY.md#vercel) has the setup. ## Known limits - No authentication and no CORS headers. - Rate limits are in memory per process and keyed on headers, `X-Real-IP` first. A client that reaches the server without a proxy that sets `X-Real-IP` can choose its own key by sending that header. - `POST /api/mcp` is limited to ten times the check limit per client (300 requests an hour by default: a client sends a few per question) and runs outside the one-check-at-a-time queue. - A check cannot be cancelled, and closing the stream does not stop it. A check whose process dies stays `running`. - Per-request settings are fixed: agent mode uses 40 turns and the server's time budget; graph mode uses 3 hops and at most 3 example runs (none in a hosted workspace). The CLI exposes `--max-steps`, `--max-hops` and `--runs`. ## Related - [CLI](cli.md): the same operations from a terminal, including `serve` - [Configuration](configuration.md): `PORT`, `HOST`, `STALEPROOF_PRELOADED`, `STALEPROOF_CHECK_BUDGET_MS` and the model variables - [Node.js library](library.md): `createApp` to mount this API in your own server - [Deploy](DEPLOY.md): Vercel, Docker Compose and a production checklist - [Troubleshooting and FAQ](troubleshooting.md) --- <!-- page: Node.js library · Integrate · source: https://staleproof.dev/docs/library.md --> # Node.js library The `staleproof` package exports the engine the CLI is built on: configuration, the FalkorDB graph client, the indexer (with tarball downloads and indexing on first use), hosted workspaces, pull request ingestion, the tools, the agent loop and graph mode, the report, the docs patch, the HTTP app and the MCP server over stdio or HTTP. This page lists every export with its signature and gives complete recipes, each type-checked against the source. It is for TypeScript and JavaScript developers who want to embed indexing or checks in their own tools, tests or servers instead of shelling out to the CLI. ## Install ```bash npm i staleproof ``` - **Node.js 20 or later** (`engines` in [package.json](../package.json)). - **ESM only**. The package is `"type": "module"` and ships no CommonJS build: use `import`. From CommonJS, load it with `const staleproof = await import('staleproof')`. - **No type declarations yet.** The build compiles JavaScript only (`dts: false` in [tsup.config.ts](../tsup.config.ts)), so TypeScript sees the module as untyped. The signatures on this page come from the source. To run the recipes as written, use `npx tsx recipe.ts` (tsx strips types without checking them). In a type-checked project, add a declaration file with `declare module 'staleproof';`; every import is then `any`, so annotate callback parameters yourself. - **What it needs at run time** is the same as the CLI: a FalkorDB server, `git` to clone (without it, `checkoutOrDownload` and `downloadRepo` fetch tarballs and pull requests are read through the GitHub API), `git` and `uv` or `python3` to run examples, and model credentials for agent mode. The library does not start a FalkorDB for you; `GraphStore.connect` fails when none answers. [Configuration](configuration.md) lists every variable. ## A first script Check a pull request in graph mode (no model) against a graph you have already indexed with `staleproof index`, and print what it reaches: ```ts import { GraphStore, loadConfig, runGraphCheck } from 'staleproof'; const config = loadConfig(); // FALKORDB_URL, STALEPROOF_ORG, STALEPROOF_HOME, ... const store = await GraphStore.connect(config.falkordbUrl); try { const result = await runGraphCheck({ graph: store.org(config.org), config }, 'https://github.com/FalkorDB/falkordb-py/pull/262'); console.log(result.status, result.toolOrder.join(' → ')); console.log(result.summary); } finally { await store.close(); } ``` Every function that touches the graph takes either an `OrgGraph` or a `ToolContext` (`{ graph, config }`), so one connection serves many calls. Close the `GraphStore` when you are done, or the process keeps the connection open. ## Exports Every export of [src/index.ts](../src/index.ts), grouped by purpose. ### Configuration | Export | Signature | What it does | |---|---|---| | `loadConfig` | `(env?: NodeJS.ProcessEnv) => Config` | Builds a `Config` from environment variables (default `process.env`). Pass `{}` for the defaults alone. Throws on an invalid `STALEPROOF_EFFORT`, an empty org name, or a `STALEPROOF_SCRATCH_URL` that carries credentials or does not parse | | `setting` | `(name: string, env?: NodeJS.ProcessEnv) => string \| undefined` | `STALEPROOF_<name>`, else `LOCKSTEP_<name>` (the former name), for example `setting('HOSTED')` | | `orgSlug` | `(raw: string) => string` | The graph name for an organisation: trimmed, lowercased, every run of characters outside `a-z`, `0-9`, `_` and `-` turned into `-`, leading and trailing `-` removed. Throws `invalid org name: "<raw>"` when nothing is left | | `DEFAULT_MODEL` | `'claude-sonnet-5-5'` | The agent model when `STALEPROOF_MODEL` is unset | | `Config` | type | See below | `Config` fields: | Field | Type | From | |---|---|---| | `falkordbUrl` | `string` | `FALKORDB_URL` | | `org` | `string` | `STALEPROOF_ORG`, slugged | | `home` | `string` | `STALEPROOF_HOME` | | `githubToken` | `string \| undefined` | `GITHUB_TOKEN`, else `GH_TOKEN` | | `anthropicApiKey` | `string \| undefined` | `ANTHROPIC_API_KEY` | | `modelBaseUrl` | `string \| undefined` | `ANTHROPIC_BASE_URL` | | `modelCompatible` | `boolean` | `true` when `ANTHROPIC_BASE_URL` names a host other than `api.anthropic.com` | | `model` | `string` | `STALEPROOF_MODEL` | | `effort` | `'low' \| 'medium' \| 'high' \| 'xhigh' \| 'max'` | `STALEPROOF_EFFORT` | | `fallbacks` | `boolean` | `STALEPROOF_FALLBACKS` is not `off` | | `scratch` | `{ host: string; port: number; managed: boolean }` | `STALEPROOF_SCRATCH_URL`, else `127.0.0.1` and `STALEPROOF_SCRATCH_PORT` | | `runTimeoutMs` | `number` | `STALEPROOF_RUN_TIMEOUT_MS` | | `checkBudgetMs` | `number \| undefined` | `STALEPROOF_CHECK_BUDGET_MS` | `Config` is a plain object: build one with `loadConfig()` and override fields (`{ ...loadConfig(), org: 'acme' }`). ### Graph | Export | Signature | What it does | |---|---|---| | `GraphStore.connect` | `(url: string) => Promise<GraphStore>` | Connects to a FalkorDB server (`falkor://`, `falkors://`, `redis://` or `rediss://`). Gives up after a 5 s connect timeout with `cannot reach FalkorDB at <url, password redacted>: ...` | | `store.org` | `(name: string) => OrgGraph` | The graph of one organisation; the name is slugged with `orgSlug` | | `store.serverVersion` | `() => Promise<string>` | The FalkorDB module version, for example `4.22.0`, or `unknown` | | `store.listGraphs` | `() => Promise<string[]>` | Every graph on the server (one per organisation) | | `store.memoryMb` | `() => Promise<{ used: number; max: number }>` | The server's memory in MB from `INFO memory`: what it uses now and its cap (`0` when uncapped) | | `store.graphMemoryMb` | `(name: string) => Promise<number \| undefined>` | One graph's size in MB (`GRAPH.MEMORY USAGE`), or `undefined` when the server cannot say | | `store.close` | `() => Promise<void>` | Closes the connection | | `graph.name` | `string` | The graph name | | `graph.run` | `<T = Record<string, unknown>>(query: NamedQuery \| string, params?: Record<string, unknown>) => Promise<QueryResult<T>>` | Runs one Cypher statement. Errors read `FalkorDB <query name or adhoc> failed on graph "<graph>": <message>` | | `graph.traced` | `(listener: (e: { name; cypher; params; ms; rows }) => void) => OrgGraph` | A view of the same graph that reports every query it runs; the tools use it to build their traces | | `graph.drop` | `() => Promise<void>` | Deletes the whole graph. Meant for tests | | `QueryResult<T>` | type | `{ rows: T[]; cypher: string; params; ms: number; stats: string[] }`; `ms` is FalkorDB's server-side execution time | | `queries` | namespace | Every named Cypher statement as `{ name, purpose, cypher, group }` (for example `queries.impact`, `queries.prChanges`), plus `queries.ALL_QUERIES`. Their text and parameters are in [Graph model and Cypher](GRAPH-MODEL.md) | ### Indexing | Export | Signature | What it does | |---|---|---| | `parseRepoName` | `(input: string) => string` | `owner/name` from `owner/name`, `https://github.com/owner/name[.git]` or `git@github.com:owner/name[.git]`. Throws `not a GitHub repository: "<input>" (expected owner/name or a github.com URL)` | | `checkoutRepo` | `(home: string, repo: string, ref?: string) => Promise<Checkout>` | Clones a repository (blob-less, over HTTPS) into `<home>/repos/<owner>/<name>`, or fetches if it is there, then checks out `ref` detached (default: the remote's default branch head). `ref` goes to `git checkout --detach` as is: use `origin/<branch>`, a tag or a SHA. With `GITHUB_TOKEN` (or `GH_TOKEN`) in the environment, git sends it to github.com, so private repositories work | | `downloadRepo` | `(home: string, repo: string, ref?: string, opts?: { token?: string; limits?: { maxBytes: number; maxFiles: number }; fetchImpl?: typeof fetch; skip?: (path: string) => boolean }) => Promise<Checkout>` | Downloads a tarball at `ref` (default `HEAD`) without git: from `codeload.github.com`, or with `token` from the API's tarball endpoint (private repositories too). Extracts only the files the indexer reads into `<home>/tarballs/<owner>/<name>/<sha>`, minus the paths `skip` accepts, and reads the commit from the archive's pax header. Default limits 150 MB and 4000 files; over them it throws a `too large` error | | `checkoutOrDownload` | `(config: Config, repo: string, ref?: string) => Promise<Checkout>` | `checkoutRepo` when git can run here, else `downloadRepo` with `config.githubToken`. What `staleproof index` uses | | `ensureIndexed` | `(graph: OrgGraph, config: Config, repos: string[], onProgress?: (p) => void) => Promise<IndexReport \| undefined>` | Indexes `repos` together, at their default branch heads, unless the graph already has every one of them; `undefined` when it indexed nothing. What `staleproof check` and the stdio MCP server do on first use | | `Checkout` | type | `{ repo: string; dir: string; sha: string }`. `dir` can be any local directory: without `.git` the indexer walks the tree, and `sha` is only used in links | | `indexOrg` | `(graph: OrgGraph, checkouts: Checkout[], onProgress?: (p: { phase: 'checkout' \| 'parse' \| 'write' \| 'resolve' \| 'docs' \| 'done'; repo?: string; message: string }) => void) => Promise<IndexReport>` | Parses every repository, resolves calls across all of them, links docs and examples to code, and writes `DEPENDS_ON`. Re-running is idempotent; see [Indexing](indexing.md) | | `IndexReport` | type | `{ org, run, repos, docs, dependsOn, ms }`. `repos[]`: `{ repo, sha, files, symbols, calls: { exact, byName, crossRepo }, gone, parseErrors }`. `docs[]`: `{ repo, pages, sections, examples: { total, python }, mentions, uses: { exact, byName }, links, gone }`. `dependsOn[]`: `{ from, to, spec }` | Cross-repository calls resolve only between repositories passed to the same `indexOrg` call. Index an organisation's repositories together. ### Hosted workspaces | Export | Signature | What it does | |---|---|---| | `Workspaces` | `new (store: GraphStore, config: Config, curated: OrgGraph, opts?: { limits?: Partial<HostedLimits>; prefix?: string; registry?: string; fetchImpl?: typeof fetch })` | Hosted mode ([Deploy](DEPLOY.md#hosted-mode)): the curated graph plus a workspace graph per repository owner (`gh-<owner>`), indexed on demand from tarballs, recorded in the registry graph (default `staleproof-hosted`). Pass it to `createApp({ workspaces })` | | `workspaces.ensure` | `(repo: string, opts?: { with?: string[]; refresh?: boolean; onEvent?: (e) => void }) => Promise<{ graph: OrgGraph; curated: boolean; report?: IndexReport }>` | The graph for `repo`, indexing its workspace first when no graph has it, when it is older than the re-index age, when a `with` repository is missing, or with `refresh`. Holds a lock across instances, evicts least recently used workspaces to make room, and throws on the limits. `onEvent` receives `index_started`, `index_progress` and `index_finished` ([trace events](http-api.md#trace-events)) | | `workspaces.locate` | `(repo: string) => Promise<{ graph; curated; entry? } \| undefined>` | The graph that holds `repo`, without indexing | | `workspaces.list`, `workspaces.graphs` | `() => Promise<WorkspaceInfo[]>`, `() => Promise<OrgGraph[]>` | Every workspace (curated first, then hosted by most recent use) with its repositories; `graphs` returns their graphs in the same order | | `workspaces.graphNameFor` | `(repo: string) => string` | The workspace graph a repository would be indexed into | | `workspaces.evict` | `(graph: string) => Promise<void>` | Drops a hosted workspace and its registry entries; refuses a graph without the prefix | | `workspaces.withinDailyBudget` | `(kind: string, limit: number) => Promise<boolean>` | Counts one use of a daily (UTC) budget shared through the registry and says whether it is still within `limit` | | `DEFAULT_HOSTED_LIMITS` | `HostedLimits` | `{ maxRepos: 4, maxGraphMb: 20, evictAboveMb: 70, evictToMb: 55, staleAfterMs: 7 days, indexesPerDay: 500, tarball: { maxBytes: 150 MB, maxFiles: 3000 } }` | | `HostedLimits` | type | The fields above | ### Pull requests | Export | Signature | What it does | |---|---|---| | `parsePrUrl` | `(input: string) => { owner; repo; fullName; number; url }` | Accepts `https://github.com/<owner>/<repo>/pull/<n>` (trailing `/files`, query or fragment allowed) and `<owner>/<repo>#<n>`; `url` is the canonical form the graph stores. Throws `not a GitHub pull request URL: ...` | | `fetchPullRequest` | `(ref, opts?: { token?: string; cacheDir?: string; fetchImpl?: typeof fetch }) => Promise<PullRequestInfo>` | PR metadata from the GitHub REST API: the `parsePrUrl` fields plus `title`, `body`, `author`, `state` (`open` \| `closed`), `mergedAt`, `mergeSha`, `baseSha`, `baseRef`, `headSha`, `headRef`. With `cacheDir`, the answer is written to `<cacheDir>/github/<owner>/<repo>/pulls/<n>.json` and read back only for merged PRs | | `diffPullRequest` | `(dir: string, repo: string, pr: PullRequestInfo) => Promise<PrDiff>` | The symbol-level diff of a PR in a local clone: first parent to merge commit when merged, merge base to head otherwise | | `diffRange` | `(dir: string, repo: string, before: string, after: string) => Promise<PrDiff>` | The same diff between any two commits | | `ChangedSymbol` | type | `{ repo, qualname, name, kind, change, before?, after?, renamedTo?, exportTarget?, path, lineno, endLineno, doc? }`; `change` is `removed`, `renamed`, `signature`, `modified`, `added` or `export-removed` (see [Change kinds](concepts.md#change-kinds)) | | `ingestPullRequest` | `(graph: OrgGraph, config: Config, prUrl: string) => Promise<IngestedPr>` | Fetches the PR, diffs it (local git, or the GitHub API when git is missing or `STALEPROOF_INGEST=github`) and records it. Always re-ingests: the tools skip PRs the graph already ingested, this function does not. Throws `<owner/repo> is not indexed in the "<graph>" graph. ...` when the repository is not in the graph | | `recordPullRequest` | `(graph: OrgGraph, pr: PullRequestInfo, diff: PrDiff) => Promise<IngestedPr>` | Writes a PR you diffed yourself: the `PullRequest` node, `CHANGED_IN` edges, recreated removed symbols and their late links. Returns `{ pr, diff, changed: { id, change }[], recreated: string[], lateLinks: number }` | `PrDiff` is `{ before: string; after: string; files: { status, path, oldPath? }[]; symbols: ChangedSymbol[]; docsTouched: string[] }`. ### Tools | Export | Signature | What it does | |---|---|---| | `AGENT_TOOLS` | `ToolDef[]` | The eight tools the agent chooses from, in this order: `impact`, `clearance`, `read_section`, `run_example`, `decisions_for`, `rank_sections`, `propose_edit`, `record_verdict` | | `ALL_TOOLS` | `ToolDef[]` | `AGENT_TOOLS` plus `write_decision` (nine). `docs_patch` exists only in the MCP server; in the library, call `buildDocsPatch` | | `modelTools` | `(tools?: ToolDef[]) => { name; description; input_schema }[]` | Tool definitions in the Anthropic Messages format, with JSON Schema generated from each tool's zod schema (default `AGENT_TOOLS`) | | `ToolDef<I, O>` | type | `{ name; description; input: z.ZodType<I>; queries: NamedQuery[]; action?: boolean; run(ctx: ToolContext, input: I): Promise<ToolResult<O>> }` | | `ToolResult<T>` | type | `{ tool: string; summary: string; data: T; queries: { name; cypher; params; ms; rows }[] }`: the sentence the agent reads first, the data, and every Cypher statement the call ran | | `ToolContext` | type | See below | `ToolContext` fields: | Field | Type | Meaning | |---|---|---| | `graph` | `OrgGraph` | The org graph | | `config` | `Config` | Settings | | `checkId` | `string?` | The check verdicts are attributed to; `''` is stored without one | | `prUrl` | `string?` | Default `pr_url` for `propose_edit` and `record_verdict` | | `log` | `(msg: string) => void` | Progress lines (ingestion, venv builds, runs) | | `checkouts` | `Record<string, string>?` | Local clones by `owner/name`, used instead of `<home>/repos/<owner>/<name>` for example installs and the patch | | `freshRuns` | `boolean?` | Re-run examples even when an identical run is recorded (what `run --fresh` sets) | `tool.run` does not validate its input. Parse it first with `tool.input.parse(input)`, as the agent loop and the MCP server do; zod then drops parameters the tool does not declare (both of those callers reject them instead). Parameters, results and errors of each tool are in the [Tools reference](tools.md). ### Agent and checks | Export | Signature | What it does | |---|---|---| | `runAgentCheck` | `(ctx: ToolContext, model: ModelClient, prUrl: string, opts?: { maxSteps?: number; maxNudges?: number; budgetMs?: number; onEvent?: (e: TraceEvent) => void }) => Promise<CheckResult>` | The agent loop (defaults: 40 turns, 2 nudges, no time budget). Ingests the PR if needed, stores a `Check` node with the trace. Failures inside the loop end the check with status `error`; invalid URLs and ingestion errors throw before the check starts | | `runGraphCheck` | `(ctx: ToolContext, prUrl: string, opts?: { maxRuns?: number; maxHops?: number; onEvent?: (e: TraceEvent) => void }) => Promise<CheckResult>` | Graph mode: `impact` (default 3 hops), `clearance`, `rank_sections` on the first impact page, and up to `maxRuns` (default 3) example runs before and after the PR. No model, no verdicts | | `checkReport` | `(ctx: ToolContext, org: string, prUrl: string, agentSummary?: string) => Promise<string>` | The Markdown report of a checked pull request, from what the graph holds: the title, the examples run before and after (the latest run on each side), the impacted sections with evidence paths in PageRank order, the cleared symbols, the latest verdicts and `agentSummary`. `org` is the graph name printed in the footer. It runs no check itself, but `impact` ingests the pull request if needed. This is the GitHub Action's comment and what `check_pull_request` returns | | `CheckResult` | type | `{ checkId, pr, status, summary, events, impacted, missing, toolOrder, since? }`. `status` is `done`, `incomplete`, `refused` or `error`. In graph mode `impacted` and `missing` are empty | | `TraceEvent` | type | The union of `check_started`, `delta`, `thinking`, `text`, `tool_call`, `tool_result`, `nudge` and `check_finished`; fields in [HTTP API](http-api.md#trace-events) | | `AnthropicModel` | `new (opts: { model: string; effort?: Effort; fallbacks?: boolean; apiKey?: string; maxTokens?: number; baseURL?: string; compatible?: boolean })` | The model client for the Claude API (defaults: effort `medium`, fallbacks on, 32,000 max tokens per turn) or, with `compatible: true`, an Anthropic-compatible provider. The constructor throws for a `claude-*` model name in compatible mode, and `the agent needs model API credentials: ...` when there is no `apiKey` and no credentials in the environment | | `ScriptedModel` | `new (script: (history, turn: number) => { thinking?: string; calls?: { name: string; input: Record<string, unknown> }[]; text?: string })` | A deterministic model for tests: each turn is computed from the conversation so far. `name` is `scripted`; `requests` keeps every request it received. Calls in one turn run one after another, in order | | `ModelClient` | type | `{ readonly name: string; turn(req: { system; tools; messages }, cb?: { onText?(delta: string): void; onThinking?(delta: string): void }): Promise<Message> }`, with the Anthropic SDK's beta Messages types. Implement it to plug in another model; the loop reads `content` (`text`, `thinking`, `tool_use` blocks), `stop_reason` and `usage` | ### Output | Export | Signature | What it does | |---|---|---| | `buildDocsPatch` | `(ctx: ToolContext, prUrl: string, opts?: { fetchImpl?: typeof fetch }) => Promise<DocsPatch>` | The docs fix for a checked PR from the latest verdict per section. `prUrl` must be the canonical URL (`parsePrUrl(x).url`). Files are read at the indexed commit from the local clone, or from `raw.githubusercontent.com` when there is none. Throws `<prUrl> has not been checked` | | `openDocsPr` | `(patch: DocsPatch, opts: { token: string; target: 'fork' \| 'upstream' \| 'same-repo'; base?: string; only?: string[]; pollMs?: number }) => Promise<{ repo; url; target; branch }[]>` | Opens the patch as a pull request per docs repository: in your fork by default, against the original repository with `target: 'upstream'`, or with `target: 'same-repo'` as a branch `staleproof/docs-<owner>-<repo>-<n>` of the docs repository itself, into `base` (default its default branch), for the repositories in `only` (the CLI's `patch --open-pr --in-repo`). Throws `opening a pull request needs GITHUB_TOKEN` and `the patch is empty: no proposed edits to open a pull request for` | | `collectFacts` | `(graph: OrgGraph, falkordbVersion?: string) => Promise<Facts>` | Every number FACTS.md quotes, computed from the graph (the `GET /api/stats` object) | | `factsMarkdown` | `(facts: Facts) => string` | FACTS.md as Markdown | `DocsPatch` is `{ pr: { url, title, repo, number }, files: { repo, path, sha, before, after, diff, sections }[], diff, title, body, verdicts, conflicts: { section, reason }[] }`. `diff` concatenates every file's unified diff; each is `git apply`-able in its own repository. ### Servers | Export | Signature | What it does | |---|---|---| | `createApp` | `(opts: { store: GraphStore; config: Config; model?: () => ModelClient; preloaded?: string[]; webDir?: string; rateLimit?: number; workspaces?: Workspaces; agentChecksPerDay?: number }) => Hono` | The HTTP API, MCP endpoint and web app of `staleproof serve` as a [Hono](https://hono.dev) app. Without `model`, agent mode answers 409. `preloaded` defaults to `DEFAULT_PRELOADED`, `webDir` to the package's `web/dist` (any missing directory gives an API-only app), `rateLimit` to 30 POST requests per client per hour. With `workspaces` it runs in hosted mode, and `agentChecksPerDay` (default 50) caps agent checks per day. Routes: [HTTP API](http-api.md) | | `defaultModelFactory` | `(config: Config) => (() => ModelClient) \| undefined` | What `serve` passes as `model`: an `AnthropicModel` factory built from `config`, or `undefined` when no credentials are found | | `DEFAULT_PRELOADED` | `string[]` | falkordb-py pull requests #262, #267 and #276 | | `createMcpServer` | `(target: OrgGraph \| ((tool: ToolDef, input: Record<string, unknown>) => Promise<ToolContext>), config: Config, opts?: { tools?: ToolDef[]; instructions?: string }) => Server` | An MCP `Server` (from `@modelcontextprotocol/sdk`) with the fourteen tools (or `opts.tools`), not yet connected to a transport. `target` is the graph every call runs on, or a function that returns the context for each call (how the stdio server indexes on first use and the hosted server routes by repository). See [MCP server](mcp.md) | | `handleMcpRequest` | `(request: Request, opts: { store: GraphStore; config: Config; graph: OrgGraph; workspaces?: Workspaces }) => Promise<Response>` | MCP over Streamable HTTP for one web-standard `Request`, stateless (a fresh server per request, JSON responses; `GET` and `DELETE` answer 405). With `workspaces`, calls are routed to the graph of the repository they name and `run_example` is left out. `POST /api/mcp` of `createApp` is this function | ## Recipes Every recipe is a complete ES module (top-level `await`). Save it as `recipe.ts` and run `npx tsx recipe.ts` with the variables of [Configuration](configuration.md) set. ### Index an organisation ```ts import { checkoutRepo, GraphStore, indexOrg, loadConfig, parseRepoName } from 'staleproof'; const config = loadConfig(); // FALKORDB_URL, STALEPROOF_ORG, STALEPROOF_HOME, ... const store = await GraphStore.connect(config.falkordbUrl); try { const graph = store.org(config.org); const checkouts = []; for (const input of ['FalkorDB/falkordb-py', 'FalkorDB/GraphRAG-SDK', 'FalkorDB/docs']) { // Clones (blob-less) or fetches into <STALEPROOF_HOME>/repos/<owner>/<name>, then checks out the default branch. checkouts.push(await checkoutRepo(config.home, parseRepoName(input))); } // Index all of them in one call: calls between repositories resolve only within one indexOrg run. const report = await indexOrg(graph, checkouts, (p) => console.error(`${p.phase.padEnd(8)} ${p.message}`)); for (const r of report.repos) { console.log(`${r.repo} @ ${r.sha.slice(0, 7)}: ${r.symbols} symbols, ${r.calls.exact} exact calls, ${r.calls.byName} by name`); } for (const d of report.docs) console.log(`${d.repo}: ${d.sections} sections, ${d.examples.total} examples`); console.log(`graph "${report.org}", run ${report.run}, ${(report.ms / 1000).toFixed(1)} s`); } finally { await store.close(); } ``` This is what `staleproof index` does. To pin a repository, pass a ref: `checkoutRepo(config.home, 'FalkorDB/docs', 'origin/main')`. ### Check a pull request in graph mode and print the impacted sections The tool results are in `result.events`. Their data types are not exported, so the script declares the fields it prints: ```ts import { GraphStore, loadConfig, runGraphCheck, type TraceEvent } from 'staleproof'; /** The fields of an impacted section this script prints (the full type is not exported). */ interface Impacted { section: string; severity: 'breaking' | 'api' | 'behavior'; hops: number; confidence: 'exact' | 'by-name' | 'late'; evidence: string[]; } type ToolResultEvent = Extract<TraceEvent, { type: 'tool_result' }>; const config = loadConfig(); const store = await GraphStore.connect(config.falkordbUrl); try { const ctx = { graph: store.org(config.org), config }; // maxRuns: 0 skips the example runs (no sandbox needed); the default is 3. const result = await runGraphCheck(ctx, 'https://github.com/FalkorDB/falkordb-py/pull/267', { maxRuns: 0 }); const impact = result.events.find((e): e is ToolResultEvent => e.type === 'tool_result' && e.name === 'impact' && !e.isError); const { sections } = (impact?.data ?? { sections: [] }) as { sections: Impacted[] }; for (const s of sections) { console.log(`${s.severity.padEnd(8)} ${s.hops} hops (${s.confidence}) ${s.section}`); console.log(` ${s.evidence.join(' ← ')}`); } console.log(`\n${result.status}: ${result.summary}\nStored as ${result.checkId}.`); } finally { await store.close(); } ``` `sections` is the first page of at most 40; `totalSections` and `truncated` on the same object say whether there are more. The evidence path runs from the changed symbol to the doc ([Evidence path](concepts.md#evidence-path)). ### Index on first use and print the report What `staleproof check <pr-url> --no-agent --format markdown` does, from a graph that may not have the repository yet: ```ts import { checkReport, ensureIndexed, GraphStore, loadConfig, parsePrUrl, runGraphCheck } from 'staleproof'; const config = loadConfig(); const store = await GraphStore.connect(config.falkordbUrl); try { const pr = parsePrUrl('https://github.com/FalkorDB/falkordb-py/pull/262'); const graph = store.org(pr.owner); // the CLI's default graph: the repository's owner, slugged // Clones (or downloads) and indexes the repository unless the graph already has it. const indexed = await ensureIndexed(graph, config, [pr.fullName], (p) => console.error(p.message)); if (indexed) console.error(`indexed ${indexed.repos.map((r) => r.repo).join(', ')}`); const ctx = { graph, config }; await runGraphCheck(ctx, pr.url, { maxRuns: 0 }); console.log(await checkReport(ctx, graph.name, pr.url)); } finally { await store.close(); } ``` Pass the docs repositories to `ensureIndexed` along with the code repository (`[pr.fullName, 'FalkorDB/docs']`) so they are indexed together. With `maxRuns` above 0 the report's examples table fills in. ### Run an agent check and stream its events ```ts import { AnthropicModel, GraphStore, loadConfig, runAgentCheck } from 'staleproof'; const config = loadConfig(); // Throws without credentials, or for a claude-* model name on an Anthropic-compatible provider. const model = new AnthropicModel({ model: config.model, effort: config.effort, fallbacks: config.fallbacks, apiKey: config.anthropicApiKey, baseURL: config.modelBaseUrl, compatible: config.modelCompatible, }); const store = await GraphStore.connect(config.falkordbUrl); try { const ctx = { graph: store.org(config.org), config, log: (m: string) => console.error(m) }; const result = await runAgentCheck(ctx, model, 'https://github.com/FalkorDB/falkordb-py/pull/262', { maxSteps: 40, onEvent(e) { switch (e.type) { case 'delta': if (e.kind === 'text') process.stdout.write(e.text); // live tokens; not stored break; case 'tool_call': console.log(`\n→ ${e.name} ${JSON.stringify(e.input)}`); break; case 'tool_result': console.log(`${e.isError ? '!' : '←'} ${e.summary} (${e.queries.length} queries, ${e.ms} ms)`); break; case 'nudge': console.log(`! ${e.missing.length} section(s) still need a verdict`); break; case 'check_finished': console.log(`\n${e.status}: ${e.verdicts} verdicts, ${e.usage.input} input / ${e.usage.output} output tokens`); break; } }, }); console.log(result.summary); if (result.missing.length) console.log(`Still open: ${result.missing.join(', ')}`); } finally { await store.close(); } ``` For DeepSeek, set `ANTHROPIC_BASE_URL`, `ANTHROPIC_API_KEY` and `STALEPROOF_MODEL` (see [Configuration](configuration.md#model-agent-mode)); `config.modelCompatible` is then `true`. Pass `budgetMs` to stop starting new model turns after a wall-clock limit, as the server does on Vercel. ### Call one tool directly `impactTool` and the other tool objects are not exported by name; find them in `ALL_TOOLS`: ```ts import { ALL_TOOLS, GraphStore, loadConfig, type ToolContext } from 'staleproof'; const config = loadConfig(); const store = await GraphStore.connect(config.falkordbUrl); try { const ctx: ToolContext = { graph: store.org(config.org), config }; const impact = ALL_TOOLS.find((t) => t.name === 'impact')!; // Validate with the tool's own zod schema first, as the agent loop and the MCP server do. const input = impact.input.parse({ pr_url: 'https://github.com/FalkorDB/falkordb-py/pull/276', max_hops: 2 }); const r = await impact.run(ctx, input); console.log(r.summary); for (const q of r.queries) console.log(`${q.name}: ${q.rows} rows, ${q.ms} ms`); console.log(JSON.stringify(r.data, null, 2)); } finally { await store.close(); } ``` `impact` ingests the pull request on first use. `propose_edit` and `record_verdict` need `pr_url` in the input or `prUrl` in the context, and write verdicts with `check: ''` unless the context has a `checkId`. ### Build the docs patch ```ts import { writeFile } from 'node:fs/promises'; import { buildDocsPatch, GraphStore, loadConfig, openDocsPr, parsePrUrl } from 'staleproof'; const config = loadConfig(); const store = await GraphStore.connect(config.falkordbUrl); try { const ctx = { graph: store.org(config.org), config }; // Pass the canonical URL: the PullRequest node is keyed by it. const pr = parsePrUrl('FalkorDB/falkordb-py#267').url; const patch = await buildDocsPatch(ctx, pr); await writeFile('docs.patch', patch.diff); await writeFile('docs.patch.md', `# ${patch.title}\n\n${patch.body}\n`); console.log(`${patch.files.length} file(s): ${patch.files.map((f) => `${f.repo}:${f.path}`).join(', ') || 'none'}`); for (const c of patch.conflicts) console.log(`not applied: ${c.section} (${c.reason})`); if (process.argv.includes('--open-pr') && patch.files.length) { const opened = await openDocsPr(patch, { token: config.githubToken ?? '', target: 'fork' }); for (const o of opened) console.log(`opened ${o.url} on ${o.branch}`); } } finally { await store.close(); } ``` The patch is built from the latest verdicts the agent recorded; graph mode records none. For falkordb-py #267 it is empty, because every verdict on that pull request is `ok`. With `--open-pr`, `openDocsPr` forks each docs repository, pushes a branch named `staleproof/<repo>-<number>` and opens the pull request in your fork. ### Mount the HTTP API in your own server `createApp` returns a Hono app. Mount it under a prefix in your own Hono app: ```ts import { serve } from '@hono/node-server'; import { Hono } from 'hono'; import { createApp, defaultModelFactory, GraphStore, loadConfig } from 'staleproof'; const config = loadConfig(); const store = await GraphStore.connect(config.falkordbUrl); const staleproof = createApp({ store, config, model: defaultModelFactory(config), // undefined without credentials: graph mode only preloaded: ['https://github.com/FalkorDB/falkordb-py/pull/262'], webDir: '/nonexistent', // API only: no web UI, no catch-all route rateLimit: 10, // POST requests per client per hour }); const app = new Hono(); app.get('/', (c) => c.text('my service')); app.route('/staleproof', staleproof); // GET /staleproof/api/health, POST /staleproof/api/check, ... serve({ fetch: app.fetch, port: 3000 }); ``` Install `hono` and `@hono/node-server` in your project for this. The web app calls `/api/...` at the root of its origin, so serve it unprefixed (omit `webDir`) if you want the UI too. Without Hono, plug the app's `fetch` into `node:http`: ```ts import { createServer } from 'node:http'; import { getRequestListener } from '@hono/node-server'; import { createApp, GraphStore, loadConfig } from 'staleproof'; const config = loadConfig(); const store = await GraphStore.connect(config.falkordbUrl); const staleproof = createApp({ store, config, webDir: '/nonexistent' }); createServer(getRequestListener(staleproof.fetch)).listen(3000); ``` In tests and scripts you can skip the network: `await staleproof.request('/api/health')` returns a `Response`, which is how `npm run snapshot` reads the API. ### Serve MCP over another transport `createApp` already serves MCP at `/api/mcp`, and `handleMcpRequest` serves it from any framework that hands you a web-standard `Request`: ```ts import { serve } from '@hono/node-server'; import { Hono } from 'hono'; import { GraphStore, handleMcpRequest, loadConfig } from 'staleproof'; const config = loadConfig(); const store = await GraphStore.connect(config.falkordbUrl); const graph = store.org(config.org); const app = new Hono(); app.all('/mcp', (c) => handleMcpRequest(c.req.raw, { store, config, graph })); serve({ fetch: app.fetch, port: 3001, hostname: '127.0.0.1' }); ``` `createMcpServer` returns an MCP `Server` without a transport, so any transport of `@modelcontextprotocol/sdk` works. The same Streamable HTTP endpoint built by hand, stateless, with a fresh server per request (the pattern of the SDK's own Hono example): ```ts import { serve } from '@hono/node-server'; import { WebStandardStreamableHTTPServerTransport } from '@modelcontextprotocol/sdk/server/webStandardStreamableHttp.js'; import { Hono } from 'hono'; import { createMcpServer, GraphStore, loadConfig } from 'staleproof'; const config = loadConfig(); const store = await GraphStore.connect(config.falkordbUrl); const graph = store.org(config.org); const app = new Hono(); // Stateless Streamable HTTP: a fresh MCP server and transport per request. app.all('/mcp', async (c) => { const server = createMcpServer(graph, config); const transport = new WebStandardStreamableHTTPServerTransport({ sessionIdGenerator: undefined }); await server.connect(transport); return transport.handleRequest(c.req.raw); }); serve({ fetch: app.fetch, port: 3001, hostname: '127.0.0.1' }); ``` Install the same major version of `@modelcontextprotocol/sdk` that `staleproof` depends on (1.x). The tools write to the graph (`propose_edit`, `record_verdict`, `write_decision`) and this endpoint has no authentication: keep it on `127.0.0.1` or behind your own auth. For tests, connect a client in the same process: ```ts import { Client } from '@modelcontextprotocol/sdk/client/index.js'; import { InMemoryTransport } from '@modelcontextprotocol/sdk/inMemory.js'; import { createMcpServer, GraphStore, loadConfig } from 'staleproof'; const config = loadConfig(); const store = await GraphStore.connect(config.falkordbUrl); try { const [clientSide, serverSide] = InMemoryTransport.createLinkedPair(); await createMcpServer(store.org(config.org), config).connect(serverSide); const client = new Client({ name: 'my-test', version: '0.0.0' }); await client.connect(clientSide); const { tools } = await client.listTools(); console.log(tools.map((t) => t.name).join(', ')); const r = await client.callTool({ name: 'impact', arguments: { pr_url: 'https://github.com/FalkorDB/falkordb-py/pull/262' } }); console.log(r.content); await client.close(); } finally { await store.close(); } ``` Each successful call returns the summary and the data as JSON as two text items, plus a third with the named Cypher statements it ran, their row counts and timings, when there were any. ### Test with ScriptedModel (no API key) `ScriptedModel` stands in for the model, and `recordPullRequest` stands in for GitHub, so a check runs with nothing but a FalkorDB. This is how the project's own tests drive the agent loop ([test/agent.test.ts](../test/agent.test.ts)). ```ts import { GraphStore, indexOrg, loadConfig, recordPullRequest, runAgentCheck, ScriptedModel } from 'staleproof'; // PullRequestInfo and PrDiff are not exported; take them from the function signature. type PrInfo = Parameters<typeof recordPullRequest>[1]; type Diff = Parameters<typeof recordPullRequest>[2]; const store = await GraphStore.connect('falkor://localhost:6379'); const graph = store.org('staleproof-scripted-test'); // a throwaway graph const config = loadConfig({}); // defaults only: ignores the environment try { // Plain directories work: without .git the indexer walks the tree. await indexOrg(graph, [ { repo: 'acme/client', dir: './fixtures/client', sha: 'fixture' }, { repo: 'acme/docs', dir: './fixtures/docs', sha: 'fixture' }, ]); // Record the pull request directly: no GitHub call, no git. const pr: PrInfo = { owner: 'acme', repo: 'client', fullName: 'acme/client', number: 1, url: 'https://github.com/acme/client/pull/1', title: 'Drop graph_names', body: '', author: 'dev', state: 'closed', mergedAt: '2026-10-01T00:00:00Z', mergeSha: 'after', baseSha: 'before', baseRef: 'main', headSha: 'head', headRef: 'drop-graph-names', }; const diff: Diff = { before: 'before', after: 'after', files: [], docsTouched: [], symbols: [{ repo: 'acme/client', qualname: 'client.Client.graph_names', name: 'graph_names', kind: 'method', change: 'removed', before: 'def graph_names(self)', path: 'client/__init__.py', lineno: 20, endLineno: 21, }], }; await recordPullRequest(graph, pr, diff); // Each turn is computed from the conversation so far. const model = new ScriptedModel((history, turn) => { if (turn === 0) return { thinking: 'Start from what the PR reaches.', calls: [{ name: 'impact', input: {} }] }; if (turn === 1) { const last = history.at(-1); const block = Array.isArray(last?.content) ? last.content.find((b) => b.type === 'tool_result') : undefined; // A tool result's content is JSON {summary, data}; an error result is plain text. const text = block?.type === 'tool_result' && !block.is_error && typeof block.content === 'string' ? block.content : '{}'; const { data } = JSON.parse(text) as { data?: { sections: { section: string }[] } }; const ids = (data?.sections ?? []).map((s) => s.section).filter((id) => id.includes('#')); if (ids.length) return { calls: [{ name: 'record_verdict', input: { section_ids: ids, verdict: 'skipped', reason: 'scripted test' } }] }; } return { text: 'Every impacted section has a verdict.' }; }); const result = await runAgentCheck({ graph, config }, model, pr.url); console.log(result.status, result.toolOrder, result.missing); // e.g. done [ 'impact', 'record_verdict' ] [] console.log(model.requests.length, 'model turns recorded'); } finally { await graph.drop(); await store.close(); } ``` The fixtures are two small directories: `fixtures/client/client/__init__.py` defines `class Client` with a method `graph_names`, and a Markdown page in `fixtures/docs/` names it in inline code as `client.Client.graph_names()`. The tool inputs can omit `pr_url`: in a check it defaults to the pull request under check. `runAgentCheck` skips ingestion because `recordPullRequest` marked the pull request ingested. A test that also calls `run_example` needs the sandbox runtime (git, and `uv` or `python3`). ### Run your own Cypher The named queries and `graph.run` are enough for reports. Precision, counted from maintainer reviews: ```ts import { GraphStore, loadConfig } from 'staleproof'; const config = loadConfig(); const store = await GraphStore.connect(config.falkordbUrl); try { const { rows } = await store.org(config.org).run<{ verdict: string; status: string; n: number }>( `MATCH (v:Verdict) WHERE v.status IN ['accepted', 'rejected'] RETURN v.verdict AS verdict, v.status AS status, count(v) AS n ORDER BY verdict, status`, ); console.table(rows); } finally { await store.close(); } ``` The node and edge properties are listed in [Graph model and Cypher](GRAPH-MODEL.md). Variable-length patterns should bind a path variable (`MATCH p = (s)<-[:CALLS*0..3]-(t)`), for the reason in [Troubleshooting](troubleshooting.md#falkordb-version-notes). ## Types that are not exported Even when compiling against the source, several types the functions use are not exported from the package entry: the tool data types (`ImpactData`, `ImpactedSection`, `SectionText`, `RunExampleData`), `RunRecord`, `DocsPatch`, `PullRequestInfo`, `PrRef`, `PrDiff`, `IngestedPr`, `IndexProgress`, `Facts`, `AppOptions`, `CheckOptions`, `AnthropicModelOptions`, `ScriptedTurn`, `QueryTrace`, `HostedEvent`, `WorkspaceInfo`, `TarballLimits`, `McpHttpOptions` and `McpResolver`. Derive them from the functions that use them, for example `Parameters<typeof recordPullRequest>[1]` or `Awaited<ReturnType<typeof buildDocsPatch>>`; `queries.NamedQuery` is reachable through the `queries` namespace. ## Limits - No type declarations are published, and the package is at version 0.3.0: exports may change between versions. - The docs audit, `docs-for` and the working-tree check have no function export yet. Reach them through the MCP tools `createMcpServer` serves (`audit_docs`, `docs_for_symbol`, `check_changes`), the HTTP routes `createApp` serves, or the CLI. - Some functions shell out: `checkoutRepo`, PR ingestion with git, and example runs need `git` (`checkoutOrDownload` and `downloadRepo` do not); example runs need `uv` or `python3`; the managed scratch server and `staleproof db` run `bash` scripts. - State is per process: example runs are serialised process-wide, and the git and sandbox availability checks are cached for the life of the process. - A few internals read the environment directly rather than `Config`: `STALEPROOF_INGEST` (indexing and PR ingestion), `GITHUB_TOKEN` and `GH_TOKEN` for git clones, and the Anthropic credential variables (`AnthropicModel`, `defaultModelFactory`). - Nothing in the library authenticates callers. Anything you expose over HTTP or MCP can write verdicts and decisions to the graph. ## Related - [HTTP API](http-api.md): the routes `createApp` serves and the trace event fields - [Tools reference](tools.md): inputs, results and errors of every tool - [Core concepts](concepts.md): impact, verdicts, checks and ids - [Configuration](configuration.md): what `loadConfig` reads - [MCP server](mcp.md) --- <!-- page: Core concepts · How it works · source: https://staleproof.dev/docs/concepts.md --> # Core concepts This page defines the vocabulary the rest of the documentation uses: the graph and its ids, how a pull request becomes symbol-level changes, impact and its evidence paths, clearance, static checks, predicted breaks and the audit, ranking, example runs, verdicts, decisions, memory, checks, the two check modes and the docs patch. Every other page links here for definitions, so each term has its own heading. The running examples are three pull requests on FalkorDB's Python client, checked against FalkorDB's three repositories (`FalkorDB/falkordb-py`, `FalkorDB/GraphRAG-SDK`, `FalkorDB/docs`). ## In one line each | Term | Plain meaning | |---|---| | [The graph](#the-graph) | One FalkorDB graph per organisation: code, docs, pull requests and everything checks conclude | | [Change kinds](#change-kinds) | What a pull request did to each definition: removed, renamed, signature, modified, added, export-removed | | [Severity](#severity) | `breaking`, `api` or `behavior`, derived from the change kind | | [Impact](#impact) | The doc sections and examples a pull request can make false | | [Evidence path](#evidence-path) | The path in the graph from a changed symbol to a doc, written node by node | | [Hops](#hops) | The number of edges on an evidence path | | [Resolution](#resolution) and [confidence](#confidence) | How each edge was resolved (`exact`, `by-name`, `late`) and what that makes the whole path | | [Clearance](#clearance) | The changed symbols that no doc reaches; the graph proves the absence | | [Static checks](#static-checks) | Example lines that no longer fit the code they call, found without running them (`MISMATCH` edges) | | [Predicted breaks](#predicted-breaks) | Example lines a change will break, found from the old and new signatures before anything runs | | [Checks before a pull request](#checks-before-a-pull-request) | The same walk from a working tree, an editor's files or one symbol | | [The audit](#the-audit) | How true the docs are with no change at all: a 0-100 health score | | [Ranking](#ranking) | PageRank over the links between doc pages, so the most central page comes first | | [Runs](#runs) | One execution of a docs example in the sandbox, and the before/after outcome | | [Verdicts](#verdicts) | The conclusion about one section for one pull request | | [Decisions](#decisions) | A maintainer's ruling that some docs stay as they are | | [Memory](#memory) | What every check writes back, and what the next check reads | | [Checks](#checks) | One run of the check loop on one pull request, stored with its full trace | | [Agent mode and graph mode](#agent-mode-and-graph-mode) | A model chooses the tools, or the same tools run in a fixed order with no model | | [The docs patch](#the-docs-patch) | The proposed edits as a `git apply`-able diff with a PR description | ## The graph Everything lives in one FalkorDB graph per organisation. The graph name is the organisation slug (`--org` or `STALEPROOF_ORG`; the CLI defaults to the owner of the repository a command is about, else `falkordb`), lowercased with anything outside `a-z 0-9 _ -` turned into `-`. Several organisations can share one FalkorDB server; FalkorDB's graphs are the tenancy boundary. A hosted server adds a workspace graph per repository owner, `gh-<owner>`, indexed on demand ([Deploy](DEPLOY.md#hosted-mode)). The code and docs half of the graph, which the [impact](#impact) walk follows from right to left: ```text Repo ──DEPENDS_ON──▶ Repo ▲ │ IN_REPO │ File ◀──DEFINED_IN── Symbol ──CALLS──▶ Symbol ──CHANGED_IN {kind}──▶ PullRequest ▲ ▲ MENTIONS │ │ USES │ │ DocSection ─┘ └─ CodeExample │ ▲ │ LINKS_TO │ └─IN_SECTION──┘ ▼ DocSection ``` `DocSection` and `CodeExample` nodes are also `DEFINED_IN` their `File`. The memory half (verdicts, runs, decisions, checks) is listed under [Edges](#edges). The full list of node properties and every Cypher statement is in [Graph model and Cypher](GRAPH-MODEL.md). ### Repo, File and Symbol - **Repo**: one per indexed GitHub repository: `name`, `url`, `sha` (the indexed commit), `indexed_at`, `index_version` (the index format, 3 since Staleproof 0.3), and `packages` (the top-level modules it defines: Python packages, and `js-<name>` for npm packages; empty for a docs-only repository). - **File**: one per indexed file (Python, TypeScript or JavaScript, a doc page, an example script), with its `kind`. - **Symbol**: one per definition in Python, TypeScript or JavaScript: `kind` is `module`, `class`, `function` or `method` (a `def` directly inside a class body; a `def` nested in a method is a `function`). It carries `qualname`, `name`, `signature`, the first line of its docstring, `decorators`, `visibility` (`special` for `__x__` and constructors, `private` for a leading `_` and for TypeScript's private or unexported declarations, else `public`), `exported` (whether the repository's packages export it; [Indexing](indexing.md#symbols)) and `status`. `status` is `live`, `gone` (the definition disappeared on a re-index; never deleted) or `removed` (recreated from a pull request that removed it; see [Pull request ingestion](#pull-request-ingestion)). ### DocSection and CodeExample - **DocSection**: one per heading of a Markdown or MDX page (or title of a reStructuredText page), from the heading line to the next heading of any level. Text before the first heading is a level-0 section with an empty anchor. It stores `heading_path` (the ancestor headings joined with `>`, for example `falkordb-py > Usage > Asynchronous Example`), `anchor`, `body` (the section's Markdown, up to 8000 characters), the code identifiers named in its inline code (`idents`) and a GitHub `url` pinned to the indexed commit. - **CodeExample**: one per code block in a page (any language), plus one per example script in a code repository (`examples/*.py`, `examples/*.js` and similar). It stores `lang`, `code`, `kind` (`fenced` or `script`), `idents` and `directive` (empty, `skip` or `expect-error`; [directives](indexing.md#example-directives)). How pages are split and linked is in [Indexing](indexing.md). ### Ids | Node | Id format | Example | |---|---|---| | Repo | `owner/name` | `FalkorDB/falkordb-py` | | File | `owner/name:path` | `FalkorDB/docs:commands/graph.explain.mdx` | | Symbol | `owner/name:qualname` | `FalkorDB/falkordb-py:falkordb.asyncio.falkordb.FalkorDB.__init__` | | DocSection | `owner/name:path#anchor` | `FalkorDB/falkordb-py:README.md#asynchronous-example` | | DocSection (text before the first heading) | `owner/name:path#` | `FalkorDB/docs:commands/graph.explain.mdx#` | | CodeExample | `owner/name:path:index` | `FalkorDB/falkordb-py:README.md:4` | | PullRequest | its URL | `https://github.com/FalkorDB/falkordb-py/pull/262` | | Decision | `dec-` + 10 hex characters | | | Verdict | `ver-` + 12 hex characters | | | Run | `run-` + 20 hex characters (`-err-<ms>` appended when the sandbox could not be prepared) | | | Check | `chk-` + 12 hex characters | | The anchor is the heading's slug (lowercase, punctuation removed, spaces turned into `-`, duplicates numbered `-1`, `-2`), or an explicit `{#id}` written after the heading. The example `index` counts every fenced block in the file from 0, whatever its language, so `README.md:4` is the fifth fenced block of the README. Tools that accept either kind of id (`decisions_for`) treat an id containing `#` as a DocSection id. ### Edges | Edge | From → to | Written by | |---|---|---| | `IN_REPO`, `DEFINED_IN` | File → Repo; Symbol, DocSection, CodeExample → File | [Indexing](indexing.md) | | `CALLS {resolved, line, text}` | Symbol → Symbol | Indexing | | `MENTIONS {resolved, span}` | DocSection → Symbol | Indexing; `late` ones by pull request ingestion | | `USES {resolved, line, text}` | CodeExample → Symbol | Indexing; `late` ones by pull request ingestion | | `IN_SECTION` | CodeExample → DocSection | Indexing | | `MISMATCH {kind, name, detail, line, text}` | CodeExample → Symbol | Indexing: an example line that no longer fits the code ([Static checks](#static-checks)) | | `LINKS_TO` | DocSection → DocSection | Indexing | | `DEPENDS_ON {spec}` | Repo → Repo | Indexing | | `CHANGED_IN {kind, before, after, renamed_to}` | Symbol → PullRequest | Pull request ingestion | | `OF`, `AT` | Run → CodeExample; Run → PullRequest | `run_example` | | `ABOUT`, `FOR`, `CITES`, `EVIDENCE` | Verdict → DocSection, PullRequest, Decision, Run | `propose_edit`, `record_verdict` | | `GOVERNS`, `SUPERSEDES` | Decision → DocSection or Symbol; Decision → Decision | `write_decision` (people only) | | `FOR`, `PRODUCED` | Check → PullRequest; Check → Verdict | The check loop | ## Pull request ingestion The first time any tool or check looks at a pull request, it is ingested: its metadata is read from the GitHub API, the two commits are compared, and the result is written to the graph. Later tools reuse the stored diff. A check of a pull request that is still open (`staleproof check`, `POST /api/check`, the MCP tool `check_pull_request`) first asks GitHub for its head: when new commits were pushed or it was merged since, the old diff is dropped and the pull request is ingested again. A merged pull request never changes, and when GitHub cannot be reached the stored diff stays. - **Which repositories**: the pull request's repository must be indexed in the graph. `staleproof check`, the MCP tools `check_pull_request`, `impact` and `clearance` (stdio and hosted) and a hosted server's checks index it first when it is missing. Elsewhere (`check --no-index`, `run --pr`, `serve` without `--hosted`, the library) the error is `<owner/repo> is not indexed in the "<graph>" graph. Run: staleproof index <owner/repo> (plus the repositories that document it).` - **Which commits**: for a merged pull request, the merge commit's first parent (before) and the merge commit (after). For an open or closed-unmerged one, the merge base of base and head (before) and the head (after). - **Which files**: only files the indexer classifies as library code (Python, and TypeScript or JavaScript in a published npm package) are diffed. Markdown files the pull request itself edits are recorded as `docs_touched` and shown as "This PR already edits: …". - **How**: with `git` in a local clone; on hosts without `git` (serverless), or with `STALEPROOF_INGEST=github`, through the GitHub API. Each changed definition is attached to the `PullRequest` node with a `CHANGED_IN` edge that records the [change kind](#change-kinds) and the signatures before and after. If the graph was indexed after the pull request, the symbols it removed no longer exist in the graph. Ingestion recreates them with `status: 'removed'` and links them to the docs that still name them: sections whose stored `idents` contain the name (or `Class.name`), and examples whose identifiers contain it, in the symbol's own repository or a repository with a direct `DEPENDS_ON` edge to it. Those edges are labelled `resolved: 'late'` (see [Resolution](#resolution)). Names shorter than 4 characters, names starting with `_` and generic names such as `get` or `run` are not late-linked. ### Change kinds | Change | When | Severity | |---|---|---| | `removed` | The definition exists before the pull request and not after, and is not part of a rename | `breaking` | | `renamed` | A removed definition pairs with an added one of the same kind, in the same class or module, with the same parameters (names and annotations) and return annotation | `breaking` | | `export-removed` | A module stops importing a name from its own package that it used to import (and does not define it either). Recorded against the definition the import pointed at | `breaking` | | `signature` | Same qualified name on both sides; the signature text or the decorators differ | `api` | | `modified` | Same name, signature and decorators, and a changed line touches the definition's own lines (not a nested definition; blank and comment-only lines ignored; a docstring edit counts) | `behavior` | | `added` | The definition exists only after the pull request | not traversed | A moved or renamed file changes the module name, so each of its definitions shows up as `removed` plus `added`. ### Severity Severity is a function of the change kind: `removed`, `renamed` and `export-removed` are `breaking`; `signature` is `api` (shown as "signature" in summaries and the PR comment); everything else is `behavior`. `added` changes are left out of every traversal. When several changes reach one section, the section takes the most severe. In all three falkordb-py examples, every impacted section is `behavior`: the changes reaching docs modified function bodies, not signatures. ## Impact Impact is the set of doc sections and code examples a pull request can make false. A doc is impacted when the graph has a path from a changed symbol, back through up to 3 callers in any repository, to a doc section that mentions one of those symbols in inline code, or to a code example that calls one. The `impact` tool runs one query for this (excerpt; the full text is in [Graph model and Cypher](GRAPH-MODEL.md#agent-tools)): ```cypher MATCH (pr:PullRequest {url: $pr})<-[c:CHANGED_IN]-(s:Symbol) WHERE c.kind <> 'added' MATCH p = (s)<-[:CALLS*0..3]-(t:Symbol)<-[:MENTIONS|USES]-(d) WHERE (d:DocSection OR d:CodeExample) AND d.status = 'live' RETURN d.id, length(p) AS hops, [n IN nodes(p) | coalesce(n.qualname, n.id)] AS evidence ``` Results are grouped by section: an example counts towards the section it sits in, and an example script (which has no section) is its own entry. Each section keeps its best path (higher severity, then fewer hops, then `exact` over other confidences), and sections are sorted by severity, then hops, then id. `impact` returns 40 sections per call; with more, the summary says where the next page starts (`offset`). The three running examples: - **falkordb-py#262** changed one function, `Is_Cluster`. It reaches one section, the falkordb-py README's "Asynchronous Example", whose example calls the async `FalkorDB` constructor, which calls `Is_Cluster`. - **falkordb-py#267** reaches 3 sections, 2 of them in another repository: the docs site's `GRAPH.EXPLAIN` and `GRAPH.PROFILE` pages, through `ExecutionPlan._operation_tree ← ExecutionPlan.__init__ ← Graph.explain`. - **falkordb-py#276** changed 4 symbols that reach 43 sections, 41 of them in other repositories. None of this is found by searching for the changed names: for the three falkordb-py pull requests, a whole-word grep for the changed names finds 0 of the 47 sections the graph reaches ([FACTS](FACTS.md#without-the-graph-grep-for-the-changed-names)). ### Evidence path The evidence path is the path behind an impacted section, written node by node: the changed symbol first, the doc last, symbols by qualified name and docs by id. For #262: ```text falkordb.asyncio.cluster.Is_Cluster ← falkordb.asyncio.falkordb.FalkorDB.__init__ ← FalkorDB/falkordb-py:README.md:4 ``` Read it as "`Is_Cluster` is called by `FalkorDB.__init__`, which is used by example 4 of the README". The path is the reason a section is listed, not proof that it is wrong. It is also a precondition: `propose_edit` refuses to edit a section unless the graph has a path from the pull request's changes to it, and the path is stored on the verdict. ### Hops Hops are the number of edges on the evidence path, the final `MENTIONS` or `USES` edge included. The #262 path has 2 hops (one `CALLS`, one `USES`); #267's path to the `GRAPH.EXPLAIN` example has 3. The walk allows 0 to 3 `CALLS` edges by default (`max_hops` on `impact`, `--max-hops` on `check --no-agent`), so a default impact reports 1 to 4 hops. "Within 3 call hops", in [clearance](#clearance) and elsewhere, counts `CALLS` edges only. ### Resolution Every `CALLS`, `MENTIONS` and `USES` edge records how it was resolved, in its `resolved` property: - `exact`: resolved through imports, package re-exports, attribute and return types, or base classes. - `by-name`: a name match used when exact resolution fails, allowed only inside repositories the module imports, never for generic names such as `get` or `run`, and only when 1 to 4 definitions match. See [Indexing](indexing.md). - `late`: created during pull request ingestion for a symbol the pull request removed, by matching its name against the identifiers stored on docs. On FalkorDB's repositories, 1,655 of 1,708 call edges and 633 of 716 example edges resolve exactly ([FACTS](FACTS.md#resolution-quality)). ### Confidence An impacted section's `confidence` summarises the edges on its evidence path: `late` if any edge is `late`, `exact` if every edge is `exact`, otherwise `by-name`. The PR comment shows it next to the hops, for example `2 (exact)`. Graph mode only runs examples from `exact` sections. ## Clearance Clearance is the list of changed symbols (other than `added`) that reach no doc section or example within 3 call hops in any repository. The graph proves the absence: there is no path. Reviewers can skip those changes as far as the docs are concerned. Each cleared symbol comes with a check you can run yourself: a `git grep -nw <name> -- '*.md' '*.mdx' '*.rst'` command, and `textMatches`, the number of matching lines across the local clones of every repository that has doc sections. A cleared symbol with text matches shares a name with text in the docs, but no path in the graph resolves to it; that is the case worth a second look. On hosts without local clones (serverless), `textMatches` is 0. The query counts bound paths (`OPTIONAL MATCH p = … WITH count(p) AS reach WHERE reach = 0`) rather than using a negated pattern: on FalkorDB 4.18 the anonymous variable-length form could miss callers reachable along two routes of different lengths. [test/traversal.test.ts](../test/traversal.test.ts) reproduces that shape. ## Static checks and the audit Impact says which docs a change can make false; a run proves it. Between the two, the graph can already tell, without running anything, when an example's call does not fit the code it reaches. This works on hosts with no sandbox (the hosted service), for examples that could never run (they need an API key or a placeholder host), and before a pull request exists. ### Static checks At index time, every call and import in a docs example is checked against what it resolves to, in any repository indexed with it: a keyword the function does not take, too many positional arguments, a required argument left out (Python only), a name the package does not export, a method the class does not have (Python, TypeScript and JavaScript). Each line that does not fit is a `MISMATCH` edge from the example to the symbol, with the doc line and a sentence such as `` `Client` has no parameter `hostname` ``. A check is made only where the source settles it: exact resolutions, signatures no unknown decorator wraps, classes and modules whose members are all visible, calls without `*args`/`**kwargs` splats. [Indexing](indexing.md#static-checks-of-examples) has every rule, and [example directives](indexing.md#example-directives) exempt examples that fail on purpose. `MISMATCH` edges describe the docs as indexed: drift that already exists, whatever the next pull request does. Impact reports the impacted ones as "already out of date before this pull request". ### Predicted breaks For a pull request, the same check runs on the change itself. For every example that uses a changed symbol, Staleproof compares the call as written with the code before and after: | Change | Predicted break when | |---|---| | `removed`, `renamed` | The example calls the symbol (for a `late` link, a call of that name must appear in the example) | | `export-removed` | The example imports the name from the module that stopped exporting it | | `signature` | The call fit the old signature and does not fit the new one: an unknown keyword, too many positional arguments, a new required argument. Python only, exact links only, and only when the decorators keep the signature | A call that never fit is not predicted to break: it was already broken, and shows as already out of date instead. Each prediction carries the doc line, the call, a reason such as `` `Graph.query` requires `timeout`, which the example does not pass `` and its confidence: `exact` when the example reached the symbol through imports and types, `by-name` for a removed symbol linked by name ([Resolution](#resolution)). Examples marked `skip` or `expect-error` are never predicted. Predictions appear in the `impact` result (`predicted`), at the top of the pull request comment, in the terminal output and on the web page; graph mode runs the predicted examples first, so a run can turn a prediction into proof; and the GitHub Action can fail on them (`fail-on: predicted`). ### Checks before a pull request The impact walk does not need a pull request, only changed symbols: - **A working tree**: `staleproof check` with no URL diffs the checkout's uncommitted and unpushed changes against its merge base ([CLI](cli.md#check-without-a-pull-request)); the MCP tool `check_changes` does the same for files an editor or agent sends, compared with the commit the graph indexed. - **One symbol**: `staleproof docs-for <symbol>` and the MCP tool `docs_for_symbol` walk from a symbol you are about to change to every doc that depends on it. Both give impact, predicted breaks and lines already out of date, and write nothing to the graph: no pull request, no check, no run. ### The audit With no change at all, the [audit](audit.md) reports how true the docs are now: every `MISMATCH` line, the example runs recorded at the indexed commits, and API coverage (the share of exported symbols some doc names or calls), combined into a health score from 0 to 100 with a grade. It is a CLI command, a web page with a README badge, an HTTP route, an MCP tool and the GitHub Action's scheduled mode. ## Ranking Ranking orders impacted sections by how central their page is in the docs: `algo.pageRank` over the `LINKS_TO` edges between doc sections, summed per page (`repo:path`), so the most linked-to page comes first. If the graph algorithm fails, ranking falls back to counting the inbound links to each page (`method: 'in-degree'`). Links resolve only within one repository. The `rank_sections` tool runs it; graph mode, the terminal output and the PR comment use it to order sections. ## Runs A run is one execution of a docs code example in the sandbox: a fresh virtualenv with the organisation's repositories installed from source at exact commits, a scratch FalkorDB answering on `localhost:6379`, and no other network. With a pull request, `run_example` runs the example twice: at the commit before the pull request (`before`) and at the pull request (`after`). Each execution is a `Run` node. Identical runs (same program, same commits) are answered from the graph instead of re-run. The details are in [Example runner](example-runner.md). ### Run statuses | Status | Meaning | |---|---| | `pass` | The program exited with code 0 | | `fail` | It exited with an error; `error` holds the last traceback line | | `timeout` | No exit within the time limit (30 s by default) | | `skipped` | Not run because it cannot be run honestly: needs an LLM key, a local file, placeholders, user input, an external service, or is not Python; a fragment that uses a name the page never defines; or nothing to install ([all the rules](example-runner.md#skip-rules)) | | `error` | The sandbox could not be prepared (clone, install or scratch server failed), for example a dependency conflict | ### Run outcomes The two runs of an example combine into one outcome (`fail` and `timeout` both count as failed): | Outcome | Before | After | What it means | |---|---|---|---| | `breaks` | passes | fails | The pull request broke the example: the strongest evidence that the docs are now wrong | | `fixes` | fails | passes | The pull request fixed an example that was broken | | `passes` | passes | passes | No observable break | | `already-broken` | fails | fails with the same error | Broken before this pull request; still worth fixing | | `changed-failure` | fails | fails with a different error | Broken both times, differently | | `fails` | | | A single run (no pull request) failed | | `skipped` | | | Either run was skipped | | `error` | | | Either run could not be prepared | From FalkorDB's repositories ([FACTS](FACTS.md#example-runs-sandbox)): - **#262**: the README's async example fails before the pull request with `TypeError: Redis.__init__() got an unexpected keyword argument 'himport_registry'` and passes after it: `fixes`. The pull request fixed a broken README example. - **#267**: the docs site's `GRAPH.EXPLAIN` example fails before and after with `Invalid graph operation on empty key` (it explains a query on a graph the page never creates): `already-broken`. - **#276**: 21 examples were tried at both commits: 10 pass both times, 6 fail both times (already broken), 3 cannot be installed together at those commits (a dependency conflict, recorded as `error`), and 2 were skipped (placeholders; not Python). ## Verdicts A verdict is the conclusion about one doc section for one pull request. It is a `Verdict` node linked to the section (`ABOUT`), the pull request (`FOR`), any decisions it cites (`CITES`) and any runs that support it (`EVIDENCE`). | Verdict | Written by | Means | |---|---|---| | `stale` | `propose_edit` | The section is wrong (the pull request falsified it, or one of its examples fails both before and after); the verdict carries the replacement Markdown (`new_text`) and the text it replaces (`old_text`) | | `ok` | `record_verdict` | The change does not falsify the section | | `governed` | `record_verdict` | A maintainer decision says it stays as it is; must cite the decision | | `skipped` | `record_verdict` | It could not be judged; the reason says why | Every verdict starts with status `proposed`. A maintainer can set it to `accepted` or `rejected` (`staleproof verdict accept|reject <id>`, or the HTTP API). Rejected edits are left out of the docs patch, and later checks see the review when they read the section's history. The evidence path is stored on the verdict; `confidence` is `path`, or `none` when `record_verdict` closed a section with no path. Only doc sections get verdicts, never individual examples. See [Decisions and memory](memory.md). ## Decisions A decision is a maintainer's ruling, in one sentence, attached to the doc sections or symbols it governs: for example "The async client keeps positional kwargs until 2.0", governing `FalkorDB/falkordb-py:falkordb.asyncio.falkordb.FalkorDB.__init__`. Decisions come only from people (the `decide` command, MCP `write_decision`, or `POST /api/decisions`); the agent cannot write them. A decision on a symbol also covers the docs of whatever calls it within 2 calls, and a decision replaced with `--supersedes` resolves to its newest active successor. When `decisions_for` finds one for a section, the agent can close the section with a `governed` verdict that cites it instead of flagging it again. See [Decisions and memory](memory.md). ## Memory Every check writes what it learned back to the graph: the pull request and its changes, every example run, every verdict, and the check itself with its full trace. Nothing is deleted on re-index: definitions and sections that disappear are marked `gone`, so verdicts and decisions keep pointing at them. The next check reads this memory in three ways: - identical example runs are answered from the graph instead of re-run; - `decisions_for` returns the decisions governing a section and its last 5 verdicts, with whether maintainers accepted them; - the agent's first message says what changed since the last check: pull requests ingested, verdicts (how many stale), new decisions, example runs (how many failed). See [Decisions and memory](memory.md). ## Checks A check is one run of the check loop on one pull request. It is a `Check` node linked to the pull request (`FOR`) and to the verdicts it produced (`PRODUCED`). It stores the model name (or `graph-only (no model)`), status, summary, the number of model turns and tool calls, the tool order, the impacted sections still open, token counts, and the full trace as JSON: every tool call with its input, its result and the Cypher it ran with server-side timings. | Status | Meaning | |---|---| | `done` | Finished; in agent mode, every impacted section (on every page of `impact`) has a verdict or an edit | | `incomplete` | Stopped with sections still open (step limit, time budget, or the model finished early after two reminders); the open ones are listed | | `refused` | The model declined to continue | | `error` | The check failed; the summary gives the error | A check that is still running has status `running`. See [The agent](agent.md). ## Agent mode and graph mode Both modes use the same tools over the same graph and are stored the same way. They differ in who chooses what to look at. | | Agent mode | Graph mode | |---|---|---| | Who picks the tools | A model, turn by turn, from what the tools return | Nobody: a fixed order | | Tool order | Varies per pull request (#262: impact → read_section → decisions_for → run_example → record_verdict) | impact → clearance → rank_sections → run_example, for up to 3 examples | | Needs | Model API credentials (Claude, or an Anthropic-compatible provider) | Only FalkorDB | | Impacted sections covered | All of them: after any `impact` call the loop collects every page itself, and the model pages through them | A sample: it ranks the first page (40 sections) | | Example runs | Whichever the model chooses | A sample: at most 3 (`--runs`) examples, those predicted to break first, then examples from `exact` sections on the first page | | Reads decisions and earlier verdicts | Yes, through `decisions_for` | No | | Writes verdicts and edits | Yes; every impacted section needs one before the check is `done` | No | | "Since the last check" summary | In the first message to the model | Not computed | | Statuses | `done`, `incomplete`, `refused`, `error` | `done`, `error` | | Model recorded on the Check | The model name, for example `deepseek-v4-pro` | `graph-only (no model)` | | Start it | `staleproof check <pr-url>`; `POST /api/check` with `mode: 'agent'` | `staleproof check <pr-url> --no-agent`; `POST /api/check` with `mode: 'graph'` (the server's default without credentials) | Agent mode fails loudly without credentials; it never falls back to graph mode silently. `check --no-agent --format markdown`, which the GitHub Action runs without a model key, and the MCP tool `check_pull_request` run graph mode and then render the report from the graph: the examples run before and after, impact, clearance, ranking and any verdicts already recorded. See [The agent](agent.md) for both modes in detail. ## The docs patch The docs patch turns the proposed edits for a pull request into a unified diff and a pull request description. It takes the latest verdict per section across all checks of that pull request and applies the `stale` ones that have replacement text and were not rejected. Each file is read at its indexed commit: from the local clone when there is one (`git show <sha>:<path>`), otherwise from `raw.githubusercontent.com` (serverless hosts). An edit whose section text no longer matches what was indexed, or whose file is missing at that commit, is reported as a conflict instead of applied. The description lists, per edited section, the reason, the evidence path, the before/after runs and the decisions cited, then the sections checked and left unchanged. Get it with `staleproof patch <pr-url>` (`--out` writes the diff and the description to files), MCP `docs_patch`, or `GET /api/patch`. `patch --open-pr` opens it as a pull request in your fork of each docs repository, and in the upstream repository only with `--upstream`. It needs `GITHUB_TOKEN`. The checks of falkordb-py #262, #267 and #276 ended with `ok` verdicts only: these pull requests are fixes, and no example they reach broke because of them. So no edit has been proposed on that graph, and their patches are empty. Proposing an edit and building the patch from it are exercised on the test suite's fixture organisation. ## Known limits - **Index together.** Cross-repository edges exist only between repositories indexed in the same `index` command. - **Late links do not survive a re-index.** A pull request is ingested once (again only when an open one moves), so the `late` links made for symbols it removed are lost when the docs repository holding them is re-indexed. - **`export-removed` can be a false positive.** Any module that stops importing a name from its own package counts, including the removal of an unused import, and it is classed as `breaking`. - **Moved files are not renames.** Moving or renaming a file reports each of its definitions as `removed` plus `added`. - **Only library files are diffed** (Python, and TypeScript or JavaScript in published npm packages). Changes to example scripts, tests and docs are not symbol changes. - **Static checks and predictions are conservative.** They judge only what the source settles, so their silence is not proof; TypeScript and JavaScript calls get no argument checks. - **Long sections always conflict in the patch.** A section longer than 8,000 characters is stored truncated, so its text never matches the file and the edit is reported as a conflict. ## Related - [Indexing](indexing.md): how code and docs become the graph - [Example runner](example-runner.md): how runs work and how to read them - [Decisions and memory](memory.md): decisions, verdict review and what checks write back - [The agent](agent.md): the loop, its stop condition and graph mode - [Tools reference](tools.md): every tool's parameters, results and errors --- <!-- page: Indexing · How it works · source: https://staleproof.dev/docs/indexing.md --> # Indexing This page explains what `staleproof index` reads from your repositories and how it turns them into the graph: which files count, how Python, TypeScript and JavaScript are parsed, how calls are resolved across repositories, how Markdown, MDX and reStructuredText sections, inline code and code examples are linked to the code, how each example is checked against the code it calls, how docs mark examples not to run, how repository dependencies are found, and what a re-index keeps and replaces. Read it if you maintain the repositories being indexed, or want to know why an edge exists (or does not). ## Index your organisation Index every repository that shares code or docs in one command. Cross-repository edges only exist between repositories indexed together. ```bash npx -y staleproof index FalkorDB/falkordb-py FalkorDB/GraphRAG-SDK FalkorDB/docs ``` Each argument is `owner/name`, `https://github.com/owner/name` (with or without `.git`) or `git@github.com:owner/name.git`. The graph is the one named by `--org` or `STALEPROOF_ORG`, else the owner of the first repository (`falkordb` here), on the server named by `--db` or `FALKORDB_URL`; see [Configuration](configuration.md). Add `--json` to print the full [index report](#the-index-report) instead of the summary. You do not have to run `index` before a first check: `staleproof check` (with or without a pull request), `staleproof audit <repos>`, the MCP tools `check_pull_request`, `impact`, `clearance`, `check_changes`, `docs_for_symbol` and `audit_docs`, and a hosted server index a repository the first time they see it, at its default branch head (`check --with` adds the repositories to index with it). Run `index` yourself to index several repositories together, to pick a ref, or to move the graph to newer commits. Progress goes to stderr, one line per phase: `checkout`, `parse`, `write`, `resolve`, `docs` and `done`. ## The pipeline 1. **Checkout**: each repository is cloned or fetched, or downloaded as a tarball where there is no git (see [Checkouts](#checkouts)). 2. **Parse**: every Python library file, and every TypeScript and JavaScript file of a published npm package, of every repository is parsed. The repository's [configuration](configuration.md#repository-configuration) is read first, and its `ignore` paths are left out. 3. **Symbol table**: one table is built over all the parsed repositories, so a call in one repository can resolve into another. 4. **Write and resolve**: `Repo`, `File` and `Symbol` nodes are written, then each repository's `CALLS` edges are resolved against the whole table. 5. **Docs**: for each repository with Markdown, MDX or reStructuredText pages or example scripts, `DocSection` and `CodeExample` nodes are written and linked to the code with `MENTIONS`, `USES` and `LINKS_TO`, and every example line that no longer fits the code it calls becomes a `MISMATCH` edge ([Static checks of examples](#static-checks-of-examples)). 6. **Dependencies**: `DEPENDS_ON` edges between repositories are written. Every run first creates the exact-match indexes the queries rely on (already-existing indexes are fine) and gets a run id such as `idx-20261016225012-k3f9`, stored as `seen` on every node it writes. Writes are batched 400 rows at a time. Each `Repo` node records the index format it was written with (`index_version`, 3 since Staleproof 0.3, which added `MISMATCH` edges, `exported` flags, reStructuredText and TypeScript/JavaScript): the [audit](audit.md#limits) says when a graph is older, and a hosted workspace written by an older version is indexed again on its next use. ## Checkouts - **Where**: `<STALEPROOF_HOME>/repos/<owner>/<name>` (`~/.staleproof` by default). - **First time**: `git clone --filter=blob:none` over HTTPS. Blobs are fetched when needed. - **Private repositories**: with `GITHUB_TOKEN` (or `GH_TOKEN`) set, git sends the token to `https://github.com/` through its environment configuration (an `http.extraheader`, never in the remote URL or on the command line), so every repository the token can read clones like a public one. Without a token, terminal prompts are disabled and a private repository fails to clone. - **Later**: `git fetch --prune origin`. - **Which commit**: the remote's default branch head (`origin/HEAD`, falling back to `origin/main`), checked out detached. Its SHA becomes `Repo.sha`, the indexed commit that doc URLs, example runs and the docs patch are pinned to. - **A specific ref**: `--ref <repo>=<ref>`, repeatable, is passed to `git checkout --detach`. Use `origin/<branch>`, a tag or a commit SHA: a bare branch name refers to the clone's local branch, which later fetches do not move. ```bash npx -y staleproof index FalkorDB/falkordb-py FalkorDB/docs --ref FalkorDB/falkordb-py=origin/main ``` Files are listed with `git ls-files`, so untracked build output never enters the graph. ### Without git: tarballs When `git` is not installed (a serverless host), or with `STALEPROOF_INGEST=github`, each repository is downloaded as a tarball instead ([src/indexer/tarball.ts](../src/indexer/tarball.ts)): - **Where from**: `https://codeload.github.com/<owner>/<name>/tar.gz/<ref>` without a token, which costs no GitHub API request. With `GITHUB_TOKEN` (or `GH_TOKEN`), the API's `repos/<owner>/<name>/tarball/<ref>` endpoint, one API request, which also reads private repositories the token can see. - **Which commit**: `<ref>` defaults to `HEAD`, the default branch head. The commit SHA comes from the archive's pax header (the `comment=<sha>` record `git archive` writes), so learning it costs no API call. `--ref` must be a branch, tag or SHA GitHub knows; `origin/<branch>` exists only in a clone. - **What is extracted**: only the files the indexer reads, by the rules in the next section, plus every `pyproject.toml`, `requirements*.txt` and `package.json`; paths with a component starting with `.` are skipped, except a root `.staleproof.toml`. They go to `<STALEPROOF_HOME>/tarballs/<owner>/<name>/<sha>`, and a directory walk replaces `git ls-files`. - **Limits**: a compressed archive over 150 MB, or more than 4000 extracted files (3000 on a hosted server), is refused with `<repo> is too large to index from an archive here (...)` and the command to run it locally. Example runs and the text-match counts of `clearance` still need a git clone, so a host without git indexes and checks but runs no examples. ## Which files are indexed Each tracked path is classified by these rules, in order: | Rule | Paths | Result | |---|---|---| | 1 | Anywhere under `.github/`, `node_modules/`, `venv/`, `.venv/`, `build/`, `dist/`, `site-packages/`, `__pycache__/`, `.tox/`, `.mypy_cache/` | Skipped | | 2 | `.md`, `.mdx` and `.rst` files | Doc pages, except housekeeping files (names starting with `CHANGELOG`, `CHANGES`, `HISTORY`, `CONTRIBUTING`, `CODE_OF_CONDUCT`, `SECURITY`, `SUPPORT`, `AGENTS`, `CLAUDE`, `GEMINI`, `LICENSE`, `NOTICE`, `PULL_REQUEST_TEMPLATE`, `ISSUE_TEMPLATE`, any case) and pages under `test/`, `tests/`, `testing/` or `fixtures/`, which are skipped | | 3 | `.ts`, `.tsx`, `.mts`, `.cts`, `.js`, `.jsx`, `.mjs`, `.cjs` files | Tests and tooling are skipped: under `__tests__/`, `__mocks__/`, `test/`, `tests/`, `test-utils/`, `e2e/`, `cypress/`, `playwright/`, `fixtures/`, `coverage/`, `out/`, `.next/`, `.docusaurus/` or `.vitepress/`; named `*.test.*`, `*.spec.*`, `*.stories.*`, `*.bench.*`, `*.d.ts`, `*.min.js` or `*.config.*`. Under `example/`, `examples/`, `sample/` or `samples/`: example scripts. Under `script/`, `scripts/`, `benchmark/`, `benchmarks/`, `docs/`, `site/` or `website/`: skipped. Everything else is library code, kept only when it belongs to a published npm package ([TypeScript and JavaScript](#typescript-and-javascript)) | | 4 | Anything else that is not `.py` | Skipped (notebooks, for example) | | 5 | Tests: under `test/`, `tests/` or `testing/`, or named `test_*.py`, `*_test.py`, `conftest.py` | Skipped | | 6 | `setup.py`, `noxfile.py`, `fabfile.py` | Skipped | | 7 | Under `example/`, `examples/`, `sample/`, `samples/`, `tutorial/`, `tutorials/`, `notebook/`, `notebooks/` | Example scripts (indexed as code examples, not library code) | | 8 | Under `script/`, `scripts/`, `benchmark/`, `benchmarks/`, `docs/` | Skipped | | 9 | Everything else ending in `.py` | Python library code | Then the repository's `ignore` globs ([Repository configuration](configuration.md#repository-configuration)) drop any file they match. Rule 7 runs before rule 8, so `docs/examples/quickstart.py` is an example script. Example scripts are how examples shipped inside a code repository (such as GraphRAG-SDK's `graphrag_sdk/examples/*.py`, or a package's `examples/quickstart.js`) become docs. A hosted server ([hosted mode](DEPLOY.md#hosted-mode)) also skips translated docs: paths under `docs/`, `doc/`, `documentation/`, `i18n/`, `locale/`, `locales/`, `translation/`, `translations/` or `content/` followed by a language directory, such as `docs/zh/`, `docs/pt-BR/` or `i18n/ja/` (`TRANSLATION_DIR` in [src/hosted/workspaces.ts](../src/hosted/workspaces.ts)). The CLI indexes them. ## Python parsing Python is parsed with tree-sitter compiled to WebAssembly (`web-tree-sitter` and the grammar vendored at `grammars/tree-sitter-python.wasm`), so no native build is needed. Tree-sitter tolerates syntax errors: a file with an error still yields the definitions it can read. A file that cannot be read or parsed at all is listed in the report's `parseErrors` as `<path>: <message>`. ### Module names A package root is the topmost directory of a chain of directories that each contain an `__init__.py`; module names start at its parent. `graphrag_sdk/src/graphrag_sdk/core/connection.py` becomes `graphrag_sdk.core.connection`, so `src/` layouts work. A file whose directory has no `__init__.py` is a loose module named after its full path (`src/util.py` becomes `src.util`). `Repo.packages` lists the first component of every module name; it is how the indexer knows which repository defines `falkordb` or `graphrag_sdk`. ### Symbols Each module yields `Symbol` nodes of four kinds: `module`, `class`, `function` and `method`. A `method` is a `def` directly in a class body; a `def` nested inside a method is a `function`. Each symbol stores: - `qualname`: `<module>.<Class>.<method>`, for example `falkordb.asyncio.falkordb.FalkorDB.__init__`; the id is `<owner>/<name>:<qualname>`. - `signature`: `def name(<parameters>)` (or `async def`) with ` -> <return>` when annotated; `class Name(<bases>)` for classes. - `doc`: the first line of the docstring. - `decorators`: as written, without `@` (`property`, `x.setter`). - `visibility`: `special` for `__name__`, `private` for a leading `_`, otherwise `public`. - `exported`: whether the repository's packages export it. In Python: a name bound in a public package's `__init__.py` (or a top-level module) that resolves to the organisation's definitions, following `from .x import *` and imports from private modules (`httpx._api.get` is exported as `httpx.get`), plus the public methods of exported classes and of their base classes in the organisation. A package with a module-level `__getattr__` (PEP 562 lazy exports) gets no export flags, because its source does not show what it offers. The [audit](audit.md#api-coverage) measures API coverage over exported symbols. - `path`, `lineno`, `end_lineno`. ## TypeScript and JavaScript TypeScript and JavaScript are parsed with the vendored tree-sitter grammars `grammars/tree-sitter-typescript.wasm` (`.ts`, `.mts`, `.cts`) and `grammars/tree-sitter-tsx.wasm` (`.tsx`, `.js`, `.jsx`, `.mjs`, `.cjs`), into the same shape as Python, so one resolver and one graph serve both. - **Packages.** Every `package.json` outside `node_modules/` that has a `name` and is not `private` is a published package. A file belongs to the nearest enclosing package directory, so monorepos with several packages work. Files in no published package (a docs site, an app, build tooling) are not library code and are not indexed. - **Module names.** `js-<package>` plus the path inside the package, with a leading `src/` dropped and `index` files naming their directory: `packages/core/src/graph/client.ts` of `@acme/core` is `js-acme-core.graph.client`. The `js-` prefix keeps an npm `falkordb` apart from the Python `falkordb`; messages show the npm name (`@acme/minigraph`). The package's own name resolves to its entry file (from `source`, `types`, `module`, `exports["."]` or `main`, mapped from `dist/`, `lib/` or `build/` back to `src/`, with `src/index.ts` and `index.js` as fallbacks), which it re-exports. - **Symbols.** Classes, functions (declarations and `const f = () => …`), methods (the constructor is `constructor`, visibility `special`; getters are recorded as properties), with their signatures and return types. Methods marked `private`, `protected` or `#name` and top-level declarations that are not exported are `private`. - **Imports and calls.** ES `import` and `export` (named, default, `* as ns`, re-exports and `export *`), `require(...)` with destructuring, relative imports resolved to the files they name (`./graph`, `./graph.js`, `../util/index`), typed bindings and `this.x` fields, `new Client()`, and structural callees such as `this.graph.query()`. `Promise<…>`, arrays and `| null` are unwrapped from types. - **Exports.** Only what a module says it exports counts; the package's API is what its entry file exports, through re-exports. - **Dependencies.** `dependencies`, `peerDependencies` and `devDependencies` that name another indexed package become `DEPENDS_ON` edges. Arguments are not recorded: JavaScript accepts any argument count, so JavaScript and TypeScript calls are checked only for names that do not exist, never for arity ([Static checks of examples](#static-checks-of-examples)). ## Call resolution Every call site is resolved against one symbol table built from all the repositories in the `index` command. The resolver is static: it follows what the source says, without running it. ### Exact resolution A call resolves exactly when the callee evaluates to a definition in the organisation. The resolver follows: - **Imports**: `import a.b`, `import x as y`, `from m import n as a`, relative imports (`from . import x`, where one dot in an `__init__.py` means the package itself) and `from m import *`. - **Package re-exports**: `pkg.Name` resolves through a `from .module import Name` in the package's `__init__.py`, following import chains through modules and star imports; `pkg.Class.method` resolves through the class. - **Attribute types**: a variable bound to `Class(...)`, or to a call whose return type is known, is an instance of that class, so `g.query(...)` resolves to the class's method. - **Return annotations**: a function annotated `-> AsyncGraph` returns an `AsyncGraph`. Without an annotation, the first `return` expression that evaluates to something is used. `Optional[...]`, `Awaitable[...]`, `Coroutine[...]` and `X | None` are unwrapped. - **Base classes**: a method not defined on the class is looked up on its bases, recursively. - **`self.x = ...` assignments**: an assignment to `self.<attr>` in any method is recorded on the class, so `self._graph.query(...)` in one method resolves through `self._graph = self._driver.select_graph(...)` in another, into another repository if that is where `select_graph`'s return type lives. When an attribute has several assignments, one that resolves into the organisation wins over one typed `Any` or external. - **Properties**: `obj.prop` on a `@property` or `cached_property` evaluates to the property's return type. - **Other bindings**: `with X as y`, and annotated parameters (`def f(g: Graph)`). Calling a class adds a second exact edge to its `__init__` (searched through bases too). That is how #262's README example reaches `FalkorDB.__init__`, and from there `Is_Cluster`. Not tracked: `for` loop targets, tuple unpacking, the walrus operator, and anything only known at run time. Each lookup has a depth limit, so cycles in imports or types end. ### By-name fallback When exact resolution fails, an attribute call `x.m()` can still get edges by name, under these rules: - Only attribute calls. A bare call `foo()` that does not resolve gets no edge. - Only when the receiver is unknown. If it evaluates to something outside the organisation (`redis.Redis().get()`), there is no edge. - Never for generic names: 105 names such as `get`, `set`, `run`, `execute`, `close`, `connect`, `build`, `create` and `data` (the `GENERIC` list in [src/indexer/resolve.ts](../src/indexer/resolve.ts)), and never for names starting with `__`. - Candidates are the functions and methods named `m` in the module's own repository and in the repositories that define the top-level packages the module imports. - Edges are written only when there are 1 to 4 candidates, one edge to each. Every edge records how it was found in `resolved`: `exact` or `by-name`. One edge is kept per caller and callee, with `exact` preferred, and calls of a function to itself are dropped. ### Why edges are labelled A by-name edge can be wrong: it says "a method with this name, in a repository this module imports". The label travels with it. An impacted section whose evidence path contains a `by-name` edge has [confidence](concepts.md#confidence) `by-name`, the PR comment and the terminal output show it, and graph mode only runs examples from `exact` sections. Nothing presents a guess as a proven path. ## Docs ### Sections Markdown and MDX pages are read with a tolerant line scanner, not a strict MDX parser: real docs sites (Mintlify components, stray braces) break strict parsers, and only headings, fenced code, inline code and links are needed. reStructuredText pages have a scanner of their own with the same output ([below](#restructuredtext-sphinx)). - **Headings**: ATX headings only (`#` to `######` at the start of a line). Setext headings, indented headings and JSX or HTML headings are not recognised, and a `#` line inside a fence is never a heading. - **Sections**: each heading starts a section that runs to the next heading of any level. Text before the first heading becomes a level-0 section with an empty anchor, kept only if it has content. - **Page title**: the front matter `title:`, else the first H1, else the file name. - **`heading_path`**: the ancestor headings joined with ` > `, starting from the page's own headings: `falkordb-py > Usage > Asynchronous Example`. - **Anchors**: an explicit `{#id}` after the heading, else a slug of the heading text (lowercase, punctuation removed, whitespace turned into `-`). Links, tags, backticks and word-boundary emphasis are removed from heading text first. Duplicates get `-1`, `-2`. - **Body**: the section's Markdown with its heading line, trimmed and truncated to 8000 characters. - **URL**: `https://github.com/<repo>/blob/<indexed sha>/<path>#L<line>`. ### Mentions A section `MENTIONS` a symbol when its inline code names it. Only inline code counts: prose and fenced blocks do not. A span names code when: - it does not start with a shell or tool word followed by a space (`pip`, `npm`, `uv`, `docker`, `git`, `curl`, `cd`, `export`, `python`, `node` and similar); - it is a dotted identifier, optionally called: `Graph.ro_query`, `db.select_graph("x")`, `select_graph()`; - it is not all capitals (configuration keys, Cypher keywords) and not a Cypher procedure (`algo.`, `db.idx.`, `GRAPH.` and similar). Each accepted identifier is matched against symbols in repositories that have code (never a module): - **Dotted names** resolve exactly through modules and re-exports (`falkordb.FalkorDB`). Otherwise symbols whose qualified name ends with `.Graph.ro_query` match by name, if there are 1 to 4. A dotted name starting with a lowercase variable that is not a package (`db.select_graph()`) falls back to its last part. - **Bare names** link only when they look like code (called, or containing `_` or a capital), are at least 3 characters, are not generic, do not start with `_`, and are not properties. They link only into the repositories the page is about: the page repository's own packages, the packages its Python, JavaScript and TypeScript examples import, and the packages named in its install commands (`pip install graphrag-sdk`) or inline code (`falkordb`). Again only with 1 to 4 candidates. The section also stores every identifier it named (and each one's last part) in `idents`. Pull request ingestion uses them to [late-link](concepts.md#resolution) symbols that were removed before the graph was indexed. ### Code examples Every fenced block (backticks or tildes, any language) is a `CodeExample`, with id `<repo>:<path>:<index>` where `index` counts all fenced blocks in the file from 0 (in reStructuredText, every code block the scanner finds). It is placed in the section that contains its opening fence (`IN_SECTION`). Details: - **Language**: the info string, normalised: `py`, `python3`, `py3`, `pycon` become `python`; `sh`, `shell`, `zsh`, `console` become `bash`; `js` becomes `javascript`; `ts` becomes `typescript`. - **Indentation**: code inside an indented fence (common in MDX components) is dedented. An unclosed fence runs to the end of the file and is kept. - **Size**: the stored code is truncated to 20000 characters. - **REPL transcripts**: when a block contains `>>> ` lines, only the `>>>` and `...` lines are kept, prompts removed; output lines are dropped. Python examples are resolved like code, cumulatively per page. The page's Python blocks are concatenated in order and parsed as one throwaway module; calls in each block are attributed to that block, but names resolve through everything above it. A later fragment that calls `g.query(...)` resolves through an earlier block's `g = db.select_graph(...)`. Each call becomes a `USES` edge (with `resolved`, the line and the call text) to a symbol in a repository with code. Definitions inside docs examples never become by-name candidates for anything else. When a mention or call has several candidates and not all are exact, the page's own exact uses decide: by-name candidates are kept only if they (or their class) are among the symbols the page's examples reach exactly. If that keeps nothing, all candidates stay. JavaScript and TypeScript examples (fences tagged `js`, `javascript`, `ts`, `typescript`, `jsx`, `tsx`, `mjs`, `cjs` or `node`) resolve the same way, cumulatively per page in their own throwaway module, against the organisation's npm packages: `import { Client } from 'falkordb'` and `require('falkordb')` reach the package `js-falkordb`. They get `USES` edges and static checks, but are never run. Example scripts (`examples/*.py`, `examples/*.js`, `examples/*.ts` and the like) are one `CodeExample` each, id `<repo>:<path>:0`, `kind: 'script'`, with no section; they are resolved on their own, not cumulatively. Examples in other languages (shell, Cypher, …) are indexed (code, identifiers) but get no `USES` edges, so they are never reached by an impact walk through calls and never run. Each `CodeExample` stores `directive`: empty, `skip` or `expect-error` ([Example directives](#example-directives)). ### reStructuredText (Sphinx) `.rst` pages are read with a tolerant line scanner that produces the same sections, examples, inline code and links as the Markdown scanner, so Sphinx docs (requests, Flask, …) link to code like Markdown docs do. Unknown directives never fail a page. - **Titles**: an unindented line underlined, or overlined and underlined, with one punctuation character repeated at least 3 times. Levels follow the order in which each style first appears on the page, as in docutils. The first title is the page title (else the file name). Anchors are slugs of the title, duplicates numbered. - **Code examples**: `.. code-block:: <lang>`, `.. code::` and `.. sourcecode::` (in their language, else the page's `.. highlight::` language, default Python); `.. testcode::`, `.. doctest::`, `.. testsetup::`, `.. ipython::` and `.. jupyter-execute::` (always Python); `::` literal blocks (Python when the default language is Python and the block looks like Python code, `bash` when it has `$ ` prompts, else `text`); and `>>>` doctest paragraphs. Directive options (`:linenos:`, `:caption:`) are skipped. `meta` records the directive (`testcode`, `doctest`, `literal`, …). - **Inline code**: ` ``literals`` ` and the API roles `:func:`, `:meth:`, `:class:`, `:attr:`, `:obj:`, `:exc:`, `:data:`, `:const:`, `:any:` (also with `:py:`), with `~` and `!` prefixes and `Title <target>` forms handled; `:func:` and `:meth:` targets count as calls. `:mod:` is not a mention. These become `MENTIONS` like inline code in Markdown. - **Links**: `` :doc:`path` `` is a page link (an absolute `/user/quickstart` is relative to the docs source directory, so it is matched against any indexed page ending in `user/quickstart.rst`, `.md` or `.mdx`); `` :ref:`label` `` points at the section a `.. _label:` line precedes on the same page. - **Skip markers**: `.. staleproof: skip`, `.. staleproof: expect-error` and Sybil's `.. skip: next` before a block ([Example directives](#example-directives)). - **Not followed**: `.. include::` and the autodoc directives (`.. autofunction::`, `.. autoclass::`): the text Sphinx generates from docstrings is not in the source. Text before the first title that holds only labels and directives is not a section. ### Links Links between doc sections become `LINKS_TO` edges, the input to [ranking](concepts.md#ranking). The scanner reads Markdown links `[text](target)` and `href="..."` attributes outside headings and fences. - Targets with a scheme (`https:`, `mailto:`) are skipped. - `#anchor` points into the same page. A path starting with `/` is relative to the repository root (site routes such as `/cypher/match`); anything else is relative to the linking file. - The target file is the first indexed page among: the path as written, the path with `.mdx`, `.md` or `.rst` (after dropping a trailing `/` and any `.md`, `.mdx`, `.rst` or `.html`), and `<path>/index.mdx`, `<path>/index.md`, `<path>/index.rst`, `<path>/README.md`. For a reStructuredText page, or an absolute path that finds nothing, any page whose path ends in the target (with one of those extensions) is used, since Sphinx paths are relative to the docs source directory rather than the repository. - If the anchor exists on that page, the link points to that section; otherwise to the page's first section. - Links resolve only within the same repository, and self-links are dropped. ## Static checks of examples While linking each example, the indexer checks its calls and imports against the code they resolve to, in any repository of the run, and records every line that no longer fits as a `MISMATCH` edge from the `CodeExample` to the symbol concerned (the callee, or the module or class that lacks the name). The edge stores `kind`, `name`, `detail` (the sentence shown to people), `line` (the line in the doc file) and `text` (the callee or import as written). Nothing runs. The [audit](audit.md) lists them, a pull request check shows the impacted ones as already out of date, and [`docs_for_symbol`](tools.md#docs_for_symbol) shows the ones on the docs it returns. | Kind | What the line does | Detail, from the test fixtures | Checked in | |---|---|---|---| | `unknown-keyword` | Passes a keyword the signature does not have | `` `Client` has no parameter `hostname` `` | Python | | `too-many-arguments` | Passes more positional arguments than the signature takes | `` `Graph.query` takes at most 2 positional arguments; the example passes 3 `` | Python | | `missing-argument` | Leaves out a required argument | `` `Graph.query` requires `timeout`, which the example does not pass `` | Python | | `missing-import` | Imports from an organisation package a name no module there defines or binds | `` `miniclient` has no `connect` ``, `` `@acme/minigraph` has no `Gone` `` | Python, JavaScript, TypeScript | | `missing-attribute` | Calls a method or function a module or class does not have | `` `Graph` has no attribute `read_only` ``, `` `Graph` has no attribute `readOnly` `` | Python, JavaScript, TypeScript | The checks only report what the source settles, so that a reported line fails the same way when run: - **Exact resolutions only.** A call is judged only when its callee resolved exactly, through imports, types and re-exports. A by-name or unresolved call is never judged. - **Signatures that can be trusted.** Argument checks need one definition under the name in its repository (no `@overload` stubs or conditional definitions), a readable signature, no `@property`, and only decorators known to keep the signature: `staticmethod`, `classmethod`, `abstractmethod`, `override`, `final`, `cache`, `lru_cache`, `deprecated` and `wraps`, also with their `functools.`, `typing.`, `typing_extensions.`, `abc.` or `warnings.` prefixes. Any other decorator (a click command, a retry wrapper) may change what the function accepts, so its calls are left alone. - **Constructors.** A call to a class checks its `__init__` (searched through base classes), without `self`, unless the class has a `__new__` (its own or a base's) or is not closed (next rule). A method called on an instance or a `classmethod` drops its first parameter; a method called through the class, or a `staticmethod`, does not. - **Closed classes and modules.** A missing attribute is reported only on a class whose members are all visible: no class decorators, no metaclass, no `__getattr__` or `__getattribute__`, and every base class in the organisation (or one of `object`, `ABC`, `Generic`, `Protocol`), recursively. A name counts as present when it is a method, a class-level assignment or annotation, or a `self.<name> = …` in any method, here or in a base class. A module must have no module-level `__getattr__` and no star import from outside the organisation, and a name it imports, assigns or star-imports from the organisation counts as present. Names starting with `__` are never judged. - **Splats.** `f(*args)` makes the positional count unknown (no `too-many-arguments` or `missing-argument`); `f(**options)` makes the keywords unknown (no `missing-argument`). A signature with `*args` takes any number of positional arguments, and one with `**kwargs` any keyword. - **Imports.** Only imports from the organisation's own packages are judged; `from x import *` lines are skipped. - **Marked examples.** Examples with a `skip` or `expect-error` directive, and examples matched by `skip-examples`, are linked but not checked. The `index` summary adds `<n> mismatches (staleproof audit <repo>)` to each docs repository's line, and the [index report](#the-index-report) has the count per repository. ### Example directives Docs can mark an example as not runnable or as failing on purpose. A marked example is still linked to the code (impact still reaches its section, and its bindings serve the examples after it on the page), but it is never run, never statically checked and never predicted to break. | Where | `skip` | `expect-error` | |---|---|---| | Markdown comment on a line before the fence (blank lines in between are fine) | `<!-- staleproof: skip -->`, `no-run` or `ignore` in place of `skip` | `<!-- staleproof: expect-error -->` or `raises` | | MDX comment before the fence | `{/* staleproof: skip */}` | `{/* staleproof: expect-error */}` | | A word in the fence's info string, such as ` ```python skip ` | `skip`, `no-run`, `norun`, `staleproof:skip`, `staleproof-skip` | `expect-error`, `staleproof:expect-error`, `raises` | | Markers other docs-testing tools use | `<!-- pytest-codeblocks:skip -->`, Sybil's `<!-- skip: next -->` | `<!-- pytest-codeblocks:expect-exception -->` | | reStructuredText comment before the block | `.. staleproof: skip` (or `no-run`, `ignore`), Sybil's `.. skip: next` | `.. staleproof: expect-error` (or `raises`) | A comment counts only when it is the nearest non-blank line above the block: prose in between cancels it. Case does not matter. Whole pages or directories are marked with `skip-examples` in the repository's [configuration](configuration.md#repository-configuration). ## Repository dependencies `DEPENDS_ON` edges say which repository builds on which. They decide what gets installed for an [example run](example-runner.md) and which repositories may [late-link](concepts.md#resolution) to a removed symbol. - **Declared**: every tracked `pyproject.toml` and `requirements*.txt`, in any directory, is read: `[project].dependencies`, all optional dependency groups, and `[tool.poetry.dependencies]`. A dependency whose normalised name (PEP 503) is the distribution name of another repository in the graph becomes an edge, `spec: "declares <name>"`. - **npm packages**: every `package.json` outside `node_modules/` is read the same way: its `dependencies`, `peerDependencies` and `devDependencies` that name another indexed npm package become edges, `spec: "declares js-<name>"`. - **Docs that import a package**: for doc pages, every Python example line `import pkg` or `from pkg... import`, and every JavaScript or TypeScript `from 'pkg'` or `require('pkg')`, adds an edge from the docs repository to each repository that defines `pkg`, `spec: "examples import <pkg>"` (`js-<name>` for npm packages), unless a declared edge already exists. On FalkorDB's repositories this gives three edges ([FACTS](FACTS.md#repository-dependencies)): GraphRAG-SDK declares `falkordb`, and the docs site's examples import both `falkordb` and `graphrag_sdk`. `setup.py` and `setup.cfg` are not read, and TOML files that fail to parse are ignored. ## Re-indexing Re-running `index` is safe and idempotent, and it never deletes nodes. - **Merged in place**: `Repo`, `File`, `Symbol`, `DocSection` and `CodeExample` nodes are matched by id and their properties refreshed. - **Marked `gone`**: a repository's symbols, sections and examples that this run did not see get `status: 'gone'`. They stay in the graph, so verdicts, decisions and runs that point at them keep their history. Impact walks only reach `live` docs. A symbol recreated as `removed` by a pull request returns to `live` if it reappears in the code. `File` nodes are never marked gone. - **Deleted and rebuilt**: the repository's outgoing `CALLS` edges, and every `MENTIONS`, `USES`, `LINKS_TO` and `MISMATCH` edge from its sections and examples. This includes the `late` edges pull request ingestion created; because a pull request is ingested only once, those late links are not restored. - **Kept**: `DEPENDS_ON` edges are never deleted. Pull requests, verdicts, decisions, runs and checks are not touched. A section's id comes from its heading, so renaming a heading (or moving the file) gives the section a new id. The old one is marked `gone`, and decisions or verdicts attached to it stay on the old id. Because the symbol table holds only the repositories in the current command, re-indexing one repository alone re-resolves its outgoing edges against that repository only: its calls into other repositories and its docs' links into other repositories disappear. Re-index repositories together. Indexing on first use (`check`, MCP) indexes the pull request's repository with the `--with` repositories only, and the GitHub Action's `reindex` input re-indexes the pull request's repository with its `also-index` repositories only; see [GitHub Action](github-action.md). A hosted workspace is always re-indexed whole. ## The index report `index --json` prints the `IndexReport`: | Field | Meaning | |---|---| | `org` | The graph name | | `run` | The run id (`idx-…`) | | `repos[]` | Per repository: `repo`, `sha`, `files` (indexed files of any kind), `symbols`, `calls` (`exact`, `byName`, `crossRepo`: edges whose target is in another repository), `gone` (symbols marked gone by this run), `parseErrors` | | `docs[]` | Per repository with pages or example scripts: `repo`, `pages`, `sections`, `examples` (`total`, `python`), `mentions`, `uses` (`exact`, `byName`), `links`, `mismatches` (example lines that no longer fit the code: `MISMATCH` edges), `gone` (sections and examples marked gone) | | `dependsOn[]` | `from`, `to`, `spec` | | `ms` | Wall time | Without `--json` the CLI prints one line per repository's code, one per repository's docs, and one per dependency, in this format (columns are padded): ```text Indexed <n> repositories into graph "<org>" in <seconds>s <owner/name> <sha7> <symbols> symbols <calls> calls (<cross-repo> cross-repo) [<n> parse errors] <owner/name> docs <sections> sections <examples> examples <uses> uses <mentions> mentions <links> links [<n> mismatches (staleproof audit <owner/name>)] <from> → <to> (<spec>) ``` ## Resolution quality on FalkorDB's repositories From [FACTS](FACTS.md#resolution-quality), generated from the graph of the three repositories: | Repository | Symbols | Doc sections | Code examples (Python) | |---|---:|---:|---:| | FalkorDB/GraphRAG-SDK | 1016 | 513 | 313 (252) | | FalkorDB/docs | 0 | 2618 | 1517 (165) | | FalkorDB/falkordb-py | 282 | 7 | 5 (2) | - `CALLS` edges: 1,708, of which 1,655 exact and 53 by name; 5 of them cross a repository boundary. - `USES` edges from code examples: 716, of which 633 exact and 83 by name. - `MENTIONS` edges: 594; `LINKS_TO` edges: 644. - Symbol-to-doc paths within 3 call hops that cross a repository boundary: 1,257. Only 5 call edges cross repositories, yet 1,257 symbol-to-doc paths do: the docs site has no code of its own, and its examples call the client and the SDK directly. ## Limits - **Python, TypeScript and JavaScript.** Code in other languages is not parsed. TypeScript and JavaScript count only inside a published npm package; their calls get no argument checks (JavaScript accepts any count) and their examples are linked and checked but never run. Examples in other languages (Cypher, shell) are indexed without being linked to code. - **Static checks follow exact resolutions.** A call reached by name, or not resolved, is never judged; neither is anything behind a decorator outside the allow-list, a dynamic class or module, or a splat. - **reStructuredText is read by a line scanner**, not by Sphinx: `.. include::`, autodoc output, substitutions and Sphinx extensions are not expanded, and `:ref:` links resolve within one page. - **Index together for cross-repository edges.** Edges between repositories exist only when the repositories are indexed in one command; indexing one alone drops its edges into the others. - **Static resolution.** Calls the types do not reveal fall back to labelled by-name edges inside imported repositories, or to no edge. - **Late links do not survive a re-index** of the docs repository that holds them. - **GitHub only.** Repositories are cloned or downloaded from github.com. Private repositories need `GITHUB_TOKEN` (or `GH_TOKEN`) with read access; a hosted server indexes what its own token can read. - **Docs syntax**: ATX headings only in Markdown; links resolve within one repository; section bodies are stored up to 8000 characters and example code up to 20000. - **Not read**: notebooks, `setup.py` and `setup.cfg` dependencies. ## Related - [Core concepts](concepts.md): the graph, ids, resolution and confidence - [Example runner](example-runner.md): what the `USES` edges and `DEPENDS_ON` edges are used for - [CLI](cli.md): `index` and its options - [Graph model and Cypher](GRAPH-MODEL.md#indexing): every indexing query - [Facts from the graph](FACTS.md): the numbers above, regenerated from the graph --- <!-- page: Example runner · How it works · source: https://staleproof.dev/docs/example-runner.md --> # Example runner This page explains how a docs code example is executed before and after a pull request: which repositories are installed and at which commits, how fragments are stitched together, how async examples run, which examples are skipped and why, how runs are cached, the virtualenvs, the scratch FalkorDB and the sandbox, and how to read the result. It is for anyone who wants to trust (or question) a "passes before, fails after". ## Run an example before and after a pull request ```bash npx -y staleproof run "FalkorDB/falkordb-py:README.md:4" --pr https://github.com/FalkorDB/falkordb-py/pull/262 ``` The argument is a [CodeExample id](concepts.md#ids) (`owner/repo:path:index`); `impact` and `read_section` return them. With `--pr`, the pull request is ingested first if needed, then the example runs at the commit before the pull request and at the pull request. The CLI prints each run (status, side, the `repo@sha` list installed, any stitched fragments, the skip reason or the traceback), then a one-line summary. For this example the summary reads: ```text Fails at 49aa012 and passes at f7c3de7: the PR fixes this example. ``` The README's "Asynchronous Example" failed before falkordb-py#262 with `TypeError: Redis.__init__() got an unexpected keyword argument 'himport_registry'` and passes after it ([FACTS](FACTS.md#example-runs-sandbox)). | Option | Default | Meaning | |---|---|---| | `--pr <url>` | none | Run before and after this pull request. Without it the example runs once, at the indexed commits | | `--fresh` | off | Re-run even if an identical run is recorded, and replace the recorded one | | `--json` | off | Print the tool result as JSON | The agent and MCP clients run examples with the [`run_example`](tools.md) tool; graph mode, and so the GitHub Action, runs up to 3 of them per check. The hosted service runs none. ## How a run works 1. **Load the example**: its code and language, the repositories of the symbols it `USES`, and every earlier example on the same page (for stitching). 2. **Choose what to install**: see [Install set](#install-set). 3. **Prepare the program**: convert a REPL transcript to code, apply the [skip rules](#skip-rules), stitch fragments. 4. **Look up an identical run** in the graph, unless `--fresh` (see [Run key and caching](#run-key-and-caching)). 5. **Execute**, one run at a time: check out each repository, build or reuse the [virtualenv](#virtualenvs), start and flush the [scratch FalkorDB](#scratch-falkordb), run the program in the [sandbox](#the-sandbox). 6. **Record** a `Run` node, linked to the example (`OF`) and to the pull request (`AT`). With a pull request, steps 3 to 6 happen twice: `before`, then `after`. If the `before` run is skipped, the `after` run is not attempted. The two runs combine into one [outcome](#statuses-and-outcomes). ## Install set The repositories installed for a run are: - the repositories that define the symbols the example `USES`, plus the pull request's repository; - their in-organisation dependencies, following `DEPENDS_ON` up to 3 steps; - keeping only repositories with Python packages (a docs-only repository is never installed). Each is installed at its indexed commit, except the pull request's repository, which is installed at the pull request's before or after commit. For #276, the GraphRAG-SDK examples on the docs site install GraphRAG-SDK at its indexed commit and falkordb-py at the pull request's commits. An example with nothing to install is recorded as skipped: `uses no symbol from an indexed repository`. ## Preparing the program Docs pages often split one program across several blocks: the first creates `db` and `g`, later ones only call `g.query(...)`. The runner runs an example the way a reader would: - A REPL transcript (`>>> ` lines) becomes plain code: prompts removed, output lines dropped. - An example with any `import` or `from` line runs as written. - An example with no import line is a fragment: every earlier Python example on the same page that is not itself skipped is prepended, in order, each marked `# --- stitched from <id>`. The run records their ids in `stitched`. - If the program then fails with `NameError: name 'x' is not defined` and nothing in it defines `x` (no `def x`, `class x`, `x =`, `import … x` or `as x`), the run is recorded as skipped (`fragment: … (the page never defines it)`), not as a failure. ### Pull requests that edit the page A pull request that changes the code and updates the docs page in the same commit should not be reported as breaking the old example. So when the pull request edits the page an example is on (a Markdown file of the pull request's own repository), the after run uses the example as the page reads at the pull request: - The example is found on the edited page by its section and its position within that section, or, when the page has as many examples as before, by its position on the page. The earlier examples used for stitching come from the edited page too. - If the edited page no longer has the example, the after run is not executed: `run_example` reports it as skipped with the reason `this pull request removes the example from its page`. - If the page cannot be read at either commit, the indexed version is run, as for any other example. The summary then ends with `The after run uses the page as this pull request edits it.`, `run_example` sets `editedByPr`, and the report marks the example's result `(after: the page as this PR edits it)`. The before run always uses the page as indexed. ### Async examples Docs for async libraries are often written for an async REPL (`python -m asyncio`, IPython, Jupyter), with `await`, `async with` or `async for` at the top level: ```python from falkordb.asyncio import FalkorDB db = FalkorDB(host="localhost", port=6379) graph = db.select_graph("social") result = await graph.query("RETURN 1") ``` Run as a script, that is a `SyntaxError`, which would report the example as broken when it is not. The sandbox's entry point, [python/run_example.py](../python/run_example.py), compiles the program first. When compiling fails with a top-level `await`, `async with` or `async for` (`... outside async function` or `'await' outside function`), it compiles it again with top-level `await` allowed and runs the resulting coroutine on an event loop with `asyncio.run`, as those REPLs do. Every other program runs unchanged, with `__name__` set to `"__main__"`, so `if __name__ == "__main__":` blocks still run. A run recorded before this entry point existed (Staleproof 0.1.0) keeps its `SyntaxError`: the program and commits are the same, so it is answered from the graph. Re-run it with `staleproof run <example-id> --pr <url> --fresh`. ## Skip rules An example the docs mark not to run is never prepared: one with a `skip` or `expect-error` [directive](indexing.md#example-directives) (`<!-- staleproof: skip -->`, a fence word such as `no-run`, a pytest-codeblocks or Sybil marker), or on a page the repository's `skip-examples` matches ([Configuration](configuration.md#repository-configuration)). `run_example` answers `Not run: the docs or the repository config mark this example skip.`, or for `expect-error` `Not run: the docs mark this example as failing on purpose (expect-error), so a run proves nothing.`, with verdict `skipped`, and records no run. `audit --run` never picks such examples; in graph mode one can take one of the run slots and come back skipped. Otherwise an example is recorded as skipped, with the reason, when it cannot be run honestly. The rules are checked in this order on the prepared code: 1. **Not Python**: the block's language is not Python: `not Python (bash)`, or `not Python (no language tag)`. 2. **Empty**: `empty example`. 3. **Needs an LLM provider**: it mentions a provider key variable (`OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `GOOGLE_API_KEY`, `GEMINI_API_KEY`, `AZURE_OPENAI`, `GROQ_API_KEY`, `COHERE_API_KEY`, `MISTRAL_API_KEY`), sets `model = "…"` to an OpenAI, Anthropic, GPT, Claude, Gemini, Azure, Groq, Ollama, Bedrock, Vertex, Mistral or Cohere model, or constructs `LiteLLM(`, `LiteLLMEmbedder(`, `OpenAI(`, `AsyncOpenAI(`, `Anthropic(`, `ChatOpenAI(` or `OpenAIEmbeddings(`. Reason: `needs an LLM provider (<what matched>); Staleproof does not hold provider keys`. 4. **Reads a local file**: a quoted file name ending in `.pdf`, `.csv`, `.txt`, `.json`, `.jsonl`, `.doc`, `.docx`, `.htm`, `.html`, `.xls`, `.xlsx`, `.parquet` or `.md`, unless the code also writes one (`.write`, `.dump`, `.to_csv`, `.to_json`). Reason: `reads a local file the docs do not provide (employees.csv)`. 5. **Placeholders**: a quoted string containing `<your…>` or `<insert…>` (either case), or a quoted `your-…` / `your_…` name ending in `key`, `password`, `token` or `host`. Reason: `contains placeholders to fill in (e.g. "<your-host>")`. 6. **User input**: it calls `input(`. Reason: `waits for user input`. 7. **External services**: `requests`, `httpx`, `urllib.request` or `aiohttp` with `get(` or `post(` on the same line, or any mention of `boto3`, `psycopg`, `psycopg2` or `neo4j`. Reason: `calls an external service`. Two more cases come later: - **Nothing to install** (checked after the next one): `uses no symbol from an indexed repository`. - **No sandbox on this host**: a host without `git`, or without both `uv` and `python3` (a serverless function), cannot execute examples. Such a run comes back skipped with `this deployment cannot execute examples (no git/uv sandbox runtime); runs recorded by other Staleproof hosts are answered from the graph`, and is not recorded, so a host that can run it still will. Skipped runs prove nothing about the docs either way. Of #276's examples, one was skipped for a placeholder and one for not being Python. ## Run key and caching Every run has a key: the first 20 hex characters of the SHA-1 of the prepared program, the sorted list of `repo@sha` installed, and the skip reason. Before running, the runner looks for the most recent `Run` with that key. If it finds one whose status is not `error`, it answers from it instead of running: the result is marked `cached`, the summary ends with `(recorded earlier; not re-run)`, and the run gets an `AT` edge to the pull request being checked. - The key does not include the example id, so two examples with the same program and commits share one `Run`. - `error` runs are never reused: the next request tries again. - `run --fresh` skips the lookup, re-runs, and replaces the recorded run (same id). It exists only on the CLI `run` command; checks, MCP and the HTTP API always reuse recorded runs. - A serverless host with no sandbox still answers every run recorded by another host, as long as the program and commits are identical. ## Virtualenvs Each distinct set of `repo@sha` gets its own virtualenv, built once and reused. 1. Each repository's source at its commit is extracted from the local clone with `git archive <sha> | tar -x` (missing blobs are fetched from the blob-less clone). 2. The project directory is the shallowest one containing `pyproject.toml` or `setup.py`, outside `test/`, `tests/`, `docs/`, `example/`, `examples/`, `benchmark/` and `benchmarks/`; the repository root otherwise. 3. With [uv](https://docs.astral.sh/uv/) on `PATH`: `uv venv --python 3.12`, then `uv pip install` of every project directory into it (uv fetches Python 3.12 if needed). Without uv: `python3 -m venv`, then `pip install`. Creating the venv may take up to 300 s and installing up to 600 s. 4. A marker file, `.staleproof-ok`, records the finished build; a venv without it is rebuilt. The repositories are installed from source at exact commits; their third-party dependencies resolve from the package index at install time. The cache key includes Python version `3.12`, but only uv pins it: the `python3` fallback uses whatever `python3` is installed. Install uv. Everything lives under `STALEPROOF_HOME` (default `~/.staleproof`, `/tmp/staleproof` on Vercel): ```text ~/.staleproof/ repos/<owner>/<name>/ blob-less clones, shared with indexing and pull request ingestion cache/github/<owner>/<repo>/pulls/ pull request metadata from the GitHub API venvs/<key>/ one virtualenv per (Python 3.12, sorted repo@sha list) src/<key>/<owner>__<name>/ sources extracted while a venv builds; removed afterwards falkordb/ falkordblite binaries and data for `db start` and the managed scratch server ``` The venv key is the first 16 hex characters of the SHA-1 of the Python version and the sorted `repo@sha` list. Delete `venvs/` to force rebuilds. Installing a repository from source runs its build code (setup scripts, build backends) at the pull request's commit. That step gets an allow-listed environment, not the server's: paths and `HOME`, user and shell, locale and time zone, XDG cache and data directories, every `UV_*` and `PIP_*` variable (so a custom index such as `UV_INDEX_URL` still applies), proxy variables and CA certificate locations ([`installEnv`](../src/runner/sandbox.ts)). `FALKORDB_URL`, `ANTHROPIC_API_KEY`, `GITHUB_TOKEN` and anything else outside the list never reach a pull request's build code. The exact list (`INSTALL_ENV`): `PATH`, `HOME`, `TMPDIR`, `TEMP`, `TMP`, `USER`, `LOGNAME`, `SHELL`, `LANG`, `LC_*`, `TZ`, `XDG_CACHE_HOME`, `XDG_DATA_HOME`, `UV_*`, `PIP_*`, `HTTP_PROXY`, `HTTPS_PROXY`, `NO_PROXY`, `ALL_PROXY` (either case), `SSL_CERT_FILE`, `SSL_CERT_DIR`, `REQUESTS_CA_BUNDLE`, `CURL_CA_BUNDLE`, and on Windows `SYSTEMROOT`, `COMSPEC`, `PATHEXT`. `VIRTUAL_ENV` is not passed, so an activated virtualenv in your shell does not leak into the build. ## Scratch FalkorDB Docs examples connect to FalkorDB on `localhost:6379`. Inside the sandbox that address goes to a scratch FalkorDB, never to the server that holds your graph. - **Managed (default)**: the runner starts a local server on `127.0.0.1:6390` (`STALEPROOF_SCRATCH_PORT` changes the port) with the same falkordblite binaries as `staleproof db start`. Starting is idempotent. - **Your own**: set `STALEPROOF_SCRATCH_URL` to a separate server (for example the `scratch` service in the Docker Compose file). Only the host and port are used (default port 6379); the scheme and TLS are ignored, and the server must not require a password. A URL with credentials is refused when the configuration loads: `STALEPROOF_SCRATCH_URL must not carry credentials: docs examples connect to the scratch server without a password`. Before every run the scratch server is wiped with `FLUSHALL`, then `GRAPH.UDF FLUSH`. The second clears user-defined function libraries, which `FLUSHALL` keeps: without it, the docs' UDF example passes once and then fails with "already registered", which would read as a pull request breaking it. Servers without UDF support are fine. The runner refuses to flush the server that holds the org graph. If the scratch host and port match `FALKORDB_URL` (with `localhost` treated as `127.0.0.1`), the run fails with: ```text refusing to flush 127.0.0.1:6379: it is the server holding the org graph (FALKORDB_URL). Point STALEPROOF_SCRATCH_URL / STALEPROOF_SCRATCH_PORT at a separate server. ``` Every example starts from an empty server. An example that assumes data the page never creates fails, and that is a finding: the docs site's `GRAPH.EXPLAIN` example explains a query on a graph it never creates, and fails with `Invalid graph operation on empty key` before and after falkordb-py#267. ## The sandbox The program is written to `example.py` and runs as `<venv>/bin/python -s _run_example.py` (the entry point [python/run_example.py](../python/run_example.py), which runs `example.py`; see [Async examples](#async-examples)) in a fresh temporary directory (`staleproof-run-*` under the system temp directory, deleted afterwards), with empty standard input, in its own process group. Two processes touch code from the pull request, and neither gets the server's environment: | Step | Runs | Environment | |---|---|---| | Install | `uv venv`, `uv pip install` (or `python3 -m venv`, `pip install`), which run the repositories' build code | The allow-list described under [Virtualenvs](#virtualenvs): paths, user, locale, `UV_*` and `PIP_*`, proxies, certificates. Network allowed | | Example | `python -s _run_example.py`, which runs `example.py` | Nothing inherited: only the variables listed below. Network refused except the scratch server | - **Environment**: nothing is inherited, not even the install step's allow-list, so no secrets reach the example. The environment is exactly `PATH=/usr/bin:/bin`, `HOME` and `TMPDIR` set to the temporary directory, `LANG=C.UTF-8`, `PYTHONPATH` pointing at the shim, `PYTHONDONTWRITEBYTECODE=1`, `PYTHONUNBUFFERED=1`, and `STALEPROOF_SCRATCH_HOST` / `STALEPROOF_SCRATCH_PORT`. - **Network shim**: a `sitecustomize.py` ([python/sitecustomize.py](../python/sitecustomize.py)) loads automatically and patches Python's sockets. A connection to `localhost`, `127.0.0.1`, `::1`, `0.0.0.0`, `host.docker.internal` or `falkordb` on port 6379 is sent to the scratch server. A connection to the scratch server itself is allowed. Anything else fails with `OSError(ECONNREFUSED)`: `Staleproof sandbox: outbound network is disabled (tried <host>:<port>); only FalkorDB on localhost:6379 is available`. - **DNS**: local names resolve to `127.0.0.1`; any other lookup fails with `Staleproof sandbox: DNS is disabled (<host>)`. - **Timeout**: `STALEPROOF_RUN_TIMEOUT_MS` (default 30000). On timeout the whole process group is killed and the run is recorded as `timeout` (`no exit after 30s`). - **Output**: up to 64 KiB per stream is captured; the stored stdout and traceback keep the last 4000 characters. ## Statuses and outcomes Each run gets a status, and a pair of runs gets an outcome. Both are defined in [Core concepts](concepts.md#runs); this is how to read them. | Outcome | Summary line | What it tells you about the docs | |---|---|---| | `breaks` | `Passes at <before> and fails at <after> with <error>: the PR breaks this example.` | The pull request broke a documented example: the section needs an edit | | `fixes` | `Fails at <before> and passes at <after>: the PR fixes this example.` | The example works now; it did not before (#262) | | `passes` | `Passes before and after the PR (<before>, <after>): no observable break.` | Good evidence the section still holds | | `already-broken` | `Fails both before and after the PR with <error>: broken already, not by this PR.` | Not this pull request's doing, and still a broken example worth fixing (#267's `GRAPH.EXPLAIN`) | | `changed-failure` | `Fails before (<error>) and after with a different error (<error>).` | Broken both times; compare the two errors | | `fails` | `Fails at the indexed commits: <error>.` | A single run without a pull request failed | | `skipped` | `Not run: <reason>.` | Proves nothing | | `error` | `The sandbox could not be prepared: <reason>` | An environment problem (install, clone, scratch server); says nothing about the docs | Commits are shown as 7-character SHAs. On #276, 21 examples were tried at both commits: 10 pass both times, 6 fail both times (already broken), 3 cannot be installed together at those commits (`error`, see the next section), and 2 were skipped. Across every recorded run in the graph: 51 runs, 24 pass, 16 fail, 6 error, 5 skipped ([FACTS](FACTS.md#example-runs-sandbox)). ## Install conflicts When the repositories cannot be installed together, the run is `error` and its reason starts with one line naming the problem, followed by the last 2000 characters of the installer's output. If the resolver reports which pin blocks the install, the line names it: ```text dependency conflict: <package>==<version> requires <requirement>, which the other checkouts don't satisfy (installing <repo@sha, …>) ``` Otherwise it reads `installing <repo@sha, …> failed:`. On #276, three GraphRAG-SDK examples hit `dependency conflict: graphrag-sdk==1.4.0 requires falkordb[hiredis]>=1.8.0,<2, …`: the SDK at its indexed commit pins the client below 2, and the falkordb-py checkout at the pull request's commits does not satisfy that pin. That is version skew between repositories, not a docs finding. ## The Run node Each execution is stored as a `Run` node: | Property | Meaning | |---|---| | `id` | `run-<key>`; a failed preparation gets `run-<key>-err-<ms>`, unique per attempt | | `key` | The [run key](#run-key-and-caching) | | `example` | The CodeExample id | | `status` | `pass`, `fail`, `timeout`, `skipped` or `error` | | `reason` | Why it was skipped or could not be prepared | | `error` | The last line of the traceback (`AttributeError: …`), or of stderr | | `traceback` | The last `Traceback (most recent call last):` block, or the end of stderr | | `stdout` | The end of standard output | | `duration_ms` | Wall time of the program | | `stitched` | Ids of the earlier examples prepended | | `installed` | `repo@sha` for every installed repository | | `side` | `before`, `after` or `current` (no pull request) | | `created_at` | When it ran | Edges: `(Run)-[:OF]->(CodeExample)`, and `(Run)-[:AT]->(PullRequest)` for every pull request it was evidence for (a reused run collects more). Verdicts that cite a run link to it with `EVIDENCE`. The tool result (`RunRecord`) has the same fields in camel case, plus `cached`. ## One run at a time Runs are serialised by a process-wide queue: the scratch server is shared and flushed before each run, so two runs never overlap in one process. The HTTP server also runs one check at a time. Do not point two Staleproof processes at the same scratch server. ## Limits - **The sandbox is a guard, not a security boundary.** It refuses connections and DNS at the Python socket level, runs with no inherited secrets and a temporary `HOME`, and kills on timeout. Native code that opens its own sockets bypasses the shim, and the program can read the file system. Run it inside a container for untrusted code. - **Runs use today's third-party dependencies.** The organisation's repositories are installed at exact commits, but their dependencies resolve at install time, so an old run may not reproduce today. A recorded run is reused until you pass `--fresh`. - **Installs have network access** (they download dependencies); only the example itself runs without it. Their environment is the allow-list above, without the server's credentials, but with any credentials you put in `UV_*` or `PIP_*` variables (an index URL) or a proxy URL. - **Python is pinned to 3.12 only with uv.** The `python3 -m venv` fallback uses the host's Python. - **Re-indexing moves commits.** The run key includes the indexed commit of every installed repository, so after a re-index the earlier runs no longer match and examples run again. - **Stitching prepends every earlier runnable Python block on the page**, related or not; a failing earlier block fails the fragment. - **Serverless hosts do not execute examples.** On Vercel, run results come only from runs recorded by a host with a sandbox (a local machine, a container or the GitHub Action), and a hosted workspace has none. ## Related - [Core concepts](concepts.md#runs): run statuses and outcomes - [Indexing](indexing.md): how examples, `USES` edges and `DEPENDS_ON` edges are found - [Tools reference](tools.md): `run_example` parameters and result - [Deploy](DEPLOY.md): the scratch server in Docker Compose and on Vercel - [Troubleshooting and FAQ](troubleshooting.md) --- <!-- page: Decisions and memory · How it works · source: https://staleproof.dev/docs/memory.md --> # Decisions and memory This page covers what the graph remembers between checks and how people steer it: recording a maintainer decision, what a decision governs, superseding one, governed verdicts, accepting or rejecting verdicts, the "since your last check" summary, and exactly what every check writes back. It is for maintainers who want a check to stop flagging something on purpose, and for anyone reading a check's history. ## Record a decision A decision is one sentence from a maintainer, attached to the doc sections or symbols it governs. Later checks find it and can cite it instead of flagging the docs again. ```bash npx -y staleproof decide "The async client keeps positional kwargs until 2.0" \ --governs "FalkorDB/falkordb-py:falkordb.asyncio.falkordb.FalkorDB.__init__" \ --author "maintainer" --source "https://github.com/FalkorDB/falkordb-py/issues/<n>" ``` It prints `Recorded decision dec-<id> governing 1 node(s).` The same decision can be written three ways: | Interface | How | Notes | |---|---|---| | CLI | `staleproof decide <text> --governs <id...> [--source <url>] [--author <name>] [--supersedes <decision-id>]` | `--governs` is required and takes several ids | | MCP | `write_decision` with `text`, `governs`, and optional `source_url`, `author`, `supersedes` | Meant for a person driving an MCP client; the tool description says so | | HTTP | `POST /api/decisions` with the same JSON body as `write_decision` | Returns `{id, governed, summary}`; text is limited to 500 characters; rate-limited to 30 writes per hour per client | In the web UI, each impacted section has a "Record a maintainer decision for this section" form; it posts to `POST /api/decisions` with that section as `governs` and `author: 'web demo'`. Rules: - `text` needs at least 5 characters; `governs` at least one id. - `governs` takes [DocSection ids](concepts.md#ids) (`owner/repo:path#anchor`) and Symbol ids (`owner/repo:qualname`), mixed. Ids that match nothing are ignored; if none matches, nothing is recorded and the error is `none of <ids> is a DocSection or Symbol id; decision not recorded`. - The `Decision` node stores `text`, `source_url` (empty by default), `author` (`unknown` by default), `status: 'active'` and `created_at`. Its id is `dec-` plus 10 hex characters. - The agent cannot write decisions: `write_decision` is never offered to the model. Decisions come from people. ## What a decision governs A decision has a `GOVERNS` edge to every section and symbol it names, and reaches further through the call graph: - **For a doc section**, a decision applies when it governs the section itself, or governs a symbol within 2 calls below a symbol the section mentions in inline code, or below a symbol one of its examples calls. - **For a symbol**, a decision applies when it governs that symbol or anything it calls within 2 hops. So a decision on a function covers the docs of everything that calls it, up to two calls above it, in any repository. A decision on `FalkorDB.__init__` covers the README section whose example constructs `FalkorDB`; a GraphRAG-SDK section whose example called a helper that constructs the async `FalkorDB` would be covered too, one call further away. The reverse does not hold: a decision on a callee does not apply when you ask about a function it calls. ## Superseding a decision Decisions change. Record the new one with `--supersedes` (or `supersedes` in MCP and HTTP): ```bash npx -y staleproof decide "The async client drops positional kwargs in 2.0; document keyword arguments only" \ --governs "FalkorDB/falkordb-py:falkordb.asyncio.falkordb.FalkorDB.__init__" \ --supersedes dec-<old-id> ``` This adds `(new)-[:SUPERSEDES]->(old)` and sets the old decision's status to `superseded`. It prints `Recorded decision dec-<new-id> governing 1 node(s), superseding dec-<old-id>.` A `--supersedes` id that does not exist is ignored without an error. ## How decisions_for resolves decisions `decisions_for` takes a section id (anything containing `#`) or a symbol id and returns the active decisions that apply, as described above. Superseded decisions resolve to their successor: when a matched decision has been replaced, the tool follows `SUPERSEDES` edges back (up to 10 steps) to the newest active decision and returns that one, with `matched` set to the decision the graph actually reached. Only `active` decisions come back, one entry per decision, preferring a direct match over one reached through a superseded decision. Each entry has `id`, `text`, `source`, `author`, `createdAt`, `matched` and `via`: the section's heading path when the decision governs the section itself, otherwise the qualified name of the governed symbol. The summary reads: ```text 1 active decision governs this: "The async client keeps positional kwargs until 2.0" (dec-<id>, via falkordb.asyncio.falkordb.FalkorDB.__init__). ``` A superseded match shows as `(dec-<new>, supersedes dec-<old>, via …)`. With no match the summary is `No decision governs this.` ## Governed verdicts When a decision says a section stays as it is, the agent closes the section with a `governed` verdict that cites it. The system prompt says: "Check decisions_for before editing: a maintainer decision can mean the section is intentionally as it is. Cite it with a governed verdict." ```json { "section_id": "FalkorDB/falkordb-py:README.md#asynchronous-example", "verdict": "governed", "reason": "The README keeps the positional-kwargs form on purpose until 2.0.", "cites": ["dec-<id>"] } ``` A `governed` verdict without `cites` is refused: `a governed verdict must cite the decision id(s)`. Each cited id that exists gets a `CITES` edge from the verdict; an id that matches no decision is skipped without an error, so check the id you cite. ## Verdicts and maintainer review Every verdict starts as `proposed`. A maintainer can accept or reject it: ```bash npx -y staleproof verdict accept ver-<id> npx -y staleproof verdict reject ver-<id> --by alice ``` The CLI prints `ver-<id> accepted` (or `rejected`); `--by` defaults to `$USER`, else `maintainer`. Over HTTP: ```bash curl -X POST http://127.0.0.1:8080/api/verdicts/ver-<id>/review \ -H 'content-type: application/json' -d '{"status": "rejected", "by": "alice"}' ``` `status` must be `accepted` or `rejected`; `by` defaults to `web` and is cut to 60 characters. An unknown id gives `no verdict <id>` on the CLI and 404 `no such verdict` over HTTP. The review sets `status`, `reviewed_by` and `reviewed_at` on the verdict. The web UI has no accept or reject buttons yet; use the CLI or the API. To find verdict ids: `GET /api/patch?pr=<url>` lists the latest verdict per section with its `id`; `check --json` prints the check's events, including each `record_verdict` and `propose_edit` result; MCP results include them too. What a review changes: - **The docs patch** leaves out `stale` verdicts that were rejected. - **The next check sees it.** `decisions_for` on a section returns its 5 most recent verdicts, on any pull request, with their status. The summary shows the status when it is not `proposed`, in the form `Earlier: stale (rejected) on <owner>/<repo>/pull/<n>, ok on …`. The system prompt tells the model to use this history. - **Precision.** Reviews are the data for measuring how often verdicts are right. No command computes it yet; this query counts reviews per verdict kind: ```cypher MATCH (v:Verdict) WHERE v.status IN ['accepted', 'rejected'] RETURN v.verdict AS verdict, sum(CASE WHEN v.status = 'accepted' THEN 1 ELSE 0 END) AS accepted, count(v) AS reviewed ORDER BY verdict ``` Run it against the org graph (the graph name is the org slug, `falkordb` by default) with any FalkorDB client. ## Since your last check At the start of an agent check, the loop looks up the most recent finished check in the graph (any pull request, either mode) and summarises what was written since it started: - pull requests ingested for the first time (this can include the one being checked); - verdicts, and how many of them are `stale`; - new decisions, with their text and id; - example runs, and how many failed. The summary is appended to the model's first message as `Since the last check: …`, recorded on the `check_started` trace event, and printed by the CLI. Its format: ```text <n> pull request(s) checked (<owner/repo/pull/n>, …); <n> verdict(s), <n> of them stale; <n> new decision(s): "<text>" (<id>); <n> example run(s), <n> failed ``` Parts with nothing to report are left out. Graph mode does not compute it. ## What every check writes Nothing a check learns is kept only in memory. In order: | When | Written | Edges | |---|---|---| | The first time the pull request is seen | `PullRequest` (URL, title, author, state, commits compared, `docs_touched`, `ingested_at`); symbols it removed that the graph no longer has, recreated with `status: 'removed'` | `CHANGED_IN {kind, before, after, renamed_to}` from each changed symbol; `late` `MENTIONS` and `USES` to recreated symbols | | The check starts | `Check` with `status: 'running'`, the model name and `started_at` | `FOR` → PullRequest | | Each `run_example` | `Run` per side, unless an identical run is reused | `OF` → CodeExample, `AT` → PullRequest | | Each `propose_edit` or `record_verdict` | `Verdict` with `check` set to the check id | `ABOUT` → DocSection, `FOR` → PullRequest, `CITES` → Decision, `EVIDENCE` → Run | | The check ends | The `Check`'s status, summary, turns, tool calls, tool order, open sections, token counts and full trace | `PRODUCED` → each Verdict of this check | Graph mode writes the pull request, the check and its runs, but no verdicts. Tool calls over MCP write runs and verdicts (with an empty `check`) but no `Check` node, except `check_pull_request`, which stores a graph-mode check. Decisions are only written by people. ## Nothing is deleted on re-index Re-indexing never deletes `Symbol`, `DocSection` or `CodeExample` nodes. Those that disappeared from the repository are marked `status: 'gone'`, so verdicts keep their `ABOUT` edge, decisions keep their `GOVERNS` edges, and runs keep their `OF` edge; the history stays readable. Gone sections are no longer reached by an impact walk. Two consequences to know: - A section's id comes from its heading. Rename the heading (or move the file) and the section gets a new id; decisions and verdicts stay on the old, gone one. Re-record a decision on the new id if it still applies. - Re-indexing a docs repository deletes and rebuilds its `MENTIONS` and `USES` edges, including the `late` ones made for removed symbols, and those are not recreated. See [Indexing](indexing.md#re-indexing). ## Worked example falkordb-py#262 reaches one section, the README's "Asynchronous Example", through the path `Is_Cluster ← FalkorDB.__init__ ← FalkorDB/falkordb-py:README.md:4`: the example constructs the async `FalkorDB`. 1. **Record the decision**, governing the constructor: ```bash npx -y staleproof decide "The async client keeps positional kwargs until 2.0" \ --governs "FalkorDB/falkordb-py:falkordb.asyncio.falkordb.FalkorDB.__init__" ``` The graph now has `(:Decision {status: 'active'})-[:GOVERNS]->(:Symbol {qualname: 'falkordb.asyncio.falkordb.FalkorDB.__init__'})`. 2. **Check the pull request again** in agent mode: ```bash npx -y staleproof check https://github.com/FalkorDB/falkordb-py/pull/262 ``` The model's first message includes `Since the last check: … 1 new decision(s): "The async client keeps positional kwargs until 2.0" (dec-<id>) …`. 3. **The decision is found through the example.** When the model calls `decisions_for` on `FalkorDB/falkordb-py:README.md#asynchronous-example`, the section's example `USES` `FalkorDB.__init__` directly (0 call hops), so the result is the decision, `via falkordb.asyncio.falkordb.FalkorDB.__init__`, followed by the section's earlier verdicts. 4. **The verdict cites it.** If the model concludes the section stays as written because of the decision, it records `governed` with `cites: ["dec-<id>"]`. The verdict gets a `CITES` edge, the PR comment's verdicts table shows `governed`, and the docs patch lists the section under "sections checked and left unchanged" with the reason. Whether the model uses the decision is its judgement: on #262 the example passes after the pull request, so an `ok` verdict is also correct. Graph mode never reads decisions. ## Limits - No command computes verdict precision yet; reviews are stored, and the query above counts them. - The web UI records decisions but cannot accept or reject verdicts. - Decisions and verdicts are tied to section ids, which change when a heading is renamed. - Unknown ids in `cites` and `--supersedes` are ignored without an error. - A check that dies mid-way (process killed) leaves its `Check` at `running`; "since your last check" ignores it. ## Related - [Core concepts](concepts.md#decisions): decisions, verdicts and memory in one paragraph each - [The agent](agent.md): when the model calls `decisions_for` and records verdicts - [CLI](cli.md): `decide` and `verdict` - [HTTP API](http-api.md): `POST /api/decisions` and `POST /api/verdicts/:id/review` - [Graph model and Cypher](GRAPH-MODEL.md#memory): every memory query --- <!-- page: The agent · How it works · source: https://staleproof.dev/docs/agent.md --> # The agent This page explains how a check runs: the agent loop in [src/agent/loop.ts](../src/agent/loop.ts), its system prompt, the tools it offers the model, input validation, the stop condition, step and time limits, what is stored, the model providers it supports (Claude by default, or any Anthropic-compatible API such as DeepSeek), graph mode, the scripted model used in tests, and real traces from FalkorDB's pull requests. Read it to understand or tune a check, or to run the agent on a different model. ## Run an agent check ```bash export ANTHROPIC_API_KEY=<your Claude API key> npx -y staleproof check https://github.com/FalkorDB/falkordb-py/pull/262 ``` The CLI streams the trace as it happens: the check id and model, the "since the last check" summary, the model's reasoning (dimmed), each tool call (`→ impact(pr_url=…)`) and its result (`← <summary>` with the Cypher queries it ran and the time it took), any reminder, and a final line with the status, turns, tool calls, tool order, verdicts and tokens, followed by the model's summary. It ends with `Patch: staleproof patch <pr> Trace: stored on <check-id>`. The exit code is 1 when the check ends `error` or `refused`. | Option | Default | Meaning | |---|---|---| | `--model <id>` | `STALEPROOF_MODEL`, else `claude-sonnet-5-5` | The model | | `--effort <level>` | `STALEPROOF_EFFORT`, else `medium` | `low`, `medium`, `high`, `xhigh` or `max` | | `--max-steps <n>` | `40` | Model turns before the loop stops | | `--json` | off | Print the whole result (events included) as JSON instead of the trace | | `--format markdown` | `text` | Also render the PR comment, with the agent's summary | | `--with <repos...>` | none | Repositories to index together with the pull request's when the graph lacks any of them | | `--no-index` | index on first use | Fail instead of indexing a repository the graph does not have | | `--out <file>` | none | Write the PR comment to a file | | `--no-agent` | off | Run [graph mode](#graph-mode) instead | To run on DeepSeek instead of Claude, see [The live demo's DeepSeek setup](#the-live-demos-deepseek-setup). The same loop runs behind `POST /api/check` in the [HTTP API](http-api.md) and the web UI. ## The loop The model chooses every tool call. The loop validates and executes the calls, streams trace events, enforces the stop condition and stores everything on a `Check` node. ### Setup 1. Ingest the pull request if it was never ingested ([Pull request ingestion](concepts.md#pull-request-ingestion)). 2. Find the most recent finished check in the graph and summarise what was written since it started (see [Since your last check](memory.md#since-your-last-check)). 3. Create the `Check` node: `status: 'running'`, the model name, `started_at`, linked `FOR` the pull request. The id is `chk-` plus 12 hex characters. 4. Send the system prompt, the tool definitions and one user message: `Check <pr-url>.`, followed by `Since the last check: …` when there is something to report. ### Each turn A step is one model turn, numbered from 1. - **Time budget**: if the check has a budget and it is used up, no new turn starts (see [Time budget](#time-budget)). - **The turn streams**: text and reasoning arrive as `delta` events (shown live, not stored); the complete message becomes `thinking` and `text` events. - **Retries**: an error that is not an API error, such as an unparseable streamed tool input, re-issues the turn, up to 2 times in a row; each retry uses a step. An API error ends the check as `error`. - **Stop reasons**: `refusal` ends the check as `refused` with the summary `The model declined to continue (<category>).`; `pause_turn` continues with the next turn. A tool call cut off by `max_tokens` ends the check as `error`: `a tool call was cut off at max_tokens; raise max_tokens and re-run the check`. - **History** is append-only: the assistant message goes back to the model exactly as it came. - **Tool calls** in one message run one after another, in the order given. ### Tools offered The model gets the eight agent tools, with their JSON schemas: `impact`, `clearance`, `read_section`, `run_example`, `decisions_for`, `rank_sections`, `propose_edit` and `record_verdict`. `write_decision` is never offered: decisions come from people. `docs_patch` is for MCP clients. Each tool and its result are described in the [Tools reference](tools.md). For `impact`, `clearance`, `run_example`, `propose_edit` and `record_verdict`, a missing `pr_url` defaults to the pull request under check. ### Input validation Every tool call is validated before it runs, and every problem goes back to the model as an error result rather than ending the check: | Problem | Error result | |---|---| | Unknown tool | `unknown tool <name>` | | A parameter the tool does not have | `<tool> has no parameter <names>; its parameters are <list>`, for example `impact has no parameter limit; its parameters are pr_url, max_hops, offset` | | Invalid input | `invalid input for <tool>: <path>: <message>; …` | | The tool throws | Its message, for example `refused: no path in the graph from <pr>'s changes to <section>. …` | Unknown parameters are an error, not ignored: if they were dropped silently, the model would get the same answer again and keep trying (see [the first #276 run](#the-first-276-run)). ### Tool results A successful result goes back to the model as JSON with the tool's `summary` and `data`, cut at 24,000 characters (`MAX_TOOL_CONTENT`) with `… [truncated; ask for fewer items]` appended. The trace keeps more: the full data (unless its JSON exceeds 60,000 characters, when it is stored as a preview marked `__trimmed`), plus every Cypher query the tool ran with its server-side time and row count. ### Stop condition The check is meant to end with a verdict or an edit for every impacted section. - **Impacted sections**: after any successful `impact` call, the loop collects the section ids of every page of that walk itself (fetching the pages the model has not asked for, without showing them to the model), so the set covers all impacted sections, not only the page the model saw. For #276 that is all 43. The set is the union over every `impact` call of the check, each with the `max_hops` it was called with. Example scripts, which have no section, are not included. - **When the model finishes** (a turn with no tool calls), the loop compares the set with the sections that got a verdict from this check. Its last text becomes the summary. - **Reminders**: if sections are still open, the loop sends up to 2 reminders: `These impacted sections still have no verdict or edit: <ids>. Give each one a verdict (record_verdict) or an edit (propose_edit), then finish.` - **Result**: `done` when nothing is open; `incomplete`, with the open sections listed in `missing`, when something still is after the reminders. A check in which the model never calls `impact` has no impacted sections to cover. ### Step limit The loop stops after `maxSteps` model turns: 40 by default, `--max-steps` on the CLI. The HTTP server always uses 40. A check that hits the limit ends `incomplete` with the summary `Stopped after 40 model turns with <n> impacted section(s) still open.`, unless the model had already written a summary. ### Time budget A serverless function has a hard time limit, so the HTTP server can give each agent check a budget: `STALEPROOF_CHECK_BUDGET_MS`, which defaults to 240,000 (240 s) when `VERCEL` is set and is unset elsewhere. The budget is checked before each turn; a turn in progress finishes. Past it, the check ends `incomplete`: ```text Stopped at this server's 240 s time limit after <n> model turns with <m> impacted section(s) still open; the CLI has no time limit. ``` The CLI never sets a budget. A 43-section check such as #276 takes about 6 minutes, so on Vercel it ends incomplete; run it from the CLI. ### Statuses | Status | When | |---|---| | `done` | The model finished and every impacted section has a verdict or an edit | | `incomplete` | Sections still open after the reminders, the step limit or the time budget | | `refused` | The model returned a refusal | | `error` | An API error, a tool call cut off at `max_tokens`, a missing credential, or any other failure: `The check failed: <message>` | While it runs, the `Check` node has status `running`. A process killed mid-check leaves it there. ### What the Check node stores | Property | Meaning | |---|---| | `pr`, `model`, `started_at`, `finished_at` | What was checked, by which model, when | | `status`, `summary` | The outcome and the model's final summary (or the loop's) | | `steps` | Model turns used | | `tool_calls`, `tool_order` | How many tool calls, and their names in order | | `missing` | Impacted sections still without a verdict or edit | | `input_tokens`, `output_tokens` | Token usage, summed over turns | | `verdicts` | Verdicts written by this check | | `trace` | Every stored event as JSON | Edges: `(Check)-[:FOR]->(PullRequest)` and `(Check)-[:PRODUCED]->(Verdict)` for every verdict whose `check` is this id. The stored trace has these event types: `check_started` (with the "since" text), `thinking`, `text`, `tool_call` (name, input, step), `tool_result` (summary, `isError`, queries, data, milliseconds, step), `nudge` (the open sections) and `check_finished` (status, summary, steps, tool calls, tool order, verdicts, missing, usage). Read a stored trace with `GET /api/checks/<check-id>`, or open `#/check/<check-id>` in the web UI. ### Usage accounting Each turn's `input_tokens`, `output_tokens` and `cache_read_input_tokens` are summed. Input and output totals are stored on the `Check`; the cache-read total appears only in the trace's `check_finished` event. The CLI's final line shows input plus output tokens. Graph mode uses no tokens. ## The system prompt The system prompt ([src/agent/prompt.ts](../src/agent/prompt.ts)) states the job and the rules the graph enforces. It does not fix a tool order: the model decides what to look at next from what the tools return, so different pull requests produce different traces. Its rules, in short: - A path is a reason to look, not proof: "A path from a changed symbol to the section is the reason to look at it, not proof that it is wrong." - Breaking changes usually make a section wrong; behaviour changes often do not. - An example `impact` lists as predicted to break (its call no longer fits the changed code) is strong evidence the section is wrong: run it to prove it when possible, and fix the call to the new signature. Its "already did not fit" list is drift that predates the pull request. - Running a Python example before and after is the strongest evidence: passes before and fails after proves a break, passes both is good evidence the docs hold, fails both means it was already broken, and skipped runs prove nothing. - Check `decisions_for` before editing, and cite a governing decision with a `governed` verdict. Use earlier verdicts, and whether maintainers accepted them. - Never propose an edit without a path from the pull request's changes; `propose_edit` refuses otherwise. - What to edit: "Edit what is wrong: what the pull request falsified, or an example you ran that fails both before and after (already broken: say so in the reason, cite both runs, and fix only what makes it fail)." Keep the author's voice and everything still true, and give the whole section, heading included, as `new_text`. - Every impacted section must end with a verdict or an edit; one reason may cover many sections. - Do not invent APIs. - Finish with a short plain-text summary for a maintainer: what changed, what was edited and why, what ran and what it proved, and what could not be checked. The path guard in `propose_edit` is the same for an already-broken example: the section must still be reachable from the pull request's changes. ## Model providers The agent was written for Claude and runs on any API that speaks Anthropic's Messages format. Both go through `AnthropicModel` ([src/agent/anthropic.ts](../src/agent/anthropic.ts)). | Option | Default | Meaning | |---|---|---| | `model` | required | The model name (`STALEPROOF_MODEL` or `--model` on the CLI) | | `effort` | `medium` | `low`, `medium`, `high`, `xhigh` or `max` | | `fallbacks` | `true` | Server-side refusal fallback (Claude API only); `STALEPROOF_FALLBACKS=off` disables it | | `apiKey` | none | Otherwise the SDK finds credentials itself | | `maxTokens` | `32000` | Output limit per turn; set only through the library | | `baseURL` | none | The API endpoint (`ANTHROPIC_BASE_URL`) | | `compatible` | `false` | Send only the portable request fields; set when `ANTHROPIC_BASE_URL` names a host other than `api.anthropic.com` | ### Claude Claude is the default (`claude-sonnet-5-5`). Each turn is a streamed request with: - adaptive thinking with summarised display, so the trace shows why each tool was chosen; - the effort level set explicitly; - the system prompt marked for prompt caching; - tool inputs streamed as they are generated (the loop validates every input before running it); - the server-side refusal fallback, unless `STALEPROOF_FALLBACKS=off`. The Claude path is tested against a local fake API only: [test/anthropic-model.test.ts](../test/anthropic-model.test.ts) checks the request it sends and that it reads a streamed answer, without paid calls. The agent runs with a real model have used DeepSeek. ### Anthropic-compatible providers Set `ANTHROPIC_BASE_URL` to a provider's Anthropic-compatible endpoint. Any host other than `api.anthropic.com` (or a URL that does not parse) switches to compatible mode, which sends only the portable fields: model, max tokens, the system prompt as plain text, the tools, the messages, thinking enabled with a budget of 16,000 tokens, and the effort level. No prompt caching, beta headers, refusal fallback or streamed tool inputs. A Claude model name is refused in compatible mode, because the provider would answer with its own model under that name and every trace would record the wrong model: ```text "claude-sonnet-5-5" is a Claude model name, but the model endpoint is an Anthropic-compatible provider, which would answer with its own model under that name; set STALEPROOF_MODEL to the provider's model (e.g. deepseek-v4-pro) ``` The default model is `claude-sonnet-5-5`, so setting `ANTHROPIC_BASE_URL` without `STALEPROOF_MODEL` hits this error. ### The live demo's DeepSeek setup The live demo runs `deepseek-v4-pro` through DeepSeek's Anthropic-compatible endpoint: ```bash export ANTHROPIC_BASE_URL=https://api.deepseek.com/anthropic export ANTHROPIC_API_KEY=<your DeepSeek API key> export STALEPROOF_MODEL=deepseek-v4-pro npx -y staleproof check https://github.com/FalkorDB/falkordb-py/pull/267 ``` For a server, set the same three variables in its environment (see [Deploy](DEPLOY.md)). `GET /api/health` then reports the agent as available, with the model, the effort and the provider's host (`api.deepseek.com`). ### Credentials The agent needs credentials: `ANTHROPIC_API_KEY`, `ANTHROPIC_AUTH_TOKEN`, a selected `ANTHROPIC_PROFILE`, a federation rule with an identity token, or an `ant auth login` profile on disk. Without any, agent mode fails before the check starts, instead of falling back to graph mode: ```text the agent needs model API credentials: set ANTHROPIC_API_KEY (or run `ant auth login`) for the Claude API, or ANTHROPIC_BASE_URL, ANTHROPIC_API_KEY and STALEPROOF_MODEL for an Anthropic-compatible provider such as DeepSeek. Use `check --no-agent` to run the graph tools without a model. ``` An authentication failure during a turn is reported the same way, with the provider's message appended. The HTTP server without credentials offers graph mode only: `POST /api/check` with `mode: 'agent'` answers 409. ## Graph mode Graph mode runs the same tools in a fixed order with no model. It needs no credentials, and it is labelled as graph mode everywhere it is shown (the `Check`'s model is `graph-only (no model)`). ```bash npx -y staleproof check https://github.com/FalkorDB/falkordb-py/pull/267 --no-agent ``` The order: 1. `impact`, with `max_hops` when it is not 3. 2. `clearance` (always 3 hops). 3. `rank_sections` over the impacted section ids of the first page, if there are any. 4. `run_example` with the pull request, for up to `maxRuns` distinct examples: first those `impact` predicts to break (with an `exact` match), so a run can prove the prediction, then examples from the `exact` sections of the first page. Graph mode samples: it ranks the first `impact` page (40 sections) and runs at most 3 examples by default, so on a pull request with more sections, the rest are listed in the impact count but not ranked or run. It writes no verdicts and no edits, does not read decisions, sends no reminders and computes no "since" summary. A tool error is recorded in the trace and the remaining steps continue; only a failing `impact` ends the check as `error` (`The check failed: impact failed`). On #262 it makes 4 calls: impact, clearance, rank_sections and one run_example. | Setting | CLI | HTTP server | MCP `check_pull_request` | |---|---|---|---| | Example runs (`maxRuns`) | `--runs <n>`, default 3; `--runs 0` runs none | 3; none in a hosted workspace | 0; 3 with `run_examples: true` | | Call hops (`maxHops`) | `--max-hops <n>`, default 3 | Always 3 | Always 3 | The summary is the impact and clearance summaries, then `Ran <n> example(s) before and after the PR: <id>: <outcome>; …` (or `Example runs not requested.` or `No exactly-linked Python example to run.`), then `Graph mode: fixed tool order and no verdicts; agent mode (a model) judges each section and proposes the edits.` The CLI prints the impacted sections, the cleared symbols, each run and the summary, and stores the check like an agent check. `check --no-agent --format markdown`, which the GitHub Action runs without a model key, runs this same graph-mode check and then renders the report from the graph: the examples run before and after, impact, clearance, ranking and any verdicts already recorded. The MCP tool `check_pull_request` does the same and returns the report. ## ScriptedModel for tests `ScriptedModel` is a deterministic stand-in for a model, exported by the library. You give it a function from the conversation so far and the turn number to the turn's content: `thinking`, tool `calls`, or final `text`. A turn with calls ends with `tool_use`, one without with `end_turn`; usage is zero. It records every request it receives in `requests`, so a test can check what the model was shown. ```ts import { GraphStore, loadConfig, runAgentCheck, ScriptedModel } from 'staleproof'; const config = loadConfig(); const store = await GraphStore.connect(config.falkordbUrl); // The check is stored like any other: use a graph where the pull request's repository is indexed. const graph = store.org(config.org); // Turn 0 calls impact; every later turn finishes without a verdict. const model = new ScriptedModel((history, turn) => turn === 0 ? { calls: [{ name: 'impact', input: {} }] } : { text: 'Done.' }, ); const result = await runAgentCheck({ graph, config }, model, 'https://github.com/FalkorDB/falkordb-py/pull/262', { maxSteps: 5 }); console.log(result.status, result.missing); // incomplete, after two reminders, with the open section listed await store.close(); ``` The test suite drives the loop this way against an indexed fixture organisation ([test/agent.test.ts](../test/agent.test.ts)): a removed method that a script reads and edits, a signature change the script forgets to close (the loop must remind it), tool errors, the time budget, unknown parameters, and the stop condition across impact pages. ## Traces from FalkorDB's pull requests These checks ran `deepseek-v4-pro` on FalkorDB's graph. Each pull request got a different tool order. **falkordb-py#262**: 5 tool calls, 1 verdict. | Step | Tool | Result summary | |---|---|---| | 1 | `impact` | `1 changed symbols reach 1 doc sections (0 breaking, 0 signature, 1 behavior; 0 in other repositories).` | | 2 | `read_section` | `falkordb-py > Usage > Asynchronous Example (README.md, lines 64-114, 1 examples).` | | 3 | `decisions_for` | `No decision governs this.`, followed by the section's earlier verdicts | | 4 | `run_example` | `Fails at 49aa012 and passes at f7c3de7: the PR fixes this example. (recorded earlier; not re-run)` | | 5 | `record_verdict` | `Verdict ok recorded for 1 section.` | The impact summary is shown as recorded then; the current wording is `1 changed symbol reaches 1 doc section (…)`. The model concluded that the pull request fixes the documented example rather than breaking it, so the section needs no edit. **falkordb-py#267**: 15 tool calls and 3 verdicts, each citing its before/after runs. A later check under the current prompt (`chk-90817863f077`, 12 tool calls: impact, read_section and decisions_for for each of the 3 sections, 3 run_example calls, then 2 record_verdict calls) again closed all 3 sections as `ok`. It considered the `GRAPH.EXPLAIN` example, which fails before and after, and declined to edit it: the page assumes a demo graph that already exists, so the model judged that fixing it was outside this pull request's scope and recommended a separate docs fix in its summary. **falkordb-py#276**: 38 tool calls paging through all 43 impacted sections with `offset`, every one with a verdict. No check on FalkorDB's graph has proposed an edit: every verdict on #262, #267 and #276 is `ok`. These pull requests are fixes, and no example they reach broke because of them. Proposing edits and building the docs patch are exercised on the test suite's fixture organisation. ### The first #276 run The first agent check of #276 stopped at the 40-turn limit with 21 sections open. At the time: - `impact` returned at most 40 sections and had no way to page, while #276 reaches 43. - The model tried parameters the tool did not have, such as `limit`, `max_results` and an `offset` that did not exist yet. They were dropped silently, so it got the same first page back each time. - The stop condition counted only the sections the model had been shown, so the 3 sections past the first page were never required. What changed: `impact` pages (40 sections per call, `offset`, and a summary that says `call impact again with offset=40 for the rest`); unknown parameters are an error in the agent loop and over MCP; and the loop collects every impact page itself, so `done` means all impacted sections have a verdict or an edit. The next check of #276 paged through all 43 sections in 38 calls. ## Limits - **The Claude path is tested against a local fake API only.** Real agent runs so far used `deepseek-v4-pro`. - **Model choices vary between runs**: the same pull request can get a different tool order and different verdicts. - **The time budget applies to the HTTP server only**; on Vercel a large check ends incomplete at 240 s. - **`max_tokens` is set only through the library**, though the cut-off error says to raise it. - **`--effort` on the CLI is passed through unchecked**; only `STALEPROOF_EFFORT` is validated. - **The HTTP server has no step, run or hop options**; it uses the defaults. ## Related - [Core concepts](concepts.md#agent-mode-and-graph-mode): agent mode and graph mode side by side - [Tools reference](tools.md): what each tool takes and returns - [Decisions and memory](memory.md): what checks read and write back - [Configuration](configuration.md): the model and budget environment variables - [Deploy](DEPLOY.md): running the agent on a server --- <!-- page: Tools reference · Reference · source: https://staleproof.dev/docs/tools.md --> # Tools reference This page is the reference for the fourteen tools that read and write the graph: the eight the agent chooses between, `write_decision` for people, and `check_pull_request`, `check_changes`, `docs_for_symbol`, `audit_docs` and `docs_patch` for MCP clients. It covers how every tool reports its work, who can call which tool, the ids they take, the usual order of calls for one pull request, and how errors come back; then each tool's parameters (generated from the code) and the shape of its result, its summary text, its side effects and the errors it raises. It is for MCP client authors, library users and anyone reading a trace. ## How tools work Every tool returns the same envelope: ```ts interface ToolResult<T> { tool: string; // the tool's name summary: string; // one or two sentences, read first by the agent and shown in the UI data: T; // the structured result (shapes under Results below) queries: QueryTrace[]; // every Cypher statement the tool ran } interface QueryTrace { name: string; // the named query from src/graph/queries.ts, or 'adhoc' cypher: string; // the statement as sent params: Record<string, unknown>; ms: number; // server-side execution time reported by FalkorDB rows: number; // rows returned } ``` Named queries are listed with their full text in [Graph model and Cypher](GRAPH-MODEL.md). Small lookups written inline in the code are traced as `adhoc`; the MCP output and the terminal trace leave those out. How the result reaches you depends on the caller: - **The agent** receives `{summary, data}` as JSON, cut at 24,000 characters. The trace stored on the check keeps the queries too. See [The agent](agent.md#tool-results). - **MCP clients** receive three text items: the summary, the data as indented JSON, and `Cypher run on FalkorDB:` followed by each named query with its row count and time (left out when there are none). `check_pull_request` returns two instead: the Markdown report, and `Stored as check <id>. <summary>`. - **The CLI and the HTTP API** call the tools inside commands and endpoints and print or return their data. ### Who can call which tool | Tool | Agent | Graph mode | MCP | CLI | HTTP API | |---|---|---|---|---|---| | `check_pull_request` | no | it runs graph mode | yes | `check --no-agent --format markdown` (the same report) | `POST /api/mcp` | | `check_changes` | no | no | yes (`dir` on the stdio server only) | `check` with no pull request (the same analysis of a working tree) | `POST /api/mcp` (with `files`) | | `docs_for_symbol` | no | no | yes | `docs-for` | `GET /api/docs-for` | | `audit_docs` | no | no | yes | `audit` (which can also run examples first) | `GET /api/audit`, `POST /api/audit` | | `impact` | yes | yes | yes | `check`, `run --pr` | inside `POST /api/check` | | `clearance` | yes | yes | yes | `check` | inside `POST /api/check` | | `read_section` | yes | no | yes | no | `GET /api/sections?id=` | | `run_example` | yes | yes (up to 3) | yes, except on a hosted server | `run` | inside `POST /api/check` | | `decisions_for` | yes | no | yes | no | no | | `rank_sections` | yes | yes | yes | `check` | inside `POST /api/check` | | `propose_edit` | yes | no | yes | no | no | | `record_verdict` | yes | no | yes | no | no | | `write_decision` | never | no | yes | `decide` | `POST /api/decisions` | | `docs_patch` | no | no | yes | `patch` (same patch) | `GET /api/patch` (same patch) | The "Available to" line in each generated entry below lists the agent tools as available to graph mode "where it applies"; graph mode uses only the four marked yes here. Over the HTTP API every MCP tool is also reachable through `POST /api/mcp`. See [MCP server](mcp.md), [CLI](cli.md) and [HTTP API](http-api.md). ## Ids | Node | Format | Example | |---|---|---| | Symbol | `owner/repo:qualname` | `FalkorDB/falkordb-py:falkordb.asyncio.falkordb.FalkorDB.__init__` | | DocSection | `owner/repo:path#anchor` | `FalkorDB/falkordb-py:README.md#asynchronous-example` | | CodeExample | `owner/repo:path:index` | `FalkorDB/falkordb-py:README.md:4` | | PullRequest | URL | `https://github.com/FalkorDB/falkordb-py/pull/262` (also accepted: `owner/repo#n`, and URLs ending in `/files` or with a query) | | Decision, Verdict, Run | `dec-…`, `ver-…`, `run-…` | returned by the tools that create them | Section ids come from `impact` and `read_section`; example ids from `impact` (`examples` on each section) and `read_section`. A DocSection id for the text before a page's first heading ends in `#`. More in [Core concepts](concepts.md#ids). ## Typical call order for one pull request An MCP client usually starts with `check_pull_request`, which runs `impact`, `clearance` and `rank_sections` in one call (a graph-mode check) and returns the report; the steps below are for judging sections one by one. The agent decides its own order; this is the order the system prompt and the MCP server's instructions describe, and what a person driving an MCP client usually does: 1. `impact` with the pull request URL. If the summary says `call impact again with offset=40 for the rest`, page with `offset`. The first call ingests the pull request. 2. `read_section` for each impacted section, to see what it claims. 3. `decisions_for` on the section before editing it: a maintainer decision may say it stays as it is, and the history shows earlier verdicts. 4. `run_example` with the pull request URL for sections that have Python examples: the before/after outcome is the strongest evidence. 5. `propose_edit` for a section that is wrong, or `record_verdict` (`ok`, `governed`, `skipped`) for one that is not; several sections can share one `record_verdict`. 6. `docs_patch` (MCP) or `staleproof patch` for the diff. `clearance` (what no doc reaches) and `rank_sections` (which page matters most) are optional. For falkordb-py#262 the agent's order was impact → read_section → decisions_for → run_example → record_verdict. Before a pull request exists, an MCP client calls `docs_for_symbol` with the function it is about to change (to know which docs to update in the same change), then `check_changes` with the files it edited (to see which examples its edits break). With no change at all, `audit_docs` reports how the docs stand. These three write nothing to the graph. ## Errors A tool that cannot do its job throws an error with a message written for whoever called it. Nothing crashes the caller: - **In the agent loop**, the error goes back to the model as an error result for that tool call, the trace records it with `isError: true`, and the check continues. - **Over MCP**, the call returns `isError: true` with the message as its only text item. - **On the CLI**, the command prints `staleproof: <message>` and exits with code 1. - **Over HTTP**, endpoints answer 400, 404 or 500 with `{error}`; see [HTTP API](http-api.md). Input problems are caught before a tool runs, with the same rules in the agent loop and over MCP: | Problem | Agent loop | MCP | |---|---|---| | Unknown tool | `unknown tool <name>` | `unknown tool <name>` | | Parameter the tool does not have | `<tool> has no parameter <names>; its parameters are <list>` | same | | Invalid value | `invalid input for <tool>: <path>: <message>; …` | `invalid input: <path>: <message>; …` | `pr_url` on `propose_edit` and `record_verdict` defaults to the pull request under check in agent mode. Over MCP there is no pull request under check, so it is required (`pr_url is required`), and verdicts written over MCP are not attached to any check. ## Tools <!-- generated:tools:start --> <!-- Generated by `npm run docs` from src/tools and src/mcp/server.ts. Do not edit by hand. --> ### `check_pull_request` Check a GitHub pull request for docs it can make false, in one call: every doc section and example the changed code reaches (with the evidence path), the changed symbols no doc reaches, ranked by PageRank, as a Markdown report. Start here; use impact/read_section/decisions_for to dig into one section. Available to MCP clients. | Parameter | Type | Required | Description | |---|---|---|---| | `pr_url` | string | yes | https://github.com/<owner>/<repo>/pull/<n> | | `run_examples` | boolean | no | Also run up to 3 affected examples before and after the PR (local servers only; takes minutes) | Cypher it runs: `impact`, `clearance`, `rankSections` ([full text](GRAPH-MODEL.md)). ### `check_changes` Check edits that are not a pull request yet: which doc sections and examples they can make false (with the evidence path), which examples are predicted to break (their calls no longer fit the changed code), and which changed symbols no doc reaches. Send the new contents of the changed files (content null deletes a file); they are compared with the commit the graph indexed. Nothing is written to the graph. On a local server, pass dir (a git checkout) instead of files to check its working tree. Available to MCP clients. | Parameter | Type | Required | Description | |---|---|---|---| | `repo` | string | no | owner/name of the repository the files belong to (needed with files) | | `files` | object[] | no | Changed files: repository-relative path and full new content (null when deleted) | | `dir` | string | no | Local servers only: a git checkout whose working tree to check (default: the server's directory) | | `base` | string | no | With dir: the commit or ref to compare against (default: merge base with the remote default branch) | Cypher it runs: `impactOfChanges`, `predictedCallsOfChanges`, `clearanceOfChanges`, `mismatchesFor` ([full text](GRAPH-MODEL.md)). ### `docs_for_symbol` Which doc sections and code examples depend on a symbol, in any repository, with the evidence path for each: symbol ← up to 3 callers ← doc. Call it before changing a function, class or method to know which docs to update in the same change. The symbol can be an id (owner/repo:pkg.mod.Class.method), a qualified name, a dotted suffix like Graph.query, or a bare name. Available to MCP clients. | Parameter | Type | Required | Description | |---|---|---|---| | `symbol` | string (non-empty) | yes | owner/repo:pkg.mod.Class.method, pkg.mod.Class.method, Class.method or method | | `repos` | string[] | no | Only symbols defined in these repositories (owner/name) | Cypher it runs: `docsForSymbol`, `findSymbols`, `mismatchesFor` ([full text](GRAPH-MODEL.md)). ### `audit_docs` Audit the docs of indexed repositories with no pull request: a 0-100 health score, every example line that no longer fits the code it calls (unknown keywords, wrong argument counts, names that no longer exist, found statically across repositories), API coverage (public symbols no doc names or calls, most used first) and the latest example runs at the indexed commits. Available to MCP clients. | Parameter | Type | Required | Description | |---|---|---|---| | `repos` | string[] | no | Repositories to audit as owner/name (default: every repository in the graph) | | `limit` | integer 0–200 | no | How many undocumented symbols to list (default 25) | Cypher it runs: `auditRepos`, `auditExamples`, `auditMismatches`, `auditRuns`, `auditSymbols` ([full text](GRAPH-MODEL.md)). ### `impact` Find every doc section and code example a pull request can falsify, in any repository of the organisation. Walks changed symbol ← callers (up to max_hops, across repositories) ← MENTIONS/USES ← doc. Returns impacted sections ordered by severity (breaking: removed/renamed; api: signature; behavior: body changed) with the evidence path for each. Call this first for a PR. At most 40 sections come back per call; when more are impacted, call again with offset to page. Available to the agent, MCP clients and graph mode. | Parameter | Type | Required | Description | |---|---|---|---| | `pr_url` | string | yes | GitHub pull request URL, e.g. https://github.com/FalkorDB/falkordb-py/pull/262 | | `max_hops` | integer 0–5 | no | Maximum call hops between the changed symbol and the doc (default 3) | | `offset` | integer ≥ 0 | no | Skip this many sections (paging: 40 per call; default 0) | Cypher it runs: `impact`, `prChanges`, `predictedCalls`, `mismatchesFor`, `upsertPullRequest`, `upsertChangedIn`, `upsertMissingSymbols`, `lateLinkSections`, `lateLinkExamples` ([full text](GRAPH-MODEL.md)). ### `clearance` Proof of absence: the changed symbols that reach no doc section or example within 3 call hops in any repository. Each comes with a git grep command and its raw text-match count so a human can check the claim. Available to the agent, MCP clients and graph mode. | Parameter | Type | Required | Description | |---|---|---|---| | `pr_url` | string | yes | GitHub pull request URL (`https://github.com/<owner>/<repo>/pull/<n>`) | Cypher it runs: `clearance` ([full text](GRAPH-MODEL.md)). ### `read_section` Read the current Markdown of a doc section and its code examples, before deciding whether and how to edit it. Available to the agent and MCP clients. | Parameter | Type | Required | Description | |---|---|---|---| | `section_id` | string | yes | DocSection id (`owner/repo:path#anchor`), as returned by impact | Cypher it runs: `readSection` ([full text](GRAPH-MODEL.md)). ### `run_example` Execute a docs code example in a sandbox (fresh virtualenv with the repositories installed from source, scratch FalkorDB on localhost:6379, no other network). With pr_url it runs twice — at the commit before the PR and at the PR — so "passes before, fails after" proves the PR broke the example. Examples needing LLM keys, local files or placeholders are recorded as skipped with the reason. Identical runs are answered from the graph. Available to the agent, MCP clients and graph mode. | Parameter | Type | Required | Description | |---|---|---|---| | `example_id` | string | yes | CodeExample id from impact (repo:path:index) | | `pr_url` | string | no | Run before and after this pull request | Cypher it runs: `exampleForRun`, `installSet`, `cachedRun`, `upsertRun`, `linkRunToPr` ([full text](GRAPH-MODEL.md)). ### `decisions_for` Before judging a doc section, read the maintainers' decisions that govern it (directly, or through any symbol it mentions or its examples use, within 2 call hops; superseded decisions resolve to the current one) and the verdicts Staleproof gave this section on earlier pull requests (with whether maintainers accepted them). A governing decision can mean the section is intentionally as it is: cite it instead of editing. Available to the agent and MCP clients. | Parameter | Type | Required | Description | |---|---|---|---| | `target_id` | string | yes | DocSection id (repo:path#anchor) or Symbol id (repo:qualname) | Cypher it runs: `decisionsForSection`, `decisionsForSymbol`, `priorVerdicts` ([full text](GRAPH-MODEL.md)). ### `rank_sections` Order doc sections by how central their page is in the docs (PageRank over LINKS_TO, summed per page), so the most-read stale page is fixed first. Available to the agent, MCP clients and graph mode. | Parameter | Type | Required | Description | |---|---|---|---| | `section_ids` | string[] (≥ 1) | yes | DocSection ids from impact | Cypher it runs: `rankSections`, `rankSectionsByLinks` ([full text](GRAPH-MODEL.md)). ### `propose_edit` Propose replacement Markdown for a stale doc section (the whole section, heading line included, as read_section returned it). Refused unless the graph has a path from this pull request's changes to the section: Staleproof never edits a paragraph without one. Keep everything that is still true; change only what the PR falsified, or what makes an already-broken example fail (fails before and after: say so). Cite the runs that prove it. Available to the agent and MCP clients. | Parameter | Type | Required | Description | |---|---|---|---| | `pr_url` | string | no | The pull request; defaults to the one under check in agent mode, required over MCP | | `section_id` | string | yes | DocSection id (`owner/repo:path#anchor`), as returned by impact | | `new_text` | string (non-empty) | yes | Replacement Markdown for the whole section, heading line included | | `reason` | string (≥ 3 chars) | yes | Why: what the pull request changed and what the section claims | | `cites` | string[] | no | Decision ids this conclusion relies on | | `runs` | string[] | no | Run ids (from run_example) that support it | Cypher it runs: `sectionReachedByPr`, `readSection`, `createVerdict` ([full text](GRAPH-MODEL.md)). ### `record_verdict` Close impacted sections without editing them: ok (the change does not falsify it — say why, e.g. the example passed before and after), governed (a decision says it stays as is — cite the decision id), or skipped (cannot be judged — say why). Pass several section_ids when one reason covers them all. Every impacted section needs a verdict or an edit before you finish. Available to the agent and MCP clients. | Parameter | Type | Required | Description | |---|---|---|---| | `pr_url` | string | no | The pull request; defaults to the one under check in agent mode, required over MCP | | `section_id` | string | no | DocSection id (`owner/repo:path#anchor`), as returned by impact | | `section_ids` | string[] | no | Several sections sharing one verdict and reason | | `verdict` | `ok` \| `governed` \| `skipped` | yes | | | `reason` | string (≥ 3 chars) | yes | Why: what the pull request changed and what the section claims | | `cites` | string[] | no | Decision ids | | `runs` | string[] | no | Run ids that support the verdict | Cypher it runs: `sectionReachedByPr`, `createVerdict` ([full text](GRAPH-MODEL.md)). ### `write_decision` Record a maintainer decision ("the async client keeps the old kwargs until 2.0") and the doc sections or symbols it governs. Future checks cite it instead of re-flagging. Decisions come from people: call this only when a human asked for it. Available to MCP clients, the CLI (`decide`) and the HTTP API (`POST /api/decisions`); never the agent. | Parameter | Type | Required | Description | |---|---|---|---| | `text` | string (≥ 5 chars) | yes | The decision, in one sentence | | `governs` | string[] (≥ 1) | yes | DocSection and/or Symbol ids | | `source_url` | string | no | Where the decision was made (issue, PR comment, ADR) | | `author` | string | no | Who decided | | `supersedes` | string | no | Id of the decision this replaces | Cypher it runs: `createDecision`, `governSections`, `governSymbols`, `supersedeDecision` ([full text](GRAPH-MODEL.md)). ### `docs_patch` The docs fix for a checked pull request as a git-apply-able unified diff, plus a PR description with the evidence path behind each edited section. Built from the latest proposed edits. Available to MCP clients; the same patch comes from the CLI (`patch`) and the HTTP API (`GET /api/patch`). | Parameter | Type | Required | Description | |---|---|---|---| | `pr_url` | string | yes | GitHub pull request URL (`https://github.com/<owner>/<repo>/pull/<n>`) | Cypher it runs: `verdictsForPr` ([full text](GRAPH-MODEL.md)). <!-- generated:tools:end --> ## Results What each tool returns in `data`, the summary it writes, what it changes in the graph, and the errors it raises. The shapes are defined in [src/tools/](../src/tools/) and, for `check_pull_request` and `docs_patch`, [src/mcp/server.ts](../src/mcp/server.ts). ### `check_pull_request` result | Field | Meaning | |---|---| | `report` | The Markdown report, the same one the GitHub Action posts: the title, the impact summary, the examples run before and after, the impacted sections with their evidence paths in PageRank order, the cleared symbols and the latest verdicts ([format](github-action.md#the-comment)) | | `checkId` | The `Check` node stored for this call (`chk-…`) | | `status` | `done`, or `error` when `impact` failed | Summary: the graph-mode check's summary: the impact and clearance summaries, then `Example runs not requested.` (or the runs' outcomes with `run_examples`), then `Graph mode: fixed tool order and no verdicts; agent mode (a model) judges each section and proposes the edits.` Over MCP the client receives the report and `Stored as check <id>. <summary>`. Side effects: a graph-mode check ([`runGraphCheck`](agent.md#graph-mode)): the pull request is ingested on the first call, a `Check` node with the trace is stored, and with `run_examples: true` up to 3 examples run before and after (Run nodes). The local stdio server first indexes the pull request's repository when the graph does not have it; a hosted server indexes it into its owner's workspace, and executes no examples. Errors: those of `impact`, and on a hosted server the [hosted limits](troubleshooting.md#hosted-service). ### `check_changes` result | Field | Meaning | |---|---| | `impact` | The shape of the [`impact` result](#impact-result), for the edits: `pr` describes them (`state: 'local'`, `url` a GitHub compare or tree link, `title` such as `2 edited files on acme/mini-client@a6214ef` or `Working tree of main against a6214ef`), `changed[]` the symbol-level changes, `sections[]` every impacted section (not paged), `predicted[]` and `mismatched[]` | | `cleared[]` | Changed symbols no doc reaches within 3 call hops, as in the [`clearance` result](#clearance-result) | With `files`, each file's new content is compared with the same path at the commit the graph indexed (`base` overrides it), read from a local clone when the server has one, else from `raw.githubusercontent.com`; `content: null` deletes the file, and a path not in that commit is a new file. With `dir` (stdio server only), the checkout's working tree is compared with `base`, else its merge base with the remote default branch, as `staleproof check` without a pull request does. Summary: `<n> changed symbol(s) reach(es) <N> doc section(s) (<b> breaking, <a> signature, <c> behavior; <x> in other repositories).`, then ` <k> example(s) is/are predicted to break, the call no longer fitting the changed code.` and ` <m> changed symbol(s) reach(es) no doc.` when they apply; or `These changes touch no Python symbol in <repo>; no docs can be falsified through the code.` Side effects: none; it writes nothing to the graph (a server may index the repository first when the graph lacks it). Errors: `repo (owner/name) is required with files`; `<repo> is not indexed in the "<graph>" graph`; `send the changed files: files: [{path, content}] with repo (this server cannot read your disk)` (`dir`, or neither, on an HTTP server); `cannot tell which GitHub repository <dir> is (origin: <url>); pass --repo owner/name`; git errors for a `dir` that is not a checkout; `GitHub raw <path>@<sha7> failed: <status>`. ### `docs_for_symbol` result | Field | Meaning | |---|---| | `query` | The symbol as given | | `symbols[]` | The symbols looked up, at most 5: `id`, `qualname`, `kind`, `repo`, `signature`, `url` (on the default branch) | | `ambiguous` | True when the name matched several symbols and none exactly; narrow it with a qualified name or `repos` | | `sections[]` | Every doc section or example script that reaches one of them: `section`, `heading`, `repo`, `url`, `evidence` (symbol first, doc last), `hops`, `confidence`, `examples` (ids of the examples on the path), `symbols` (which of the looked-up symbols reach it) | | `mismatched[]` | Lines in those examples that already do not fit the code: `example`, `url`, `line`, `kind`, `detail`, `call` | An id or qualified name that matches exactly wins over suffix and bare-name matches. Sections are ordered by hops, then repository and id. Summary: `<n> doc section(s) reach(es) <qualname> (<d> name or call it directly; <r> repositor(y|ies)).`, followed by ` <m> line(s) in those examples already do not fit the code.` and ` The name is ambiguous: pass a qualified name or repos to narrow it.` when they apply; `No doc section or example reaches <qualname> within 3 call hops: changing it cannot make a doc false through the code.`; or `No live symbol in the "<graph>" graph is named <symbol>.` (not an error). For several symbols, `<qualname>` reads `<n> symbols named <symbol>` (with `(of <total>)` when more matched). No side effects. ### `audit_docs` result The [docs audit](audit.md) of the given repositories, or of every repository in the graph. | Field | Meaning | |---|---| | `org`, `repos[]` | The graph, and each audited repository: `name`, `sha`, `symbols`, `sections`, `indexVersion` | | `outdatedIndex` | True when a repository was indexed by a Staleproof older than 0.3 (no `MISMATCH` edges or export flags) | | `score`, `grade` | 0-100 and `A`-`F` ([the formula](audit.md#the-score)) | | `deductions[]` | `points` and `why` for each deduction | | `examples` | `total`, `python`, `linked` (calling at least one indexed symbol), `mismatched` (with at least one `MISMATCH` line) | | `mismatches[]` | Every example line that no longer fits the code: `example`, `url` (to the line), `repo`, `path`, `heading`, `line`, `kind`, `detail`, `call`, `symbol` | | `coverage` | `api` (public symbols), `documented`, `percent` | | `undocumented[]` | The most used public symbols no doc names or calls, at most `limit`: `qualname`, `repo`, `kind`, `signature`, `url`, `callers`, `crossRepoCallers` | | `runs[]`, `runCounts` | The latest run of each example at the indexed commits (`example`, `url`, `status`, `error`, `reason`), and the counts `pass`, `fail` (with timeouts), `skipped`, `error` | | `generatedAt` | Milliseconds since the epoch | Summary: `Docs health <score>/100 (<grade>) for <repo or "<n> repositories">.`, then `<m> of <linked> examples that call the code no longer fit it (<lines> lines).` (or `All <linked> examples that call the code fit it.`), `<documented> of <api> public symbols (<percent>%) are named or called by a doc.`, `<f> of <ran> examples run at the indexed commits fail.` when runs are recorded, and the older-index sentence when `outdatedIndex`. Side effects: none; over MCP it runs no example. Errors: `not indexed in the "<graph>" graph: <repos>` for a named repository the graph does not hold (the stdio server indexes it first). ### `impact` result | Field | Meaning | |---|---| | `pr` | `url`, `title`, `repo`, `state` (`merged`, `open` or `closed`), `before` and `after` (the commits compared), `docsTouched` (Markdown files the pull request itself edits) | | `changed[]` | Every changed symbol, `added` ones included: `symbol` (qualified name), `change` (the [change kind](concepts.md#change-kinds)), and when present `before` and `after` signatures and `renamedTo` | | `sections[]` | The current page of impacted sections, at most 40 (fields below) | | `totalSections` | Impacted sections across all pages | | `offset` | Index of the first section on this page | | `truncated` | True when more sections follow this page | | `counts` | `breaking`, `api`, `behavior`: sections per severity, over all pages | | `predicted[]` | Over all pages, the example lines [predicted to break](concepts.md#predicted-breaks): `example`, `url` (to the line), `section`, `repo`, `path`, `line` (in the doc file), `symbol`, `change`, `kind` (`removed`, `missing-import`, `unknown-keyword`, `too-many-arguments` or `missing-argument`), `detail`, `call` (as written) and `confidence` (`exact`, or `by-name` for a removed symbol linked by name) | | `mismatched[]` | Impacted example lines that already did not fit the code when it was indexed (`MISMATCH` edges), except those the predictions already report: `example`, `url`, `line`, `kind`, `detail`, `call` | Each impacted section: | Field | Meaning | |---|---| | `section` | The DocSection id, or the CodeExample id for an example script (which has no section) | | `heading` | The section's heading path | | `repo`, `path`, `url` | Where it is; `url` links to the doc node on GitHub at the indexed commit | | `severity` | `breaking`, `api` or `behavior`, from the most severe change that reaches it | | `evidence` | The [evidence path](concepts.md#evidence-path): changed symbol first, doc last | | `hops` | Edges on that path ([hops](concepts.md#hops)) | | `confidence` | `exact`, `by-name` or `late` ([confidence](concepts.md#confidence)) | | `changes[]` | Every change that reaches the section: `symbol`, `change`, `before`, `after` | | `examples[]` | Ids of the section's code examples the walk reached | | `crossRepo` | True when the section is in a different repository from the pull request | Summary: `<n> changed symbols (and <m> added) reach <N> doc sections (<b> breaking, <a> signature, <c> behavior; <x> in other repositories).` The added count appears only when there are added symbols, and the wording agrees in number (`1 changed symbol reaches 1 doc section`). With predictions it continues ` <k> examples are predicted to break, the call no longer fitting the changed code (<id>, <id>, <id>, …).`, and with lines already out of date ` <m> impacted example(s) already did not fit the code before this pull request.` With more than 40 sections, or an `offset`, it continues `Showing <i>–<j> of <N>; call impact again with offset=<k> for the rest.` (or ends `Showing <i>–<j> of <N>.` on the last page). A pull request that changed no symbol of the indexed code (Python, TypeScript or JavaScript) gives `<url> changed no Python symbols in <repo>; no docs can be falsified through code.` Side effects: the first call for a pull request ingests it: the `PullRequest` node, `CHANGED_IN` edges, removed symbols recreated with `late` links ([Pull request ingestion](concepts.md#pull-request-ingestion)). With `git`, ingestion also fetches the pull request's repository and checks out its default branch in the local clone; the GitHub metadata of a merged pull request is cached under `STALEPROOF_HOME/cache`. Errors: - `not a GitHub pull request URL: "<input>" (expected https://github.com/<owner>/<repo>/pull/<n>)` - `<owner/repo> is not indexed in the "<graph>" graph. Run: staleproof index <owner/repo> (plus the repositories that document it).` - `GitHub API rate limit reached (60 requests/hour without a token; set GITHUB_TOKEN to raise it). It resets in <n> min.` - `GitHub: <path> not found (a private repository needs GITHUB_TOKEN with read access)` (without the parenthesis when a token is set) - `GitHub <METHOD> <path> failed: <status> <body>`, `git <args> failed in <dir>: <stderr>`, `merge commit <sha> has no parent` ### `clearance` result | Field | Meaning | |---|---| | `cleared[]` | Changed symbols (not `added`) with no doc within 3 call hops: `symbol` (qualified name), `name`, `repo`, `change`, `visibility`, `textMatches`, `grep` | | `docsRepos[]` | The repositories that have doc sections, which `textMatches` searched | `textMatches` is the number of lines matching `git grep -cw <name> -- '*.md' '*.mdx' '*.rst'` in the local clones of `docsRepos` (0 for repositories with no local clone, so always 0 on serverless hosts). `grep` is the command to check it yourself: `git grep -nw <name> -- '*.md' '*.mdx' '*.rst'`. Summary, one of: - `The PR changed no existing symbols; there is nothing to clear.` - `Every changed symbol reaches at least one doc; nothing is cleared.` - `<n> changed symbols reach no doc within 3 hops (<q> also have zero text matches in the docs; <r> share a name with text in the docs but no path resolves to them).` Side effects: ingests the pull request if needed, as `impact` does. Errors: the same as `impact`'s ingestion errors. ### `read_section` result | Field | Meaning | |---|---| | `id`, `repo`, `path` | The section | | `heading` | Its heading path | | `body` | Its Markdown, heading line included (stored up to 8000 characters) | | `url` | GitHub link at the indexed commit | | `lineno`, `endLineno` | Its lines in the file | | `examples[]` | Its code examples, in order: `id`, `lang`, `code`, `lineno` | Summary: `<heading> (<path>, lines <a>-<b>, <n> examples).` For example `falkordb-py > Usage > Asynchronous Example (README.md, lines 64-114, 1 examples).` An unknown id is not an error: `data` is `null` and the summary is `No doc section <id>.` No side effects. ### `run_example` result | Field | Meaning | |---|---| | `example` | The CodeExample id | | `url` | Its GitHub link | | `verdict` | The [outcome](concepts.md#run-outcomes): `breaks`, `fixes`, `passes`, `already-broken`, `changed-failure`, `fails`, `skipped` or `error` | | `runs[]` | One record per run, `before` then `after` with a pull request, otherwise one `current` run | | `editedByPr` | `true` when the pull request edits the example's page and the after run used the page as the pull request leaves it ([details](example-runner.md#pull-requests-that-edit-the-page)) | Each run record: `id`, `example`, `status` (`pass`, `fail`, `timeout`, `skipped`, `error`), `reason`, `error` (last traceback line), `traceback`, `stdout`, `durationMs`, `stitched` (ids of earlier examples prepended), `installed` (`repo@sha` list), `side`, `cached` (answered from the graph) and `createdAt`. See [The Run node](example-runner.md#the-run-node). Summary, by outcome (commits shortened to 7 characters): | Outcome | Summary | |---|---| | `breaks` | `Passes at <before> and fails at <after> with <error>: the PR breaks this example.` | | `fixes` | `Fails at <before> and passes at <after>: the PR fixes this example.` | | `passes` | `Passes before and after the PR (<before>, <after>): no observable break.`, or `Passes at the indexed commits.` without a pull request | | `already-broken` | `Fails both before and after the PR with <error>: broken already, not by this PR.` | | `changed-failure` | `Fails before (<error>) and after with a different error (<error>).` | | `fails` | `Fails at the indexed commits: <error>.` | | `skipped` | `Not run: <reason>.` | | `error` | `The sandbox could not be prepared: <first line of the reason>` | ` The after run uses the page as this pull request edits it.` is appended when the pull request edits the example's page, and ` (recorded earlier; not re-run)` when a run was answered from the graph. If the `before` run is skipped, the `after` run is not attempted. Side effects: a `Run` node per side (unless an identical run is reused), with `OF` to the example and `AT` to the pull request; clones or fetches the repositories, builds a virtualenv, and flushes the scratch FalkorDB. See [Example runner](example-runner.md). Errors: `no code example <id>`; `<pr-url> has not been checked yet; call impact first` (the pull request is not in the graph yet); the pull request URL error. Problems inside the sandbox are not errors: they come back as runs with status `error`, `fail` or `timeout`. ### `decisions_for` result | Field | Meaning | |---|---| | `decisions[]` | Active decisions that apply: `id`, `text`, `source`, `author`, `createdAt`, `matched` (the decision the graph reached; differs from `id` when it was superseded), `via` (the section's heading path for a direct match, else the governed symbol's qualified name) | | `history[]` | For a section id only: its 5 most recent verdicts on any pull request: `id`, `verdict`, `status`, `reason`, `pr`, `title`, `createdAt` | Summary: `<n> active decision(s) govern(s) this: "<text>" (<id>[, supersedes <matched>], via <via>); …` (text cut at 80 characters), or `No decision governs this.`, followed when there is history by `Earlier: <verdict>[ (<status>)] on <owner>/<repo>/pull/<n>, …`. The status is shown when it is not `proposed`. An id containing `#` is treated as a DocSection id, anything else as a Symbol id. No side effects; an unknown id returns no decisions. See [How decisions_for resolves decisions](memory.md#how-decisions_for-resolves-decisions). ### `rank_sections` result | Field | Meaning | |---|---| | `ranked[]` | The given sections that exist, most central page first: `id`, `heading`, `page` (`repo:path`), `pageScore`, `score` | | `method` | `pagerank`, or `in-degree` when the graph algorithm failed | With PageRank, `score` is the section's own score and `pageScore` the sum over its page. With the in-degree fallback, `pageScore` is the number of links into the page and `score` is 0. Summary: `Ranked <n> sections by PageRank of their page; most central: <heading> (<page>).` (`by inbound links` for the fallback), or `None of these ids are doc sections.` No side effects. It does not raise errors of its own: a failing PageRank call falls back. ### `propose_edit` result | Field | Meaning | |---|---| | `id` | The new Verdict id | | `evidence` | The shortest path from the pull request's changes to the section (or one of its examples) | Summary: `Edit proposed for <section> (evidence: <changed symbol> ← … ← <doc>).` Side effects: a `Verdict` with `verdict: 'stale'`, `status: 'proposed'`, `new_text`, `old_text` (the section body as stored), `evidence`, `confidence: 'path'` and the check id (empty over MCP), with `ABOUT`, `FOR`, `CITES` (existing decisions only) and `EVIDENCE` (existing runs only) edges. The edit reaches files only through the [docs patch](concepts.md#the-docs-patch). Errors: - `pr_url is required` (over MCP, without `pr_url`) - `refused: no path in the graph from <pr-url>'s changes to <section>. Staleproof never edits a paragraph without a path; record_verdict instead if it needs no change.` The path must have at most 3 call hops and start from a change other than `added`. - `new_text is identical to the current section; use record_verdict with ok` - `no doc section <section> or pull request <pr-url> in the graph` ### `record_verdict` result | Field | Meaning | |---|---| | `ids` | One new Verdict id per distinct section | | `evidence` | Per section id, the path from the pull request's changes, or `[]` when there is none | Summary: `Verdict <ok|governed|skipped> recorded for <n> section(s).` Side effects: one `Verdict` per section, `status: 'proposed'`, with `confidence: 'path'`, or `'none'` when no path reaches the section (unlike `propose_edit`, a missing path does not block a verdict), and the same edges as `propose_edit`. Errors: - `give section_id or section_ids` (reported as invalid input) - `pr_url is required` (over MCP, without `pr_url`) - `a governed verdict must cite the decision id(s)` - `no doc section <section> or pull request <pr-url> in the graph` ### `write_decision` result | Field | Meaning | |---|---| | `id` | The new Decision id (`dec-…`) | | `governed` | How many of the given ids matched a DocSection or Symbol | Summary: `Recorded decision <id> governing <n> node(s).`, with `, superseding <old-id>` before the period when `supersedes` is given. Side effects: a `Decision` (`status: 'active'`, `author` default `unknown`), `GOVERNS` edges to the matching sections and symbols, and with `supersedes`, a `SUPERSEDES` edge to the old decision, whose status becomes `superseded` (an unknown old id is ignored). See [Decisions and memory](memory.md). Errors: `none of <ids> is a DocSection or Symbol id; decision not recorded`. `POST /api/decisions` also limits `text` to 500 characters. ### `docs_patch` result | Field | Meaning | |---|---| | `diff` | A unified diff of every edited file, `git apply`-able in each docs repository | | `title` | `docs: update for <owner/repo>#<n> (<pull request title>)` | | `body` | The pull request description: per edited section, the reason, evidence path, runs and decisions; then the sections left unchanged; then any edits not applied | | `files[]` | `<repo>:<path>` for each changed file | Summary: `<n> file(s) changed: <repo>:<path>, ….` or `No proposed edits for this pull request yet.` It reports no queries. The patch is built from the latest verdict per section: `stale` verdicts with replacement text that were not rejected. Files are read at the indexed commit, from the local clone when there is one, otherwise from `raw.githubusercontent.com`. An edit whose section changed since indexing, or whose file is missing at that commit, is listed in the body under "Not applied" instead of being applied. No side effects. `GET /api/patch` returns the same patch with the verdicts and conflicts as separate fields; `staleproof patch` prints it. Errors: `<pr-url> has not been checked`; `GitHub raw <path>@<sha> failed: <status>`; `git show` errors from a local clone. ## Related - [Core concepts](concepts.md): the terms these results use - [The agent](agent.md): how the agent calls the tools and what it is told - [MCP server](mcp.md): connecting an MCP client to these tools - [Graph model and Cypher](GRAPH-MODEL.md): the full text of every query - [Example runner](example-runner.md): what `run_example` does underneath --- <!-- page: Configuration · Reference · source: https://staleproof.dev/docs/configuration.md --> # Configuration `staleproof` is configured by environment variables and a few command-line flags; it has no configuration file of its own and does not load `.env` files. Nothing has to be set: with no variables, the CLI starts a local FalkorDB and names each graph after the repository's owner. A repository being indexed can carry a small file of its own, `.staleproof.toml`, to leave paths out or mark examples not to run. This page lists every variable the code reads, with its default, its meaning and the commands that use it, then the global CLI options and their precedence, the files kept under `STALEPROOF_HOME`, the repository configuration and example directives, and complete configurations for local development, FalkorDB Cloud with DeepSeek, CI and Vercel. It is for anyone setting up the CLI, the server, the GitHub Action or a deployment. ## The variables you usually set ```bash export FALKORDB_URL='falkor://localhost:6379' # the FalkorDB holding the org graph (this is the default) export STALEPROOF_ORG=acme # graph name, one per organisation (default: the repository's owner) export GITHUB_TOKEN=<token> # private repositories; raises the GitHub API limit of 60 requests/hour export ANTHROPIC_API_KEY=<key> # agent mode; without it, use graph mode (--no-agent) ``` Everything else has a working default. Agent mode on an Anthropic-compatible provider such as DeepSeek also needs `ANTHROPIC_BASE_URL` and `STALEPROOF_MODEL` ([below](#model-agent-mode)). ## Environment variables `loadConfig` ([src/config.ts](../src/config.ts)) reads most of them once, when a command starts. Every command except `db` loads it, so an invalid value (for example in `STALEPROOF_EFFORT`) stops every command, not only the ones that use it. Staleproof was first published as Lockstep (npm `lockstep-docs`). Every `STALEPROOF_*` variable is also read under its former name, `LOCKSTEP_*`, when the new one is unset, and an existing `~/.lockstep` is used as the home when `~/.staleproof` does not exist, so deployments and caches made under the old name keep working. | Variable | Default | Meaning | Used by | |---|---|---|---| | `FALKORDB_URL` | `falkor://localhost:6379` | The FalkorDB server holding the org graph. Unset (and no `--db`), the CLI starts a local server on that address when nothing answers there | every command except `db`; Vercel function; `npm run snapshot` | | `STALEPROOF_ORG` | the owner of the repository a command is about, else `falkordb` | Organisation, which names the graph. In a hosted server: the curated graph | same | | `STALEPROOF_HOME` | `~/.staleproof`; `/tmp/staleproof` when `VERCEL` is set | Clones, downloaded tarballs, GitHub cache, virtualenvs, local FalkorDB | `index`, `check`, `run`, `patch`, `mcp`, `serve`, `db` | | `GITHUB_TOKEN` | — | GitHub token for the API, git clones, tarball downloads and file reads, so private repositories work | indexing, pull request ingestion (`check`, `run --pr`, `serve`, `mcp`), the docs patch, hosted pull request reads (hosted downloads never send it, so a hosted server indexes public repositories only); required by `patch --open-pr` | | `GH_TOKEN` | — | Used when `GITHUB_TOKEN` is unset | same | | `STALEPROOF_INGEST` | — | `github`: work as if git were missing even when it is installed: download repositories as tarballs and ingest pull requests through the GitHub API | indexing, pull request ingestion | | `STALEPROOF_HOSTED` | Vercel function: on; `serve`: off | Hosted mode: any public repository is indexed on demand into a workspace graph of its own. The Vercel function turns it off only for `off`; `serve` turns it on only for `on` (or `--hosted`) | Vercel function, `serve` | | `STALEPROOF_AGENT_CHECKS_PER_DAY` | `50` | Agent checks a hosted server starts per day (UTC), across all callers; over it, `POST /api/check` with `mode: "agent"` answers 429 | Vercel function and `serve --hosted` | | `ANTHROPIC_API_KEY` | — | Model API key (Claude API or an Anthropic-compatible provider) | `check` (agent mode), `serve`, Vercel function | | `ANTHROPIC_BASE_URL` | Claude API | Model endpoint; a host other than `api.anthropic.com` switches to compatible mode | same | | `STALEPROOF_MODEL` | `claude-sonnet-5-5` | Agent model | same; `check --model` overrides | | `STALEPROOF_EFFORT` | `medium` | `low`, `medium`, `high`, `xhigh` or `max` | same; `check --effort` overrides | | `STALEPROOF_FALLBACKS` | on | `off` disables the server-side refusal fallback (Claude API only) | same | | `STALEPROOF_SCRATCH_URL` | — | A separate FalkorDB that docs examples run against (host and port only) | example runs: `run`, `check`, `serve`, `mcp` | | `STALEPROOF_SCRATCH_PORT` | `6390` | Port of the scratch server Staleproof starts itself, when `STALEPROOF_SCRATCH_URL` is unset | same | | `STALEPROOF_RUN_TIMEOUT_MS` | `30000` | Time limit of one example run, in milliseconds | same | | `STALEPROOF_CHECK_BUDGET_MS` | `240000` when `VERCEL` is set, else none | Wall-clock budget of one agent check in the server | `serve`, Vercel function (not `check`) | | `STALEPROOF_PRELOADED` | falkordb-py #262, #267, #276 | Comma-separated pull request URLs listed on the home page | `serve` (`--preload` overrides), Vercel function, `npm run snapshot`, `scripts/seed.sh` | | `PORT` | `8080` | Port `serve` listens on | `serve` (`--port` overrides) | | `HOST` | `127.0.0.1` | Interface `serve` binds | `serve` (`--host` overrides) | | `VERCEL` | — | Set by Vercel; changes the defaults of `STALEPROOF_HOME` and `STALEPROOF_CHECK_BUDGET_MS` | all | | `USER` | — | Default reviewer name for `verdict --by` (else `maintainer`) | `verdict` | | `FALKORDBLITE_VERSION` | `0.10.0` | The `falkordblite` wheel that provides the local FalkorDB binaries | `db`, the managed scratch server | | `FACTS_OUT` | `docs/FACTS.md` | Where `scripts/seed.sh` writes FACTS.md | `scripts/seed.sh` | Credential detection also reads `ANTHROPIC_AUTH_TOKEN`, `ANTHROPIC_PROFILE`, `ANTHROPIC_FEDERATION_RULE_ID`, `ANTHROPIC_IDENTITY_TOKEN`, `ANTHROPIC_IDENTITY_TOKEN_FILE`, `ANTHROPIC_CONFIG_DIR` and `XDG_CONFIG_HOME` ([details](#model-agent-mode)). The sandbox shim reads `STALEPROOF_SCRATCH_HOST`, `STALEPROOF_SCRATCH_PORT` and `STALEPROOF_DB_PORTS` inside the example process ([details](#example-runner)). Empty values: `ANTHROPIC_API_KEY`, `ANTHROPIC_BASE_URL`, `STALEPROOF_SCRATCH_URL`, `STALEPROOF_CHECK_BUDGET_MS` and `VERCEL` treat an empty string as unset. The others take it literally: an empty `GITHUB_TOKEN` hides `GH_TOKEN`, an empty `STALEPROOF_PRELOADED` lists no pull requests, an empty `STALEPROOF_ORG` fails with `invalid org name: ""`, an empty `STALEPROOF_EFFORT` fails validation, and an empty number becomes 0. Unset a variable instead of setting it to an empty string; note that Docker Compose's `${VAR:-}` passes an empty string. ### Graph and organisation **`FALKORDB_URL`** is passed to the `falkordb` client, which turns a leading `falkor` into `redis` and hands the URL to node-redis: | Form | Meaning | |---|---| | `falkor://host:port` | Plain TCP (the FalkorDB Cloud free tier endpoint is plain TCP) | | `falkors://host:port` | TLS | | `falkor://user:password@host:port` | With credentials; FalkorDB Cloud's user is `falkordb` | | `redis://...`, `rediss://...` | Accepted as is | The user name and password are URL-decoded, so percent-encode any reserved character in them (`@` is `%40`, `:` is `%3A`, `/` is `%2F`, `#` is `%23`, `%` is `%25`). Error messages show the URL with the password replaced by `***`. The connection times out after 5 seconds; after a dropped connection the client retries 4 times, waiting 200 ms and doubling up to 2 s, then gives up with `FalkorDB unreachable`. One server can hold any number of org graphs. **`STALEPROOF_ORG`** names the graph. It is slugged: trimmed, lowercased, every run of characters other than `a-z`, `0-9`, `_` and `-` replaced by `-`, leading and trailing `-` removed. `Acme Inc` becomes `acme-inc`. A name with nothing left fails with `invalid org name: "<name>"`. Each organisation is a separate graph on the same server; indexing, checks and memory never cross between graphs. **The default graph.** With neither `--org` nor `STALEPROOF_ORG`, a CLI command uses the graph of the owner of the repository it is about, slugged: the first repository of `index`, the pull request of `check` and `patch`, the example id of `run`, the first `--governs` id of `decide`. `check https://github.com/encode/httpx/pull/<n>` reads `encode`; FalkorDB's repositories still land in `falkordb`. Commands that name no repository (`mcp`, `serve`, `verdict`, `facts`) use `falkordb`. Set `STALEPROOF_ORG` when one graph holds repositories of several owners, so every command and the MCP server use the same one. **Without `FALKORDB_URL`**, a command that cannot reach `localhost:6379` starts a local FalkorDB there (the falkordblite server of `staleproof db start`, installed on first use) instead of failing. With `FALKORDB_URL` or `--db` set, an unreachable server is an error. **`STALEPROOF_HOME`** is where files are kept ([layout](#files-under-staleproof_home)). On Vercel the default is `/tmp/staleproof`, because a function's file system is read-only except `/tmp`. The Docker image sets it to `/data`. ### GitHub **`GITHUB_TOKEN`**, else **`GH_TOKEN`**, is the one GitHub credential. With it set, private repositories the token can read work everywhere a public one does: | Where | How the token is sent | |---|---| | `api.github.com` (pull request metadata and diffs, opening docs pull requests) | `Authorization: Bearer <token>` | | git clones and fetches (indexing, example installs) | Through git's environment configuration (`GIT_CONFIG_COUNT`, `GIT_CONFIG_KEY_0`, `GIT_CONFIG_VALUE_0`), as an `http.https://github.com/.extraheader`, so only to `https://github.com/`, and never in the remote URL or on a command line. Not added when the environment already sets `GIT_CONFIG_COUNT` | | Tarball downloads (no git) | From the API's `repos/<owner>/<name>/tarball/<ref>` endpoint instead of `codeload.github.com` | | `raw.githubusercontent.com` (file contents without a clone) | `Authorization: token <token>` | Without a token, GitHub allows 60 API requests per hour per IP address, and the error is `GitHub API rate limit reached (60 requests/hour without a token; set GITHUB_TOKEN to raise it). It resets in <n> min.`; a 404 adds `(a private repository needs GITHUB_TOKEN with read access)`. - **What uses the API**: ingesting a pull request (once per pull request; later checks reuse the graph). With git, that is one request for the metadata, cached on disk for merged pull requests. Without git, or with `STALEPROOF_INGEST=github`, it is up to five (metadata, the commit range, the changed files and the file list at both commits); file contents come from `raw.githubusercontent.com`, which the code treats as outside the API limit. Indexing uses no API request with git (it clones over HTTPS) or without a token (the tarball comes from `codeload.github.com`); a tarball download with a token is one API request. - **`patch --open-pr`** refuses to run without a token (`opening a pull request needs GITHUB_TOKEN`). The token must be able to fork the docs repository, push a branch to the fork and open a pull request; for a classic token that is the `public_repo` scope. With `--in-repo` there is no fork: the token needs `contents: write` and `pull-requests: write` on the docs repository itself, which is what a GitHub Actions job token can be given for its own repository. - **A hosted server** downloads with its token, so it can index any repository that token reads, for whoever asks. Give it a fine-grained token limited to public repositories, read-only ([Deploy](DEPLOY.md#hosted-mode)). **`STALEPROOF_INGEST=github`** makes Staleproof work as it does where git is missing: `index` and indexing on first use download tarballs, and pull request ingestion reads commits through the GitHub API, the paths serverless hosts take automatically. Example runs still need git. ### Model (agent mode) | Variable | Effect | |---|---| | `ANTHROPIC_API_KEY` | Passed to the Anthropic SDK as the API key. Its presence turns on agent mode in `serve` | | `ANTHROPIC_BASE_URL` | Unset or empty: the Claude API. A URL whose host is not `api.anthropic.com` (or that does not parse) switches on compatible mode | | `STALEPROOF_MODEL` | Model name; default `claude-sonnet-5-5`. `check --model` overrides it | | `STALEPROOF_EFFORT` | `low`, `medium` (default), `high`, `xhigh` or `max`. Any other value fails every command except `db` with `STALEPROOF_EFFORT must be one of low, medium, high, xhigh, max`, even when `--effort` is also given. `check --effort` itself is not checked locally; an unknown value is sent to the model API as is | | `STALEPROOF_FALLBACKS` | Exactly `off` disables the server-side refusal fallback. It only matters on the Claude API: compatible mode never sends it, so there is no need to set it for DeepSeek | **Compatible mode** sends only the portable fields of the Messages API: `model`, `max_tokens`, `system` (as a plain string), `tools`, `messages`, `thinking` (`{type: "enabled", budget_tokens: min(16000, max_tokens - 1000)}`) and `output_config.effort`. It sends no beta flags, no refusal fallback, no prompt caching and no eager input streaming. It refuses a model name starting with `claude-` (any case), because a compatible provider would answer with its own model under that name and every trace would be mislabelled: ```text "claude-sonnet-5-5" is a Claude model name, but the model endpoint is an Anthropic-compatible provider, which would answer with its own model under that name; set STALEPROOF_MODEL to the provider's model (e.g. deepseek-v4-pro) ``` Setting `ANTHROPIC_BASE_URL` without `STALEPROOF_MODEL` therefore always hits this error. **Credentials** are found when any of these holds; otherwise agent mode fails with `the agent needs model API credentials: ...` and `serve` runs graph mode only: - `ANTHROPIC_API_KEY`, `ANTHROPIC_AUTH_TOKEN` or `ANTHROPIC_PROFILE` is set; - `ANTHROPIC_FEDERATION_RULE_ID` is set with `ANTHROPIC_IDENTITY_TOKEN` or `ANTHROPIC_IDENTITY_TOKEN_FILE`; - the Anthropic config directory exists: `ANTHROPIC_CONFIG_DIR`, else `$XDG_CONFIG_HOME/anthropic`, else `~/.config/anthropic` (created by `ant auth login`). Each model turn may produce up to 32,000 tokens. There is no variable for it; the library's `AnthropicModel` takes `maxTokens`. ### Example runner The runner installs the repositories at the commits before and after a pull request and runs a docs example against a scratch FalkorDB ([Example runner](example-runner.md)). | Variable | Effect | |---|---| | `STALEPROOF_SCRATCH_URL` | Use this server as the scratch database. Only the host and port are read (`[scheme://]host[:port]`, default port 6379); the scheme is ignored and TLS is not used. A URL with credentials is refused: `STALEPROOF_SCRATCH_URL must not carry credentials: docs examples connect to the scratch server without a password`. Staleproof does not start or stop this server | | `STALEPROOF_SCRATCH_PORT` | Without `STALEPROOF_SCRATCH_URL`, Staleproof starts its own scratch server on `127.0.0.1:<port>` (default 6390) with `scripts/falkordb-local.sh`, the same falkordblite binaries as `staleproof db start` | | `STALEPROOF_RUN_TIMEOUT_MS` | Each run (before and after are separate runs) is killed after this many milliseconds and recorded as `timeout` with the reason `no exit after <s>s`. Default 30000. Creating a virtualenv (300 s) and installing into it (600 s) have fixed limits | The scratch server is flushed before every run (`FLUSHALL`, then `GRAPH.UDF FLUSH`). Staleproof refuses to flush the server that holds the org graph: when the scratch host and port equal those of `FALKORDB_URL` (`localhost` and `127.0.0.1` count as the same host), each run ends with status `error` and the reason `refusing to flush <host>:<port>: it is the server holding the org graph (FALKORDB_URL). Point STALEPROOF_SCRATCH_URL / STALEPROOF_SCRATCH_PORT at a separate server.` Inside the sandbox, the example process inherits no environment. The runner sets `STALEPROOF_SCRATCH_HOST` and `STALEPROOF_SCRATCH_PORT` for the network shim, which sends connections to a local host on the ports in `STALEPROOF_DB_PORTS` (default `6379`) to the scratch server. Staleproof never sets `STALEPROOF_DB_PORTS`, and because nothing is inherited you cannot set it either: the remapped port is always 6379. The install step (`uv venv` and `uv pip install`, or `python3 -m venv` and `pip install`) runs the repositories' build code at the pull request's commits, so it gets an allow-listed environment, not the server's: only the variables matching `INSTALL_ENV` in [src/runner/sandbox.ts](../src/runner/sandbox.ts) are passed on. | Passed to the install step | Variables | |---|---| | Paths and user | `PATH`, `HOME`, `TMPDIR`, `TEMP`, `TMP`, `USER`, `LOGNAME`, `SHELL`, `XDG_CACHE_HOME`, `XDG_DATA_HOME` | | Locale and time | `LANG`, `LC_*`, `TZ` | | Package tools | every `UV_*` and `PIP_*` variable | | Proxies | `HTTP_PROXY`, `HTTPS_PROXY`, `NO_PROXY`, `ALL_PROXY` (and their lowercase forms) | | Certificates | `SSL_CERT_FILE`, `SSL_CERT_DIR`, `REQUESTS_CA_BUNDLE`, `CURL_CA_BUNDLE` | | Windows | `SYSTEMROOT`, `COMSPEC`, `PATHEXT` | Everything else, including `FALKORDB_URL`, `ANTHROPIC_API_KEY` and `GITHUB_TOKEN`, is withheld. uv and pip settings such as `UV_INDEX_URL` or `PIP_INDEX_URL` (a package mirror) still apply, and the Docker image's `UV_PYTHON_INSTALL_DIR=/data/python` and `UV_CACHE_DIR=/data/uv-cache` reach uv the same way. Any other variable a repository's build reads is not set. ### Server | Variable | Effect | |---|---| | `PORT`, `HOST` | Address of `staleproof serve`; `--port` and `--host` override them. The Docker image sets `HOST=0.0.0.0` and `PORT=8080`. Not used on Vercel | | `STALEPROOF_PRELOADED` | Comma-separated pull request URLs listed on the home page and by `GET /api/prs`; spaces around entries are trimmed and empty entries dropped. `serve --preload <url...>` overrides it. Unset: falkordb-py #262, #267 and #276. Every entry must be a pull request URL, or `GET /api/prs` answers 500 | | `STALEPROOF_CHECK_BUDGET_MS` | After this many milliseconds, an agent check in the server starts no new model turn and ends `incomplete`, listing the sections still open. Default 240000 on Vercel (whose function limit is 300 s in [vercel.json](../vercel.json)), none elsewhere. `staleproof check` ignores it. When a hosted server indexes first, indexing counts against it, but the agent always gets at least 45 seconds. `0` ends every agent check before its first turn | | `STALEPROOF_HOSTED` | Hosted mode ([Deploy](DEPLOY.md#hosted-mode)). In the Vercel function it is on unless the value is `off`; for `staleproof serve` it is off unless the value is `on` or `--hosted` is given | | `STALEPROOF_AGENT_CHECKS_PER_DAY` | In hosted mode (the Vercel function and `serve --hosted`): how many agent checks start per day (UTC) across the service, default 50 | | `VERCEL` | Any non-empty value. Vercel sets it, which moves `STALEPROOF_HOME` to `/tmp/staleproof` and sets the 240 s check budget | The HTTP rate limit (30 POST requests per client per hour) has no variable; see [HTTP API](http-api.md#rate-limits). The hosted workspace limits have none either; they are `DEFAULT_HOSTED_LIMITS` in [src/hosted/workspaces.ts](../src/hosted/workspaces.ts). ### Scripts and helpers - **`FALKORDBLITE_VERSION`** (default `0.10.0`) is the version of the `falkordblite` Python wheel that `scripts/falkordb-local.sh` installs into `$STALEPROOF_HOME/falkordb/venv` (with uv, else `python3 -m venv` and pip). It provides `redis-server` and the FalkorDB module for `staleproof db start` and for the managed scratch server. That build is FalkorDB 4.18.3 ([version notes](troubleshooting.md#falkordb-version-notes)). - **`FACTS_OUT`** is where `scripts/seed.sh` writes FACTS.md (default `docs/FACTS.md`, relative to the working directory). - **`USER`** is the default for `verdict --by`. ## Global CLI options and precedence | Option | Overrides | Default | |---|---|---| | `--org <org>` | `STALEPROOF_ORG` | the owner of the repository the command is about, else `falkordb` | | `--db <url>` | `FALKORDB_URL` | `falkor://localhost:6379`, started locally when nothing answers | Both may come before or after the command (`staleproof --org acme index ...` and `staleproof index ... --org acme` are the same). The general rule is flag, then environment variable, then default: | Setting | Flag | Variable | Default | |---|---|---|---| | Graph name | `--org` | `STALEPROOF_ORG` | the repository's owner, else `falkordb` | | FalkorDB URL | `--db` | `FALKORDB_URL` | `falkor://localhost:6379` | | Agent model | `check --model` | `STALEPROOF_MODEL` | `claude-sonnet-5-5` | | Effort | `check --effort` | `STALEPROOF_EFFORT` | `medium` | | Server port | `serve --port` | `PORT` | `8080` | | Server interface | `serve --host` | `HOST` | `127.0.0.1` | | Preloaded pull requests | `serve --preload` | `STALEPROOF_PRELOADED` | falkordb-py #262, #267, #276 | | Hosted mode | `serve --hosted` | `STALEPROOF_HOSTED` | off for `serve`, on in the Vercel function | | Reviewer | `verdict --by` | `USER` | `maintainer` | | Local FalkorDB port | `db --port` | — | `6379` | The other limits are flags only: `check --max-steps` (40), `check --no-agent --max-hops` (3) and `--runs` (3). `check --with` and `--no-index` control indexing on first use. Every command and option is in [CLI](cli.md). ## Files under STALEPROOF_HOME ```text ~/.staleproof/ STALEPROOF_HOME repos/<owner>/<name>/ blob-less git clones: indexing, PR ingestion, example installs, the docs patch, clearance's text-match counts tarballs/<owner>/<name>/<sha>/ files extracted from a downloaded tarball (no git); hosted mode deletes them once the repository is indexed cache/github/<owner>/<repo>/pulls/<n>.json pull request metadata (read back only for merged PRs) venvs/<key>/ one virtualenv per set of repo@sha, marked done by .staleproof-ok src/<key>/<owner>__<name>/ sources extracted while a virtualenv builds; removed after a successful build falkordb/venv/ falkordblite, installed on first `db start` or scratch start falkordb/data-<port>/ data (RDB snapshots), redis.log and redis.pid of each local server ``` - **`falkordb/data-6379`** holds the org graph when you run `staleproof db start`, including every verdict, decision and run. Deleting it deletes the graph. `data-6390` is the scratch server, which is flushed before every run anyway. - **`venvs/` and `src/`** can be deleted at any time; virtualenvs are rebuilt on the next run that needs them. - **`repos/`** can be deleted; clones come back on the next index or check. Until then, `clearance` reports 0 raw text matches and the docs patch reads files from GitHub instead. - **`tarballs/`** can be deleted; it is only read while indexing. - **`cache/`** can be deleted; metadata is fetched again. - Each example run also uses a temporary directory, `$TMPDIR/staleproof-run-*`, removed when the run ends. ## Repository configuration A repository can tell the indexer what to leave out, in `.staleproof.toml` at its root: ```toml ignore = ["docs/legacy/**", "CHANGELOG.md"] # never indexed skip-examples = ["docs/v1/**"] # indexed and linked, never run or checked ``` or in the `[tool.staleproof]` table of its root `pyproject.toml`: ```toml [tool.staleproof] ignore = ["docs/legacy/**"] skip-examples = ["docs/v1/**"] ``` | Key | Type | Effect | |---|---|---| | `ignore` | list of globs | Matching files are not indexed at all: no symbols, sections or examples. | | `skip-examples` | list of globs | Examples on matching pages, and matching example scripts, are indexed and linked to the code as usual, so a pull request check still lists them, but they are never run, statically checked, predicted to break or counted as mismatched by the audit (as if each had a `skip` directive). | - **Which file**: `.staleproof.toml` when it exists, even if empty; otherwise `[tool.staleproof]` when it sets either key. Unknown keys are ignored. - **Globs** are matched against repository-relative paths, gitignore style: `**` matches any number of directories, `*` anything within one path segment, `?` one character; a leading `./` or `/` is dropped; a pattern also matches everything under it, so `docs/v1` covers `docs/v1/guide.md` (but not `docs/v10/`). - **When**: read at index time. After editing it, index the repository again. - **Errors** stop the index: a key that is not a list of strings fails with `<file>: ignore must be a list of glob strings` (or `skip-examples`), and a file that is not valid TOML with the parser's message. - **On hosts without git** (tarball downloads: the hosted service, Vercel, `STALEPROOF_INGEST=github`), hidden paths are not extracted except a root `.staleproof.toml`, so both files are read there too. ### Example directives One example at a time is marked in the docs themselves, with a comment on the line before it (blank lines in between are fine) or a word in its fence's info string: | Directive | Means | |---|---| | `<!-- staleproof: skip -->`, `{/* staleproof: skip */}` (MDX), fence word `skip` or `no-run` | Not runnable here (a long-running server, a cloud account): never run or statically checked | | `<!-- staleproof: expect-error -->`, fence word `expect-error` | Fails on purpose (it shows an error): never run or statically checked, because a failure proves nothing | | `<!-- pytest-codeblocks:skip -->`, `<!-- skip: next -->` (Sybil) | Read as `skip`, so docs already marked for those tools need no second marker | | `<!-- pytest-codeblocks:expect-exception -->` | Read as `expect-error` | | `.. staleproof: skip`, `.. staleproof: expect-error`, `.. skip: next` (reStructuredText) | The same, as a comment before the block | A marked example stays linked to the code it calls: a pull request check still lists its section, and its definitions still serve the examples after it on the page. `run_example` answers `Not run: the docs or the repository config mark this example skip.` (or `… failing on purpose (expect-error), so a run proves nothing.`) without running anything. [Indexing](indexing.md#example-directives) has every accepted spelling. ## Configurations that work together ### Local development ```bash npx -y staleproof db start # FalkorDB on 127.0.0.1:6379, data in ~/.staleproof/falkordb/data-6379 export GITHUB_TOKEN=<token> # optional: more than 60 GitHub requests per hour export ANTHROPIC_API_KEY=<key> # optional: agent mode npx -y staleproof index FalkorDB/falkordb-py FalkorDB/GraphRAG-SDK FalkorDB/docs npx -y staleproof check https://github.com/FalkorDB/falkordb-py/pull/262 --no-agent npx -y staleproof serve ``` No other variable is needed: the `FALKORDB_URL` default matches `db start` (and without `db start`, the first command starts the same server), the graph is named after the repositories' owner (`falkordb`), and the scratch server for example runs starts by itself on port 6390. The host needs `git`, plus `uv` or `python3` (uv is recommended: it pins Python 3.12 for the sandbox; with `python3` the version is whatever the host has). ### FalkorDB Cloud and DeepSeek ```bash export FALKORDB_URL='falkor://falkordb:<url-encoded password>@<host>:<port>' # falkors:// if the endpoint has TLS export STALEPROOF_ORG=falkordb export ANTHROPIC_BASE_URL=https://api.deepseek.com/anthropic export ANTHROPIC_API_KEY=<DeepSeek API key> export STALEPROOF_MODEL=deepseek-v4-pro export GITHUB_TOKEN=<token> npx -y staleproof check https://github.com/FalkorDB/falkordb-py/pull/267 ``` The live demo's agent mode uses this provider and model. Leave `STALEPROOF_SCRATCH_URL` unset so example runs use the local scratch server on 127.0.0.1:6390, never the cloud database. `STALEPROOF_FALLBACKS` needs no setting: compatible mode never sends the fallback. ### CI (GitHub Action) The [GitHub Action](github-action.md) needs no variables: it starts FalkorDB in the job and sets `FALKORDB_URL=falkor://localhost:6379` and `STALEPROOF_SCRATCH_URL=falkor://localhost:6390` for its own containers (or lets the CLI start falkordblite when the runner has no Docker). Its inputs set the rest: `github-token` (default: the workflow's token) becomes `GITHUB_TOKEN`, `anthropic-api-key` becomes `ANTHROPIC_API_KEY`, `base-url` becomes `ANTHROPIC_BASE_URL`, `model` becomes `STALEPROOF_MODEL`, and with `falkordb-url` set, `falkordb-url` becomes `FALKORDB_URL` and `org` becomes `STALEPROOF_ORG`. With an empty key the action runs `check --no-agent --format markdown`: ```yaml jobs: docs: runs-on: ubuntu-latest steps: - uses: rajkaria/staleproof@v1 with: anthropic-api-key: ${{ secrets.ANTHROPIC_API_KEY }} base-url: https://api.deepseek.com/anthropic # only for an Anthropic-compatible provider model: deepseek-v4-pro ``` The workflow also needs `permissions: pull-requests: write` to post the comment ([examples/staleproof.yml](../examples/staleproof.yml)). On the runner, `STALEPROOF_HOME` is the default `~/.staleproof`, which starts empty on every job. ### Vercel ```bash vercel env add FALKORDB_URL production # required vercel env add GITHUB_TOKEN production # recommended: a fine-grained token, public repositories, read-only vercel env add ANTHROPIC_API_KEY production # optional: agent mode vercel env add ANTHROPIC_BASE_URL production # with STALEPROOF_MODEL, for an Anthropic-compatible provider vercel env add STALEPROOF_MODEL production vercel env add STALEPROOF_PRELOADED production # optional vercel env add STALEPROOF_AGENT_CHECKS_PER_DAY production # optional, default 50 vercel env add STALEPROOF_HOSTED production # optional: `off` serves the curated graph only ``` Vercel sets `VERCEL`, so `STALEPROOF_HOME` is `/tmp/staleproof` and agent checks stop at 240 s. Hosted mode is on unless `STALEPROOF_HOSTED=off`. `PORT`, `HOST` and the scratch variables are not used: the function serves no static files and executes no examples. [Deploy](DEPLOY.md#vercel) has the full procedure. ## Related - [CLI](cli.md): every command and flag - [Deploy](DEPLOY.md): Vercel, Docker Compose, FalkorDB Cloud and a production checklist - [Example runner](example-runner.md): the sandbox and the scratch server - [The agent](agent.md): model providers, budgets and graph mode - [Troubleshooting and FAQ](troubleshooting.md) --- <!-- page: Graph model and Cypher · Reference · source: https://staleproof.dev/docs/graph-model.md --> # Graph model Generated by `npm run docs` from `src/graph/queries.ts`: every Cypher statement Staleproof sends to FalkorDB, named, with what it is for. One graph per organisation; the graph name is the org slug. ## Indexes - `CREATE INDEX FOR (n:Repo) ON (n.name)` - `CREATE INDEX FOR (n:File) ON (n.id)` - `CREATE INDEX FOR (n:Symbol) ON (n.id)` - `CREATE INDEX FOR (n:Symbol) ON (n.name)` - `CREATE INDEX FOR (n:Symbol) ON (n.repo)` - `CREATE INDEX FOR (n:DocSection) ON (n.id)` - `CREATE INDEX FOR (n:DocSection) ON (n.repo)` - `CREATE INDEX FOR (n:CodeExample) ON (n.id)` - `CREATE INDEX FOR (n:CodeExample) ON (n.repo)` - `CREATE INDEX FOR (n:PullRequest) ON (n.url)` - `CREATE INDEX FOR (n:Decision) ON (n.id)` - `CREATE INDEX FOR (n:Verdict) ON (n.id)` - `CREATE INDEX FOR (n:Verdict) ON (n.check)` - `CREATE INDEX FOR (n:Run) ON (n.id)` - `CREATE INDEX FOR (n:Run) ON (n.key)` - `CREATE INDEX FOR (n:Check) ON (n.id)` ## Agent tools **`impact`** — Every doc section and code example a pull request reaches: changed symbol ← up to 3 callers (any repository) ← doc. The path is the evidence. ```cypher MATCH (pr:PullRequest {url: $pr})<-[c:CHANGED_IN]-(s:Symbol) WHERE c.kind <> 'added' MATCH p = (s)<-[:CALLS*0..3]-(t:Symbol)<-[:MENTIONS|USES]-(d) WHERE (d:DocSection OR d:CodeExample) AND d.status = 'live' OPTIONAL MATCH (d)-[:IN_SECTION]->(sec:DocSection) RETURN s.qualname AS changed, c.kind AS change, c.before AS before, c.after AS after, t.qualname AS reached, labels(d)[0] AS kind, d.id AS doc, d.repo AS repo, d.path AS path, coalesce(sec.id, d.id) AS section, coalesce(sec.heading_path, d.heading_path, d.path) AS heading, d.url AS url, length(p) AS hops, [n IN nodes(p) | coalesce(n.qualname, n.id)] AS evidence, [x IN relationships(p) | coalesce(x.resolved, 'exact')] AS resolution ORDER BY hops, repo, doc ``` **`clearance`** — Proof of absence: changed symbols with no doc section or example within 3 call hops in any repository. ```cypher MATCH (pr:PullRequest {url: $pr})<-[c:CHANGED_IN]-(s:Symbol) WHERE c.kind <> 'added' OPTIONAL MATCH p = (s)<-[:CALLS*0..3]-(:Symbol)<-[:MENTIONS|USES]-() WITH s, c, count(p) AS reach WHERE reach = 0 RETURN s.qualname AS symbol, s.name AS name, s.repo AS repo, c.kind AS change, s.visibility AS visibility ORDER BY symbol ``` **`prChanges`** — What a pull request changed, symbol by symbol. ```cypher MATCH (pr:PullRequest {url: $pr})<-[c:CHANGED_IN]-(s:Symbol) RETURN s.qualname AS symbol, s.kind AS kind, c.kind AS change, c.before AS before, c.after AS after, c.renamed_to AS renamedTo, s.repo AS repo, s.path AS path, s.status AS status ORDER BY change, symbol ``` **`rankSections`** — Order doc sections by how central their page is: PageRank over LINKS_TO, summed per page. ```cypher CALL algo.pageRank('DocSection', 'LINKS_TO') YIELD node, score WITH node.repo + ':' + node.path AS page, node.id AS id, node.heading_path AS heading, score WITH page, sum(score) AS pageScore, collect({id: id, heading: heading, score: score}) AS sections UNWIND sections AS sec WITH page, pageScore, sec WHERE sec.id IN $ids RETURN sec.id AS id, sec.heading AS heading, page, pageScore, sec.score AS score ORDER BY pageScore DESC, score DESC ``` **`rankSectionsByLinks`** — Fallback ranking when graph algorithms are unavailable: inbound links per page. ```cypher MATCH (d:DocSection) WHERE d.id IN $ids OPTIONAL MATCH (other:DocSection {repo: d.repo, path: d.path})<-[l:LINKS_TO]-() RETURN d.id AS id, d.heading_path AS heading, d.repo + ':' + d.path AS page, count(l) AS pageScore, 0 AS score ORDER BY pageScore DESC ``` **`readSection`** — The current text of a doc section, its code examples and its source link. ```cypher MATCH (d:DocSection {id: $id}) OPTIONAL MATCH (e:CodeExample)-[:IN_SECTION]->(d) WITH d, e ORDER BY e.index RETURN d.id AS id, d.repo AS repo, d.path AS path, d.heading_path AS heading, d.body AS body, d.url AS url, d.lineno AS lineno, d.end_lineno AS endLineno, collect(CASE WHEN e IS NULL THEN NULL ELSE {id: e.id, lang: e.lang, code: e.code, lineno: e.lineno} END) AS examples ``` **`exampleForRun`** — A code example, the repositories its calls resolve into, and the earlier examples on its page (for stitching fragments). ```cypher MATCH (e:CodeExample {id: $id}) OPTIONAL MATCH (e)-[:USES]->(s:Symbol) WITH e, collect(DISTINCT s.repo) AS usedRepos OPTIONAL MATCH (prev:CodeExample {repo: e.repo, path: e.path, status: 'live'}) WHERE prev.index < e.index WITH e, usedRepos, prev ORDER BY prev.index RETURN e.id AS id, e.lang AS lang, e.code AS code, e.repo AS repo, e.path AS path, e.url AS url, e.directive AS directive, usedRepos, collect(CASE WHEN prev IS NULL THEN NULL ELSE {id: prev.id, lang: prev.lang, code: prev.code} END) AS earlier ``` **`installSet`** — Repositories to install for a run: the given ones plus their in-org dependencies (DEPENDS_ON), code repositories only. ```cypher MATCH (r:Repo) WHERE r.name IN $repos OPTIONAL MATCH dep = (r)-[:DEPENDS_ON*1..3]->(d:Repo) WITH collect(DISTINCT r) + collect(DISTINCT d) AS repos UNWIND repos AS repo WITH DISTINCT repo WHERE repo IS NOT NULL AND size(repo.packages) > 0 RETURN repo.name AS repo, repo.sha AS sha ORDER BY repo ``` **`decisionsForSection`** — Active decisions governing a doc section directly, or any symbol its prose or examples reach within 2 call hops; superseded decisions resolve to their successor. ```cypher MATCH (d:DocSection {id: $id}) OPTIONAL MATCH (dec0:Decision)-[:GOVERNS]->(d) OPTIONAL MATCH p1 = (d)-[:MENTIONS]->(:Symbol)-[:CALLS*0..2]->(g1:Symbol)<-[:GOVERNS]-(dec1:Decision) OPTIONAL MATCH p2 = (d)<-[:IN_SECTION]-(:CodeExample)-[:USES]->(:Symbol)-[:CALLS*0..2]->(g2:Symbol)<-[:GOVERNS]-(dec2:Decision) WITH collect(DISTINCT {dec: dec0, via: d.heading_path}) + collect(DISTINCT {dec: dec1, via: g1.qualname}) + collect(DISTINCT {dec: dec2, via: g2.qualname}) AS hits UNWIND hits AS hit WITH hit.dec AS dec, hit.via AS via WHERE dec IS NOT NULL OPTIONAL MATCH chain = (newer:Decision {status: 'active'})-[:SUPERSEDES*1..10]->(dec) WITH coalesce(newer, dec) AS current, dec AS matched, via WHERE current.status = 'active' RETURN DISTINCT current.id AS id, current.text AS text, current.source_url AS source, current.author AS author, current.created_at AS createdAt, matched.id AS matched, via ``` **`decisionsForSymbol`** — Active decisions governing a symbol or anything it calls within 2 hops; superseded decisions resolve to their successor. ```cypher MATCH p = (s:Symbol {id: $id})-[:CALLS*0..2]->(g:Symbol)<-[:GOVERNS]-(dec:Decision) OPTIONAL MATCH chain = (newer:Decision {status: 'active'})-[:SUPERSEDES*1..10]->(dec) WITH coalesce(newer, dec) AS current, dec AS matched, g.qualname AS via WHERE current.status = 'active' RETURN DISTINCT current.id AS id, current.text AS text, current.source_url AS source, current.author AS author, current.created_at AS createdAt, matched.id AS matched, via ``` **`sectionReachedByPr`** — Guard for edits: is there a path from this pull request's changes to the section (or one of its examples)? ```cypher MATCH (pr:PullRequest {url: $pr})<-[c:CHANGED_IN]-(s:Symbol) WHERE c.kind <> 'added' MATCH p = (s)<-[:CALLS*0..3]-(:Symbol)<-[:MENTIONS|USES]-(d) WHERE d.id = $section OR (d)-[:IN_SECTION]->(:DocSection {id: $section}) WITH p ORDER BY length(p) LIMIT 1 RETURN [n IN nodes(p) | coalesce(n.qualname, n.id)] AS path ``` **`predictedCalls`** — Examples that call a symbol the pull request removed, renamed, stopped exporting or re-signed, with the call as written (checked against the new signature in code). ```cypher MATCH (pr:PullRequest {url: $pr})<-[c:CHANGED_IN]-(s:Symbol) WHERE c.kind IN ['removed', 'renamed', 'signature', 'export-removed'] MATCH (e:CodeExample)-[u:USES]->(s) WHERE e.status = 'live' AND coalesce(e.directive, '') = '' AND (e.lang = 'python' OR (c.kind <> 'signature' AND e.lang IN ['javascript', 'typescript', 'js', 'ts', 'jsx', 'tsx', 'mjs', 'cjs', 'node'])) OPTIONAL MATCH (e)-[:IN_SECTION]->(sec:DocSection) RETURN e.id AS example, e.code AS code, e.lang AS lang, e.lineno AS lineno, e.url AS url, e.repo AS repo, e.path AS path, coalesce(sec.id, e.id) AS section, u.line AS line, u.text AS text, u.resolved AS resolved, s.qualname AS symbol, s.name AS name, s.kind AS symbolKind, s.decorators AS decorators, c.kind AS change, c.before AS before, c.after AS after, c.renamed_to AS renamedTo ORDER BY example, line ``` **`mismatchesFor`** — Example lines that already did not fit the code when indexed (MISMATCH edges), for the given examples. ```cypher MATCH (e:CodeExample)-[m:MISMATCH]->(s:Symbol) WHERE e.id IN $ids AND e.status = 'live' RETURN e.id AS example, e.url AS url, m.line AS line, m.kind AS kind, m.detail AS detail, m.text AS call ORDER BY example, line ``` **`findSymbols`** — Symbols a name refers to: an id, a qualified name, a dotted suffix (Graph.query) or a bare name. ```cypher MATCH (s:Symbol) WHERE s.status = 'live' AND s.kind <> 'module' AND (s.id = $name OR s.qualname = $name OR s.qualname ENDS WITH $suffix OR s.name = $name) AND (size($repos) = 0 OR s.repo IN $repos) RETURN s.id AS id, s.qualname AS qualname, s.name AS name, s.kind AS kind, s.repo AS repo, s.signature AS signature, s.path AS path, s.lineno AS lineno ORDER BY qualname LIMIT 50 ``` **`docsForSymbol`** — Every doc section and example that reaches a symbol: symbol ← up to 3 callers (any repository) ← doc, with the path. Impact without a pull request. ```cypher MATCH (s:Symbol) WHERE s.id IN $ids MATCH p = (s)<-[:CALLS*0..3]-(t:Symbol)<-[:MENTIONS|USES]-(d) WHERE (d:DocSection OR d:CodeExample) AND d.status = 'live' OPTIONAL MATCH (d)-[:IN_SECTION]->(sec:DocSection) RETURN s.qualname AS changed, 'lookup' AS change, s.signature AS before, s.signature AS after, t.qualname AS reached, labels(d)[0] AS kind, d.id AS doc, d.repo AS repo, d.path AS path, coalesce(sec.id, d.id) AS section, coalesce(sec.heading_path, d.heading_path, d.path) AS heading, d.url AS url, length(p) AS hops, [n IN nodes(p) | coalesce(n.qualname, n.id)] AS evidence, [x IN relationships(p) | coalesce(x.resolved, 'exact')] AS resolution ORDER BY hops, repo, doc ``` **`impactOfChanges`** — Impact of changes that are not a pull request yet (a working tree, an editor's edits): the impact walk from a list of changed symbols. ```cypher UNWIND $changes AS ch MATCH (s:Symbol {id: ch.id}) WHERE ch.kind <> 'added' MATCH p = (s)<-[:CALLS*0..3]-(t:Symbol)<-[:MENTIONS|USES]-(d) WHERE (d:DocSection OR d:CodeExample) AND d.status = 'live' OPTIONAL MATCH (d)-[:IN_SECTION]->(sec:DocSection) RETURN s.qualname AS changed, ch.kind AS change, ch.before AS before, ch.after AS after, t.qualname AS reached, labels(d)[0] AS kind, d.id AS doc, d.repo AS repo, d.path AS path, coalesce(sec.id, d.id) AS section, coalesce(sec.heading_path, d.heading_path, d.path) AS heading, d.url AS url, length(p) AS hops, [n IN nodes(p) | coalesce(n.qualname, n.id)] AS evidence, [x IN relationships(p) | coalesce(x.resolved, 'exact')] AS resolution ORDER BY hops, repo, doc ``` **`predictedCallsOfChanges`** — Examples that call a changed symbol (removed, renamed, stopped exporting, re-signed) in changes that are not a pull request yet. ```cypher UNWIND $changes AS ch MATCH (s:Symbol {id: ch.id}) WHERE ch.kind IN ['removed', 'renamed', 'signature', 'export-removed'] MATCH (e:CodeExample)-[u:USES]->(s) WHERE e.status = 'live' AND coalesce(e.directive, '') = '' AND (e.lang = 'python' OR (ch.kind <> 'signature' AND e.lang IN ['javascript', 'typescript', 'js', 'ts', 'jsx', 'tsx', 'mjs', 'cjs', 'node'])) OPTIONAL MATCH (e)-[:IN_SECTION]->(sec:DocSection) RETURN e.id AS example, e.code AS code, e.lang AS lang, e.lineno AS lineno, e.url AS url, e.repo AS repo, e.path AS path, coalesce(sec.id, e.id) AS section, u.line AS line, u.text AS text, u.resolved AS resolved, s.qualname AS symbol, s.name AS name, s.kind AS symbolKind, s.decorators AS decorators, ch.kind AS change, ch.before AS before, ch.after AS after, ch.renamed_to AS renamedTo ORDER BY example, line ``` **`clearanceOfChanges`** — Proof of absence for changes that are not a pull request yet: changed symbols no doc reaches within 3 call hops. ```cypher UNWIND $changes AS ch MATCH (s:Symbol {id: ch.id}) WHERE ch.kind <> 'added' OPTIONAL MATCH p = (s)<-[:CALLS*0..3]-(:Symbol)<-[:MENTIONS|USES]-() WITH s, ch, count(p) AS reach WHERE reach = 0 RETURN s.qualname AS symbol, s.name AS name, s.repo AS repo, ch.kind AS change, s.visibility AS visibility ORDER BY symbol ``` ## Memory **`cachedRun`** — An identical earlier run (same program, same commits): the graph remembers what already ran. ```cypher MATCH (r:Run {key: $key}) RETURN properties(r) AS props ORDER BY r.created_at DESC LIMIT 1 ``` **`upsertRun`** — Record an example execution: status, error line, traceback, commits installed. ```cypher MATCH (e:CodeExample {id: $example}) MERGE (r:Run {id: $id}) SET r += $props MERGE (r)-[:OF]->(e) ``` **`linkRunToPr`** — Attach a run to the pull request it was evidence for. ```cypher MATCH (r:Run {id: $id}) MATCH (pr:PullRequest {url: $pr}) MERGE (r)-[:AT]->(pr) ``` **`priorVerdicts`** — What Staleproof concluded about this section on earlier pull requests, and what maintainers did with it. ```cypher MATCH (v:Verdict)-[:ABOUT]->(:DocSection {id: $id}) MATCH (v)-[:FOR]->(pr:PullRequest) RETURN v.id AS id, v.verdict AS verdict, v.status AS status, v.reason AS reason, pr.url AS pr, pr.title AS title, v.created_at AS createdAt ORDER BY createdAt DESC LIMIT 5 ``` **`createDecision`** — Record a maintainer decision. ```cypher CREATE (dec:Decision {id: $id}) SET dec += $props RETURN dec.id AS id ``` **`governSections`** — Attach a decision to the doc sections it governs. ```cypher MATCH (dec:Decision {id: $id}) MATCH (d:DocSection) WHERE d.id IN $governs MERGE (dec)-[:GOVERNS]->(d) RETURN count(d) AS n ``` **`governSymbols`** — Attach a decision to the symbols it governs. ```cypher MATCH (dec:Decision {id: $id}) MATCH (s:Symbol) WHERE s.id IN $governs MERGE (dec)-[:GOVERNS]->(s) RETURN count(s) AS n ``` **`supersedeDecision`** — Mark an older decision superseded by a newer one. ```cypher MATCH (newer:Decision {id: $newer}) MATCH (older:Decision {id: $older}) MERGE (newer)-[:SUPERSEDES]->(older) SET older.status = 'superseded' ``` **`createVerdict`** — Record the agent's verdict on one section for one pull request, with the decisions and runs it cites. ```cypher MATCH (d:DocSection {id: $section}) MATCH (pr:PullRequest {url: $pr}) CREATE (v:Verdict {id: $id}) SET v += $props MERGE (v)-[:ABOUT]->(d) MERGE (v)-[:FOR]->(pr) WITH v OPTIONAL MATCH (dec:Decision) WHERE dec.id IN $cites FOREACH (_ IN CASE WHEN dec IS NULL THEN [] ELSE [1] END | MERGE (v)-[:CITES]->(dec)) WITH DISTINCT v OPTIONAL MATCH (r:Run) WHERE r.id IN $runs FOREACH (_ IN CASE WHEN r IS NULL THEN [] ELSE [1] END | MERGE (v)-[:EVIDENCE]->(r)) RETURN DISTINCT v.id AS id ``` **`verdictsForPr`** — The latest verdict per section for a pull request (what the docs patch is built from). ```cypher MATCH (v:Verdict)-[:FOR]->(:PullRequest {url: $pr}) MATCH (v)-[:ABOUT]->(d:DocSection) OPTIONAL MATCH (v)-[:CITES]->(dec:Decision) OPTIONAL MATCH (v)-[:EVIDENCE]->(r:Run) WITH v, d, collect(DISTINCT dec.id) AS cites, collect(DISTINCT {id: r.id, status: r.status, side: r.side, error: r.error}) AS runs ORDER BY v.created_at DESC WITH d, collect({v: v, cites: cites, runs: runs})[0] AS latest RETURN d.id AS section, d.repo AS repo, d.path AS path, d.heading_path AS heading, d.lineno AS lineno, d.end_lineno AS endLineno, d.url AS url, latest.v.id AS id, latest.v.verdict AS verdict, latest.v.status AS status, latest.v.reason AS reason, latest.v.new_text AS newText, latest.v.old_text AS oldText, latest.v.evidence AS evidence, latest.cites AS cites, latest.runs AS runs ORDER BY repo, path, lineno ``` **`reviewVerdict`** — A maintainer accepts or rejects a verdict; precision is measured from these. ```cypher MATCH (v:Verdict {id: $id}) SET v.status = $status, v.reviewed_by = $by, v.reviewed_at = $at RETURN v.id AS id ``` **`sinceLastCheck`** — What changed in the graph since a moment: pull requests checked, verdicts, decisions, runs. ```cypher OPTIONAL MATCH (pr:PullRequest) WHERE pr.ingested_at > $since WITH collect(DISTINCT {url: pr.url, title: pr.title}) AS prs OPTIONAL MATCH (v:Verdict) WHERE v.created_at > $since WITH prs, count(v) AS verdicts, sum(CASE WHEN v.verdict = 'stale' THEN 1 ELSE 0 END) AS stale OPTIONAL MATCH (dec:Decision) WHERE dec.created_at > $since WITH prs, verdicts, stale, collect(DISTINCT {id: dec.id, text: dec.text}) AS decisions OPTIONAL MATCH (r:Run) WHERE r.created_at > $since RETURN prs, verdicts, stale, decisions, count(r) AS runs, sum(CASE WHEN r.status = 'fail' THEN 1 ELSE 0 END) AS failedRuns ``` **`lastCheck`** — When the previous agent check ran (for "since your last run"). ```cypher MATCH (c:Check) WHERE c.status <> 'running' RETURN c.id AS id, c.started_at AS startedAt, c.pr AS pr ORDER BY c.started_at DESC LIMIT 1 ``` **`createCheck`** — Start an agent check of a pull request. ```cypher MATCH (pr:PullRequest {url: $pr}) CREATE (c:Check {id: $id}) SET c += $props MERGE (c)-[:FOR]->(pr) ``` **`finishCheck`** — Close an agent check: status, summary, and the full trace (every tool call with its Cypher). ```cypher MATCH (c:Check {id: $id}) SET c += $props WITH c MATCH (v:Verdict {check: $id}) MERGE (c)-[:PRODUCED]->(v) RETURN count(v) AS verdicts ``` **`sectionsWithVerdict`** — Sections that received a verdict during this check (the stop condition). ```cypher MATCH (v:Verdict {check: $id})-[:ABOUT]->(d:DocSection) RETURN DISTINCT d.id AS id ``` ## Indexing **`upsertPullRequest`** — Record a pull request and forget its previous change set (re-checking is idempotent). ```cypher MERGE (pr:PullRequest {url: $url}) SET pr += $props WITH pr OPTIONAL MATCH (pr)<-[c:CHANGED_IN]-() DELETE c ``` **`upsertMissingSymbols`** — Symbols a PR removed or renamed that the graph (indexed after the PR) no longer has: recreate them as removed. ```cypher UNWIND $rows AS row MERGE (s:Symbol {id: row.id}) ON CREATE SET s += row.props, s.status = 'removed' WITH s, row MERGE (f:File {id: row.file}) ON CREATE SET f.repo = row.props.repo, f.path = row.props.path, f.kind = 'python' MERGE (s)-[:DEFINED_IN]->(f) ``` **`lateLinkSections`** — Link docs that still name a removed symbol (by the identifiers stored on each section), within repositories that document its repository. ```cypher UNWIND $rows AS row MATCH (s:Symbol {id: row.id}) MATCH (d:DocSection {status: 'live'}) WHERE (row.name IN d.idents OR row.dotted IN d.idents) AND (d.repo = s.repo OR (:Repo {name: d.repo})-[:DEPENDS_ON]->(:Repo {name: s.repo})) MERGE (d)-[m:MENTIONS]->(s) ON CREATE SET m.resolved = 'late', m.span = row.name RETURN count(m) AS linked ``` **`lateLinkExamples`** — Link code examples that still use a removed symbol's name. ```cypher UNWIND $rows AS row MATCH (s:Symbol {id: row.id}) MATCH (e:CodeExample {status: 'live'}) WHERE row.name IN e.idents AND (e.repo = s.repo OR (:Repo {name: e.repo})-[:DEPENDS_ON]->(:Repo {name: s.repo})) MERGE (e)-[u:USES]->(s) ON CREATE SET u.resolved = 'late', u.text = row.name RETURN count(u) AS linked ``` **`upsertChangedIn`** — Attach each changed symbol to the pull request with the kind of change. ```cypher UNWIND $rows AS row MATCH (s:Symbol {id: row.id}) MATCH (pr:PullRequest {url: $url}) MERGE (s)-[c:CHANGED_IN]->(pr) SET c.kind = row.kind, c.before = row.before, c.after = row.after, c.renamed_to = row.renamed_to ``` **`upsertRepo`** — Create or refresh a repository node. ```cypher MERGE (r:Repo {name: $name}) SET r += $props ``` **`upsertFiles`** — Create or refresh files and attach them to their repository. ```cypher UNWIND $rows AS row MERGE (f:File {id: row.id}) SET f += row.props, f.seen = $run WITH f, row MATCH (r:Repo {name: row.repo}) MERGE (f)-[:IN_REPO]->(r) ``` **`upsertSymbols`** — Create or refresh symbols (modules, classes, functions, methods) in their files. ```cypher UNWIND $rows AS row MERGE (s:Symbol {id: row.id}) SET s += row.props, s.status = 'live', s.seen = $run WITH s, row MATCH (f:File {id: row.file}) MERGE (s)-[:DEFINED_IN]->(f) ``` **`deleteRepoCalls`** — Drop the outgoing call edges of a repository before re-resolving them. ```cypher MATCH (s:Symbol {repo: $repo})-[c:CALLS]->() DELETE c ``` **`upsertCalls`** — Write resolved call edges (exact = through imports and types; by-name = name match in an imported repository). ```cypher UNWIND $rows AS row MATCH (a:Symbol {id: row.from}) MATCH (b:Symbol {id: row.to}) MERGE (a)-[c:CALLS]->(b) SET c.resolved = row.resolved, c.line = row.line, c.text = row.text ``` **`markGoneSymbols`** — Symbols no longer in the repository are kept (verdicts and decisions point at them) but marked gone. ```cypher MATCH (s:Symbol {repo: $repo}) WHERE s.seen <> $run AND s.status = 'live' SET s.status = 'gone' RETURN count(s) AS gone ``` **`upsertDependsOn`** — Record that one repository depends on another (from pyproject/requirements). ```cypher UNWIND $rows AS row MATCH (a:Repo {name: row.from}) MATCH (b:Repo {name: row.to}) MERGE (a)-[d:DEPENDS_ON]->(b) SET d.spec = row.spec ``` **`upsertSections`** — Create or refresh doc sections (one per heading) with their text and identifiers. ```cypher UNWIND $rows AS row MERGE (d:DocSection {id: row.id}) SET d += row.props, d.status = 'live', d.seen = $run WITH d, row MATCH (f:File {id: row.file}) MERGE (d)-[:DEFINED_IN]->(f) ``` **`upsertExamples`** — Create or refresh code examples (fenced blocks and example scripts) and place them in their section. ```cypher UNWIND $rows AS row MERGE (e:CodeExample {id: row.id}) SET e += row.props, e.status = 'live', e.seen = $run WITH e, row MATCH (f:File {id: row.file}) MERGE (e)-[:DEFINED_IN]->(f) WITH e, row OPTIONAL MATCH (d:DocSection {id: row.section}) FOREACH (_ IN CASE WHEN d IS NULL THEN [] ELSE [1] END | MERGE (e)-[:IN_SECTION]->(d)) ``` **`deleteRepoDocEdges`** — Drop a docs repository's MENTIONS/USES/LINKS_TO/MISMATCH edges before re-linking. ```cypher MATCH (d {repo: $repo})-[r:MENTIONS|USES|LINKS_TO|MISMATCH]->() WHERE d:DocSection OR d:CodeExample DELETE r ``` **`upsertMentions`** — Link a doc section to a symbol its prose names in inline code. ```cypher UNWIND $rows AS row MATCH (d:DocSection {id: row.from}) MATCH (s:Symbol {id: row.to}) MERGE (d)-[m:MENTIONS]->(s) SET m.span = row.span, m.resolved = row.resolved ``` **`upsertUses`** — Link a code example to a symbol it calls (resolved like code: imports, then types). ```cypher UNWIND $rows AS row MATCH (e:CodeExample {id: row.from}) MATCH (s:Symbol {id: row.to}) MERGE (e)-[u:USES]->(s) SET u.line = row.line, u.resolved = row.resolved, u.text = row.text ``` **`upsertMismatches`** — Record an example line that no longer fits the code it calls: an unknown keyword, a wrong argument count, a name that does not exist. ```cypher UNWIND $rows AS row MATCH (e:CodeExample {id: row.from}) MATCH (s:Symbol {id: row.to}) MERGE (e)-[m:MISMATCH {kind: row.kind, name: row.name, detail: row.detail}]->(s) SET m.line = row.line, m.text = row.text ``` **`upsertLinks`** — Link doc sections that hyperlink to each other (input to PageRank). ```cypher UNWIND $rows AS row MATCH (a:DocSection {id: row.from}) MATCH (b:DocSection {id: row.to}) MERGE (a)-[:LINKS_TO]->(b) ``` **`resetPullRequest`** — An open pull request moved (new pushes): drop its old symbol diff so it is ingested again. ```cypher MATCH (pr:PullRequest {url: $url}) SET pr.ingested_at = null WITH pr MATCH (pr)<-[c:CHANGED_IN]-() DELETE c ``` **`markGoneDocs`** — Doc sections and examples no longer present are kept but marked gone. ```cypher MATCH (d {repo: $repo}) WHERE (d:DocSection OR d:CodeExample) AND d.seen <> $run AND d.status = 'live' SET d.status = 'gone' RETURN count(d) AS gone ``` ## Reporting **`recentChecks`** — Agent checks, newest first. ```cypher MATCH (c:Check)-[:FOR]->(pr:PullRequest) RETURN c.id AS id, pr.url AS pr, pr.title AS title, c.status AS status, c.model AS model, c.started_at AS startedAt, c.finished_at AS finishedAt, c.steps AS steps, c.tool_calls AS toolCalls, c.verdicts AS verdicts, c.summary AS summary, c.tool_order AS toolOrder ORDER BY startedAt DESC LIMIT $limit ``` **`checkTrace`** — The stored trace of one agent check. ```cypher MATCH (c:Check {id: $id}) RETURN c.trace AS trace ``` **`auditRepos`** — The repositories a docs audit covers, with the commit each was indexed at. ```cypher MATCH (r:Repo) WHERE size($repos) = 0 OR r.name IN $repos OPTIONAL MATCH (s:Symbol {repo: r.name}) WITH r, count(s) AS symbols OPTIONAL MATCH (d:DocSection {repo: r.name}) WHERE d.status = 'live' RETURN r.name AS name, r.sha AS sha, r.indexed_at AS indexedAt, coalesce(r.index_version, 0) AS indexVersion, symbols, count(d) AS sections ORDER BY name ``` **`auditMismatches`** — Every example line that does not fit the code it calls, in the audited repositories' docs or against their code. ```cypher MATCH (e:CodeExample)-[m:MISMATCH]->(s:Symbol) WHERE e.status = 'live' AND (e.repo IN $repos OR s.repo IN $repos) OPTIONAL MATCH (e)-[:IN_SECTION]->(sec:DocSection) RETURN e.id AS example, e.url AS url, e.repo AS repo, e.path AS path, coalesce(sec.heading_path, e.path) AS heading, m.line AS line, m.kind AS kind, m.detail AS detail, m.text AS call, s.qualname AS symbol, s.repo AS symbolRepo ORDER BY repo, path, line ``` **`auditSymbols`** — Every live symbol of the audited code repositories with how many docs reference it directly and how many symbols call it (API coverage). ```cypher MATCH (s:Symbol) WHERE s.repo IN $repos AND s.status = 'live' AND s.kind <> 'module' OPTIONAL MATCH (s)<-[:MENTIONS|USES]-(d) WHERE d.status = 'live' WITH s, count(DISTINCT d) AS docs OPTIONAL MATCH (s)<-[:CALLS]-(caller:Symbol) WITH s, docs, collect(DISTINCT caller.repo) AS callerRepos, count(DISTINCT caller) AS callers RETURN s.id AS id, s.qualname AS qualname, s.name AS name, s.kind AS kind, s.repo AS repo, s.path AS path, s.lineno AS lineno, s.signature AS signature, s.visibility AS visibility, s.exported AS exported, docs, callers, callerRepos ``` **`auditExamples`** — The live examples in the audited repositories' docs, and examples elsewhere that call their code, with how many symbols each calls. ```cypher MATCH (e:CodeExample) WHERE e.status = 'live' AND e.repo IN $repos OPTIONAL MATCH (e)-[u:USES]->(:Symbol) RETURN e.id AS id, e.lang AS lang, e.url AS url, e.repo AS repo, coalesce(e.directive, '') AS directive, count(u) AS uses UNION MATCH (e:CodeExample)-[:USES]->(s:Symbol) WHERE e.status = 'live' AND s.repo IN $repos AND NOT e.repo IN $repos OPTIONAL MATCH (e)-[u:USES]->(:Symbol) RETURN e.id AS id, e.lang AS lang, e.url AS url, e.repo AS repo, coalesce(e.directive, '') AS directive, count(u) AS uses ``` **`auditRuns`** — Recorded runs of the audited examples at the indexed commits (not before/after a pull request). ```cypher MATCH (r:Run)-[:OF]->(e:CodeExample) WHERE e.id IN $ids AND r.side = 'current' RETURN e.id AS example, e.url AS url, r.status AS status, r.error AS error, r.reason AS reason, r.installed AS installed, r.created_at AS at ORDER BY at ASC ``` **`repoSymbols`** — List the live symbols of a repository. ```cypher MATCH (s:Symbol {repo: $repo, status: 'live'}) RETURN s.id AS id, s.qualname AS qualname, s.kind AS kind, s.signature AS signature ORDER BY s.qualname ``` **`callsFrom`** — Outgoing calls of one symbol. ```cypher MATCH (a:Symbol {id: $id})-[c:CALLS]->(b:Symbol) RETURN b.id AS to, c.resolved AS resolved, c.line AS line ORDER BY b.id ``` **`graphCounts`** — Live node counts for FACTS.md (symbols and docs a re-index marked gone, or a PR removed, are not counted). ```cypher MATCH (n) WHERE NOT (n:Symbol OR n:DocSection OR n:CodeExample) OR n.status = 'live' RETURN labels(n)[0] AS label, count(n) AS n ORDER BY label ``` **`edgeCounts`** — Edge counts by type for FACTS.md. ```cypher MATCH ()-[r]->() RETURN type(r) AS type, count(r) AS n ORDER BY type ``` **`factsRepos`** — Per-repository counts for FACTS.md. ```cypher MATCH (r:Repo) OPTIONAL MATCH (s:Symbol {repo: r.name, status: 'live'}) WITH r, count(s) AS symbols OPTIONAL MATCH (d:DocSection {repo: r.name, status: 'live'}) WITH r, symbols, count(d) AS sections OPTIONAL MATCH (e:CodeExample {repo: r.name, status: 'live'}) RETURN r.name AS name, r.sha AS sha, r.packages AS packages, symbols, sections, count(e) AS examples, sum(CASE WHEN e.lang = 'python' THEN 1 ELSE 0 END) AS pythonExamples ORDER BY name ``` **`factsCalls`** — How CALLS edges were resolved, and how many cross repositories. ```cypher MATCH (a:Symbol)-[c:CALLS]->(b:Symbol) RETURN sum(CASE WHEN c.resolved = 'exact' THEN 1 ELSE 0 END) AS exact, sum(CASE WHEN c.resolved = 'by-name' THEN 1 ELSE 0 END) AS byName, sum(CASE WHEN a.repo <> b.repo THEN 1 ELSE 0 END) AS crossRepo ``` **`factsUses`** — How USES edges (example → symbol) were resolved. ```cypher MATCH (:CodeExample)-[u:USES]->(:Symbol) RETURN sum(CASE WHEN u.resolved = 'exact' THEN 1 ELSE 0 END) AS exact, sum(CASE WHEN u.resolved = 'by-name' THEN 1 ELSE 0 END) AS byName ``` **`factsDependsOn`** — Repository dependency edges. ```cypher MATCH (a:Repo)-[d:DEPENDS_ON]->(b:Repo) RETURN a.name AS from, b.name AS to, d.spec AS spec ORDER BY from, to ``` **`factsCrossRepoDocPaths`** — Symbol → doc pairs connected through ≤ 3 call hops where the doc lives in another repository. ```cypher MATCH (s:Symbol {status: 'live'}) MATCH p = (s)<-[:CALLS*0..3]-(:Symbol)<-[:MENTIONS|USES]-(d) WHERE (d:DocSection OR d:CodeExample) AND s.repo <> d.repo RETURN count(DISTINCT s.id + '|' + d.id) AS n ``` **`factsSamplePaths`** — For public symbols, the longest path where a call crosses a repository boundary before reaching a doc. ```cypher MATCH (s:Symbol {status: 'live', visibility: 'public'}) WHERE NOT s.qualname CONTAINS '._' MATCH p = (s)<-[:CALLS*1..3]-(t:Symbol)<-[:MENTIONS|USES]-(d) WHERE (d:DocSection OR d:CodeExample) AND s.repo <> t.repo WITH s, d, p, [n IN nodes(p) | coalesce(n.qualname, n.id)] AS names ORDER BY length(p) DESC, d.id, reduce(k = '', x IN names | k + x + ' ') WITH s, collect({doc: d.id, hops: length(p), path: names})[0] AS best RETURN s.qualname AS symbol, best.doc AS doc, best.hops AS hops, best.path AS path ORDER BY hops DESC, symbol LIMIT 5 ``` **`factsPullRequests`** — Pull requests checked so far, with how many symbols they changed and how many doc sections those reach. ```cypher MATCH (pr:PullRequest) OPTIONAL MATCH (pr)<-[c:CHANGED_IN]-(s:Symbol) WHERE c.kind <> 'added' WITH pr, collect(s) AS changed OPTIONAL MATCH p = (pr)<-[c2:CHANGED_IN]-(s2:Symbol)<-[:CALLS*0..3]-(:Symbol)<-[:MENTIONS|USES]-(d) WHERE c2.kind <> 'added' AND (d:DocSection OR d:CodeExample) AND d.status = 'live' OPTIONAL MATCH (d)-[:IN_SECTION]->(sec:DocSection) WITH pr, changed, collect(DISTINCT coalesce(sec.id, d.id)) AS sections, collect(DISTINCT CASE WHEN d.repo <> pr.repo THEN coalesce(sec.id, d.id) END) AS elsewhere RETURN pr.url AS url, pr.title AS title, size(changed) AS changed, size(sections) AS sections, size(elsewhere) AS otherRepos ORDER BY url ``` **`factsGrepBaseline`** — Per pull request: the names of the symbols it changed and the doc sections the impact traversal reaches from them (the grep baseline in FACTS compares the two). ```cypher MATCH (pr:PullRequest)<-[c:CHANGED_IN]-(s:Symbol) WHERE c.kind <> 'added' WITH pr, collect(DISTINCT s.name) AS names OPTIONAL MATCH p = (pr)<-[c2:CHANGED_IN]-(:Symbol)<-[:CALLS*0..3]-(:Symbol)<-[:MENTIONS|USES]-(d) WHERE c2.kind <> 'added' AND (d:DocSection OR d:CodeExample) AND d.status = 'live' OPTIONAL MATCH (d)-[:IN_SECTION]->(sec:DocSection) RETURN pr.url AS url, names, collect(DISTINCT coalesce(sec.id, d.id)) AS reached ORDER BY url ``` **`factsSectionBodies`** — The text of every live doc section, for the grep baseline. ```cypher MATCH (d:DocSection) WHERE d.status = 'live' RETURN d.id AS id, d.body AS body ``` **`factsRuns`** — Every recorded example execution (the latest attempt of each identical run), latest first per example and side. ```cypher MATCH (r:Run)-[:OF]->(:CodeExample) WITH r.key AS key, max(r.created_at) AS latest MATCH (r:Run {key: key})-[:OF]->(e:CodeExample) WHERE r.created_at = latest OPTIONAL MATCH (r)-[:AT]->(pr:PullRequest) RETURN e.id AS example, e.url AS url, r.side AS side, r.status AS status, r.error AS error, r.reason AS reason, r.installed AS installed, pr.url AS pr, r.created_at AS at ORDER BY example, at DESC ``` **`impactFromSymbol`** — Docs reachable from one symbol (FACTS timing; the PR-scoped version is the impact tool). ```cypher MATCH (s:Symbol {qualname: $qualname}) MATCH p = (s)<-[:CALLS*0..3]-(t:Symbol)<-[:MENTIONS|USES]-(d) WHERE d:DocSection OR d:CodeExample RETURN d.id AS doc, length(p) AS hops ``` --- <!-- page: Facts from the graph · Reference · source: https://staleproof.dev/docs/facts.md --> # FACTS Generated from the `falkordb` graph by `staleproof facts` at 2026-10-16T17:19:38.971Z (FalkorDB 4.22.0). Every number in the README comes from this file. Regenerate it; never edit it by hand. ## Repositories | Repository | Commit | Symbols | Doc sections | Code examples (Python) | |---|---|---:|---:|---:| | [FalkorDB/GraphRAG-SDK](https://github.com/FalkorDB/GraphRAG-SDK) | `35e67a0` | 1016 | 513 | 313 (252) | | [FalkorDB/docs](https://github.com/FalkorDB/docs) | `f804fd7` | 0 | 2618 | 1517 (165) | | [FalkorDB/falkordb-py](https://github.com/FalkorDB/falkordb-py) | `31f63f4` | 282 | 7 | 5 (2) | ## Graph | Nodes (live) | Count | |---|---:| | Check | 13 | | CodeExample | 1835 | | DocSection | 3138 | | File | 329 | | PullRequest | 4 | | Repo | 3 | | Run | 57 | | Symbol | 1298 | | Verdict | 70 | | Edges | Count | |---|---:| | ABOUT | 70 | | AT | 57 | | CALLS | 1708 | | CHANGED_IN | 9 | | DEFINED_IN | 6271 | | DEPENDS_ON | 3 | | EVIDENCE | 756 | | FOR | 83 | | IN_REPO | 329 | | IN_SECTION | 1824 | | LINKS_TO | 644 | | MENTIONS | 594 | | OF | 57 | | PRODUCED | 70 | | USES | 716 | ## Resolution quality - CALLS edges: 1708 (1655 exact through imports and types, 53 by name inside an imported repository) - CALLS edges that cross a repository boundary: 5 - USES edges from code examples: 716 (633 exact, 83 by name) - Symbol → doc paths (≤ 3 call hops) that cross a repository boundary: 1257 ## Repository dependencies - FalkorDB/GraphRAG-SDK → FalkorDB/falkordb-py (declares falkordb) - FalkorDB/docs → FalkorDB/GraphRAG-SDK (examples import graphrag_sdk) - FalkorDB/docs → FalkorDB/falkordb-py (examples import falkordb) ## Sample cross-repository paths - 4 hops: `falkordb.asyncio.falkordb.FalkorDB` ← `graphrag_sdk.core.connection.FalkorDBConnection._ensure_client` ← `graphrag_sdk.core.connection.FalkorDBConnection.query` ← `graphrag_sdk.storage.vector_store.VectorStore.backfill_entity_embeddings` ← `FalkorDB/GraphRAG-SDK:docs/api-reference.mdx#finalize` - 4 hops: `falkordb.asyncio.falkordb.FalkorDB.select_graph` ← `graphrag_sdk.core.connection.FalkorDBConnection._ensure_client` ← `graphrag_sdk.core.connection.FalkorDBConnection.query` ← `graphrag_sdk.storage.vector_store.VectorStore.backfill_entity_embeddings` ← `FalkorDB/GraphRAG-SDK:docs/api-reference.mdx#finalize` - 4 hops: `falkordb.asyncio.graph.AsyncGraph.query` ← `graphrag_sdk.core.connection.FalkorDBConnection.query` ← `graphrag_sdk.storage.graph_store.GraphStore.delete_document_chunks` ← `graphrag_sdk.api.main.GraphRAG.delete_document` ← `FalkorDB/GraphRAG-SDK:README.md#examples` ## Pull requests checked | Pull request | Changed symbols | Doc sections reached (≤ 3 hops) | …in other repositories | |---|---:|---:|---:| | [fix(ingestion): preserve document structure on update](https://github.com/FalkorDB/GraphRAG-SDK/pull/336) | 2 | 73 | 3 | | [fix: only forward accepted kwargs from Is_Cluster to redis.Redis](https://github.com/FalkorDB/falkordb-py/pull/262) | 1 | 1 | 0 | | [test: assert the client parses execution plans, and fix a plan parser bug](https://github.com/FalkorDB/falkordb-py/pull/267) | 2 | 3 | 2 | | [fix(asyncio): prevent connection pool kwargs mutation and await schema refresh](https://github.com/FalkorDB/falkordb-py/pull/276) | 4 | 43 | 41 | ## Without the graph: grep for the changed names For each pull request, a whole-word search of every live doc section for the names of the symbols it changed (what a reviewer can do without the graph), next to the sections the impact traversal reaches. | Pull request | Changed names | Reached by the graph | …that grep also finds | grep matches outside them | |---|---|---:|---:|---:| | FalkorDB/GraphRAG-SDK/pull/336 | `update`, `run` | 73 | 30 | 219 | | FalkorDB/falkordb-py/pull/262 | `Is_Cluster` | 1 | 0 | 0 | | FalkorDB/falkordb-py/pull/267 | `_operation_tree`, `Is_Cluster` | 3 | 0 | 0 | | FalkorDB/falkordb-py/pull/276 | `Cluster_Conn`, `_query`, `__init__` | 43 | 0 | 8 | Total: the graph reaches 120 sections; grep finds 30 of them and matches 227 others. ## Example runs (sandbox) 51 recorded runs: 5 skipped, 6 error, 24 pass, 16 fail. | Example | Side | Status | Error / reason | Installed | |---|---|---|---|---| | [FalkorDB/GraphRAG-SDK:docs/api-reference.mdx:52](https://github.com/FalkorDB/GraphRAG-SDK/blob/35e67a0b27fdcc7f2c28508b09e86169851a8187/docs/api-reference.mdx#L716) | before of FalkorDB/GraphRAG-SDK/pull/336 | skipped | reads a local file the docs do not provide (employees.csv) | FalkorDB/GraphRAG-SDK@d703d4a, FalkorDB/falkordb-py@31f63f4 | | [FalkorDB/GraphRAG-SDK:docs/graph-schema.mdx:7](https://github.com/FalkorDB/GraphRAG-SDK/blob/35e67a0b27fdcc7f2c28508b09e86169851a8187/docs/graph-schema.mdx#L267) | after of FalkorDB/falkordb-py/pull/276 | error | dependency conflict: graphrag-sdk==1.4.0 requires falkordb[hiredis]>=1.8.0,<2, which the other checkouts don't satisfy ( | FalkorDB/GraphRAG-SDK@35e67a0, FalkorDB/falkordb-py@2a970dc | | [FalkorDB/GraphRAG-SDK:docs/graph-schema.mdx:7](https://github.com/FalkorDB/GraphRAG-SDK/blob/35e67a0b27fdcc7f2c28508b09e86169851a8187/docs/graph-schema.mdx#L267) | before of FalkorDB/falkordb-py/pull/276 | error | dependency conflict: graphrag-sdk==1.4.0 requires falkordb[hiredis]>=1.8.0,<2, which the other checkouts don't satisfy ( | FalkorDB/GraphRAG-SDK@35e67a0, FalkorDB/falkordb-py@e307df3 | | [FalkorDB/GraphRAG-SDK:docs/graph-schema.mdx:8](https://github.com/FalkorDB/GraphRAG-SDK/blob/35e67a0b27fdcc7f2c28508b09e86169851a8187/docs/graph-schema.mdx#L276) | after of FalkorDB/falkordb-py/pull/276 | error | dependency conflict: graphrag-sdk==1.4.0 requires falkordb[hiredis]>=1.8.0,<2, which the other checkouts don't satisfy ( | FalkorDB/GraphRAG-SDK@35e67a0, FalkorDB/falkordb-py@2a970dc | | [FalkorDB/GraphRAG-SDK:docs/graph-schema.mdx:8](https://github.com/FalkorDB/GraphRAG-SDK/blob/35e67a0b27fdcc7f2c28508b09e86169851a8187/docs/graph-schema.mdx#L276) | before of FalkorDB/falkordb-py/pull/276 | error | dependency conflict: graphrag-sdk==1.4.0 requires falkordb[hiredis]>=1.8.0,<2, which the other checkouts don't satisfy ( | FalkorDB/GraphRAG-SDK@35e67a0, FalkorDB/falkordb-py@e307df3 | | [FalkorDB/GraphRAG-SDK:docs/storage.mdx:16](https://github.com/FalkorDB/GraphRAG-SDK/blob/35e67a0b27fdcc7f2c28508b09e86169851a8187/docs/storage.mdx#L314) | after of FalkorDB/falkordb-py/pull/276 | error | dependency conflict: graphrag-sdk==1.4.0 requires falkordb[hiredis]>=1.8.0,<2, which the other checkouts don't satisfy ( | FalkorDB/GraphRAG-SDK@35e67a0, FalkorDB/falkordb-py@2a970dc | | [FalkorDB/GraphRAG-SDK:docs/storage.mdx:16](https://github.com/FalkorDB/GraphRAG-SDK/blob/35e67a0b27fdcc7f2c28508b09e86169851a8187/docs/storage.mdx#L314) | before of FalkorDB/falkordb-py/pull/276 | error | dependency conflict: graphrag-sdk==1.4.0 requires falkordb[hiredis]>=1.8.0,<2, which the other checkouts don't satisfy ( | FalkorDB/GraphRAG-SDK@35e67a0, FalkorDB/falkordb-py@e307df3 | | [FalkorDB/GraphRAG-SDK:graphrag_sdk/examples/07_incremental_updates.py:0](https://github.com/FalkorDB/GraphRAG-SDK/blob/35e67a0b27fdcc7f2c28508b09e86169851a8187/graphrag_sdk/examples/07_incremental_updates.py#L1) | before of FalkorDB/GraphRAG-SDK/pull/336 | skipped | needs an LLM provider (OPENAI_API_KEY); Staleproof does not hold provider keys | FalkorDB/GraphRAG-SDK@d703d4a, FalkorDB/falkordb-py@31f63f4 | | [FalkorDB/docs:agentic-memory/graphiti-mcp-server.mdx:19](https://github.com/FalkorDB/docs/blob/f804fd70f915fbd6f2651b56088739c0227c36fc/agentic-memory/graphiti-mcp-server.mdx#L384) | after of FalkorDB/falkordb-py/pull/276 | pass | | FalkorDB/falkordb-py@2a970dc | | [FalkorDB/docs:agentic-memory/graphiti-mcp-server.mdx:19](https://github.com/FalkorDB/docs/blob/f804fd70f915fbd6f2651b56088739c0227c36fc/agentic-memory/graphiti-mcp-server.mdx#L384) | before of FalkorDB/falkordb-py/pull/276 | pass | | FalkorDB/falkordb-py@e307df3 | | [FalkorDB/docs:commands/graph.config-get.mdx:0](https://github.com/FalkorDB/docs/blob/f804fd70f915fbd6f2651b56088739c0227c36fc/commands/graph.config-get.mdx#L16) | after of FalkorDB/falkordb-py/pull/276 | pass | | FalkorDB/falkordb-py@2a970dc | | [FalkorDB/docs:commands/graph.config-get.mdx:0](https://github.com/FalkorDB/docs/blob/f804fd70f915fbd6f2651b56088739c0227c36fc/commands/graph.config-get.mdx#L16) | before of FalkorDB/falkordb-py/pull/276 | pass | | FalkorDB/falkordb-py@e307df3 | | [FalkorDB/docs:commands/graph.config-set.mdx:1](https://github.com/FalkorDB/docs/blob/f804fd70f915fbd6f2651b56088739c0227c36fc/commands/graph.config-set.mdx#L24) | after of FalkorDB/falkordb-py/pull/276 | pass | | FalkorDB/falkordb-py@2a970dc | | [FalkorDB/docs:commands/graph.config-set.mdx:1](https://github.com/FalkorDB/docs/blob/f804fd70f915fbd6f2651b56088739c0227c36fc/commands/graph.config-set.mdx#L24) | before of FalkorDB/falkordb-py/pull/276 | pass | | FalkorDB/falkordb-py@e307df3 | | [FalkorDB/docs:commands/graph.constraint-create.mdx:3](https://github.com/FalkorDB/docs/blob/f804fd70f915fbd6f2651b56088739c0227c36fc/commands/graph.constraint-create.mdx#L119) | after of FalkorDB/falkordb-py/pull/276 | fail | AttributeError: 'FalkorDB' object has no attribute 'create_constraint' | FalkorDB/falkordb-py@2a970dc | | [FalkorDB/docs:commands/graph.constraint-create.mdx:3](https://github.com/FalkorDB/docs/blob/f804fd70f915fbd6f2651b56088739c0227c36fc/commands/graph.constraint-create.mdx#L119) | before of FalkorDB/falkordb-py/pull/276 | fail | AttributeError: 'FalkorDB' object has no attribute 'create_constraint' | FalkorDB/falkordb-py@e307df3 | | [FalkorDB/docs:commands/graph.constraint-drop.mdx:1](https://github.com/FalkorDB/docs/blob/f804fd70f915fbd6f2651b56088739c0227c36fc/commands/graph.constraint-drop.mdx#L54) | after of FalkorDB/falkordb-py/pull/276 | fail | AttributeError: 'FalkorDB' object has no attribute 'drop_constraint' | FalkorDB/falkordb-py@2a970dc | | [FalkorDB/docs:commands/graph.constraint-drop.mdx:1](https://github.com/FalkorDB/docs/blob/f804fd70f915fbd6f2651b56088739c0227c36fc/commands/graph.constraint-drop.mdx#L54) | before of FalkorDB/falkordb-py/pull/276 | fail | AttributeError: 'FalkorDB' object has no attribute 'drop_constraint' | FalkorDB/falkordb-py@e307df3 | | [FalkorDB/docs:commands/graph.explain.mdx:0](https://github.com/FalkorDB/docs/blob/f804fd70f915fbd6f2651b56088739c0227c36fc/commands/graph.explain.mdx#L17) | after of FalkorDB/falkordb-py/pull/276 | fail | redis.exceptions.ResponseError: Invalid graph operation on empty key | FalkorDB/falkordb-py@2a970dc | | [FalkorDB/docs:commands/graph.explain.mdx:0](https://github.com/FalkorDB/docs/blob/f804fd70f915fbd6f2651b56088739c0227c36fc/commands/graph.explain.mdx#L17) | before of FalkorDB/falkordb-py/pull/276 | fail | redis.exceptions.ResponseError: Invalid graph operation on empty key | FalkorDB/falkordb-py@e307df3 | | [FalkorDB/docs:commands/graph.explain.mdx:0](https://github.com/FalkorDB/docs/blob/f804fd70f915fbd6f2651b56088739c0227c36fc/commands/graph.explain.mdx#L17) | after of FalkorDB/falkordb-py/pull/267 | fail | redis.exceptions.ResponseError: Invalid graph operation on empty key | FalkorDB/falkordb-py@a9c7510 | | [FalkorDB/docs:commands/graph.explain.mdx:0](https://github.com/FalkorDB/docs/blob/f804fd70f915fbd6f2651b56088739c0227c36fc/commands/graph.explain.mdx#L17) | before of FalkorDB/falkordb-py/pull/267 | fail | redis.exceptions.ResponseError: Invalid graph operation on empty key | FalkorDB/falkordb-py@031eb04 | | [FalkorDB/docs:commands/graph.list.mdx:0](https://github.com/FalkorDB/docs/blob/f804fd70f915fbd6f2651b56088739c0227c36fc/commands/graph.list.mdx#L14) | after of FalkorDB/falkordb-py/pull/276 | pass | | FalkorDB/falkordb-py@2a970dc | | [FalkorDB/docs:commands/graph.list.mdx:0](https://github.com/FalkorDB/docs/blob/f804fd70f915fbd6f2651b56088739c0227c36fc/commands/graph.list.mdx#L14) | before of FalkorDB/falkordb-py/pull/276 | pass | | FalkorDB/falkordb-py@e307df3 | | [FalkorDB/docs:commands/graph.profile.mdx:0](https://github.com/FalkorDB/docs/blob/f804fd70f915fbd6f2651b56088739c0227c36fc/commands/graph.profile.mdx#L22) | after of FalkorDB/falkordb-py/pull/276 | pass | | FalkorDB/falkordb-py@2a970dc | | [FalkorDB/docs:commands/graph.profile.mdx:0](https://github.com/FalkorDB/docs/blob/f804fd70f915fbd6f2651b56088739c0227c36fc/commands/graph.profile.mdx#L22) | before of FalkorDB/falkordb-py/pull/276 | pass | | FalkorDB/falkordb-py@e307df3 | | [FalkorDB/docs:commands/graph.profile.mdx:0](https://github.com/FalkorDB/docs/blob/f804fd70f915fbd6f2651b56088739c0227c36fc/commands/graph.profile.mdx#L22) | after of FalkorDB/falkordb-py/pull/267 | pass | | FalkorDB/falkordb-py@a9c7510 | | [FalkorDB/docs:commands/graph.profile.mdx:0](https://github.com/FalkorDB/docs/blob/f804fd70f915fbd6f2651b56088739c0227c36fc/commands/graph.profile.mdx#L22) | before of FalkorDB/falkordb-py/pull/267 | pass | | FalkorDB/falkordb-py@031eb04 | | [FalkorDB/docs:commands/graph.slowlog.mdx:0](https://github.com/FalkorDB/docs/blob/f804fd70f915fbd6f2651b56088739c0227c36fc/commands/graph.slowlog.mdx#L26) | after of FalkorDB/falkordb-py/pull/276 | fail | redis.exceptions.ResponseError: Invalid graph operation on empty key | FalkorDB/falkordb-py@2a970dc | | [FalkorDB/docs:commands/graph.slowlog.mdx:0](https://github.com/FalkorDB/docs/blob/f804fd70f915fbd6f2651b56088739c0227c36fc/commands/graph.slowlog.mdx#L26) | before of FalkorDB/falkordb-py/pull/276 | fail | redis.exceptions.ResponseError: Invalid graph operation on empty key | FalkorDB/falkordb-py@e307df3 | | [FalkorDB/docs:genai-tools/graphrag-sdk.mdx:1](https://github.com/FalkorDB/docs/blob/f804fd70f915fbd6f2651b56088739c0227c36fc/genai-tools/graphrag-sdk.mdx#L25) | before of FalkorDB/GraphRAG-SDK/pull/336 | skipped | needs an LLM provider (LiteLLM(); Staleproof does not hold provider keys | FalkorDB/GraphRAG-SDK@d703d4a, FalkorDB/falkordb-py@31f63f4 | | [FalkorDB/docs:getting-started/index.mdx:6](https://github.com/FalkorDB/docs/blob/f804fd70f915fbd6f2651b56088739c0227c36fc/getting-started/index.mdx#L121) | after of FalkorDB/falkordb-py/pull/276 | pass | | FalkorDB/falkordb-py@2a970dc | | [FalkorDB/docs:getting-started/index.mdx:6](https://github.com/FalkorDB/docs/blob/f804fd70f915fbd6f2651b56088739c0227c36fc/getting-started/index.mdx#L121) | before of FalkorDB/falkordb-py/pull/276 | pass | | FalkorDB/falkordb-py@e307df3 | | [FalkorDB/docs:index.mdx:1](https://github.com/FalkorDB/docs/blob/f804fd70f915fbd6f2651b56088739c0227c36fc/index.mdx#L88) | after of FalkorDB/falkordb-py/pull/276 | fail | redis.exceptions.ResponseError: Invalid graph operation on empty key | FalkorDB/falkordb-py@2a970dc | | [FalkorDB/docs:index.mdx:1](https://github.com/FalkorDB/docs/blob/f804fd70f915fbd6f2651b56088739c0227c36fc/index.mdx#L88) | before of FalkorDB/falkordb-py/pull/276 | fail | redis.exceptions.ResponseError: Invalid graph operation on empty key | FalkorDB/falkordb-py@e307df3 | | [FalkorDB/docs:operations/opentelemetry.mdx:10](https://github.com/FalkorDB/docs/blob/f804fd70f915fbd6f2651b56088739c0227c36fc/operations/opentelemetry.mdx#L206) | after of FalkorDB/falkordb-py/pull/276 | fail | ModuleNotFoundError: No module named 'opentelemetry' | FalkorDB/falkordb-py@2a970dc | | [FalkorDB/docs:operations/opentelemetry.mdx:10](https://github.com/FalkorDB/docs/blob/f804fd70f915fbd6f2651b56088739c0227c36fc/operations/opentelemetry.mdx#L206) | before of FalkorDB/falkordb-py/pull/276 | fail | ModuleNotFoundError: No module named 'opentelemetry' | FalkorDB/falkordb-py@e307df3 | | [FalkorDB/docs:operations/opentelemetry.mdx:4](https://github.com/FalkorDB/docs/blob/f804fd70f915fbd6f2651b56088739c0227c36fc/operations/opentelemetry.mdx#L81) | before of FalkorDB/falkordb-py/pull/276 | skipped | contains placeholders to fill in (e.g. "<your-host>") | FalkorDB/falkordb-py@e307df3 | | [FalkorDB/docs:udfs/index.mdx:0](https://github.com/FalkorDB/docs/blob/f804fd70f915fbd6f2651b56088739c0227c36fc/udfs/index.mdx#L20) | after of FalkorDB/falkordb-py/pull/276 | pass | | FalkorDB/falkordb-py@2a970dc | | [FalkorDB/docs:udfs/index.mdx:0](https://github.com/FalkorDB/docs/blob/f804fd70f915fbd6f2651b56088739c0227c36fc/udfs/index.mdx#L20) | before of FalkorDB/falkordb-py/pull/276 | pass | | FalkorDB/falkordb-py@e307df3 | | [FalkorDB/docs:udfs/index.mdx:6](https://github.com/FalkorDB/docs/blob/f804fd70f915fbd6f2651b56088739c0227c36fc/udfs/index.mdx#L158) | after of FalkorDB/falkordb-py/pull/276 | pass | | FalkorDB/falkordb-py@2a970dc | | [FalkorDB/docs:udfs/index.mdx:6](https://github.com/FalkorDB/docs/blob/f804fd70f915fbd6f2651b56088739c0227c36fc/udfs/index.mdx#L158) | before of FalkorDB/falkordb-py/pull/276 | pass | | FalkorDB/falkordb-py@e307df3 | | [FalkorDB/falkordb-py:README.md:2](https://github.com/FalkorDB/falkordb-py/blob/31f63f44a5745b9bec65969a1db706906718cb82/README.md#L32) | before of FalkorDB/falkordb-py/pull/276 | skipped | not Python (bash) | FalkorDB/falkordb-py@e307df3 | | [FalkorDB/falkordb-py:README.md:3](https://github.com/FalkorDB/falkordb-py/blob/31f63f44a5745b9bec65969a1db706906718cb82/README.md#L39) | after of FalkorDB/falkordb-py/pull/276 | pass | | FalkorDB/falkordb-py@2a970dc | | [FalkorDB/falkordb-py:README.md:3](https://github.com/FalkorDB/falkordb-py/blob/31f63f44a5745b9bec65969a1db706906718cb82/README.md#L39) | before of FalkorDB/falkordb-py/pull/276 | pass | | FalkorDB/falkordb-py@e307df3 | | [FalkorDB/falkordb-py:README.md:4](https://github.com/FalkorDB/falkordb-py/blob/31f63f44a5745b9bec65969a1db706906718cb82/README.md#L67) | after of FalkorDB/falkordb-py/pull/276 | pass | | FalkorDB/falkordb-py@2a970dc | | [FalkorDB/falkordb-py:README.md:4](https://github.com/FalkorDB/falkordb-py/blob/31f63f44a5745b9bec65969a1db706906718cb82/README.md#L67) | before of FalkorDB/falkordb-py/pull/276 | pass | | FalkorDB/falkordb-py@e307df3 | | [FalkorDB/falkordb-py:README.md:4](https://github.com/FalkorDB/falkordb-py/blob/31f63f44a5745b9bec65969a1db706906718cb82/README.md#L67) | after of FalkorDB/falkordb-py/pull/267 | pass | | FalkorDB/falkordb-py@a9c7510 | | [FalkorDB/falkordb-py:README.md:4](https://github.com/FalkorDB/falkordb-py/blob/31f63f44a5745b9bec65969a1db706906718cb82/README.md#L67) | before of FalkorDB/falkordb-py/pull/267 | fail | TypeError: Redis.__init__() got an unexpected keyword argument 'himport_registry' | FalkorDB/falkordb-py@031eb04 | | [FalkorDB/falkordb-py:README.md:4](https://github.com/FalkorDB/falkordb-py/blob/31f63f44a5745b9bec65969a1db706906718cb82/README.md#L67) | after of FalkorDB/falkordb-py/pull/262 | pass | | FalkorDB/falkordb-py@f7c3de7 | | [FalkorDB/falkordb-py:README.md:4](https://github.com/FalkorDB/falkordb-py/blob/31f63f44a5745b9bec65969a1db706906718cb82/README.md#L67) | before of FalkorDB/falkordb-py/pull/262 | fail | TypeError: Redis.__init__() got an unexpected keyword argument 'himport_registry' | FalkorDB/falkordb-py@49aa012 | ## Impact query timing (server-side, FalkorDB) | Changed symbol | Rows | ms | |---|---:|---:| | `falkordb.asyncio.falkordb.FalkorDB` | 49 | 0.74 | | `falkordb.asyncio.falkordb.FalkorDB.select_graph` | 49 | 0.79 | | `falkordb.asyncio.graph.AsyncGraph.query` | 670 | 2.85 | --- <!-- page: Architecture · Reference · source: https://staleproof.dev/docs/architecture.md --> # Architecture This page is for contributors: where each part of the code lives, what happens during a check from end to end, the design decisions worth knowing before changing anything, how the documentation is published from `docs/`, and how the test suite is organised. ## Source layout ```text src/ cli.ts index | check | docs-for | audit | run | decide | verdict | patch | mcp | serve | facts | db index.ts the library's exports config.ts environment → Config audit.ts the docs audit: mismatches, API coverage, runs at the indexed commits, the score (audit_docs) pkg.ts package root and version graph/client.ts FalkorDB connection; one graph per org; every query can be traced graph/queries.ts every Cypher statement, named (the README's Cypher section and GRAPH-MODEL.md are generated from it) graph/schema.ts the exact-match indexes created before every index run indexer/ repo.ts (git checkout with the token in git's environment config, file classification, module names), tarball.ts (download a GitHub tarball and extract what the indexer reads, commit from the pax header; for hosts without git), ensure.ts (clone or download; index on first use), files.ts, treesitter.ts and python.ts (tree-sitter Python extractor), typescript.ts and tsmodules.ts (TypeScript/JavaScript into the same module shape; npm packages as js-<name> modules), resolve.ts (org-wide symbol table, call resolver, static checks of examples, exported API), signature.ts (argument checks against a signature), markdown.ts (Markdown/MDX scanner, example directives), rst.ts (reStructuredText scanner), repo-config.ts (.staleproof.toml), docs.ts (MENTIONS/USES/LINKS_TO/MISMATCH), deps.ts (DEPENDS_ON), write.ts, index.ts (indexOrg) hosted/workspaces.ts hosted mode: a workspace graph per repository owner (gh-<owner>) indexed on demand from tarballs; the registry graph, cross-instance locks, size limits, least-recently-used eviction and daily budgets pr/ github.ts (PR metadata), changes.ts (symbol-level diff between the PR's commits), github-diff.ts (the same diff through the GitHub API when there is no git), ingest.ts (and re-reading an open PR that moved), open-pr.ts (the docs PR in a fork, upstream or the docs repository itself), predict.ts (predicted breaks), local.ts (a working tree or an editor's files as changes; check_changes) tools/ graph-tools.ts (impact, clearance, rank_sections, read_section), run-example.ts, docs-for.ts (docs_for_symbol), memory-tools.ts (decisions, verdicts, review, "since the last check"), types.ts (ToolResult, Tracer, unknown-parameter check), index.ts (tool sets, JSON schemas) runner/ index.ts (runExample, Run nodes, one run at a time), prepare.ts (skip rules, fragment stitching), sandbox.ts (venv per repo@sha set built with an allow-listed environment, the sandboxed process), scratch.ts (scratch FalkorDB start and flush) agent/ loop.ts (runAgentCheck), prompt.ts, model.ts (ModelClient), anthropic.ts (Claude and Anthropic-compatible providers), scripted.ts (ScriptedModel), graph-check.ts (graph mode), trace.ts (trace events) mcp/server.ts MCP server over the tools, plus check_pull_request, check_changes, docs_for_symbol, audit_docs and docs_patch; stdio transport (indexes a pull request's repository on first use) mcp/http.ts MCP over Streamable HTTP, stateless; in hosted mode routed to the graph of the repository each call names, without run_example server/app.ts HTTP API (+ SSE), hosted routes (/api/index, /api/repos, /api/workspaces), /api/audit, /api/docs-for, /api/badge (server/badge.ts), /api/mcp, the static web UI and the docs as plain text patch.ts verdicts → unified diff + PR description with evidence facts.ts FACTS.md from the graph format/ text.ts (terminal output), markdown.ts (the PR comment's Markdown), audit.ts (the audit as text and Markdown), report.ts (the report from the graph: impact, clearance, ranking, verdicts and the example runs before and after; what the Action posts and check_pull_request returns) web/ index.html, vite.config.ts Vite + React; the build also emits the docs as plain text (llms.ts) llms.ts /llms.txt, /llms-full.txt, /docs/<slug>.md, robots.txt, sitemap.xml src/main.tsx, App.tsx entry and routes (home, a stored check, the query catalogue, docs) src/route.ts hash routes: #/, #/<home section>, #/try?pr=<url>, #/check/<id>, #/audit?repo=<repo>, #/install, #/model, #/docs/<slug>/<anchor> src/layout.tsx header, footer, icons, code blocks src/Landing.tsx the home page's content (also prerendered) src/TryIt.tsx the live demo: paste a PR, watch the check stream in, recent checks src/Audit.tsx the docs audit page: score, mismatches, coverage, the badge src/Install.tsx, install-links.ts the install page: the prefilled workflow link, MCP buttons and commands src/CheckResults.tsx a check's findings: changed symbols, sections, runs, verdicts; a stored check replayed src/components.tsx evidence paths, Cypher lists, the trace, runs, diffs, the decision form src/state.ts trace events → view (the same reducer live and replayed) src/api.ts API client and the snapshot fallback src/Docs.tsx the docs pages: navigation, table of contents, search src/docs/manifest.ts the page list (file, slug, title, summary, group) and llms.txt builders src/docs/render.ts Markdown → HTML with GitHub-style anchors and in-app links src/prerender.tsx the home page as static HTML src/format.ts, types.ts, styles.css public/snapshot.json what the API returned at one moment, for hosts without FalkorDB python/sitecustomize.py the sandbox shim (socket and DNS) python/run_example.py the sandbox entry point: runs example.py, on an event loop when it has top-level await grammars/ tree-sitter-python.wasm scripts/ falkordb-local.sh (FalkorDB without Docker), seed.sh (index + graph-mode checks + facts), gen-docs.ts (generated docs), snapshot.ts (web/public/snapshot.json), prerender.ts (the home page into web/dist/index.html) api/index.js the Vercel Function wrapping server/app.ts (vercel.json rewrites /api/* to it) action.yml composite GitHub Action at the repository root (uses: rajkaria/staleproof@v1) examples/staleproof.yml workflow using the Action test/ vitest suite and the fixture organisation (test/fixtures/org) ``` ## A check, end to end Before the check, the repository must be in a graph. `ensureIndexed` (CLI `check`, stdio MCP) indexes the pull request's repository and any `--with` repositories when the graph lacks one, cloning with git or downloading tarballs where there is none (`checkoutOrDownload`). In hosted mode `Workspaces.ensure` finds the graph that holds the repository, or indexes it into the owner's workspace under a lock, making room by evicting the least recently used workspaces first. Then: 1. `ingestPullRequest`: PR metadata from GitHub → commits before/after → `diffRange` parses both versions of each changed Python file and compares definitions → `PullRequest` + `CHANGED_IN`; removed symbols recreated and late-linked to docs through the identifiers stored on each section. Without `git`, the same diff is read through the GitHub API. 2. `runAgentCheck` creates a `Check` node and gives the model the system prompt, the PR URL and a "since the last check" summary from the graph. Each turn streams; tool inputs are checked for unknown parameters and validated (zod) before running; tool errors go back to the model as error results. 3. Tools read and write the graph through a traced view, so every Cypher statement and its server time lands in the trace. `run_example` builds (or reuses) a virtualenv per set of repo@sha, flushes the scratch FalkorDB, and runs the example with the shim through `python/run_example.py`; identical runs are answered from `Run` nodes. 4. After any `impact` call, the loop collects the section ids of every impact page itself. When the model stops, the loop checks that each of those sections has a verdict from this check; if not, it sends up to two reminders, then records the check as incomplete. The trace is stored on the `Check` node, linked to the PR and the verdicts it produced. Graph mode (`runGraphCheck`) runs impact, clearance, rank_sections and up to 3 example runs in that order instead, with no model and no verdicts. `checkReport` then renders the report from what the graph holds: the comment the GitHub Action posts and what `check_pull_request` returns. 5. `buildDocsPatch` applies the latest stale verdicts to each file at the indexed commit, read from the local clone or from raw.githubusercontent.com (refusing sections whose text changed since), and renders a unified diff and a PR description with the evidence per section. The details: [The agent](agent.md), [Example runner](example-runner.md), [Decisions and memory](memory.md). ## Decisions worth knowing - One graph per organisation (graph name = org slug): FalkorDB's multi-graph is the tenancy boundary. - Nothing is deleted on re-index: symbols and sections are marked `gone`, so decisions and verdicts never dangle. - By-name resolution is allowed only inside repositories the module imports, never for generic names, and is narrowed by what the same docs page uses exactly; every by-name edge is labelled. - Every variable-length pattern binds a path variable and clearance counts bound paths, because an anonymous variable-length pattern missed callers on FalkorDB 4.18; [test/traversal.test.ts](../test/traversal.test.ts) keeps it that way. - The agent never edits without a path: `propose_edit` checks the graph before writing a verdict. - Hosted mode drops and re-indexes only graphs named with its `gh-` prefix; the curated graph is never touched, and a workspace is always indexed whole, so its cross-repository edges stay resolved. - Unknown tool parameters are an error, in the agent loop and over MCP, so a model never gets the same answer to an argument that was silently dropped. - Code from a pull request never sees the server's environment: the install step (which runs build code) gets an allow-list of paths, locale, proxies, certificates and `UV_*`/`PIP_*` settings; the example gets nothing inherited. - Example runs are serialised in one process (the scratch server is flushed per run), and the server runs one check at a time. - Agent mode fails loudly without credentials; graph mode is a separate, labelled mode, not a silent fallback. ## How the docs are published The documentation is the Markdown in `docs/` plus the repository README. One list, `DOCS` in [web/src/docs/manifest.ts](../web/src/docs/manifest.ts), names every page (file, slug, title, one-sentence summary, group) and drives every place they are published: | Output | Built by | Served at | |---|---|---| | Web docs pages | `web/src/Docs.tsx` loads `docs/<file>.md` and renders it with `web/src/docs/render.ts` (GitHub-style anchors; links to other pages become in-app routes, links to code go to GitHub) | `#/docs/<slug>` | | `/llms.txt` | `llmsTxt()` in the manifest: a summary, quick facts, and one line per page | the site root | | `/llms-full.txt` | `llmsFullTxt()`: every page in reading order, each under a header | the site root | | `/docs/<slug>.md` | each page's Markdown as is | the site | | `robots.txt`, `sitemap.xml` | `web/llms.ts` | the site root | | `llms.txt` in the repository | `npm run docs`, linking to `docs/<file>` | GitHub | `web/llms.ts` produces the plain-text files; the Vite build emits them into `web/dist`, the Vite dev server serves them, and `staleproof serve` serves them from the built web directory. On Vercel they are static files. Some parts are generated from the code and never edited by hand. `npm run docs` ([scripts/gen-docs.ts](../scripts/gen-docs.ts)) rewrites the README's Cypher section, [docs/GRAPH-MODEL.md](GRAPH-MODEL.md), the parameter reference between the generated markers in [docs/tools.md](tools.md), and the repository `llms.txt`. `staleproof facts` writes [docs/FACTS.md](FACTS.md) from the graph. Tests keep them in sync: [test/docs.test.ts](../test/docs.test.ts) fails if the README's Cypher section or GRAPH-MODEL.md drift from `src/graph/queries.ts`, and [test/docs-site.test.ts](../test/docs-site.test.ts) fails if the tools reference or `llms.txt` drift, if a page in `docs/` is missing from the manifest (or the reverse), if a page lacks its single H1 and opening paragraph, or if any relative link or heading anchor in `docs/*.md` or the README is broken. To add a page: write `docs/<file>.md` with one H1 and a summary paragraph, add an entry to `DOCS`, then run `npm run docs` and `npm run verify`. ## Testing ```bash npm install npm run verify # typecheck (engine + web), then the whole vitest suite ``` The integration tests need a FalkorDB. Without `FALKORDB_TEST_URL`, the global setup ([test/global-setup.ts](../test/global-setup.ts)) starts the Docker-free local server on port 6380 with `scripts/falkordb-local.sh` and leaves it running for the next run. Test files run one at a time, against a fixture organisation of three small repositories (`test/fixtures/org`: a client, an SDK built on it, and a docs site). No test calls a paid model API, and GitHub API responses are faked. | Area | Test files | |---|---| | Indexing | `python.test.ts` (extractor), `resolve.test.ts` (module names, symbol table, exact and by-name resolution), `markdown.test.ts` (scanner, identifiers, links), `index-code.test.ts`, `index-org.test.ts` (the whole organisation in FalkorDB) | | Pull requests | `changes.test.ts` (change kinds, diff parsing, the GitHub API source) | | Graph queries and tools | `tools.test.ts` (impact paging, clearance, ranking, read_section), `traversal.test.ts` (variable-length paths, bound path variables) | | Example runner | `prepare.test.ts` (skip rules, stitching, outcomes, install conflicts, the install environment allow-list), `sandbox.test.ts` (the shim, timeout, no inherited secrets), `run-example.test.ts` (before/after runs in FalkorDB) | | Memory and patch | `memory.test.ts` (decisions, supersession, edit guard, verdict history, the docs patch, opening the docs PR, "since the last check") | | Agent | `agent.test.ts` (the loop with a scripted model, reminders, errors, time budget, unknown parameters, every impact page), `anthropic-model.test.ts` (the Claude and compatible request shapes against a local fake API) | | Interfaces | `mcp.test.ts`, `server.test.ts` (HTTP API and SSE), `format.test.ts` (the PR comment), `action.test.ts` (the Action calls real commands and flags), `web-state.test.ts` (the web view reducer) | | Hosted mode | `hosted.test.ts` (tarball download and the pax header, translated docs, workspaces on first use, limits and daily budgets, hosted checks and repository status, MCP over HTTP and its routing) | | Package and facts | `package.test.ts` (shipped assets, the built UI and the plain-text docs served), `facts.test.ts` | | Docs | `docs.test.ts` (generated sections, no internal material in tracked files), `docs-site.test.ts` (manifest, page shape, links and anchors, generated reference, llms.txt) | ## Related - [Core concepts](concepts.md): the vocabulary the code uses - [The agent](agent.md): the check loop in detail - [Graph model and Cypher](GRAPH-MODEL.md): every query - [Deploy](DEPLOY.md): running it on Vercel, Docker Compose or one machine --- <!-- page: Deploy · Operate · source: https://staleproof.dev/docs/deploy.md --> # Deploy A deployment is one Node.js process (`staleproof serve`: the web app, the HTTP API and MCP over HTTP) plus FalkorDB. This page covers four ways to run it (Vercel with FalkorDB Cloud, Docker Compose on any VM, a plain host without Docker, and FalkorDB Cloud as the database for any of them), hosted mode and its limits (how staleproof.dev indexes any public repository on demand), connecting Claude Code over MCP, the GitHub Action, and a checklist for running it for a team. It is for whoever operates the service; every variable is described in [Configuration](configuration.md). ## Choose a host | | Vercel | Docker Compose, a VM or your machine | |---|---|---| | Web app, API and MCP at `/api/mcp` | yes | yes | | Hosted mode (any public repository on demand) | on by default (`STALEPROOF_HOSTED=off` turns it off) | with `serve --hosted` or `STALEPROOF_HOSTED=on` | | Agent mode and graph mode | yes; agent checks stop at 240 s by default | yes; no time limit unless you set one | | Indexing | tarball downloads (no git), in hosted mode | git for `staleproof index`; tarball downloads in hosted mode | | Pull request ingestion | through the GitHub API (no git) | git | | Example runs | not executed; runs recorded on the graph by another host are answered | yes, in the sandbox | | Docs patch | yes, files read from GitHub at the indexed commit | yes, from the local clones | | Raw text-match counts in `clearance` | 0 (no clones to search) | yes | The usual split: index and seed from a machine that can run examples, and serve the demo from Vercel against the same FalkorDB. ## The minimum you need | Variable | Why | |---|---| | `FALKORDB_URL` | The FalkorDB holding the org graph. Default `falkor://localhost:6379`; `serve` starts a local server there when it is unset and nothing answers | | `STALEPROOF_ORG` | The graph the server answers from (the curated graph in hosted mode), if it is not `falkordb` | | `GITHUB_TOKEN` (or `GH_TOKEN`) | Without it GitHub allows 60 API requests per hour; private repositories need it; required for `patch --open-pr` | | `ANTHROPIC_API_KEY` | Agent mode. For an Anthropic-compatible provider such as DeepSeek, also `ANTHROPIC_BASE_URL` and `STALEPROOF_MODEL` (for example `deepseek-v4-pro`). Without a key the server runs graph mode and says so | The full list, with defaults and the commands that read each one, is in [Configuration](configuration.md#environment-variables). A host that runs examples also needs `git`, `tar`, and `uv` or `python3`. With [uv](https://docs.astral.sh/uv/) each virtualenv is created for Python 3.12; without it the runner falls back to `python3 -m venv` and pip, and the examples run on whatever version `python3` is. uv is recommended. Hosts without git (serverless) skip example runs with a reason and answer recorded runs from the graph. ## Vercel The repository deploys to Vercel as is ([vercel.json](../vercel.json)): the web app is static (`web/dist`) and the API, MCP included, runs as one Vercel Function ([api/index.js](../api/index.js)); a rewrite in vercel.json sends every `/api/...` path to it. On Vercel, pull requests are read through the GitHub API (no `git`), repositories are downloaded as tarballs, `STALEPROOF_HOME` is `/tmp/staleproof`, and examples are not executed (no sandbox runtime): runs recorded by a local or container host are answered from the graph. The function runs in [hosted mode](#hosted-mode) unless `STALEPROOF_HOSTED=off`. 1. Create a FalkorDB the function can reach ([FalkorDB Cloud](#falkordb-cloud)'s free tier works) and index into it from your machine: ```bash export FALKORDB_URL='falkor://falkordb:<url-encoded password>@<host>:<port>' # falkors:// if the endpoint has TLS npx -y staleproof index FalkorDB/falkordb-py FalkorDB/GraphRAG-SDK FalkorDB/docs bash scripts/seed.sh # from a clone of the repository: re-index, graph-mode checks of the preloaded PRs with example runs, FACTS.md ``` `scripts/seed.sh` is not in the npm package; it uses `staleproof` from your `PATH`, else the clone's `dist/cli.js`. Its example runs execute on your machine and are stored on the graph, which is what lets the Vercel deployment show them. 2. Add the variables and redeploy: ```bash vercel env add FALKORDB_URL production vercel env add GITHUB_TOKEN production # recommended: fine-grained, public repositories, read-only vercel env add ANTHROPIC_API_KEY production # optional: agent mode vercel env add ANTHROPIC_BASE_URL production # with STALEPROOF_MODEL, only for an Anthropic-compatible provider vercel env add STALEPROOF_MODEL production ``` `STALEPROOF_PRELOADED` (the pull requests on the home page), `STALEPROOF_CHECK_BUDGET_MS`, `STALEPROOF_AGENT_CHECKS_PER_DAY` (default 50) and `STALEPROOF_HOSTED` are optional. `PORT`, `HOST` and the scratch variables are not used there. Without a reachable FalkorDB the function answers `503 {"error": "the Staleproof API is not connected to FalkorDB (<message>); set FALKORDB_URL", "offline": true}` and the web app shows the snapshot bundled at build time, labelled with the time it was taken ([snapshot mode](http-api.md#snapshot-mode); `npm run snapshot` regenerates `web/public/snapshot.json` from a graph). Function duration is capped at 300 s (`maxDuration`). An agent check in the server starts no model turn after 240 s (`STALEPROOF_CHECK_BUDGET_MS`) and ends `incomplete`, listing the sections still open; a 43-section agent check takes about 6 minutes, so run large checks from the CLI, which has no time limit. The function runs in `bom1` (Mumbai), next to the FalkorDB Cloud instance in AWS `ap-south-1`: a check makes dozens of graph round trips, so set `regions` in `vercel.json` to the Vercel region closest to your FalkorDB. ## Hosted mode Hosted mode is how staleproof.dev checks pull requests in repositories nobody indexed for it. It is on by default in the Vercel function, and `staleproof serve --hosted` (or `STALEPROOF_HOSTED=on`) turns it on anywhere else. The code is [src/hosted/workspaces.ts](../src/hosted/workspaces.ts). - **The curated graph** is the server's own (`STALEPROOF_ORG`, else `falkordb`), indexed and seeded from the CLI as before. A repository it holds answers from it; hosted mode never re-indexes, evicts or adds to it. - **Workspaces.** Any other public repository is indexed on demand into the graph of its owner, `gh-<owner>` (the owner slugged), when a check, an audit (`POST /api/audit`, the `#/audit` page), `POST /api/index` or an MCP call first names it. The repositories named with it (`with`: a docs site, an SDK) join the same workspace, and every repository of a workspace is indexed together on every refresh, so calls and doc links across them resolve. - **Downloads.** Each repository is downloaded as a tarball at its default branch head and only the files the indexer reads are extracted ([Indexing](indexing.md#without-git-tarballs)); translated docs directories, such as `docs/zh/` or `i18n/pt-BR/`, are skipped. The extracted files are deleted once the repository is indexed. - **Registry.** The graph `staleproof-hosted` records which workspace holds each repository, its commit, when it was indexed and last used, a lock per workspace (one indexing run at a time across every server instance; others wait up to 150 seconds), and the daily counters. - **No examples.** Graph-mode checks in a workspace run no examples, and the MCP endpoint does not offer `run_example`. A server with a sandbox still runs them for the curated graph. ### Limits `DEFAULT_HOSTED_LIMITS`: | Limit | Value | What happens | |---|---|---| | Repositories per workspace | 4 | Indexing a fifth is refused, with the `npx -y staleproof index …` command to run it yourself | | Workspace graph size | 20 MB | A workspace bigger than this after indexing is dropped and refused (needs `GRAPH.MEMORY USAGE` on the server; without it, the size is not checked) | | Eviction | above 70 MB, down to 55 MB | Before indexing, when the FalkorDB server uses more than 70 MB, the least recently used workspaces are dropped until it uses less than 55 MB. If it still uses more than 70 MB, indexing is refused: `the hosted service is full right now.` A dropped workspace is indexed again the next time someone asks for it | | Re-index | after 7 days | A workspace indexed more than 7 days ago, or by an older Staleproof (before index format 3: no static checks or export flags), is indexed again at its default branch heads on its next use | | Index runs | 500 per day | Across the service, per UTC day | | Archive size | 150 MB | Compressed download, checked against `Content-Length` and while streaming | | Files | 3000 | Files the indexer would read (Python, TypeScript and JavaScript, Markdown, MDX and reStructuredText, dependency manifests) | | Agent checks | 50 per day | `STALEPROOF_AGENT_CHECKS_PER_DAY` in the Vercel function; over it, agent checks answer 429 and graph mode still works | The eviction thresholds fit FalkorDB Cloud's free tier, which is 100 MB. `GET /api/health` reports the server's memory and the limits, and `GET /api/workspaces` lists every workspace ([HTTP API](http-api.md#get-apiworkspaces)). ### A GitHub token for hosted mode Give the server a `GITHUB_TOKEN`. Unauthenticated GitHub API requests are limited to 60 an hour per IP address, and Vercel's egress IP addresses are shared with other deployments, so without a token pull request ingestion (up to five API requests per new pull request) runs into other people's limit. Use a fine-grained token with access to public repositories only and no write permission. Hosted mode downloads repositories without it (from codeload), so it indexes public repositories only whatever the token can read; the token only raises the API limit for reading pull requests. ## Docker Compose (any VM) ```bash docker compose up -d --build docker compose exec staleproof node dist/cli.js index FalkorDB/falkordb-py FalkorDB/GraphRAG-SDK FalkorDB/docs docker compose exec -e FACTS_OUT=/data/FACTS.md staleproof bash scripts/seed.sh # optional: index + graph-mode checks of the preloaded PRs ``` The compose file ([docker-compose.yml](../docker-compose.yml)) runs two FalkorDB containers: `falkordb` (the graph, persisted in the `falkordb-data` volume) and `scratch` (examples only, no volume, flushed before every run). The `staleproof` container serves on port 8080 with `FALKORDB_URL=falkor://falkordb:6379` and `STALEPROOF_SCRATCH_URL=falkor://scratch:6379`. - **Image**: Node 22 on Debian bookworm, with `git`, `python3`, `tar` and uv 0.11.6. `STALEPROOF_HOME` is `/data` (the `staleproof-data` volume, which keeps clones and virtualenvs across restarts), uv keeps Python and its cache under `/data`, and a health check calls `/api/health` every 30 s. - **Variables passed through** from your shell or `.env`: `STALEPROOF_ORG` (default `falkordb`), `ANTHROPIC_API_KEY`, `GITHUB_TOKEN` and `STALEPROOF_PRELOADED`. Compose passes each one as an empty string when it is unset. For `STALEPROOF_PRELOADED` that means the home page lists no pull requests, so set it explicitly if you want the preloaded list. `ANTHROPIC_BASE_URL`, `STALEPROOF_MODEL` and `GH_TOKEN` are not passed through: to use an Anthropic-compatible provider, add them to the service's `environment` in the compose file. - **`FACTS_OUT`**: `scripts/seed.sh` ends by writing FACTS.md to `docs/FACTS.md` by default, and the image has no `docs/` directory; the command above writes it to the volume instead. - **Exposure**: the API has no authentication. Put a TLS-terminating proxy in front of port 8080 for a public demo, and see the [production checklist](#production-checklist). > The Dockerfile and compose file are written but were not built on the machine Staleproof was developed on (no Docker there). Everything else in this document was run. ## Without Docker ```bash npx -y staleproof db start # FalkorDB on :6379 from the falkordblite wheel (uv fetches Python 3.12) npx -y staleproof index FalkorDB/falkordb-py FalkorDB/GraphRAG-SDK FalkorDB/docs npx -y staleproof serve --host 0.0.0.0 --port 8080 ``` `db start` installs the `falkordblite` wheel once into `~/.staleproof/falkordb/venv` (with uv, else `python3 -m venv` and pip; the wheel needs Python 3.12 or later) and keeps the data in `~/.staleproof/falkordb/data-6379`; without it, `serve` starts the same server itself. Add `--hosted` to `serve` to index any public repository on demand. The scratch server for example runs starts by itself on port 6390 the first time an example runs. `--host 0.0.0.0` exposes the unauthenticated API on every interface; keep the default `127.0.0.1` unless a proxy sits in front. ## FalkorDB Cloud Create a free instance; its Connectivity tab gives the public endpoint and Instance Details the password for user `falkordb`. The free tier's endpoint is plain TCP: `FALKORDB_URL=falkor://falkordb:<password>@<host>:<port>`. URL-encode the password (`@` becomes `%40`, `:` becomes `%3A`, and so on), and use `falkors://` for an endpoint with TLS ([URL forms](configuration.md#graph-and-organisation)). Keep the scratch server separate and disposable: leave `STALEPROOF_SCRATCH_URL` unset to use the local one, or point it at another server. The scratch server cannot require a password: docs examples connect to it without one, so a `STALEPROOF_SCRATCH_URL` with credentials is refused (`STALEPROOF_SCRATCH_URL must not carry credentials: ...`). Staleproof also refuses to flush the server named in `FALKORDB_URL`. ## Claude Code (MCP) ```bash claude mcp add staleproof -e FALKORDB_URL=falkor://localhost:6379 -e STALEPROOF_ORG=falkordb -- npx -y staleproof mcp ``` Add `-e GITHUB_TOKEN=...` to raise the GitHub limit for pull requests the graph has not ingested yet. A deployment also serves MCP over Streamable HTTP at `/api/mcp`, for example `claude mcp add --transport http staleproof https://<your host>/api/mcp`. [MCP server](mcp.md) covers other clients and the fourteen tools. ## GitHub Action See [examples/staleproof.yml](../examples/staleproof.yml) and [GitHub Action](github-action.md). The action needs no deployment: it starts a FalkorDB in the job. The workflow needs `pull-requests: write` to post its comment; add `ANTHROPIC_API_KEY` as a secret for agent verdicts. - Without a key, the action runs a graph-mode check: impact, clearance, ranking and up to 3 examples before and after the pull request. - To keep decisions, verdicts and runs between jobs, point `falkordb-url` at your deployment's FalkorDB (stored as a secret such as `STALEPROOF_FALKORDB_URL`) and set `org` to its graph. - With `reindex: 'true'` and `falkordb-url`, the action re-indexes the pull request's repository and `also-index` before checking, at the remote default branch head (no ref is passed). Repositories of the graph that are not listed are not re-indexed with them, so cross-repository edges into them are dropped until the organisation is indexed together again. - The action has run end to end on a private test repository with the one-click workflow (no inputs). ## Production checklist - **A separate scratch database.** Examples run against a FalkorDB that is flushed (`FLUSHALL` and `GRAPH.UDF FLUSH`) before every run. Use a dedicated, unauthenticated server on a private network, or the managed local one (127.0.0.1:6390). Never point it at the org graph's server: Staleproof refuses when the host and port match, but a different address for the same server is not detected. - **Authentication and rate limits.** The API has no authentication, only a limit of 30 POST requests per client per hour, counted in memory per process; `/api/mcp` is not limited. Put an authenticating proxy in front of a public deployment. The limit keys on `X-Real-IP`, else `Fly-Client-IP`, else the last `X-Forwarded-For` entry: have your proxy set `X-Real-IP`, or a client can pick its own key by sending it; without a proxy, all clients share one bucket. - **`GITHUB_TOKEN`.** Without it, ingestion shares GitHub's 60 requests per hour per IP address; through the GitHub API (Vercel) one new pull request costs up to five requests. In hosted mode, use a token limited to public repositories ([why](#a-github-token-for-hosted-mode)). - **Hosted mode's memory.** Workspaces are evicted by the FalkorDB server's total memory, so the curated graph counts toward the 70 MB threshold too. On a larger server, raise the limits through the library (`new Workspaces(store, config, curated, { limits })`). - **Function region next to FalkorDB.** Set `regions` in `vercel.json` to the region nearest your FalkorDB; a check makes dozens of graph round trips. - **Snapshot fallback.** Run `npm run snapshot` against the production graph before you build, so the page a visitor sees when the database is unreachable is recent. It is labelled with the time it was taken. - **Time budget.** On Vercel, keep `STALEPROOF_CHECK_BUDGET_MS` well under the function's 300 s. A function killed mid-check leaves its `Check` node `running`. - **Model settings.** For an Anthropic-compatible provider, set `STALEPROOF_MODEL` to the provider's model; a `claude-*` name is refused and every agent check fails. `GET /api/health` shows the model and provider the server will use. - **Persistent storage.** On a VM, keep `STALEPROOF_HOME` on a volume, or virtualenvs and clones are rebuilt after every restart. Back up the FalkorDB data: verdicts, decisions, runs and checks live only in the graph. - **Re-indexing.** Index all of an organisation's repositories in one `index` command, so cross-repository edges are resolved together. A re-index marks vanished symbols and sections `gone`, and it removes the late links that earlier pull request ingestion created; already-ingested pull requests are not ingested again. - **Isolation for runs.** The sandbox is a guard, not a security boundary: it blocks network and DNS at the Python socket level, the example process inherits no environment, and the install step, which runs the repositories' build code, gets only an allow-list (paths, locale, proxies, certificates, `UV_*` and `PIP_*`), so neither sees `FALKORDB_URL`, model keys or `GITHUB_TOKEN`. Installs do reach the network to fetch dependencies. Run the server in a container when it executes examples from repositories you do not control. ## Related - [Configuration](configuration.md): every variable and its default - [HTTP API](http-api.md): routes, rate limits and snapshot mode - [GitHub Action](github-action.md) - [Example runner](example-runner.md): the sandbox and the scratch server - [Troubleshooting and FAQ](troubleshooting.md) --- <!-- page: Troubleshooting and FAQ · Operate · source: https://staleproof.dev/docs/troubleshooting.md --> # Troubleshooting and FAQ This page explains the error messages you can meet when running `staleproof` (each with what it means and how to fix it), the FalkorDB version behaviour the queries are written around, the limits of what the tool does today, and answers to the questions developers ask most. It is for anyone using the CLI, the server, the MCP tools or the library. Messages are quoted as the code writes them; `<...>` marks a value that varies. ## Common errors The CLI prints errors as `staleproof: <message>` and exits with code 1. The HTTP API returns them as `{"error": "<message>"}`, tools return them as error results, and failed example runs record them as the run's reason. ### Setup and configuration #### FalkorDB is not reachable ```text cannot reach FalkorDB at <url>: <message> Start one locally with `npx staleproof db start` (or scripts/falkordb-local.sh start), or set FALKORDB_URL. ``` Nothing answered at `FALKORDB_URL` (or `--db`) within 5 seconds, or the credentials were refused. The password in `<url>` is shown as `***`. With neither of them set, CLI commands start a local server on `localhost:6379` instead of failing, so from the CLI this error means a URL you configured did not answer, or the local server did not come up. Fix the URL (`falkors://` for TLS endpoints, the password URL-encoded; [URL forms](configuration.md#graph-and-organisation)), or unset it to use the local server. After a connection drops, the client retries four times and then reports `FalkorDB unreachable`. When the automatic start itself fails, the command stops, after the script's own message ([Local FalkorDB does not start](#local-falkordb-does-not-start)), with: ```text could not start a local FalkorDB (see above). Start one yourself, e.g. docker run -d -p 6379:6379 falkordb/falkordb, or set FALKORDB_URL to a FalkorDB you have. ``` #### A query failed `FalkorDB <query name> failed on graph "<graph>": <message>` FalkorDB rejected or failed a statement; `<query name>` is the name in [Graph model and Cypher](GRAPH-MODEL.md), or `adhoc` for unnamed statements. Check the server version ([version notes](#falkordb-version-notes)) and that the server has the FalkorDB module loaded. #### Invalid organisation name `invalid org name: "<name>"` `--org` or `STALEPROOF_ORG` contains no letter, digit, `_` or `-` (an empty `STALEPROOF_ORG` also gives this). Names are slugged: `Acme Inc` becomes the graph `acme-inc`. #### Invalid effort `STALEPROOF_EFFORT must be one of low, medium, high, xhigh, max` The variable is checked when the configuration loads, so this stops every command except `db`, even with `--effort` on the command line. Fix or unset the variable. `check --effort` itself is not checked locally: an unknown value is sent to the model API as is, and if the API rejects it the check ends with `error`. #### Scratch URL with credentials `STALEPROOF_SCRATCH_URL must not carry credentials: docs examples connect to the scratch server without a password` Docs examples connect to `localhost:6379` without a password, so the scratch server cannot require one. Because the configuration loads in every command, this stops all of them except `db`. Point `STALEPROOF_SCRATCH_URL` at an unauthenticated server on a private network, or unset it to use the managed local server on port 6390. A value that cannot be parsed at all gives `invalid STALEPROOF_SCRATCH_URL: <url>`. #### Broken installation `tree-sitter-python.wasm not found (expected in <package>/grammars/)` (or `tree-sitter-typescript.wasm`, `tree-sitter-tsx.wasm`) A grammar is missing from the package, which happens with a partial install or a copied `dist/` without `grammars/`. Reinstall the package; in a custom image, copy `grammars/` and `python/` next to `dist/`. #### Local FalkorDB does not start ```text falkordb-local: need uv (https://docs.astral.sh/uv/) or Python 3.12+ to install falkordblite; or run FalkorDB in Docker: docker run -d -p 6379:6379 falkordb/falkordb falkordb-local: falkordblite needs Python 3.12+ (python3 is <version>): install uv (https://docs.astral.sh/uv/), which fetches one, or run FalkorDB in Docker: docker run -d -p 6379:6379 falkordb/falkordb ``` `staleproof db start` (and the managed scratch server, and the local server a command starts when no FalkorDB is configured) installs the `falkordblite` wheel once, with uv or with `python3 -m venv` and pip. The wheel needs Python 3.12 or later: install uv, which fetches a Python 3.12, or run FalkorDB in Docker and leave `FALKORDB_URL` at its default. `falkordb-local: server did not answer PING on port <port> — see <data>/redis.log` The server did not come up within 5 seconds. The log usually says why, for example that the port is already in use. `staleproof db status --port <port>` reports whether a server is running there. ### Indexing #### Not a repository name `not a GitHub repository: "<input>" (expected owner/name or a github.com URL)` `index` accepts `owner/name`, `https://github.com/owner/name[.git]` and `git@github.com:owner/name[.git]`. #### git failed `git <arguments> failed in <directory>: <git's error>` Common causes: - **A `--ref` git cannot check out.** The ref is passed to `git checkout --detach` unchanged. A bare branch name resolves to the clone's local branch, which fetching does not advance; use `origin/<branch>`, a tag or a SHA. - **A private or missing repository.** Terminal prompts are disabled, so a private repository fails here unless `GITHUB_TOKEN` (or `GH_TOKEN`) is set to a token that can read it; git then sends it to github.com. Check the name, and the token's access. - **No network.** Clones and fetches need github.com. Without git installed at all, indexing downloads tarballs instead (next section). #### Tarball download failed Without git, or with `STALEPROOF_INGEST=github`, repositories are downloaded as tarballs ([Indexing](indexing.md#without-git-tarballs)): | Message | Meaning | |---|---| | `GitHub: <owner/repo>[@<ref>] not found (is it public? private repositories need GITHUB_TOKEN)` | GitHub answered 404: a wrong name or ref, or a private repository without a token that can read it | | `GitHub: downloading <owner/repo> failed: <status>` | Any other failed download | | `GitHub: the archive of <owner/repo> carries no commit id` | The archive had no pax header with the commit; nothing was indexed | | `<owner/repo> is too large to index from an archive here (<what>; the limit is 150 MB and <n> files). Run Staleproof where git is installed (npx -y staleproof check <pr-url>, which clones instead) or in the GitHub Action.` | The archive is over 150 MB, or it has more than 4000 files the indexer reads (3000 on a hosted server). The CLI writes the same text, so where git is available, install it and index again | #### Repository configuration error `.staleproof.toml: ignore must be a list of glob strings` (or `skip-examples`, or `pyproject.toml: …`) The repository's [configuration](configuration.md#repository-configuration) has a key with the wrong type, for example `ignore = "docs/old"` instead of `ignore = ["docs/old"]`. A file that is not valid TOML stops the index with the TOML parser's message. Fix the file in the repository and index again. ### Pull requests and GitHub #### Not a pull request URL ```text not a GitHub pull request URL: "<input>" (expected https://github.com/<owner>/<repo>/pull/<n>) ``` Accepted forms: `https://github.com/<owner>/<repo>/pull/<n>`, with an optional trailing `/files`, query or fragment, and `<owner>/<repo>#<n>`. Issue URLs, commit URLs and URLs of other hosts are refused. The HTTP API answers 400 for this. #### Repository not indexed ```text <owner/repo> is not indexed in the "<graph>" graph. Run: staleproof index <owner/repo> (plus the repositories that document it). ``` The pull request's repository is not in this graph, and the caller does not index on first use: `check --no-index`, `run --pr`, the library, or the check form and `POST /api/check` of `staleproof serve` without `--hosted`. `check`, both MCP servers and hosted servers index the repository themselves. Index it together with the repositories that document it, and check that the graph is the one you indexed into: the CLI and the stdio MCP server default to the graph of the repository's owner, `serve` to `STALEPROOF_ORG`, else `falkordb`. #### GitHub rate limit `GitHub API rate limit reached (60 requests/hour without a token; set GITHUB_TOKEN to raise it). It resets in <n> min.` Unauthenticated GitHub API requests are limited per IP address. Set `GITHUB_TOKEN` (or `GH_TOKEN`). Ingesting a pull request costs one request with git and up to five without it (Vercel, or `STALEPROOF_INGEST=github`); merged pull requests are cached under `STALEPROOF_HOME`. With a token the message is `GitHub API rate limit reached. It resets in <n> min.` The reset time is left out when GitHub does not send it. On a hosted server, whose token you cannot change, a check that hits the limit ends with the same message followed by `This shared server reads pull requests through GitHub's API, which limits it per hour. Until then, run the same check yourself (npx -y staleproof check <pr-url>) or add the GitHub Action: both use your own GitHub quota.` #### Not found `GitHub: <path> not found (a private repository needs GITHUB_TOKEN with read access)` GitHub answered 404: the pull request number is wrong, the repository is private, or it was renamed. With a token the message is `GitHub: <path> not found`, and a private repository then means the token cannot read it. Other GitHub failures read `GitHub <METHOD> <path> failed: <status> <start of the response>`, and a failed file read reads `GitHub raw <path>@<sha7> failed: <status>`. #### Merge commit without a parent `merge commit <sha> has no parent` The merge commit GitHub reports for the pull request has no parent to compare against. This is unusual; check the pull request on GitHub. ### Before a pull request | Message | Meaning | |---|---| | `cannot tell which GitHub repository <dir> is (origin: <url>); pass --repo owner/name` | `staleproof check` without a pull request (or `check_changes` with `dir`) could not read a github.com `origin` remote (`origin: none` when there is none). Pass `--repo` (`repo` over MCP) | | `<owner/repo> is not indexed in the "<graph>" graph: staleproof index <owner/repo>` | The working tree's repository is not in the graph and indexing on first use is off (`--no-index`) or not available | | `repo (owner/name) is required with files` | `check_changes` with `files` but no `repo` | | `send the changed files: files: [{path, content}] with repo (this server cannot read your disk)` | `check_changes` with `dir` (or with neither) on the hosted server or `staleproof serve`: only the stdio server reads a checkout | | `No live symbol in the "<graph>" graph is named <symbol>.` | `docs-for` (or `docs_for_symbol`) found nothing: check the spelling, the graph (`--repo` or `--org`; a bare name with neither uses `falkordb`), and that the repository is indexed | | `git <arguments> failed in <directory>: …` | The directory is not a git checkout, or `--base` names a ref that does not exist there ([git failed](#git-failed)) | ### Docs audit | Message | Meaning | |---|---| | `not indexed in the "<graph>" graph: <repos>` | `audit --no-index`, or `audit_docs`, for repositories the graph does not hold. Drop `--no-index`, or index them | | `unknown --format <x> (text, markdown or json)`, `--run must be a whole number, got <x>`, `--limit must be a whole number, got <x>`, `--fail-under must be a score from 0 to 100, got <x>` | A bad option value; nothing ran | | `health score <s> is below --fail-under <n>`, `<n> example(s) no longer fit the code (--fail-on-mismatch)` | Not errors: the reason `audit` exits 1 after printing the report | | `… Indexed by an older Staleproof: examples are not checked against the code until the repositories are indexed again (staleproof index).` | The graph was written before Staleproof 0.3, so it has no `MISMATCH` edges or export flags and the audit cannot judge the examples. Index the repositories again | | A grey `docs health: not audited` badge | The server behind the badge URL has not indexed the repository. Audit it there once ([the page](https://staleproof.dev/#/audit)); the badge is cached for 5 minutes | | `Could not open the audit issue (the job needs issues: write). The report is in the job summary.` | The GitHub Action's workflow lacks `permissions: issues: write` | ### Agent and models #### Agent mode without credentials ```text the agent needs model API credentials: set ANTHROPIC_API_KEY (or run `ant auth login`) for the Claude API, or ANTHROPIC_BASE_URL, ANTHROPIC_API_KEY and STALEPROOF_MODEL for an Anthropic-compatible provider such as DeepSeek. Use `check --no-agent` to run the graph tools without a model. ``` `check` runs agent mode by default and found no credentials ([where it looks](configuration.md#model-agent-mode)). Set a key, or run `check --no-agent`. The same message with the API's own error appended in parentheses means the key was found but rejected; the check then ends with status `error`. On the server, the equivalent is `409 {"error": "agent mode needs model API credentials on the server (ANTHROPIC_API_KEY); graph mode works without them", "agentUnavailable": true}` for `mode: "agent"`. Graph mode needs no credentials. A hosted server also limits agent checks per day: `429 {"error": "today's hosted agent checks are used up; graph mode still works, and the agent runs anywhere with your own key (npx -y staleproof check <pr-url>)", "agentUnavailable": true}`. The count resets at midnight UTC. #### A Claude model name on a compatible provider ```text "claude-sonnet-5-5" is a Claude model name, but the model endpoint is an Anthropic-compatible provider, which would answer with its own model under that name; set STALEPROOF_MODEL to the provider's model (e.g. deepseek-v4-pro) ``` `ANTHROPIC_BASE_URL` points at a host other than `api.anthropic.com`, and the model name starts with `claude-` (here the default, because `STALEPROOF_MODEL` is unset). Set `STALEPROOF_MODEL` (or `--model`) to the provider's model, for example `deepseek-v4-pro`. The server still reports `agent.available: true` in this state, and every agent check fails with this message as an `error` event. If `ANTHROPIC_BASE_URL` is not a valid URL at all, the server treats it as a compatible provider and `GET /api/health` answers 500 (`Invalid URL`). #### The check stopped early These are summaries of checks that ended `incomplete`, not errors; the sections still open are listed in `missing`: - `Stopped after 40 model turns with <n> impacted section(s) still open.` Raise the limit with `check --max-steps <n>` (the server always uses 40). - `Stopped at this server's 240 s time limit after <n> model turns with <m> impacted section(s) still open; the CLI has no time limit.` The server's `STALEPROOF_CHECK_BUDGET_MS` (240 s by default on Vercel) ran out. Run large checks from the CLI. - A check where the model finished with sections still open, after two reminders. Each reminder appears as a `nudge` event. `The model declined to continue (<category>).` is the summary of a `refused` check: the model stopped with a refusal. #### A tool call was cut off `The check failed: a tool call was cut off at max_tokens; raise max_tokens and re-run the check` A model turn hit its output limit (32,000 tokens) in the middle of a tool call. The CLI has no option for the limit; the library's `AnthropicModel` takes `maxTokens`. Re-running often succeeds, because the model chooses differently. #### Errors the model sees and recovers from These come back to the model as error results, so a check continues; they also appear in the trace and, over MCP, to your client: | Message | Meaning | |---|---| | `unknown tool <name>` | The tool does not exist | | `<tool> has no parameter <names>; its parameters are <list>` | An argument the tool does not declare. The agent loop and the MCP server both refuse it instead of dropping it | | `invalid input for <tool>: <path>: <message>` (MCP: `invalid input: ...`) | The input failed the tool's schema; `give section_id or section_ids` is the `record_verdict` rule | | `<pr> has not been checked yet; call impact first` | `run_example` with a `pr_url` the graph has not ingested. Call `impact` first; `staleproof run --pr` does it for you | | `no code example <id>` | No example with that id (`owner/repo:path:index`) | | `refused: no path in the graph from <pr>'s changes to <section>. Staleproof never edits a paragraph without a path; record_verdict instead if it needs no change.` | `propose_edit` on a section the pull request does not reach within 3 call hops | | `new_text is identical to the current section; use record_verdict with ok` | The proposed edit changes nothing | | `a governed verdict must cite the decision id(s)` | `record_verdict` with `governed` needs `cites` | | `pr_url is required` | `propose_edit` or `record_verdict` without a pull request. In agent mode it defaults to the pull request under check; over MCP and in the library pass it | | `no doc section <id> or pull request <pr> in the graph` | The section id or the pull request is unknown | | `none of <ids> is a DocSection or Symbol id; decision not recorded` | `write_decision` (or `decide`, or `POST /api/decisions`) with no valid id in `governs` | ### Example runs A run that cannot be prepared is recorded with status `error`, and `run_example` reports `The sandbox could not be prepared: <first line of the reason>`. Error runs are never answered from the cache, so the next request tries again. #### Scratch server is the org server ```text refusing to flush <host>:<port>: it is the server holding the org graph (FALKORDB_URL). Point STALEPROOF_SCRATCH_URL / STALEPROOF_SCRATCH_PORT at a separate server. ``` Every run starts by flushing the scratch database, and its host and port equal those of `FALKORDB_URL`. Staleproof refuses rather than delete your graph. Unset `STALEPROOF_SCRATCH_URL` to use the managed scratch server on port 6390, or point it at another server. #### Install dependency conflict ```text dependency conflict: <pin> requires <requirement>, which the other checkouts don't satisfy (installing <repo@sha>, <repo@sha>) ``` The repositories an example needs cannot be installed together at those commits. On FalkorDB's repositories, `graphrag-sdk==1.4.0` at its indexed commit requires `falkordb[hiredis]>=1.8.0,<2`, which the falkordb-py checkout under test does not satisfy, so three examples of falkordb-py#276 could not be run ([FACTS](FACTS.md#example-runs-sandbox)). The pull request is not to blame: it is version skew between the checkouts. There is nothing to fix in Staleproof; the agent records such sections as `skipped`. Other install failures read `installing <repo@sha, ...> failed:` followed by the end of the installer's output, and a failed virtualenv reads `uv venv failed: ...` or `python3 -m venv failed: ...`. #### Network refused in the sandbox `Staleproof sandbox: outbound network is disabled (tried <host>:<port>); only FalkorDB on localhost:6379 is available` `Staleproof sandbox: DNS is disabled (<host>)` The example tried to reach a server other than FalkorDB on `localhost:6379` (which the sandbox redirects to the scratch server). The run is recorded as `fail` with this error. Examples that call external services are not runnable here; when the code mentions `requests.get(`, `httpx.post(`, `boto3`, `neo4j` and similar, it is skipped before it runs with `calls an external service`. #### Timeouts A run that does not exit within `STALEPROOF_RUN_TIMEOUT_MS` (30 s by default) is killed and recorded as `timeout` with the reason `no exit after 30s`. Raise the variable for slow examples. Creating a virtualenv and installing into it have fixed limits of 300 s and 600 s. #### Not executed on this host `this deployment cannot execute examples (no git/uv sandbox runtime); runs recorded by other Staleproof hosts are answered from the graph` The host lacks `git`, or both `uv` and `python3` (Vercel, for example). Recorded runs for the same program and commits are still answered. Run examples from a host that has them (`scripts/seed.sh`, `staleproof run`) to record them on the graph. #### An async example fails with a SyntaxError `SyntaxError: 'await' outside function`, or `'async with' outside async function` Since 0.2.0, examples with a top-level `await`, `async with` or `async for` run on an event loop ([Async examples](example-runner.md#async-examples)). A run recorded before that has the same program and commits, so it is answered from the graph with its old error. Re-run it with `staleproof run <example-id> --pr <url> --fresh`. #### Skipped runs A skipped run is not an error: it says why the example cannot prove anything. The reasons are `not Python (<lang>)`, `empty example`, `needs an LLM provider (<match>); Staleproof does not hold provider keys`, `reads a local file the docs do not provide (<file>)`, `contains placeholders to fill in (e.g. "<your-host>")`, `waits for user input`, `calls an external service`, `uses no symbol from an indexed repository`, and `fragment: <error> (the page never defines it)` for a fragment that uses a name nothing on its page defines. [Example runner](example-runner.md) has the exact rules. ### Docs patch #### Not checked `<url> has not been checked` `patch` (or `GET /api/patch`, or `docs_patch`) for a pull request the graph has never ingested. Run a check first. #### No proposed edits `patch` prints `(no proposed edits)` when no section has a `stale` verdict with new text that a maintainer has not rejected. Graph mode records no verdicts, so the patch of a graph-mode-only pull request is always empty. On falkordb-py #262, #267 and #276 the patch is empty too: every verdict the agent recorded on them is `ok`. #### Edits not applied The patch description lists them under "Not applied (file changed since indexing)", and the API under `conflicts`: - `<path> changed since it was indexed at <sha7>`: the section's text at the indexed commit differs from the text the verdict was written against, usually because the docs repository was re-indexed after the check. Check the pull request again so the edit is proposed against the current text. - `<path> not found in <repo>@<sha7>`: the file is missing at the indexed commit. - A section longer than 8,000 characters always conflicts: its stored text is cut at 8,000 characters, so it never matches the file. #### Opening the pull request - `opening a pull request needs GITHUB_TOKEN`: set `GITHUB_TOKEN` or `GH_TOKEN`. - `the patch is empty: no proposed edits to open a pull request for`: there is nothing to open. - `fork <owner/name> did not become available`: GitHub did not finish creating the fork within 30 polls 2 seconds apart. Run the command again; the existing fork is reused. - `--upstream and --in-repo are different targets: pick one`: pass one of them. - `no pull request opened: no proposed edit is in a repository this token may write to`: with `--in-repo --only <repos>`, every proposed edit is in another repository. Open those with `patch --open-pr` from an account that can write to them. - `Could not open the docs fix pull request (the job needs contents: write and pull-requests: write).`: the GitHub Action's `open-docs-pr` step, without those permissions (or on a branch the token cannot push to). ### Hosted service These come from a server in hosted mode (staleproof.dev, `serve --hosted`): as an `error` event in the stream of `POST /api/check` or `POST /api/index`, as an MCP error result, or as `400 {"error": ...}` elsewhere. The limits are in [Deploy](DEPLOY.md#limits). | Message | Meaning | |---|---| | `the hosted workspace gh-<owner> would hold <n> repositories (limit 4: <repos>). Run Staleproof yourself for bigger organisations: npx -y staleproof index <repos>` | A workspace holds at most 4 repositories, counting the ones already in it | | `<repos> came to <mb> MB of graph, over the hosted limit of 20 MB. Run it on your own FalkorDB: npx -y staleproof check <pr-url>, or the GitHub Action.` | The indexed workspace was too big; it was dropped | | `the hosted service is full right now. Run Staleproof on your own machine or in CI instead: npx -y staleproof check <pr-url>, or the GitHub Action.` | Evicting the least recently used workspaces did not bring the server under 70 MB | | `the hosted service has indexed 500 workspaces today, its daily limit. Run it yourself: npx -y staleproof check <pr-url>` | The daily indexing budget is spent; it resets at midnight UTC | | `gh-<owner> is being indexed by another request; try again in a minute` | Another request held the workspace's lock for more than 150 seconds | | `<owner/repo> is in the curated "<graph>" graph, which the hosted service does not re-index; index other repositories on their own.` | `with` named repositories for a repository of the curated graph | | `<owner/repo> is too large to index from an archive here (...)` | The archive is over 150 MB or has more than 3000 files the indexer reads ([above](#tarball-download-failed)) | | `name a repository: pass a pull request URL (pr_url), repo or repos ("owner/name"), or ids that start with owner/repo:` | MCP: the call names no repository, so the hosted server cannot pick a graph | | `<owner/repo> is not indexed yet: call check_pull_request with one of its pull requests first` | MCP: a tool that does not index on first use (for example `read_section` or `docs_patch`; the indexing ones are `check_pull_request`, `impact`, `clearance`, `check_changes`, `docs_for_symbol` and `audit_docs`) for a repository no workspace holds | | `404 this server indexes from the CLI only: staleproof index <owner/repo>...` | `POST /api/index` on a server that is not in hosted mode | ### Server and web app | Status and message | Meaning | |---|---| | `429 rate limited: try again later` | Over 30 POST requests from one client in the last hour ([rate limits](http-api.md#rate-limits)) | | `404 no such check`, `no such section`, `no such verdict` | The id is unknown in this graph (in hosted mode, in every graph); check `--org` | | `404 <owner/repo> is not indexed` | Hosted mode: `GET /api/patch` or `POST /api/decisions` for a repository no graph holds | | `404 <owner/repo> is not indexed yet: POST /api/audit {"repo": "owner/name"} indexes and audits it` | `GET /api/audit` for a repository no graph holds (without hosted mode: `…: staleproof index <owner/repo>`) | | `400 symbol required, e.g. ?symbol=Graph.query&repo=owner/name`, `400 repo required (owner/name), or a symbol id that starts with owner/repo:` | `GET /api/docs-for` without `symbol`, or in hosted mode without a repository | | `400 id required` | `GET /api/sections` without `id`. `GET /api/patch` without `pr` answers `400 not a GitHub pull request URL: ""…` | | `400 status must be accepted or rejected` | The review body needs `status` | | `400 decision text is limited to 500 characters` | The HTTP API caps decision text; the CLI `decide` does not | | `503 the Staleproof API is not connected to FalkorDB (<message>); set FALKORDB_URL` | Vercel only: the function cannot reach FalkorDB | The CLI's `verdict` command reports `action must be accept or reject` and `no verdict <id>`. The web app shows `the Staleproof API is unreachable and no snapshot is bundled` when the API fails and there is no snapshot for that page, and `Live checks need the Staleproof API connected to FalkorDB; this deployment is showing a snapshot. Run it locally: npx staleproof serve` when you try to run a check or record a decision in [snapshot mode](http-api.md#snapshot-mode). ## FalkorDB version notes Staleproof has run on two FalkorDB versions: 4.18.3 (the falkordblite build that `staleproof db start` and the test suite use) and 4.22.0 (FalkorDB Cloud, where [FACTS.md](FACTS.md) was generated). Each has a traversal behaviour the queries are written around. - **4.18.3: an unbound variable-length pattern can drop nodes.** When a node is reachable along two routes of different length, a pattern such as `(err)<-[:CALLS*0..3]-(t)` that binds no path variable can miss some of the callers. In GraphRAG-SDK, `RetrievalStrategy.search` calls `RetrieverError` directly and through `_validate`, and the pattern found 4 of the 6 callers. Clearance then used `WHERE NOT (pattern)` and "proved" a symbol undocumented while the docs' quickstart was three calls away. Binding the path (`MATCH p = (s)<-[:CALLS*0..3]-(t)`) gives the right answer on both versions, so every variable-length pattern in Staleproof binds one, and clearance counts bound paths (`OPTIONAL MATCH p = ... WITH count(p) AS reach WHERE reach = 0`) instead of using a pattern predicate. - **4.22.0: an exact length of 2 or more toward a labelled node returns nothing.** For example `MATCH (:N {name: 'd'})<-[:R*2]-(t:N)` returns no rows on the chain `a->b->c->d`. No Staleproof query uses an exact length: every range starts at 0 or 1. [test/traversal.test.ts](../test/traversal.test.ts) reproduces the GraphRAG-SDK shape and fails if any query reintroduces an unbound variable-length pattern or a range that does not start at 0 or 1. Follow the same two rules in your own Cypher. Other version-dependent behaviour: - `rank_sections` calls `algo.pageRank`; on any error it falls back to counting inbound links per page and reports `method: "in-degree"`. - Between example runs the scratch server is cleared with `FLUSHALL` and `GRAPH.UDF FLUSH`; servers without UDF support reject the second command, and that error is ignored. - `GET /api/health` reads the version with `MODULE LIST` and shows `unknown` when the server does not allow it. - [docker-compose.yml](../docker-compose.yml) uses `falkordb/falkordb:latest` unless `FALKORDB_IMAGE` is set, so the version you get there has not been tested; pin `FALKORDB_IMAGE` for reproducible runs. ## Limits Stated plainly; most are "not yet" rather than "never". **What is analysed** - **Python, TypeScript and JavaScript code.** Python is read anywhere in a repository; TypeScript and JavaScript only inside a published npm package (a `package.json` with a `name`, not `private`). Other languages are not parsed, and notebooks are not read. - **TypeScript and JavaScript get name checks, not argument checks.** JavaScript accepts any argument count, so their examples are flagged only for names a package does not export and methods a class does not have, and a signature change predicts no JavaScript break. JavaScript and TypeScript examples are linked and checked but never run. - **reStructuredText is scanned, not built.** Titles, code directives (`code-block`, `testcode`, `doctest`, …), `::` literal blocks, `>>>` doctests, API roles and `:doc:`/`:ref:` links are read. `.. include::`, autodoc output (`.. autofunction::` and the like), substitutions and Sphinx extensions are not expanded, and `:ref:` resolves only to labels on the same page. - **Static checks follow exact resolutions only.** An example call is judged only when it resolved through imports and types, to a function whose signature no unknown decorator wraps, on a class or module whose members are all visible, without `*args`/`**kwargs` splats. The rest is silent, so no mismatch and no predicted break is not proof that an example works; runs are. - **Static call resolution.** Calls are resolved through imports, re-exports, attribute types, return annotations and base classes. Where the types do not say, a method call falls back to a by-name edge when 1 to 4 functions of that name exist in repositories the module imports; such edges are labelled `by-name` everywhere. Generic names (`get`, `run`, `close`, ...) are never matched by name. - **Cross-repository resolution needs the repositories indexed together.** Edges between repositories are resolved only within one `index` command. Indexing a repository alone (including indexing on first use without `--with`, and the Action's `reindex` without `also-index`) drops its cross-repository edges until the organisation is indexed together again. - **Re-index semantics.** A re-index re-resolves calls and doc links and marks vanished symbols and sections `gone`. It also removes the late links that pull request ingestion created for removed symbols, and already-ingested pull requests are not ingested again (unless an open one moves). The Action's `reindex` uses the remote default branch head, not the base branch. - **Change detection.** `export-removed` is reported for any module that stops importing a name from its own package, not only `__init__.py`, so it can flag internal refactors. A moved or renamed file is reported as every definition removed and added, because renames are paired only within the same parent. - **Private repositories need a token.** The CLI, a server you run and the GitHub Action read private GitHub repositories with `GITHUB_TOKEN` (the Action's default token reads its own repository). The hosted site, staleproof.dev, indexes public repositories only. Nothing outside GitHub is read. **Runs** - **The sandbox is a guard, not a security boundary.** The example refuses network connections and DNS at the Python socket level, runs with no inherited environment and a temporary HOME, and its process group is killed on timeout. The install step before it runs the repositories' build code with network access and an allow-listed environment (paths, locale, proxies, certificates, `UV_*`, `PIP_*`; [details](configuration.md#example-runner)), without `FALKORDB_URL`, model keys or `GITHUB_TOKEN`. Run it inside a container for untrusted code. - **Runs use today's third-party dependencies.** The repositories are installed from source at the exact commits; their dependencies resolve at install time, so an old commit can fail for reasons unrelated to the pull request. A recorded run is reused for the same program and commits even if the dependencies have moved since; `staleproof run --fresh` re-runs it. - **Without uv, Python is not pinned.** The fallback uses whatever `python3` is installed. - **Graph mode samples.** It ranks the first impact page (up to 40 sections) and runs at most 3 exactly-linked examples on it. It writes no verdicts. **Checks and patches** - **Proposed edits are tested on fixtures, not shown on the live graph.** The agent proposing a docs fix with its evidence path is tested on the project's fixture organisation. On FalkorDB's own repositories, every verdict on falkordb-py #262, #267 and #276 is `ok`: those pull requests fixed code, and no edit was proposed. - **Model paths.** The agent was written for Claude; the Claude API path is tested against a local fake API only. It has run on `deepseek-v4-pro` through DeepSeek's Anthropic-compatible API. The GitHub Action's pull request check has run end to end on a private test repository; the Docker image is written but has not been built. - **An agent may edit an already-broken example only when it proved it.** Besides what a pull request falsified, the agent may fix an example that it ran and that fails both before and after the pull request, saying so in the reason and citing both runs. The path guard still applies: no edit without a path from the pull request's changes to the section. - **Sections over 8,000 characters always conflict in the patch**, because their stored text is truncated. - **The hosted service runs no examples.** Checks and audits on staleproof.dev and its MCP endpoint report impact, predicted breaks, clearance, ranking and the static checks; runs come from the GitHub Action, the CLI or a local MCP server. - **Checks before a pull request are static.** `check` without a URL, `check_changes` and `docs-for` predict and trace; they run no example and record nothing. - **Pull requests are re-read only when they move.** A later check reuses the stored diff unless the pull request is open and its head changed (or it was merged since); `impact` and the other tools alone never re-read it. The library's `ingestPullRequest` always re-ingests. The GitHub Action's default FalkorDB starts empty on every run, so there each run diffs the pull request afresh. - **Verdicts written over MCP** have no check id, so no check counts them. - **Verdict ids are not printed** in the CLI's text output or in the pull request comment; they are in the trace (`check --json`, `GET /api/checks/<id>`). - **The HTTP API has no authentication**, and its rate limit is in memory, per process. ## FAQ ### Which languages does it support? Code in Python, and TypeScript and JavaScript in published npm packages; docs in Markdown, MDX and reStructuredText (Sphinx) from any repository. Python examples are linked to code, checked against its signatures and run; JavaScript and TypeScript examples are linked and checked for names that do not exist, but not run. Other languages are not parsed. ### Do I need an LLM API key? No. Indexing, graph-mode checks (`check --no-agent`), example runs, the patch, the MCP server and the HTTP API's graph mode need no key. Over MCP, your own client's model (Claude Code, for example) drives the tools. Only agent mode, where Staleproof's loop calls a model, needs credentials. ### Which models can run the agent? The agent was written for Claude: the default model is `claude-sonnet-5-5`, with adaptive thinking and the server-side refusal fallback. It runs on any Anthropic-compatible API through `ANTHROPIC_BASE_URL`, sending only the portable request fields; the live demo runs `deepseek-v4-pro` through DeepSeek's Anthropic-compatible endpoint. The Claude path is tested against a local fake API only. Any model works through the library by implementing `ModelClient` ([library](library.md#agent-and-checks)). ### What does a check cost? There is no published figure. Every agent check records the tokens it used: the CLI prints the total at the end, the `check_finished` event has `usage` (input, output, cache reads), and the `Check` node stores `input_tokens` and `output_tokens`. Multiply by your provider's prices. Spending is bounded by the turn limit (40 by default), the output limit per turn (32,000 tokens) and the size of each tool result handed back (24,000 characters). Graph mode uses no model and costs nothing beyond your FalkorDB and GitHub usage. ### What is sent to the model provider? In agent mode: the system prompt, the tool definitions, the pull request URL, a one-sentence summary of what changed in the graph since the previous check (pull request URLs, counts, and the text of new decisions), and every tool result the model asks for. Tool results contain the text of doc sections and their code examples, symbol names and signatures, evidence paths, example run results (status, the last line of the error, the end of the traceback and of the output), and decisions with their text, author and source. All of it comes from the public repositories you indexed, plus what maintainers wrote into decisions. Your `GITHUB_TOKEN`, model key and other environment variables are never part of a tool result; example processes inherit no environment, and the install step that builds the repositories gets only an allow-list without those secrets. Graph mode sends nothing to a model. ### Does it open pull requests on its own? No. A check writes verdicts and proposed edits to the graph only. `staleproof patch` prints the diff and the description; `patch --open-pr` opens the pull request, and only when you pass that flag: in your fork by default, against the original repository with `--upstream`, or as a branch of the docs repository itself with `--in-repo`. The GitHub Action posts or updates one comment, keeps one audit issue in audit mode, and opens a docs fix pull request into the checked pull request's branch only with `open-docs-pr: 'true'`. ### How long does a check take? The impact query itself takes about 1 to 2 ms server-side on FalkorDB 4.22 ([FACTS](FACTS.md#impact-query-timing-server-side-falkordb)). The rest depends on the model and on example runs. A 43-section agent check takes about 6 minutes; the server on Vercel stops agent checks at 240 s, and the CLI has no time limit. Each example run is limited to 30 s by default, and the first run against a new set of commits also builds a virtualenv. ### Can I use private repositories? Yes, with the CLI, a server you run and the GitHub Action. Set `GITHUB_TOKEN` (or `GH_TOKEN`) to a token that can read them: it is sent to the GitHub API, to `raw.githubusercontent.com` and, through git's environment configuration, to git clones of github.com ([how](configuration.md#github)). In the Action the workflow's default token already reads its own repository. The hosted site, staleproof.dev, indexes public repositories only. ### Do I need to set up FalkorDB? No. The CLI starts a local FalkorDB (falkordblite) when no `FALKORDB_URL` is set and nothing answers on `localhost:6379`; the GitHub Action starts one inside the job; the website and the hosted MCP server bring their own. Set up your own only to keep one graph for a team, as in [Deploy](DEPLOY.md). ### Can one server hold several organisations? Yes. Each organisation is its own graph on the same FalkorDB server, selected with `--org` or `STALEPROOF_ORG` (the CLI defaults to the repository's owner); a second graph built from encode/httpx sits next to `falkordb` on the same server. Graphs never share edges, verdicts or decisions. One `staleproof mcp` process, or `staleproof serve` without `--hosted`, serves one organisation; run one per organisation. `serve --hosted` answers for every owner from one process, indexing each into its own workspace graph on demand. ### How do I measure precision? Review verdicts: `staleproof verdict accept <verdict-id>` or `verdict reject <verdict-id>` (with `--by <name>`), or `POST /api/verdicts/<id>/review`. Each review sets the verdict's `status`, `reviewed_by` and `reviewed_at`, and a rejected edit is left out of the patch. Verdict ids are in the check's trace: `check --json` or `GET /api/checks/<check-id>`, in the `tool_result` of `propose_edit` (`data.id`) and `record_verdict` (`data.ids`). Count reviews by verdict with Cypher: ```cypher MATCH (v:Verdict) WHERE v.status IN ['accepted', 'rejected'] RETURN v.verdict AS verdict, v.status AS status, count(v) AS n ORDER BY verdict, status ``` The [library](library.md#run-your-own-cypher) shows how to run it from Node.js. ### Why FalkorDB? Because the question is a graph traversal. Impact is a variable-length path from a changed symbol through callers in any repository to a doc section; clearance is the same pattern counted to zero, a proof that no doc is reachable; decisions govern symbols transitively through `CALLS*0..2` and resolve through `SUPERSEDES` chains; ranking runs PageRank over the docs' links. FalkorDB runs the impact query in about 1 to 2 ms server-side, and its multiple graphs per server give one graph per organisation. Every statement is in [Graph model and Cypher](GRAPH-MODEL.md). ### Does a re-check notice new commits on the pull request? Yes, since 0.3.0. Before a check (`staleproof check`, `POST /api/check`, MCP `check_pull_request`) of an open pull request, Staleproof compares its head with the one it ingested; when it moved, the old diff is dropped and the current one is read, and the CLI logs `<pr> has new commits since it was last checked: reading its current diff`. `impact` and the other tools on their own reuse the stored diff, so start with a check. A merged pull request is never re-read. Example runs are still answered from recorded runs of the same program and commits; `staleproof run --fresh` re-runs one. ### Can I check changes before opening a pull request? Yes, statically. `staleproof check` with no URL checks the working tree of the checkout you are in, and `--fail-on-predicted` makes it a pre-push hook; `staleproof docs-for <symbol>` lists the docs that depend on a function before you change it; over MCP, `check_changes` takes edited files and `docs_for_symbol` a symbol. They predict which examples break; running examples before and after needs the pull request ([CLI](cli.md#check-without-a-pull-request)). ### Why is an example I know is wrong not flagged? The static checks report only what the source settles: a call reached by name rather than through imports and types, a function behind an unknown decorator, a class with an external base or a `__getattr__`, or a call with `*args`/`**kwargs` is left alone, and JavaScript calls get no argument checks ([the rules](indexing.md#static-checks-of-examples)). Run the example (`staleproof run <example-id>`, or `audit --run <n>`) to prove it, and check that it is not marked `skip` or matched by `skip-examples`. ## Related - [Configuration](configuration.md): every variable and its default - [Deploy](DEPLOY.md): hosts and the production checklist - [Example runner](example-runner.md): the sandbox, skip rules and run outcomes - [The agent](agent.md): the loop, its stop condition and model providers - [HTTP API](http-api.md)