OSS Validation

SwarmVault has a second release-validation lane that runs the published npm package against a small pinned corpus of public repositories.

This is deliberately different from the normal smoke flow:

  • smoke proves the core product flows quickly
  • the OSS corpus proves those flows on real repo shapes people actually work with

The default corpus stays intentionally small so provider cost and run time remain bounded.

SwarmVault also keeps a separate tiny controlled fixture matrix in the OSS repo. That matrix covers the core code-language baseline plus the local non-code file kinds the CLI consumes, and it runs through the same installed-package smoke path before broader public-repo validation.

That tiny matrix now also includes:

  • local ingest for the full Word family (.docx, .docm, .dotx, .dotm), Excel family (.xlsx, .xlsm, .xlsb, .xls, .xltx, .xltm), PowerPoint family (.pptx, .pptm, .potx, .potm), OpenDocument (.odt, .odp, .ods), Jupyter notebooks (.ipynb), plus Rich Text (.rtf), BibTeX (.bib), Org-mode (.org), and AsciiDoc (.adoc), each with extracted text and metadata sidecars
  • local .rst ingest with normalized headings and searchable extracted text
  • browser-style markdown and HTML inbox bundles with copied local assets
  • an optional packaged browser check that opens both graph serve and the exported HTML graph in headless Chromium and verifies selection, path highlighting, and deselection

Default Gated Repos

  • sindresorhus/ky: compact TypeScript library with source, tests, and docs
  • remarkjs/react-markdown: docs-heavy JS/TS repo with architecture and examples
  • pallets/itsdangerous: small Python library with code plus documentation
  • necolas/normalize.css: small web/docs repo with stylesheet-first content

Optional Canary

  • apple/sample-food-truck

That canary is not part of the default gated lane. It is there to exercise a mixed-language Apple-style project layout without slowing the normal release gate.

Run It

From the OSS repo:

pnpm live:cli-surface
pnpm live:oss:corpus

Maintainers publish the full release sequence with the same installed-package checks wired in:

pnpm release:publish -- --dry-run --skip-preflight
pnpm release:publish -- --otp <npm-2fa-code>

pnpm release:preflight also writes a compact evidence bundle under .release-preflight/:

.release-preflight/
├── summary.json
└── summary.md

The summary records every local gate, including the parser-backed direct CLI surface smoke, tarball install specs, browser-smoke status, OSS-corpus status, and the smoke artifact roots. If npm or ClawHub propagation lags immediately after publish, use the summary plus npm view @swarmvaultai/cli@<version> version --json to confirm whether to wait and resume with skip flags instead of publishing again.

Target one or two repos while iterating:

pnpm live:oss:corpus -- --repo ky --repo react-markdown

Include the canary explicitly:

pnpm live:oss:corpus -- --include-canary

Run the same flow against a provider-backed lane:

OPENAI_API_KEY=... pnpm live:oss:corpus -- --lane openai

What It Verifies

For each repo, the runner uses the installed CLI path and executes:

swarmvault init
swarmvault ingest <repo> --repo-root <repo>
swarmvault compile
swarmvault check-update
swarmvault benchmark
swarmvault graph query "<repo prompt>"
swarmvault graph stats
swarmvault graph validate --strict
swarmvault cluster-only
swarmvault tree --output <tree.html>
swarmvault merge-graphs state/graph.json state/graph.json --out <merged.json>
swarmvault query "<repo prompt>"
swarmvault chat "<repo prompt>"
swarmvault graph export --html <output>
swarmvault graph export --neo4j <output.cypher>
swarmvault export ai --out <output-dir>

It then checks:

  • source, page, and module-page counts
  • first-party source classification
  • benchmark output
  • graph query results
  • graph stats, compatibility update checks, cluster-only recompute, tree/merge aliases, and strict graph-artifact validation
  • saved query output
  • saved chat transcript and structured session state
  • wiki/graph/report.json
  • standalone graph export output plus the Neo4j/Cypher compatibility export
  • static AI export files, including llms.txt and graph.jsonld

Why It Uses Small Repos

The goal is repeatable release validation, not heroic one-off stress tests.

Keeping the default corpus small means:

  • lower provider costs
  • faster local reruns while fixing issues
  • clearer regression signals when behavior changes
  • fewer false failures from giant vendored or generated trees

The tiny matrix complements that by giving release validation a stable per-language and per-file-type baseline:

  • code baseline: JavaScript, JSX, TypeScript, TSX, Bash, Python, Go, Rust, Java, Kotlin, Scala, Dart, Lua, Zig, C#, C, C++, PHP, Ruby, and PowerShell
  • additional parser-backed language coverage is exercised separately for Elixir, OCaml, Objective-C, ReScript, Solidity, HTML, CSS, and Vue single-file components
  • local files: markdown, text, reStructuredText, HTML, PDF, the Word family (.docx/.docm/.dotx/.dotm), RTF, EPUB, CSV/TSV, the Excel family (.xlsx/.xlsm/.xlsb/.xls/.xltx/.xltm), the PowerPoint family (.pptx/.pptm/.potx/.potm), OpenDocument (.odt/.odp/.ods), Jupyter notebooks, BibTeX, Org-mode, AsciiDoc, structured config/data (JSON/YAML/TOML/XML/INI/ENV/PROPERTIES), images (including modern .heic/.avif/.jxl), and code

The browser-backed packaged lane is intentionally optional because it needs a local Chromium install, but when enabled it validates the same installed-package artifacts people actually run:

pnpm exec playwright install chromium
pnpm live:smoke:heuristic:browser

When a real bug is found in one of these repos, reduce it into a stable regression test or small derived fixture before treating the fix as done.