Overview¶
Why aveloxis-cve exists¶
Aveloxis collects open source project data, including every commit and an scc complexity snapshot. Two of its limits matter for security research:
It walks only the default branch. Many projects fix CVEs on maintenance branches (release, stable, LTS) by cherry-pick or manual backport, and some fixes never reach the default branch at all. Those fixes are invisible in Aveloxis's tables.
It keeps complexity only at HEAD.
aveloxis_data.repo_laborholds the latest scc snapshot, so it cannot show how a codebase's complexity changed over time.
aveloxis-cve is a sidecar that fills both gaps for a chosen subset of the repositories Aveloxis already collects. It writes only its own schema, aveloxis_cve, in the same database.
The questions it answers¶
Which CVEs has a project fixed, and by which commits, including copies on other branches?
On which branches did each fix land, and when?
How long do backports take, and which maintained branches never got a fix, or lost one to a revert?
Who fixes CVEs, and what code do the fixes touch?
How did size and complexity rise or fall over time, window by window?
Which languages, files and changes drive that?
How do CVE fixes relate to complexity: where do they land in the series, and are fixed files more complex?
The analysis guide maps each question to tested queries.
What it records¶
Area |
Tables |
What it holds |
|---|---|---|
Tracked set |
|
The repositories you chose, with their last sync and labor outcomes |
Commits |
|
Every commit reachable from any branch, live or archived, with per-file counts. Commit metadata includes CVE and GHSA ids from the message, cherry-pick and revert references, the patch-id and the resolved Aveloxis contributor. |
Branches |
|
Every branch, plus archived tips of deleted and force-pushed branches. Full membership: one row per branch and commit, with when the change landed on that branch. |
CVE links |
|
Fixes, their copies, their reverts, and when they reached each branch |
Rolling labor |
|
Sliding windows on the default branch. Each has a full scc snapshot at its end, with Aveloxis's |
Operations |
|
Recompute requests, graceful stop requests, the migration ledger |
The model is drawn in schema.md, and every column is described in data.md.
How it works¶
flowchart TB
subgraph aveloxis["Aveloxis database (read only)"]
repos["aveloxis_data.repos<br/>aveloxis_ops.collection_queue"]
contributors["contributors, aliases,<br/>identities, commits"]
end
forge[("forge<br/>(GitHub, GitLab, ...)")]
subgraph sidecar["aveloxis-cve"]
add["repos add"]
sync["sync"]
resolve["resolve"]
labor["labor"]
end
clones[("own bare clones<br/>clone_dir/repo_ID.git")]
scc[["scc"]]
schema[("schema aveloxis_cve")]
repos --> add --> schema
repos -- "readiness, clone URL" --> sync
forge -- "clone / fetch" --> clones
sync --- clones
sync -- "branches, commits, membership,<br/>links, patch-ids, file counts" --> schema
contributors -- "author identity" --> sync
contributors -- "author identity" --> resolve
resolve -- "commits.cntrb_id" --> schema
clones --> labor
scc --> labor
labor -- "windows, snapshots,<br/>changed files" --> schema
run does sync, then resolve, then labor, for each repository whose sync succeeded.
sync¶
For each tracked repository that Aveloxis has finished collecting:
Clone it from the forge into aveloxis-cve's own bare clone, or fetch into that clone. aveloxis-cve never reads, seeds from or writes Aveloxis's clones.
Archive the tip of any branch deleted or force-pushed upstream under
refs/cve-archive/<deleted|forced>/<time>/<name>, so history on end-of-life and rewritten branches stays.Ingest every new commit reachable from any branch or archived tip, with its files, CVE and GHSA ids, cherry-pick and revert references.
Resolve cherry-pick and revert references to commits, and recompute which commits undo a CVE fix.
Record branches and full membership. Each commit's
landed_aton a branch is when it reached that branch's first-parent history.Resolve authors to Aveloxis contributors from Aveloxis's own data, and compute patch-ids and Aveloxis-parity file counts.
resolve¶
Re-runs author resolution for commits still unresolved, or for every commit with --all, as Aveloxis's contributor data improves. A changed resolution marks the affected labor windows for recompute.
labor¶
On each synced repository's default branch:
Plan windows from the first dated landing. Each window is
window_dayslong (60), and window ends advancestep_monthsat a time (1). Only complete windows are planned.For each window not yet stored, or marked stale, take the default-branch commit at its end. List the tree, apply scc's own checkout walk, and measure each file not already in the shared
scc_blob_cache.Store the window, its full snapshot, and the files changed by commits that landed in it, with start and end metrics, in one transaction.
Key concepts¶
Concept |
Meaning |
|---|---|
Full membership |
Each branch row holds its whole reachable history, not just commits unique to it. |
|
When a commit reached a branch: the committer time of the first-parent commit that brought it in. It never runs backwards along a branch, and implausible future clocks are capped. |
Archived ref |
A tip of a deleted or force-pushed branch, kept so its history (and fixes on it) is never lost |
Fix match |
A commit fixes a CVE when its message names it, when it is a resolved cherry-pick of one that does, or when it has the same patch-id as one that does (a manual backport). |
Undo |
A commit that names a CVE and reverts its fix (an odd revert chain). It is not a fix. |
Window |
|
Count scheme |
Which files a snapshot counts. Currently scc 3.7.0's checkout walk, as Aveloxis's |
Stale window |
A window marked |
Design rules¶
Zero coupling with Aveloxis.
Aveloxis does not know aveloxis-cve exists.
aveloxis-cve reads Aveloxis's schemas through one package (
internal/source) and writes onlyaveloxis_cve. It readsaveloxis_dataandaveloxis_opstoday. The rolesql/roles.sqlcreates may also readaveloxis_scanandaveloxis_augur_data.No foreign key, view, function or trigger in
aveloxis_cvereferences an Aveloxis object. Aveloxis ids are copied by value, soDROP SCHEMA aveloxis_cve CASCADEremoves the sidecar completely and Aveloxis's DDL is never blocked.No Aveloxis Go code is imported.
Only Aveloxis creates objects, indexes included, in its own schemas.
Independent clones. aveloxis-cve clones every repository itself from the URL Aveloxis records.
Readiness. A repository is analysed only after Aveloxis has collected it at least once.
Never fabricate identity. An ambiguous or unknown author stays unresolved, and contributors merged in Aveloxis are never chosen.
Safe concurrency. Every repository is worked on under a database advisory lock and a lock on its clone. Every write proves in its own transaction that the lock is still held. Several aveloxis-cve processes can therefore run at once without corrupting each other's work. aveloxis-cve stop asks all of them to finish their current repositories and exit.
Components¶
Package |
Role |
|---|---|
|
The binary's entry point |
|
Commands, flags, signal handling, output |
|
The JSON configuration and its defaults |
|
Embedded migrations, the migrate lock, the schema version |
|
The only reader of Aveloxis's schemas (readiness, selection, identity) |
|
Every write to |
|
Per-repository sync and labor, locks, workers, stop and interrupt handling |
|
git: clones, fetches, archiving, commit streams, graphs, patch-ids, trees |
|
Membership and landing-time computation |
|
CVE and GHSA id, cherry-pick and revert extraction from messages |
|
Window planning, scc's walk emulation, running scc |
|
Running git and scc, interrupts, credential redaction |
|
Source-level tripwires: what may import or write what |
|
Test fixtures: scratch databases and git repositories |
|
The DBA script that creates the least-privilege role |