Overview

Why aveloxis-cve exists

Aveloxis collects open source project data, including every commit and an scc complexity snapshot. Two of its limits matter for security research:

  • It walks only the default branch. Many projects fix CVEs on maintenance branches (release, stable, LTS) by cherry-pick or manual backport, and some fixes never reach the default branch at all. Those fixes are invisible in Aveloxis's tables.

  • It keeps complexity only at HEAD. aveloxis_data.repo_labor holds the latest scc snapshot, so it cannot show how a codebase's complexity changed over time.

aveloxis-cve is a sidecar that fills both gaps for a chosen subset of the repositories Aveloxis already collects. It writes only its own schema, aveloxis_cve, in the same database.

The questions it answers

  1. Which CVEs has a project fixed, and by which commits, including copies on other branches?

  2. On which branches did each fix land, and when?

  3. How long do backports take, and which maintained branches never got a fix, or lost one to a revert?

  4. Who fixes CVEs, and what code do the fixes touch?

  5. How did size and complexity rise or fall over time, window by window?

  6. Which languages, files and changes drive that?

  7. How do CVE fixes relate to complexity: where do they land in the series, and are fixed files more complex?

The analysis guide maps each question to tested queries.

What it records

Area

Tables

What it holds

Tracked set

tracked_repos

The repositories you chose, with their last sync and labor outcomes

Commits

commits, commit_files, commit_facade_files

Every commit reachable from any branch, live or archived, with per-file counts. Commit metadata includes CVE and GHSA ids from the message, cherry-pick and revert references, the patch-id and the resolved Aveloxis contributor.

Branches

branch_repos, branch_commits

Every branch, plus archived tips of deleted and force-pushed branches. Full membership: one row per branch and commit, with when the change landed on that branch.

CVE links

commit_cherry_picks, commit_reverts, commit_cve_undoes; views cve_fix_commits, cve_fix_propagation, commit_equivalence

Fixes, their copies, their reverts, and when they reached each branch

Rolling labor

repo_labor_windows, repo_labor_rolling, repo_labor_rolling_changed, scc_blob_cache

Sliding windows on the default branch. Each has a full scc snapshot at its end, with Aveloxis's repo_labor columns, and the files changed within it.

Operations

labor_recompute_requests, stop_requests, schema_migrations

Recompute requests, graceful stop requests, the migration ledger

The model is drawn in schema.md, and every column is described in data.md.

How it works

        flowchart TB
    subgraph aveloxis["Aveloxis database (read only)"]
        repos["aveloxis_data.repos<br/>aveloxis_ops.collection_queue"]
        contributors["contributors, aliases,<br/>identities, commits"]
    end
    forge[("forge<br/>(GitHub, GitLab, ...)")]
    subgraph sidecar["aveloxis-cve"]
        add["repos add"]
        sync["sync"]
        resolve["resolve"]
        labor["labor"]
    end
    clones[("own bare clones<br/>clone_dir/repo_ID.git")]
    scc[["scc"]]
    schema[("schema aveloxis_cve")]
    repos --> add --> schema
    repos -- "readiness, clone URL" --> sync
    forge -- "clone / fetch" --> clones
    sync --- clones
    sync -- "branches, commits, membership,<br/>links, patch-ids, file counts" --> schema
    contributors -- "author identity" --> sync
    contributors -- "author identity" --> resolve
    resolve -- "commits.cntrb_id" --> schema
    clones --> labor
    scc --> labor
    labor -- "windows, snapshots,<br/>changed files" --> schema
    

run does sync, then resolve, then labor, for each repository whose sync succeeded.

sync

For each tracked repository that Aveloxis has finished collecting:

  1. Clone it from the forge into aveloxis-cve's own bare clone, or fetch into that clone. aveloxis-cve never reads, seeds from or writes Aveloxis's clones.

  2. Archive the tip of any branch deleted or force-pushed upstream under refs/cve-archive/<deleted|forced>/<time>/<name>, so history on end-of-life and rewritten branches stays.

  3. Ingest every new commit reachable from any branch or archived tip, with its files, CVE and GHSA ids, cherry-pick and revert references.

  4. Resolve cherry-pick and revert references to commits, and recompute which commits undo a CVE fix.

  5. Record branches and full membership. Each commit's landed_at on a branch is when it reached that branch's first-parent history.

  6. Resolve authors to Aveloxis contributors from Aveloxis's own data, and compute patch-ids and Aveloxis-parity file counts.

resolve

Re-runs author resolution for commits still unresolved, or for every commit with --all, as Aveloxis's contributor data improves. A changed resolution marks the affected labor windows for recompute.

labor

On each synced repository's default branch:

  1. Plan windows from the first dated landing. Each window is window_days long (60), and window ends advance step_months at a time (1). Only complete windows are planned.

  2. For each window not yet stored, or marked stale, take the default-branch commit at its end. List the tree, apply scc's own checkout walk, and measure each file not already in the shared scc_blob_cache.

  3. Store the window, its full snapshot, and the files changed by commits that landed in it, with start and end metrics, in one transaction.

Key concepts

Concept

Meaning

Full membership

Each branch row holds its whole reachable history, not just commits unique to it.

landed_at

When a commit reached a branch: the committer time of the first-parent commit that brought it in. It never runs backwards along a branch, and implausible future clocks are capped.

Archived ref

A tip of a deleted or force-pushed branch, kept so its history (and fixes on it) is never lost

Fix match

A commit fixes a CVE when its message names it, when it is a resolved cherry-pick of one that does, or when it has the same patch-id as one that does (a manual backport).

Undo

A commit that names a CVE and reverts its fix (an odd revert chain). It is not a fix.

Window

(window_start, window_end] on the default branch; the first window is closed. Windows overlap.

Count scheme

Which files a snapshot counts. Currently scc 3.7.0's checkout walk, as Aveloxis's repo_labor counts them.

Stale window

A window marked needs_recompute after history, identity or a recompute request changed it. It is kept until its replacement is written.

Design rules

Zero coupling with Aveloxis.

  • Aveloxis does not know aveloxis-cve exists.

  • aveloxis-cve reads Aveloxis's schemas through one package (internal/source) and writes only aveloxis_cve. It reads aveloxis_data and aveloxis_ops today. The role sql/roles.sql creates may also read aveloxis_scan and aveloxis_augur_data.

  • No foreign key, view, function or trigger in aveloxis_cve references an Aveloxis object. Aveloxis ids are copied by value, so DROP SCHEMA aveloxis_cve CASCADE removes the sidecar completely and Aveloxis's DDL is never blocked.

  • No Aveloxis Go code is imported.

  • Only Aveloxis creates objects, indexes included, in its own schemas.

Independent clones. aveloxis-cve clones every repository itself from the URL Aveloxis records.

Readiness. A repository is analysed only after Aveloxis has collected it at least once.

Never fabricate identity. An ambiguous or unknown author stays unresolved, and contributors merged in Aveloxis are never chosen.

Safe concurrency. Every repository is worked on under a database advisory lock and a lock on its clone. Every write proves in its own transaction that the lock is still held. Several aveloxis-cve processes can therefore run at once without corrupting each other's work. aveloxis-cve stop asks all of them to finish their current repositories and exit.

Components

Package

Role

cmd/aveloxis-cve

The binary's entry point

internal/cli

Commands, flags, signal handling, output

internal/config

The JSON configuration and its defaults

internal/db

Embedded migrations, the migrate lock, the schema version

internal/source

The only reader of Aveloxis's schemas (readiness, selection, identity)

internal/store

Every write to aveloxis_cve, fenced by the holder's locks

internal/pipeline

Per-repository sync and labor, locks, workers, stop and interrupt handling

internal/gitx

git: clones, fetches, archiving, commit streams, graphs, patch-ids, trees

internal/branches

Membership and landing-time computation

internal/cve

CVE and GHSA id, cherry-pick and revert extraction from messages

internal/labor

Window planning, scc's walk emulation, running scc

internal/procx

Running git and scc, interrupts, credential redaction

internal/archtest

Source-level tripwires: what may import or write what

internal/testdb, internal/gittest

Test fixtures: scratch databases and git repositories

sql/roles.sql

The DBA script that creates the least-privilege role