Operations¶

How to install, run, upgrade, monitor, stop and remove aveloxis-cve. Every configuration key is in configuration.md, and every table in data.md.

Before you start¶

Aveloxis. A complete Aveloxis install that has finished at least one collection of each repository you will track, in the same PostgreSQL database.
Tools.
- Go 1.26 or later, to build.
- git. 2.45 or later lets clones use reftable refs, which hold branch names a file-based ref store cannot.
- scc. The snapshot walk emulates scc 3.7.0's, and parity with Aveloxis was checked with that version.
Disk. Space for a bare clone of every tracked repository under clone_dir.

Install¶

Build or install the binary:

go install github.com/aveloxis/aveloxis-cve/cmd/aveloxis-cve@latest

Have a DBA create the least-privilege role, as a superuser, in the Aveloxis database. The role reads Aveloxis's schemas, except the tables that hold secrets, and owns aveloxis_cve. See the header of sql/roles.sql and the README.
```
psql -d aveloxis -v ON_ERROR_STOP=1 \
     -v cve_role=aveloxis_cve -v cve_password="$AVELOXIS_CVE_DB_PASSWORD" \
     -v aveloxis_owner=aveloxis -f sql/roles.sql
```
Configure. Copy aveloxis-cve.example.json to aveloxis-cve.json, set the database block and clone_dir, then chmod 600 the file. To reuse Aveloxis's database settings instead, set aveloxis_config_path and delete the database block: Aveloxis's block is used only when this file has none. aveloxis-cve finds its configuration through --config, then $AVELOXIS_CVE_CONFIG, then ./aveloxis-cve.json.

Install the schema, and check everything:

aveloxis-cve init-db     # first installation only
aveloxis-cve doctor      # read-only checks; fix every FAIL

doctor checks:

tools and directories;
the connection;
every Aveloxis column aveloxis-cve reads;
that the role cannot write Aveloxis tables or read its secret tables;
the schema version;
whether Aveloxis's email indexes exist;
the readiness of each tracked repository.

Choose repositories¶

aveloxis-cve repos add --repo-id 123 --repo-id 132        # Aveloxis repo_id
aveloxis-cve repos add --url https://github.com/redis/redis
aveloxis-cve repos add --url-file urls.txt                # one URL per line; - reads stdin
aveloxis-cve repos add --repo-group-id 3                  # a whole Aveloxis repo group
aveloxis-cve repos add --query "SELECT repo_id FROM aveloxis_data.repos WHERE repo_name LIKE 'linux%'"
aveloxis-cve repos list
aveloxis-cve repos disable --repo-id 132                  # keeps its data; enable brings it back

Rules for repos add:

All or nothing. An unknown id or URL adds nothing.
Readiness. A repository Aveloxis has not finished collecting is added with a warning, and skipped by sync until it is ready.
--query. It runs read-only, with a 60-second timeout.

Run¶

aveloxis-cve run                       # sync, resolve, labor for every enabled repository
aveloxis-cve run --repo-id 123         # only this one (disabled repositories too)

The steps can also run alone:

Command	Does	When to run it alone
`sync`	Clone or fetch, then record branches, commits, membership, links, identities, patch-ids and file counts	To refresh commits without measuring
`resolve [--all]`	Re-resolve unresolved authors (`--all`: every author) from Aveloxis's contributor data	After Aveloxis improves its contributor data
`labor`	Compute windows not yet stored or marked stale	To measure without fetching

Order. run resolves before labor, so windows count newly resolved authors.

Schedule. Run aveloxis-cve run periodically, for example from cron after Aveloxis's collection cycle. A run only fetches what is new, and measures windows that completed since the last run or were marked stale.

First runs. The first sync clones, and the first labor measures every window since the repository's first commit. Both are slow for large repositories. The scc cache, keyed by file content, makes later windows and other repositories sharing content much faster.

Concurrency.

workers (default 2) sets how many repositories one process works on at once.
Several processes may run at once, even on different machines. A repository another process is working on is reported busy and skipped. The next run picks it up.

Exit status. A command exits non-zero when any repository failed. Each repository's outcome is printed, and its last error is kept in tracked_repos (aveloxis-cve status).

Reading results¶

aveloxis-cve status                                 # per repository: default branch, branches, commits, CVE commits, last sync
aveloxis-cve cves --repo-id 123                     # when each CVE fix landed on each branch
aveloxis-cve cves --repo-id 123 --cve CVE-2024-1234

For anything deeper, query the schema. The analysis guide has tested queries for every research question, and data-quality queries to check the data first.

Recompute¶

Situation	Command
You changed scc, or want every window of the current settings remeasured	`aveloxis-cve labor --recompute [--repo-id N]`
scc failed on some files (for example under a memory limit) and recorded them as not counted	Fix the cause, then `aveloxis-cve labor --retry-uncounted`
You changed `labor.window_days` or `labor.step_months`	Nothing: the next labor run starts a new series. The old series stays, unchanged.
History changed (a late push, a force-push) or an author's resolution changed	Nothing: the next labor run recomputes the affected windows.

Stop¶

aveloxis-cve stop

This records a stop request in the database:

Who stops. Every sync, labor, resolve and run already running, on any machine, finishes the repositories it is working on, starts no other, and exits.
What it reports. The rest are reported in one line: N repositories not started: stop requested. The exit status is 0 unless something failed. resolve --all exits non-zero and names the repositories to rerun.
Who does not stop. Commands started after the request carry on. stop returns at once; it does not wait for them.

Ctrl-C or SIGTERM interrupts a command immediately instead. The repositories in progress are cut short (sync records them as interrupted), the rest are reported not started: interrupted, and the next run redoes them.

Upgrade¶

Install the new binary.
Apply its schema changes:
```
aveloxis-cve migrate
```
Run aveloxis-cve doctor.

migrate applies only the migrations not yet applied, under a lock, and never touches Aveloxis's schemas. It refuses to run:

on a database where init-db never ran;
on a schema migrated by a newer aveloxis-cve.

Until you migrate, doctor fails with the schema version and says to run migrate, and commands that need a newer table fail with a missing-relation error (SQLSTATE 42P01). The troubleshooting table below covers it.

When Aveloxis itself is upgraded, nothing is needed: aveloxis-cve holds no object in Aveloxis's schemas. doctor tells you if a column it reads is gone.

Troubleshooting¶

Symptom	Meaning and fix
`repository not ready in Aveloxis`	Aveloxis has not finished a collection of it, has no commits for it, or marked it gone. Wait for Aveloxis.
`skipped: ... repository is being processed by another aveloxis-cve run` (busy)	Another process holds the repository's lock or its clone's lock. If no aveloxis-cve is running, a leftover git may still hold the clone's lock: `lsof <clone_dir>/repo_<id>.git.lock` names it. See `clone_dir` in configuration.md.
`warning: git maintenance of the clone failed`	`git gc --auto` failed or timed out. The sync still succeeded. See `git.timeout_seconds` in configuration.md.
labor `skipped: ... no complete default-branch membership; run sync first`	The default branch has no complete membership yet. Run `sync`.
`doctor` warns about missing `lower(email)` indexes	Author resolution reads Aveloxis's contributor tables in full. Upgrade Aveloxis to 0.29.141 or later, which creates them. Never create them by hand.
`doctor` fails on a readable secret table	Aveloxis created a secret-bearing table after `sql/roles.sql` ran. Re-run the script.
A schema error naming a missing `aveloxis_cve` table	The schema is older than the binary: run `aveloxis-cve migrate`.

Remove¶

To remove aveloxis-cve completely:

DROP SCHEMA aveloxis_cve CASCADE;   -- touches no Aveloxis object

Then delete clone_dir. To remove one repository's data, follow data.md.

Personal data¶

aveloxis-cve stores author and committer names and emails, commit messages, branch names, and links to Aveloxis contributors. data.md lists every such column and how to remove it.