feat(scanning): continuous SBOM + image scanners via Deployments, unified jobs + retention #15

Merged

jonas merged 20 commits from feat/continuous-sbom-scanning into main 2026-06-26 10:55:04 +02:00

Conversation 0 Commits 20 Files changed 74 +4659 -1588

jonas commented 2026-06-24 13:54:36 +02:00

Owner

Copy link

Converts both vulnerability scanners from batch/CronJob models to long-running poller Deployments, unifies SBOM scanning into the jobs table, and centralizes grype into a single image — so repo runs, image scans, and SBOM scans share one queue//runs view, and grype runs in exactly one place. Continuous scanning also brings retention sweeps (to keep the now-constant artifacts bounded) and a batch of hardening fixes shaken out by the live lab deploy.

Why

Vulnerability counts only refreshed after the nightly sbom-scanner CronJob; image-scanner pods lingered ~30 min just to keep grype's DB warm; and grype was installed in two images. SBOMs/images are produced continuously — scanning should be too.

Phase 1 — SBOM scanning (commit 1)

• SBOM_SCAN job type — peer of CREATE_RUN/IMAGE_SCAN. GET /api/sbom-scans/next (claim via ClaimNextJobOfType) + POST /api/sbom-scans/{id}/complete. Jobs-table RUNNING/locked_by + stale-requeue replaces the bespoke sbom_scan_leases queue (retired).
• Producer (internal/sbomscan): enqueue on ingest + periodic re-sweep over the bounded active set (latest-per-repo commits + image digests still present in a cluster), deduped per sbom_id.
• scan_config + admin GET/PUT /api/admin/sbom/config for the UI-tunable re-sweep interval (default 24h); "Run Scan Now" forces a full re-sweep.
• Binary → poll loop; Helm → Deployment; UI → rescan control + sbom filter on /runs.

Phase 2 — image scanning → Deployment (commit 2)

• Binary → long-running poll loop (graceful drain).
• Retired the scanner operator: deleted operator.go + the scanner_controller_lease table, removed the operator ticker, dropped the PodController impl. imagescan.Reconciler stays the IMAGE_SCAN producer.
• Helm → Deployment replacing the suspended CronJob; operator env + cronjobs:get RBAC removed.

Phase 3 — one grype (commit 3)

• image-scanner no longer runs grype — it produces only the SBOM (syft) + signature/labels/secrets. Vuln scanning is the sbom-scanner's job (it greps the image SBOM, like repos already work). grype ships only in Dockerfile.sbom-scanner.
• After an image SBOM is stored, an SBOM_SCAN is enqueued so image_vuln_findings are produced within seconds.
• New imageScanner.restrictEgress toggle drops the image-scanner's public-443 egress once images route through an in-cluster registry (e.g. Harbor) — leaving DNS + worker + that registry.

Phase 4 — retention & cleanup

Continuous re-scanning produces a superseded copy every cycle and nothing pruned them — re-scan artifacts, old grype results, orphaned SBOMs and non-latest manifests piled up unbounded (a ~1.9GB DB). Four daily, self-rescheduling, batched prune sweeps keep the latest artifact per asset and delete only older copies past a configurable minimum-age window:

• PRUNE_IMAGE_SCANS — artifacts (+empty runs) of non-latest runs per digest
• PRUNE_SBOM_SCANS — grype results that aren't the latest per SBOM
• PRUNE_SBOMS — SBOMs bound to no asset (cascade results/state/leases)
• PRUNE_MANIFESTS — non-latest manifest per repo (cascade dependencies)

The current artifact per asset is always kept (even if months old), and the historic vuln graph is untouched — it reads vuln_dashboard_snapshots / vuln_canonical_*, never the raw scan history these prune. Windows live on scan_config (default 7d, retention on), editable in Settings → Database → "Retention & cleanup" dialog with a "Run cleanup now" button (POST /api/admin/retention/run) that expedites/enqueues all four immediately.

Registry pull-through proxy

New registry_proxies mappings let an admin route upstream registries through one internal pull-through proxy, so the (now egress-pinned) image-scanner only talks to a single host. One credential row fronts many upstreams (e.g. ghcr.io → example.com/ghcr, docker.io → example.com/dockerhub). ResolveForImage prefers a proxy mapping over a direct credential, rewriting only the scanner's pull coordinates (new pull_registry/pull_repository lease fields, omitempty → backward compatible) — the image's stored identity is never rewritten. Admin UI gains a Credential/Proxy tab group.

Production hardening (shaken out by the live lab deploy)

• Resilient stale-job reaper + scanner heartbeat — reclaim each stale job in its own tx (one 23505 on a singleton type can't poison the whole batch); scanners send X-Scanner-Id + new /heartbeat (RenewLease), so a dead scanner is caught in ~minutes instead of the 45m backstop.
• VULN_META_FETCH cooldown — 72h re-fetch window stops permanent-miss CVEs (vuldb-only, OSV-404) being re-fetched on every scan-completion (~800× fewer upstream calls).
• asset_risk guard — asset_risk_src only explodes findings when it is genuinely a JSON array, so one 'null' findings row no longer fails every REFRESH_MV.
• UTF-8 sanitization — shared dbutil.SanitizeUTF8 at the raw-byte sinks (runner.storeLog, secretprobe audit log) so binary tool output / response bodies no longer fail the insert.
• /runs/<id> SBOM detail — RunGetHandler now serves SBOM_SCAN (404'd before); web renders asset + severity grid + View SBOM / View vulnerabilities links.
• Scanner startup banners — both scanners log the real name|version|digest and report to /api/tool-versions.
• Misc: rescan-cooldown arg cast fix; dialog chrome unified (native alert()/confirm() → styled dialogs); devcontainer/seed fixes.

Scoping note

Re-sweeps cover only the active set (latest-per-repo + running-in-cluster images), so they stay bounded. New assets are scanned immediately on ingest; already-scanned assets are re-evaluated against a newer grype DB on the interval — for images via the cached SBOM (no re-pull).

NB: fixed scanner replicas hold standing per-pod scratch reservations — size replicaCount accordingly.

Verification

• go vet ./... (golang:1.26.4) clean on both api and runner modules (incl. tests).
• helm template renders four Deployments and no scanner CronJobs; restrictEgress=true drops the image-scanner's public-443 rule.
• svelte-check: no errors in changed files.

🤖 Generated with Claude Code

Converts both vulnerability scanners from batch/CronJob models to **long-running poller Deployments**, unifies SBOM scanning into the jobs table, and centralizes grype into a single image — so repo runs, image scans, and SBOM scans share one queue/`/runs` view, and grype runs in exactly one place. Continuous scanning also brings retention sweeps (to keep the now-constant artifacts bounded) and a batch of hardening fixes shaken out by the live lab deploy. ## Why Vulnerability counts only refreshed after the nightly `sbom-scanner` CronJob; image-scanner pods lingered ~30 min just to keep grype's DB warm; and grype was installed in two images. SBOMs/images are produced continuously — scanning should be too. ## Phase 1 — SBOM scanning (commit 1) - **`SBOM_SCAN` job type** — peer of `CREATE_RUN`/`IMAGE_SCAN`. `GET /api/sbom-scans/next` (claim via `ClaimNextJobOfType`) + `POST /api/sbom-scans/{id}/complete`. Jobs-table `RUNNING`/`locked_by` + stale-requeue replaces the bespoke `sbom_scan_leases` queue (retired). - **Producer** (`internal/sbomscan`): enqueue on ingest + periodic re-sweep over the **bounded active set** (latest-per-repo commits + image digests still present in a cluster), deduped per `sbom_id`. - **`scan_config`** + admin `GET/PUT /api/admin/sbom/config` for the UI-tunable re-sweep interval (default 24h); "Run Scan Now" forces a full re-sweep. - Binary → poll loop; Helm → Deployment; UI → rescan control + `sbom` filter on `/runs`. ## Phase 2 — image scanning → Deployment (commit 2) - Binary → long-running poll loop (graceful drain). - **Retired the scanner operator**: deleted `operator.go` + the `scanner_controller_lease` table, removed the operator ticker, dropped the `PodController` impl. `imagescan.Reconciler` stays the `IMAGE_SCAN` producer. - Helm → Deployment replacing the suspended CronJob; operator env + `cronjobs:get` RBAC removed. ## Phase 3 — one grype (commit 3) - **image-scanner no longer runs grype** — it produces only the SBOM (syft) + signature/labels/secrets. Vuln scanning is the sbom-scanner's job (it greps the image SBOM, like repos already work). grype ships only in `Dockerfile.sbom-scanner`. - After an image SBOM is stored, an `SBOM_SCAN` is enqueued so `image_vuln_findings` are produced within seconds. - New `imageScanner.restrictEgress` toggle drops the image-scanner's public-443 egress once images route through an in-cluster registry (e.g. Harbor) — leaving DNS + worker + that registry. ## Phase 4 — retention & cleanup Continuous re-scanning produces a superseded copy every cycle and nothing pruned them — re-scan artifacts, old grype results, orphaned SBOMs and non-latest manifests piled up unbounded (a ~1.9GB DB). Four daily, self-rescheduling, batched prune sweeps keep the **latest artifact per asset** and delete only older copies past a configurable minimum-age window: - `PRUNE_IMAGE_SCANS` — artifacts (+empty runs) of non-latest runs per digest - `PRUNE_SBOM_SCANS` — grype results that aren't the latest per SBOM - `PRUNE_SBOMS` — SBOMs bound to no asset (cascade results/state/leases) - `PRUNE_MANIFESTS` — non-latest manifest per repo (cascade dependencies) The current artifact per asset is always kept (even if months old), and the historic vuln graph is untouched — it reads `vuln_dashboard_snapshots` / `vuln_canonical_*`, never the raw scan history these prune. Windows live on `scan_config` (default 7d, retention on), editable in Settings → Database → "Retention & cleanup" dialog with a "Run cleanup now" button (`POST /api/admin/retention/run`) that expedites/enqueues all four immediately. ## Registry pull-through proxy New `registry_proxies` mappings let an admin route upstream registries through one internal pull-through proxy, so the (now egress-pinned) image-scanner only talks to a single host. One credential row fronts many upstreams (e.g. `ghcr.io → example.com/ghcr`, `docker.io → example.com/dockerhub`). `ResolveForImage` prefers a proxy mapping over a direct credential, rewriting **only the scanner's pull coordinates** (new `pull_registry`/`pull_repository` lease fields, omitempty → backward compatible) — the image's stored identity is never rewritten. Admin UI gains a Credential/Proxy tab group. ## Production hardening (shaken out by the live lab deploy) - **Resilient stale-job reaper + scanner heartbeat** — reclaim each stale job in its own tx (one `23505` on a singleton type can't poison the whole batch); scanners send `X-Scanner-Id` + new `/heartbeat` (RenewLease), so a dead scanner is caught in ~minutes instead of the 45m backstop. - **`VULN_META_FETCH` cooldown** — 72h re-fetch window stops permanent-miss CVEs (vuldb-only, OSV-404) being re-fetched on every scan-completion (~800× fewer upstream calls). - **`asset_risk` guard** — `asset_risk_src` only explodes `findings` when it is genuinely a JSON array, so one `'null'` findings row no longer fails every `REFRESH_MV`. - **UTF-8 sanitization** — shared `dbutil.SanitizeUTF8` at the raw-byte sinks (`runner.storeLog`, secretprobe audit log) so binary tool output / response bodies no longer fail the insert. - **`/runs/<id>` SBOM detail** — `RunGetHandler` now serves `SBOM_SCAN` (404'd before); web renders asset + severity grid + View SBOM / View vulnerabilities links. - **Scanner startup banners** — both scanners log the real `name|version|digest` and report to `/api/tool-versions`. - Misc: rescan-cooldown arg cast fix; dialog chrome unified (native `alert()`/`confirm()` → styled dialogs); devcontainer/seed fixes. ## Scoping note Re-sweeps cover only the **active set** (latest-per-repo + running-in-cluster images), so they stay bounded. New assets are scanned immediately on ingest; already-scanned assets are re-evaluated against a newer grype DB on the interval — for images via the cached SBOM (no re-pull). **NB:** fixed scanner replicas hold standing per-pod scratch reservations — size `replicaCount` accordingly. ## Verification - `go vet ./...` (golang:1.26.4) clean on both `api` and `runner` modules (incl. tests). - `helm template` renders four Deployments and **no scanner CronJobs**; `restrictEgress=true` drops the image-scanner's public-443 rule. - svelte-check: no errors in changed files. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

jonas added 1 commit 2026-06-24 13:54:37 +02:00

feat(sbom-scan): continuous scanning via SBOM_SCAN jobs + Deployment ... 31c5342d5b

Replace the nightly sbom-scanner CronJob with a long-running poller
Deployment and unify SBOM vulnerability scanning into the jobs table, so
newly-ingested SBOMs are scanned within seconds and appear on /runs and
the admin jobs page alongside repo runs (CREATE_RUN) and image scans
(IMAGE_SCAN).

- New SBOM_SCAN job type; consumer endpoints GET /api/sbom-scans/next
  (claim via ClaimNextJobOfType) and POST /api/sbom-scans/{id}/complete.
  The jobs-table RUNNING/locked_by + stale-requeue replaces the bespoke
  sbom_scan_leases pull queue (now retired).
- Producer (internal/sbomscan): enqueue on ingest + periodic re-sweep
  over the bounded active set — latest-per-repo commit SBOMs plus image
  digests still present in a cluster. Deduped per sbom_id.
- scan_config table + admin GET/PUT /api/admin/sbom/config for the
  UI-tunable full re-sweep interval (default 24h); "Run Scan Now"
  repurposed to force a full re-sweep immediately.
- sbom-scanner binary: long-running poll loop with backoff, periodic
  grype db update, graceful SIGTERM drain.
- Helm: sbom-scanner is now a Deployment (ephemeral per-replica grype
  cache, no RWX); worker gets SBOM_SCAN_ENABLED to drive the producer.

Image-scanner conversion (Deployment + retire operator) is a separate
follow-up.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

jonas added 1 commit 2026-06-24 14:03:25 +02:00

feat(image-scan): continuous scanning via Deployment, retire operator ... a28b471e35

Phase 2: convert the image-scanner from the suspended-CronJob +
scanner-operator pod-spawning model to a long-running Deployment, the
same shape as the sbom-scanner. The worker's imagescan.Reconciler stays
as the IMAGE_SCAN producer; the Deployment replicas are the consumers
leasing jobs over HTTP.

- Binary: drop the linger-then-exit; loop forever with backoff, refresh
  the grype DB on an interval, drain the in-flight scan on SIGTERM. Warm
  the DB at startup unconditionally (no pre-download queue probe — a
  persistent replica has no cron tick to optimise).
- Retire the scanner operator: delete operator.go + the
  scanner_controller_lease leader-election table, remove the operator
  ticker from the worker, and drop the PodController impl
  (CreateScannerJob / CountActiveScannerPods) from the runner k8s client.
- Helm: replace image-scanner-cronjob.yaml with a Deployment
  (replicaCount, pollInterval, dbUpdateInterval; ephemeral per-replica
  caches). Worker keeps IMAGE_SCAN_ENABLED to drive the reconciler; the
  operator env (IMAGE_SCAN_CRONJOB_NAME / MAX_PARALLELISM /
  SPAWN_SPACING) is gone. Worker no longer clones any CronJob, so the
  cronjobs:get RBAC is dropped.

NB: fixed image-scanner replicas hold a standing per-pod scratch
reservation (work + caches), unlike the old on-demand Jobs — size
replicaCount accordingly.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

jonas changed title from feat(sbom-scan): continuous scanning via SBOM_SCAN jobs + Deployment to feat(scanning): continuous SBOM + image scanners via Deployments + unified jobs 2026-06-24 14:03:59 +02:00

jonas added 1 commit 2026-06-24 15:03:51 +02:00

feat(image-scan): drop grype from image-scanner, centralize in sbom-scanner ... f31240f61b

Phase 3: the image-scanner now only produces the SBOM (syft) plus
signature/labels/secrets — grype is removed. Vulnerability scanning for
images is done by the sbom-scanner, which greps the stored image SBOM
(the same way repos already work). grype is now installed in exactly one
image (sbom-scanner), and the image-scanner needs no vuln-DB egress.

- runner/imagescan: remove the vuln category (grype + trivy_vuln),
  DefaultScanner grype entry, runVuln, and drop CategoryVuln from
  ImageDependentCategories.
- image-scanner binary: remove the grype DB download + periodic refresh
  (it no longer runs grype); keep the continuous poll loop + graceful
  shutdown. Drop the dead pending-probe.
- Dockerfile.image-scanner: drop the grype build stage, binary, and
  grype-cache; keep syft/cosign/crane/betterleaks/trivy.
- Server: image results no longer accept grype/trivy_vuln artifacts;
  instead, after the image SBOM is stored, enqueue a SBOM_SCAN (deduped)
  so the sbom-scanner produces image_vuln_findings within seconds.
- Helm: image-scanner Deployment loses the grype-cache volume + grype
  env; new imageScanner.restrictEgress toggle drops the public-443 egress
  rule once images route through an in-cluster registry (e.g. Harbor),
  leaving DNS + worker + that registry.

Repos already excluded grype (repo-runner = syft only), so this brings
images onto the same "produce SBOM, scan it once, in one place" model.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

jonas added 2 commits 2026-06-24 18:32:34 +02:00

feat(scanners): log tool name|version|digest at startup ... a3124b8293

The sbom-scanner banner printed `Tool: grype | Application: grype | <digest>`
because it took the first line of `grype version` (an "Application:" header)
instead of the version. Parse a Version:/GitVersion: field (fallback: first
non-empty line) so it logs the real version (e.g. 0.114.0).

The image-scanner logged no tool inventory at all. Add a startup banner that
prints name|version|digest for each bundled tool (syft, cosign, crane,
betterleaks, trivy) and reports them to /api/tool-versions under source
"image-scanner", mirroring the sbom-scanner.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

fix(runs): serve SBOM_SCAN detail so /runs/<id> stops 404ing ... 13e9761258

Continuous SBOM scanning added SBOM_SCAN jobs to the runs list (and a `sbom`
type filter), but the detail path was never wired for them: RunGetHandler only
accepted CREATE_RUN/IMAGE_SCAN, so clicking an SBOM-scan row returned 404
"run not found", and the frontend had no SBOM rendering branch.

API: RunGetHandler now short-circuits SBOM_SCAN to writeSBOMScanRunResponse,
returning the scanned SBOM, its asset (type/ref), format, and severity counts +
scanned-at from sbom_scan_results.

Web: add a /runs/[id] branch rendering the asset, scanned-at, a
crit/high/med/low/unknown grid, and View SBOM + View vulnerabilities links;
skip the K8s/SSE/artifact plumbing for SBOM scans (no pod/logs/artifacts).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

jonas added 15 commits 2026-06-26 10:40:39 +02:00

feat(retention): admin-configurable supersession cleanup sweeps ... 607526f4e8

Continuous scanning re-scans the same images/SBOMs/repos constantly, and
nothing pruned the superseded copies — re-scan artifacts (~298MB), old grype
results (~96MB), orphaned SBOMs (~75MB), and non-latest manifests (~122MB)
accumulated unbounded (a ~1.9GB DB).

Add four daily, self-rescheduling, batched prune sweeps that keep the LATEST
scan/SBOM/manifest per asset and delete only older, superseded copies past a
configurable minimum-age window:
  - PRUNE_IMAGE_SCANS  — artifacts (+empty runs) of non-latest runs per digest
  - PRUNE_SBOM_SCANS   — grype results that aren't the latest per SBOM
  - PRUNE_SBOMS         — SBOMs bound to no asset (cascade results/state/leases)
  - PRUNE_MANIFESTS     — non-latest manifest per repo (cascade dependencies)

The current artifact per asset is always kept, even if old, so an image/repo
untouched for months never loses its current scan. Windows live on the
scan_config singleton (default 7 days, retention on) and are editable in
Settings → Database → "Data retention & cleanup", with a "Run cleanup now"
button (POST /api/admin/retention/run) that enqueues all four immediately.

The historic vulnerability graph is unaffected: it reads vuln_dashboard_
snapshots (daily rollup) and vuln_canonical_* (latest-per-asset), neither of
which depends on the raw scan history these sweeps prune. vuln_dashboard_
snapshots is never touched.

Verified: go vet ./... clean; svelte-check at pre-existing baseline; prune
SELECTs dry-run against the live DB confirm 0 latest-per-asset rows match.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

fix(retention): "run now" expedites a queued sweep instead of no-op ... f1dd84eec1

AdminRetentionRunHandler enqueued each sweep with OnConflictDoNothing. But the
ux_jobs_prune_*_active unique index covers any QUEUED/RETRY row, so when a sweep
already had an active job — e.g. a daily run, or a future-dated row left by a
self-rescheduled skip (the boot-time migration race) — the insert silently
no-opped. Net effect: "Run cleanup now" could not run a sweep that was queued
for a future slot; it stayed stuck until that slot arrived. This is exactly
what wedged PRUNE_IMAGE_SCANS (queued +24h after a skipped boot run).

Fix: for each sweep, UPDATE the active QUEUED/RETRY row's run_at to now
(expedite it), and only INSERT a fresh job when there's no active row to bump.
"Run now" now always runs now.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

style(retention): align cleanup panel with the design system ... eb4915dc49

Rebuild the Data retention & cleanup section on the Database settings page
using the shared primitives (per the /playground reference) instead of ad-hoc
Tailwind: Toggle component for the enable switch, .input for the window fields
in playground-style cards, .btn-primary/.btn-secondary for Save / Run cleanup
now (with the canonical spinner), and .pill-success/.pill-error for feedback.
Disabled windows dim as a group. No behaviour change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

refactor(retention): move cleanup settings into a dialog ... a08dfc1fe3

These windows are set-and-forget, so an always-visible panel just crowded the
page. Move the retention form into a Dialog (header / scrollable body / footer
with Save + Run cleanup now) opened by a "Retention & cleanup" button in the
Database storage header. The page now leads with the live storage view; the
knobs are one click away. No behaviour change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

chore(devcontainer): start the full stack on up and fix host access ... ea16ab3b0a

- postAttachCommand runs .devcontainer/start-dev.sh, which starts the
  SvelteKit frontend (:5178) and Go API (:8080) in the background
  (idempotent, port-guarded) so `devcontainer up` brings up the whole app
  and returns instead of hanging on a foreground dev server.
- Publish :5178 (web) and :19999 (mocc OIDC) to the host via appPort instead
  of forwardPorts, so the localhost OIDC redirect works under the
  `devcontainer up` CLI (which does not honor forwardPorts).
- remoteEnv GOTOOLCHAIN=auto so `go run` fetches the toolchain api/go.mod
  requires when the base image ships an older patch release.
- Single dev env in .devcontainer/dev.env, shared by start-dev.sh and
  .vscode/launch.json (envFile) to avoid drift.
- README: rewrite the local-development section to match.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

fix(seed): give the dev_seed repo a provider instance ... a522f08608

repos.provider_instance_id is NOT NULL (per the Repo model), so the single
seeded repo failed to insert against the current schema, crashing API
startup during seeding. Insert a provider instance (idempotent DELETE+INSERT,
matching the file's convention) and reference it from the repo. enabled=false
keeps the seed self-contained: an enabled github instance makes the poller /
cache-warmer hit the real github.com/NorskHelsenett org.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Merge remote-tracking branch 'origin/main' into feat/data-retention-cleanup 3ab33bcb85

refactor(web): unify dialog chrome and replace native confirms ... d5f8bd77f7

Portal Dialog and ConfirmDialog to <body> so they center on the viewport and dim/blur the whole page instead of a transformed parent. Move the close X to the top-right corner. Share pill-button styling (dialogButton.ts) between both dialogs and give Dialog a right-aligned button footer. Drop the divider under titles and above footers across all dialogs, and enlarge the ConfirmDialog icons. Replace the four native window.confirm() prompts in the database admin page with the styled ConfirmDialog via an askConfirm() helper.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

refactor(runs): replace native alert() with styled error dialog ... 80cac3d58d

The re-queue failure now surfaces in a single-button ConfirmDialog (danger variant, XCircle icon) instead of a browser alert().

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

fix(jobs): resilient stale-job reaper + scanner lease heartbeat ... 75ab52b94c

The stale-job reaper requeued every stale RUNNING job in one transaction.
A singleton type (e.g. FETCH_EPSS, whose ux_jobs_fetch_epss_active index
covers QUEUED/RETRY but not RUNNING) flipping RUNNING->RETRY collides with
an already-queued instance and raises 23505, rolling back the reclaim of
EVERY other stale job. In prod a couple of stuck FETCH_EPSS rows kept
zombie IMAGE_SCAN / VULN_META_FETCH locks (held by long-dead pods) RUNNING
indefinitely; those locks block re-enqueue of the same image/CVE via the
active unique indexes, so those assets silently stopped scanning.

- Reclaim each stale job in its own transaction (one bad row can't poison
  the batch); a 23505 on the RETRY means an active instance already exists,
  so retire the stale duplicate as FAILED (in no active partial index).
  Status-guarded UPDATE keeps concurrent replicas safe.

Leased scanner jobs also had no liveness signal: locked_by was the worker
that brokered the lease, not the scanner pod running it, and scanners never
heartbeat, so a dead scanner was only caught by the 45m backstop.

- Scanners send X-Scanner-Id; lease handlers record it as locked_by so the
  job is attributable to its real executor.
- New /api/{image,sbom}-scans/{id}/heartbeat (RenewLease) bumps locked_at
  while a scan runs; 409 when the lease was lost so the scanner stops and
  doesn't clobber the new owner's completion.
- Reaper reclassified: IMAGE_SCAN/SBOM_SCAN reclaim on a short 2m
  heartbeat window; CREATE_RUN keeps the long backstop (reconciled
  separately). A dead scanner is now caught in ~minutes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

fix(asset_risk): guard asset_risk_src against non-array findings ... 14b3df935f

repo_active_secrets explodes the latest run_secrets row per repo with
jsonb_array_elements(COALESCE(lr.findings, '[]'::jsonb)). COALESCE only
substitutes for SQL NULL; a jsonb scalar (a row whose findings is
'null'::jsonb / a string / an object) passes through and makes
jsonb_array_elements raise 22023 "cannot extract elements from a scalar".
asset_risk rebuilds as INSERT ... SELECT * FROM asset_risk_src, so one
such row (observed: one 'null' findings row in prod) failed every
REFRESH_MV / nightly rebuild -- asset_risk silently stopped refreshing
while the worker logged the error every cycle.

Replace asset_risk_src via CREATE OR REPLACE (output columns unchanged,
table/indexes untouched) and only explode findings when it is genuinely a
JSON array, treating any other shape as no findings -- the same guard
pattern already used for ingress rules/backends.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

fix(vulnmeta): add re-fetch cooldown to stop VULN_META_FETCH storm ... 962dd603eb

EnqueueMissingVulnMeta runs on every scan-completion hook and enqueues a
VULN_META_FETCH for every vuln_id with no vuln_metadata row. The
not_found_upstream / upstream_error branches deliberately write no row
(caching a miss would suppress a later successful fetch), and the only
dedup was an active-job check that dedupes within a cycle but not across
them. Permanent-miss vuln_ids — vuldb-only CVE-2026-* disclosures that
OSV 404s and EUVD only returns coincidental numeric-id collisions for —
were therefore re-fetched on every cycle forever.

Prod: 116k VULN_META_FETCH rows, ~7.8k/day, dominated by the same ~26
permanent misses (26 ids x ~280 scan-completions/day), hammering both
upstream APIs to no effect.

Add a 72h re-fetch cooldown: skip any vuln_id with a VULN_META_FETCH
created within the window. New vuln_ids still enrich on first sight; only
retries of misses are throttled, and a genuinely late-published CVE is
re-checked every 72h. Validated on live data: old query enqueues all 26
missing ids per cycle, new query enqueues 0 (~800x fewer jobs / upstream
calls). idx_jobs_vuln_meta_recent backs the created_at lookup; the jobs
table is ~120k rows so the non-concurrent build is sub-second.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

fix(imagescan): cast rescan-cooldown arg so the rescan query stops erroring ... 82ff830d0d

The pass-3 rescan query (digests that scanned but never bound an SBOM)
interpolated the cooldown seconds with `(? || ' seconds')::interval`. The
`||` operator made Postgres infer the bound parameter as text, but the Go
caller passes int(rescanCooldown.Seconds()) (43200) as an int — so pgx
failed every tick with:

  rescan query: failed to encode args[4]: unable to encode 43200 into
  text format for text (OID 25): cannot find encode plan

The whole query aborted before returning rows, so pass 3 never ran:
digests stuck without a bound SBOM were never rescanned. Add the `::int`
cast (matching the working idiom in secrets_dashboard.go) so the param is
typed int and Postgres casts it to text for the concat.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

feat(registrycreds): pull-through proxy mappings for the image scanner ... 42c4eff3d1

Let an admin route upstream registries through an internal pull-through
proxy so the scanner's egress can be pinned to one host. One credential
row can front many upstreams: register example.com with its token, then
attach mappings (ghcr.io -> example.com/ghcr, docker.io ->
example.com/dockerhub) that reuse that credential.

- New registry_proxies child table (upstream_registry -> proxy_path),
  owned by an upstream_repos row; upstream_registry is globally unique.
- ResolveForImage prefers a proxy mapping over a direct credential:
  rewrites the pull onto proxy_path and authenticates against the proxy
  host with the owning row's token. The image's stored identity is never
  rewritten, only the scanner's pull coordinates (new pull_registry/
  pull_repository lease fields, omitempty + scanner fallback = backward
  compatible). Docker Hub aliases fold to docker.io on both sides.
- Admin API + UI: create/update take a proxies[] array; a token OR >=1
  mapping is required. The dialog gains a Credential/Proxy tab group, and
  the Proxy tab edits mappings as single "upstream -> proxy-path" fields.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

fix(db): sanitize raw external bytes to valid UTF-8 before storing ... 501c251888

Postgres rejects any text/jsonb value that isn't valid UTF-8 with
`invalid byte sequence for encoding "UTF8"`, failing the whole INSERT.
Prod logged this (byte 0x89, a binary/non-UTF-8 byte) for an unnamed
portal parameter — a string built from raw external bytes hitting a text
column with no sanitization.

Add a shared dbutil.SanitizeUTF8 (strip NUL + drop invalid UTF-8 runs —
the same logic assets.sanitizeForDB already applied to git author/commit
fields) and apply it at the two raw-byte sinks that lacked it:

  - runner.storeLog: RunLog.Line/Container are raw process output and can
    carry binary bytes a scanned tool printed; an unsanitized line failed
    the insert and the log line was silently dropped ("failed to store
    log"). Best fit for the observed burst (websocket-fed log inserts on
    the API pool while a run is in progress).
  - secretprobe audit log: ResponseBody is a raw HTTP response body (a
    probed URL may return an image/gzip), and the existing byte-slice
    truncation at 4096 can itself split a UTF-8 rune. Sanitize after
    truncating so both cases are storable.

assets.sanitizeForDB now delegates to the shared helper (identical
behavior, no duplication).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

jonas changed title from feat(scanning): continuous SBOM + image scanners via Deployments + unified jobs to feat(scanning): continuous SBOM + image scanners via Deployments, unified jobs + retention 2026-06-26 10:54:33 +02:00

jonas merged commit 6dd6caec76 into main 2026-06-26 10:55:04 +02:00

jonas referenced this pull request from a commit 2026-06-26 10:55:05 +02:00

feat(scanning): continuous SBOM + image scanners via Deployments, unified jobs + retention (#15)

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels   

area/api

Public/UI API surface

  

area/cluster

SCAM cluster data

  

area/llm

LLM advisory / agents

  

area/notifications

Notifications & actions

  

area/triage

Vulnerability triage

  

epic

Large multi-PR feature

  

feature

New functionality

  

needs-design

Open design questions to resolve

No labels

area/api

area/cluster

area/llm

area/notifications

area/triage

epic

feature

needs-design

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

Motstandskraft/spam!15

No description provided.