fix(vuln): stop image-vuln extract dropping other scanners' findings #52

Merged

jonas merged 3 commits from fix/image-vuln-extract-cross-scanner-loss into main

2026-07-08 22:42:03 +02:00

jonas commented

2026-07-08 22:26:26 +02:00

Owner

Problem (found while investigating the vuln-metrics dup-key issue)

The incremental extract for view_unified_image_vulnerabilities can silently drop findings. This is a correctness / data-loss bug, distinct from the duplicate-key freeze (that's PR #51).

Root cause

ExtractDerivedOnArrival runs, per table, a DELETE then a scoped INSERT (internal/db/derived_tables.go). For the image table:

DELETE (before): removed all rows for any image with a finding newer than the watermark — WHERE image_id IN (SELECT image_digest_id FROM image_vuln_findings WHERE created_at > $1).
INSERT: view_unified_image_vulnerabilities_scoped($1) only re-emits findings with created_at > $1.

image_vuln_findings is delete+reinserted per (image_digest, scanner) scan, so only the scanner that just re-ran carries a fresh created_at; a different scanner's findings for the same image keep their older created_at. So when e.g. grype rescanned an image, trivy's rows for that image were deleted by the image-wide DELETE but not re-added by the created_at-scoped INSERT — gone until the next full RebuildDerivedTables.

The 20260626 migration comment assumed "a changed image's whole finding set carries a fresh created_at" — true for a single scanner, but not across scanners.

Fix

Scope the DELETE to the (image_id, source) pairs that were actually rescanned, matching the INSERT's per-scanner scope:

DELETE FROM view_unified_image_vulnerabilities
WHERE (image_id, source) IN (
    SELECT DISTINCT image_digest_id::text, scanner
    FROM image_vuln_findings
    WHERE created_at > $1::timestamptz
)

Now an untouched scanner's rows (older created_at, not in the delete set, not re-emitted) simply survive. Eviction semantics for the rescanned scanner are unchanged (its rows are deleted and replaced with the fresh set, so fixed vulns still disappear). The repos extract already scopes by changed-repo membership and doesn't have this bug.

Verification

go vet ./internal/db/... — clean.
Logic: DELETE scope now equals INSERT scope, per (image, scanner).

🤖 Generated with Claude Code

## Problem (found while investigating the vuln-metrics dup-key issue) The incremental extract for `view_unified_image_vulnerabilities` can silently drop findings. This is a **correctness / data-loss** bug, distinct from the duplicate-key freeze (that's PR #51). ## Root cause `ExtractDerivedOnArrival` runs, per table, a `DELETE` then a scoped `INSERT` (`internal/db/derived_tables.go`). For the image table: - **DELETE** (before): removed **all** rows for any image with a finding newer than the watermark — `WHERE image_id IN (SELECT image_digest_id FROM image_vuln_findings WHERE created_at > $1)`. - **INSERT**: `view_unified_image_vulnerabilities_scoped($1)` only re-emits findings with `created_at > $1`. `image_vuln_findings` is delete+reinserted **per `(image_digest, scanner)`** scan, so only the scanner that just re-ran carries a fresh `created_at`; a *different* scanner's findings for the same image keep their older `created_at`. So when e.g. grype rescanned an image, trivy's rows for that image were deleted by the image-wide DELETE but **not** re-added by the `created_at`-scoped INSERT — gone until the next full `RebuildDerivedTables`. The `20260626` migration comment assumed "a changed image's whole finding set carries a fresh created_at" — true for a single scanner, but not across scanners. ## Fix Scope the DELETE to the `(image_id, source)` pairs that were actually rescanned, matching the INSERT's per-scanner scope: ```sql DELETE FROM view_unified_image_vulnerabilities WHERE (image_id, source) IN ( SELECT DISTINCT image_digest_id::text, scanner FROM image_vuln_findings WHERE created_at > $1::timestamptz ) ``` Now an untouched scanner's rows (older `created_at`, not in the delete set, not re-emitted) simply **survive**. Eviction semantics for the rescanned scanner are unchanged (its rows are deleted and replaced with the fresh set, so fixed vulns still disappear). The repos extract already scopes by changed-repo membership and doesn't have this bug. ## Verification - `go vet ./internal/db/...` — clean. - Logic: DELETE scope now equals INSERT scope, per `(image, scanner)`. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

jonas added 1 commit

2026-07-08 22:26:27 +02:00

fix(vuln): stop image-vuln extract dropping other scanners' findings

Build and Deploy / build-image-scanner (push) Successful in 1m6s

Details

Build and Deploy / build-repo-runner (push) Successful in 1m26s

Details

Build and Deploy / build-sbom-scanner (push) Successful in 1m26s

Details

Build and Deploy / deploy-helm (push) Has been cancelled

Details

Build and Deploy / build-app (push) Has been cancelled

Details

81168c4bc4

The incremental extract for view_unified_image_vulnerabilities deleted ALL
rows for any image with a fresh finding, but the scoped INSERT only re-emits
findings with created_at > watermark. image_vuln_findings is delete+reinserted
per (image_digest, scanner), so only the scanner that just re-ran carries a
fresh created_at. When one scanner rescanned an image, the other scanners'
findings for that image were deleted and never re-inserted — silent data loss
until the next full RebuildDerivedTables.

Scope the DELETE to the (image_id, source) pairs that were actually rescanned,
matching the INSERT's per-scanner scope, so untouched scanners' rows survive.
The repos extract already scopes by changed-repo membership and isn't affected.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

jonas added 1 commit

2026-07-08 22:30:18 +02:00

Merge branch 'main' into fix/image-vuln-extract-cross-scanner-loss

Build and Deploy / build-image-scanner (push) Successful in 1m9s

Details

Build and Deploy / build-repo-runner (push) Successful in 1m25s

Details

Build and Deploy / build-sbom-scanner (push) Failing after 1m16s

Details

Build and Deploy / build-app (push) Successful in 4m4s

Details

Build and Deploy / deploy-helm (push) Has been skipped