Dependency maintenance intelligence: detect stale/abandoned libs, per-library threat view, deepsearch harness #28

New issue

Open

opened 2026-06-30 15:53:42 +02:00 by jonas · 0 comments

jonas commented

2026-06-30 15:53:42 +02:00

Owner

Goal

Detect stale / abandoned / unmaintained / archived third-party dependencies and make that a first-class input into the triage buckets — not just a passive metric. Plus, give every dependency a rich per-library view (like the npm / NuGet package page) with a threat analysis. Supported by a new long-running "deepsearch" job harness / agent so deep per-library investigation can run as a background task rather than a quick synchronous lookup.

This is the dependency-intelligence sub-feature called out in #26, promoted to its own epic. #26 stays focused on the per-repo multi-breakdown audit; this issue owns the library-centric model, staleness scoring, triage tracing, and the deepsearch harness.

What exists today (extend, don't duplicate)

internal/dephealth already fetches library health metadata (activity, deprecation, stars) and produces ArchivedDepCount, DeprecatedDepCount, MaxMajorBehind, MajorBehindDepCount, WorstDepHealthScore (see assetrisk/score.go:19). Job type FETCH_DEP_HEALTH refreshes it (weekly, self-rescheduling).
internal/manifests parses lockfiles / package.json / etc. → the dependency graph + PURLs.
asset_risk scoring: dep-health is currently a TrustScore / context input only — it never moves a triage tier (assetrisk/score.go). Today there is no per-library page and no "is the upstream repo abandoned?" signal beyond archived/deprecated flags.
internal/providers / providerconfig can reach GitHub/GitLab/Gitea source repos (encrypted PATs) — the source for commit cadence / archive status / maintainer activity.

Proposed work

1. Staleness / abandonment detection

Per library (PURL or canonical package identity), compute a maintenance signal from upstream source repo + registry metadata:

No human commits in N weeks (configurable threshold; e.g. 26 / 52 weeks → stale, longer → dormant).
Only bot commits — if the only recent activity is Renovate / Dependabot / github-actions[bot] / mergify, treat as effectively unmaintained (humans aren't touching it). Requires classifying commit authors as bot vs. human.
Archived repo flag (already partly in dephealth — make explicit).
Deprecated in registry (npm deprecated, NuGet unlisted, etc.).
Release cadence: last published release age; releases trailing off.
Single-maintainer / bus-factor (optional, later): few distinct recent authors.

Output a dependency_maintenance record: {package_key, ecosystem, status: active|stale|dormant|abandoned|archived|deprecated, last_human_commit_at, last_release_at, bot_only_recent: bool, distinct_recent_authors, signals_json, computed_at}.

2. Trace into triage buckets

Add maintenance fields to assetrisk.Signals (e.g. AbandonedDepCount, StaleDepCount, BotOnlyDepCount) and decide their effect on tiers (open question below). At minimum they enrich TrustScore + context reasons; ideally an unmaintained dependency carrying a fixable CVE with no upstream activity (i.e. the fix will likely never come) is a distinct, surfaced situation in the triage detail panel. The "trace" means: from a triage bucket entry you can see which deps are dragging it down on maintenance grounds, and from a library page you can see which assets/clusters it endangers.

3. Per-library view (npm / NuGet style)

A dedicated page per dependency showing:

Identity (name, ecosystem, PURL), source repo link, license.
Maintenance scorecard (status, last human commit, release cadence, bot-only badge, archived/deprecated).
Threat analysis: known CVEs/KEV/EPSS for this library (already have via vuln views), transitive risk, advisory narrative (LLM), and a maintenance-driven "fix-availability" verdict.
Blast radius: which repos / images / clusters use it and at what versions (from manifests + cluster_image_inventory + bindings) — ACL-filtered.
Version history / "you are N majors behind" (from MaxMajorBehind).

4. Long-running "deepsearch" job harness / agent

A new job kind (e.g. DEEPSEARCH or DEP_DEEPSEARCH) + harness for long-running, possibly multi-step agent investigations that don't fit the quick FETCH_DEP_HEALTH poll:

Runs in the sandboxed K8s runner (internal/runner) like other heavy jobs, with checkpointing / progress so it can run for minutes and survive worker churn (multi-replica safe via the existing SELECT … FOR UPDATE SKIP LOCKED claim).
An LLM agent (internal/llmadvisory) drives: pull source repo signals, read README/CHANGELOG/issues, cross-reference advisories, assess maintainer trust & abandonment, and write the structured threat analysis for the library page.
Cache by a signals hash (advisory pattern) so deepsearch only re-runs when inputs change; expose progress/status (reuse run/status streaming).

Edge cases / decisions

Bot classification. Maintain an allowlist of bot author patterns (renovate[bot], dependabot[bot], github-actions[bot], …) and a heuristic for the long tail; risk of misclassifying a human named "...bot". Make it auditable/overridable.
Source repo resolution. Mapping PURL → upstream repo is imperfect (especially transitive / vendored deps). Reuse the OCI org.opencontainers.image.source resolution where present; otherwise registry metadata; mark confidence.
Does maintenance move triage tiers, or only context? Recommend: context + a specific escalation only when abandoned + fixable-or-KEV CVE (the "no fix is coming" case). Default off / configurable, mirroring the posture decision in #27 and escalation logic in #22.
Thresholds are ecosystem-dependent. A 2-year-quiet, complete, single-file lib may be fine; a quiet web framework is alarming. Make N configurable; allow per-library suppression of the "stale" flag (ties into #22).
Rate limits / cost. Source-repo API calls + LLM deepsearch are expensive — dedup per package across all assets, cache aggressively, run deepsearch on-demand or for high-blast-radius libs first.
Privacy/ACL. Library page blast-radius must be ACL-filtered (a user only sees assets/clusters they can read).
Renovate-as-positive-signal nuance. Bot-only activity means no human review, but a repo with active Renovate that humans merge is healthy — distinguish "bot commits only" from "bot PRs merged by humans" where data allows.

Tasks

dependency_maintenance model + computation (commit cadence, bot-vs-human author classification, archived/deprecated, release age) extending internal/dephealth
Source-repo signal fetch via providers/providerconfig (cadence, archive status, authors)
New maintenance fields on assetrisk.Signals + decide context-vs-escalation (resolve open question)
Per-library page: identity, maintenance scorecard, threat analysis, blast radius (ACL-filtered), version-behind
DEEPSEARCH job kind + long-running harness (sandboxed runner, checkpoint/progress, multi-replica claim, status streaming)
LLM deepsearch agent producing structured library threat analysis (cached by signals hash, evidence-cited)
"Trace" UX: from a triage bucket entry → offending deps; from a library → endangered assets
Config: staleness thresholds, bot allowlist, per-library stale suppression

Relates to: #26 (per-repo audit shares dep graph + LLM plumbing), #22 (suppress a stale flag / escalation policy), #27 (context-vs-escalation precedent), #24 (dependency.abandoned event), #23 (expose in scan results).

## Goal Detect **stale / abandoned / unmaintained / archived** third-party dependencies and make that a **first-class input into the triage buckets** — not just a passive metric. Plus, give every dependency a rich **per-library view** (like the npm / NuGet package page) with a **threat analysis**. Supported by a new **long-running "deepsearch" job harness / agent** so deep per-library investigation can run as a background task rather than a quick synchronous lookup. > This is the dependency-intelligence sub-feature called out in #26, promoted to its own epic. #26 stays focused on the per-repo multi-breakdown audit; this issue owns the *library-centric* model, staleness scoring, triage tracing, and the deepsearch harness. ## What exists today (extend, don't duplicate) - **`internal/dephealth`** already fetches library health metadata (activity, deprecation, stars) and produces `ArchivedDepCount`, `DeprecatedDepCount`, `MaxMajorBehind`, `MajorBehindDepCount`, `WorstDepHealthScore` (see `assetrisk/score.go:19`). Job type **`FETCH_DEP_HEALTH`** refreshes it (weekly, self-rescheduling). - **`internal/manifests`** parses lockfiles / package.json / etc. → the dependency graph + PURLs. - **`asset_risk`** scoring: dep-health is currently a **TrustScore / context input only — it never moves a triage tier** (`assetrisk/score.go`). Today there is no per-library page and no "is the upstream repo abandoned?" signal beyond archived/deprecated flags. - **`internal/providers`** / `providerconfig` can reach GitHub/GitLab/Gitea source repos (encrypted PATs) — the source for commit cadence / archive status / maintainer activity. ## Proposed work ### 1. Staleness / abandonment detection Per library (PURL or canonical package identity), compute a **maintenance signal** from upstream source repo + registry metadata: - **No human commits in N weeks** (configurable threshold; e.g. 26 / 52 weeks → `stale`, longer → `dormant`). - **Only bot commits** — if the only recent activity is Renovate / Dependabot / github-actions[bot] / mergify, treat as effectively unmaintained (humans aren't touching it). Requires classifying commit authors as bot vs. human. - **Archived** repo flag (already partly in dephealth — make explicit). - **Deprecated** in registry (npm `deprecated`, NuGet unlisted, etc.). - **Release cadence:** last published release age; releases trailing off. - **Single-maintainer / bus-factor** (optional, later): few distinct recent authors. Output a `dependency_maintenance` record: `{package_key, ecosystem, status: active|stale|dormant|abandoned|archived|deprecated, last_human_commit_at, last_release_at, bot_only_recent: bool, distinct_recent_authors, signals_json, computed_at}`. ### 2. Trace into triage buckets Add maintenance fields to `assetrisk.Signals` (e.g. `AbandonedDepCount`, `StaleDepCount`, `BotOnlyDepCount`) and **decide their effect on tiers** (open question below). At minimum they enrich TrustScore + context reasons; ideally an *unmaintained dependency carrying a fixable CVE with no upstream activity* (i.e. the fix will likely never come) is a distinct, surfaced situation in the triage detail panel. The "trace" means: from a triage bucket entry you can see *which* deps are dragging it down on maintenance grounds, and from a library page you can see *which assets/clusters* it endangers. ### 3. Per-library view (npm / NuGet style) A dedicated page per dependency showing: - Identity (name, ecosystem, PURL), source repo link, license. - **Maintenance scorecard** (status, last human commit, release cadence, bot-only badge, archived/deprecated). - **Threat analysis:** known CVEs/KEV/EPSS for this library (already have via vuln views), transitive risk, advisory narrative (LLM), and a maintenance-driven "fix-availability" verdict. - **Blast radius:** which repos / images / clusters use it and at what versions (from `manifests` + `cluster_image_inventory` + bindings) — ACL-filtered. - Version history / "you are N majors behind" (from `MaxMajorBehind`). ### 4. Long-running "deepsearch" job harness / agent A new job kind (e.g. `DEEPSEARCH` or `DEP_DEEPSEARCH`) + harness for **long-running, possibly multi-step agent investigations** that don't fit the quick `FETCH_DEP_HEALTH` poll: - Runs in the sandboxed K8s runner (`internal/runner`) like other heavy jobs, with checkpointing / progress so it can run for minutes and survive worker churn (multi-replica safe via the existing `SELECT … FOR UPDATE SKIP LOCKED` claim). - An LLM agent (`internal/llmadvisory`) drives: pull source repo signals, read README/CHANGELOG/issues, cross-reference advisories, assess maintainer trust & abandonment, and write the structured threat analysis for the library page. - Cache by a signals hash (advisory pattern) so deepsearch only re-runs when inputs change; expose progress/status (reuse run/status streaming). ## Edge cases / decisions 1. **Bot classification.** Maintain an allowlist of bot author patterns (renovate[bot], dependabot[bot], github-actions[bot], …) and a heuristic for the long tail; risk of misclassifying a human named "...bot". Make it auditable/overridable. 2. **Source repo resolution.** Mapping PURL → upstream repo is imperfect (especially transitive / vendored deps). Reuse the OCI `org.opencontainers.image.source` resolution where present; otherwise registry metadata; mark confidence. 3. **Does maintenance move triage tiers, or only context?** Recommend: context + a *specific* escalation only when `abandoned + fixable-or-KEV CVE` (the "no fix is coming" case). Default off / configurable, mirroring the posture decision in #27 and escalation logic in #22. 4. **Thresholds are ecosystem-dependent.** A 2-year-quiet, complete, single-file lib may be fine; a quiet web framework is alarming. Make N configurable; allow per-library suppression of the "stale" flag (ties into #22). 5. **Rate limits / cost.** Source-repo API calls + LLM deepsearch are expensive — dedup per package across all assets, cache aggressively, run deepsearch on-demand or for high-blast-radius libs first. 6. **Privacy/ACL.** Library page blast-radius must be ACL-filtered (a user only sees assets/clusters they can read). 7. **Renovate-as-positive-signal nuance.** Bot-only activity means *no human review*, but a repo with active Renovate that humans merge is healthy — distinguish "bot commits only" from "bot PRs merged by humans" where data allows. ## Tasks - [ ] `dependency_maintenance` model + computation (commit cadence, bot-vs-human author classification, archived/deprecated, release age) extending `internal/dephealth` - [ ] Source-repo signal fetch via `providers`/`providerconfig` (cadence, archive status, authors) - [ ] New maintenance fields on `assetrisk.Signals` + decide context-vs-escalation (resolve open question) - [ ] Per-library page: identity, maintenance scorecard, threat analysis, blast radius (ACL-filtered), version-behind - [ ] **`DEEPSEARCH` job kind + long-running harness** (sandboxed runner, checkpoint/progress, multi-replica claim, status streaming) - [ ] LLM deepsearch agent producing structured library threat analysis (cached by signals hash, evidence-cited) - [ ] "Trace" UX: from a triage bucket entry → offending deps; from a library → endangered assets - [ ] Config: staleness thresholds, bot allowlist, per-library stale suppression **Relates to:** #26 (per-repo audit shares dep graph + LLM plumbing), #22 (suppress a stale flag / escalation policy), #27 (context-vs-escalation precedent), #24 (`dependency.abandoned` event), #23 (expose in scan results).

jonas added the

labels

2026-06-30 15:53:42 +02:00