Reverse search on cluster by IP, hostname etc #53

New issue

Open

opened 2026-07-08 22:41:38 +02:00 by jonas · 0 comments

jonas commented

2026-07-08 22:41:38 +02:00

Owner

Problem

SPAM today centers on clusters for non-global / non-admin roles, which makes it hard for regular users to answer broad questions across the clusters they can view, e.g.:

which cluster has a given hostname / FQDN exposed?
which cluster is a given exposed IP assigned to?
and similar "reverse lookup" questions.

Add a feature so users can search more broadly across the clusters they have view access to.

Scope: externally-exposed endpoints — the public-facing IPs/hosts behind ingresses, HTTPRoutes, Gateways, IngressRoutes. Not internal ranges (ClusterIPs, Node IPs, pod IPs).

Current state (what already exists)

Much of the hostname side already works for regular users; the gap is narrower than "add search".

FQDN / hostname + LB-IP substring search already exists, ACL-scoped, via HostsHandler (GET /api/clusters/hosts, approved group). ?q= runs ILIKE '%q%' across he.host, he.backends, he.lb_ips, namespace, name, kind, ingress_class (internal/scam/handler.go → buildHostFilterClauses), scoped by he.cluster_id through clusterACLFilterCol. Surfaced in Clusters → Hosts tab (web/src/routes/(app)/clusters/+page.svelte) with an IP column from lb_ips.
ClusterSummaryHandler (/api/clusters/summary?q=) already does ACL-scoped name/slug/environment search.
host_exposure (migration 20260509) already ingests all relevant kinds: k8s Ingress (rules[].host + lb_ips), Gateway API HTTPRoute/GRPCRoute/TLSRoute (hostnames), Traefik IngressRoute (hosts).
host_resolution.public_ips / ips (populated by the internal/hostresolve worker) holds the DNS-resolved public IP of every exposed FQDN, kind-agnostically.

Because the exposed IPs are already captured (lb_ips) and resolved (host_resolution.public_ips), no SCAM-agent ingest or schema change is required.

Gaps / the actual work

Reverse-IP lookup is weak. IP search today is only a substring ILIKE over the he.lb_ips string. The resolved public IPs in host_resolution.public_ips/ips are not searched. Reverse-IP should match against those columns — that covers ingress + all Gateway/route kinds with data we already have.
Advanced search is admin-only and doesn't cover this. /api/search/* (AdvancedSearchHandler, target=cluster) sits behind APIGuard (admin / global_reader) and only searches cluster_summary names. Do not extend that surface — build on the approved group like /api/clusters/hosts.
No search index on host/IP columns. pg_trgm is repos-only; host search is sequential ILIKE. If this becomes primary UX, add a trigram/btree index on the host/IP columns (or a small dedicated projection).

Constraints

Must reuse clusterACLFilterCol(r, "<table>.cluster_id") before any DB read. Grants come from ROR (RORProvider) and resolve across three identity domains (cluster_id / ror_slug / ror_cluster_uid). Deny → empty response, never unfiltered.

Suggested scope

Extend the existing Hosts search (approved group) into a first-class "reverse search" entry point.
Add reverse-IP matching against host_resolution.public_ips/ips (not just the lb_ips substring).
Add an index if perf warrants it once the query shape is settled.

Open design questions (needs-design)

Reverse-IP UX: dedicated search box vs. generalized query field on the Hosts tab.
Whether to add a trigram/btree index now or after measuring ILIKE performance.
Optional/separable edge case: a Gateway/HTTPRoute whose LB address isn't in public DNS wouldn't be found via resolution; capturing Gateway .status.addresses in the agent would close it, but is non-blocking.

Key files

internal/scam/handler.go (HostsHandler, buildHostFilterClauses)
internal/scam/acl.go (clusterACLFilterCol)
internal/hostresolve + host_resolution table; host_exposure (migration 20260509)
internal/uiapi/advanced_search.go (admin-only — don't reuse)
web/src/routes/(app)/clusters/+page.svelte (Hosts tab)

## Problem SPAM today centers on clusters for non-global / non-admin roles, which makes it hard for regular users to answer broad questions across the clusters they can view, e.g.: - which cluster has a given hostname / FQDN exposed? - which cluster is a given **exposed** IP assigned to? - and similar "reverse lookup" questions. Add a feature so users can search more broadly across the clusters they have view access to. **Scope:** externally-**exposed** endpoints — the public-facing IPs/hosts behind ingresses, HTTPRoutes, Gateways, IngressRoutes. **Not** internal ranges (ClusterIPs, Node IPs, pod IPs). ## Current state (what already exists) Much of the hostname side already works for regular users; the gap is narrower than "add search". - **FQDN / hostname + LB-IP substring search already exists, ACL-scoped**, via `HostsHandler` (`GET /api/clusters/hosts`, `approved` group). `?q=` runs `ILIKE '%q%'` across `he.host`, `he.backends`, `he.lb_ips`, namespace, name, kind, ingress_class (`internal/scam/handler.go` → `buildHostFilterClauses`), scoped by `he.cluster_id` through `clusterACLFilterCol`. Surfaced in **Clusters → Hosts tab** (`web/src/routes/(app)/clusters/+page.svelte`) with an IP column from `lb_ips`. - `ClusterSummaryHandler` (`/api/clusters/summary?q=`) already does ACL-scoped name/slug/environment search. - `host_exposure` (migration `20260509`) already ingests all relevant kinds: k8s Ingress (`rules[].host` + `lb_ips`), Gateway API HTTPRoute/GRPCRoute/TLSRoute (`hostnames`), Traefik IngressRoute (`hosts`). - `host_resolution.public_ips` / `ips` (populated by the `internal/hostresolve` worker) holds the **DNS-resolved public IP of every exposed FQDN**, kind-agnostically. Because the exposed IPs are already captured (`lb_ips`) and resolved (`host_resolution.public_ips`), **no SCAM-agent ingest or schema change is required.** ## Gaps / the actual work 1. **Reverse-IP lookup is weak.** IP search today is only a substring `ILIKE` over the `he.lb_ips` string. The resolved public IPs in `host_resolution.public_ips`/`ips` are **not** searched. Reverse-IP should match against those columns — that covers ingress + all Gateway/route kinds with data we already have. 2. **Advanced search is admin-only and doesn't cover this.** `/api/search/*` (`AdvancedSearchHandler`, `target=cluster`) sits behind `APIGuard` (admin / global_reader) and only searches `cluster_summary` names. Do **not** extend that surface — build on the `approved` group like `/api/clusters/hosts`. 3. **No search index on host/IP columns.** `pg_trgm` is repos-only; host search is sequential `ILIKE`. If this becomes primary UX, add a trigram/btree index on the host/IP columns (or a small dedicated projection). ## Constraints - Must reuse `clusterACLFilterCol(r, "<table>.cluster_id")` before any DB read. Grants come from ROR (`RORProvider`) and resolve across three identity domains (cluster_id / ror_slug / ror_cluster_uid). Deny → empty response, never unfiltered. ## Suggested scope - Extend the existing Hosts search (`approved` group) into a first-class "reverse search" entry point. - Add reverse-IP matching against `host_resolution.public_ips`/`ips` (not just the `lb_ips` substring). - Add an index if perf warrants it once the query shape is settled. ## Open design questions (needs-design) - Reverse-IP UX: dedicated search box vs. generalized query field on the Hosts tab. - Whether to add a trigram/btree index now or after measuring `ILIKE` performance. - Optional/separable edge case: a Gateway/HTTPRoute whose LB address isn't in public DNS wouldn't be found via resolution; capturing Gateway `.status.addresses` in the agent would close it, but is non-blocking. ## Key files - `internal/scam/handler.go` (`HostsHandler`, `buildHostFilterClauses`) - `internal/scam/acl.go` (`clusterACLFilterCol`) - `internal/hostresolve` + `host_resolution` table; `host_exposure` (migration `20260509`) - `internal/uiapi/advanced_search.go` (admin-only — don't reuse) - `web/src/routes/(app)/clusters/+page.svelte` (Hosts tab)

jonas added the

labels

2026-07-08 22:54:27 +02:00