Feedback button (+ optional context-aware chat bot) #25

New issue

Open

opened 2026-06-30 13:51:19 +02:00 by jonas · 0 comments

jonas commented

2026-06-30 13:51:19 +02:00

Owner

Goal

A lightweight, always-available feedback button so users can report bugs, request features, and react to specific features in context. Explore combining it with a context-aware "chat with a bot" experience (the user is already viewing some asset/page — the bot should know that).

What already exists (reuse)

LLM advisory + chat infra: internal/llmadvisory (OpenAI-compatible, pluggable Open WebUI / vLLM, AES-GCM encrypted keys, per-use-case settings, streaming via ConverseStream). Use cases today: advisory_summary, triage_verdict, finding_chat.
Finding Chat UI: web/src/lib/components/FindingChat.svelte → POST /api/triage/chat (asset_type, asset_id, message history, server-injected evidence). This is the template for a context-aware chat surface.
Audit / admin patterns in internal/audit, internal/uiapi.
No feedback mechanism exists today.

Proposed design

Part A — Feedback capture (ship first, low risk)

Global floating button in web app shell. On click: small form — type (bug / feature / general), free-text, optional screenshot, and auto-attached context: current route, asset in view (type+id), app version/commit, user (from session).
POST /api/feedback → new feedback table {id, user_id, type, body, route, context_json, app_version, created_at, status}.
Admin view to triage feedback (/api/admin/feedback, list + status: new/triaged/closed). Optionally fan a feedback.created event into the notifications epic so it can post to Slack.

Part B — Context-aware bot (build on A)

Reuse llmadvisory with a new use case assistant_chat (separate system prompt, model, settings — admin-configurable like the others).
The chat panel receives page context (route, visible asset, maybe current filters) and injects it as system context, similar to how triage_chat injects finding evidence. So "why is this image in fix_now?" or "how do I suppress this?" can be answered with the user's actual data + product help.
Bridge to feedback: a "this answer didn't help / report a problem" affordance inside the chat creates a feedback row pre-filled with the conversation.

Open questions / decisions

Scope of v1. Recommend shipping Part A (feedback) standalone first — fast, no LLM cost/latency — then layer Part B. (Keeps the two decoupled if bot is deprioritized.)
Privacy. Feedback context may include asset IDs / route params. Define what's captured; respect ACL (don't leak others' asset names into a shared admin queue beyond what the reporter could see).
Bot grounding. Does the assistant get tool-access to live SPAM data (read endpoints), or only the injected page context? Start with injected context only (cheaper, safer); tools later.
Cost / rate limits on assistant chat (reuse llmadvisory token caps).
Storage of chat transcripts — keep for product insight? PII considerations.
Where the button lives — every page vs. excluded flows (login).

Tasks

feedback model + POST /api/feedback + auto-context capture
Floating feedback button + form in web app shell (screenshot optional)
Admin feedback queue (/api/admin/feedback, statuses) + optional feedback.created event
(Part B) assistant_chat llmadvisory use case + admin settings entry
(Part B) Context-aware chat panel reusing FindingChat patterns + page-context injection
(Part B) Chat→feedback bridge

Relates to: notifications epic (route feedback.created to Slack); LLM repo-check epic (shared llmadvisory plumbing).

## Goal A lightweight, always-available **feedback button** so users can report bugs, request features, and react to specific features in context. Explore combining it with a **context-aware "chat with a bot"** experience (the user is already viewing some asset/page — the bot should know that). ## What already exists (reuse) - **LLM advisory + chat infra:** `internal/llmadvisory` (OpenAI-compatible, pluggable Open WebUI / vLLM, AES-GCM encrypted keys, per-use-case settings, streaming via `ConverseStream`). Use cases today: `advisory_summary`, `triage_verdict`, `finding_chat`. - **Finding Chat UI:** `web/src/lib/components/FindingChat.svelte` → `POST /api/triage/chat` (asset_type, asset_id, message history, server-injected evidence). This is the template for a context-aware chat surface. - **Audit / admin patterns** in `internal/audit`, `internal/uiapi`. - **No feedback mechanism exists today.** ## Proposed design ### Part A — Feedback capture (ship first, low risk) - Global floating button in `web` app shell. On click: small form — `type` (bug / feature / general), free-text, optional screenshot, and **auto-attached context**: current route, asset in view (type+id), app version/commit, user (from session). - `POST /api/feedback` → new `feedback` table `{id, user_id, type, body, route, context_json, app_version, created_at, status}`. - Admin view to triage feedback (`/api/admin/feedback`, list + status: new/triaged/closed). Optionally fan a `feedback.created` event into the notifications epic so it can post to Slack. ### Part B — Context-aware bot (build on A) - Reuse `llmadvisory` with a new use case `assistant_chat` (separate system prompt, model, settings — admin-configurable like the others). - The chat panel receives **page context** (route, visible asset, maybe current filters) and injects it as system context, similar to how `triage_chat` injects finding evidence. So "why is this image in fix_now?" or "how do I suppress this?" can be answered with the user's actual data + product help. - Bridge to feedback: a "this answer didn't help / report a problem" affordance inside the chat creates a `feedback` row pre-filled with the conversation. ## Open questions / decisions 1. **Scope of v1.** Recommend shipping **Part A (feedback) standalone first** — fast, no LLM cost/latency — then layer Part B. (Keeps the two decoupled if bot is deprioritized.) 2. **Privacy.** Feedback context may include asset IDs / route params. Define what's captured; respect ACL (don't leak others' asset names into a shared admin queue beyond what the reporter could see). 3. **Bot grounding.** Does the assistant get tool-access to live SPAM data (read endpoints), or only the injected page context? Start with injected context only (cheaper, safer); tools later. 4. **Cost / rate limits** on assistant chat (reuse llmadvisory token caps). 5. **Storage of chat transcripts** — keep for product insight? PII considerations. 6. **Where the button lives** — every page vs. excluded flows (login). ## Tasks - [ ] `feedback` model + `POST /api/feedback` + auto-context capture - [ ] Floating feedback button + form in web app shell (screenshot optional) - [ ] Admin feedback queue (`/api/admin/feedback`, statuses) + optional `feedback.created` event - [ ] (Part B) `assistant_chat` llmadvisory use case + admin settings entry - [ ] (Part B) Context-aware chat panel reusing FindingChat patterns + page-context injection - [ ] (Part B) Chat→feedback bridge **Relates to:** notifications epic (route `feedback.created` to Slack); LLM repo-check epic (shared `llmadvisory` plumbing).

jonas added the

area/llm

area/api

feature

labels

2026-06-30 13:51:19 +02:00