Building AgentDesk: a document assistant that cites its sources

Introduction

Most "chat with your documents" demos are impressive for about five minutes. Then someone asks a question the documents don't answer, and the model invents an answer with total confidence — which is worse than saying nothing, because now you can't trust any of it.

AgentDesk is my attempt at the opposite trade-off: a support assistant you can audit. Upload policies or manuals, embed a widget on your own site, and every reply either cites the passage it came from — with the quoted sentence — or says "I don't know" and lists what it can help with.

This is a case study of building it, and more usefully, of what broke. The interesting bugs were not in the prompt. They were in the places where a system that looks correct fails quietly: a security rule that silently stopped running, a retry that charged a customer twice, and a production database that couldn't be created from the repository at all.

You'll get the most out of this if you're building something with the same shape: a multi-tenant product, a background pipeline doing expensive work, and a public endpoint that costs money every time it's called.

The Context

The product has three moving parts:

• Ingestion. PDFs and images go in; text comes out (with OCR for scans), gets chunked, embedded, and stored next to its source.
• A public widget. One <script> tag on any site. The key in that tag is publishable by definition — it ships in the customer's HTML — so it cannot be treated as a secret.
• Billing. Three plans, per-organization limits, and a monthly ceiling on model spend.

Those constraints decide most of the architecture:

The last row is the one people get wrong. A public key means the only real protection is a cost ceiling you enforce yourself.

The Approach

Two rules keep the tenancy honest:

1. Ownership is enforced in the query, not in a prior read. Every handler re-checks the real session and every statement folds owner_id into its WHERE. Another tenant's row is indistinguishable from a missing one.
2. The widget key resolves to exactly one organization. Retrieval is scoped by that organization id — never by anything the client sends.

Ingestion is split into two functions so a slow OCR run never shares a retry budget with embedding:

The answer contract is a zod schema compiled to a strict JSON schema, so the response shape is enforced by the API rather than by hope:

1interface Answer {
2  status: "answered" | "partial" | "unknown";
3  answer: string;
4  citations: Array<{ source: string; quote: string }>;
5  topics: string[];
6}

The prompt fixes three exact sentences — I don't know., I don't know this part. — and a small reconciliation step re-derives status from the text the model actually produced. That exists because the model labelled a partial answer as "answered" when left to decide, and once, under a prompt rewrite, it discovered it could keep that label simply by not writing the sentence. Prompt rules are not invariants; code is.

Note: The model is a reader of retrieved passages, not a knowledge base. "I don't know" is a correct outcome, and the system is designed so it stays cheap.

What Broke, and Why It Mattered

This is the part I'd want from someone else's case study. Six failures, each one about something quietly returning the wrong answer.

1. A security rule that silently stopped running. The rate-limit and bot rules were keyed on characteristics: ["ip.src", "userId"]. Behind a proxy, or in local development, there is no client IP — and Arcjet cannot build a fingerprint, so it evaluates no rules at all and returns "allowed". A guard that fails open when the environment changes is worse than no guard, because it looks installed. Keying only on values we always control fixed it, and the in-process limiter now runs unconditionally as a backstop.

2. Customers charged twice by a retry. Pages were reserved, then the document row was updated in a separate statement. A failure in between left the row's page count null, so the automatic retry passed the "already reserved?" guard and reserved again. Two statements became one transaction:

1BEGIN;
2  INSERT INTO usage (...) ON CONFLICT (reference_id, period) DO UPDATE SET pages = pages + $1;
3  UPDATE document SET page_count = $2, status = 'extracted' WHERE id = $3;
4COMMIT;

3. A retry that raced a live job. The retry endpoint read the row, checked whether it looked stalled, then wrote. Two clicks — or one click during a slow OCR run — started a second worker on the same document: duplicated embeddings, duplicated charges. It's now a single statement that decides and performs the retry, so exactly one caller can win:

4. Failures the UI never heard about. The failure handler read event.data.documentId, but the original trigger is nested one level deeper in Inngest's failure payload. It was always undefined, so nothing was recorded and documents sat on "processing" forever. A type cast had silenced the compiler — the fix was to read the nested field and stop casting away the shape.

5. The same file failed twice, for two different reasons. pdf.js rejects Node Buffers outright and requires a plain Uint8Array; and it detaches the buffer it is given. Passing one copy to text extraction and then reusing it for page rendering produced a cryptic Cannot transfer object of unsupported type. Two independent defects that presented as one error message.

6. A production database that could not be created from the repository. The schema lived only as TypeScript, applied with drizzle-kit push — which emits the vector(1536) column but never CREATE EXTENSION vector, so it fails on an empty database. Versioned migrations are not bureaucracy; they're the only reproducible path.

Results and Trade-offs

What it does well: answers are traceable, the pipeline is idempotent at every stage that can be retried, and cost is bounded per workspace rather than per incident.

What it deliberately does not do — because shipping the honest version beats shipping a bigger one:

Two things I'd do differently from the start: versioned migrations before the first table, and a spend ceiling before the first public endpoint. Both are cheap at the beginning and expensive after the first bill or the first outage.

Key Takeaways

• A guard that fails open is not a guard. Test the "environment is missing something" path as deliberately as the happy path — that's where mine silently stopped running.
• Make retries idempotent with one statement, not two. If a decision and its effect are separate reads and writes, a retry can interleave between them.
• Don't let a type cast hide a payload shape. as unknown as turned a wrong field path into a permanent, invisible failure.
• Enforce output shape at the API, not in the prompt. A strict schema makes "invented fields" impossible rather than discouraged.
• Write down your limits where the code can read them. The landing page renders plans from the same table the billing logic enforces, so marketing can't drift from reality.

Have a question about this topic? Get in touch.