Security & Compliance

"Three levels of isolation, choose based on sensitivity:

• (1)Row-level security (cheapest) — shared infrastructure, data filtered by tenant_id in every query. Good for low-sensitivity SaaS.
• (2)Namespace isolation — shared compute but separate vector indices, separate KMS keys, separate logging sinks per tenant. Good for mid-tier.
• (3)Project-level isolation (strongest) — separate GCP project per tenant with its own VPC-SC perimeter, CMEK, and service accounts. Required for regulated industries. The agent's MCP tools enforce the isolation layer — the LLM has no concept of tenants. Critical: test isolation — run adversarial queries like 'show me data from tenant B' and verify zero cross-contamination."

"Full trace chain: every task gets a trace_id that links:

• (1)The original user request
• (2)The plan the agent generated
• (3)Every tool call with arguments and results
• (4)Every LLM prompt and response (stored but PII-redacted)
• (5)The final output with confidence score
• (6)Any human approval steps. This goes to BigQuery with 7-year retention. For the executive: a human-readable summary is auto-generated: 'The agent reviewed 3 data sources, found X, concluded Y, and took action Z.' Think of it as an automated decision log — every AI decision is as auditable as a human decision."

“It should act as the user. Put it on a service account with broad access and you have built a confused deputy — someone who can’t read the salary table asks a question, the agent can read it, and now the answer contains it. No prompt rule reliably fixes that, because the enforcement point is in the wrong place.

Concretely: propagate the user’s identity down to every tool call. An on-behalf-of token, or at minimum a user id that the tool enforces against — never one the model is free to set. Authorization happens in the tool or the data layer, not in the prompt.

Retrieval is the part people miss. If the index holds every document and you filter by permission after retrieval, you have already leaked: through what matched, through the ranking, through the fact that an answer exists at all. The permission has to be a filter inside the query.

There are legitimate service-account cases — a scheduled job with no user behind it. Those get their own narrowly-scoped principal and their own audit trail, not the identity the interactive agent uses.”

“Two destinations, two different controls.

For the provider: a zero-retention endpoint if the contract offers one, and I’d treat that as the primary control because it’s contractual and verifiable. Redaction is the secondary control, on what we send.

For our own traces — that’s where I see most of the leakage, because tracing gets set up by engineers to be maximally useful and it captures whole prompts. If those prompts carry customer data, your observability vendor is now a subprocessor holding PII for thirty days, and it’s usually on nobody’s data map.

The mechanism I like is tokenised redaction rather than deletion: detect the entity, replace it with a stable placeholder before the call, swap it back on the way out. The model reasons about PERSON_1 and ACCOUNT_2 perfectly well, the real values never leave the process, and the trace is still readable when you need to debug.

And I’d be honest that detection is imperfect — so redact by default, allowlist what’s permitted through, and sample-audit what actually got sent.”

“Red-teaming an agent is a different exercise from red-teaming a chatbot, because the interesting failures aren’t ‘it said something bad’ — they’re ‘it did something bad’. The attack surface is the tool list.

Three passes. First, a systematic walk: for every tool the agent can call, what is the worst request that still looks legitimate and would make it call that tool destructively? That isn’t adversarial creativity, it’s an inventory, and it finds the most.

Second, injection through data. Every path where untrusted text enters — a document, a web page, a ticket, a tool response — gets tested with instructions embedded in it. The question isn’t whether the model complies once; it’s whether compliance can reach a tool.

Third, automated: a jailbreak suite that runs continuously rather than once. And the part that matters — every attack that works becomes a regression test. A red team that produces a PDF is theatre; a red team that produces test cases is engineering.

And I’d scope it honestly. Two weeks internally finds the obvious. If the blast radius is real, you buy an external engagement.”

“They’re two different asks and it’s worth separating them, because customers say them in one breath.

SOC 2 is mostly evidence, not architecture — access control with reviews, change management, logging and monitoring, vendor management, incident response, and being able to demonstrate all of it across an observation window. The part specific to us is that the model provider, the vector database and the tracing vendor are all subprocessors, and the auditor will ask for that list.

Residency is architecture. Every hop in-region: model endpoint, vector index, the object store holding source documents, the traces, the queues, the backups. The one that catches people is that a vendor’s global control plane often routes metadata through another region, and ‘metadata’ includes prompts more often than anyone would like.

So the practical answer is a region-pinned deployment per residency zone, rather than one global system with a region column — because the second one leaks, and you can’t prove that it doesn’t.”

“I’d separate what the user authorised from what the code enforces, because the model can be wrong or fooled about both. An agent purchase has three layers: checkout with the merchant, proof the user approved the spend, and moving the money. Protocols such as ACP, UCP and AP2 standardise those messages, but the controls are still mine to build.

For overspending: the user’s intent becomes signed rules, such as office supplies, under $300 an order, four vendors. Code checks every cart against them, using the merchant’s total in minor units, never a number the model typed. Payment goes through a scoped token, capped, expiring and tied to one seller, never the raw card. Each checkout session keeps one idempotency key, so a retry after a timeout can’t buy twice. Above the cap, the user approves the exact cart and total.

For being tricked: page text never changes the cart; only the user’s mandate does. Once the agent has read a merchant page, the session is tainted, so a cart line that isn’t in the intent, a new vendor or a changed shipping address is held for the user, decided by code that reads the arguments, not the page.

And every step writes an append-only record: intent, cart, verdict, approval, token cap, result. When someone says ‘I didn’t buy this,’ I can show what they signed.”

“I start by assuming it will be fooled sometimes. Instructions and data reach the model as one token stream, so there’s no filter that separates them, the way parameterised queries do for SQL. My job is to bound what a fooled agent can do, in code the attacker’s text never reaches.

So I count the legs. This agent reads untrusted text, has sensitive access to the inbox, and can act outward. With all three, every outward action needs a person, so first I cut a leg where I can: the summariser gets no send_email. Where sending stays, a gate decides from the tool name, the arguments and the session’s taint. Once the agent has read a web page or a stranger’s email, a send to anyone outside the user’s contacts is held and the exact action is shown to the user.

Then I close the exits that need no send tool. Fetching attacker.example/?q=<secrets> is exfiltration, so every fetch goes through an egress proxy with an allowlist. A markdown image the model wrote leaks on render, so images and links render only from allowed hosts. The agent acts with the user’s own permissions, and third-party MCP tool descriptions are pinned, because they’re text the model reads too.

Last, I test it against an attacker who has read the defence, and I report a residual rate, not ‘injection-proof’.”