57
Scenario: Preventing Private Code Leakage in Coding Agents
"Meta trains coding agents using private source-code repositories. Design a system to ensure the agent does not output verbatim or near-verbatim code from private repos."
Types of Leakage
| Type | Description | Severity | Detection Difficulty |
|---|---|---|---|
| Verbatim memorization | Model outputs exact copy of training code (50+ tokens identical) | Critical — clear IP violation | Easy — string matching |
| Near-verbatim | Minor changes (variable rename, whitespace) but structurally identical | High — still recognizable as private code | Medium — need AST/structural comparison |
| Structural leakage | Unique algorithm, architecture pattern, or API design reproduced | Medium — reveals proprietary design decisions | Hard — requires semantic understanding |
| Metadata leakage | Internal file paths, team names, internal API endpoints, secrets | High — reveals internal infrastructure | Easy — pattern matching |
| Style/pattern leakage | Reproduces distinctive coding style or naming conventions | Low — hard to prove, often acceptable | Very hard — subjective |