Production LLM application security is not a new model provider. It is the same access, logging, and change-control story with an untrusted string in the middle. OWASP's Top 10 for LLM Applications names prompt injection, insecure output handling, training-data leakage, and unbounded agency (OWASP LLM Top 10). Mid-market teams need a map onto AWS they can ship in a sprint: identity for the caller, scoped tools, no standing keys, logs you can show an auditor.
Rutagon builds that map. We are not an agentic-SOC vendor. We are not writing a generic AI-automation playbook. We secure the application plane so a CTO can put Bedrock or an HTTP model API behind the same org they already use for SOC 2-aligned workloads. We do not claim Rutagon holds a SOC 2 report.
The Control Map We Actually Implement
| OWASP LLM theme | AWS-shaped control | Evidence | | --- | --- | --- | | Prompt injection / agency | Tool allowlist; bind tenant from identity, not from the prompt | Gateway audit log | | Insecure output handling | Encode/allowlist before the output hits a browser or a shell | Code review + tests | | Sensitive information | No production dumps in prompts; redact logs | Log samples | | Model DoS / cost | Quotas, max tokens, per-tenant budgets | CloudWatch + bills | | Supply chain | Pin model IDs and tool versions; scan the app | Change ticket | | Unbounded consumption | IAM on the model API; no shared admin key | Credential report |
NIST SP 800-53 Rev. 5 AC-6 and AU-2 still apply: least privilege and audit events (NIST SP 800-53 Rev. 5). CISA's secure-by-design line on least privilege and short-lived credentials is the same physics (CISA Secure by Design).
Production LLM Application Security on the AWS Path
Identity. The user is Cognito or your IdP. The application assumes an IAM role to call Amazon Bedrock or to read a secret for a third-party API. There is no long-lived provider key in Lambda environment screenshots. See workload identity federation.
Tools. If the app can call refund, deploy, or export, those calls go through a gateway with an allowlist. The model text cannot choose tenant_id. That is the same pattern as MCP tool security for AI agents and AI agent least privilege.
Data. Prompts do not include raw production rows if a retrieval index can return a handle. If you must send data, it is the minimum fields, in the region you already scoped, with a retention story for provider logs. Bedrock model invocation logging, if enabled, goes to the log account — not to a playground bucket.
Output. Treat model text as untrusted. If it is rendered in the admin panel, encode it. If it can become a SQL fragment or a shell argument, it does not ship. That is application security, not "AI magic."
def complete(ctx, user_text):
if not ctx.role.can_use_llm:
raise PermissionError("llm")
prompt = render_template(ctx.template_id, user_text=sanitize(user_text), tenant=ctx.tenant_id)
resp = bedrock.invoke_model(
modelId=ctx.pinned_model_id,
body=prompt,
# credentials from the task role, not from env keys
)
audit.emit(ctx, tokens=resp.usage, model=ctx.pinned_model_id)
return safe_render(resp.text)
Pin modelId. A console change to a larger model is a cost and a behavior change. Treat it as a production change ticket.
Capability: security automation and full-stack development. Cost of runaway tokens belongs next to Lambda cost optimization.
What a Sprint Actually Ships
Week 1: inventory call paths, keys, tools, and log destinations. Kill standing provider keys. Put quotas on the model API.
Week 2: gateway or bind-from-context on any tool; invocation logging to the log account; WAF or app-level abuse limits on the public prompt endpoint.
That is enough for a CTO to say "this is production" without buying an AI-SOC. Further work — retrieval ACLs, evals, red-team prompts — is a second increment. We do not pretend a sprint deletes prompt injection as a class. We shrink agency and we log.
What We Will Not Write
We will not publish jailbreak recipes. We will not specify controlled algorithms. We will not frame this as AI for any vertical. The buyer is a team already calling a model from a production app and staring at a key in Secrets Manager that three functions share.
RAG (retrieval) is a second sprint if the first sprint has no tool calls. Retrieval ACLs must match the user's tenant. A shared index with no filter is a data leak with better UX. We do not build a vector database product. We enforce the same bind-from-context rule on the retrieve step that we enforce on tools.
Human feedback loops — thumbs-down, red-team prompts, eval sets — belong in nonprod until they are logged and access-controlled. Do not paste production tickets into a public eval notebook. That is the same data-handling rule as any other support tool.
Cost alarms on token spend are a security control when a prompt-injection or a buggy loop would otherwise print money and data. We set them in the same sprint as quotas. A silent unlimited Bedrock call from a public endpoint is both a bill and an abuse path.
Streaming responses do not change the control model. Tokens still count. Logs still need a request id. The output is still untrusted when it finishes. We do not disable logging "because streaming made it hard." We log metadata and hashes if the full stream is too large or too sensitive to retain. Evaluation prompts that include real customer text stay in the log account's retention policy, not in a personal notebook that outlives the observation window.
Discuss securing your production LLM application →
907-841-8407 · contact@rutagon.com
Frequently Asked Questions
Does production LLM application security require Amazon Bedrock?
No. Bedrock is the AWS-native path we like for IAM and logging. A third-party HTTP API can be just as locked down if the caller is a role and the key is not standing in six places. We still prefer federation or a single broker.
Is prompt injection "solved" by a system prompt?
No. System prompts help. They are not a control. The control is what the app is allowed to do with the output and which tools exist.
How do we evidence this for SOC 2?
Treat the LLM path like any privileged feature: access (who can invoke), change (model and prompt-template versions), monitoring (invocation logs and errors). Rutagon does not hold the attestation.
Should we put the model inside a VPC?
Network placement is not authorization. Private connectivity helps. The IAM and the tool gateway still have to be right.
Are you competing with LLM security vendors?
No. Those products specialize in scanners and guardrail SaaS. We implement the AWS and application controls behind whatever you already bought.