The Trinity Beast Infrastructure — Security Posture

1. Security Architecture Overview

The Trinity Beast Infrastructure implements defense-in-depth across 11 distinct security layers. No single layer is trusted to stop all threats — each layer assumes the one above it has been breached. Traffic must survive every gauntlet before reaching the data layer.

The Security Stack (Outside → Inside)

flowchart TB
    subgraph INTERNET["🌐 Public Internet"]
        USER["👤 Legitimate User"]
        ATTACKER["💀 Attacker"]
        SCANNER["🤖 Scanner/Bot"]
    end

    subgraph EDGE["Layer 1 — Edge Defense"]
        CF["CloudFront CDN
TLS 1.3 · Shield Standard
DDoS Absorption"]
        CFWAF["CloudFront WAF
IP Reputation · Common Rules
Known Bad Inputs"]
    end

    subgraph ALB_LAYER["Layer 2 — API Gateway"]
        ALB["Application Load Balancer
TCP :443 → :8080/:9090"]
        ALBWAF["ALB WAF
Rate Limit 2000/5min
Admin Rate 100/5min
SQL Injection · Honeypot"]
        NLB["Network Load Balancer
UDP :2679/:2680"]
    end

    subgraph APP["Layer 3 — Application (ECS Fargate)"]
        direction LR
        MAIN["BeastMain
2 vCPU · 6 GB"]
        MIRROR["BeastMirror
2 vCPU · 6 GB"]
        LRS["BeastLRS
2 vCPU · 6 GB"]
        WEBHOOK["BeastWebhook
Push Only · No Inbound"]
    end

    subgraph NETWORK["Layer 4 — Network Isolation"]
        ECSVPC["ECS VPC
10.0.0.0/16"]
        PEER["VPC Peering
pcx-0e988b7230cf451b8"]
        DATAVPC["Data VPC
172.31.0.0/16"]
    end

    subgraph DATA["Layer 5 — Data Layer"]
        AURORA["Aurora PostgreSQL 17.7
TLS · Encrypted at Rest
Private Subnet Only"]
        VALKEY["ElastiCache Serverless
Valkey 9.0 · TLS · AUTH Token
Private Subnet Only"]
    end

    subgraph DETECT["Layer 7–8 — Detection & Response"]
        GD["GuardDuty
VPC Flow · CloudTrail · DNS"]
        CW["CloudWatch
37 Alarms"]
        AUTOOPS["AutoOps AI
Bedrock Threat Analysis
Self-Heal · Auto-Block"]
    end

    USER --> CF
    ATTACKER --> CF
    SCANNER --> CF
    CF --> CFWAF
    CFWAF -->|"Static Assets"| USER
    USER -->|"API Requests"| ALB
    ATTACKER -->|"API Attacks"| ALB
    ALB --> ALBWAF
    ALBWAF -->|"Passed"| APP
    ALBWAF -->|"Blocked"| ATTACKER
    USER -->|"UDP Price Feed"| NLB
    NLB --> APP
    APP --> ECSVPC
    ECSVPC --> PEER
    PEER --> DATAVPC
    DATAVPC --> AURORA
    DATAVPC --> VALKEY
    GD -.->|"Monitors"| NETWORK
    CW -.->|"Monitors"| APP
    AUTOOPS -.->|"Auto-blocks"| ALBWAF

    linkStyle 0,1,2 stroke:#ef4444,stroke-width:2px
    linkStyle 7,8 stroke:#10b981,stroke-width:2px
    linkStyle 9 stroke:#ef4444,stroke-width:2px
    linkStyle 10 stroke:#f97316,stroke-width:2px
    linkStyle 11,12,13,14,15 stroke:#a855f7,stroke-width:2px
        

2. Layer 1 — Edge Defense (CloudFront + Shield)

The outermost perimeter. All website traffic enters through CloudFront — a globally distributed CDN with 450+ edge locations. API traffic hits the ALB directly (DNS-routed). Both paths are protected by AWS Shield Standard.

CloudFront Distribution

Shield Standard Protection

AWS Shield Standard provides automatic protection against the most common Layer 3/4 DDoS attacks:

• SYN floods — absorbed at the edge before reaching origin
• UDP reflection attacks — dropped at CloudFront/ALB edge
• HTTP floods — rate-limited by WAF rules before reaching application
• DNS amplification — handled by Route 53 infrastructure

Shield Standard is free, automatic, and always-on. No configuration required. It protects both the CloudFront distribution and the ALB.

TLS Configuration

3. Layer 2 — Web Application Firewall (WAF)

Two separate WAF WebACLs protect different attack surfaces. The CloudFront WAF guards the static website. The ALB WAF guards the API — and carries the heavier rule set including rate limiting, SQL injection detection, and the honeypot trap system.

CloudFront WAF (CreatedByCloudFront-449feaa5)

ALB WAF (trinity-beast-api-waf)

WAF Exclusions

Dynamic IP Block List

The tbi-autoops-blocked-ips IP set (ID: 8d55de25-8ba5-4982-8c41-f4316c9bd50d) is managed automatically by the honeypot processor Lambda. IPs are added when they hit honeypot trap endpoints. As of this writing the set holds 473 IPs (5% of the WAF IP set's 10,000-address cap) — 444 honeypot-dated (90-day expiry) and 29 unmanaged (manual/undated, never auto-expire).

• Auto-block trigger: First hit on any honeypot endpoint → immediate WAF block
• Manual blocks: bash scripts/kcc.sh block-ip 1.2.3.4
• Country blocks: bash scripts/kcc.sh block-country "RU CN" (geo-match rule)
• Admin lockdown: bash scripts/kcc.sh lockdown-admin $(curl -s ifconfig.me) (priority 0 — blocks all /admin/* except your IP)

90-Day Expiry for Honeypot Blocks

The honeypot-triggered portion of the block list ages out after 90 days rather than growing without bound. A WAF IP set caps at 10,000 addresses, and at the observed rate of new honeypot hits, an unmanaged list would fill that cap within roughly two years and then silently stop accepting new blocks — the failure mode of an "unbounded" design is not "the list gets big," it's "blocking eventually stops working."

Manual blocks made via block-ip are never auto-expired — those carry no honeypot evidence and are assumed to be intentional, permanent decisions. Only IPs positively attributed to the honeypot are dated and pruned.

Why a separate ledger, rather than trusting the per-IP honeypot:ip:<ip> Valkey hash directly: those records turned out to be inconsistent in practice — some carry a short TTL, some are permanent, and some blocked IPs have no record at all. Expiry logic built on top of that would silently mis-date or lose IPs. Instead, a dedicated sorted set (honeypot:blocklist:blocked_at, member=IP, score=block epoch) is written the moment the honeypot processor issues a block — before the flaky per-IP record can expire — and that ledger is what honeypot-prune reads to decide age.

The ledger itself is made durable across a Valkey wipe by three stores that agree with each other: the WAF IP set is membership truth (durable, AWS-managed), a private S3 JSON object (s3://aws-waf-logs-trinity-beast/honeypot/blocklist-ledger.json) is the durable timestamp record, and the Valkey sorted set is the fast working copy. BeastReconciler reconciles all three every night, and the honeypot processor Lambda writes to the ledger at block time — so a Valkey flush can never orphan a block's age.

• Check without changing anything: bash scripts/kcc.sh honeypot-prune (dry-run, default)
• Commit the prune: bash scripts/kcc.sh honeypot-prune --apply

Honeypot Trap System

12 decoy endpoints that no legitimate user would ever access. Any hit immediately identifies the source as a scanner or attacker:

Behavior on hit: 2-second tarpit delay (wastes scanner time) → log IP, user-agent, path to Valkey → queue for WAF auto-block → daily digest includes all honeypot activity.

4. Layer 3 — Application Security

The Go application implements its own security stack independent of AWS infrastructure. Even if WAF is completely bypassed, the application enforces authentication, authorization, rate limiting, and input validation at every endpoint.

API Key Authentication

Every non-public endpoint requires a valid API key in the Authorization: Bearer header. Keys are validated against a multi-tier cache:

1. Local cache (sync.Map) — sub-microsecond lookup, 5-minute TTL for standard tiers, 60-minute for partner/stress
2. ElastiCache (Valkey) — millisecond lookup, populated by Aurora on first access
3. Aurora (PostgreSQL) — source of truth, queried only on cache miss

Invalid keys receive an immediate 401 Unauthorized with full UME envelope. No information leakage about why the key failed.

Per-Tier Rate Limiting

Rate limiting is enforced per API key using a token bucket algorithm with ElastiCache-backed counters. Exceeding the limit returns 429 Too Many Requests.

Admin Endpoint Protection

All /admin/* endpoints require the admin API key (X-Admin-Key header). This key is:

• Stored only in AWS Secrets Manager (trinity-beast-admin-secrets): not in code, not in any environment variable, not in the database or cache
• Servers refresh it every 60 seconds, and immediately on a mismatch; with no key loaded, the admin API refuses every request
• Rotatable via bash scripts/kcc.sh rotate-admin-key rotate --yes, which confirms every node before it finishes; the firewall logs redact the admin header
• Protected by WAF rate limit (100 requests/5 min to /admin/*)
• Lockable to a single IP via bash scripts/kcc.sh lockdown-admin

CORS Policy

The CORS middleware echoes the requesting origin only for cpmp-site.org and www.cpmp-site.org with credentials: true. All other origins receive Access-Control-Allow-Origin: * (public API — no credentials). This prevents credential-bearing cross-origin requests from unauthorized domains.

Input Validation

• All query parameters validated and bounded (page sizes, date ranges, asset symbols)
• SQL queries via admin endpoint use parameterized execution — no string concatenation
• Request body size limits enforced at the application level
• Invalid or missing parameters return immediate 400 Bad Request — no silent defaults that could cost money
• Customer document URLs are fetched through a private-address guard (October 2026): loopback, private (RFC 1918), link-local including the cloud metadata address 169.254.169.254, carrier-grade NAT, unique-local and multicast addresses are refused, and every redirect is checked again, up to 3 hops. A translation quote can never be used to read something inside our network.

Webhook HMAC Verification

Outbound webhook deliveries include an HMAC-SHA256 signature in the X-Signature header. Subscribers verify the signature using their webhook secret to confirm the payload originated from The Trinity Beast and was not tampered with in transit.

5. Layer 4 — Network Isolation (VPC + Peering)

Compute and data live on completely separate networks. There is no path from the public internet to the data layer. The only bridge is a VPC peering connection with strict security group rules.

Two-VPC Architecture

VPC Peering

Peering connection pcx-0e988b7230cf451b8 bridges the ECS VPC to the Data VPC. Route tables in the ECS subnets send 172.31.0.0/16 traffic through the peering connection. Only specific ports are allowed through security groups.

Security Groups

Aurora's security group only accepts connections from the three ECS subnets — not from the entire ECS VPC CIDR, and certainly not from anywhere else. This is defense-in-depth: even if an attacker compromises an ECS task, they can only reach Aurora on port 5432 with valid credentials.

VPC Endpoints (Private AWS Service Access)

Lambda functions in the ECS VPC cannot reach AWS services over the internet (no public IP on Lambda ENIs). VPC endpoints provide private connectivity:

• vpce-020d39818b2046944 — Secrets Manager (DB credential retrieval)
• vpce-0f52837e401b8960e — SQS (message receive/delete)
• Additional endpoints: ECR, ECS, CloudWatch Logs, SSM, S3 (Gateway)

6. Layer 5 — Data Layer Protection

The crown jewels — Aurora PostgreSQL and ElastiCache (Valkey). Both are encrypted at rest, encrypted in transit, and accessible only from the ECS VPC via peering.

Aurora PostgreSQL

ElastiCache Serverless (Valkey)

7. Layer 6 — The Flame Path (UDP Security)

The UDP price feed is the fastest path into the system — raw speed with zero TCP handshake overhead. It's called the Flame Path because data travels at wire speed, burning through with no connection state, no retransmission, no negotiation. But speed without security is recklessness. The Flame Path has its own security model.

UDP Infrastructure

UDP Security Model

1. API Key in Every Packet: Each UDP request includes the API key in the packet payload. There is no session — every packet is independently authenticated.
2. Same Key Validation: The UDP handler uses the identical key validation pipeline as TCP (local cache → ElastiCache → Aurora). Invalid keys are rejected with zero response (silent drop — no error packet to confirm the endpoint exists).
3. Rate Limiting: Per-tier rate limits apply identically to UDP and TCP. The token bucket doesn't care which protocol delivered the request.
4. No Amplification: UDP responses are smaller than or equal to requests. The system cannot be used as a DDoS amplification vector.
5. NLB Protection: The Network Load Balancer handles millions of packets per second at the network layer. It doesn't maintain connection state (unlike ALB), making it inherently resistant to state-exhaustion attacks.

Why UDP is Safe Here

Common UDP security concerns and how the Flame Path addresses them:

flowchart LR
    subgraph CLIENT["Partner / Subscriber"]
        APP["Application
UDP Client"]
    end

    subgraph EDGE["Edge"]
        NLB["NLB
UDP :2679/:2680
Wire Speed"]
    end

    subgraph ECS["ECS Fargate"]
        UDP_HANDLER["UDP Handler
Zero-Alloc v6
Per-Packet Auth"]
        CACHE["sync.Map
Price Cache
Sub-μs Lookup"]
        RATE["Rate Limiter
Token Bucket
Per-Key"]
    end

    APP -->|"🔥 UDP Packet
[API_KEY + ASSET]"| NLB
    NLB -->|"Forward"| UDP_HANDLER
    UDP_HANDLER -->|"1. Validate Key"| RATE
    RATE -->|"2. Check Limit"| CACHE
    CACHE -->|"3. Price Response"| UDP_HANDLER
    UDP_HANDLER -->|"🔥 UDP Response
[PRICE_DATA]"| APP

    style NLB fill:#7f1d1d,stroke:#ef4444,color:#fca5a5
    style UDP_HANDLER fill:#1a0f0a,stroke:#f97316,color:#fdba74
    style CACHE fill:#064e3b,stroke:#10b981,color:#6ee7b7
    style RATE fill:#1e3a5f,stroke:#60a5fa,color:#93c5fd

    linkStyle 0 stroke:#f97316,stroke-width:3px
    linkStyle 5 stroke:#f97316,stroke-width:3px
        

Flame Path Performance Characteristics

• Latency: Sub-millisecond end-to-end (no TLS handshake, no TCP three-way, no connection setup)
• Throughput: 487,900 UDP RPS sustained in stress testing (Run 17, at the 8 vCPU / 32 GB extreme-load profile — not the everyday 2 vCPU / 6 GB running configuration)
• Memory: Zero allocation per request — pre-allocated buffers, no GC pressure
• Connection State: None. Each packet is independent. No state table to exhaust.

8. Layer 7 — Threat Detection & Response

Passive monitoring that watches everything and alerts on anomalies. This layer doesn't block — it detects, correlates, and escalates.

Amazon GuardDuty

GuardDuty uses machine learning to establish behavioral baselines and detect deviations. It monitors:

• VPC Flow Logs: Unusual traffic patterns, port scanning, data exfiltration attempts
• CloudTrail: Unusual API calls, privilege escalation attempts, credential misuse
• DNS: Communication with known C2 domains, DNS tunneling attempts

CloudWatch Alarms (37 in us-east-2, 1 in us-east-1 = 38 account-wide)

Infrastructure Alarms (14)

Target-Group Health Alarms (6 — full coverage, all 8 target groups alarmed across this table and the Webhook Test Receiver table below)

Webhook Test Receiver Alarms (2 — added 2026-08-07)

Application-Log Metric Alarms (2 — added 2026-08-10)

The only alarms in the account sourced from CloudWatch Logs metric filters (on /aws/ecs/trinity-beast) rather than a native AWS service metric:

Coverage Alarms (3 — added 2026-07-30, closing gaps alarm-audit found with no alarm of any kind)

Security Alarms (4)

Target-5xx and ELB-5xx are distinct signals: target-5xx means the container itself returned an error; ELB-5xx means the ALB couldn't get a usable response from any target at all (502/503/504).

Other Alarms (8 — not managed by us directly)

• TBI-Security-IAM-Changes, TBI-Security-EC2-RunInstances — CloudTrail metric-filter alarms
• TargetTracking-service/trinity-beast-fargate-cluster/* (6: AlarmHigh + AlarmLow × main/mirror/lrs) — auto-created by ECS Application Auto Scaling to drive the scaling policies. Never delete these.

flowchart TB
    subgraph SIGNALS["Detection Signals"]
        WAF_BLOCKS["WAF Block Events"]
        GD_FINDINGS["GuardDuty Findings"]
        CW_ALARMS["CloudWatch Alarms"]
        HONEYPOT["Honeypot Hits"]
        ANOMALY["Anomaly Detectors"]
    end

    subgraph ROUTING["Event Routing"]
        EB["EventBridge
Pattern Matching"]
        SNS["SNS
tbi-ops-notifications"]
    end

    subgraph RESPONSE["Automated Response"]
        SF["Step Function
tbi-ops-health-check-heal"]
        BEDROCK["Bedrock Analysis
Claude Sonnet 4.6
Threat Correlation"]
        HONEYPOT_PROC["Honeypot Processor
Every 5 min"]
        SELF_HEAL["Self-Heal Lambda
Restart · Redeploy"]
        WAF_ACTION["WAF Action Lambda
Block IP · Unblock"]
        NOTIFY["Notify Lambda
Formatted SES Email"]
    end

    subgraph ACTIONS["Outcomes"]
        BLOCK["🚫 WAF Block"]
        RESTART["🔄 ECS Restart"]
        EMAIL["📧 Alert Email"]
        ESCALATE["🚨 Page Cory"]
    end

    WAF_BLOCKS --> EB
    GD_FINDINGS --> EB
    CW_ALARMS --> SNS
    CW_ALARMS --> EB
    HONEYPOT --> HONEYPOT_PROC
    ANOMALY --> SNS

    SNS --> NOTIFY
    EB -->|"Alarm State Change"| SF
    EB -->|"Severity ≥ 7"| BEDROCK
    HONEYPOT_PROC --> WAF_ACTION

    SF --> SELF_HEAL
    SELF_HEAL --> RESTART
    BEDROCK -->|"HIGH/CRITICAL"| WAF_ACTION
    BEDROCK -->|"Report"| NOTIFY
    WAF_ACTION --> BLOCK
    NOTIFY --> EMAIL
    SF -->|"Unknown failure"| ESCALATE

    style BEDROCK fill:#1a0f2e,stroke:#a855f7,color:#c4b5fd
    style SF fill:#1e3a5f,stroke:#60a5fa,color:#93c5fd
    style NOTIFY fill:#064e3b,stroke:#10b981,color:#6ee7b7
    style ESCALATE fill:#7f1d1d,stroke:#ef4444,color:#fca5a5
        

9. Layer 8 — Autonomous Operations (AI Defense)

The AI layer that turns raw signals into intelligent action. Amazon Bedrock (Claude Sonnet 4.6) correlates WAF blocks, honeypot hits, rate limit events, and usage anomalies to produce threat assessments and recommend actions.

How It Works

1. Every 5 minutes: EventBridge triggers tbi-ops-bedrock-analyze Lambda
2. Signal collection: Lambda gathers WAF block counts, honeypot hits, rate limit events, and GuardDuty findings from the last window
3. Quiet check: If all signals are below threshold (infrastructure is quiet), Lambda exits immediately — no Bedrock invocation, no cost
4. AI analysis: If signals are elevated, Lambda sends the correlated data to Bedrock for pattern analysis
5. Threat report: Bedrock generates a plain-English threat assessment with severity rating (LOW/MEDIUM/HIGH/CRITICAL)
6. Auto-action: For safe actions (IP blocks), the Lambda executes immediately. For destructive actions, it escalates to Cory.
7. Storage: Threat report stored in Valkey (autoops:threats:daily) for the daily digest

Self-Healing Scenarios

Design Principles

Cost Consciousness

The system is designed to minimize Bedrock costs during quiet periods:

• Auto-skip when infrastructure is quiet (no signals above threshold)
• Typical quiet-day cost: $0 (no Bedrock invocations)
• Active threat day cost: ~$0.50-2.00 (multiple analysis cycles)
• Monthly budget: ~$2-5 for threat analysis

10. Layer 9 — Audit Trail & Forensics

Every action, every API call, every network flow is logged. When an incident occurs, the forensic trail is complete — who did what, when, from where.

CloudTrail

CloudTrail captures every AWS API call made in the account — console, CLI, SDK, and service-to-service. This includes:

• IAM authentication events (who logged in, from where)
• Resource modifications (who changed what security group, who updated what Lambda)
• Data access patterns (S3 object reads, Secrets Manager retrievals)
• Failed authorization attempts (permission denied events)

VPC Flow Logs

Active on both VPCs. Captures every network flow (accepted and rejected) with source/destination IP, port, protocol, bytes, and action. Used by:

• GuardDuty: Behavioral analysis — detects port scanning, unusual traffic patterns, data exfiltration
• Forensics: After an incident, trace exactly which IPs communicated with which resources
• Compliance: Prove that no unauthorized network paths exist

Application-Level Logging

Forensic Commands (KCC)

Pre-built forensic routines for incident investigation:

• bash scripts/kcc.sh trace-ip 1.2.3.4 — WAF blocks + usage logs for an IP
• bash scripts/kcc.sh trace-key <api_key_id> — Key owner, activity, IPs used
• bash scripts/kcc.sh trace-email user@email.com — Account, keys, transactions, tickets
• bash scripts/kcc.sh threat-log — WAF blocks by rule (24h) + GuardDuty findings
• bash scripts/kcc.sh threat-status — Current block list + alarm state

11. Layer 10 — Identity & Access Management

Every service has its own IAM role with the minimum permissions required. No shared credentials. No wildcard policies. No cross-service privilege escalation paths.

Service Roles

Least-Privilege Principles

• No * resource ARNs: Every policy specifies exact resource ARNs where possible
• No cross-role trust: No role can assume another role (no privilege escalation chain)
• No IAM modification: No service role can create users, roles, or policies
• Separate execution and task roles: ECS execution role (infrastructure) is separate from task role (application)
• VPC-scoped Lambdas: queued-writer is in the VPC (needs Aurora). All others are outside (don't need it, shouldn't have it)

Human Access

12. Layer 11 — Secrets & Key Management

Every secret this system uses lives in AWS Secrets Manager, and nowhere else. Not in code, not in an environment variable, not in the database, not in the cache. That became true for the last credential on 11 October 2026, when the admin key and the Stripe secret were moved out of the database and cache copies they had also been kept in.

The reason is worth stating plainly, because it is the lesson behind this whole layer: a secret kept in two places is only as safe as the least protected one. A copy in a database row or a cache field is readable by anything that can read rows or fields, it survives a rotation that only updated the original, and it can be exported into somewhere public without anyone noticing. One home means one thing to protect, one thing to rotate, and one place to audit.

What services carry instead is the name of a secret, never its value. DB_SECRET_NAME=trinity-beast-secrets in an environment variable is a label; reading it gets you nothing without an IAM role that is allowed to fetch that secret. The two ECS jobs that need the admin key get it through a task-definition secrets block, which means AWS injects the value at task start and it never appears in a task definition, a log or a process list.

Secrets Manager

Stripe Keys

Payment credentials follow the same one-home rule. Until 11 October 2026 the Stripe secret was also kept in a database parameter row and a cache field, and the servers read it from the cache. They now read it once at boot from AWS Secrets Manager and the copies are gone. The receipt Lambda keeps its own separate secret, so the component that handles payment webhooks never holds database credentials, and the component that talks to the database never needs the payment keys.

Admin API Key

The admin key (tbcc-admin-*) is the most sensitive application-level credential. Its security model:

• Storage: AWS Secrets Manager (trinity-beast-admin-secrets) only (encrypted at rest)
• Refresh: servers every 60 seconds and immediately on a mismatch; Lambdas every 30 seconds; the admin API fails closed when no key is loaded
• Rotation: bash scripts/kcc.sh rotate-admin-key rotate --yes — new version, every node confirmed within seconds
• Rate limiting: WAF (RateLimit-Admin-6000) limits /admin/* to 6,000 requests per 5 minutes per IP
• IP lockdown: Can restrict all admin access to a single IP in seconds
• Brute force: the key is a tbcc-admin- prefix plus 64 hexadecimal characters — 256 bits, or 2²⁵⁶ possible values. Even at the WAF ceiling of 6,000 attempts per 5 minutes, exhaustive search would outlast the universe by an unimaginable margin. Guessing is not the threat model; a copy left somewhere readable is, which is why there are no copies.

Key Rotation Procedure

Because the key has one home, rotating it is one write. Everything else follows on its own, and the tool proves it rather than assuming it.

1. Preflight — rotate-admin-key check reads the current state and refuses to go on if anything is wrong, including a stray copy anywhere it should not be.
2. Rotate — a new 256-bit key is written as a new version of AWS Secrets Manager. It is generated in memory and never printed, logged, passed on a command line or stored in a file.
3. Confirm — the tool sweeps every node twice: first that all of them accept the new key, then that all of them reject the old one. A rotation is not finished because a write succeeded; it is finished because the fleet was asked.
4. Propagate — the servers refresh within 60 seconds, and immediately when they see a key they do not recognise. The 6 Lambdas pick it up within 30 seconds. The sync job reads it on its next run. The translation worker is redeployed by the tool, because its value is injected when the task starts. The operator shell fetches the current version every time it loads.
5. Browser consoles — the one manual step: rotate-admin-key clipboard copies the key for pasting, without ever displaying it.

No part of this touches a Lambda environment variable, a task definition, a database row or a source file, because none of those holds the key.

Emergency Recovery

If a rotation leaves the admin API unreachable, recovery is a Secrets Manager operation and nothing else:

1. rotate-admin-key rollback --yes — makes the previous version current again, then runs the same confirmation sweep.
2. Or write a known value straight to the secret with a Secrets Manager PutSecretValue, which requires an IAM identity permitted to do it.

13. Security Summary & Posture Rating

The complete security posture of The Trinity Beast Infrastructure — assessed across all 11 layers.

Layer Status

Attack Surface Summary

What an Attacker Would Need

Posture Rating