The Trinity Beast Infrastructure — Message Queue Configuration

1. Overview

Problem

During Run 17 stress testing (746,374 RPS, 1.34B requests), the in-process usage log writer's worker pool saturated under extreme load. Background work was silently shed — only ~12K usage_logs appeared in Aurora after 1.34 billion requests. The price response was never affected (0ms), but usage logging completeness was sacrificed.

Solution

Decouple usage logging from the hot path using SQS. The LPO server sends a fire-and-forget message to SQS for every price request. A purpose-built Go Lambda (trinity-beast-queued-writer) consumes batches from the queue and batch-inserts into Aurora. Zero logs shed. Zero latency added to the price response.

Data Flow

graph LR
    Client["Client"] -->|"price request"| LPO["LPO Server"]
    LPO -->|"0ms response"| Client
    LPO -->|"fire-and-forget"| SQS["SQS Queue
trinity-beast-queued-usage-logs"]
    SQS -->|"batch of 100"| Lambda["Lambda Consumer
trinity-beast-queued-writer"]
    Lambda -->|"batch INSERT"| Aurora["Aurora
usage_logs table"]
        

2. Architecture

Producer: SQSProducer

Source: internal/database/sqs_producer.go

• Channel-buffered pipeline (configurable buffer size via sqs_buffer_size parameter)
• Background flusher goroutine drains in batches via SendMessageBatch (up to 10 per API call)
• Non-blocking Send() — if buffer full, message dropped (protects hot path)
• Runtime-adjustable: flusher re-reads config on every tick

Message Envelope

All messages use a standard envelope format. The type field routes processing in the Lambda consumer:

{
  "type": "usage_log",
  "timestamp": "2026-05-02T15:30:00.000Z",
  "node": "BeastMain",
  "payload": {
    "api_key_id": "ak_live_abc123",
    "asset": "BTC",
    "source": "binance",
    "response_time_us": 42,
    "cache_tier": "L1_MEMORY"
  }
}

Consumer: trinity-beast-queued-writer Lambda

Queue: trinity-beast-queued-usage-logs

3. Application Parameters

All 4 SQS parameters are stored in the application_parameters table and are runtime-adjustable — no redeploy needed.

4. Environment Variables

5. Profile Integration

The 4 SQS parameters are included in all 16 system profiles. Each profile tunes the queue behavior for its intended workload:

6. Cost

7. Extensibility

The message envelope pattern supports future write types without infrastructure changes:

Adding a New Message Type

1. Add h.SQSProducer.Send("analytics_event", data) in any handler
2. Add case "analytics_event": in the Lambda consumer
3. Same queue, same Lambda, same IAM — just a new case statement

// Producer side — any handler
h.SQSProducer.Send("analytics_event", AnalyticsPayload{
    Event:  "page_view",
    Path:   "/pricing",
    UserID: session.UserID,
})

// Consumer side — Lambda handler
switch msg.Type {
case "usage_log":
    batchInsertUsageLogs(msg.Payload)
case "analytics_event":
    batchInsertAnalytics(msg.Payload)
}

8. IAM Permissions

ECS Task Role (ecsTaskRole)

Lambda Role (trinity-beast-queued-writer-role)