Allan Simões AdegasPortuguês
Menu
Back to home

BTG Pactual · Software Engineer (Operations) · 2024 – 2026

Transaction monitoring with event replay

Critical processes moved millions of dollars and had to be tracked without losing events when something failed along the way. I built event-driven microservices on AWS with a backup of every payload for reprocessing, async APIs built for high availability, and React dashboards for risk and operations.

US$ milhões
in transactions monitored
Replay
from the original payload
20+
automated workflows delivered

Context

Financial operations in Brazil, Chile, Colombia and Mexico depended on processes that needed close tracking: if something stalled, risk and operations had to know quickly.

The challenge

In monitoring, losing an event is worse than delaying it. The system had to absorb peaks and dependency failures, and reprocess what went wrong without risky manual intervention.

What I built

Async ingestion
FastAPI services in a serverless environment take events in quickly and respond without waiting for processing.
A queue in the middle
SQS decouples producers from processors. A spike or a temporary failure becomes a queue, not an error.
A backup of every payload
Payloads are kept in DynamoDB and SQS. If processing fails, it can be redone from the original, without depending on anyone resending it.
A view for the people who decide
React dashboards for risk and operations analysts, fed by pipelines that pull from several sources at once.

Architecture

Architecture: Transaction monitoring with event replaySources → APIs FastAPI. APIs FastAPI → Amazon SQS. Amazon SQS → AWS Lambda. AWS Lambda → DynamoDB. DynamoDB → Dashboards. AWS Lambda → Reprocessing. Reprocessing → Amazon SQSreadsfailedSourcescritical processesAPIs FastAPIasync, serverlessAmazon SQSabsorbs spikesAWS LambdaprocessorsDynamoDBstate and backupDashboardsReactReprocessingfrom the backup
Simplified view. Internal names and details are omitted.

Decisions and trade-offs

Durability before speed
Storing the payload before processing costs a little storage and guarantees no event depends on luck.
Serverless for the load profile
Irregular, spiky loads fit Lambda and SQS: you pay for use and scale follows the queue.
Reprocessing is a normal path
Treating failure as part of the flow (not a manual exception) reduced the operational risk of fixing data by hand.

Results

  • Millions of dollars in transactions monitored, with replay from the original payload.
  • More than 20 automated workflows in production, saving over 300 hours a month for Operations teams across Latin America.
09001,8002,7003,6001234567891011123,600 hoursmonth
Twelve-month projection at 300 hours saved per month.

Architecture described at a high level for confidentiality.

What I'd do next

  • Infrastructure as code for the whole environment, to recreate and audit it with one command.
  • End-to-end distributed tracing, to measure the time spent at each queue stage.

Stack

PythonFastAPIAWS LambdaAmazon SQSDynamoDBS3ReactPandasPlaywright