CodePark/ questions / CloudWatch, logs & X-Ray
START PRACTICINGPRACTICE
Backend question · Cloud & AWS › CloudWatch, logs & X-Ray

A request goes through API Gateway, a Lambda function, SQS and an ECS worker that writes to DynamoDB. How would you trace it end to end on AWS?

Use distributed tracing. A trace is the full path of one request. A span is one step of that path, such as one service call. Each component creates spans that share the same trace ID. The trace context (the trace ID and the parent span) travels with the request, in HTTP headers or in message attributes.

On AWS, X-Ray collects the traces. It draws a service map that shows the latency and errors for each hop. API Gateway and Lambda can turn on X-Ray tracing with a single setting.

For the Node services, instrument the code with OpenTelemetry. OpenTelemetry is an open standard for traces, metrics and logs. AWS supports it through the AWS Distro for OpenTelemetry (ADOT). Auto-instrumentation creates spans for you for:

  • HTTP calls
  • the AWS SDK
  • common database clients

A collector or a sidecar container then exports the traces to X-Ray or to another backend.

The queue needs extra care. The producer must put the trace context on the SQS message. The consumer must read it and continue the same trace. If not, the trace breaks into two separate traces.

Also write the trace ID in every structured log line. Then you can jump from a slow trace straight to its logs.

  • All spans of one request share a trace ID, and the trace context travels across every hop.
  • X-Ray draws a service map, and you turn on tracing for API Gateway and Lambda with a setting.
  • Node services use OpenTelemetry through ADOT, which auto-instruments HTTP calls and the AWS SDK.
  • Pass the trace context through SQS message attributes so the trace does not break.
  • Write the trace ID in your logs so you can connect logs to traces.