Uploading a large OpenTelemetry trace to a local OTLP/HTTP traces receiver

Featured image for sharing metadata for article

I've recently been doing some work with improving the performance of Renovate on larger repositories.

To guide this investigation, I'm leveraging our OpenTelemetry support to more meaningfully understand how Renovate operates in parts of the codebase, and hone in on areas of improvement.

One of the repositories I'm running these tests is particularly large, and leads to traces of roughly ~30MB in size.

When trying to ingest them into my local Jaeger instance, I hit HTTP size limits, and the documentation I'd written in the past (with help from Claude Sonnet 5) for ingesting these doesn't seem to handle the size limits.

With a bit of tweaking with Claude Sonnet 5, this is now better handled with:

while IFS= read -r line || [ -n "$line" ]; do
  curl -s -X POST http://localhost:4318/v1/traces \
    -H 'Content-Type: application/json' \
    --data-binary @- <<< "$line"
done < /tmp/traces.jsonl

Notice that we redirect (via a herestring) the line's contents in via stdin, instead of passing it as an argument, which is what led to the shell crashing.

With this few-liner this leads to us being able to ingest very large traces!

Written by Jamie Tanna's profile image Jamie Tanna on , and last updated on .

Content for this article is shared under the terms of the Creative Commons Attribution Non Commercial Share Alike 4.0 International, and code is shared under the Apache License 2.0.

#blogumentation #opentelemetry #command-line.

πŸ€– Content in this blog post (prose or code snippets) includes code derived from the following LLMs:

  • claude:sonnet-5

This post was filed under articles.

Interactions with this post

Interactions with this post

Below you can find the interactions that this page has had using WebMention.

Have you written a response to this post? Let me know the URL:

Do you not have a website set up with WebMention capabilities? You can use Comment Parade.