fix docker container name extraction in fluent bit#270
Merged
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #269
Problem
The shipped Fluent Bit pipeline has a
Copy container_name servicefilter, but nothing ever setcontainer_name:extract_container_id.luaonly extracted the container ID from the log filepath (/var/lib/docker/containers/<ID>/<ID>-json.log), and the container name simply is not part of that path. The copy was a silent no-op, so logs reached LogTide with noservicefield at all and onlycontainer_id/container_short_idin metadata, which is what showed up in the UI.Fix
extract_container_id.luanow resolves the container name from the container'sconfig.v2.json, which Docker keeps next to the log file and which is already available through the existing read-only/var/lib/docker/containersmount in both compose files (no compose changes needed):"Name":"/<name>"); container names always carry the leading slash there, so volume mountNamekeys in the same file cannot be picked up by mistakeconfig.v2.jsonis missing or unreadable,container_namefalls back to the 12-char short ID, soserviceis always populatedVerification
Tested end-to-end against the real
fluent/fluent-bit:4.2.2image with the repo'sparsers.conf, the production filter chain and a fake/var/lib/docker/containerstree (one container with aconfig.v2.jsonincluding a decoy volumeName, one without the file):Before (current main): records had no
serviceat all.After:
container_nameis still stripped from the final record by the existingrecord_modifierfilter, and the backend ingest fallback chain (service || container_name || ... || 'unknown') is untouched.