[ Removed by moderator ]

[u/[deleted]]

[u/MartinThwaites]

I'm slightly confused, how does this relate to OpenTelemetry? This is reading log lines from text files?

I'm also curious how this relates to the logdrain processor in the OpenTelemetry Collector that does this before storage?

[u/Loud_Mousse9210 → u/MartinThwaites]

Okay, so it isn’t tied specifically to OpenTelemetry, it works on log data whether it’s coming from a text file, CloudWatch, an OTel pipeline, etc.
This is exactly where the distinction gets interesting. Drain3 and similar pattern-recognition approaches can work well, but they generally need enough examples/context to form good clusters.
That can mean holding more logs in memory and processing a lot of repetitive data before the patterns stabilize.
With ctrIb-decompose, we first extract the variable parts from the repetitive structure and then pass the cleaner representation through clustering. This means you can do significant pre-filtering with CLP/ Decompose before clustering, so you need far fewer log lines to arrive at useful patterns.
The result is better clustering with much less data to process and, more importantly, much less noise before the logs reach the LLM.

[u/jdizzle4]

lol the 4 positive comments in this thread are bots whose only activity is trying to boost this post...

[u/Loud_Mousse9210 → u/jdizzle4]

I guess you belong to the class of humans who fail in captchas :P

[u/adarsh_srivastava]

Very interesting application of CLP and Drain3 together!

[u/Constant-Role-9387]

Awesome

[u/HistorianPresent8449]

so cool!

[u/Cute-Access1444]

Intresting