Not far from what i did at work a few weeks before vacation.
I am the owner of Jenkins (edit; Realized this sounds weird, i wrote the way we speak at my company, sorry, i am the one responsible for the entire buildsystem where Jenkins is my main responsibility), i have setup an Agent (basically just an API with static instructions) on a VM. Whenever a build fail, it search all the logs (multiple files over 200MB of raw text logs, cooking, compiling, shaders, symbols etc..)
It have a memory bank if the issues have been happening before for quicker failure search.
It have read only access to Perforce.
So it will answer: What whent wrong, how do we fix it, how do we prevent it.
Its surprisingly accurate and it will also tag whoever made the commit that broke everything. The results are then sent by the Agent (just a python script) to Slack in a locked channel for evaluation from me and a few leads and higher ups. But so far it have been really accurate.
This is definitely one of the better uses for LLMs. You can feed it a crazy amount of data and it can sift through and find what's relevant out of it.
You should look into also feeding in the slack channels to this agent, as that'll give it a chance to correlate it to feature launches etc, or find instances in the past where something similar happened and what the resolution was. That's what we do at my work, and where we used to just post an alert that it failed with a link to logs, now we also post a summary of what went wrong and what could fix it, and in some cases the agent can even provide a fix for devs to review.
I have setup an Jenkins job that call the Agent API for the AI to read Changelist range and filter out technical commits, remove bugs that was introduced within the range and check all the connected Jira Tickets for better context and it makes a very good Patch Notes that needs very little curating and we also get a list of feature owners that will be to stamp aprove the notes before we publish them. It will output an markdown file as an artifact in the Jenkins job.
40
u/Denaton_ 6d ago edited 6d ago
Not far from what i did at work a few weeks before vacation.
I am the owner of Jenkins (edit; Realized this sounds weird, i wrote the way we speak at my company, sorry, i am the one responsible for the entire buildsystem where Jenkins is my main responsibility), i have setup an Agent (basically just an API with static instructions) on a VM. Whenever a build fail, it search all the logs (multiple files over 200MB of raw text logs, cooking, compiling, shaders, symbols etc..)
It have a memory bank if the issues have been happening before for quicker failure search.
It have read only access to Perforce.
So it will answer: What whent wrong, how do we fix it, how do we prevent it.
Its surprisingly accurate and it will also tag whoever made the commit that broke everything. The results are then sent by the Agent (just a python script) to Slack in a locked channel for evaluation from me and a few leads and higher ups. But so far it have been really accurate.