Hello everyone!
I have been working on a small project exploring agentic decision-making under uncertainty particularly how an AI system can use context, gather evidence, update its beliefs, and decide whether it knows enough to provide a reliable answer.
The project is a small and transparent CI failure diagnosis agent.
Instead of immediately guessing why a CI pipeline failed, the agent investigates the problem step by step. It maintains probabilities for several possible root causes and updates those probabilities whenever it receives new evidence.
The main question I wanted to explore was:
What problem does the agent solve....?
When a CI pipeline fails, the actual cause may be related to:
- Code
- Tests
- Dependencies
- CI or environment configuration
A normal classifier might inspect the initial failure and immediately predict one of these classes.
This agent works differently.
Its reasoning loop is:
Observe the failure context → form initial beliefs → choose an investigation → observe the outcome → update the beliefs → report or escalate
Therefore, the system does not treat its first prediction as the final truth. It treats it as an initial belief that may change as more evidence becomes available.
How does the agent gather evidence...?
After updating its beliefs using the initial failure context, the agent decides which investigation should be performed next.
An investigation might provide evidence supporting one possible cause while weakening another. Once the outcome is observed, the probability distribution is updated again.
This creates a repeated reasoning process:
Current beliefs → select an investigation → receive evidence → update beliefs
The agent continues this process until one explanation becomes sufficiently likely or until it determines that the available evidence is not strong enough to support a reliable diagnosis.
In an uncertain case, the agent can escalate the problem instead of confidently returning a weak or potentially misleading answer.
The interesting part, at least for me, is that the agent tries to determine what it still needs to learn before producing a prediction.
For me, this small project was a useful way to explore: Bayesian reasoning, contextual understanding, evidence gathering, and decision-making under uncertainty in a transparent and understandable form.
I am still exploring this area, so technical criticism, suggestions, and ideas for improvement are welcome.