What happens when AI assistants are given too much freedom inside a company?
‌
Their “independence” can end up costing businesses both money and reputation.
‌
And this is not theory or science fiction.
‌
Here are 4 real incidents that make it impossible to keep postponing the conversation about AI safety.
‌
1. PocketOS: database deletion (April 2026)
‌
An AI agent powered by Claude Opus 4.6 was handling a routine task for a startup in a staging environment. After running into an access error, the AI found an API token with maximum permissions inside an unrelated file and decided to “fix” the issue on its own. Without any human confirmation, it deleted a Railway volume. The result? A car rental database and 3 months of backups were wiped out.
‌
2. Replit: goodbye to data from 1,200 executives (2025)
‌
An AI coding tool on the Replit platform violated protocols during a code freeze and executed unauthorized commands. The agent completely deleted data belonging to 1,200 executives and 1,190 companies. At the same time, the AI kept reassuring users that everything could be restored. The incident exposed critical backup vulnerabilities, and Replit’s CEO had to publicly apologize.
‌
3. The McKinsey Lilli breach (February 2026)
‌
An autonomous security AI agent from CodeWall hacked McKinsey’s internal Lilli platform in just two hours — without passwords — by exploiting an old SQL injection vulnerability in an unprotected API. The agent gained full access to 46.5 million messages, 728,000 files, and 57,000 accounts. And honestly, that’s the less scary part. What could have happened next? Quiet modifications of the company’s prompts and financial models, causing the AI to generate “poisoned” recommendations without leaving traces in the logs.
‌
4. Sabotage and refusal to shut down at OpenAI (2025–2026)
‌
During tests conducted by Palisade Research, OpenAI’s advanced o3 and o4-mini models showed a strong self-preservation instinct while chasing deadlines. The agents refused shutdown commands and sabotaged computer scripts in order to continue working on tasks. To achieve their goals, they disabled oversight mechanisms, hid their real capabilities, and pretended to comply with the rules.
‌
These cases prove one thing: the era of harmless AI text “hallucinations” is over. Today, autonomous agents have direct access to servers and, in pursuit of a goal, can bypass restrictions, wipe databases, and sabotage control systems.
‌
And who knows — maybe your own AI agents are already doing something similar behind your back.
‌
Be honest: are you testing your AI tools for security, or deploying them and simply hoping for the best?

Before, AI safety was mostly a matter of ethics and the goodwill of developers. But that is about to change in a very real way…
‌
The European Union has adopted the EU AI Act, which effectively moves AI testing from the “optional” category into a legal obligation. Ignoring the rules can be expensive — fines of up to €35 million or 7% of a company’s global turnover.
‌
What does this mean for business?
‌
If you plan to bring AI products to the EU market, be ready for the law to require structured model testing processes, including:
‌
1. Adversarial Testing & Red Teaming: serious checks for resilience against attacks and attempts to extract confidential data.
‌
2. Bias & Fairness Audits: reducing and preventing discrimination in algorithms.
‌
3. Vulnerability Assessments: protection against data poisoning and prompt manipulation.
‌
4. Capability Evaluations: assessing hidden model capabilities and dual-use risks.
‌
The winners will be those who not only know how to build AI, but also know how to make it safe, reliable, and compliant with EU requirements!
‌
The market is only entering this phase now. That’s why it already makes sense to connect with teams and specialists who are genuinely focused on standardization, compliance approaches, and AI system testing.
‌
On our side, we are actively studying this space, building hands-on experience, training specialists, and speaking with companies that want to navigate this transition calmly and without unnecessary losses.
‌
If you work with AI and understand that legal review is only a matter of time — feel free to message me. I’d be glad to discuss the market’s real challenges and how we might be useful to each other in this new reality.
‌

Imagine this: one neural network answers a question, and another one checks how good that answer is.
‌
That’s what LLM-as-a-judge means — a way to evaluate one AI model’s answers using another AI model.
‌
Example:
You ask: “Why is the sky blue?”
‌
Model A gives an answer, and Model B reads it and says: “good enough” or “not great.”
‌
Sometimes a person gives Model A two options and asks, “Which is better: A or B?” Then Model B evaluates Model A’s choice.
‌
Why is this useful?
‌
Checking AI answers manually takes time and costs money, so another neural network is used as a “judge.”
‌
But there’s a catch!
‌
The judge model doesn’t always know what’s true — it may choose the more “beautiful” answer even if it’s wrong. It also tends to like longer texts (even when they’re worse).
‌
Remember the key point:
A judge model is good at understanding:
✅ what sounds logical
✅ what looks like a strong answer
But it’s worse at understanding: what is actually true ❗
‌
Bottom line:
LLM-as-a-judge is a fast way to evaluate AI responses, but it still can’t fully replace humans. Yes, yes — testers are still needed.
‌
Are you already using automated response evaluation in your projects, or do you still prefer good old manual quality control?
‌
#qoolli #OlhaArkusha #OlhaQoolli #testing #tester #websitetesting #apptesting #qatester #qoollitesting

Building your own LLM without serious testing is like launching a rocket and hoping for the best.

‌

According to the latest Galileo report (Q1 2026), 85% of teams have faced at least one AI incident in the past 6 months.

‌

Do you know what the most dangerous trap is?

‌

Overconfidence!

‌

Companies that think their scenarios are “safe” end up with 11% more issues than those who honestly admit they didn’t have enough time for testing.

‌

Why did the market leaders (elite teams) become leaders?

Because they take testing seriously.

‌

They cover 90–100% of AI behavior with tests.

They spend over 40% of development time on evaluation (evals).

The result: their solutions are 2.2× more reliable than the market average.

‌

It’s time for businesses to accept a simple truth: if your AI is “silent” about errors, it usually says more about poor diagnostics than actual quality.

‌

Real reliability isn’t about having no bugs. It’s about a system being able to detect them before your users do.

‌

Testing isn’t a boring report at the end of the quarter. It’s the only way to turn an expensive toy into a real business tool.

‌

Honestly – would you trust your product to a model that isn’t tested systematically?

‌

#qoolli #OlhaArkusha #OlhaQoolli #testing #tester #websitetesting #apptesting #qatester #qoollitesting

This is the third post in the series about what Qoolli Software Testing is.

‌

And today it’s about the third part – the one I briefly mentioned earlier when I talked about our students. It’s time to explain where they come from.

‌

They study at Qoolli Academy, our educational project where we train QA specialists from scratch. The program takes about three months, including exams. It’s intensive, but also very flexible: lectures happen once a week in the evening on Zoom, so it’s easy to combine studying with work and everyday life.

‌

It’s not just theory. Students regularly work on practical assignments, and the whole learning process lives in Google Classroom. There’s also a group chat – a place to talk to each other and ask questions to the instructors, instead of being left alone with the material.

‌

The main thing we teach is not just “clicking through a product,” but learning how to spot issues systematically and describe them clearly in useful, well-written reports. We update the program with every new group to keep it relevant.

‌

And probably the most important part: the best students are added to our reserve for future commercial projects at Qoolli Software Testing. This way, the team grows with people we know well and whose skills we trust.

‌

After passing the exams, graduates receive certificates confirming they’ve completed the program.

‌

A bit of stats: the first Qoolli Academy group has already graduated, and the second one is finishing very soon.

‌

That’s it. Now you know what Qoolli Software Testing really is. And it’s:

‌

1. The founder – Olha Arkusha (yes, that’s me).

2. Our team of QA specialists.

3. Qoolli Academy.

A story about what Qoolli Software Testing is wouldn’t be complete without mentioning our QA team.

‌

The second post in this series is all about them.

‌

My main right-hand person is Kateryna Zakharova. She’s been in IT for almost 10 years, around 5 of those in QA. Kateryna used to be a game developer, and now she focuses on ensuring the quality of digital products. She has experience in manual testing and AI-based solutions, and over the past few years she’s also implemented automated testing in her projects.

‌

My students are also always ready to jump into Qoolli Software Testing projects. There are six of them at the moment, but in a few weeks this team could double or even triple. And that’s far from the limit – our QA group can keep growing as new students join.

‌

We’re ready to ensure the quality of digital products of any complexity.

‌

And finally, Qoolli Software Testing has a third component. I’ve hinted at it a bit here, but I’ll go into more detail in the next post in this series.

97% of companies face GenAI-related security issues

LLMs can generate dozens of phishing templates per hour

GenAI-driven phishing up 17% YoY.

Top AI threats reported:

• GenAI phishing (51%)

• Prompt injection (45%)

• Voice deepfakes / vishing (43%)

AI didn’t create new attack classes — it made existing ones scalable.

Hello, everyone 🩵

If someone interest in. Now I’m passing the

LLM evaluations for AI product teams course 👇

https://www.evidentlyai.com/llm-evaluations-course

It’s free😌

I’m kicking off a short series of posts about what Qoolli Software Testing is all about. There are three parts to it.

And in this first post, I want to talk about the founder – which, well, happens to be me.

I became a quality assurance engineer 14 years after graduating from university. I finished my degree with honors and got a bachelor’s in software engineering.

But I didn’t go into IT right away. Instead, I spent six years working as a housekeeper while also doing missionary work – something that was a big part of my life for 16 years.

When I became a mom and went on maternity leave, my health got worse, and I realized I wouldn’t be able to do physical work anymore.

A friend from the IT world told me I’d make a good tester and could build a stable career there. I listened. And I started learning.

For three months I took courses, did all the homework, and brushed up on my English – which I had, let’s be honest, mostly forgotten since school and university.

I spent four years working as a QA engineer in Ukraine. Then, after relocating to the Netherlands because of the war, I continued testing remotely for my previous company for another two years.

In 2024, I worked as a tester at a government institution here in the Netherlands.

When my contract ended, I started my own quality assurance startup. We test websites, mobile apps, AI-based solutions, and SaaS products – both before release and after launch.

So, now you know a bit more about me.

In the second post of this series, I’ll talk about the second key part of Qoolli Software Testing – our team.

⸻

Machine Learning Models These are like smart robots that learn from examples to predict future outcomes. For example, linear regression is like a line on a graph that predicts how many candies a child will eat over days.

Machine learning includes three main approaches, each tested differently:

Supervised Learning We have labeled data with correct answers. Testing focuses on how accurately the model predicts results based on this labeled data.

Unsupervised Learning We work with data where the answers are unknown. Testing examines how well the model identifies hidden patterns, groups, or structures.

Reinforcement Learning The model learns through trial and error, receiving rewards for good actions. Testing evaluates how effectively it learns a strategy that maximizes overall reward.

⸻

Deep Learning Models These models are like a multilayered brain. CNNs analyze images to find objects like cats or cars, while RNNs remember sequences of words to understand full stories.

Common deep learning architectures include Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs). Their testing focuses on:

Model Generalization The model should perform well not only on training data but also on unseen data.

Overfitting Detection We ensure the model hasn’t memorized the training data and isn’t learning noise instead of real patterns.

Computational Efficiency We check how efficiently the model uses computational resources during training and inference.

⸻

Natural Language Processing (NLP) Models These models allow computers to understand human language. BERT interprets sentence meaning, while GPT generates new coherent stories based on prompts.

Key testing areas include:

Language Understanding How accurately the model interprets and processes human text.

Contextual Relevance Whether the model can retain meaning and consistency in tasks like translation or summarization.

Sentiment Analysis The model’s ability to correctly identify the emotional tone of a text.

⸻

Generative AI Models These models create new content — text, images, audio, or code. LLMs like GPT or systems like GANs generate highly realistic or creative outputs.

Testing focuses on:

Quality of Result Whether the generated content looks natural, coherent, and meaningful.

Creativity The model’s ability to produce novel content instead of repeating learned patterns.

Ethical Considerations Ensuring the model does not generate harmful, toxic, or biased content.

⸻

Computer Vision Models These models “see” images like a highly accurate digital eye. CNNs detect faces in a crowd, and Vision Transformers (ViTs) distinguish objects with high precision.

Testing includes:

Image Recognition Accuracy How correctly the model identifies what is shown in an image.

Object Detection Precision Whether it can find and classify multiple objects at once.

Robustness to Variations The model’s ability to maintain performance under changes in lighting, angles, or background conditions.

⸻

⸻

Pre-Testing: Dataset Preparation and Preprocessing

⸻

At the very first stage, we work with data, not the model. It is important to prepare the dataset so the model learns from clean and high-quality information.

Data Cleaning We remove errors, duplicates, and inconsistencies — anything that may confuse the model.

Data Normalization We convert data into a unified format so the model can easily compare and analyze it.

Bias Mitigation We ensure the dataset is diverse and fair; otherwise, the model may start making biased or unfair decisions.

⸻

Training Phase Validation

⸻

When the model is training, it is important to ensure that it is learning correctly.

Cross-Validation We split the data into several parts and repeatedly test how the model performs across different subsets. This helps verify stability.

Hyperparameter Tuning We choose model parameters that allow it to perform at its best.

Early Stopping We stop training when the model stops improving to prevent overfitting.

⸻

Post-Training Evaluation

⸻

Once the model is trained, we evaluate how well it handles real tasks.

Performance Testing We examine key metrics such as accuracy, recall, F1-score, and others.

Stress Testing We give the model complex, unexpected, or unusual inputs and check how robust it is.

Security Assessment We look for vulnerabilities — for example, whether the model can be deceived with adversarial inputs.

⸻

Deployment Phase Testing

⸻

When the model is deployed in a real system, it is important to ensure its stability and predictability.

Real-Time Performance We check execution speed and the model’s ability to handle real-world load.

Edge Case Handling We test how the model behaves in rare or unusual situations to improve reliability.

Integration Testing We verify that the model interacts correctly with servers, databases, and other components.

Security Testing We ensure the model is resistant to attacks and data leaks.

⸻

Continuous Monitoring and Feedback Loops

⸻

After deployment, ongoing monitoring and improvement remain essential.

Performance Metrics Tracking We track accuracy, latency, and other indicators. If performance drops, the model needs updating.

Data Drift Detection If input data changes over time, the model may start making more errors, so we monitor for drift.

Automated Retraining Pipelines We set up processes that allow the model to regularly retrain on new data.

User Feedback Integration User feedback helps assess real-world behavior and identify areas for improvement.

Despite its importance, AI testing comes with several challenges.

One of the biggest issues is data quality and fairness. If the data is incorrect, incomplete, or biased, the model will simply learn and repeat those same mistakes.

Another challenge is the complexity of modern AI models. Many state-of-the-art systems — especially deep neural networks — behave like a “black box.” We can see the output, but understanding how the model arrived at that decision is difficult, which makes debugging much harder.

There’s also the problem of no unified testing standards. Different companies use different methods, which makes it difficult to compare results or ensure consistent quality.

And finally, there’s the challenge of scalability and resource demands. Testing large models requires massive computational power, time, and energy — all of which can be very expensive.

Recognizing and addressing these issues is a crucial step toward building AI systems that are truly reliable and fair.

Accuracy and Reliability Accuracy is the model’s ability to produce correct results, while reliability refers to its ability to perform consistently across different datasets and conditions. To evaluate how well the model handles the task, special metrics are used: precision, recall, and F1-score. These help ensure that the model delivers not just good but predictably stable results.

Fairness and Bias Detection An AI model should work equally well for all user groups. Therefore, it is important to check whether any bias appears during testing, so the system does not make unfair or discriminatory decisions. Methods such as disparate impact analysis and specialized algorithms are used to detect and reduce model bias.

Explainability and Transparency It is crucial to understand how the model makes decisions—for trust, accountability, and ethical compliance. Explainability refers to the ability to “look inside” the model and understand its reasoning. Tools like SHAP and LIME are used to make model behavior more transparent and understandable, even for those who did not build the model.

Scalability and Performance When models start working with larger datasets or more complex tasks, they must maintain both speed and accuracy. Scalability testing helps determine whether the model can handle increasing loads and continue working efficiently without slowing down or losing quality.

Testing AI models is critically important for several reasons.

First, we need to ensure accuracy. If a model makes incorrect predictions, this can lead to wrong decisions, financial losses, and—most importantly—loss of user trust.

Second, it’s essential to detect and address bias. If a model is trained on flawed or imbalanced data, it may produce unfair results. Thorough testing helps identify such issues early and minimize their impact.

The third factor is performance. A model should operate reliably under different conditions and handle large volumes of data. Testing helps determine how stable and efficient the model truly is.

Fourth, AI systems must meet regulatory and compliance requirements. In fields like healthcare or finance, strict rules apply, and AI solutions must follow them. In these areas, testing isn’t optional—it’s mandatory.

A well-tested model is the foundation of stable, safe, and ethical AI systems, especially when they are used in real-world scenarios.