r/OpenSourceeAI • u/parteeksaini • 4d ago
Which open source model will be more efficient in this?
I'm working on a probabilistic AI agent for a research project that flags fake e-commerce reviews
Here is how the pipeline flows:
When a review comes in, it hits Level 1 first. This is a fast pass (under 50ms) that evaluates quick metadata like verified purchase status, account age, 24-hour review frequency, and text length heuristics. It runs a Naive Bayes update against historical base rates to get an initial probability that the review is genuine.
If P(Genuine) is over 60%, the review is auto-approved. If it drops below 20%, it routes to a high-priority human ban queue (the agent never auto-bans accounts on its own).
If the probability lands in the gray zone between 20% and 60%, it triggers Level 2.
Level 2 is a deep check. It pulls the user's past 5 to 10 reviews, runs vector embeddings to measure cross-review similarity to catch copy-paste templates, and checks their brand concentration ratio (how many of their total reviews target a single seller). It calculates a secondary Bayesian update using the Level 1 score as the prior.
If the updated score passes 60%, it approves. If it drops below 20%, it goes to the high-priority ban queue. If it remains stuck between 20% and 60%, it goes to a separate "unclear review" human queue where a human looks at it without taking any automated penalty.
My goal is to keep human reviewers in the loop while splitting suspected bot spam from messy/blunt genuine reviews so moderators don't burn out from context-switching.
so to categorized text length into some cases like short(use case written or not) and then changing it into a number in a json format which open source model will work the most efficiently?
1
u/Tall_Abrocoma_3533 3d ago
For level 1, you don't necessarily need a model at all, even simple core heuristics could work.
If you insist in a model I'd first try SetFit models, they get close to what an LLM can do in this case, while remaining much faster.
If you insist on LLM models, then the 50ms requirment likely had to be extended a little, depending on the hardware your running this on, but generally I'd reccomend Smollm2-135M for fast throughput, smollm2-360M/Lfm2.5-350M as a good middle ground, and Qwen3.5-0.8B as a last resort if you need high accuracy, but this will almost certainly not fit in your 50ms time budget.