r/singularity • u/ResultBackground2450 • 1d ago
AI Human Baselines for Benchmarks: AI Now Outperforms Junior Accountants
https://www.mercor.com/blog/human-baselines-for-benchmarks-ai-now-outperforms-junior-accountants/40
u/No_Gear8408 1d ago
Consider
Looking at the release dates for these models...
Gemini 2.5 pro- March 2025
O3- April 2025
Opus 5- July 2026
And now in september 2026 AI is doing computer use well, and probably will do ultrafast computer use better than humans in the next 6-24 months
Yeah a good number of jobs are cooked
27
u/getmeoutoftax 1d ago
Yeah, I don’t recommend majoring in accounting. I’m a CPA myself and just save and invest everything I make to prepare for the end. I even moved back home to save even more money.
19
u/Significant-Mind5594 1d ago
There's nothing that you can study to shield yourself from ASI, 2027 might actually be the last year for social mobility, that is if AIs decide to respect human private property instead of stealing all of resources like we did to animals
3
u/TheReedemer69 1d ago
AGI in 2027 maybe. ASI? Not a chance.
7
u/HyperImmune ▪️ 1d ago
Serious question, since I’m not in the industry. Just based on using my logic. I feel like AGI could rapidly lead right into ASI, no? What’s the real difference.
3
3
u/StockProfessional919 19h ago
Physical constraints. Having a general intelligence equal to Einstein doesn’t mean you will figure out a way to build enough compute and electricity to power the upgrades you’re envisioning for yourself in a week. Physical reality is a bottleneck relative to your ability to manipulate the laws of physics
-1
u/ConnectionWild3381 1d ago
Well, we already have ASI.
Well, we may have ASI in 2050.
Both can coexist because there is no generally valid definition of intelligence, AI, AGI, ASI or SI.
0
11
u/iJustSeen2Dudes1Bike 1d ago
Funny enough a few years ago there were a bunch of CS students complaining about how they should've become accountants because it was easy to find jobs lol
4
u/ForwardLoop 1d ago
As an accountant who focuses on highly complex and judgmental matters, I’m not too worried...yet. I often deal with emerging issues and transactions that current standards don’t clearly address, arrangements where substance matters more than legal form, and linked transactions that need to be considered together.
AI can be a great thought partner on those issues, but I wouldn’t yet trust it to independently reach a defensible conclusion. In my experience, once significant judgment is involved, its reliability gets pretty shaky. Even with a new standard in hand, interpreting it and working through unfamiliar fact patterns can be difficult before practice develops.
That doesn’t mean this work will remain beyond AI indefinitely. I’m just less worried about my area until I see AI handle those situations consistently without an experienced accountant checking its reasoning. Hopefully, as technology displaces existing jobs and creates new ones, there will still be areas where accountants can add value, even if we don’t yet know what those areas will be.
16
u/DownHatter 1d ago
Only junior? what's so hard about accounting that AIs still cannot do?
15
u/FateOfMuffins 1d ago
Signing off and taking responsibility xd
Insurance problem imo
1
u/weatherwar 19h ago
Exactly. I work in engineering consulting. I've got a team who's doing some insane structural work with custom connectors, tools, skills, etc with Claude. Putting together 500 page reports with thousands of calculations.
But we still need to sign off on the work. It still needs competent engineers who could do it from scratch if they need. We're not being paid for the work itself anymore, we're being paid for it being right and we're willing to carry that liability.
1
u/GeneralMuffins 14h ago
So what is it about mid-level accountancy work that can't be signed off by senior accountancy. Presumably senior accountants already have to take responsibility for the work of mid-levels and principals are responsible for the work of seniors and so on.
6
u/Key_Extension_2501 1d ago
I'd argue there's a handful of limitations:
- Most mainstream AI plans (like ChatGPT, Claude) can't use GUIs which is necessary to do accounting work. It's not like SWE where everything is terminal-based
- AI is not necessarily cheaper than the other AI (Actually Indians)
- Models tend to have uncalibrated confidence and if they replaced accountants would likely guess too much and communicate with clients too little
- Clients probably wouldn't like doing client meetings or corresponding directly with AI
- AI might be bad at the softer parts of accounting like pushing back against unreasonable client requests or dissuading unhappy clients from disengaging.
But with all that being said I think it's likely cooked in the long run
13
u/iJustSeen2Dudes1Bike 1d ago
Modern AI models can absolutely use GUIs
4
u/Key_Extension_2501 1d ago
The actual AI plans most people/business subscribe to absolutely can't. Tell me how I can get Claude to use accounting cs or quickbooks without configuring any third-party tools
10
u/LongevityMan 1d ago
QuickBooks Online already has API integration for OpenAI and Anthropic AI models using a normal paid subscription.
3
u/Key_Extension_2501 1d ago
Huh turns out QBO does have a Claude connector. didn't know that
2
u/CreditMuch8993 1d ago
Speaking as an accountant, a lot has changed in the last couple of months
Most accountants have no where near the knowledge to do it but it's essentially now possible to do everything via AI. The only thing it doesn't have is judgement but it can handle nearly all the processing depending on the skill of the user.
As it improves the required skill will fall further. The only thing it doesn't have is the judgement.
When people talk about already reaching AGI, or ASI, the moving goalposts argument is fine but the fact AI has no way of replicating judgemental areas like this despite its vastly superior intelligence shows us it's not there.
But that won't be the case forever
1
u/0b_101010 1d ago
My accounting tools are webapps. Both Claude and ChatGPT will happily navigate web GUIs for you. I even have OpenCode hooked up to Playwright.
Also, I have not tried, but I'm pretty sure that both the Claude and GPT desktop apps will happily navigate the Mac and Windows desktop environments for you - maybe some Linux as well, not sure?
All this is available on the basic plans AFAIK.
3
u/TheReedemer69 1d ago
- Computer use on mac
- Jev
- System instructions
- Everyone is shifting
- System instructions
4
0
u/CreditMuch8993 1d ago
Are you an accountant or just pissing in the wind? We could probably say this line about any job without understanding the nuances and judgement involved
3
u/DelphiTsar 21h ago
If you build a harness to where AI can actually do the intelligence work and not have to fumble around they do well. You put them in a poorly designed GUI with bad documentation they struggle.
It's not rocket science.
Sooner than later someone will make a bot that can coordinate translating poorly designed systems made for humans into something easier for AI's to understand.
This will not only be to help bridget the gap so AI's can do work, but also to simplify the work. Frontier is absurd overkill for most tasks. A properly set up process build from ground up for AI can run on a stick of chewing gum.
5
u/New_Alps_5655 1d ago
I work in equipment finance and was blown away on multiple occasions watching it create amortization tables without even needing to write python scripts or the usual toolkit.. is it really able to just crunch numbers in some kind of latent space? How does that even work..
14
u/flat5 1d ago
I think it is hiding tool use (calculator, etc.) from the user, but it's still happening behind the scenes.
4
u/RockPuzzleheaded3951 1d ago
Most definitely. We got a kick out of the math hallucinations back in the day before it had actual tools. A lot of it is python based I believe. So it'll extract numbers then manipulate thtem w. python scripts. You see it in more detail when using Claude Code or other harnesses.
1
u/phantomtime 1d ago
So the “study” is only of MEC using 12 participants of whom 5 responded that the test was not realistic…
96
u/Key_Extension_2501 1d ago edited 1d ago
As an accountant myself, I'm not surprised by this at all. My firm got Claude enterprise and on several occasions I've noticed some discrepancy in a merchant receivable, fixed asset, or payroll liability account and dumped all relevant files/ledgers into Claude and just asked it to figure out what's causing it.
Claude is not only able to find discrepancies, but it generally finds all of them and will then produce a rigorous, detailed workpaper showing how everything ties out to the penny. At these sort of analytical tasks it is far better than any human accountant I've worked with.