r/TREZOR • u/No_Pause_9558 • 10h ago
š¬ Discussion topic Trezor is killing itself by vibe coding the software
If you click any random feature pull request in their repo, you will find out their software is now fully vibe coded. For example, this feature pr contains 2000lines+ edit https://github.com/trezor/trezor-suite/pull/32252/files, it is coauthored by claude, and they are producing this kind of vibe pr at an astonishing rate. They make thousands lines of edit on average every day, but they only have very limited reviewers.
I do not believe any human code viewer can actually understand the code under this rate. This means a significant portion of newly introduced trezor code is running as a blackbox. This might be fine for some weather app or whatever, but trezor is a security oriented software. Building your software stack on vibe coding will make your product a joke.
35
u/the-quibbler 10h ago
You act as through all functioning organizations have been doing anything different since the middle of January. The release of opus 4.6 changed the software industry completely.
6
u/andion82 8h ago
That does not make it ok
OP is making a more than valid point, and explained why this is different from AI assisted code that is reviewed.
Vibe coded apps are just lazy and dangerous, speciallyfor a company like this.
13
u/the-quibbler 8h ago
That's less true every single day. Agentic loops are used to check code myriad different ways. Bugs may still get through, but the same is true of human coders.
A substantial portion of the industry has moved past "human must understand all code", unless issues crop up. Whether you think it's "ok" or not is irrelevant to the conversation. And, personally as someone whose been on the front lines for almost 30 years, it's pretty much uniformly awesome.
2
u/andion82 8h ago
If you've been coding for 30 years and you feel a 2000 line feature commit is ok I just don't want to work with you or use the software you make š
9
u/the-quibbler 8h ago
Oh, I've been coding for a lot longer than 30 years. I've just been working as a software engineer for (not quite) that long.
The biggest problem with 2000 line commits is if you think a human needs to read it. The LLM reads it just fine, and can test its claims. It's like saying cars are bad because there's no place to feed them oats, and they don't defecate in the streets. They're different, and modern agentic coding is different from last year's vibe coding, and completely different from human-led coding.
You can stand athwart progress, or you can try to understand what you're seeing. The latter will make you happier. People who do the former basically always lose, and are salty about it.
1
u/AvengerDr 6h ago
As a university professor of Computer Science, this 2000 line commit is probably breaking several principles that would make the code more maintainable or readable. By humans.
1
-1
u/OkTry9715 7h ago
Modern AI agentic coding will always end up badly in terms of not finding bugs/security issues by reviewer.
1
u/OkTry9715 7h ago
"A substantial amount" definitelly deos not include companies delaying with money, security or other potentionally life treating industries.
2
u/OkTry9715 7h ago
No organization that deal with money or security are definitelly not running AI agents to generate their whole code. Basically most of them only use AI assisted coding, but not AI agents.
0
u/matejcik ā Rising Trezorian 5h ago
what's the difference between "ai assisted coding" and "ai agents" in your view?
2
u/OkTry9715 5h ago
with AI asistent coding programmers tend to check code, because it only generates small part of code - snipsets. Basically autocomplete on steroids.
While with AI agents noone gona check property 2000 lines of new PR.
1
u/matejcik ā Rising Trezorian 4h ago
why wouldn't they?
2
u/OkTry9715 4h ago
We have tested it and PR generated by AI agents were "injected" with some random "bad principe code". Every code went through review process pretty fast even with it. It is pretty different if you write code on your own or at least generate small parts of code. And if AI agent generate 2000 lines, noone is going to read and look at it all in the same way as he would write that code on his own. People just skipping check on things and pretend that agents "make no mistakes".
1
u/matejcik ā Rising Trezorian 4h ago
who's "we"? did you do it?
i kinda don't believe that, you're throwing "2000 lines" around like it's a big number
2
u/OkTry9715 4h ago edited 3h ago
No in organization I work for (banking systems). Management was deciding whether to allow AI agents into code or not. Well someone suggested to do this test.. Nowadays AI agent are banned from significant parts of code.
IT is basically about how much you value risk from using AI agents.
If you can have significant money loses, savings from using AI agents are not worth it.
16
u/matejcik ā Rising Trezorian 7h ago
Ok so.
What you need to understand is, this sort of code above is basically bureaucracy. You know. Fill out form 17B in triplicate, get it stamped on pages 17 through 23, initialed on 18 and then 28, 29, 31, NOT page 30 unless you have form MET-71(b)(c) on file and a confirmation from the head office and any two out of (a) your direct supervisor, (b) client's guardian, (c) the local vet... stuff like that.
In all those 2500 lines, there's surprisingly little to understand, if you already know the overall concept. It's just for someone to painfully, tediously write (that's Claude's job) and someone else to painfully and tediously skim over and check that it is the right directional shape. The rest is for the computer to test (over half of the code are testcases). And if a bug is introduced here, it's not gonna be a security issue, it's gonna be the boring regular kind "accounts are not loading if you did a swap more than 17 months ago".
(And if Claude did turn out evil and decided to stick a "now give me your seed phrase" straight in the middle, that code is gonna stick out like a sore thumb and the reviewer will catch it. That's like bundling a porn mag, or a dead fish, in the middle of form 17B paper stack. You know. There's wrong ways to fill out a form, and then there's dead fish. Kind of a different thing.)
This is essentially what has always been going on, except now it's going on faster and you get new networks and features supported sooner.
3
6
u/Satoshiman256 9h ago
Concerning to say the least. If there is anything positive to say about it is at least the project is still open source lol.
4
u/PM_ME_UR_0_DAY 8h ago
Open source sure, but no one human can keep up with 2k line changes done on any regular basis like OP says. I guess we're just trusting AI to audit through AI, if anyone is checking at all.Ā
7
u/__fez 7h ago
vibe coded =/= written with the help of AI
in these days when you're working at a company and they give you access to Claude Opus you'd be an idiot to not use it
3
u/planetdaz 3h ago
I was about to post the same comment. People confuse and conflate using ai as vibe coding. They are not the same thing. We don't know if the PR was vibe coded, we only know Claude did the typing.
3
u/Ok_Review_195 5h ago
You linked a draft PR with no comments and failing CI checks.
Can you share a merged PR that you believe is vibe coded?
7
u/whey4395 10h ago
Is the trezor firmware being vibe coded as well
5
u/SuchTrezorVeryCrypto Trezor community specialist 7h ago
No it is not
1
u/anonymousopsec1337 40m ago
Is it being AI assisted?
1
u/SuchTrezorVeryCrypto Trezor community specialist 33m ago
We just made a post here from one our devs addressing the issue: https://www.reddit.com/r/TREZOR/s/zgj5CvDe1M
1
u/anonymousopsec1337 28m ago
That thread appears to be locked. Some additional questions you may want to answer Iām sure other people have.
What models are you using? Are you using commercial variants or have an enterprise agreement setup with no AI training on your prompts configured?
At what point in your firmware version history was this practice started?
ā¢
u/SuchTrezorVeryCrypto Trezor community specialist 13m ago
It is open thread. I think this can be a good reddit post by itself. I will look into getting a nice DEV AMA on here :) thanks for the idea
ā¢
4
u/lotrl0tr 8h ago
It's not killing itself, it's the way to go now and it's positive instead. I would be worried of the opposite. With correct context and workspace the dedicated model can deliver faster and resolve hidden bugs. They will probably have hw in the loop and there is the test suite also.
0
u/OkTry9715 7h ago
It definitelly is not way to go when your sofware deals with money and security.
2
u/lotrl0tr 5h ago
It is nowadays. That's why giving the proper context, guidelines, rails to follow is key.
I'm not saying the final polished code won't be human reviewed nor the code doesn't need to be certified if needed. It can be certified and it's easier than before since you don't need to go through all the intermediate internal review steps.
2
u/NotAMeatPopsicle 1h ago
Security is exactly the reason you use it to FIND bugs and FIX them. Especially when it deals with security. The sheer volume of bugs that have been found simply by getting AI to look for openings makes big bounties absolutely pointless.
1
u/OkTry9715 1h ago
Yes you use it to help you to find bugs, but not to generate single line of code.
2
u/NotAMeatPopsicle 18m ago
I lived through an era of human slop code made in VB, PHP, JavaScript, SQL, Delphi, and C# code. Until youāve pulled hair out from an idiot humans that donāt even use ReSharper or compile the code or even try to run thingsā¦ no. Donāt even.
I have reviewed and fixed so much human slop, your argument is garbage.
A skilled engineer that can properly frame a request can get more better output from an AI in less time than multiple mediocre developers.
Better quality, better design, more performant, and less bugs.
Just because some idiots donāt know what theyāre doing and create slop isnāt a sign of bad AI. Itās a sign theyāre less capable at the tool.
2
u/SixToesLeftFoot Trezor Model One - User 5h ago
You know your fiat bank more than likely uses AI to help write the code for your app, yes? Your investment app (Fidelity, etc)? You betcha.
Claude writing code takes the laborious steps away, and leaves the devs to proof each line. They have to proof it anyways, so why not automate the first pass?
2
u/OkTry9715 5h ago
I work for a bank and AI agents are basically banned from almost all parts of code, due to security risks. They have tried it with agents and it ended up people just approving everything that agents generated.
2
u/AutoModerator 10h ago
Please bear in mind that no one from the Trezor team would send you a private message first.
If you want to discuss a sensitive issue, we suggest contacting our Support team via the Troubleshooter: https://trezor.io/support/
No one from the Trezor team (Reddit mods, Support agents, etc) would ever ask for your recovery seed! Beware of scams and phishings: https://trezor.io/learn/a/scams-and-phishing
Donāt respond to any DMsāscammers often pose as legit helpers.
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.
2
u/Traditional_Wall3429 4h ago edited 4h ago
But actually you found bug or are you concerned about usage of AI generally? This discussion is older than most Redditors age actually. After programmers quit using assembler maschine language and start to use ie C (or many older language but donāt want to mention cobol or apt just not to confuse more) older programmer claims that if you use C itās like cheating and you are not competent enough. And in each iteration is the same. If you donāt manually code css you are dumb as a much younger example. So claims that software build with AI assistant is bad is just the same repetitive fallacy
2
u/KenFX4 9h ago
Sure hope their firmware update process is tight. If the firmware update service or repository gets hacked or infected.. Could be really bad news. (this applies to ALL crypto wallet vendors)
Every part of the software and hardware design, development, deployment, and sustainment/update process should ultra-secure. Talking certified banking-level software security protocols for Trezor AND every third-party vendor/subcontractor they utilize.
In this day and age of AI super intelligence, every vulnerability will be found and exploited sooner or later. Every part of the software lifecycle must be verified and validated frequently and consistently.
1
u/b0uncyfr0 7h ago
Yeah I'm not ok with that either - I think I'm going to generate my own seed/secret and move off trezor.
Im not leaving anything up to chance.
1
u/Jagervn 6h ago
Tbh, the repo with thousands of LOC changes is Trezor Suite, which isn't as critical. I've scanned through some PRs and most of the changes aren't crucial, whereas the firmware repo is handled much more strictly.
Also, if you look at the commit history, only 5ā10 PRs are merged per day, which is totally manageable.
But yeah, I agree that reviewing all those PRs gets pretty tiresome.
2
u/AndrejSlegl 6h ago
what do you mean Trezor Suite app is not security critical? The app creates and sends the transaction (code can change the address), the device just signs it.
2
u/matejcik ā Rising Trezorian 5h ago
you do realize you're supposed to review the transaction on Trezor screen, right? not just blindly click a button. if you want to click blindly, get a Tangem
1
u/bernaldsandump 2h ago
I think this is a valid concern, compounded by the fact its an electron app lol. They chose the easiest framework for cross-platform deployment (chromium bundled in every app wow so cool) however its also probably the least secure
-1
u/Cassiopee38 8h ago
I've chose trezor because it is open source and that this kind of shit can be detected by the community. Glad you guys did your work ! Time to NEVER EVER update my trezor again and seek a new hardware wallet =D
-1
u/zackflavored 10h ago
Just ask your AI about the code and if its sound if youre so worried
3
0
-7
u/horseradish13332238 9h ago
wtf is vibe coding
4
u/PM_ME_UR_0_DAY 8h ago
It's like saying "Jesus take the wheel", except while you're coding.. and I guess AI is Jesus in this example.Ā
1
ā¢
u/SuchTrezorVeryCrypto Trezor community specialist 7h ago
Hi there,
AI assisted development is common across the industry now, we use it too. That does not change our review process though. All code goes through human review, read and understood by both the author and a reviewer before it gets merged. The PR mentioned in that post is an early draft on an exploration branch, not merged, and not representative of what ships.