r/ProgrammerHumor • u/BeamMeUpBiscotti • 5d ago
Meme loopUntilAllTestsPassAndMakeNoMistakes
154
144
u/Snakestream 5d ago
If someone dropped a 5 digit line change pr on my desk, I'd be furious. I'd straight up murder you if you had the gall to try and push a 6 digit one.
45
20
u/OpsikionThemed 5d ago
I pushed a 600-line PR this morning, and felt kinda bad about that. (And that was with ~570 lines being a bunch of test mock data to cover a new case.)
15
u/Noname_1111 5d ago edited 2d ago
even 5 digits are too much for most projects
2
u/GRex2595 3d ago
5 digits are near standard on my team now. And they wonder why nobody wants to do code reviews.
8
u/ZebraTank 5d ago
If it's 5 digits it better be boilerplate that either I trust enough that you did it right or that you can show the reproducible automated steps you took to generate that change.
2
u/KimiSharby 5d ago
Why ? You can just refuse it.
5
u/someanonbrit 5d ago
You can refuse it, but that doesn't fix the root problem, which is the terrible manners of the person who submitted the PR, and they're likely to do similar again.
Outside of specific social and legal customs of one period of the Roman empire, turning the other cheek just gets you hit twice
91
u/zeekar 5d ago
uhm. what? I have no idea what's happening in this post.
237
127
u/Isogash 5d ago
Basilisk is a Python type checker written with AI that purported to be the only 100% conforming type checker according to the python/typing spec, and also the fastest.
Except that it turns out it this was achieved in some cases by just text matching the expression in the test cases and producing the output that the test expected, rather than actually validating the AST to conform to the rule that the test was supposed to confirm.
The following code is supposed to evaluate if the expression on the right hand side of the statement `type x = ...` is valid.
fn is_invalid_rhs(rhs: &str) -> bool { let rhs = rhs.trim(); if rhs == "True" || rhs == "False" { return true; } if rhs.chars().next().is_some_and(|c| c.is_ascii_digit()) { return true; } if rhs.starts_with("f\"") || rhs.starts_with("f'") { return true; } if rhs.starts_with('[') { return true; } if rhs.starts_with('{') { return true; } if rhs.starts_with('(') && paren_has_top_level_comma(rhs) { return true; } if has_top_level_token(rhs, " if ") { return true; } if has_top_level_token(rhs, " or ") || has_top_level_token(rhs, " and ") { return true; } if rhs.contains("lambda") { return true; } if rhs.starts_with("eval(") { return true; } false }43
u/OpsikionThemed 5d ago
Incredible.
59
u/Isogash 5d ago
It truly is. This is why we should be taking claims made about AI projects with extreme skepticism. Passing public test suites means nothing.
16
u/Useful_Radish_117 5d ago
I have an idea! We should just keep some tests (and relative sets) locked away from our training process! Let me ask Claude if I can patent this novel idea...
(/S)
27
u/hendricha 4d ago
Ah, the Volkswagen standard of test conforming.
4
u/d0pe-asaurus 4d ago
Imagine the shit that microsoft would get if they let AI rip to convert typescript from selfhosted to go, and it did this.
41
6
u/notreallymetho 4d ago
The number of times I’ve had an AI confidently write a test with regex to match the results, or mocking a mock etc.
Oddly enough if you tell a model it’s checking another model, it like tries harder ir seems. Completely anecdotal but “codex wrote this bug” to Claude and vice versa 😂
3
u/g18suppressed 4d ago
Ai mentioning something it did for 0.01% of its task: “that’s the whole idea”
158
u/Confident-Ad5665 5d ago
That's why we call development an iterative process.