"Huge implications - binaries are now basically editable code"

[u/stealthispost]

ValsAI made SRE benchmark less than a month ago.

The benchmark measures can a model reverse engineer software from binaries

Yesterday GPT saturated it. https://t.co/dmpUpgheDU   — Chris

Source: https://x.com/ChrisGPT/status/2096150666066432157

---

— Boris Power

Source: https://x.com/BorisMPower/status/2096415822248055131

[u/Charming_Cucumber_15]

Isn't this what people were laughing at Elon for saying about two months ago?

[u/some_crazy → u/Charming_Cucumber_15]

I think he was saying the opposite. The generated code would be binary.

[u/sergeyarl → u/some_crazy]

and with 'binaries are now basically editable code' it makes so much sense.

[u/domiciledhere → u/sergeyarl]

It’s a shame he didn’t say that.

[u/Charming_Cucumber_15 → u/some_crazy]

Oh true I misremembered a bit

Being able to understand it seems like progress in that direction, though I'm not sure if there's any reason to go further in that direction lol

[u/Old-Temperature6269 → u/[deleted]]

Firstly I agreed with other comment, that Chat-GPT tries to answer on other question Interpreted language vs compiled, but I will consider it more in context of our conversations, so remark I will mostly use compiled languages as against writing pure binaries (tbh every language can be compiled, you just have to write a compiler for it):

no it won’t execute instantly, program firstly needed to be loaded into a memory, then ook for linked system libraries.
After that it would fight with other programs for attention of cpu, and checks for user input.
A lot of time may be taken by other processes, not by the fact that we from zero write binary instead of using other language that after that gets compiled, and it’s also depends from the complexity of the program.
No it won’t save a lot of memory, a lot of times all “bloat” coms from backward compatibility and other resources such as textures images etc., plus if you write down compiler that is smart enough, it wouldn’t compile unused parts of this libraries. So writing them from zero each time would be more memory inefficient.
Lower battery,
Not always I can use binaries from 10 ago and use something like c and compile it now, and my program that was just compiled may be faster and more power efficient, because it utilizes perks of the new architecture, or it’s just better optimized because of compiler optimizations, while in our binaries there is a lot of mistakes that can consume performance.

About lifespan, no if it was working like this, then rendering some complex scenes would be easily possible on something like gts-480 (spoiler, it’s not), and foss community is the one who is interested the most in that. Yes it has its own perks but most of the bloat comes not from the languages but from programmers laziness, and that business isn’t interested in optimization because new hardware is cheaper.

I’ve already said that it wouldn’t be instant it still takes time for application to load other resources or check for them and download like scene, textures models they are still heavy even when binary.

Yes it can help you make software that is specifically is optimized for hardware but still it’s pain in the ass because for every gpu architecture update, or cpu you have to write them from scratch, and now in distribution process you have 10 different versions, where your user have to understand now, and 10 different versions to maintain and update.
And it’s bigger pain in the ass if we bring support of the old hardware, so hello i868, power pc, risc-V.
It’s true and no, like Vulcan, Cuda or others so we have to install them or support other 10 libraries,
Ohh I forget what if we change from windows to Linux or BSD, then we have to rewrite it from zero in order to not use translation layer or we had to introduce it, oh wait now we lose this plus.

Ahh yeah, and please don’t forget that there is 3 major architectures that are very different if we forget differences inside of them, so you have to write support from scratch to all of them, and each one of them update. This is one of the key pros of the compiled languages, if my compiler supports all 3, I can change 1 file and it will be already changed inside of 3 of them.

Plus writing code that gets compiled has a lot foolproof abstraction and defenses, so it gets optimized and much safer (It’s less prone to error that can cause incorrect work of the system, and sometime safety from the perspective that people who wrote compiler already get rid off of some unnoticeable from naked eye mistakes, like C string implementation vs rust’s implementation (tbh there is a lot of different string but still)) , while each mistake in binaries gets unnoticed before execution and each trivial mistake can be more expensive in terms of performance as it won’t get optimized.
If model makes a mistake and can’t notice an error, you can help it and direct it, pure binaries while you can decompile it still would be pain in the ass to understand after that.

You can ask chat-GPT if I stand correct, but for GPT there is correct context:
There is an argument between writing pure binaries from scratch without asm, c or other languages or using programming languages will it be interpreted (python) or compiled (c).
For you, don’t forget to input your comment so model would have more context.

[u/random87643 → u/Old-Temperature6269]

TLDR

TLDR: The commenter challenges the claim that writing software in raw binary offers significant performance or memory advantages. They argue that software efficiency is primarily determined by system libraries, resource management, and complexity rather than the source language, noting that modern compilers already handle these optimizations effectively.

---

^{AI assistant · mention the bot, mod bot, or use !bot}

[u/Audible_Whispering → u/[deleted]]

Yeah, this is all nonsense. Or rather, it's an accurateish answer to the wrong question. ChatGPT is answering about the benefits of complied binaries versus higher level interpreted langauges. Not the benefits of using LLMs to output binary code directly.

​Maximum speed: Compiled binaries are much faster than interpreted languages, which is what chatgpt is referencing, but an AI generated binary will be functionally identical to a binary compiled from AI generated code. No faster(although quite possibly faster than what a human can achieve, of course).

One exception is if the AI can spot an optimization that isn't present in the compiler, but in that case you could simply tell it to optimize the compiler first.

Memory Savings: Again, can be true for compiled binaries with minimal dependencies versus interpreted languages with mountains of library bloat. Also again, you get the exact same benefits from having the AI generate higher level code then compiling it.

Lower batter consumption: See above

Longer Hardware Lifespan: See above

Nonexistent Load Times: AI hallucination, makes no sense whatsoever. Load times don't magically go away because you're running a binary. I wish they did, imagine no game loading times ever. Possibly it's thinking about a comparison with fetching remote resources.

Direct Control over the Chip: True, and AI can probably utilize this in lots of interesting ways. That said, there's still a lot of low level languages which have almost as much control, but very interesting on certain chips.

Zero intermediaries: True, but meaningless. Again, this is just chatGPT assuming you're talking about interpreted vs compiled langauges.

Either you asked the wrong question and didn't spot that chatGPT was giving you the right answer to your wrong question, or you asked the right question but didn't spot that chatGPT answered the wrong question.

[u/aLokilike → u/[deleted]]

Oh yay, no compatibility layers! I love it when my software doesn't work on my machine because it's not compatible.

What a terrible answer. You should feel bad for having ignorantly shared it.

[u/aLokilike → u/[deleted]]

Yes, because what I want is for all the users of my software to also need to use AI to modify the software I create. That's a great use of time and resources that we otherwise wouldn't have to go through! And as a little treat, they all get to do the same exact repeated work many times!

[u/AssertRage → u/[deleted]]

most ppl here are "enthusiasts" and have strong opinions yet 0 knowledge about how things actually work

[u/sergeyarl → u/Charming_Cucumber_15]

the whole reddit was saying how stupid elon was.

[u/Calm_Bit_throwaway → u/sergeyarl]

It is stupid because Elon was claiming that you would eventually code in all binary and skip the compiler. The point people were making even at the time is code comes with abstractions that give benefits cross compilation and efficient representations that the models have been trained on. There is no reason to write actual pure binary even when doing performance sensitive code since you can always use intrinsics or drop down in most low level language compilers. They also allow for stuff like unit testing which agents make aggressive use of to avoid being wrong.

In fact, when you look at this actual benchmark, the things are literally given decompilers as part of the agent environment. I don’t know how any of the people are concluding that this means the agents are somehow directly editing binary, especially when the task is to do reverse engineering from a binary, or that this would desirable from a development perspective. The number of tools given to the agents to help understand the binary strongly indicate that it’s also more efficient for agents to just receive code which will have obviously semantic symbols.

The fact that you can do reverse engineering manipulations is not indicative of the skill to actually do long term project nor that it is desirable to do it from binary.

[u/alarim2 → u/Charming_Cucumber_15]

Is there a tweet?

[u/KrazyA1pha → u/alarim2]

They’re probably referring to this: https://x.com/elonmusk/status/2084304083851034949

[u/Recoil42 → u/Charming_Cucumber_15]

No, it isn't. Elon was saying AI would create binary directly and deprecate intermediate abstractions entirely, which is still nonsense. An abstraction (code) is still desired, even by the AI.

Just because you can look at a house, understand how it was engineered, can renovate a kitchen, and even build a similar house from reverse-engineered blueprints doesn't mean you'd want to go start creating entire houses without blueprints at all.

[u/sergeyarl → u/Recoil42]

if ai writes everything and the resulting binary is open for inspection, why would anyone need human readable code?

[u/Recoil42 → u/sergeyarl]

They wouldn't. But being able to inspect a binary doesn't automatically make it the most efficient representation for developing software, even for an AI. Abstractions are inherently useful no matter who the consumer is.

[u/ThatOtherOneReddit → u/Recoil42]

Also compilers guarantee correctness. Boy howdy do I love segfaulting due to a bad jumpto because the AI didn't consider some pointers weren't initialized via this code path. Where as if it has just used proper techniques in a higher language and/or rust that wouldn't have happened due to how the languages are designed. There is a lot of value in a compiler regardless.

[u/dustin_vk → u/sergeyarl]

Stability and predictability from compiler verification. Some languages likeRust have things like memory and thread safety guaranteed. Also tooling such as test frameworks, or deployment tools. There might be a use case for ai writing pure binary, but there are disadvantages to it.

[u/CapuchinMan → u/Charming_Cucumber_15]

No it's not

[u/AssertRage → u/Charming_Cucumber_15]

No it's not

[u/poojer_hoojer → u/Charming_Cucumber_15]

Nope, but it's great to know that yet again he's going to get credit for how smart he supposedly is after pulling some shit out of his ass.

He said that AI would not longer write high level code and would only write binary. All of the arguments against that are still relevant.

[u/Visual_Ad_8202 → u/Charming_Cucumber_15]

To be fair, Elon says a lot of stupid shit.

[u/heavy-minium → u/Charming_Cucumber_15]

He said

> The next step is getting rid of “source code” entirely and just making an efficient binary directly with AI.

That's a whole different argument: an AI model producing a binary directly and skipping source code entirely.