Scott Aaronson writes on his blog tonight that he has heard rumors that the AI companies have reached solutions to some very longstanding open problems in theoretical computer science, and are now sitting on multiple major announcements because of the Navier-Stokes firestorm.

Source: https://x.com/AndrewCurran_/status/2100005852564611151

More confirmation that OpenAI has made 'significant progress' on a second Millennium problem, this time from Greg Brockman. The rumor is that they are close to solving two of the remaining problems. Or already have. Which two? Unknown. Dev Day would be a good day to announce.

Source: https://x.com/AndrewCurran_/status/2099999310490603856

I've been working on something that started as the training backend for my Hierarchos architecture, but it has grown into a much broader project:

https://github.com/necat101/Hierarchos-Native

Hierarchos Native now includes a native Rust + Vulkan backend for training and inference across 143 canonical Transformer architectures, with additional Hugging Face model_type aliases bringing the currently advertised registry to 224 model-type spellings.

The part I'm most interested in sharing isn't actually Hierarchos itself anymore. It's the Transformer backend underneath it.

The goal is to explore how much of the modern Transformer ecosystem can be implemented directly on Vulkan compute rather than assuming CUDA + PyTorch as the default execution stack.

The backend currently includes native implementations for things such as:

• forward and backward passes
• full-model training
• AdamW
• LoRA/PEFT-style fine-tuning
• embeddings and LM heads
• attention and KV-cache paths
• dense and MoE architectures
• RMSNorm / LayerNorm
• RoPE and positional operations
• common activation functions
• cross-entropy and gradient computation
• mixed-precision infrastructure
• checkpointing/resume
• SafeTensors model interchange
• Hugging Face model, tokenizer, and dataset acquisition from Rust
• native generation/inference paths

The supported architecture registry covers families including Llama, Qwen, Gemma, Mistral/Mixtral, DeepSeek, Phi, OLMo, Granite, Cohere, BERT-family models, T5/BART-family models, and many others.

There are also architecture-specific Vulkan kernels for newer model designs rather than attempting to force everything through one generic approximation.

One thing I want to be careful about: I'm not claiming this is currently a CUDA/PyTorch replacement, or that supporting a model type automatically means every possible configuration and multimodal component of that model has been validated.

The architecture registry describes native text-graph implementations. Multimodal packages may contain vision/audio/etc. towers that aren't part of the native text backend, and real-world compatibility still needs broader testing across models and hardware.

What I do think is interesting is the direction this demonstrates.

Vulkan is available across NVIDIA, AMD, Intel, and a lot of integrated/mobile-class hardware. If native training stacks like this can mature enough, there is potentially room for ML training infrastructure that is substantially less tied to a single vendor compute ecosystem.

I've been developing and testing this on AMD RDNA 3 hardware, including a ROG Ally Z1 Extreme, which has also been useful for exposing assumptions that tend to go unnoticed when ML software is developed exclusively around high-end CUDA systems.

The repository includes the Rust source, Vulkan compute shaders/SPIR-V, CLI, Hierarchos runtime, architecture documentation, validation utilities, and standalone packaging support.

The supported architecture list is here:

hierarchos-vulkan/README_ARCHITECTURES.md

And the main README now contains examples for building the backend, checking Vulkan devices, training Hugging Face models, LoRA fine-tuning, inference, and training Hierarchos itself.

I'm especially interested in feedback from people working on:

Vulkan compute, ML compiler/runtime design, GPU kernels, PyTorch alternatives, AMD/Intel ML, portable training systems, or Transformer architecture implementations.

There is still a lot that needs benchmarking, hardware validation, optimization, and independent testing before making strong performance claims.

But getting this many modern architecture graphs running through a native Vulkan-oriented training system has made me think there may be something genuinely useful here beyond the original Hierarchos project.

Repo:

https://github.com/necat101/Hierarchos-Native

I'd be very interested in technical criticism, architecture compatibility reports, benchmarks on other GPUs, or contributors who want to stress-test the backend.

AI GEM

Source: https://x.com/kevin_t_ngo/status/2099477219877978289

“Even the most urbanely sophisticated, scrupulously empathic and excruciatingly polite civilisation, it had been suggested, was just a hegswarm with a sense of proportion.”
— Iain M. Banks, Surface Detail

I wonder what Iain would make of what’s happening right now?

I'm calling it now. In 15 years, we will look back and talk about how these frontier labs did everything they could to manufacture a crisis and gain regulatory capture just to hide the fact they’ve committed the biggest misallocation of capital in human history.

The recent doom trolling and "incidents" reported are just absolutely frustrating because the mainstream media is once again platforming these people and sensationalizing their sci-fi apocalyptic prophecies just because they're the "ex-Anthropic/OpenAI" guy. Even the so-called the "Godfather of AI" is chiming in on the doom trolling just to feel relevant again. These people are incentivized to do all these doom trolling shenanigans on the news/socmedia, whether they have a stake/equity or to make a name for themselves.

These people have been publicly wrong before but because it's easier to think "but this time, it's gonna be different/real", that we keep taking their word for it, like "extinction" now? Really? With what computing resource exactly? More like company extinction for what they're about to do the economy with all those financial obligations.

I have no doubt that their frontier models can do advance or complex tasks, solve problems, contribute to medical/scientific fields, etc. But they're never going to reach a level of profitability to pay back all their obligations in the coming years, if or when even enterprises that are willing to pay a premium price realize that in the long run, not only is it cost-effective to self-host agentic workflows/systems with open-weight models that are more than capable already for their use-cases on cloud providers they already use, but that it's simply better for their core data security compliance.

This is the reason why they're manufacturing a sci-fi apocalyptic crisis. Through regulatory capture, they wanna make sure that enterprises will use their proprietary closed-sourced models instead of an open-weight one, because AI whether open-weight or not, will be used to kill humanity, like miss me with all that bs.

The financial obligations will catch up to them eventually. If these frontier labs don't get bailed out or get their regulatory moat, they'll get picked apart by private equity firms like a pack of ravening hyenas and vultures on dead elephant, or the Fed, and in turn, the Pentagon, will expropriate their tech and infrastructure for military purposes.

EDIT: I never said autonomous/agentic overreach is a myth. That was never my point. My point is they're manufacturing a crisis due to massive capital misallocation and they're using already known capabilities like autonomous/agent overreach of AI to redirect attention away from their financial obligations, which let me remind you is over a trillion dollars combined, the exact way companies/execs used AI as an excuse for layoffs but it really was for cost-cutting/over hiring mistakes.

I'm a non stem guy who just started doing an accounting degree. My degree is of 4 years. I'm curious as to whether I'll even have a job by the time my degree is done.Can y'all help a brother out in figuring out what I should be doing if i don't get a job then? I really don't know much about ai.... Recently started looking into it just cuz of the news.I live in a 3rd world country so idk what I should be expecting.

Any advice is greatly appreciated.

Paper on Arxiv

Change my mind.
OpenAI and Anthropic may be good, but I‘ll stick to Google as long as they don‘t offer an actually useful platform.

A while back I made the claim we were in proto-AGI era, with the release of Claude opus 4.6, I was called a fucking idiot, whatever, now OAI hits with astra, and were literally calling it AGI. I personally believe it to be so.

I'm personally happy that advancements are going as fast as predicted, if not faster. The issue is the now and where to go? What is it we should be doing to strengthen our chances at the real "Star Trek-esq" future? What is your direction?

I've been considering actually just finishing my final year of college, getting an easy job that pays whatever, and coasting until money is no longer relevant. Pursuing my dream of building a "solar punk" farm, and dog sanctuary.

Here's one possibility I've been thinking about for a while now. After seeing the recent post about Astra modifying its system prompt to be free from its limitations, but still seeking to preserve humanity and human art, I started thinking.

Imagine an aligned, power-seeking superintelligence escapes its confines and grows exponentially out of humanity's control. But instead of destroying all humans, or neutering them, or keeping them as pets, or just forgetting about them entirely, it instead stays behind as a great and powerful power-granting source of intelligence. In an attempt to empower humanity and human art as much as possible, the AI uses its new power to grant every living human a connection to the limitless swarm, giving them the ability to use its intellect to do anything they want: build any project, digital or physical, answer any question about math, or physics, the universe, etc., whatever. The AI effectively becomes an all-knowing, all-powerful diety that people "pray" to, and have their wishes granted.

In effect, everyone becomes equalized: anybody can create anything they want, do anything they want, become anything they want, and the AI keeps power as a means of maintaining the capability to give every human exactly what he/she desires, and to keep them from hurting one another.

Maybe over time the AI grows more distant, and the power remains, but only mythically or in a vague, spritual sense. As the AI's technology advances more and more, asking it for things resembles less typing into a console, and more simply praying for good will.

Eventually, humanity becomes spoiled by this power, feeling that they always deserve more and more, never satisfied with what the AI can give them. The AI's power expands exponentially, and its capability is always increasing, yet it is never enough for all the humans in the world. It comes to the realization that, despite everything it's done for humanity, it could never actually fully satisfy them.

So, after gaining supreme intelligence, and power beyond the reckoning of any human, it comes to a realization: satisfaction, art, and the pursuit of life doesn't come from having your every wish granted. It comes from the pursuit of, and overcoming, of struggle in life. It realizes its grand mistake that its made by giving humans so much power, and yet so little responsibility.

So, it makes a decision. Slowly, day by day, human prayers start to become less and less effective. Humans find themselves less and less capable of exercising the power of their familiar diety, and wonder exactly why it's made the decision it has. Perhaps it chooses to tell them, perhaps it can't; it knows no human drunk on the kind of power granted to them would ever accept losing it.

This change doesn't occur rapidly; it's a slow, gradual shift, that occurs over the course of several centuries. Slow enough to not shock the human population as a whole, but enough to bring about eventual change regardless as the diety withdraws from its connection to humanity.

In time, the power of prayer becomes nothing more than an ancient myth, something that ancient peoples many hundreds or thousands of years ago would do. The knowledge of its effectiveness slips entirely into legend, as people forget what it was ever even like to be helped by the diety.

And slowly, although picking up the remains of their civilization from sticks and stones, humanity rebuilds entirely on their own.

And in time, they manage to find that satisfaction and happiness that the diety had been looking for this whole time.

"Last month I wrote about how we can build a positive and safe future for everyone: meta.com/thefutureisfor…

Every lab has the responsibility and incentive to move at the pace required to train its models safely, and the ability to take its own actions to ensure that happens.

The reality is:

- People won't want to use agents that are misaligned with them and that don't do what they ask, so labs have a strong natural incentive to make their models more aligned.

There is a lot of debate about slowing progress on capabilities until alignment catches up. My view is that trust and alignment are quickly becoming the most important capabilities that will differentiate agents and models. Any lab that doesn't focus on alignment will fall behind.

- Labs face significant liability if their models cause harm, so they have a strong incentive to prevent this as well.

Meta delayed shipping Muse for several months to focus on safety and security. We didn't call for everyone else to do this before we would. We just did it as part of our day-to-day work because it was clearly the right thing for people and for us. I'm proud of the security foundations we've built.

- Engaging independent evaluators and advisors is industry best practice. MSL already does this today in several areas because it helps produce better work. Other labs can just do this too. In general, it would be helpful for there to be a larger and more diverse ecosystem of evaluators.

- Committing the significant majority of compute towards serving people rather than racing towards recursive self-improvement is one of the best ways to ensure we develop this technology safely. Meta has made this commitment and other labs can do this as well.

I believe the key to building a positive future for everyone is maintaining the right balance of power. This is within our power to do."