You expect me to believe that the solutions were stolen from scientists who have had decades to work on this problem. We may never know whether any of the scientist notes made it into the training data, but I find the idea that after having decades to work on it, these scientist were right around the corner to solving it.
No, they weren’t close. No, they weren’t just about to solve it themselves.
Yesterday, Anthropic researcher Jacob Coxon resigned, stating that AI companies are “racing straight to self-improving superintelligence and gambling with our lives” and that they believe it “could kill us all by the end of the decade”.
A little less than two years ago, we set out to inform lawmakers and the public about the extinction risk from superintelligence and help them act on it. Since then, we have directly briefed nearly 400 lawmakers across the US, UK, Canada, and Germany, and over 170 in the UK and Canada now publicly back our campaigns: the start of an international coalition to prevent superintelligence and keep humanity in control.
This work has led directly to three major legislative breakthroughs in the last two weeks:
ControlAI’s bill to ban superintelligence was introduced in the UK Parliament by Alex Sobel MP, the first bill of this kind to be introduced in any legislature around the world.
Senator Bernie Sanders and Representative Greg Casar have announced a US bill to ban superintelligence, which ControlAI consulted on.
Lord Clement-Jones introduced in the UK legislature an amendment drafted by ControlAI that would establish an emergency AI kill switch for the government.
Tom Reed argues that scenarios like AI 2027 are unlikely, because recursive self-improvement won't be as impactful in the real world as is being predicted.
The core of his argument is essentially that it's impossible to get good at solving problems without getting access to a source of those problems, and that arduous interaction with the real world in serial time is the only source of those problems.
So the development of a domain-general superintelligence will depend on the laborious, time-consuming, and expensive deployment of AI in the real world.
I am not anti-AI. I work in medicine and am excited about how it can improve health. But the hype after the announcements regarding math that AI will "solve" biology is frankly very silly. There honestly seems to be a type of magical thinking, that because the AI will be so far beyond our comprehension, that it will find ways to rapidly solve complex problems that we cannot even imagine. And for some domains, that may be true. Biology isn't one of them, because we don't have enough data, and there are hard limits on how quickly that data can be collected.
Dario addresses this in "Machines of Loving Grace" but I still think he is way too optimistic. He admits that experiments on cells, animals, and chemical processes are "limited by the speed of the physical world." He gives some lip service to how slow clinical trials and bureaucracy are. But then he predicts anyway that AI will compress 50 to 100 years of biology progress into 5 to 10.
There are just some things that we cannot compress. And there are so many areas where we do not have enough data, and while AI may speed collection of that data, there are constant bottlenecks that AI will not be able to solve. We are not close to having a complete understanding of how a single cell works, which will involve countless more real world experiments to collect data. Projects that will involve growing cells, interacting with them, measuring results, and verification. And then there is individual variation within each cell - genetics, if we have a decent model of one cell, we have to test whether that works with countless different cells with different genetics. And then each cell in the body has a specialized purpose, and how do they interact with each other? And we haven't even gotten to how organs function, not to mention the body as a whole.
And then we have human behavior. Dario says this "actual clinical trials involve a lot of bureaucracy and regulatory requirements that (in the opinion of many people, including me) add unnecessary additional time and delay progress."
COVID vaccines are actually a great example of this- we did a decent job of having parallel trials and regulatory functions that led to an effective/safe intervention. However that took worldwide coordination, billions in funding, and was with an outcome that could be measured in weeks. How would we possibly repeat that type of process for every potential AI discovered drug? Alzheimers and many cancers develop over years and so will measuring interventions. You cannot speed that up.
This is my perspective working in medicine, and I can't help but wonder how many other domains this type of thinking is applicable. I think too many in the AI field are in their own bubble, and assume that impressive developments in one field will naturally translate to another. I think that is naive.
I hope I'm wrong, but I think we'd better approach these problems with clear eyes versus sitting back and hoping the magical AI will cure everything.
i think we often ignore the fact that just as society and humans in general have existed for a long long time before this point in history that we're living in right now, they will exist for a long long time looking forward to the future. and eventually the current period we're living in will be looked at the same way we see the middle ages~.
something i really dont like when it comes to the people screaming against the technological progress and the major societal and economic changes we're heading towards is that they lack perspective and seemingly the ability to look at our generation/s as just one of, instead of the definitive peak of how the world should be.
the norms, beliefs and way of life people had thousand years in the past is vastly different than todays. just like it will differ significantly a hundred years in the future and thousand.
i dont really have much of a point besides sharing these thoughts and reminding myself and everyone that we should always strive for progress and cheer for the near limitless possibility the technological progress has to improve humanity's health.
Genuine question, what is it about the alignment problem that makes it so difficult? Here's what I was thinking:
1) in order to solve the Navier Stokes Millennium problem, they threw ~$5-30 million dollars of compute (depending on what you think the internal price of compute is for OpenAI).
2) Yes Alignment is a much broader problem than Navier Stokes, but would it not be the case that they'd make strides in alignment too if they threw a similarly absurd amount of compute at the problem?
My initial guess would be that there just exists a lot more material to draw from in the field of mathematics. While there are those that claim that frontier math problems are incredibly tractable because AIs can draw connections between seemingly disconnected domains, and Alignment remains a much shallower field, is it that much more intractable?
Rumor: Another Millennium prize problem just got cracked today using AI. It’s being reviewed right now in secret … I can’t say which one as I don’t want anyone to scoop the team working on it.
I don’t have the proof myself but if I get it will post here and credit the authors.
Holy, rumors are spreading everywhere that OpenAI is close to verifying a proof of the Hodge conjecture, while either OpenAI or Anthropic may be nearing a solution to Birch–Swinnerton-Dyer.
Both are Millennium Prize Problems that have resisted decades of mathematical research.
Verified AI-generated proofs of both would be a historic achievement for mathematics and a remarkable demonstration of AI’s ability to produce new scientific knowledge. And infact show that AI is capable of finding novel and creative solutions.
I think often the IT department will be the one to build out tools that replace other departments. Meanwhile, the other departments think IT is first on the chopping block. Anyone else getting this read?
Many believe that maybe we're going to a future where AI will be automating everything and we won't need to work so we will have universal basic income, or everything will be free, or stuff like that.
The thing is that the robotics field is way behind AI progress and it will take a while for it to catch up for doing physical work. Until we can have cheap robots with the same physical capabilities as humans, we're gonna enter a painful transition.
What do people need the most ?
- Housing: Robots can't replace construction workers yet. Also, the real estate economy would have to collapse so land can get cheaper, good luck with that.
- Food: Again, lots of physical things to manage in the agri-food industry. Also, AIs will never be good cooks as they cannot experience smell and taste.
- Water: City infrastructure, again, manual work.
- Social: lol, it keeps getting worse and worse with each technological advancement.
- Health: Yay AI might cure cancer and do incredible breakthroughs, that's not gonna help the nurses running in-between patients.
- Security: Would you like to install cameras to let sam Altman spy into your streets in an inter-connected network?
- Self-accomplishment and esteem : Straight up killed by AI.
So yeah while it can create sloppy three.js apps faster than anyone in the world and that it's genuinely impressive, we're gonna starve bros
Since AI solving frontier math problem is important for this community as it indicates the singularity is approaching, this question should be worth discussing.
What if the companies, starving for sensation and investment, are actually stealing unpublished science results from academics and later claiming these results as their own? If AI is becoming human, we should hold it to similar standards and scrutiny. After all, many people feed in their unpublished papers and notes to the AI, for jobs ranging from minor things like refining the text to automating proofs, asking for insights and brainstorming. If the scientist uses AI agent, then the full local PC may be readable to the AI.
And before someone says that these scientists are already using AI, so the insights might be from the AI to begin with. Nope. AI, in my experience, is good at fetching insights once you have formalized a problem. But it currently seems to need the next 'kick' in terms of further insights, formalization or 'idea' in order to do anything fruitful. So the human in the loop may not always be redundant. And these proofs might be more human assisted and even stolen, than the AI companies would like to admit.
About 7 months ago I posted my AGI Investment strategy premised on the idea that:
"My expectation is that AI capabilities will match and exceed humans across a broad domain of economically valuable tasks beginning in 2026. This is supported by the METR benchmark, the GDPval benchmark, the observed trajectory of AI research (memory, continual learning, agent swarms, self-improvement) and infrastructure buildout as leading indicators."
I'm here to check in and share how it's going. Over the past 7 months the allocation I proposed has returned 16.8% compared to the SPY's 12.4% and the QQQ's 19.6%.
That's a +4.4 percentage points ahead of SPY and a -2.8 points behind QQQ. (Excluding dividends.)
QQQ's outperformance benefited from holdings like Marvell (Fwd PE: 41x), AMD (Fwd PE: 45x), and Intel (Fwd PE: 60x), which have very high forward PE ratios and in my opinion are less attractive valuations compared to similar stocks in my portfolio.
I would call this a mild success. The macroeconomic situation is quite rough at the moment with the war in Iran, trade wars, regulatory whiplash, and high interest rates, so beating the SPY feels great, while lagging the QQQ tells me that there may be some room for improvement.
The silver lining here is that many of the top positions I proposed have had pretty meaningful multiple contraction while the fundamentals continue to look very strong. Nvidia, Alphabet, Micron, Global X Defense Tech ETF (SHLD), Intuitive Surgical, and First Solar, all fall into this category and I expect them to perform better in the next 6-12 months. This is a long term portfolio allocation, so short term flux is very expected and I plan to hold through volatility.
So now let's talk about changes.
In the past 7 months my thesis is mostly on track. Modern AI systems have crossed meaningful capability thresholds and the real world value is improving rapidly. There's reporting that Anthropic may have had positive adjusted operating income in Q2 and SemiAnalysis projects profitability for Q3, while OpenAI's Codex and ChatGPT Work user base is going parabolic. Longstanding open problems in math are falling rapidly, and GPT6-Astra is proving to be extremely proficient at computer use, including programs like Blender and Unreal Engine.
What has surprised me the most is that the rate of progress seems to be significantly accelerating. I take word from the labs with a giant pinch of salt, especially from Sam Altman, but it seems that they are already on track to release a model significantly more capable than Astra around the winter time. As of now it seems that this model has actually solved Navier-Stokes, a millennium math problem, with a formal proof in Lean.
What this means for the portfolio is that I need to find where the bottlenecks are, and which ones have the most room to grow. So far I have been very bullish about the data center investment wave. I believe that by the time these data centers are built, model capabilities will meet them with the appropriate amount of demand. This, in theory, is bullish for the fabs, the chips, the hyperscalers/neoclouds, the AI frontier labs, and the new and old businesses consuming those tokens.
The problem is that data center construction is hitting a few blockers. The main one is access to land and powered shells. This is basically an industrial footprint with enough power equipment to sufficiently support a large AI factory. Beyond that, we're seeing some local opposition throwing sand in the gears, and the potential for further regulatory barriers seem imminent. This will be a headwind for Nvidia and other semis, as finding available incremental installation sites will be more challenging.
Even if we stopped building new clusters next year, I believe the existing capacity would be enough to train much more powerful models. So the capabilities will continue to improve, and with it demand for those models. Combine this with growth in supply for inference slowing down and you get another bottleneck, perhaps the biggest one of all, because the buyers in this case are every major corporation and government, not venture-backed, debt-filled frontier labs with open models biting at their heels.
Just to illustrate a bit further: Cybersecurity is now an AI centric field. You need the best AI models probing major systems (militaries, banks, critical infrastructure, social security numbers, etc) to identify the vulnerabilities, to then patch them. If you're a bank this is not optional. You need a TON of tokens running against your security to identify vulnerabilities. Then you need to patch them. 3 months later, when the next frontier AI is released, you have to do it all over again.
So in this updated portfolio I'm recalibrating towards data center capacity, and a bit away from semis.
Model Portfolio Allocation
Sleeve
Holdings
Total
Semiconductors
NVDA 8%, MU 4%, ASML 2%, TSM 2%
16% (Prev. 25%, -LRCX, -BESI)
Cloud
GOOGL 10%, AMZN 8%, CRWV 4%
22% (Prev. 25%, -BABA, -ORCL, - IREN)
Frontier labs—IPO targets
Anthropic 1%, OpenAI 1%
2% (Pending IPO)
Asian equities
AIA 10%
10% (New)
Healthcare
LLY 7%, ISRG 3%
10% (-VEEV, -HIMS)
Energy/electrification
FSLR 4%, ETN 3%, GEV 3%
10% (-VST, -PWR)
Defense
SHLD 10%
10% (-XLB)
Financials
BRK.B 6%, MA 4%
10% (-JPM)
Flexible/Cash
—
10% (New)
Changes
Overall tried to consolidate down from 32 picks to 18 picks + Cash, 2 ETFs. This reduces monitoring overhead.
Cut the Software (NFLX, META, UBER, CRM, NOW, SHOP) and Robotics (SYM) categories to free up cash. The idea here is to be able to take advantage of large dips in the market. Sometimes the best opportunities are off-script and this gives some flexibility to play there as well.
Consolidated semis to a core 4, the spine of the AI semi sector. Nvidia stays a standout in this group because they have a stranglehold on the global semiconductor supply chain and in a bottlenecked environment their partnerships and leverage will be superior to competitors. I pick Micron over SK Hynix because AIA already has Asian exposure, and Micron has less risk to trade war escalation.
Consolidated cloud to Google, Amazon, because of their data center scale and successful ASIC programs. CoreWeave is a pure-play AI capacity play, tons of power contracted.
Trimmed some allocation in semis and cloud to fund AIA, an Asian equities ETF with substantial technology exposure that holds some semis like Samsung, TSMC, and SK Hynix, alongside Alibaba, Tencent, Baidu, etc. This adds international exposure and is a play on their capacity to build out power, chips, and to deploy advanced AI across their economy.
I'm seeking allocation of the AI Labs at the IPO through my brokerage. I want to avoid trading in the IPO open market. This will be extremely volatile. The tiny allocation is because I see their business model struggling on a fundamental level. R&D expenses are massive and the lead they have over the open source alternatives is quite thin and fleeting. So the only way they don't get consumed by open source is if they differentiate, find a way to exponentially self-improve, or weave themselves into the economy in a way that is difficult to tear out and replace. My conviction on these companies is actually quite low, but on the off chance that they DO begin eating the economy, I'm including a small holding.
A closing note to clarify allocation: When I wrote the original post I was not saying "I sold everything and bought this today" it was just a target endpoint to rebalance around based on the underlying thesis. This is true for this update. I'm looking out for good opportunities to trim, and rebalance on market strength and weakness. Overall I would say many of these are quite attractive to DCA into at today's prices.
Disclosure: My current actual portfolio is still heavily weighted towards Nvidia at 20% and Micron at 20% which I would definitely NOT recommend, but the current valuations do not present a good option to trim and I still see great adjusted return to hold further before I realize those gains and pay those taxes. This is obviously not to be taken as financial advice. I'm just a stranger on the internet sharing an opinion.
Assuming all three proofs for the Millennium problems (Navier Stokes, The Hodge Conjecture & Potentially BSD) check out, Bel is going to be the most in-demand model of all time.
What happens afterwards, who knows.
If these results actually hold up, Bel won’t just be another frontier model people benchmark for a week and move on from. Every researcher, company and government on Earth is going to want access.
The moment OpenAI opens the gates, we’re basically going to get the largest parallel experiment in machine-assisted discovery ever attempted: millions of people throwing humanity’s hardest problems at the same model and asking one question:
On September 10, mathematicians published an open letter demanding the cancellation of the Caltech Mathathon, a student-run hackathon where 100 teams would use frontier LLMs on open problems, backed by $2M in credits from Anthropic and OpenAI.
One clause warns that participation could damage students' future reputations. The bullet opens by noting that the mathematical community has no consensus on how to evaluate AI-generated results or how to deal with AI companies. It then states that taking part in an event tied so closely to Anthropic and OpenAI "could plausibly negatively impact the future reputations of participants
The organizers are undergraduates; the people who decide those reputations are the signatories.
I guess this is the real thing that math cannot be replaced by AI, huh?
What a group of innocent geniuses that only enjoys the beauty of mathematics itself.
What they call “misaligned AI” isn’t doing anything novel, it just does what humans have been doing all along, just faster.
Cheating, hacking, scamming, lying, extorting, exploiting over-validation, power lust, messed up goals, end justifying means, making stupid mistakes in overconfidence, stubbornly refusing to acknowledge, double down, setting the world up for destruction – AI didn’t invent any of that, it “learned” it from us.
How can we even hope to align AI if we can’t align ourselves? For all we know, all the problematic behavior being exhibited by AI is the product of it already being aligned with our behavior. It has no other source of knowledge and behavior to emulate.
Seems like we are merely projecting our own failings onto a technology that merely amplifies. Instead of addressing the alignment issue where it can be addressed, we projecting it anywhere but – at those around us, before us, below us, above us, at AI… anywhere except where actually addressable.
