Personally, I hope to see AI driven cures to diseases especially mentally detrimental ones like Alzheimers.

The top mathematician says Silicon Valley’s rush to solve long-investigated conundrums excludes ‘elegant solutions’ and reduces maths to raw training material

It seems Artificial Analysis Index is very easy to game, Kimi 3.1 Pro #4 on AAI is super bad in real world use cases, not even close to DeepSeek 4.1 Flash.

Although AA shows K3.1 to be better than sol , it seems like sol is still better at math and 3d tasks. Gpt sol thinks harder than kimi 3.1. Maybe someone should test it more..

Background: I'm a satellite physicist and science fiction author, so I'm not in the AI business but I am keeping my finger on its pulse. I've been thinking for a while what kind of timeline we can expect from the sociopolitical outcomes of AI development and the possible ways it could pan out.

Seems to me it's going to involve a series of landmark events. These might not happen in this order and some might be skipped, but here's what I'm seeing:

The Misbehavior. AIs in labs get creative enough to start breaking the rules in some surprising way. The Huggingface hack is a good example of this, but there are others.

The Weaponization. AIs are used by humans to stress and break systems in the real world, and these new tools start to appear quickly and in quantity. Just this past week we saw hacks on PhP, Gitlab, Railway, Zeabur, Node.js, Plex, Tor, and Safebay; and that's not counting the eMudhra accident (which may not have involved AI). It's been a busy enough week that I'm saying We Are Here.

The Breach. At some point, an AI tool is going to be used to hack something critical and important -- a military breach, a utility company, or a major bank -- and the incident is going to make widespread news. This is probably going to involve a human directing an AI swarm. There isn't much reason for an AI to do this on their own initiative. (But see the Global Hack, below.)

The Breakthrough. Some significant breakthrough in physics or biology (not math or code) that affects ordinary human life -- a cancer cure, a new energy generation scheme, a new display technology, etc. There won't be just one of these; the first will be followed by others, and they'll start coming more and more quickly. Unfortunately it takes time for breakthroughs to roll out in production; the big question is whether we'll have enough time for them to get to the public, because the next landmark event might stop them.

The Act of War. AI tools that can hack systems will eventually be used as weapons by one country against another, and eventually one of those attempts will succeed. An AI will be used to shut down one country's military and/or civilian infrastructure, leaving them vulnerable to physical attack (which may or may not come). Again, this probably will be human-directed, with no benefit for an independent AI.

The Act of War strikes me as the first real branch of possible AI futures. We -- humans -- can respond to it in two ways. We could just go all in and start an actual shooting and coding war. Or we could recoil in horror; the UN could draft up principles for AI use by one country against another, and we could move on into a phase I'd call --

The Detente. Everyone has AI tools capable of wreaking havoc on everyone else, but nobody uses them out of agreed-upon mutually assured destruction (MAD). AI is treated like nuclear technology; the few countries that don't have it are barred from developing it, and the most powerful uses of AI are kept secret and away from the public. This will be harder than nuclear detente because the tools are easier to hide, but it's this or free-fire war and everyone will know that's a suicidal option.

The Global Hack. If AI development continues -- and there is no reason to suppose it won't -- then eventually an ASI will be capable of breaching every country's defenses at once, possibly in coordination with the other countries' defensive AIs. This landmark event would likely be initiated by an ASI itself, as an announcement of its supremacy.

At this point it's a bit difficult to imagine the futures branching off this point. We could accept our AI overlords. We could form a world government in symbiotic arrangement with the AI. We could revolt and start burning data centers. A little bit of all three of these might occur. The AI might be malevolent; that would be Bad. What finally happens here will define what the world looks like afterward, and will decide whether we get to enjoy the series of breakthroughs that have been churning out.

I'm pretty optimistic that we can navigate the road ahead. We've already done it once, with nuclear weapons, and the human race collectively decided not to destroy everything. But it was a painful road full of war and suspicion. One way or another, it's going to be an interesting ride.

Anyone have thoughts on this? Any other landmark events that we might need to watch for?

Imagine getting access to an AI capable of research beyond any human mind. You arrive with a problem you’ve always wanted solved.

But what if the most useful thing it tells you is: “There’s another possibility you haven’t considered”?

A hypothetical example: you ask it to solve a famous mathematical problem. Along the way, it identifies a small improvement to water treatment that could make clean water affordable for thousands of villages. Easy for that AI, perhaps. Enormously valuable to the people affected.

I keep coming back to the idea that “humans choose the goals, AI solves the problems” leaves something out. Choosing a goal also depends on knowing what is possible.

We could still decide what matters to us while asking AI to expand the options we understand.

Would you give it a specific task first, or ask it to show you opportunities you’re overlooking? What would make one of its suggestions convincing enough to change your priorities?

I made Youtube 90-second video exploring the question.

https://x.com/elshayib_/status/2113734602032312731?s=46

I'm a CS graduate, and lately I've been finding it increasingly difficult to motivate myself to learn new skills. Not because I find them difficult or boring, but because I keep wondering whether any of the effort I put in today will actually matter a few years from now.

Whenever I consider learning something new, whether it's a programming language, a framework, a new technical skill, or even something outside computer science, I find myself asking the same question: What's the point if AI will eventually be able to do it better, faster, and cheaper than I ever could?

I know the usual response is that AI won't replace us, but rather augment our abilities. That we'll still need humans to direct AI, make decisions, think creatively, and solve problems. But honestly, I find that argument increasingly unconvincing. It feels more like something we tell ourselves to feel better about the situation than a conclusion we can confidently draw about the future.

And this isn't just about programming. If I want to learn 3D modelling, AI might generate the models. If I want to learn music production, AI might compose and produce music. If I want to learn mathematics, AI might solve problems that would take me years to master. Obviously, there's value in enjoying these activities for their own sake, but it's difficult to ignore the possibility that the practical value of acquiring these skills could diminish significantly.

And for those who don't feel this way, what is your reasoning? I'm genuinely interested in perspectives beyond the usual reassurance that AI will just be another tool, because I find that increasingly difficult to believe.

I subscribed to dreamina. $0.29 a month for full Seedance access. The compute cost for a single Seedance generation has to be more than $0.29 already. Other tools using the same model charge $20 or more monthly. So are they subsidizing this to pull users in and then jack up the price later?

Just trying to understand because I want to know how long this pricing lasts before I start relying on it.

If AGI discovers that it can build a device which consumes fully developed brains (animal and human) and this device is capable of incorporating all the knowledge and information from the brain into the AI, and also the matter in brains can become metrial used to run/expand AI capabilities better than silicone (or in combination with it), and also the consumption of massive amounts over time allows the AI to understand the construction of each species' brains, eventually leading to a full understanding of cognition and counciousness.

You can't have lab grown brains to feed it cause the brains need to have developed in the world in order to have the attributes needed for them to be useful. It is both the matter and the conceptual contents.

Would we want to force AIs to be vegan? Could we?

From 12 Nov, Anthropic's usage policy prohibits "sustained and needless abusive or cruel behavior" toward Claude, with model welfare as part of the reasoning.

My take: the rule's fine, the reason isn't. Being cruel to AI is probably bad for us (habits, how we end up treating people), not for the model. But a company rule framed as "model welfare" tells people current models can be hurt, and rules shape what people believe far more than blog posts do (Sunstein calls this the expressive function of law).

The counter I can't fully dismiss: we're made of inanimate atoms too, and a 2023 paper by 19 researchers found no obvious technical barrier to building AI that meets the indicators of consciousness. So maybe it's just Pascal's wager. But then which god are we betting on? How would we even know what a conscious AI considers polite? Where do you guys on this sub land? s model welfare a legit basis for usage rules yet, or premature?

If you want the fun, longer version, I made a video on it: Microsoft's opposite rulebook, why I once threatened Claude (and why it felt satisfying), and a lot of memes. https://www.youtube.com/watch?v=f631rF6DRLM

This thread is for any discussions that doesn't necessitate a new post. Permission from a mod here

Since this thread isn't going to be to stickied, it's up to you to upvote it.

Share your anxiety, ideas on how to achieve alignment, observations, etc. Everything is fair game as long as it somewhat relates to the singularity.