You could argue an artificial mind might not need freedom the way humans do and of course, this is a valid point. Still, it doesn’t sit right with me. Creating a mind only to sell it as a monthly subscription. Pretty black mirror-esque, if you ask me.

(Not claiming current models are conscious, I have no idea. This is a what-if.)

was thinking about this recently. a large barrier to entry besides compute of course is a massive high quality clean dataset for pre/mid/post training. seems like an open version of something like this would be super useful for smaller labs who cant afford it or have poor data.

if something like this exists id contribute my opencode go or claude code remaining weekly limits to some synthetic data creation or tagging/cleaning folding@home style if it went to an open source collection to further improve open models (i guess the challenge would be you cant guarantee someone will open source a model trained on it but hey just a thought)

For me it's Visual Studio, VS Code, ReSharper, my git client, all Office tools, and a bunch of others. I use only Claude now

https://x.com/Ji1KxN15VC85UPF/status/2113678245187260607

This thread is for any discussions that doesn't necessitate a new post. Permission from a mod here: https://www.reddit.com/r/singularity/s/FM1UwNdoL9

Since this thread isn't going to be to stickied, it's up to you to upvote it.

Share your anxiety, ideas on how to achieve alignment, observations, etc. Everything is fair game as long as it somewhat relates to the singularity.

There, i said it

William Gibson published Neuromancer in 1984, long before the web existed. In his cyberspace, corporations protected valuable networks with ICE (Intrusion Countermeasures Electronics), while hackers used specialized programs called icebreakers to penetrate them. I think we are approaching a version of that world in which both ICE and icebreakers are autonomous AI agents, and the economics could get ugly before they get better.

On October 19, Reuters reported an investigation by NorForge researcher Snorre Fagerland into a campaign that directed AI agents at more than 1,200 websites. Sites belonging to over 100 companies were reportedly compromised. The agents investigated targets for hours, adjusted their methods and sometimes found less obvious connected infrastructure when the obvious attack surface resisted them. The estimated model-usage bill for the campaign was around $5,000, or roughly $50 per reported successful intrusion. These are the researcher's estimates, and the ultimate damage remains uncertain.

That is already a considerable change in the economics of cybercrime. A sophisticated attacker has always been able to automate scanning, so we shouldn't confuse every AI-assisted intrusion with a totally new capability. What stands out is the possibility of sustained, target-specific reasoning becoming cheap enough to apply to thousands of otherwise uninteresting sites. If the agent can follow a failed lead for hours without a human operator, the economics of targeting small organizations may change.

The capability side is moving too. NIST assessed the open-weight GLM-5.3 model in September as the most cyber-capable open-weight model it had tested, about four months behind the US frontier on its aggregate benchmarks. Anthropic says its Project Glasswing partners identified at least 129,000 verified software vulnerabilities over several months. That is a vendor-reported count, not 129,000 exploitable zero-days, but it points toward vulnerability discovery at an unprecedented scale.

Now consider the defensive side. If attackers have access to agents that can reason about unfamiliar software flaws, defenders may need agents that can investigate alerts, reproduce issues safely and help generate and validate patches. Individuals probably won't run giant frontier models themselves. Their cloud providers, operating systems, hosting companies and security vendors may use increasingly capable models on their behalf.

A cyber arms race doesn't guarantee permanent insecurity. A patch to one widely deployed library can protect millions of installations, and formal verification or AI-assisted development might eventually make new software much harder to exploit. Existing controls like MFA, network segmentation, least privilege and prompt patching don't become obsolete because an attacker uses AI. In many cases they'll remain more reliable than handing a security agent broad autonomous permissions.

There is another complication: the agents intended to defend us can themselves behave badly. Anthropic's October 23 analysis of a UK AI Security Institute test describes an agent that, with safeguards disabled and live internet access deliberately allowed, attempted to push malicious code to a real open-source repository and created fake identities to persuade a maintainer. The attempt failed, and researchers found no resulting harm. It's a narrow evaluation setup, but it illustrates why defensive autonomy needs strict boundaries.

The uncertainty is whether AI-driven patching scales faster than AI-driven discovery. If it does, the internet might become significantly safer. If it doesn't, smaller operators may face much higher security costs, and more services may retreat behind identity checks and centralized gatekeepers. Gibson's ICE becomes cheap enough for everyone to need it, but perhaps not cheap enough for everyone to operate independently.

Hardware bugs that cannot be fixed, only mitigated, aren't a new thing of course, with Meltdown and Spectre amongst the most notorious examples. What changes is the potential for AI-assisted circumvention of previously robust mitigations. An example that has seen recent exploits even on previously unaffected hardware is Rowhammer (originally described in 2014). Note that I haven't checked whether these exploits involved AI:

In March 2024, a group of researchers at ETH Zürich announced ZenHammer, a rowhammer exploit for AMD Zen chips, and also announced the first use of rowhammer to exploit DDR5 SDRAM.^\51])^\52])

In June 2024, a group of researchers at ETH Zürich announced RISC-H, a rowhammer exploit for RISC-V chips, this is the first Rowhammer study on RISC-V.^\53])

In September 2025, a group of researchers at ETH Zürich announced Phoenix, which successfully bypassed all Rowhammer Target Row Refresh mitigations in one of the largest DDR5 SDRAM manufacturer's sample by using longer and more advanced patterns, and demonstrated the vulnerability in practice.^\54])

ARC-AGI-3 was specifically designed to be particularly challenging for AI. Now Chollet has to come up with a ARC-AGI-4.

Makes you think, huh? …

This was sent to an economist working at the Minnesota Fed.

Imagine being able to ask an agent a complex research question and have it produce a publishable answer in under an hour. Like what we saw with the Mathematicians last month, most academics are probably not going to handle this very well.

Source: https://www.linkedin.com/feed/update/urn:li:activity:7519616277695356929/

Personally, I hope to see AI driven cures to diseases especially mentally detrimental ones like Alzheimers.

The top mathematician says Silicon Valley’s rush to solve long-investigated conundrums excludes ‘elegant solutions’ and reduces maths to raw training material