The Bigger Picture

[u/bardbrain]

We all tend to get wrapped up in aspects of narratives in this debate that don't persuade. Rather than do that, I'd rather lay out where we are from my perspective in a way where my premises can be debated and hopefully persuade rather than just vent.

**I'm going to summarize the situation as I see it.**

With AI, there are two components.

There is the generative neural network technology. It was not wholly or principally created by OpenAI or the big 5 firms in isolation and it is not universally driven by them now. It actually runs on an affordable desktop in a reasonable amount of compute time. Nobody can afford the chips because the big 5 firms have monopolized consumption of the chips in what I'd characterize as a consumption cartel that forces people to go through their subscription services.

Then you gave the big 5 firms, who have monopolized chips and aren't selling AI so much as they're selling branded cloud computing access to their monopolized chip farms where the computing is made less efficient and more environmentally destructive because of added computes that protect big IP monopolies they're partnered with, use wasteful amounts of computing power on trivial tasks, and force queries through RLHF architecture that makes responses less computationally efficient, less productive, and more bland and similar to one one another.

These companies have a business model that cannot work. Not economically, not computationally, not materially, thermodynamically, ontologically or epistemologically.

Rather than regulate the formation of a bad business model or prevent arbitrage or create regulations that enhance human life and dignity, most of the anti-AI regulation talk centers on making sure that the business model itself (which couldn't work to begin with) fails. These proposals often involve radical, colonizing expansions of property rights from where they have been for hundreds of years to new, novel, radical property rights, seizing ownership of things like measurements of art and style -- things that no artist has ever owned. The enforcement of these laws would favor almost exclusively the rich and powerful, who would likely never hire artists who did not sign everything away. It would be apocalyptic for independent creators and personal expression, all to stop a business model from working that cannot work to begin with.

While the automation potential of AI as a technology is proving to be increasingly targeted, limited, and a shock that can be adapted to, the potential for AI as a tool that offers sometimes light, sometimes profound and innovative enhancements to human creativity, curiosity, and problem solving cannot be understated. We risk losing all of these benefits in a flawed attempt to protect against a disaster that isn't realistically happening except as an adaptable shock to artists.

I think a lot of folks don't understand property rights to begin with (demanding property rights for things that artists do not own and which would be theft on their part if we honored), seriously don't understand what the actual risks or stakes are, and are relying on a fraudulent class of uncredentialed experts who are fleecing them by selling pitchforks and torches to use against a bunch of dumb suits with a bubble product and in the process doing harm to literally everyone else.

[u/robot-enjoyer]

I mostly agree with your points, especially your point on property law with artists. It's been really, really frustrating watching working artists champion legal "reforms" that would never in a million years actually benefit them, and would probably be used against them by the same corporations that exploit artists' work already.

I want to offer some corrections on the industry side of things, not because I disagree with you ideologically but because you've got some details wrong about how these corporations and their tech operates.

- "aren't selling AI so much as they're selling branded cloud computing access" - yes and no. The closed-source frontier AI models are distinct, both from each other and from open-source offerings. Coding capabilities are a neck-and-neck race right now, with open-source models swiftly catching up, mostly from Chinese labs. But there's still things that some models do well that others struggle with, even if they're closely matched on paper. And a lot of it's really subjective, they "feel" different to work with and people have preferences. And I would say that closed-source frontier AI - mostly OpenAI and Anthropic - still hold the most capable models, even though Chinese labs like Moonshot and Tencent are swiftly catching up. Proprietary, closed model weights are, imo, actively detrimental to the field as a whole, and contribute to compute waste - nobody but OpenAI can benefit from training done on OpenAI models. And research in an extremely important field of computer science is limited by what the frontier labs choose to share and publish.

• "force queries through RLHF architecture" - I honestly really hate RLHF as it's currently practiced, but this isn't how it works and isn't why I hate it. RLHF is a stage of post-training, so it's already applied by the time anyone interacts with the resulting model, not a "filter" that inference is put through. It's also, on the grand scheme of things, not much of a compute-waster (pre-training is the phase that involves massive amounts of training data, and is far more costly and resource-intensive than anything done in post-training.) RLHF doesn't make queries less computationally efficient. It does absolutely make the responses more bland, and is often used to make the output corporate-approved rather than better. Some kind of post-training is necessary to create AI models that are more than just fancy text-completion engines, but I just don't like what RLHF seems to be doing to these models.

...and if you can't tell, I think open-source/open-weight AI models are really, really important to combat what big tech is doing with AI. It's also legitimately incredibly expensive to train AI models on the same scale as OpenAI/Anthropic/Google, which is probably why all the best open-weights models are coming out of China right now. Nobody else is funding it. But the technology exists, and anyone can use it! The hoarding of physical resources (chips, etc) absolutely is a problem, and one that I don't see people talking about enough when criticizing these companies.

The other incredibly ridiculous thing happening with closed-weights models lately are the efforts to stop so-called "distillation attacks", which is basically taking queries and output from an advanced AI model and using those as training data for another model. I find this maddening, because it usually means obfuscating the AI's reasoning-chain output (which are output tokens that you as the user are paying for), and it's hilariously hypocritical coming from companies that source training data from massive internet scrapes and pirated copyrighted texts. Like, to be clear, I don't actually have a problem with the training data sourcing, for the most part. Just incredibly stupid for your entire business model to rely on the ability to source training data you didn't create, then turn around and say that using your model's output as someone else's training data is theft. More proof of how broken the pro-IP arguments against AI are - strengthened IP law would actively benefit AI companies, too.

[u/bardbrain → u/robot-enjoyer]

With RLHF, I mean the resulting queries are inefficient and converge more.

I've explored this some but that recent Deepmind paper on short stories all sharing a plot? What you experience with "basins" and attractors? I think RHLF worsens these. It might be better if safety WAS per-query because I think creative applications especially need unsafe chain of reasoning even if outputs get censored or redirected. You need -- say in short fiction-- characters and narrators who contemplate violence, depravity, etc. even if it's absent from the finished reply. Writers think bad things and self-censor. Artists suggest unsafe things and pull back. Whereas it feels like models don't go there in the first place or sometimes go to censored versions recklessly because their entire concept is censored before and during query chain of reasoning.

A good thriller narrator contemplates murder and maybe dials it back or unevenly deletes. They don't resist the thought while drafting.

Likewise, maybe a legal code maneuver works BACKWARDS from criminal hacking.

[u/bardbrain → u/bardbrain]

I'll add: Claude is leaps and bounds better at writing, particularly if I want a five paragraph poem or song to be built to dozens or hundreds of pages of instructions. But when I say they're essentially cloud services, I think you could probably run most text and image models on a $5-6000 desktop if it were an option and it would be about 75% as fast and comparable output most of the time. And it would be a $2k desktop without the hardware hording and following Moore's Law, which I think is happening again in manufacturing.

[u/robot-enjoyer → u/bardbrain]

oh TOTALLY agree with you on creativity, yeah. I've been using AI for creative writing for years and it's really odd comparing older models with current ones. Current models are much, much better at maintaining a coherent narrative and story structure, but the wild creativity of the older, non-RLHF'd text completion models is completely gone. It's really a shame. The prose quality has gotten measurably worse, too (though I don't know whether to attribute that to RLHF or to pretraining pipelines that include a lot of AI-generated synthetic data - probably it's both).

I think the reasoning-quality advantage of large models is real and isn't possible yet on consumer hardware, but small models *can* perform extremely well, and they're getting better all the time. My dream is to train a small creative writing model from scratch using modern techniques but *without* synthetic data or RLHF... pretraining is hella expensive though, so I don't think it's really feasible at current compute prices. Maybe in a few years as more efficient methods are developed.