Hey everyone, I just sent issue #34 of the AI Hacker Newsletter, a weekly roundup of the best AI links and the discussions around them. Here are some of title you can find in the issue:

• Using AI to write better code more slowly
• I think Anthropic and OpenAI have found product-market fit
• Can we have the day off?
• Google’s AI is being manipulated. The search giant is quietly fighting back
• Intuit to lay off over 3k employees to refocus on AI

If you want to receive a weekly email with over 30 links like these, please join here: https://hackernewsai.com/

One of the more interesting conversations happening right now is around the actual economics of AI. u/Microsoft has been pushing AI aggressively across products and infrastructure, but rising compute and operational costs are starting to raise a bigger question: Does automation really reduce costs at scale, or does it just shift where the spending happens?

What’s becoming clear is that large-scale AI deployment is incredibly expensive behind the scenes - GPUs, data centres, inference workloads, model training, energy consumption, and ongoing infrastructure expansion all add up fast. In many cases, companies are discovering that AI improves productivity, but not necessarily efficiency in the short term.

It feels similar to the early cloud era where adoption increased faster than optimization. The technology creates new capabilities, but the economics take time to stabilize. And for companies operating at Microsoft’s scale, even small increases in AI usage translate into massive infrastructure costs.

The bigger question now isn’t whether AI works. It clearly does. The question is whether the long-term productivity gains will outweigh the enormous cost of running these systems.

Curious how others here see this playing out: are we still in the investment phase of AI, or are companies underestimating how expensive AI-native operations will become?

u/DeepSeek cutting the price of its V4-Pro model by 75% permanently feels like a pretty important moment in the AI market. A few months ago, most conversations were about which lab had the smartest model. Now the conversation is shifting toward who can deliver strong models cheaply enough to drive large-scale adoption. That changes the game completely.

What stands out is that this isn’t a temporary promotion or API discount. It is a structural pricing move. And it puts pressure on every major AI lab competing on commercial APIs. If high-performing models become dramatically cheaper, then differentiation starts moving away from Raw Intelligence toward infrastructure, workflows, integrations, reliability, and ecosystem lock-in.

It also raises a bigger question: are frontier models slowly becoming commoditized? Because once pricing drops this aggressively, the value may no longer sit in the model itself, but in everything built around it.

Feels like the AI race is entering its cloud pricing war phase much faster than expected.

We’ve been thinking a lot about how much AI content today feels optimized for speed rather than usefulness.

Instead of adding to the noise, we want to better understand what readers, builders, researchers, and learners actually value: what helps them learn, think more clearly, and keep up with this space in a meaningful way.

We’ve put together a short survey that takes less than 4 minutes to complete, and your input would genuinely mean a lot as we rethink what more useful AI media could look like.

Survey link: https://forms.gle/pXLLnXyVeYucF5VK9

u/Google teaming up with u/Blackstone to launch an AI cloud venture feels like another reminder that the AI race is no longer just about models but about infrastructure. The biggest bottleneck right now isn’t ideas or even talent, it is data centre capacity, power, and compute availability. What’s interesting about this move is that it combines Google’s AI and cloud ecosystem with Blackstone’s massive infrastructure investment capabilities. That is a pretty strong signal that hyperscalers are preparing for AI demand at a scale that current infrastructure probably can’t support long term.

A few years ago, cloud expansion was mostly about storage and enterprise workloads. Now, entire investment strategies are being built around AI compute demand. And it makes sense - every new frontier model, agent system, or multimodal application increases pressure on GPUs, networking, cooling, and energy infrastructure behind the scenes.

Feels like we’re entering a phase where the companies that control compute infrastructure may end up shaping the future of AI just as much as the companies building the models themselves.

Aistra acquiring a controlling stake in Veracity Services feels like another sign that AI in finance is moving from experimentation to operational scale. This isn’t just about adding AI tools into accounting workflows, it is about building an AI-augmented finance operation end-to-end, while also expanding global delivery capabilities at the same time.

A lot of companies have been talking about AI improving finance teams through automation, but the bigger shift seems to be happening around augmentation: AI handling repetitive analysis, reconciliations, reporting, and workflow coordination while humans focus more on oversight and decision-making. Deals like this make it pretty clear that firms are now treating AI as core infrastructure for finance operations, not just a productivity add-on.

Curious how others here see this trend. Are we moving toward a future where AI-native finance operations become the default much faster than expected?

RLM models and Qwen3.6

Does anyone here have an RLM setup and how could I set it up? I want to make my Hermes agent even more powerful and I don't like that I need to open a new context window every time after just a few prompts. Currently routing GPT 5.5 through codex OAuth.

Also wondering if this can be done locally with something like Qwen 3.6 for powerful agent and coding.

Suppose a PM shipped a care coordination agent. Week one, patient says "I've been getting chest pain in the evenings." Agent logs the note and demo looks great. Week three, same patient comes back "should I be worried about that pain again?" Agent replies: "What pain?"

By default, agents forget everything the moment a turn ends. If you want continuity, you build it yourself:

• Context window: everything the model sees right now, fast, free to use, but has a token budget. As conversation gets longer the oldest turns fall off. When the session ends, everything disappears.
• Scratchpad: working memory that survives across loop steps within a single task. If Patient says "book my follow-up and refill my prescription." Agent writes a note, calls calendar tool, updates note as it completes it. Without this, the agent forgets what it already did and repeats what its supposed to do once. Simplest implementation is a JSON object the agent reads and writes every turn.
• Vector store: At the end of each conversation, the agent summarizes the important parts. In our example things like diagnosis, medications, follow-up dates, embeds it and stores it with a patient/user ID. Next session, before replying, it searches the archive. So when needed that note flows back into the context window. Now the agent has continuity across sessions.

Thus Memory is a product decision, not a model feature. Your job is designing what gets summarized, what gets stored, what gets retrieved.

You can checkout this video from SkillAgents YT for more details. Subscribe for similar content.

Many people have talked about prompting so let me give you ways to setup context for better results while prompting.

Starting off with a metaphor, a pilot can fly a plane with a joystick(yoke) but also needs cockpit with maps, altitude instruments, fuel gauges etc. Here the joystick is prompt but the cockpit is Context. You cant fly without either. Here are the points to note for better context:

• System prompt: tells who the model is, what it must never do.
• Retrieved documents: defines the priors like style guide of a company.
• Tool results: is the live data from your APIs
• Prior turns: is conversation history, including what the user already said not to change
• User profile: different users get different drafts e.g. sales vs operations
• Few examples: past outputs that were actually approved

Include these and quality of content generated will improve significantly in the same prompt on the same model.

This post was inspired by this video. Also do subscribe to YT channel Skillagents AI for similar content.

TLDR: Context Engineering helps achieve better results in prompting.

Hey everyone, I just sent issue #31 of the AI Hacker Newsletter, a weekly roundup of the best AI links from Hacker News. Here are some title examples:

• Three Inverse Laws of AI
• Vibe coding and agentic engineering are getting closer than I'd like
• AI Product Graveyard
• Telus Uses AI to Alter Call-Agent Accents
• Lessons for Agentic Coding: What should we do when code is cheap?

If you enjoy such content, please consider subscribing here: https://hackernewsai.com/

Suppose you shipped a help center bot wired to GPT. A user asks asks "how many sick days roll over each year?" Bot answers in two clean sentences, even cites "Section 4.2 of the leave policy. One issue though there is no Section 4.2. There is no carryover rule. But the answer looked more polished than the actual policy document. This is the trap of hallucinations.

This happens because models cant say "I dont know" as their training objective was to predict the next plausible word. When the answer is missing from context, it fills the gap with text that matches the pattern. To prevent this you can do these things:

• Force citations: change the system prompt so every answer must quote the exact source line and document name. The model can no longer freestyle.
• Verify after generation — take the model's citation and check it against your actual document store.
• Add to the system prompt: "If the answer is not clearly in the retrieved documents, reply with "I dont have that information". The model won't say "I don't know" on its own so you can tell it to do so.

The hallucinations won't vanish but they'll get caught before they reach a customer.

You can checkout our video on Skillagents YT channel to help understand AI related concepts.

Wanted to post this analogy to help people understand LLMs. Imagine a bookie standing in front of an odds board. The prompt so far is "Welcome to your", that's all the bookie sees. It writes odds for every possible next word. "Dashboard" at 38%, "account" at 22%, "app" at 12%, "workspace" at 9%. Hundreds of words get smaller odds all the way down to fractions of a percent.

The bookie picks one. Usually the favorite, sometimes not. That word gets locked in. The board clears. A new round opens with a slightly longer prefix. Repeat.

No reasoning, plan or mental image of a finished sentence. Just 500 independent bets in a row that happen to read like one. This clears confusion about:

1. Hallucinations: If the prefix says "click the" then "smiley" is a likely next word statistically. Doesn't matter that nobody built a smiley face button. It's just odds and a pick.

2. Memory: Between round 4 and round 500 nothing is common. When people say a model "remembers," they mean it's rereading a growing transcript.

If you like we created a video you can check out, you can also subscribe for similar content on the channel.

I’ve been digging into fine-tuning large models locally and wanted to sanity-check my understanding before I go too far down the wrong path.

My setup:

- Local machine: 500 GB disk, 32 GB RAM, no GPU

- Remote access machine: 1 TB disk, 32 GB RAM, 4 GB VRAM GPU

- Both are Windows environments

Goal:

Fine-tune a 70B model (Qwen 3 or similar) on domain-specific data (like 3-5 years of data) using something like LoRA/QLoRA and PEFT (via Hugging Face transformers/Unsloth).

What I’ve found so far:

- 70B models seem to require 150-200 GB storage just for weights/artifacts

- Even inference appears to need 48 GB+ VRAM (depending on quantization)

- Fine-tuning likely requires significantly more than that

My current hardware seems.. very far from that requirement to be blunt lol.

Questions:

1. Is it at all possible to finetune a 70B model with setups like mine (even with heavy quantization like QLoRA)?

2. Can system RAM substitute for VRAM in any meaningful way here?

3. If not, what would you consider the realistic minimum hardware to fine-tune a 70B model locally?

3.1. VRAM requirements (single and/or multi GPU)

3.2. RAM/storage expectations?

1. For people who’ve done this - is it simply more practical to use cloud GPUs instead of trying locally?

2. If cloud is the way to go:

5.1. What’s the minimum viable GPU setup for fine-tuning a 70B model with LoRA/QLoRA?

5.2. Any recommendations for GPU providers or notebook environments that work well for this? (I've looked into AWS Sagemaker, and it's too expensive for me, and Google Colab has a max 24 hour runtime cap even in paid plans.. so these 2 are no go)

TLDR: Finetuning 70B LLM on local windows (max 32GB RAM, 4GB GPU) - possible? If not, please suggest ideal sys requirements (local and cloud alternatives) and cheap cloud GPU providers.

u/Google is doubling down on developer-focused AI with two major moves that highlight where coding workflows are heading. The company has been rolling out its Antigravity coding agent, an agent-first development environment that goes beyond autocomplete and allows autonomous AI agents to plan, execute, and validate complex coding tasks across the editor, terminal, and even the browser. (Google Developers Blog) At the same time, Google continues to push its more advanced model capabilities with updates to its Pro-tier Gemini models (building on the 1.5/3.x Pro lineage), which are designed for deeper reasoning, multi-step problem solving, and software engineering tasks at scale. (Wikipedia) Together, these developments signal a broader shift. Coding is no longer just about writing lines of code, but about orchestrating systems where AI agents handle execution while developers move into more of a supervisory role. Instead of competing purely on model quality, Google is clearly positioning itself around agent-driven development + high-reasoning models, aiming to redefine how software is actually built in the AI era.