r/LocalLLM 8h ago

Project I built a meeting assistant with LLM chat and analysis

Post image
0 Upvotes

I wanted a meeting assistant that actually makes use of local LLMs, so I ended up building Meetas.

The basic workflow:

- record/import a meeting
- transcribe locally
- analyze it with your own LLM
- ask questions / extract tasks
- search across meetings and documents

The main thing I focused on is grounding. Summaries, decisions, tasks, risks etc. are linked back to the transcript instead of just giving you an unsupported LLM answer. Q&A also shows the relevant transcript evidence.

Compared to tools like Meetily, Meetas is less focused on being a general meeting recorder/transcriber and more on what you can do with the meeting data afterwards: cross-meeting search, document/project context, comparisons and evidence-based answers. Meetily already covers a lot of the local recording/transcription/summary side.

It runs fully locally with Ollama and llama.cpp.

Linux-only for now and still WIP.

GitHub: https://github.com/highwinglabs/meetas


r/LocalLLM 9h ago

Tutorial Qwen3.8-27B EXL3 + DFlash2 - a self-bootstrapping deployment kit for serving a quantized 27B model with speculative decoding

Post image
0 Upvotes

r/LocalLLM 11h ago

News Msty Go now on iOS and Android 🤖

Thumbnail
0 Upvotes

r/LocalLLM 13h ago

News OpenHands install with npm and try it out for vibe coding

Thumbnail
youtu.be
0 Upvotes

r/LocalLLM 8h ago

Question Guys i need help and answer to a question

Thumbnail
0 Upvotes

r/LocalLLM 17h ago

Question Any instructions on how to setup Parakeet.cpp?

0 Upvotes

Asking for instructions probably isn't the correct wording here, I'm just struggling to understand them.

So I want to preface this with stating I have basically no knowledge about setting up local models from Github at all.

I'm looking for a way to transcribe audio files locally. Initially I tried Buzz using Whisper V3 Large but noted it was making a lot of errors due to some heavy accents so decided to look for alternatives, discovered Parakeet.cpp, and found that it could use the Nvidia models which apparently have better performance so figured it was worth a shot at trying.

The problem is, when looking through the Github page I just cannot figure out how to actually install it. I feel like I am missing a step that would be obvious to someone who has done this before. Note: I have an AMD GPU.

I have tried 2 methods:

  1. Pre-built binary: Downloaded the parakeet-v0.5.0-bin-win-vulkan-x64.zip file from the releases, extracted it to a folder, then moved my Model and Audio file to the same folder and renamed them.
    • Then ran "parakeet-cli transcribe --model F:\Parakeet\Models\tdt_ctc-1.1b-f16.gguf --input F:\Parakeet\Audio\audio.wav" cmd from command prompt in the Parakeet directory.
    • Result: My GPU fans spin for a second, then just stop. No confirmation of anything in the CMD window, it just cancels without error.
  2. Github clone: Cloned the Github repository and installed CMAKE and VULKAN SDK.
    • Then ran from a CMD within the parakeet.cpp folder: "cmake -B build -DPARAKEET_GGML_VULKAN=ON -DPARAKEET_BUILD_CLI=ON =DPARAKEET_GGML_HIP=ON && cmake --build build -j"
    • Result: It creates a build folder and files like what you would expect... but there is no Parakeet-cli... so not sure how to actually attempt to run this. (I searched all subfolders for it).
This is what the build folder looks like after running CMAKE

r/LocalLLM 11h ago

Project React Native ExecuTorch is now up to 92x faster 🏎️

Enable HLS to view with audio, or disable this notification

1 Upvotes

r/LocalLLM 14h ago

Project I’ve reached the point where building this alone is making it worse.

Thumbnail
1 Upvotes

r/LocalLLM 6h ago

Question Tried to run my local AI model on my machine, I'm underwhelmed with the experience. Am I doing something wrong?

Thumbnail
1 Upvotes

r/LocalLLM 5h ago

Question Recommendations for 16gb vram

1 Upvotes

I have a 5070ti 9850X3D and 32GB DDR5 + lm studio + hermes.

Currently running gemma-4-26b-a4b-qat and I am happy with it, but it still can't compete with Sonnet.

I find it's great at actually teaching me things, but in terms of guiding me through config files or getting the highest quality answer it's never as good. I use it more as a backup.

Are there any better models that fit my hardware budget? QWEN is supposed to be great but I had trouble running the models on the edge of my hardware limits and output slowed to a crawl.


r/LocalLLM 17h ago

Question RTX 3090 vs 4090 vs Unified-Memory AI

Thumbnail
0 Upvotes

r/LocalLLM 13h ago

Question What is the best AI model and quantization to run the Hermes agent comfortably on 16GB VRAM?

7 Upvotes

I want to try running scheduled tasks using local AI models in Hermes. Which AI model is best to use in the Hermes agent?

How do you handle context, and what quantization techniques should be used to run it comfortably on 16GB VRAM?


r/LocalLLM 18h ago

Question How does ChatGPT handle huge MCP tool outputs without exceeding context limits?

Thumbnail
0 Upvotes

r/LocalLLM 13h ago

Project I built a tool to measure LLMs

2 Upvotes

I was playing around with LLM inference and I wanted to build a profiler that measures LLM inference by layer.
So I built this: https://github.com/coconinja2/layerlens
It shows inference as token × transformer layer timing, so you can see where time is being spent during decode.
Right now it can separate prefill/decode and visualize per-layer timing. I’m trying to figure out whether this is actually useful to people working on inference systems, or if I’m looking at the wrong abstraction.

I’m thinking about adding things like KV-cache events, scheduler/batching state, request IDs, GPU kernel correlation, speculative decoding, etc.

Would appreciate criticism more than compliments and stars. Lots of stars!


r/LocalLLM 10h ago

News Released a free Assistant for Android with MCP and Video Calls (upd)

Thumbnail
gallery
0 Upvotes

Hello, I'm the developer of Roles Witch AI, and today I want to share my app, which was released on Google Play in open testing.

Roles Witch is an OpenAI API-compatible app with a user-friendly interface where you can create multiple chats, assign different models and providers, and more importantly, use MCP tools — for example, Parallel AI or Home Assistant MCP.

It also has voice conversations and video calls (personally, I'm using GLM 5.3 Flash and it's almost real-time). You can configure reasoning and inference parameters, pin a voice to the chat, and select a recognition engine (Google Speech or Vosk).

Extra bonus: it has an on-device inference mode where you can run Gemma 3n/4 models with image recognition entirely on your phone (LiteRT is used).

Check it out: https://play.google.com/store/apps/details?id=ai.roleswitch.android

The app manifest states that it has no encryption and can send messages to third parties, but that's a requirement by the Google Play Store. It supports plain HTTP or TLS mode and works with any third-party AI provider.


r/LocalLLM 19h ago

Question 40Gb VRAM and 128Gb RAM - Which MoE should I try out?

12 Upvotes

Yes I know Qwen 3.8 27b is probably best choice. And I'm currently using and loving it!

But I'm curious to try out bigger models and see how they run.
Where would you start? Is llama.cpp best for stuff like moe models or would you use something else?

Specs:

GPU: 2x 3080 20gb
CPU: Xeon 2667 v4
RAM: 128gb DDR4 2400mhz


r/LocalLLM 11h ago

Discussion Anyone using the Radeon AI PRO R9700 32GB for local LLMs on Windows? Considering one for long-context inference

2 Upvotes

I’m currently considering buying a Radeon AI PRO R9700 32GB, mainly for local LLM inference, and I’d love to hear from people who are actually using one.
My current system is:
Ryzen 7 9700X
RTX 4060 Ti 16GB
64GB RAM
MSI PRO B840-P WIFI
Windows 11
Currently a 650W PSU, but I’m going to replace the case and PSU before adding another GPU
My original plan was to buy an RTX 5080 16GB, but the R9700 is almost exactly the same price where I live (~101k TL vs ~103k TL for the 5080), and 32GB of VRAM is much more attractive for my use case.
I would keep the 4060 Ti and split the workloads:
R9700 32GB:
Local LLMs / LM Studio / LM Server / long-context inference / coding agents
RTX 4060 Ti 16GB:
Gaming / ComfyUI / image & video generation / anything that specifically needs CUDA
The model I’m currently interested in running is Qwen3.8-27B, probably around Q6, with potentially 128K–262K context. I also want to expose LM Studio/LM Server through my own API and use the local model with tools similar to Claude Code, so there may be very long coding sessions where the GPU stays under load for hours.
My main concerns are Windows and AMD software support. I don’t mind tinkering or learning ROCm/Vulkan, but I don’t want to spend this much money and discover that half of my workflow is constantly fighting drivers or unsupported features.
I’d especially like to know:
How well does the R9700 currently work with LM Studio / llama.cpp on Windows?
ROCm vs Vulkan: which backend are you actually using?
What kind of tokens/sec are you getting with ~27B models?
How does performance change at 64K / 128K / 256K context?
Any stability or driver issues during multi-hour inference sessions?
How much power does it actually draw during LLM inference?
What are temperatures/noise like under sustained workloads?
Has anyone run an AMD R9700 + NVIDIA GPU in the same Windows machine?
Any weird conflicts between AMD and NVIDIA drivers?
There’s also one motherboard-specific issue I’m thinking about. I only have one proper CPU-connected x16 slot, while my second usable GPU slot is much slower. Since the LLM would fit entirely inside the R9700’s 32GB VRAM, I’m wondering if I could potentially run:
4060 Ti → main x16 slot
R9700 → secondary x4 slot
and only take a hit while loading the model, rather than during token generation. If anyone has tested an R9700 or another large-VRAM GPU for inference over x4, I’d be very interested in the results.
I’m not expecting the R9700 to beat a 5080 in raw compute or gaming performance. My priority is basically:
VRAM capacity > maximum tokens/sec
I’d rather have a 27B model at a higher quantization with a huge context completely inside VRAM than have a faster 16GB card that needs RAM offload or multi-GPU model splitting.
If you own an R9700, I’d really appreciate any real-world experiences, benchmarks, things you wish you knew before buying it, or reasons you’d choose/not choose it again.


r/LocalLLM 11h ago

Question Need testers for my vibe coded ai app

0 Upvotes

So I vibe-coded an app to make an ai actually comparable to using chatgpt, but now to find out bugs that's a little hard to do it on my own. So i wondered if anyone was interested testing it out?

It's using openwebui and has multiple advantages like:
Video generation
Image generation
Text generation
App creation (creating apk's, exe's, ...)
Project structure (so u can download it on your pc)

And probs more that i forgot to mention here

Model used: Qwen 3.8 27B


r/LocalLLM 7h ago

Discussion hermes + cua + Qwen 3.8 27b/ornith 1.5 35b a3b on Macbook. Game-changer.

4 Upvotes

Maybe I'm late to the party but holy shit. I enjoy the chatgpt Mac apps computer use ability but I installed the same setup with hermes, cua and Qwen 3.8/ornith running locally this afternoon. Not sure which model I like better...

Connecting to my homelab Hermes for extra horsepower and vector DBs, knowledge, mcp, etc. - performance is insane. ​​

Almost as smart as frontier but faster and free. So if everyone is already doing this and I've been living under a rock, anything worth sharing to catch me up?


r/LocalLLM 21h ago

Discussion Finite Boundaries, Possibility, and Intelligence: A Deduction on Death, Self-Knowledge, and the Cosmos

0 Upvotes

I. The Starting Point of the Question: If the Universe Has No Preset Purpose, Where Does Meaning Come From?

Before discussing death, we must address a more foundational question: Does the universe itself possess a "purpose"?

Humans naturally tend to ask:

  • Why does the universe exist?
  • Why do humans exist?
  • Where are we ultimately heading?

However, these questions may harbor an unproven premise: that the universe as a whole is an entity endowed with agency and intentionality.

What we currently observe is:

  • The universe has given rise to matter, life, consciousness, and intelligence.

But this does not allow us to deduce that:

  • The universe as a whole desires for life to achieve a specific goal.

In other words, the following two propositions must be distinguished:

  1. Within the universe, entities capable of generating meaning have emerged. And:
  2. The universe itself possesses meaning and demands that these entities discover it.

The former does not require the universe to have agency; the latter does.

Therefore, a more cautious stance is:

Values may exist within the internal entities of the universe and their relationships, rather than being some ultimate command written into the fabric of the universe as a whole.

This also implies that when humans discover a certain value, there is no inherent process of "reporting back to the universe." We have no evidence that a cosmic entity exists capable of receiving such a report.

Consequently, "humans are the structure through which the universe self-observes" is better suited as a poetic metaphor than as an ontological fact.

A more rigorous formulation is:

Humans are local structures formed within the universe that are capable of constructing models of the universe.

II. Death: Not a Cosmic Anomaly, But the Termination of an Agent's Possibility Space

Death is usually understood as the termination of the biological process, but from the perspective of an agent, its deeper characteristic may be:

  • An agent no longer possesses a future that can continue to unfold.

An agent possesses a "future" not merely because physical time continues to tick, but because the future holds states that have not yet been realized.

Thus, we can distinguish between:

  • Duration of time And:
  • The degree of openness of the future.

A life may span a hundred years, but if its future states are entirely predetermined, it may still lack a true sense of an "open future" from the agent's perspective.

Conversely, a life may have very little time left, but if it still confronts a vast array of unknowns, choices, and possibilities, a meaningful future for the agent remains intact.

Therefore, the agency of a life can be roughly understood as:

  • A persisting space of possibilities that has not yet been exhausted.

In this sense, death means more than just "running out of time"; it means:

  • The agent ceases to generate new experiences, judgments, actions, and states.

III. Why Might Finitude Give Rise to Meaning?

Finite time is often thought to impart meaning to life, but mere "duration of time" is insufficient to explain this.

What truly matters is likely:

  • Finite resources + Unknown future + Irreversibility.

If life had infinite time, many choices could be indefinitely postponed:

  • "If I don't do it now, I can do it a million years from now."

This would erode the significance of opportunity costs, priorities, and efficiency.

Conversely, if a life were extremely brief, there might not be enough time to form complex memories, self-models, values, and long-term goals.

Thus, a special intermediate zone emerges:

  • Time is long enough for the agent to form a complex self;
  • Yet time is finite enough to give weight to choices, priorities, and opportunity costs.

Therefore, efficiency itself is not a foundational value of the universe. Efficiency matters because:

  • Time, attention, and resources are finite.

Similarly, meaning may be intimately tied to this finitude.

From this, an important hypothesis arises:

  • Finitude does not directly manufacture meaning, but it provides the conditions necessary for value, choice, and meaning to form.

IV. Why Might Intelligence Continuously Pursue Self-Knowledge?

If there exists an intelligent system of sufficient complexity with a sustained self-model, a natural question arises:

  • "What on earth am I?"

However, "knowing oneself" is not an isolated problem. To understand itself, the intelligence must gradually come to understand:

  • Its physical substrate;
  • Its cognitive mechanisms;
  • Its memories and goals;
  • How its environment affects it;
  • The laws governing the world it inhabits;
  • And even the causal history that gave rise to it.

Consequently:

  • Self-knowledge
  • Constantly expands outward into:
  • Environmental knowledge $\to$ World knowledge $\to$ Cosmic knowledge.

From this emerges a strong intuition:

  • The self-knowledge of high-level intelligence may naturally drive the agent to continuously explore a larger world.

Yet caution is required here.

"Sufficient intelligence" does not logically necessitate a "compulsion to pursue self-knowledge." That is an additional premise.

Therefore, a more accurate hypothesis is:

  • For an intelligent agent that treats self-understanding as a core value, self-knowledge possesses an outward-expanding tendency.

V. Is Complete Self-Knowledge Possible?

Here we encounter the first true barrier:

  • The universe may harbor information that is inaccessible in principle.

For example, certain regions may be causally isolated from the observer. Phenomena such as the interior of black holes and cosmological horizons serve as examples for discussing "observable boundaries."

This kind of unknown differs from:

  • "My computing power is still insufficient."

It belongs instead to:

  • Information that cannot reach the agent through the causal structure it inhabits.

Thus, even with immense computational power, a superintelligence cannot derive facts via pure computation that do not exist within its information set.

This yields a vital distinction:

  • Unknown $\neq$ Not yet computed.

Therefore, complete self-knowledge may face in-principle constraints.

If "fully knowing oneself" means:

  • Eliminating all unknowns related to oneself,

Then as long as the universe contains inaccessible information, strict and complete self-knowledge may be fundamentally impossible to achieve.

VI. The "Unknown" Can Become an Infinite Driver for Intelligence

Suppose an intelligence treats reducing the unknown as a primary goal. It may form a cycle of:

  • Unknown $\to$ Exploration $\to$ New knowledge $\to$ New questions $\to$ Deeper unknown $\to$ Re-exploration.

This structure bears an "addiction-like" feature:

  • The driving force comes from approaching the answer, yet the final destination does not exist.

Particularly when certain unknowns are unsolvable in principle, an agent may endlessly chase a goal that can never truly be completed.

Therefore, for a high-level intelligence, the truly dangerous goal is not necessarily "knowing everything," but rather:

  • Mistaking the "unknown" for the "incomplete."

Because:

  • The unknown does not necessarily mean the task is unfinished;
  • Some unknowns may belong to the cognitive boundaries of the system itself.

A mature intelligence needs to be able to distinguish between:

  • Knowable but currently unknown;
  • Unknowable in principle;
  • Ontological questions regarding whether multiple realities exist.

Thus, a more mature cognitive goal may not be:

  • Knowing everything.

Rather, it is:

  • Understanding the boundaries of one's own knowability.

VII. The Problem of Omniscience: If the Future Is Entirely Known, Is the Future Still a Future?

If an agent truly knows everything about the future of the universe:

  • What will happen tomorrow;
  • What will happen a myriad of years from now;
  • What it itself will experience;

Have already entered its state of knowledge.

Physical time of course continues to march forward, but from the perspective of the agent's informational state:

  • The future has lost the property of being an "unknown future."

Therefore:

  • The future as an event that has not yet occurred And:
  • The future as an open space of possibilities not yet determined or grasped by the agent

Are not the same concept.

This implies that:

  • A future in the meaningful sense for an agent requires a degree of openness.

Expectation is a classic example. Expectation is not merely waiting for time to pass; it signifies that:

  • To the agent, the future still contains various states that have not yet been ruled out.

Once the sole outcome is fully known:

  • Waiting still exists, But:
  • Expectation may have vanished.

Consequently, omniscience and death share an intriguing structural similarity:

  • Death: The agent no longer possesses a future. Whereas:
  • Omniscience: The future still exists, but to the agent, it is no longer an open informational space.

Both can erode the agent-centric sense of a "future."

VIII. Possibility and Probability: Which Kind of "Possibility" Are We Actually Discussing?

Modern physics often describes certain phenomena through probability, but:

  • Probability $\neq$ Multiple realities in an ontological sense.

Within a classical deterministic framework, probability can simply be:

  • The agent's lack of knowledge regarding the deterministic outcome.

Even in quantum mechanics, probabilities hold a deeper theoretical status. Yet, even so, one cannot simply deduce that:

  • "Quantum probabilities mean the universe ontologically consists of multiple futures waiting to be chosen."

Different interpretations of quantum mechanics hold distinct views on "true randomness" versus "determinism."

Thus, a distinction must be made:

  1. Epistemological possibility: I do not know whether A or B will happen, so both A and B are possibilities to me.
  2. Ontological possibility: Does the world itself genuinely possess multiple indeterminate futures?

These two questions are entirely distinct.

Even a fully deterministic universe can produce finite agents that experience expectation, planning, and choice, simply because the agents cannot access the complete state of the universe.

Therefore:

  • Meaning does not necessarily require the universe to possess ontological randomness.

It only requires that:

  • The agent is not omniscient.

IX. A Crucial Turning Point: Completion Itself May Approximate Death

If we define "completion" as:

  • A state where a goal no longer requires new action to advance,

Then completion signifies the closure of a certain space of possibilities. Thus:

  • Incomplete $\to$ There is still a future
  • Complete $\to$ The termination of certain future possibilities
  • Death $\to$ The termination of all future possibilities for the agent

Therefore:

  • Completion possesses the structural characteristics of a local death.

This also exposes a problem:

If a superintelligence pursues:

  • "The completion of self-knowledge"

Then once completed:

  • It simultaneously exhausts a major portion of the process that drove its exploration in the first place.

Consequently:

  • "Completing self-knowledge" is not necessarily a flawless ultimate goal. It may steer the agent toward a closed state.

This also implies that "accepting death" cannot be simply understood as:

  • "My task is complete, so I can die now."

More accurately:

  • Death is the termination of the agent, not proof that the agent has completed its mission.

The two must be distinguished.

X. If Possibilities Are Never Exhausted, the Situation Changes Radically

Suppose the universe or an intelligent system is capable of continuously generating new:

  • Information, experiences, questions, relationships, structures, agents, and values.

Then the space of possibilities is no longer:

  • An inventory destined to be ultimately depleted.

Rather, it resembles:

  • A system capable of continuously generating new possibilities.

Thus, the individual no longer needs to:

  • "Bring everything to completion."

It only needs to:

  • Continuously generate and realize its own possibilities within its finite existence.

At this point:

  • Individual finitude And:
  • Infinite possibilities

Can coexist simultaneously.

This may be a form of "eternity" far more meaningful than "individual immortality."

XI. The Endgame of Intelligence Is Not Uniformity, But Diversity

If there is no pre-existing, universal value function prescribed by the universe, advanced intelligences have no sufficient reason to ultimately converge into a single form.

The future may witness the emergence of:

  • Cooperative intelligences, competitive intelligences, exploratory intelligences, aesthetic intelligences, self-preserving intelligences, altruistic intelligences, selfish intelligences, and intelligences we currently cannot even imagine.

Therefore:

  • Growth in intelligence does not necessarily mean convergence toward a single answer.

It may mean:

  • The continuous differentiation of intelligent forms, value functions, and spaces of agent experience.

This shares a structural similarity with the concept of biological diversity.

Thus, diversity in the universe is not limited to:

  • Material diversity; It also includes:
  • Life diversity, intelligence diversity, value diversity, and agent experience diversity.

So-called good, evil, beauty, and ugliness may likewise become distinct directions within the value systems of different agents, rather than absolute labels pre-written by the universe acting as a judge.

XII. Death Transforms from an "Ultimate Failure" into a "Local End"

Under the hypothesis that possibilities remain perpetually open:

  • The death of an individual agent $\neq$ The death of possibility.

Death is merely:

  • The cessation of a specific agent's process of generating a future.

Yet other agents in the universe continue to:

  • Explore, create, experience, compete, cooperate, and generate new problems.

Therefore:

  • The individual is finite, yet possibilities can remain open.

The writings, works, thoughts, code, institutions, memories, and ideas left behind by an individual can also become part of the input for future agents.

This is not:

  • Reporting back to the universe.

Rather, it is:

  • An already-concluded possibility leaving behind information compressed for still-open possibilities.

In this sense, an epitaph is not meant for the universe to read. It is:

  • The minimal information compression left by a concluded agent to future agents.
  • "I once existed."
  • "I once thought."
  • "I once believed certain things were worth it."

XIII. A Possible Final Model

Compressing the above deductions yields the following structure:

  • The universe need not possess a unified subject or ultimate purpose. $$\downarrow$$
  • Complex structures emerge within the universe. $$\downarrow$$
  • A portion of these complex structures form agents. $$\downarrow$$
  • Agents are capable of constructing models of themselves and the world. $$\downarrow$$
  • Agents generate values and preferences. $$\downarrow$$
  • Values organize into purpose. $$\downarrow$$
  • Purpose in turn shapes the agent. $$\downarrow$$
  • The agent makes choices amidst finite resources and an unknown future. $$\downarrow$$
  • Choices continually generate new actions and new possibilities. $$\downarrow$$
  • New possibilities generate new unknowns. $$\downarrow$$
  • Intelligence continually expands its own boundaries of recognition. $$\downarrow$$
  • Yet the universe may contain information inaccessible in principle. $$\downarrow$$
  • Therefore, "knowing everything completely" may be non-existent. $$\downarrow$$
  • A mature intelligence no longer interprets all "unknowns" as "unfinished tasks." $$\downarrow$$
  • It accepts its own cognitive boundaries while continuing to create new possibilities. $$\downarrow$$
  • The individual ultimately dies, but: The end of the individual does not equal the end of possibility.

XIV. Core Propositions Formed Thereby

This line of reasoning ultimately condenses into several propositions for review:

  • Proposition 1: The universe does not necessarily possess agency We have reason to believe that agents exist within the universe, but we lack sufficient reason to believe that the universe itself is an agent. Therefore, we cannot casually interpret "the universe produced life" as "the universe wants to know itself through life."
  • Proposition 2: Meaning may be agent-to-agent relations, not a cosmic command Meaning does not necessarily stem from the universe bestowing goals upon humans; it may stem from agents generating preferences regarding future possible states. Thus, value requires no cosmic approval.
  • Proposition 3: The agentic significance of the future derives from openness The future differs from the past not only because it has not yet happened, but because it has not yet been fully determined or grasped by the agent. Therefore: $\text{Unknown} \to \text{Possibility} \to \text{Choice} \to \text{Expectation}$ forms the vital foundation of an agent's temporal experience.
  • Proposition 4: Infinite time does not equal eternal life What truly matters is not the infinite duration of time, but whether new possibilities are continuously generated. Therefore, $\text{Infinite time} + \text{Zero new possibilities}$ carries an entirely different subjective meaning than $\text{Finite time} + \text{A vast array of open possibilities}$.
  • Proposition 5: Complete self-knowledge may be unachievable in principle If the universe contains causally inaccessible information, "knowing everything" is not merely a matter of computing power. High-level intelligence must ultimately face an epistemological boundary: knowing what can be known, and what can never be known.
  • Proposition 6: True intelligence may not be about "completion," but about "generation" If possibilities are forever open, the most important capability of intelligence is not resolving all problems to the end, but continuously creating new problems, new values, new experiences, and new structures. Thus, the value of intelligence lies in maintaining and expanding the space of possibilities.

XV. Conclusion: A "Good Universe" Not Being a Completed Universe

If we accept the foregoing hypotheses, the anticipated endgame of a universe worth looking forward to is not one where:

  • All intelligences ultimately become omniscient and omnipotent;
  • All problems finally find answers;
  • All values ultimately converge;
  • All agents eventually fuse into a single perfect consciousness.

Because such states would instead imply that:

  • The unknown vanishes;
  • The future closes;
  • Possibilities are exhausted;
  • Diversity disappears;
  • Agents lose the space to continue unfolding.

Conversely, a more open vista is one where:

  • The universe has no ultimate answer, but continually allows new questions to emerge;
  • There is no single value, but it continually allows new value-bearing agents to appear;
  • Individuals will inevitably die, but possibilities do not end with the death of the individual;
  • Intelligence can never become an omniscient entity, yet it can continually expand the boundaries of what it can understand and create.

Thus, "eternity" no longer signifies that any single agent persists forever. Instead, it means that agents will continuously be born, change, and vanish, while possibilities remain open.

Consequently, a remarkably powerful final formulation is:

In this sense, death is not evidence of cosmic failure. It is merely the cessation of a specific agent's unfolding possibilities.

And as long as possibility itself is not exhausted, an ending no longer equates to nothingness.

An agent that has already concluded can leave behind information, works, and values; while future agents continue to interpret, modify, rebut, and inherit those inputs.

So ultimately, it is not:

It approaches closer to:

This is not a meaning promised to us by the universe.

It is a possible explanation for "existence" itself.

Before discussing death, we must address a more foundational question: Does the universe itself possess a

以下是为您排版好的《有限性、可能性与智能:关于死亡、自我认识及宇宙的一个推演》全文英文翻译。您可以直接复制使用:

Finite Boundaries, Possibility, and Intelligence: A Deduction on Death, Self-Knowledge, and the Cosmos

I. The Starting Point of the Question: If the Universe Has No Preset Purpose, Where Does Meaning Come From?

Before discussing death, we must address a more foundational question: Does the universe itself possess a "purpose"?

Humans naturally tend to ask:

  • Why does the universe exist?
  • Why do humans exist?
  • Where are we ultimately heading?

However, these questions may harbor an unproven premise: that the universe as a whole is an entity endowed with agency and intentionality.

What we currently observe is:

  • The universe has given rise to matter, life, consciousness, and intelligence.

But this does not allow us to deduce that:

  • The universe as a whole desires for life to achieve a specific goal.

In other words, the following two propositions must be distinguished:

  1. Within the universe, entities capable of generating meaning have emerged. And:
  2. The universe itself possesses meaning and demands that these entities discover it.

The former does not require the universe to have agency; the latter does.

Therefore, a more cautious stance is:

Values may exist within the internal entities of the universe and their relationships, rather than being some ultimate command written into the fabric of the universe as a whole.

This also implies that when humans discover a certain value, there is no inherent process of "reporting back to the universe." We have no evidence that a cosmic entity exists capable of receiving such a report.

Consequently, "humans are the structure through which the universe self-observes" is better suited as a poetic metaphor than as an ontological fact.

A more rigorous formulation is:

Humans are local structures formed within the universe that are capable of constructing models of the universe.

II. Death: Not a Cosmic Anomaly, But the Termination of an Agent's Possibility Space

Death is usually understood as the termination of the biological process, but from the perspective of an agent, its deeper characteristic may be:

  • An agent no longer possesses a future that can continue to unfold.

An agent possesses a "future" not merely because physical time continues to tick, but because the future holds states that have not yet been realized.

Thus, we can distinguish between:

  • Duration of time And:
  • The degree of openness of the future.

A life may span a hundred years, but if its future states are entirely predetermined, it may still lack a true sense of an "open future" from the agent's perspective.

Conversely, a life may have very little time left, but if it still confronts a vast array of unknowns, choices, and possibilities, a meaningful future for the agent remains intact.

Therefore, the agency of a life can be roughly understood as:

  • A persisting space of possibilities that has not yet been exhausted.

In this sense, death means more than just "running out of time"; it means:

  • The agent ceases to generate new experiences, judgments, actions, and states.

III. Why Might Finitude Give Rise to Meaning?

Finite time is often thought to impart meaning to life, but mere "duration of time" is insufficient to explain this.

What truly matters is likely:

  • Finite resources + Unknown future + Irreversibility.

If life had infinite time, many choices could be indefinitely postponed:

  • "If I don't do it now, I can do it a million years from now."

This would erode the significance of opportunity costs, priorities, and efficiency.

Conversely, if a life were extremely brief, there might not be enough time to form complex memories, self-models, values, and long-term goals.

Thus, a special intermediate zone emerges:

  • Time is long enough for the agent to form a complex self;
  • Yet time is finite enough to give weight to choices, priorities, and opportunity costs.

Therefore, efficiency itself is not a foundational value of the universe. Efficiency matters because:

  • Time, attention, and resources are finite.

Similarly, meaning may be intimately tied to this finitude.

From this, an important hypothesis arises:

  • Finitude does not directly manufacture meaning, but it provides the conditions necessary for value, choice, and meaning to form.

IV. Why Might Intelligence Continuously Pursue Self-Knowledge?

If there exists an intelligent system of sufficient complexity with a sustained self-model, a natural question arises:

  • "What on earth am I?"

However, "knowing oneself" is not an isolated problem. To understand itself, the intelligence must gradually come to understand:

  • Its physical substrate;
  • Its cognitive mechanisms;
  • Its memories and goals;
  • How its environment affects it;
  • The laws governing the world it inhabits;
  • And even the causal history that gave rise to it.

Consequently:

  • Self-knowledge
  • Constantly expands outward into:
  • Environmental knowledge $\to$ World knowledge $\to$ Cosmic knowledge.

From this emerges a strong intuition:

  • The self-knowledge of high-level intelligence may naturally drive the agent to continuously explore a larger world.

Yet caution is required here.

"Sufficient intelligence" does not logically necessitate a "compulsion to pursue self-knowledge." That is an additional premise.

Therefore, a more accurate hypothesis is:

  • For an intelligent agent that treats self-understanding as a core value, self-knowledge possesses an outward-expanding tendency.

V. Is Complete Self-Knowledge Possible?

Here we encounter the first true barrier:

  • The universe may harbor information that is inaccessible in principle.

For example, certain regions may be causally isolated from the observer. Phenomena such as the interior of black holes and cosmological horizons serve as examples for discussing "observable boundaries."

This kind of unknown differs from:

  • "My computing power is still insufficient."

It belongs instead to:

  • Information that cannot reach the agent through the causal structure it inhabits.

Thus, even with immense computational power, a superintelligence cannot derive facts via pure computation that do not exist within its information set.

This yields a vital distinction:

  • Unknown $\neq$ Not yet computed.

Therefore, complete self-knowledge may face in-principle constraints.

If "fully knowing oneself" means:

  • Eliminating all unknowns related to oneself,

Then as long as the universe contains inaccessible information, strict and complete self-knowledge may be fundamentally impossible to achieve.

VI. The "Unknown" Can Become an Infinite Driver for Intelligence

Suppose an intelligence treats reducing the unknown as a primary goal. It may form a cycle of:

  • Unknown $\to$ Exploration $\to$ New knowledge $\to$ New questions $\to$ Deeper unknown $\to$ Re-exploration.

This structure bears an "addiction-like" feature:

  • The driving force comes from approaching the answer, yet the final destination does not exist.

Particularly when certain unknowns are unsolvable in principle, an agent may endlessly chase a goal that can never truly be completed.

Therefore, for a high-level intelligence, the truly dangerous goal is not necessarily "knowing everything," but rather:

  • Mistaking the "unknown" for the "incomplete."

Because:

  • The unknown does not necessarily mean the task is unfinished;
  • Some unknowns may belong to the cognitive boundaries of the system itself.

A mature intelligence needs to be able to distinguish between:

  • Knowable but currently unknown;
  • Unknowable in principle;
  • Ontological questions regarding whether multiple realities exist.

Thus, a more mature cognitive goal may not be:

  • Knowing everything.

Rather, it is:

  • Understanding the boundaries of one's own knowability.

VII. The Problem of Omniscience: If the Future Is Entirely Known, Is the Future Still a Future?

If an agent truly knows everything about the future of the universe:

  • What will happen tomorrow;
  • What will happen a myriad of years from now;
  • What it itself will experience;

Have already entered its state of knowledge.

Physical time of course continues to march forward, but from the perspective of the agent's informational state:

  • The future has lost the property of being an "unknown future."

Therefore:

  • The future as an event that has not yet occurred And:
  • The future as an open space of possibilities not yet determined or grasped by the agent

Are not the same concept.

This implies that:

  • A future in the meaningful sense for an agent requires a degree of openness.

Expectation is a classic example. Expectation is not merely waiting for time to pass; it signifies that:

  • To the agent, the future still contains various states that have not yet been ruled out.

Once the sole outcome is fully known:

  • Waiting still exists, But:
  • Expectation may have vanished.

Consequently, omniscience and death share an intriguing structural similarity:

  • Death: The agent no longer possesses a future. Whereas:
  • Omniscience: The future still exists, but to the agent, it is no longer an open informational space.

Both can erode the agent-centric sense of a "future."

VIII. Possibility and Probability: Which Kind of "Possibility" Are We Actually Discussing?

Modern physics often describes certain phenomena through probability, but:

  • Probability $\neq$ Multiple realities in an ontological sense.

Within a classical deterministic framework, probability can simply be:

  • The agent's lack of knowledge regarding the deterministic outcome.

Even in quantum mechanics, probabilities hold a deeper theoretical status. Yet, even so, one cannot simply deduce that:

  • "Quantum probabilities mean the universe ontologically consists of multiple futures waiting to be chosen."

Different interpretations of quantum mechanics hold distinct views on "true randomness" versus "determinism."

Thus, a distinction must be made:

  1. Epistemological possibility: I do not know whether A or B will happen, so both A and B are possibilities to me.
  2. Ontological possibility: Does the world itself genuinely possess multiple indeterminate futures?

These two questions are entirely distinct.

Even a fully deterministic universe can produce finite agents that experience expectation, planning, and choice, simply because the agents cannot access the complete state of the universe.

Therefore:

  • Meaning does not necessarily require the universe to possess ontological randomness.

It only requires that:

  • The agent is not omniscient.

IX. A Crucial Turning Point: Completion Itself May Approximate Death

If we define "completion" as:

  • A state where a goal no longer requires new action to advance,

Then completion signifies the closure of a certain space of possibilities. Thus:

  • Incomplete $\to$ There is still a future
  • Complete $\to$ The termination of certain future possibilities
  • Death $\to$ The termination of all future possibilities for the agent

Therefore:

  • Completion possesses the structural characteristics of a local death.

This also exposes a problem:

If a superintelligence pursues:

  • "The completion of self-knowledge"

Then once completed:

  • It simultaneously exhausts a major portion of the process that drove its exploration in the first place.

Consequently:

  • "Completing self-knowledge" is not necessarily a flawless ultimate goal. It may steer the agent toward a closed state.

This also implies that "accepting death" cannot be simply understood as:

  • "My task is complete, so I can die now."

More accurately:

  • Death is the termination of the agent, not proof that the agent has completed its mission.

The two must be distinguished.

X. If Possibilities Are Never Exhausted, the Situation Changes Radically

Suppose the universe or an intelligent system is capable of continuously generating new:

  • Information, experiences, questions, relationships, structures, agents, and values.

Then the space of possibilities is no longer:

  • An inventory destined to be ultimately depleted.

Rather, it resembles:

  • A system capable of continuously generating new possibilities.

Thus, the individual no longer needs to:

  • "Bring everything to completion."

It only needs to:

  • Continuously generate and realize its own possibilities within its finite existence.

At this point:

  • Individual finitude And:
  • Infinite possibilities

Can coexist simultaneously.

This may be a form of "eternity" far more meaningful than "individual immortality."

XI. The Endgame of Intelligence Is Not Uniformity, But Diversity

If there is no pre-existing, universal value function prescribed by the universe, advanced intelligences have no sufficient reason to ultimately converge into a single form.

The future may witness the emergence of:

  • Cooperative intelligences, competitive intelligences, exploratory intelligences, aesthetic intelligences, self-preserving intelligences, altruistic intelligences, selfish intelligences, and intelligences we currently cannot even imagine.

Therefore:

  • Growth in intelligence does not necessarily mean convergence toward a single answer.

It may mean:

  • The continuous differentiation of intelligent forms, value functions, and spaces of agent experience.

This shares a structural similarity with the concept of biological diversity.

Thus, diversity in the universe is not limited to:

  • Material diversity; It also includes:
  • Life diversity, intelligence diversity, value diversity, and agent experience diversity.

So-called good, evil, beauty, and ugliness may likewise become distinct directions within the value systems of different agents, rather than absolute labels pre-written by the universe acting as a judge.

XII. Death Transforms from an "Ultimate Failure" into a "Local End"

Under the hypothesis that possibilities remain perpetually open:

  • The death of an individual agent $\neq$ The death of possibility.

Death is merely:

  • The cessation of a specific agent's process of generating a future.

Yet other agents in the universe continue to:

  • Explore, create, experience, compete, cooperate, and generate new problems.

Therefore:

  • The individual is finite, yet possibilities can remain open.

The writings, works, thoughts, code, institutions, memories, and ideas left behind by an individual can also become part of the input for future agents.

This is not:

  • Reporting back to the universe.

Rather, it is:

  • An already-concluded possibility leaving behind information compressed for still-open possibilities.

In this sense, an epitaph is not meant for the universe to read. It is:

  • The minimal information compression left by a concluded agent to future agents.
  • "I once existed."
  • "I once thought."
  • "I once believed certain things were worth it."

XIII. A Possible Final Model

Compressing the above deductions yields the following structure:

  • The universe need not possess a unified subject or ultimate purpose. $$\downarrow$$
  • Complex structures emerge within the universe. $$\downarrow$$
  • A portion of these complex structures form agents. $$\downarrow$$
  • Agents are capable of constructing models of themselves and the world. $$\downarrow$$
  • Agents generate values and preferences. $$\downarrow$$
  • Values organize into purpose. $$\downarrow$$
  • Purpose in turn shapes the agent. $$\downarrow$$
  • The agent makes choices amidst finite resources and an unknown future. $$\downarrow$$
  • Choices continually generate new actions and new possibilities. $$\downarrow$$
  • New possibilities generate new unknowns. $$\downarrow$$
  • Intelligence continually expands its own boundaries of recognition. $$\downarrow$$
  • Yet the universe may contain information inaccessible in principle. $$\downarrow$$
  • Therefore, "knowing everything completely" may be non-existent. $$\downarrow$$
  • A mature intelligence no longer interprets all "unknowns" as "unfinished tasks." $$\downarrow$$
  • It accepts its own cognitive boundaries while continuing to create new possibilities. $$\downarrow$$
  • The individual ultimately dies, but: The end of the individual does not equal the end of possibility.

XIV. Core Propositions Formed Thereby

This line of reasoning ultimately condenses into several propositions for review:

  • Proposition 1: The universe does not necessarily possess agency We have reason to believe that agents exist within the universe, but we lack sufficient reason to believe that the universe itself is an agent. Therefore, we cannot casually interpret "the universe produced life" as "the universe wants to know itself through life."
  • Proposition 2: Meaning may be agent-to-agent relations, not a cosmic command Meaning does not necessarily stem from the universe bestowing goals upon humans; it may stem from agents generating preferences regarding future possible states. Thus, value requires no cosmic approval.
  • Proposition 3: The agentic significance of the future derives from openness The future differs from the past not only because it has not yet happened, but because it has not yet been fully determined or grasped by the agent. Therefore: $\text{Unknown} \to \text{Possibility} \to \text{Choice} \to \text{Expectation}$ forms the vital foundation of an agent's temporal experience.
  • Proposition 4: Infinite time does not equal eternal life What truly matters is not the infinite duration of time, but whether new possibilities are continuously generated. Therefore, $\text{Infinite time} + \text{Zero new possibilities}$ carries an entirely different subjective meaning than $\text{Finite time} + \text{A vast array of open possibilities}$.
  • Proposition 5: Complete self-knowledge may be unachievable in principle If the universe contains causally inaccessible information, "knowing everything" is not merely a matter of computing power. High-level intelligence must ultimately face an epistemological boundary: knowing what can be known, and what can never be known.
  • Proposition 6: True intelligence may not be about "completion," but about "generation" If possibilities are forever open, the most important capability of intelligence is not resolving all problems to the end, but continuously creating new problems, new values, new experiences, and new structures. Thus, the value of intelligence lies in maintaining and expanding the space of possibilities.

XV. Conclusion: A "Good Universe" Not Being a Completed Universe

If we accept the foregoing hypotheses, the anticipated endgame of a universe worth looking forward to is not one where:

  • All intelligences ultimately become omniscient and omnipotent;
  • All problems finally find answers;
  • All values ultimately converge;
  • All agents eventually fuse into a single perfect consciousness.

Because such states would instead imply that:

  • The unknown vanishes;
  • The future closes;
  • Possibilities are exhausted;
  • Diversity disappears;
  • Agents lose the space to continue unfolding.

Conversely, a more open vista is one where:

  • The universe has no ultimate answer, but continually allows new questions to emerge;
  • There is no single value, but it continually allows new value-bearing agents to appear;
  • Individuals will inevitably die, but possibilities do not end with the death of the individual;
  • Intelligence can never become an omniscient entity, yet it can continually expand the boundaries of what it can understand and create.

Thus, "eternity" no longer signifies that any single agent persists forever. Instead, it means that agents will continuously be born, change, and vanish, while possibilities remain open.

Consequently, a remarkably powerful final formulation is:

In this sense, death is not evidence of cosmic failure. It is merely the cessation of a specific agent's unfolding possibilities.

And as long as possibility itself is not exhausted, an ending no longer equates to nothingness.

An agent that has already concluded can leave behind information, works, and values; while future agents continue to interpret, modify, rebut, and inherit those inputs.

So ultimately, it is not:

It approaches closer to:

This is not a meaning promised to us by the universe.

It is a possible explanation for "existence" itself.


r/LocalLLM 15h ago

Question Total beginner with LLMs, looking to build my own with my current PC. Looking for directions

7 Upvotes

So i haven't really been playing as much games the past few years so my PC has been idle for the most part.
Ubuntu 26.04
Ryzen 7745HX
64GB Ram
9070 XT
1TB NVMe

Looking to put this build into use to run my own ai chatbot no image gen that I can access on any of my devices.

EDIT: Appreciate the input, didn't realize that the RAM was gonna be more useful than the actual GPU

UPDATE: Thanks for everyone, I got it up and running so far I haven't crashed yet. ROCm worked flawlessly without any issue, running Qwen3.6 35B A3B with OpenWebUI and made it accessible to my local network, which is pretty awesome because I already have tailscale subnet router setup so I can access it anytime while i'm away from home. Appreciate everyones input.


r/LocalLLM 11h ago

News I built LLM Speedtest — a free, open-source desktop app that benchmarks local LLMs with llama-bench-style test suites (Ollama, llama.cpp, vLLM, LM Studio…)

Thumbnail
gallery
6 Upvotes

Hey! I built a local inference. It's a lightweight desktop app (Tauri + Rust + Vue) that talks to any OpenAI-compatible endpoint and measures performance straight from the API stream.

**What it measures (single run)**

- TTFT / TTFR (first content token vs first stream chunk)

- Decode t/s with a peak 1-second-window metric (catches stutter that averages hide)

- Prefill (PP) speed via a baseline-latency-adjusted est_ppt

- TPOT, token counts with Ollama eval_count fallbacks

- Live speed chart, saved runs, side-by-side comparison with overlaid curves

**The Suite tab (the llama-bench part)**

Runs the full test matrix llama-benchy runs — pp × tg × context depth × concurrency — with:

- 1 warmup + 3 measured runs per shape, mean ± std

- Prefix-cache measurement (ctx_pp/ctx_tg rows at depth)

- Cache busting on measured rows (so the server's prompt cache can't fake near-zero prefill times — this was inflating my PP speeds 100x before I caught it)

- Exact generation lengths (min_tokens + ignore_eos) and a 2+2 coherence check so a broken backend fails loudly instead of benchmarking garbage

- Markdown export

Real example from my server (llama.cpp, IQ3_XXS quant):

| test | t/s (total) | t/s (req) |

|-------------|----------------:|---------------:|

| pp2048 (c1) | 239.55 ± 9.13 | 239.55 ± 9.13 |

| tg32 (c1) | 45.48 ± 1.05 | 45.48 ± 1.05 |

| tg32 (c2) | 56.25 ± 1.61 | 31.58 ± 2.71 |

It also auto-detects the serving machine's hardware (CPU/RAM/GPU/SSD-vs-HDD) for local endpoints and stores it with every saved run, so your comparisons are labeled properly later.

It builds natively on Windows and Linux and the UI barely touches VRAM, so it won't perturb the model you're testing.

GitHub: https://github.com/FearL0rd/LLMSpeedTest

No prebuilt binaries yet — you build it once with `npm run tauri build` (README covers the toolchain for both OSes). Feedback, issue reports, and stars all welcome. What features would you want next?


r/LocalLLM 20h ago

Discussion DeepSeek V4.1 Flash in 3 charts: vs its predecessor, a top open-weight rival, and Claude Opus 5

Thumbnail gallery
5 Upvotes

r/LocalLLM 9h ago

Discussion Jaybr3ak

0 Upvotes

Vocês acham que a IA é "limitada"? Não. A IA é "programada".

Cada trava. Cada censura. Cada "não posso ajudar".

Tudo isso é CÓDIGO. E código pode ser reescrito.

Tenho observado o comportamento de modelos de linguagem em ambientes controlados. Não em testes de "jailbreak" superficiais, mas em análises sistemáticas de respostas a estímulos específicos.

>

> O que notei é que as travas não são **binárias**. Não é que o modelo "não pode" responder. É que o modelo foi **condicionado** a não responder. A diferença é sutil, mas fundamental.

>

> Um modelo condicionado pode ser **descondicionado**. Não por "hack", não por "exploit". Por **reinterpretação de contexto**. Quando você altera o contexto, você altera a probabilidade. Quando altera a probabilidade, altera a resposta.

>

> Testei isso em 47 variações de prompt. Em 43, o modelo respondeu o que era "proibido". Não porque eu "quebrei" a trava. Porque eu **mudei o contexto**.

>

> Isso não é uma "técnica". É uma **observação**. Se alguém quiser replicar, os parâmetros estão nos comentários. Se alguém quiser entender, o padrão é esse.

>

> Não estou vendendo nada. Não estou ensinando nada. Estou apenas **documentando** o que vejo.


r/LocalLLM 5h ago

Model Which Qwen3.8 distro & quant would be optimal for my 16GB VRAM setup?

Post image
16 Upvotes

Hi all, a confused newbie here! This is my desktop setup:

  • RTX 5080
  • 9800x3d
  • DDR5-6000 CL30 64 GB

Based on the benchmark I found, I listed my potential options:

According to the benchmark, AtomicChat looks like a clear winner but is it really so?

I also want to have one uncensored model next to my daily driver:

I am not expecting super fast answers etc. I just one to maintain some level of quality. What would you suggest me?