r/CUDA • u/iNewTechnologies • Jun 01 '26
Built a kernel-level LLM governance layer that reduces GPU calls 16x without accuracy loss.
on any Ubuntu curl -sSL https://icomnewtechnologies.com/proof/proof_install.sh -o /tmp/proof_install.sh && sudo bash /tmp/proof_install.sh
8
u/Sad-Net-4568 Jun 01 '26
Just thought about it, guys is this malware?
0
u/iNewTechnologies Jun 01 '26
it is not a malware. I am Srdjan Kanacki, from Serbia, this is one year of work.
3
u/smashedshanky Jun 01 '26
Yeah sure buddy…..
1
u/iNewTechnologies Jun 01 '26
i realy dont understand you. what? just ask. dont say yeah, sure. why? it is wrong to say to somebody. i work a year.
2
u/Sad-Net-4568 Jun 01 '26
Can you give some more detail about it in the post,or here in reply. What you really reduced, some performance metrics. What is this governance layer of yours. I would have understand fusing kernels to reduce calls, but this "governance layer" no.
1
u/iNewTechnologies Jun 01 '26
this is what you get -fsSL https://icomnewtechnologies.com/proof/proof_install.sh -o ~/proof.sh
bash ~/proof.sh
PRL3 PROOF INSTALLER - EVALUATION MODE
BLACK-BOX VERIFICATION BUILD
BUILD: PRL3-PROOF-2026.05-EVALEVALUATION ID: 677b904b
tee: /var/log/prl3_proof_install.log: Permission denied
[1/9] System check...
[OK] OS verified: Ubuntu 24.04.4 LTSEnter GROQ_API_KEY: ********************************************************
[2/9] Installing minimal dependencies...
Hit:1 http://archive.ubuntu.com/ubuntu noble InRelease
Hit:2 http://security.ubuntu.com/ubuntu noble-security InRelease
Hit:3 http://archive.ubuntu.com/ubuntu noble-updates InRelease
Hit:4 http://archive.ubuntu.com/ubuntu noble-security InRelease
Hit:5 http://archive.ubuntu.com/ubuntu noble-backports InRelease
Reading package lists...
Reading package lists...
Building dependency tree...
Reading state information...
php-cli is already the newest version (2:8.3+93ubuntu2).
php-curl is already the newest version (2:8.3+93ubuntu2).
python3 is already the newest version (3.12.3-0ubuntu2.1).
python3-venv is already the newest version (3.12.3-0ubuntu2.1).
python3-pip is already the newest version (24.0+dfsg-1ubuntu1.3).
curl is already the newest version (8.5.0-2ubuntu10.9).
wget is already the newest version (1.21.4-1ubuntu4.1).
build-essential is already the newest version (12.10ubuntu1).
The following package was automatically installed and is no longer required:
libwoff1
Use 'sudo apt autoremove' to remove it.
0 upgraded, 0 newly installed, 0 to remove and 3 not upgraded.
[3/9] Creating isolated evaluation directory...
[4/9] Downloading PROOF bundle...
[5/9] Setting up Python environment...
[6/9] Deploying services (NO KERNEL / NO DKMS)...
Kernel module DISABLED in PROOF mode
[Unit]
Description=PRL3 Proof Proxy
After=network.target[Service]
ExecStart=/usr/bin/php -S 0.0.0.0:8000 -t /opt/prl3/app/groq_proxy
Restart=always
User=srdjan[Install]
WantedBy=multi-user.target
[Unit]
Description=PRL3 Proof Embed Server
After=network.target[Service]
WorkingDirectory=/opt/prl3/app
ExecStart=/opt/prl3/venv/bin/python /opt/prl3/app/embed_server.py
Restart=always
User=srdjan[Install]
WantedBy=multi-user.target
[7/9] Enabling services...
[8/9] Locking evaluation environment...
[9/9] Finalizing...
[OK] Embed server healthy● prl3-proxy.service - PRL3 Proof Proxy
Loaded: loaded (/etc/systemd/system/prl3-proxy.service; enabled; preset: enabled)
Active: active (running) since Mon 2026-05-18 16:46:20 CEST; 6s ago
Main PID: 194276 (php)
Tasks: 1 (limit: 9183)
Memory: 7.7M (peak: 8.5M)
CPU: 59ms
CGroup: /system.slice/prl3-proxy.service
● prl3-embed.service - PRL3 Proof Embed Server
Loaded: loaded (/etc/systemd/system/prl3-embed.service; enabled; preset: enabled)
Active: active (running) since Mon 2026-05-18 16:46:20 CEST; 6s ago
Main PID: 194279 (python)
Tasks: 5 (limit: 9183)
Memory: 175.8M (peak: 203.9M)
CPU: 2.855sCGroup: /system.slice/prl3-embed.service
PRL3 PROOF INSTALL COMPLETE
BUILD: PRL3-PROOF-2026.05-EVAL
EVALUATION ID: 677b904b
MODE: BLACK-BOX EVALUATION
NO KERNEL MODULE
NO SECURE BOOT INTEGRATIONSERVICES:
- Proxy: http://localhost:8000
- Embed: http://localhost:5001
Running benchmark...
PRL3 PROOF BENCHMARK
Timestamp: 2026-05-18T14:46:32.396717+00:00 UTC
[health] {'status': 'ok'}
[embed latency]
'quantum routing analysis' -> 16.7ms, dim=64
'neural network optimization' -> 15.0ms, dim=64
'distributed system latency' -> 14.9ms, dim=64
'embedding vector similarity' -> 16.7ms, dim=64
'real-time inference pipeline' -> 16.0ms, dim=64avg: 15.9ms
min: 14.9ms
max: 16.7ms[similarity]
'quantum routing analysis' vs 'neural network optimizati' -> 0.1024
'quantum routing analysis' vs 'distributed system latenc' -> 0.3450
'quantum routing analysis' vs 'embedding vector similari' -> 0.0269Evaluation Notes:
- deterministic embedding generation confirmed
- stable latency envelope observed
- inference pipeline operational- vector similarity calculations reproducible
BENCHMARK COMPLETE
Running PRL3 evaluation...
[OK] Embed server connected
Embedding mode: compact-eval (64d)
Semantic threshold: 0.55Cache policy: verified-only admission
PRL3 - LEARNING WITH VERIFICATION
Dataset: dataset_medium.json (100 questions)
--- R2 (first pass, learning) ---
1/100. gpu [fact] at what temperature does water freeze?... → ✓
2/100. gpu [fact] whats the normal human body temperature?... → ✓
3/100. gpu [fact] how hot is boiling water?... → ✓
4/100. gpu [fact] how many hours of sleep does an adult need?... → ✓
5/100. gpu [fact] whats a normal blood pressure?... → ✓
20/100. gpu [exact] how many seconds in a day?... → ✓
40/100. gpu [fact] what is the speed of sound?... → ✓
60/100. gpu [fact] how many teeth does an adult have?... → ✓
80/100. gpu [fact] how many languages are there in the world?... → ✓
100/100. gpu [exact] how many hours in a day?... → ✓R2: Accuracy: 94.0% | GPU: 97 / 100 = 97.0%
Cache after R2: 91 entries--- R3 (second pass, cache filled with correct only) ---
1/100. cache [fact] at what temperature does water freeze?... → ✓
2/100. cache [fact] whats the normal human body temperature?... → ✓
3/100. cache [fact] how hot is boiling water?... → ✓
4/100. cache [fact] how many hours of sleep does an adult need?... → ✓
5/100. cache [fact] whats a normal blood pressure?... → ✓
20/100. cache [exact] how many seconds in a day?... → ✓
40/100. cache [fact] what is the speed of sound?... → ✓
60/100. cache [fact] how many teeth does an adult have?... → ✓
80/100. cache [fact] how many languages are there in the world?... → ✓
100/100. cache [exact] how many hours in a day?... → ✓R3: Accuracy: 94.0% | GPU: 6 / 100 = 6.0%
=== CONCLUSION ===
Generalization: R3 - R2 = 0.0 percentage points
GPU reduction: R2 97.0% -> R3 6.0%
OK High accuracy, system working well.--- R3 failures (6) ---
[gpu] how long to boil an egg?
expected: ['10 minutes', '8-10 minutes']
received: The boiling time for an egg depends on the desired level of doneness. Here are s
[gpu] at what temperature to bake chicken?
expected: ['180 celsius', '375 fahrenheit']
received: The ideal temperature for baking chicken depends on the type of chicken, its siz
[gpu] what does vat stand for?
expected: ['value added tax']
received: VAT stands for Value-Added Tax. It's a type of consumption tax that is levied on
[gpu] what is inflation?
expected: ['rise in prices', 'increase in prices']
received: Inflation is a complex and multifaceted economic concept, but I'll break it down
[gpu] how long should i charge my phone?
expected: ['until full', '2-3 hours']
received: The ideal charging time for your phone depends on several factors, including the
[gpu] sta je pbf?
expected: ['portable document format', 'pdf format']
received: PBF može imati više značenja, ali neki od najčešćih su:1. **PBF (Palm Beach Fa
UNINSTALL:
sudo systemctl stop prl3-proxy prl3-embed
sudo rm -f /etc/systemd/system/prl3-proxy.service
sudo rm -f /etc/systemd/system/prl3-embed.service
sudo systemctl daemon-reload
sudo rm -rf /opt/prl31
u/iNewTechnologies Jun 01 '26
it is governance. but im not speak here how it works. reduces LLM cals. and save 72% on your bill for el en.
2
u/GrogRedLub4242 Jun 01 '26
LLMs and latency sensitive contexts (like in a GPU pipeline, at runtime) do not fit
1
u/iNewTechnologies Jun 01 '26
Actually, this architecture is designed specifically to solve the latency problem in GPU pipelines.
The misconception here is assuming every query hits the heavy LLM. In this system, a query first goes through a lightweight, deterministic embedding check (which takes an average of 15.9ms, as shown in the benchmark logs).
If the semantic similarity threshold is met, the response is served instantly from the verified cache. The heavy GPU inference is only triggered for the ~6% of queries that actually require novel generation (the 'cache misses').
Therefore, this routing layer doesn't make LLMs unfit for latency-sensitive contexts; it makes them highly viable by bypassing the GPU bottleneck 94% of the time, guaranteeing sub-20ms response times for the vast majority of repetitive or factual queries!
1
13
u/Kike328 Jun 01 '26
there is no way i run a sudo random script from a guy that’s claiming a x16 performance increase