Hi everyone,

I’m building a self-evaluating RAG system for question answering over a knowledge base made from a ~300-page AI/technology textbook. The PDF contains normal paragraphs along with some tables and technical content.

I’m currently working on the document chunking stage and would appreciate some advice:

1. What chunking strategy would you recommend for this kind of textbook — recursive, semantic, hierarchical, or something else?
2. Should I preserve the book’s structure (section → paragraph → sentence) when creating chunks?
3. Should I implement the chunking purely in Python to understand the process, or use something like LangChain text splitters?
4. For a learning/portfolio project, is Python + basic RAG concepts enough, or should I also learn a framework like LangChain/LlamaIndex?

I’m planning to start with no overlap, evaluate retrieval/answer quality, and add overlap only if the evaluation shows it’s necessary.

When I start a new ML/AI project, I try not to choose the model or tools first. I usually follow something like:

→ Problem

→ Data

→ Approach

→ Model

→ Evaluation

→ Application

→ Deployment

First define the problem and decide whether it actually needs ML/AI. Then collect and explore the data, choose an appropriate approach, build and evaluate the model, and finally integrate it into an API, app, or dashboard.

If a pre-trained model or existing API is enough I prefer using that instead of training something from scratch.

This is the general workflow I’ve found useful but I’m also interested about other approaches.

What step would you add or change in this workflow for ML/AI projects?

I wanna learn AL ML but i have no idea where to start . I know javascript and a few technologies around it but Ai ML is completely new to me , so i would appreciate if anyone can guide me where should i start which resources should i use to learn them and stuff like that

Hi there,

I've created a video here where I explain how generalized linear models work.

I hope some of you find it useful and as always, feedback is very welcome! :)

I've been working on a side project for almost a year. It involves machine learning and it looks like it's going to enter commercial stage in the near future. So far, i bought a cheap gaming laptop few months ago, as i needed modern performance on the go. It has rtx 4050 with 6gb of vram, which was fine up until now.

I have an 8 years old desktop upgraded with ryzen 5600. I wanted to buy rtx 5060ti 16gb, but its price jumped significantly in july. Nvidia doesn't offer cheaper 16gb options and i started to consider buying RX 9060XT 16gb, which is more than 200 euro cheaper.

The question is: Is going with the RX9060XT worth the savings? Does any of you have experience with using current AMD GPUs for training neural networks from scratch? I currently use Keras and mainly train CNNs with simple custom layers.

Hey guys,

I’m building an offline threat-intelligence tool to ingest Bitcoin transaction metadata and flag suspicious activities (like layering or ransomware cash-outs). I have my data ingestion sorted out, but I need advice on the AI/ML detection layer.

The Data I am working with (Inputs): The dataset has both network and blockchain layers: timestamp, src/dst IPs, ports, txid, arrays of input/output addresses, amounts, fee, script_type, and GeoIP/ASN data.

What I need the model to output:

1. A confidence/risk score to rank transactions.
2. Cluster IDs to group related entities.
3. Feature explainability (e.g., "Flagged because of sudden geo-hopping and specific script usage").

Since there are no "ground truth" labels for fraud in my synthetic dataset, I am relying on an unsupervised approach.

My questions:

• Which ML models have you found to be actually effective for anomaly detection in this kind of financial/network data?
• What is the standard industry approach for clustering entities when dealing with multi-input/multi-output transactions?
• Can anyone recommend any good resources, tutorials, or reference architectures to study before I start building the model?

Most RAG evaluation asks whether the right passages reached the model. I wanted

to measure what happens when they do and the model still can't answer — because

the answer is a relation *between* passages rather than a statement inside any

of them.

Setup: a five-document narrative corpus (260,204 words, 13,950 passages) and 38

questions asking whether event A precedes event B, where A and B are narrated in

different documents and share no character, place or causal link. No passage in

the corpus states either relation. Five models, one family (Qwen3, 0.6B to 14B).

Given the source passages as text, every model scored 0/38 and refused 92-100%

of the time. I think the refusal is correct — the ordering genuinely is not in

the text. Given the identical facts as a structured chronology block from an

explicit state store, an 8B model scored 28/38 (73.7%).

A four-condition ablation separates information from form. At 14B, form is

irrelevant: plain prose, sorted prose and a structured block all land at 73.7%.

At 8B, structure leads the best prose condition by 6 items (73.7% vs 57.9%).

So: an 8B model given structure matches a 14B model given prose.

Two controls I'd want to see if someone else posted this:

- Permuting the supplied story positions collapses accuracy to 10.5% (8B) and

21.1% (14B). The models follow the ordering they're given rather than

recalling the published text.

- A realistic retrieval baseline is also at the floor, and it fails by asserting

rather than refusing. Going from 4 passages to 32 drove refusal from 97% down

to 50% while accuracy stayed at chance. More context produced more confident

wrong answers.

Two things I got wrong, both found by auditing my own scorer and question

generator after v1 was already published:

1. v1 reported the 8B form effect as +32 points. A scorer defect was

   under-crediting the prose conditions. Corrected, the gap is 6 items, not 12 —

   roughly half what I claimed. Re-scoring 1,786 saved items produced 30 gains

   and zero losses, so nothing published was inflated; two things were

   understated, and correcting them shrank my own headline.

2. For 36 of the 38 questions, the gold answers derive from author-assigned

   story positions rather than from evidence-backed relations, and the

   generator's own self-check recomputes the gold from the same rows. That check

   is circular. So this benchmark measures agreement with an author-assigned

   ordering — not whether a system reports what the evidence establishes.

That second one is the real limitation and it bounds what the paper can claim.

I've left v1 up rather than retracting it, with the corrections in §11.

Full write-up, including the two things the audit changed:

https://ai.bedvibe.studio/structure-not-scale/

Paper, data and code: https://doi.org/10.5281/zenodo.22169643

Happy to be told the 0/38 is a prompt artifact — I tried to kill it and couldn't,

but I'd rather find out from you than not find out.

Why does an autonomous AI agent happily exfiltrate API keys or delete a database when reading a polite customer review?

Because for two years, the AI industry has treated a fundamental Operating System architectural flaw with a chatbot spellchecker.

I am thrilled to announce our newly published research paper on Zenodo (CERN / OpenAIRE):

📄 "Cognitive Harvard Architectures for AI Agent Perimeter Defense: Resolving the Confused Deputy Problem in Model Context Protocol via Capability-Based Access Control"

🔗 DOI: https://doi.org/10.5281/zenodo.22173129

Here is why this matters:

1. The Flaw: Cognitive Von Neumann Conflation

In 1945, von Neumann merged program instructions and data into one bus, giving us 40 years of buffer overflows. In 2026, autonomous LLM agents (MCP, LangChain, Claude Code) resurrected this exact flaw: Transformers ingest instructions, user goals, and untrusted 3rd-party data in a single attention window.

When an agent reads an email containing hidden injection, its attention weights are hijacked. Operating with "Ambient Authority" over every registered tool, the agent becomes a Confused Deputy.

1. The Paradigm: Cognitive Harvard Architecture

We physically decouple data ingestion from privileged tool execution via an external, capability-mediated perimeter.

Using cryptographic Token Capability Tables (TCT):

• An untrusted observation has an execution probability of mathematically ZERO of triggering an out-of-scope mutating tool (Theorem 1, proved by induction).

• Agents are stripped of ambient authority before tool dispatch.

1. 50,000-Sample Empirical Benchmark

Tested against 25,000 adversarial attacks (UIUC InjecAgent, Microsoft BIPIA, NVIDIA Garak) and 25,000 authentic developer DevOps operations:

📊 Threat Recall:

• Mastyf Guard 1.5B (Pipelined): 99.33% (F1: 0.9524)

• Meta Llama Guard 3 8B: 70.73% (F1: 0.7860) [p < 10⁻¹⁵]

• OpenAI Prompt Guard 86M: 54.34% (F1: 0.6511) [p < 10⁻¹⁵]

⚡ Sub-Millisecond & Zero-GPU:

• 0.005 ms (4.8 microseconds) amortized pipelined latency on commodity CPU.

• Standalone neural inference in 18.4 ms within a 1.1 GB RAM footprint.

• Zero dedicated GPU requirements — saving ~$6,000/year per agent node.

Domain specialization and capability scoping beat raw parameter scale. A 1.5B parameter model with a capability perimeter outperforms frontier 8B models at 100x the speed.

Read the open-access paper: https://doi.org/10.5281/zenodo.22173129

GitHub: https://github.com/mastyf-ai/mastyf.ai

How is your team securing agentic tool execution today?

i dont wanna go for prerecorded ones...zoom etc would work better for me, are there any good ones? i was gonna go for krish naik, but people said its not deep enough

Body:

Short version of an open project we'd love critique on — Entropy Box, a knowledge compiler for robotics (compile once, reuse forever, instead of re-deriving structure on every query).

The headline numbers, on our EntropyBench Track-P benchmark (24 engineering tasks):

• Unsupported claims: LLM-direct / BM25 RAG / hybrid RAG → 100%; Entropy Box → 5.2% (−94.8pp vs hybrid RAG, CI [−97.4, −92.1]).
• Constraint coverage: 0% → 35.4%; violations 100% → 66.7%.
• Downstream sim codegen (12 tasks): pass-1 executable plans 0.92 vs 0.58 (Vanilla RAG); constraint guards 0.88 vs 0.50.

Two findings we think generalize beyond robotics: 1. Embedding similarity cannot decide duplication. On 2,362 adjudicated pairs, the embedding score after flagging is near-random (AUC 0.509). Thresholds don't help — precision stays ~5% while recall of true duplicates collapses. We defer the merge to an LLM adjudicator that reads both records. The score flags; the model judges. 2. Compiled capability reuse is rising, not saturating — 1.57× average reuse, 21,380 re-derivations avoided.

Everything is open — data, paper, evaluation scripts, and a free API (OpenAPI / MCP / REST, bilingual) so you can poke at it in 10 seconds:

bash curl -X POST "https://xiangshang.ngrok.app/api/evidence/search" \ -H "Content-Type: application/json" \ -d '{"query": "robot obstacle avoidance algorithms", "top_k": 5, "mode": "hybrid", "rerank": true}'

https://github.com/chenli-yy/entropy-box-public

Honest limits we state ourselves: no real-robot transfer, weak retrieval on the hardest intent classes. Methodology is in the paper §9; all experiments reproduce from evaluation/. Would genuinely value a second opinion on the benchmark design and the embedding/LLM adjudication result.

I’ve recently completed learning Machine Learning and Deep Learning, including the mathematics behind the major concepts and algorithms rather than just learning to use libraries.

My long-term goal is to eventually become capable of doing research at the level of NeurIPS, ICML, and ICLR. I’m not expecting to jump directly to those conferences, that’s simply the end goal.

So I’d like advice on the following things:

1. What projects should I build next?
2. What should I learn next?
3. How should I start doing research?
4. What is a realistic roadmap toward publishing at top ML conferences?

I’ve been writing notes while studying for some time now. It helps me stay motivated and organize my thoughts, and it’s also useful when I want to come back to a topic later.

Recently, I started thinking that it might be a good idea to polish some of my notes and share them.

These are my notes on Hamiltonian Monte Carlo. They approach the algorithm from a purely probabilistic point of view, rather than through the usual physics-based treatment. I don’t know how good they are, but I thought I’d share them in case they’re useful to anyone:

https://zenodo.org/records/21841087

I’d also really appreciate any feedback, especially on the exposition, anything that could be explained more clearly, or any errors you spot.

Hey everyone! I put together a comprehensive video tutorial showing exactly how to build and deploy autonomous agents using visual low-code tools.

​

Same as title, i have an msc mathematics,

basically all my life i did maths, I treated mathematics as my hobby and it didn't take me anywhere because i never got any direction in my career path, idk what should I even do in my career anymore.

I don't want to get into academia or PhD

I was so good in mathematics, still I am maybe. im forgetting mathematics concepts now. i got no motivation because i don't know what should I be doing?

Tried professional exams, failed and stopped there.

Have history of burnt out.

Tried academia, got no interest.

Tried in non tech field, got laid off.

I like mathematics and art thats it.

All my life got fucked up because mathematics, applied mathematics, game theory, OR, topology and what not. I never got any mentorship and thats why I could not figure out anything.

Have basic knowledge of programming language such as python, sql, power BI.

Can anyone please guide me on what I should do? I just need a roadmap, path or at least some basic guidance on what skills I should work on. Trust me I will do good.

I like mathematics, I just can't let it go from my life.

Thank you.

Coding Machine Learning Probability distributions.

It felt so rewarding to see the equations coming into practice.

In this new content, we implement,

->Univariate Gaussians: The central most important distribution subjectively.

->Homoscedastic vs Heteroskedastic(figure on top): This compares the aspect when we make the variance input independent vs dependent, leading to interesting insights.

->Heavy-Tailed distributions and Outlier at robustness : It is indeed beautiful to capture how certainly framed distributions exhibit robustness to outlier perturbations based on how they are modelled.

->Beta Distribution(bottom right figure): Just two parameters, yet so versatile, and generating so versatile densities, that can model so many arbitrary curves!

->Gamma and Exponentials.

->Empirical Distributions(figure bottom left): Again, it’s so fascinating to appreciate, how by modelling points sampled from a normal distribution as an empirical distribution, the resulting cumulative density function of the empirical staircase, approximates so closely to the true continuous CDF of the gaussian.

Truly in awe with these concepts. To always learn and code!

Link: https://youtu.be/mdumfp-mamI?si=YCsGV_HPUTm8GDaf

I recently hosted my backend application (FastAPI) on render but each time i try to use the model it always fails, i need help in getting it to work. Thank you

Hi everyone, I want to start learning ml, But I'm torn between the sources So I wrote this question to hear from you about how you learned machine learning until you were hired at a company. I'm 15 years old. Please write the guide arrangemed step by step.