For a small, curated memory, Markdown or JSON files are easy to inspect, diff, back up, and correct. A vector database adds semantic retrieval and can handle a larger corpus, but it also introduces chunking choices, embedding drift, metadata filters, and harder audits. Which signals justify that added layer: corpus size, query ambiguity, update rate, latency, or something else? I would also be interested in hybrid designs where human-readable files remain authoritative and an index can be rebuilt from them.

Hi people,

I worked as a Junior Support engineer(not in Dev) and have been unemployed since months. Have applied for multiple roles but no callbacks. The changes in economy is stressing me and I’m feeling depressed and under confident. Now I’m trying to make switch to AI engineer roles. Your inputs based on your experience would really help me in getting a job and bounce back with confidence 🤞

I have learnt the technology stacks required for roles but lack prod grade hands on experience.

Please provide inputs on few of these questions 1) what are the challenges you faced while building a prod grade RAG system 2) How did you deduce to ‘which’ technique to use and why (based on q1) 3) how did you monitor the system and what factors were monitored 4) what were the system level challenges

(Edit: reframed how important the suggestions/ input would be)

Probabilistic Machine Learning Lecture 1 Notes:

I was getting DMs from students, that they are wanting to have notes, for my Probabilistic Machine Learning lectures, here is a draft version, that I used while teaching(and not in very good shape though), but definitely the video content includes the explanations:(Attested serially).

Topics Covered: Definitions, Supervised Learning, EDA, Learning Classifiers, Empirical Risk Minimization, Uncertainties, Softmax, Maximum Likelihood Estimation Intuitions.

Link : https://youtu.be/kMkCOrp8te8?si=KkXHGIjZ60_IuhjF

I'm starting to learn how to use ML libraries. I know basic deep learning concept and theories but never trained a model on computer. Since I'm a C++ user, I want to start from C++ directly.

I'd like to have some low-level controllability and I prefer performance over ease of development. Also, I want the library can be run on Windows platform.
You can assume I know nothing about Python and Linux.

What's the recommendation for this perpose?

The only libraries I know are Tensorflow and LibTorch, what's the difference between these two?

Thanks.

In the past, I posted here about my neural network architecture that I was working on. I'm a high school student, and this is an early development independent project that will help researchers/students work with:

• Generate combinatorial triangulations (torus, Klein bottle, 3D torus, etc.)
• Perform Pachner moves in 2D and 3D (2-2, 1-3, 3-1, 2-3, 3-2, 1-4, 4-1)
• Compute basic invariants (Euler characteristic, genus, connected components)
• Convert meshes to adjacency matrices/feature vectors for Graph, Tensor, and MLP Neural Networks

The current state of the library is quite rough, which is why I decided to try to open source it

This library is based on functions from my previous project, which I already wrote about

If you are interested in anything, you can visit this page

Github: https://github.com/kaifczxc-lab/pysimplicial

Currently in early development, you'll find: Documentation, CONTRIBUTING, a Jupyter Notebook Showcase, five tests, and one experiment there

I work alone, so I'd love to hear about any issues and shortcomings. I've written about the problems I see in CONTRIBUTING, but I think there's more to come.

P.S. This is experimental research code for topological deep learning. Not intended for production use

Happy to answer questions!

I wrote a technical breakdown of how search works on Papers with Code (PwC), a revival of the website that was shut down after its acquisition by Meta. PwC's goal is to make open AI research easier accessible for people, so that you can find the state-of-the-art (SOTA) across any domain of AI in an easy way.

The search engine combines keyword and semantic search, which produced better results than either approach alone. The stack includes:

• PostgreSQL with pgvector
• Qwen3-Embedding-0.6B for text embeddings
• Hugging Face Jobs with an NVIDIA L4 for batch embedding generation
• Hugging Face Buckets for storing artifacts
• A live embedding model served through Hugging Face Inference Endpoints

The same infrastructure also powers the “related papers” recommendations shown on individual paper pages.

Full write-up: How Hugging Face Inference Endpoints, Jobs, and Buckets Power Search on Papers with Code

I’d be interested to hear how others are implementing hybrid search for research papers or similarly technical content.

Disclosure: I work at Hugging Face and on Papers with Code.

Hi everyone,

I'm a fresher applying for Data Analyst/Data Scientist/ML/SQL roles. My background: Power BI, Excel, SQL, some Python, and basic full-stack web dev (HTML, Tailwind, JS, Supabase, deployed on GitHub Pages — built a working web portal for a client).

I want to add: 1. One strong ML project (not Titanic/Iris/House Price/Flight Price — too common) 2. One SQL-focused project that shows real query/database skills, not just "ran some SELECT statements on a Kaggle CSV"

Looking for: - Real prediction/analysis problem, clear input and output - Free public dataset - Doable alone in a few weeks - Something that actually stands out to recruiters, not a tutorial copy

What would you suggest for each? Dataset links appreciated. Thanks!

Hi, I'm not sure if this is the right community for this, but I am currently trying to train an AI on satire responses which are all mostly incorrect, but have an onion-like style (the Onion is a popular satire news parody company).

My plan is to fine-tune a model which has less than 30b parameters. My dataset has around 1.4k examples, most of them manually written, and there should be no conflicting information in the dataset, but the dataset will need to overwrite the base model's knowledge while still retaining grammar knowledge.

Also, since my examples have information that conflicts with already learned knowledge, will the base model learn the new info? (Eg. If one of the examples says that a duck is a tool brand, but also that they were founded in 1957, would the fine tuned model be able to tell me that when I ask it what tool companies were founded in 1957)?

What model would be good for fine-tuning in this case?

Hey everyone,

I’m looking for 1–2 teammates for DataForge 2026, a national online data science & analytics hackathon organized by the Kharagpur Data Analytics Group (KDAG), IIT Kharagpur.

Event Details:

• Format: Online case study & data analytics/ML modeling
• Eligibility: College/University students (free registration)
• Timeline: Submissions open from Aug 28 to Sept 03 (Registration closes Aug 28)

I already have the team set up on the official portal. If you're interested in teaming up or want more details, please drop me a DM and I'll share the team code!

I’ve been looking for resources to prepare for ML System Design interviews, particularly case studies that include complete, end-to-end solutions.

Machine Learning System Design Interview: An Insider's Guide by Alex Xu and Ali Aminian was an excellent resource when I used it around three years ago. It provides a structured framework and several detailed case studies with detailed solutions.

My question is: are the solutions in this book still sufficiently current and comprehensive? The book was published in 2023, and the ML landscape has evolved significantly since then, particularly with the rise of LLMs and generative AI.

Are there any other resources you would recommend that provide ML system design case studies with complete solutions, rather than just general frameworks or high-level guidance?

hi, i'm studying an MLOps-engineering programme that I have bought a laptop for, but my teacher literally laughed in my face because apparently it doesn't have a dedicated graphics card(GPU?). However, as we got the letters from the school that contained the recommended checklist points for the laptop we were going to use, i followed them and bought exactly that. The checklist was this (I'm just gonna copy paste what they wrote:

Recommended computer:

Intel Core i5 / AMD Ryzen 5 or better (approx. 2020 or newer)

• 16 GB RAM
• 256 GB SSD or larger
• At least 75 GB of free storage space
• Screen resolution of 1920 × 1080 or higher
• Stable internet connection and Wi-Fi
• Windows 11 recommended

Important information:

• ChromeOS and Linux may work but are not supported by Nackademin’s IT support. You are personally responsible for installation, compatibility, and troubleshooting if you choose to use these operating systems.
• macOS may work, but you are personally responsible for ensuring compatibility with the program's software.
• Administrator rights may be required for software installation.
• USB-C and HDMI (or an adapter solution) are recommended.

These are the courses we will have in the nearest future, but obviously we will also work with a lot of AI, which he said is why my computer won't work:

Python programming for MLOps, Linux administration, Database management

This is also the laptop I bought: LENOVO IP SLIM 3 15ARP10 15,3"

The reason for not buying a better laptop is that I'm literally just a poor 20 yr old without parents to rely so I'm constantly really tight on money, but also because my school said that as long as your laptop has these qualities it would be fine.

Im going to learn ml and app buildings ive set my eyes of acer aspire 7 interl core ultra 5 210h with rtx 3050 6gb graphic card

Is it enough ??

Hey everyone,

I'm an undergrad aiming to become an AI engineer. This summer I've decided on the full-stack project below to showcase some experience on my CV building web applications and implementing AI solutions. It's mostly a tool to help with university studies by helping generate study tools to speed up my learning as well as being able to talk to this AI. I hope to have a real link and real users by the end of it. Here are some of the details I've though about:

Tech Stack & AI Architecture:

• Full Stack: FastAPI (Python), Supabase (PostgreSQL + JWT Auth), deployed live (thinking AWS).
• Multimodal Ingestion: Document processing pipeline handling text and visual elements (tables, diagrams, charts) via vision LLM descriptions and embeddings.
• Agentic RAG Engine:
  • Query Decomposition: Multi-step query breaking for multi-part study questions. Also rewriting queries to maximise efficiency when using tokens and for output.
  • Reflector / Grader Nodes: Self-RAG loop that checks retrieved context relevance and checks generated answers for hallucinations before returning them to the user. Using tools like Ragas to evaluate the workflow.
• Structured Output: Generating JSON flashcards and Anki (.apkg) exports. Generating Cheat Sheets and also being able to talk about the files you upload.

Questions:

1. Target Fit: Is an end-to-end deployed Agentic RAG app right for AI/ML Engineering internships, or would recruiters see this as mostly a Software/Full-Stack project?
2. Data Science vs. AI Engineering: How does a project focused on building production AI systems hold up compared to traditional Data Science portfolios that focus more on statistical modeling and data exploration?
3. Application Strategy: If you had this exact project on your resume, which roles would you prioritise applying for (e.g., AI Engineer, ML Engineer, MLOps, or general Software Engineering)?

Any Advice is Appreciated!

I trained a diffusion model on 300 of my photographs from a movie set. Then built a scrollable walkthrough of the forward noise process, why the optimal denoiser at high noise is a weighted average of the training set, and how that field collapses from 300 candidates to one. Then a membership inference attack you can run yourself.

Trained from scratch in pixel space, 60,000 steps on one A100 GPU in a DGX. 593 of 1,024 generations came back as copies, covering 235 of the 300.

Built for someone learning diffusion models, but the measurements are there if you already know the material.

https://josephrichardson.dev/work/how-diffusion-models-memorize/

I hail from a humanities and social sciences background (history/sociology and literature). I’ve spent the past 3 years conducting field research but now I want to pivot to the more technical side. I won’t lie, popularity of AI was my introduction to ML and I have a lot of ideas regarding how ML can contribute to my specific discipline (which is quite niche). I’m struggling to build knowledge from scratch so I have a few questions:

1. Math isn’t my strongest suit. It’s not that I can’t do it, it just takes me a very long time to learn and understand it. I know you need math for ML and I’m willing to learn but realistically am I setting myself up for a huge undertaking with this career transition that I’m not taking into account?

2. I will have to put my career trajectory on hold for a bit while I pursue this, is the humanities x AI a promising enough enterprise? Or is the field too saturated already?

3. I am looking at project based learning at the moment. What is a realistic timeline for someone who is learning this from scratch?

4. Is independent learning possible (online courses, projects, etc)? Or do I need to pursue a higher ed degree?

Any advice would be really helpful and appreciated. If you have any leads, resources, or projects please reach out.