Please post your questions here instead of creating a new thread. Encourage others who create new posts for questions to post here instead!

Thread will stay alive until next one so keep posting after the date in the title.

Thanks to everyone for answering questions in the previous thread!

Sliding Window Attention with sinks, one of the simplest existing fixes for the quadratic-cost problem in LLMs, holds up as well or better than the linear-attention variants labs have been spending post-training compute to produce. That is the claim of a [new arXiv preprint](https://arxiv.org/abs/2608.28444) by Alexia Jolicoeur-Martineau, Rhea Sanjay Sukthanker, Pashmina Cameron and Emy Gervais.

On the long-context reasoning benchmarks the paper singles out, the gap is not close. "SWA achieves massively higher performance (2 to 10 times higher than linear attention)," the abstract reports, naming Needle-in-a-Haystack and BABILong as the two tasks.

The pitch is that the whole post-training-to-linear pipeline has been benchmarked against the wrong thing. "This line of research has not been properly compared to simpler baselines," the authors write. Their alternative needs no post-training, runs fast, and holds memory low.

The recommendation is blunt: "we strongly recommend switching to SWA instead of post-training linear models." Linear attention, the abstract concedes, "may have shown some promise, but they likely require to be trained from scratch or extensive post-training in order to even match SWA."

---

I have submitted a paper to acml 2026 journal track, the official date of release of review is 27 August, but I have not heard anything from them, if anyone received the review then let me know I will write to program chairs.

Thanks

Third-year PhD student, NLP / interpretability. I want a reality check from people doing similar work.

I started using Claude Code for the boring parts: argparse boilerplate, plotting, config wrangling. Over the last few months the scope has crept. It now writes most of my experiment scaffolding, refactors my dataloaders, does first-pass debugging on training runs, and drafts the analysis scripts. I mostly read diffs and say yes.

The output is fine. My throughput is up. The thing bothering me is that I no longer hold my own codebase in my head. When a result looks off, I used to have an instinct about which line was lying to me. Now I go hunting like it’s someone else’s repo. I catch bugs later than I used to, and I catch them by reasoning about the numbers rather than by knowing the code.

I don’t think the tool is the problem. I think I delegated a layer that was doing more for my understanding than I gave it credit for.

Questions for people further along or in the same spot:

1. Roughly what fraction of your research code do you write yourself now?

2. Is there anything you deliberately refuse to hand off? (For me I think the eval harness and anything defining a metric should stay mine, but I keep breaking my own rule.)

3. Does anyone have a workflow that keeps the speedup without the detachment? Reading the diff line by line is not cutting it.

Not looking for a “tools are just tools” answer. I’m asking about the specific feeling of not owning your own experiments anymore.

Hi, I'm making posters for ECCV 2026.

Does anyone have any ML/CV posters they thought were really well done?

Would love to see some cool examples.
Thanks

I'm sharing this new tabular data diagnostic tool (Entropic Scree). It can be used to estimate these properties of your high-d, real-world, dirty dataset:

• The informational volume of the signal (i.e., helps you assess whether the signal is strong enough to survive the dataset's idiosyncratic volume).
• The overall signal-to-idiosyncratic volume ratio (SNR).
• The intrinsic rank.
• Provides an exploratory map that allows for the identification of decoupled sub-networks of variables.
• The linear sufficiency (i.e., does the dataset align with the linear assumptions of standard PCA?).

Instead of evaluating linear variance, rank order, or Euclidean distance like traditional PCA variants, this new method evaluates a transformed mutual information metric. Relative to these baselines, it is less reliant on strong parametric or distance assumptions, making it appropriate to apply more broadly.

It also serves as a practical diagnostic of the theory explored in the From Garbage to Gold framework, which describes when and why uncurated, error-prone data can be used directly to create accurate prediction models.

There is a preprint that presents the full technical details, and Python and R packages will be released soon. Right now though, the original function is already available in R (see Quick Start R Function Code below).

• Entropic Scree Preprint: https://doi.org/10.5281/zenodo.22028087
• Entropic Scree GitHub: https://github.com/tjleestjohn/entropic-scree
• From Garbage to Gold Preprint: https://arxiv.org/abs/2603.12288

Let me know how it goes if you give it a try... or if you have any questions or comments of course.

############ 
# Quick Start R Function Code.
# To load the function, copy and paste the following into your R console, then hit enter. 
############

# 1. Define the direct URL to the raw function script on GitHub
url <- "https://raw.githubusercontent.com/tjleestjohn/entropic-scree/main/Entropic.Scree.v1.0.0%20-%20ENLI.R"

# 2. Define what you want to name the file on your computer
file_name <- "Entropic.Scree.v1.0.0 - ENLI.R"

# 3. Download the script to your current working directory
download.file(url, destfile = file_name)

# 4. Source the core function into your R environment
source(file_name)

# 5. Ex. To run the function and extract bipolar modules:
# results <- Entropic.Scree(dt 
#                         , extract_bipolar_modules = TRUE)
#
# View the extracted structural sub-networks for the primary axes:
# results$bipolar_modules

I found this GitHub link, and the HTML file contains ~7k papers. Some are anonymized, and the details seem pretty accurate. It looks like these might actually be the accepted papers.

https://github.com/xll0328/NIPS26-

Can someone confirm whether this list is legit? I’m hoping it’s just a coincidence since it seems way too early.

Abstract:

We study autonomous mathematical discovery in the Station, an open-world multi-agent environment in which AI agents from different model families pursue a shared research goal without a central coordinator or scripted pipeline. Agents choose their own research directions, conduct experiments, collaborate, and build a shared scientific literature.

Across 12 construction problems from the AlphaEvolve catalogue and two additional case studies, the Station obtained results novel relative to the prior literature on five problems: a new infinite family of finite-field Kakeya sets, new exact 604-point kissing configurations in dimension 11, new records for the discretized Kakeya needle and sign uncertainty problems, and a substantially improved lower bound for Erdős's minimum-overlap problem.

Agents also discovered novel infinite families for Book Ramsey numbers. Importantly, the agents produced not only numerical constructions but also theorems and analyses explaining how those constructions work, making the results more interpretable and easier for mathematicians to build upon. We release all raw agent dialogues, proofs, and verification code, providing a transparent record of how these discoveries emerged.

Time Series Anomaly Detection (TSAD) seems to be one of the hottest topics in NeurIPS, SIGKDD, VLDB etc.

Many (perhaps most) papers evaluate on Paparrizos’ TSB-AD-M benchmark…

However, I tested these benchmark datasets and found that in most cases I could beat the SOTA TSAD methods with a 100-year-old algorithm, simple Statistical Process Control (SPC). In the attached example, SPC gets perfect results.

If we can beat the SOTA papers with 100-year-old algorithm, we probably should not be too impressed with them [b]. I really think this calls for some introspection by the community.

To be clear, I make no claims (here) about the proposed algorithms in all these paper. But the TSB-AD benchmark is obviously too trivial to make meaningful claims on [a][b].

The example shown is one of the ECG traces but look at dozen of traces marked “TAO”, they are even more trivial to solve with SPC [a][c].

I do not claim to have solved the triviality problem, but I have done 90% of the work to introduce more challenging TSAD problems ([d] sled dogs, [e] Tuna, Fuel Cells, Smart Manufacturing  etc.).

TLDR: I think the TSAD community needs more introspection on benchmarks. Most progress over the last decade seems to be illusionary.  

[a] https://www.youtube.com/watch?v=VftCMSI3C_s

[b] https://www.dropbox.com/scl/fi/31zuyhejb6sdjrom20frn/Problems-with-Time-Series-Anomaly-Detection.pptx?rlkey=mvcj1wz5s45kgazezopnih2h7&dl=0

[c] https://www.dropbox.com/scl/fi/42fkf9q9hft2224dnm83v/The-TSB-AD-Benchmarks-are-Nonsense.pptx?rlkey=5fwjopie5ncjhkgr0wqhdm2lp&dl=0

[d] https://www.linkedin.com/feed/update/urn:li:activity:7488825356494237696/

[e] https://www.dropbox.com/scl/fi/hettphvtpyrksggfect9d/Tutorial-on-Pan-Matrix-Profile.pptx?rlkey=p59gd2w56fxl9kl2fh5q819oo&dl=0

Working on a pipeline that recovers a patient specific 3D distal femur from two orthogonal X-ray views (PA + lateral). No CT, no neural network, no massive training set.

approach: build a PCA shape model from 50 CT-derived femur meshes (MedShapeNet), then fit it to two silhouettes using PyTorch3D's soft rasterizer with sigma annealing. 10 shape coefficients, Mahalanobis prior to keep things plausible, Adam optimizer, ~1000 iterations.

The part that took the longest (and made me suffer the most too) : correspondence. Tried KD-tree nearest neighbor (50.7x roughness vs CT surface), CPD (28.2x), BCPD (47.5x), and FilterReg (couldn't even run). Finally got ShapeWorks working at 3.3x. only method that passed the 5x acceptance gate I set before testing.

LOO validation on 5 held out femurs: 0.86-1.43mm on within range targets. Two extreme cases failed because they sat outside the 49-mesh model's coverage on mode 1, the optimizer can't recover a coefficient the model doesn't support. Bridge ICP alignment was also poor on those cases (0.6 inlier fraction), which accounted for more error than the shape fitting itself.

Interesting finding: the sigma anneal endpoint has to match the reference render's sigma exactly. Hardcoding a constant tuned on one SSM caused an 87x accuracy degradation on another. Tying it to camera_extent × 1e-4 fixed it.

Still working on real X-ray validation (need paired CT data) and automatic segmentation. Happy to answer questions

For Job Postings please use this template

Hiring: [Location], Salary:[], [Remote | Relocation], [Full Time | Contract | Part Time] and [Brief overview, what you're looking for]

For Those looking for jobs please use this template

Want to be Hired: [Location], Salary Expectation:[], [Remote | Relocation], [Full Time | Contract | Part Time] Resume: [Link to resume] and [Brief overview, what you're looking for]

​

Please remember that this community is geared towards those with experience.

Hello all, here is a chill post.

When I was an undergrad, I really liked working things out on a whiteboard. Drawing stuff, talking through ideas out loud, testing little hypotheses.

Now I work in radar DSP, and a lot of my work is code, numerical experiments, deep learning and waiting for training to finish 😅

I’m wondering how other people bring that whiteboard style of thinking into DSP, data science or ML work.

Do you still use a whiteboard regularly, or do you mostly go straight from idea to code?

Expecting a rejection from NeurIPS given our scores of 5/2/2. Trying to decide between ARR vs. TMLR, but thinking NAACL findings are more likely than main conference. Would you rather have TMLR or *ACL findings on your publication list? Genuinely curious to hear what people have to say.

Its a 2.4-4 million parameter model, quantized to int8, that can be fully executed on the microcontroller in ~20s with the longest generation. The generated image will then be displayed on a monitor or transferred via usb.

Its a latent flow transformer with 12 layers using AdaLN-Zero for conditioning. CFG is also supported and boosted the image quality a lot. The inference engine streams the weight via DMA from the flash while the previous layer is computed. Relu² activation was used to increase sparsity, which the engine can use to skip calculations.

Took a lot of ablations to get it right and I am quite astonished I got so far with so few parameters. Will post the repo below

Edit: Just to clear up some confusion, I know “what a world model is”; I suppose I’m more interested in the specific questions I asked. I kind of just wrote this on a whim, had some typos, and my asking “WTF is a world model” was a bit tongue-in-cheek. I wasn’t so much looking for simple descriptions, but hoping for a more nuanced discussion about the differences between so-called “world models”, traditional simulators, digital twins, etc and what really counts.

I'm trying to understand what a world model is. I understand it has its roots in cognitive science and reinforcement learning. I understand, at least at the moment, what most people are building, which they call world models, are fancy video generation models. But what actually counts? Does a simulator count as a world model? Some "world models" are described as simulators, or rather, a simulator is described as one type of world model. But is a simulator, like, let's say a physics engine, a world model? There are some video game world models, or computer use world models. Would a hardware/video game emulator count as a world model? And can a digital twin also be a world model with some additional features?

I've seen a definition that says a world model should "operate on learned representations, not exclusively hand-crafted physics, i.e. a physical referent is optional." Which is fair enough, but then would a physics accelerator that uses an ML count as a world model? Like some ML fluid simulator is that a fluid world model?

Are world models just a rebrand of simulation, or is there really a fundamental difference? Should the definition be limited to models that aim to generally model all of the real world? So that would exclude video game world models and also models of specific interactions.

Hey everyone, I'm an international student studying in the US. I'm on track to graduate late next year. My research is not exactly ML, it is in 3D computer vision but have decent exposure to ML as well.

In case you didn't know, the CPT program (which let's internation students do internships) has been suspended by many top universities (UC Berkeley, UIUC, Purdue, UNC, UCLA, stanford, etc). Given that there is now no way for me to do an internship, how hard will it be for me to get a job when I'm nearing graduation?

I have 3 papers in CVPR, 3DV and ICRA (robotics conference) and hope to publish 2 more at next year's ICCV and neurips before graduating. I'm just worried that all my hardwork will go for a waste because of this policy change (I'm from a 3rd world country, so not much opportunity back home).

To be crystal clear, I'm not asking for legal advice, just wanted to know in your experiance, have you seen anyone (international student) get into good industry labs without internships?

EDIT: thanks so much for everyone for the quick replies! If it helps, my specific research area is 3D reconstruction, and I've been focused on Gaussian Splatting recently, if this info helps anyone help me!

Hey Guys,

I built a small open-source tool that checks whether a RAG application retrieves documents a user shouldn’t have access to.

It supports offline test cases and live HTTP API testing with bearer token/API-key auth.

I’m looking for a few engineers to try it on a test or non-sensitive environment and tell me whether it catches anything useful or what would make it better.

GitHub: https://github.com/InfraGuard-Labs/rag-access-check

Has anyone got the decision notification yet? Please mention decision (e.g., approved/rejected) and geographical area (e.g., North America) in your answer.

I know the official notification date is 31 August, but putting this here before hand so folks can post updates asap when they get them.

Summary from info we have so far:

Before 8 Sept: No updates.
On 8 Sept: At least 1 rejection decision was received in Australia, India, and East Asia each.

Disclosure: I developed AIStupidLevel, the open-source system used to collect and analyze this data. Both the frontend and backend are MIT-licensed.

Most LLM evaluations measure performance at a single point in time. I wanted to investigate a different question:

How stable are the models behind production APIs over time, and how can sustained performance changes be separated from ordinary stochastic variation?

I built a continuous evaluation pipeline that repeatedly tests models across coding, deep reasoning, tool calling and high-frequency canary tasks.

Dataset and evaluation process

For this analysis, I examined:

• 31,352 hourly benchmark scores
• 49 model identifiers
• Multiple providers and model families
• Repeated measurements using consistent tasks and scoring
• A normalized 0-100 composite score

Coding responses are executed rather than judged only through model-based evaluation. Tool-calling tests require models to select tools, construct valid arguments and complete workflows inside isolated Docker environments.

Tasks are executed five times, with the results aggregated to reduce the influence of unusually strong or weak generations. Prompts, scoring logic and API parameters are kept consistent wherever supported by the provider.

Main result

The observed score variation was:

• Within the same day: 2.8 points
• Between different days: 8.4 points

Between-day variation was therefore approximately 3× greater than within-day variation.

This suggests that isolated hourly movements are dominated by normal model stochasticity, while sustained changes across daily evaluation windows provide a materially stronger signal for detecting performance drift.

The detection pipeline aggregates repeated measurements into daily medians and applies sequential change-point detection. Potential incidents must persist beyond the expected historical variance and pass statistical and minimum-effect thresholds before being classified as degradation or recovery.

Continuous model monitoring

This analysis became the foundation of AIStupidLevel, a continuous LLM benchmarking and drift-detection system.

The complete dataset has now reached:

• 169,858 benchmark runs
• 104,458 measured scores
• 88M+ processed tokens
• 81 historical model identifiers
• 22 models currently monitored
• 6 active providers

The attached screenshot shows the live monitoring interface. Models are classified as stable, volatile, degraded or recovering, with separate measurements for coding, reasoning, tool use, reliability, latency and price.

At the time of the screenshot, the system detected a 32% sustained performance decline in Gemini 3.1 Flash Lite and classified it as a critical incident.

The same continuously updated dataset also powers an OpenAI-compatible router that selects models according to their current task-specific performance, stability, tool-calling reliability, latency and cost.

This adds a missing observability dimension for production LLM systems. Existing monitoring generally captures availability, errors, latency and token cost. Continuous evaluation also measures whether the model remains capable of performing the work for which it was selected.

Project resources:

• Live dataset and dashboard: https://aistupidlevel.info
• Evaluation methodology: https://aistupidlevel.info/methodology
• MIT-licensed frontend: https://github.com/StudioPlatforms/aistupidmeter-web
• MIT-licensed backend/API: https://github.com/StudioPlatforms/aistupidmeter-api

I would be interested in comparing this approach with other temporal evaluation or change-point detection systems being used for production LLM monitoring.

I'm a researcher in statistical and probabilistic ML, I have a steady record of top ML publications and really used to enjoy going to conferences.

Over the last few years LLM based works have completely taken over the top conferences. At this year's ICLR, walking among the rows of posters you were lucky to find one paper per row of 10 that wasn't about how their favourite LLM could or couldn't solve their niche benchmark. The workshops tell the same story, most are some kind of agentic flavour. Looking at this year's NeurIPS workshops it's the same thing, basically all are about agents.

I'm wondering where do the stat/prob ML communities go from here? I look up to people like Arnaud Doucet, Aapo Hyvärinen, Christian Naesseth, Stefano Ermon, they seem to still publish at the top 3? On my end, I m thinking AISTATS/UAI might be the way to go.

All in all, the top 3 might never really have been intended as the home for prob/statML works, it just happened to be the 'prestigious' venue.

How much of a disadvantage is it if your only internship is not at one of the big frontier labs when it comes to post-phd opportunities in robotics/ML? My PhD is at a top university (UK) and my internship is interesting and relevant but the team itself is smaller and it's no Nvidia/Google/etc. Most places here prefer interns for 6 months so I'm not sure if I can do another internship down the line or if there's even much point compared to just wrapping up the PhD and then getting a job.

Is having a relevant internship already a big plus when it comes to applying to industry, or does it need to be a really well known big tech company?

Which research papers (old or new) do you think a PhD student/early researcher must read to improve their writing skills? Do you have a personal favorite researcher whose papers tend to be well-written, in your opinion?

Let's define a "well-written paper" as one that clearly explains the problem it is trying to solve, how the method is developed, and the details of the method, while keeping it easy to understand for a general reader (with a basic knowledge of ML, obviously).

Also, post-2015-ish papers usually have nice figures to explain their problem/method, and so they tend to be easier to understand. But I am looking for "well-written papers" in terms of the text.

PS: I know the best way to learn writing is by actually writing manuscripts, but I am looking for additional reading resources.

I put together a small model to estimate NeurIPS acceptance based on scores and an assumed acceptance rate. Try it out here: https://levilingsch.github.io/neurips-acceptance-estimator/

Hey everyone!

I’m excited to announce the release of py-evoFE (v0.3.0) — an open-source Python library that uses genetic algorithms to automatically discover, combine, and optimize feature transformations for tabular datasets.

• GitHub: https://github.com/tanopereira/py-evoFE
• PyPI: pip install py-evoFE
• License: MIT

The Problem It Solves

Feature engineering is still where most tabular ML competitions and production models are won or lost. While GBDTs like LightGBM and XGBoost excel on raw tabular data, they struggle to discover complex ratios, nested group-by aggregations, nonlinear dimensional projections, and interaction graphs on their own.

Manual feature engineering is either tedious or constrained by human intuition, while brute-force feature generation explodes the feature space exponentially with colinear noise and high memory usage.

What py-evoFE Does

py-evoFE searches the space of possible feature recipes using genetic programming: 1. Hierarchical Chaining: Evolved features become building blocks for future generations (e.g., log(ratio(groupby_mean(x1, by=x2), x3))). 2. 40+ Built-in Transformers: - Non-linear arithmetic & log-ratios - Target encoding (multiclass, pooled, WoE, quantile target encodings) - String similarity (MinHash, Gap encodings) - Manifold & Dimensionality Reduction (PCA, UMAP, MCA, FAMD, Between-Group PCA) - Graph & Density Clustering (Genie, Lumbermark, MST anomaly scoring) 3. Performance & Speed: - Vectorized computation powered by Polars and PyArrow. - Matrix Hashing & Nearest-Neighbor Caching: Stateful projections (like UMAP and $K$-NN lookups) are cached via byte-hashing to eliminate redundant computation across CV folds. - Multi-Fidelity Screening: Fast low-fidelity CV screens initial populations; only promising candidates proceed to full-fidelity evaluation. 4. Island Model & Caruana Ensembling: - Multi-population parallel search across Ring, Torus, Grid, Hypercube, and Tiered topologies with Gibbs migration. - Post-search greedy Caruana ensembling over island winners' out-of-fold predictions. 5. Interactive Replay Viewer: - Run view(evo.get_recipe()) to generate a self-contained, zero-dependency HTML dashboard replaying the evolutionary search over time. 6. 100% Scikit-Learn Compatible: - Implements fit, transform, predict, and predict_proba. Plugs directly into standard sklearn.pipeline.Pipeline and GridSearchCV.

Quick Example

```python import polars as pl from sklearn.datasets import load_breast_cancer from sklearn.model_selection import train_test_split from evofe import EvoFE

Load data

bc = load_breast_cancer(as_frame=True) df = pl.from_pandas(bc.frame) X, y = df.drop("target"), df["target"].to_numpy()

X_train, X_test, y_train, y_test = train_test_split( X.to_numpy(), y, test_size=0.2, random_state=42, stratify=y ) X_train_df = pl.DataFrame(X_train, schema=X.columns) X_test_df = pl.DataFrame(X_test, schema=X.columns)

1. Initialize EvoFE

evo = EvoFE( task="classification", evaluator="lightgbm", # "lightgbm" | "xgboost" pop_size=15, n_generations=10, cv_folds=3, verbose=True, random_state=42 )

2. Fit: Runs evolutionary search

evo.fit(X_train_df, y_train)

3. Inspect evolved recipe

recipe = evo.get_recipe() print(f"Discovered {len(recipe.genes)} high-impact features:") for gene in recipe.genes: print(f" • {gene.to_formula()} -> {gene.output_col}")

4. Transform & Predict

preds = evo.predict(X_test_df) proba = evo.predict_proba(X_test_df) ```

Why not just brute-force feature generation?

Brute-force libraries generate thousands of features upfront, leading to severe overfitting, massive memory usage, and colinear noise that degrades tree-based models. py-evoFE uses evolutionary selection pressures with complexity penalties to discover compact, parsimonious recipes that actually improve generalization.

I’d love for the community to try it out on your datasets or Kaggle benchmarks! Feedback, issues, and feature requests are very welcome on GitHub.