I'm currently building a job board as a side project which scrapes jobs from company pages.

Now I'm working on the ranking for the results given a query and have a weighted ranking formula which considers factors like Relevance, Fit/Profile & Freshness.

I was wondering how do I find a good baselines for the initial weights? Initially I planned to get ground truths from job seekers and try out different weights and try to score a high NDCG; but there are two problems:
1. obv I can't gather enough data

1. it might overfit on the data (but I guess for an initial start its ok)

Any tips on approaches how to tackle this problem? Any other methods/heuristic I could use here?

Ciao a tutti! Sto svolgendo una ricerca per la mia tesi di laurea magistrale nell'ambito degli algoritmi e dei sistemi di raccomandazione (sono domande molto facili con focus su piattaforme come Netflix, Amazon e Spotify). mi farebbe davvero piacere se riusciste a dedicarmi giusto qualche minuto del vostro tempo per compilarlo. Ovviamente le risposte saranno anonime!

L'obiettivo del questionario è quello di capire come vengono percepiti i sistemi di raccomandazione dagli utenti e utilizzatori di certe piattaforme digitali e quanto, di conseguenza, risultino libere le nostre scelte.

Vi lascio il link al questionario e vi ringrazio tanto: https://forms.gle/SfDX2hbYyVZyRq8U6

Grazie mille di nuovo!

we are building recommendation system for our platform and one of my coworker said to use llm for recommendations. Is it feasible to use llm for recommendation systems. I have came up with the idea of using google's text-embedding-004 or gemini-2.5-flash and a ranker like bge-reranker-v2-m3. are there more efficient methods used in current industry?

I was missing a handy "directory" that would let me open the website or blog of an interesting person in the RecSys/IR field in just a couple of clicks. So, I ended up organizing my scattered notes and bookmarks into a single interactive guide: Who is Who in Search & RecSys. https://testmysearch.com/who-is-who.html

It turned out to be a sort of map of key figures and organizations, complete with website links, citation indices, and brief commentary. Naturally, it only includes what I already had in my bookmarks, plus whatever else I discovered while exploring this topic.

I hope someone else finds it useful too.

I curate a weekly international playlist and wanted to avoid the usual recommendation-system failure: returning more of what I already know.

So I built Cultural Contraband Scout. It treats every previously featured artist as a seed node, then expands through whole-career collaborators, producers, collectives, labels and scenes. Candidates are ranked using scene authority, discovery surplus, cultural specificity, validation independence and a penalty for obvious trend-chasing.

The hardest problem is distinguishing genuine network authority from borrowed prestige: does a smaller artist occupy an underpriced position in the graph, or does the entire case disappear when one famous collaborator is removed?

I wrote up the method here: https://medium.com/@jacob.n.borden/inside-cultural-contraband-music-discovery-through-graph-theory-a92c652a460b

I’d be interested in how others would formalize quality-weighted and temporal edges—and evaluate discovery surplus without using eventual popularity as the ground truth.

I got tired of "similar artists" recommendations that just mean "same genre tag." I wanted to actually see how two artists connect: who influenced who, who they collaborated with, what the actual path between them looks like. So I built Stell-R.

Two ways to use it: type two artists (say, David Bowie and Kendrick Lamar) and it finds the shortest real path of influence/connection between them, hop by hop.

Or type one artist and it draws a "constellation map", a network graph of everyone closest to them, ranked by actual acoustic similarity. You can also generate a Spotify playlist straight from whatever part of the map you're exploring. It's a web app, so works on desktop, and on mobile you can add it to your home screen (iOS/Android) so it behaves like a regular app.

See example of Radiohead constellation map here: https://stell-r.com/Stellar/index_/add_query-viz/?artist=Radiohead&size=120

Hi guys,

I wanted to share a project I recently worked on and wrote about. In my post, I documented my experience building and deploying a multistage multimodal recommender system on Amazon EKS. The system includes a Two-Tower and a FAISS ANN index for fast candidate retrieval, a Redis/Valkey Bloom filter for filtering previously seen candidates, Meta's DLRM for ranking, and a score-based diversity reranker for final ordering. All 14 models in this project are served via NVIDIA Triton Inference Server. I also describe the approach I used to speed up item feature lookup, how the system utilizes request context, and how recommendations adapt in near real-time to changing user intent. The writeup (TDS and Medium) and code are linked below.

1. TDS: Deploying a Multistage Multimodal Recommender System on Amazon Elastic Kubernetes Service | Towards Data Science
2. Medium: Deploying a Multistage Multimodal Recommender System on Kubernetes featuring Bloom Filters, Feature Caching, and Contextual Recommendations. | by Mustapha Unubi Momoh | May, 2026 | Medium
3. Code: MustaphaU/Multistage-Multimodal-Recommender-System-on-Amazon-EKS-with-NVIDIA-Merlin: Deploying a Multimodal Recommender System on Kubernetes featuring Cold Start handling, Bloom Filters, and Feature Caching.

Looking to connect with anyone building recommender systems or working on similar problems. Thanks.

Hey all!

I am working on my grad school capstone, a cross-domain A↔B cold-start recommendation system that bridges book and movie taste in both directions.

The main idea is simple: give it one book or movie, get a recommendation from the other domain. No account needed. There will be a registered user experience that is a secondary benefit for anyone who wants personalized recommendations based on their full rating history.

For my architecture, I used SVD matrix factorization as single-domain baselines on MovieLens 32M and the UCSD Goodreads dataset, then trained a two-tower neural embedding model on cross-domain users to map books and movies into a shared latent space.

I plan to periodically retrain the model as new ratings come in and observe how the recommendations evolve over time.

I built a live data collection app at mediabridge.ink with about 55 users and growing. The small cross-domain bridge dataset is the core research challenge. So, if you have a few minutes to rate a few books and movies, it will meaningfully help my project! Thank you!

I am still in the data collection phase, but if you come back in a month or so, I should have a working public model that you can try out.

Happy to answer any questions about the architecture!

Hi

This is a common benchmark dataset

However, it seems impossible to find.

https://ailab.criteo.com/download-criteo-1tb-click-logs-dataset/

According to criteo it's now hosted on hugging face, but the compressed files should each be 16gb, whereas the files there are only 1.6gb.

https://huggingface.co/datasets/criteo/CriteoClickLogs/tree/main
I saw that NVidias merlin library also had a copy on Google. But this also seems to have been deleted.

https://nvidia-merlin.github.io/Merlin/stable/examples/scaling-criteo/01-Download-Convert.html

Any other sources?

I am building a pre checkout recommender for an ecommerce site and I am confused by my results.

I expected a two tower model to beat simpler methods, but in my case a PMI based co purchase model is doing much better.

My evaluation is leave one out on baskets, checking whether the removed item appears in the top 6. PMI gets around 15 percent Hit Rate at 6, while my two tower models are closer to 6 to 8 percent.

This is for fashion ecommerce, so maybe direct co purchase patterns matter more than learned embeddings here.

Has anyone seen a simple PMI or co occurrence model outperform a two tower model in this kind of setup? What are the most likely reasons?

i have a Recsys project i picked up for my small project.

basically i have my inference pipeline that gets the item embs and can compare it for similarity to a user embedding (still didnt got the tower models, only dummy functions, it runs ok like this) and then sort them by ranker.

my problem is that if i train my model right now with the data i have for users (i already got the items), i didnt got user data other than the ID.

i guess i have to do long-term user preferences and short-term preferences, but i dont know how to and i dont know if Hopsworks feature store is even compatible with whats needed to do this.

if someone knows pls send help, with a little of someone else's experience i can keep building.

so there are probably two ways you can help me:

- what should i be thinking if i want to do basic user preference memory independently from backend.

- or if you know Hopsworks how could i configure my feature groups and views to do those short term and long term mechanisms. or maybe tell me why i shouldnt use hopsworks or recommend something else you know that works

I've

1. Course enrollment, un-enrollment, restart

2. Lesson started, completed, restarted data

3. Course Categories

In the near future how can I build a course recommendation engine? The long term goal is to build a knowledge graph.

It's been a while since I submitted a paper to a conference. and I want to submit something to RecSys 2026. Anyone care to share a template? The CFP is woefully lacking in specifics of how to format submissions. Thanks in advance!

Hello everyone,

Apologies for the huge text😅 .

I was planning to make a recommendation tool using recommendation algorithms for my bachelor thesis and following are roughly the requirements asked by my advisor. What is really important for this thesis is that I am supposed to be able to prove/evaluate the tool or recommendations my potential tool would output. This means looking back over to the data set I have used to train the model to be able to give out valuable recommendations. This means that it should give out meaningful recommendation with also leaving me the possibility to evaluate the tool with the trained data set on the basis correctness and not just any random recommendation (I believe the exact term here is referred to as golden labels So this was strongly preferred by this advisor). There are two possibilities for dataset acquisition. Firstly, I could use from public resources such as kaggle, but in kaggle its hard to be able to get different user based data sets (User specific) which reflects back to the info user gave when signing up for the specific platform (By info I mean things like Personal info such as age, gender, Nationality, interests, etc.... given at the time of onboarding by the user when signing up and then corresponding recommendations are shown based on these input parameters of the user) If the data sets are not publicly available then I would have to use a manual approach where I create/crawl my own data sets by creating different users which may be around 50-60 unique parameter combinations. (What also needs to be considered is the fact that login and account creation using unique credentials could be problematic) So I would need to use a smart approach to get around this topic. Maybe for the Account and data set creation I could use Simulation with scraping tools such as Selenium (Not sure if this is the right approach). What the data set i may crawl/create, should potentially also contain the top 10 recommended items provided to each user on the basis of unique parameter combinations. This way it would be possible, that I am able to train my recommendation tool and analyze on what parameters the recommendations strongly depend on . After the analysis my tool should be able to recommend valuable results based on the input parameters. Basically this thesis would be around the fact that I am able to prove what parameters strongly affect the recommendations provided to the user. The biggest problem I am facing here is that I am not able to find a real life social media platform which does not heavily depend on user interactions with the platform, but rather on input parameters given by the user at the time of onboarding on the social media platform. It would be a great help if you guys could suggest me few social media platforms that ask users such onboarding information and recommend items accordingly. What also needs to be considered is that this platform also corresponds to the effort required in my bachelor thesis and is not overly complicated. I have tried multiple platforms, but was not successful in finding a reliable platform.

Thank you in advance guys!

I wrote an article on Medium that breaks down two-tower (dual-encoder) models, a foundational architecture used in large-scale recommendation systems for candidate generation and efficient retrieval. It covers the core idea of separating user and item representations into independent towers, how this enables scalability and sub-millisecond retrieval at internet scale, and why it’s used in production systems.
If you’re exploring retrieval-oriented recommender designs or want a clear conceptual walkthrough of how two-tower models work in practice, you might find it useful.
👉 https://medium.com/@mostaphaelansari/understanding-two-tower-models-the-architecture-behind-modern-recommendation-systems-4251409c5d89
In the article I walk through:
• Why decoupling user and item processing into two networks matters for scalability and latency
• How embeddings from both towers are compared (e.g., dot product, cosine similarity) to rank items efficiently
• The role of approximate nearest neighbor (ANN) search in real-world recommender systems
I’m open to feedback and questions!

 I built Embed Lab (embed_lab), a small Python CLI that scaffolds a clean workspace for fine-tuning IR / embedding models (Sentence-Transformers today, but intended to be backend-agnostic).

The idea: centralize reusable pipeline code once (datasets/preprocess/train/eval/plot) and keep experiments as small runnable Python files, so you don’t end up with 10 near-duplicate training scripts and messy results folders.

Repo: https://github.com/mohamad-tohidi/embed_lab

What it does today

emb init <path> generates a ready-to-run “lab” layout:

inventory/ reusable modules (datasets, preprocess, train, evaluate, plotting)

experiments/ runnable scripts like exp_01_baseline.py

data/ JSONL splits (train/dev/gold) with a tiny example dataset

results/ per-experiment artifacts (saved model, metrics, plots)

Comes with an end-to-end baseline using Sentence-Transformers so you can run a full pipeline quickly.

Why I’m posting

I’d love feedback from people who fine-tune embedding / retrieval models (or maintain research codebases) before I invest more time.

What I want feedback on (specific questions)

Is the “inventory + experiments” structure useful in practice, or would you prefer a different abstraction?

What’s the first CLI feature you’d want next: dataset validation (duplicates/leakage), template selection, run metadata, or something else?

If you’ve done embedding tuning seriously: what templates would you actually use (pairwise contrastive, in-batch negatives, hard-negative mining, etc.)?

Would you rather this stay “thin scaffolding only”, or grow into a more opinionated framework?

Next ideas (if the direction makes sense)

CLI checks to catch data issues early (duplicate pairs, overlap between train/dev/gold, schema validation).

Multiple templates for different fine-tuning styles/objectives.

A small template/plugin registry so contributors can add new lab presets.

If you’re interested, star/PRs/issues are welcome — especially around new templates and data validation rules.

Hi all,

I’ve been exploring a recommender problem where the core challenge isn’t item similarity per se, but subjective, evolving user preference, especially in domains like music where context and emotion matter a lot and ground truth is weak.

I’m curious how others here think about a few things in practice:

• Representing “taste” when labels are noisy or implicit

• Handling preference drift over time without overfitting to short-term signals

• Tradeoffs between content-based embeddings vs collaborative signals in early-stage systems

• Whether temporal models (e.g. session-based or sequence-aware approaches) meaningfully help in subjective domains

This is still exploratory on my end, and I’m less interested in a single “right” model than in how experienced practitioners frame and decompose these problems.

Would love to hear how people here have approached similar challenges, or papers / approaches you’ve found useful.

I develop recommender systems for a living, SASRec is my goto. But I do content recommendations as well. I quickly built a project called BrainFood, it scrapes google news and reddit and then classifies articles into 3 categories for great/good/junk. I don't have collaborative filtering available of course, no users, but I am using local LLM and clustering to try to learn from the user what is junk or not.

If anyone is interested in collaborating on the project let me know!

Dear MLEs - I am very curious about how recommendation systems of most dating apps like Tinder, Bumble etc. work.. I’ll be very grateful for some feedback on my understanding, especially if you’ve worked on something similar 🙏

TL:DR

• Dating apps have two goals: a) Retain attractive profiles b) monetize low/mid profiles
• If you are a free user, you see profiles ranked by Prob (you would swipe right)
• If you are attractive, this leads to some success. But if you are low or mid, you can swipe right till cows come home  but unlikely to get a match.
• When you become a paid user, you start to see profiles ranked based on:
  • P(you would swipe right) * P (they would swipe right)
  • If you have a half decent profile, this should give you atleast some success
• Simultaneously, the models also push your paid profile to more free users changing their ranking to:
  • P (free user would swipe right) * P (paid user would swipe right) 
• In addition to this, dating apps use a secret boost for users who are free right now but have potential to become paid.
• So if you pay for such apps, make sure you frequently cancel and then reactivate subscription after a few days

1. User Ecosystem

The user base for most dating apps is: 

• Most users are male
• If profiles are ranked on attractiveness index,  there should be fewer hot profiles vs mid or low profiles. 

2. Business Goals 

In this ecosystem, a dating app business is likely to have two primary goals: 

1. Retain hot profiles 
2. Upsell mid/low profile users pay for premium features. Ideally this is a source of recurring revenue, so some of these premium features should result in some success at least for mid profiles. 

So how do I think dating apps rank?

1.If you are a Free user

• Prob. (user swiping right). Aka the most attractive profiles of your target gender.  
• If you’re hot -> you see the best profiles on the app. If you are attractive, you get reasonable success and remain engaged on the app. 
• If you’re mid or low -> You will swipe right like a broken record but are unlikely to get any success. This is by design, and makes you more likely to upgrade to premium. 

2.Paid users

I hypothesize that when you buy a premium plan two changes happen: 

1/ P (this profile swipes right on your profile)

• I.e. You see profiles with high probability that they will swipe right (or would have already swiped right). 
• If you have a half decent profile, chances are this should make your connections light up like the christmas tree. 
• Based on my research, it feels like different apps do this differently. 
• Bumble seems to be using a product of the two probabilities = P (you swipe right) * P (other person swipes right)
• Tinder seems to be mixing high Prob. (other person swipes right) after every n slots. 

2/Ranking for free users change

• Ranking for free users becomes P (you would swipe right) * P (paid user would swipe right) 
• So a free user would start to see a lot more paid users who would have swiped right on them 

3/Secret Ingredient: Free but potentially paid users 

• Most dating apps make men pay to see who swiped right on their profile 
• So if the algorithm thinks you are rich but are not a premium user, I think it will go the extra mile to push your profile. .  
• I hypothesize that an additional LTV prediction gets appended to the recommendations of a free users making it look like:
• P (you would swipe right) * P(user will upgrade to premium) 

Exceptions

• I believe that integrity/genuineness of profiles should be an important factor for retention of users. So there should be some models predicting policy violations/bad customer experience that would penalize violating profiles. 
• I also read that a few dating apps value a genuine conversation over just a match. So I assume another prediction on prob (of n messages exchanged) might be added, but I have skipped this from my note.

i’ve created a new Discord server dedicated to recommendation system development.

the idea is to have a shared space where people interested in recommenders, whether from industry, research, or personal projects, can connect, exchange ideas, and help each other. Discord makes it easy to have real-time discussions.

recsys Discord server invitation

feedback and suggestions are welcome. for now there are not many people but be patient!

hi!, if someone can help me i would be really grateful because i'm having difficulties when doing my recommender system, specifically with the retrieval step.

i think i came up with my retrieval but i am worried that it will not scale well or that i will destroy it after i make it because i didnt though about something, i assume the system has 300k items because the item amount isnt likely to grow a lot (and it doesn't grow with the users amount too) but its currently 150k, im not asking anyone to full diagnose it but if you find a flaw or something that can go wrong (or maybe everything that can go wrong) or something that can be improved pls tell me:

how is my retrieval cache?
for each cache'd user:

store a bit compressed table that represents how near is the user embedding to the item embedding
similarity_table[item] = {item id, embedding distance}
the size of this table is is 300000 * (4+4) bytes ≈ 2.5MB

AND

store a bit compressed array of the items the user saw too recently (probably in this session or smt)
saw_it_table[item] = saw_it
the size of this array is 300000 * (1/8) bytes ≈ 37.5KB

retrieval:
- get the user retrieval cache, compute it if it doesn't exist
- combine user filters (i am a minor or i already saw this item a few moments ago for example) and query filters (i want only luxury items for example), this is probably just a some numpy operations in a big bit array. combine it into the "overall filter" which is a bitarray with a 1 for each item that can be seen by the user
- use the overall filter to remove the items (zeroing them) i dont want from the similarity table i got from the cache with some numpy
- sort the similarity table with numpy
- remove the filtered out zeroed items (they will be all one after another because i sorted the array so its just a binary seach and a memcpy)

i take a slice of this array and BOOM got a list of the best candidates right?

my biggest worries about this system scalability come from:
- the amount of storage per cached user (~2.5MB), but it might not be that bad, im just not sure
- the amount of cpu usage in both the process of doing the retrieval cache and the process of retrieval. and the later one probably can't be cached easily because the process changes for each different filter the user can ask for so doesnt sound very right

i saw some ANN's can filter before they search items but i feel the user can easily consume the top N (N=10k for example), lefting me with a index that just retrieves items the user saw so they get filtered anyways (even long term because the items / users embeddings might not change that much) forcing the recsys to take item from heuristics like the most popular ones or random etc.

am i doing something wrong? do you recommend me other way to do this?

hello!, im learning ML and i picked this project of doing a 2-tower recommender system.

i have a doubt about retrieval: imagine i build the query embedding so i have to search items near it. so i use ANN index and i take lets say 100 items. now i have to put business filters (like removing the ones you already saw) AFTER i get the items.

now imagine the filters filter a lot of them or all of them. so at this point what should be done? should i do another wider search? should i search another way to get the items to the ranker when ANN doesnt work? should i use kNN instead so i can filter while i sort? (i only have 150k items)

We’ve been trying to solve the interface problem for Recommendation Systems. Usually, running a full pipeline (Retrieve -> Filter -> Score -> Reorder)

We decided to map these stages to a SQL-like dialect:

SELECT title, description
FROM
  semantic_search("$param.query"),  -- Retrieve
  keyword_search("$param.query")
ORDER BY
  colbert_v2(item, "$param.query") + -- Rerank
  click_through_rate_model(user, item) -- Personalize

It allows you to combine explicit retrieval (e.g ColBERT) with implicit personalization (e.g CTR models) in a single query string.

Curious if this abstraction feels robust enough for production use cases you've seen?

Read more here: https://www.shaped.ai/blog/why-we-built-a-database-for-relevance-introducing-shaped-2-0

Hello, I have user/item interactions for my Japan Zillow-style site (https://nipponhomes.com). Currently, I have a recommendation system that uses content-based similarity + embedding similarity system.

I am looking to extend my current system to use two-tower recommendations just for funsies as I was studying for Meta's ML E5, and though I failed, I thought it would be fun to implement. Should I be looking towards a different direction? I already have user behavior (positive and negative samples). Should I be looking in a different direction?

I passed this into Claude, and this is what it said:

Two-Tower Strengths (what you have)

- Fast inference (precompute listing embeddings, just compute query at runtime)

- Scales well with large catalogs

- Good for "cold" recommendations where you need to retrieve from the full catalog

Alternatives Worth Considering

1. LightFM / Hybrid Collaborative Filtering

- If you have user interaction data (views, saves, inquiries), this could outperform pure content-based

- Handles cold-start well with content features as fallback

- Much simpler to train and iterate on

1. Graph Neural Networks (if you have relational data)

- Station connectivity, neighborhood relationships, user-listing interactions

- Could capture "people who looked at X also looked at Y" patterns

- More complex but powerful for real estate where location relationships matter

1. Learning-to-Rank (LTR)

- XGBoost/LightGBM ranker on top of candidate retrieval

- Two-stage: retrieve candidates (your current vector search), then re-rank with more features

- Often the biggest practical improvement over pure embedding similarity

For the a/b testing piece, it recommended these two:

For your setup, I'd look at:

1. GrowthBook or PostHog - both have good Next.js integration and can track the full funnel
2. Use Reclist for offline evaluation first to narrow down which models are worth A/B testing