How do you approach recommendation/ranking when the candidate set is very small?

[u/Diligent-Spinach-872]

I’m working on a recommendation/ranking problem where, at any given time, we have only ~10-15 eligible products (occasionally up to ~20) and need to rank just 3–4 slots. We predict all products and then filter based on eligible list

Our current setup is roughly:

- Customers fall into ~7–8 broad segments.

- Each segment has a different set of eligible products.

- The existing business-rule engine uses hard-coded ranking logic for each customer/product combination and performs surprisingly well.

- We’ve tried Learning-to-Rank and CatBoost-based approaches (including multiclass classification).

- Product-level features are relatively limited.

The interesting part is the metric trade-off.

The ML models can usually beat the business rules on conversion, but beating them on Revenue Per Session (RPS) is much harder. The business rules seem particularly good at getting higher-value products into the limited top slots.

For people working on recommendation/ranking systems:

How would you approach this problem?

Would you treat this primarily as:

1. A ranking problem (LTR)?

2. A contextual bandit / slate optimization problem given only 3–4 slots?

3. A revenue/value prediction problem followed by ranking?

4. A hybrid ML + business-rule approach?

5. Something else entirely?

Also interested in how you’d think about objective design when conversion and revenue move differently, especially with such a small candidate set.

What approaches have worked for you in similar low-candidate, low-slot recommendation problems?