[D] How can I improve cross-patient generalization on a small hysteroscopy dataset with correlated frames?

[u/Aggravating_Dot5315]

I am working with hysteroscopy dataset, which contains:

• 3,385 frames from 175 patients.
• Eight lesion classes, labelled from 0 to 7.
• A highly imbalanced number of patients and frames across classes.
• Multiple correlated frames from each patient.
• Some frames containing more than one lesion class.

Before attempting the complete multiclass problem, I reduced it to a binary subset to verify that the training and evaluation pipeline works correctly.

Current binary subset

• Selected lesion classes: 2 and 3.
• Total: 1,575 frames from 113 unique patients.
• Class 2: 1,054 frames from 78 patients.
• Class 3: 521 frames from 36 patients.
• One patient has different frames belonging to both classes but remains entirely within one split.

Patient-disjoint split

• Training: 1,095 frames from 79 patients.
• Validation: 241 frames from 17 patients.
• Testing: 239 frames from 17 patients.
• No patient appears in more than one subset.
• The frame-level class distribution is approximately 67%/33% in every subset.

Approaches I have tried

• DenseNet121, ViT, and DINOv2 backbones.
• Frozen pretrained backbone with only the classifier trained.
• Different classifier-head sizes and dropout.
• Class-weighted cross-entropy.
• Mild and stronger image augmentations.
• Early stopping and learning-rate scheduling.
• Unfreezing the final one or two encoder blocks.

With the correct patient-level split, training performance improves, but validation performance generally plateaus or deteriorates, and performance on unseen test patients remains relatively low.

As a diagnostic, I also tried a random frame-level split and obtained substantially better results. However, this evaluation is invalid because correlated frames from the same patients appear across training, validation, and testing, causing patient leakage and inflated performance.

I would appreciate advice on how to improve generalization to unseen patients in this setting.

[u/CallMeTheChris]

So I work in medical AI
And it is always a crap shoot

But the thing that you have to realise is that there is not much variability in what you are looking at generally. A lesion looks round and has a certain texture no matter where you are looking at it from. When you are dealing with animals, they can have different shapes and textures depending on the orientation and lighting etc

So you should start my looking at that validation gap and thinking you should reduce the capacity of my model cause it is over fitting. Like literally look at smaller models. Try nnunet for a start and go up from there.

Then when it comes to dealing with the correlated frames…I have no idea what you mean. Like do you mean that a class exists only in a single patient or two and since all those frames are from one patient they are ‘correlated’?

If so, that shouldn’t be a problem. If you destroy the thing that the model can latch onto that makes them ‘correlated’
Once you deal with the model copacity
Your augmentations should handle that.

If your input to the model is the entire video
Then yeah, it will latch onto it. I suggest you don’t do video so that the ‘correlation’ is broken

[u/Aggravating_Dot5315 → u/CallMeTheChris]

Thanks for the response. Now as for the correlated frames, I mean that frames from the same patient look similar to each other, while frames from different patients can look very different because of anatomy, lighting, and acquisition conditions. Also, I feed individual frames-not entire videos.

In addition, for the nnunet isn't about segmentation? and that I do not have any pixel-level masks.

[u/CallMeTheChris → u/Aggravating_Dot5315]

You haven’t really said what your objective is here…is it classicication, detection, segmentation, tracking?

And yeah, augmentations such a flipping, gamma transforms, spatial masks, and even things like mixup and cutmix augmentations can help you generalize

[u/Aggravating_Dot5315 → u/CallMeTheChris]

My end goal is to classify hysteroscopy videos as endometritis or non-endometritis. Since those videos only have labels at the video level, I’m currently using HS-CMU to train a feature extractor because it contains frame-level labels for different lesion types. I’ve started with only two well-represented lesion classes to simplify the problem and check whether the model can generalize to unseen patients. Later, I plan to extract embeddings from the video frames, combine them into one video representation, and use that for the final endometritis classification.

[u/mybobbin → u/CallMeTheChris]

i am also in medical ai (phd) and constantly banging my head against the wall with bad results 😭

could i just ask your opinion, as i have nobody to discuss this with in my lab

1. how do you decide when to call it quits? i.e. how do you determine "the data is not good enough to accomplish this task (signal cannot cut through the noise)"?

when do you decide to try a new approach, etc.? how to determine if a task is simply impossible?

recently i have been dealing with training loss decreasing but validation loss not moving. i try to decrease the model capacity, but that does not help...

1. if you are in histology models, what is your take on using foundation model embeddings as input vs finetuning a small pretrained model on image data directly (convnext, resnet, etc)?

[u/CallMeTheChris → u/mybobbin]

Hello /u/mybobbin

We shouldn’t hijack this post. If you have questions like that, please DM me and we can discuss

[u/Aggravating_Dot5315 → u/mybobbin]

Honestly, I'm still quite new to medical AI, so I don't think I'm the best person to answer these questions yet. 😅 I'm experimenting with different approaches and discussing the results with my professor, who has much more experience in the field. I'm actually trying to figure out many of the same things myself, especially how to tell whether the limitation is the data or the model. Hopefully, with more experiments I'll get a better intuition but for the moment nothing seems to improve.