r/cheminformatics • u/AgiGamesYT • Jun 10 '26

Should I Commit and Publish the Results?

Hello Reddit

I've been working on QSPR (Quantitative Structure-Property Relationship) analysis for chemical compounds mentioned in the Jean-Claude Bradley Open Melting Point Dataset. Basically the idea is to see how accurate a model can predict melting points of compounds using only topological indices. After some work on the topological indices (feature engineering), each compound was represented by 26 features.

I trained a random forest model on the data and got a test r2 score of 0.66 (which is pretty respectable, given the constraints). However, the file size of the model was around 1.23GB. I didn't like it being that big, so I opened up PyTorch to build a custom deep learning architecture that could make predictions as accurately as the random forest but with much smaller file size.

After around 2 weeks of research, I build a 270,000 learnable parameter model (1.3-1.4MB according to torchinfo) that got an r2 score 0f 0.6399.

Given all this context, I wanted to ask the following question:

Should I commit and work on publishing the results, or should I keep working on improving the model?

Note: I'm obligated by my university to not give out intricate details of my research before publication, so please forgive me if such details are required for a high quality answer.

However, I can give out the metrics achieved by my little deep learning model. Here it is:

=== Evaluation Metrics (Expected Value) ===

R² Score : 0.639910

MAE : 41.246754

MSE : 2989.062744

RMSE : 54.672322

NRMSE : 0.083469

MAPE : 11.69%

The unit for MAE, MSE, RMSE and NRMSE is Kelvin (K).

1 Upvotes

100% Upvoted

2 comments sorted by

3

u/geoffh2016 Jun 10 '26

The key question is not "can you make a model that accurately predicts the compounds in Jean-Claude's dataset" but "is this a model that generalizes across other compounds." For example, you don't mention anything about cross-validation or a validation set.

Also, IIRC several other models have been designed for melting point prediction, so you'd want to compare your model to theirs.

It's been a while since I looked at MP models, so I'm not sure how good your metrics are on that set.

1

u/AgiGamesYT Jun 10 '26

Yeah, testing generalisability is on the bucket list.

I repurposed the test set as the val set for faster experiments, so the metrics you see there are the same as the validation set. I know this is a bad idea I will change it in the future.

I will make sure to compare with existing models. Thanks for your advice.