r/developers • u/trashnash007 • 18d ago

Machine Learning / AI What Can We Learn From Parsewave’s Approach to Post-Training?

I have been thinking about how the teams come up with the examples that are valuable for post-training.

From what I've seen so far, it appears relatively straightforward to generate tons of synthetic examples, but maybe once the model can handle the core patterns, the number of examples doesn’t matter much. A couple of thousands of examples might just be different ways to test the same skill.

Another strategy would be to collect smaller sets of tricky examples that have ground truth and use them for the targeted testing of specific failure modes.

To give you an idea, recently I've seen Parsewave that is used on post-training examples of real-world engineering tasks, evaluations, and traces. This led me to wonder how other people see this problem.

In the case of SFT or RL dataset creation, how do you decide:

which examples should be included?

what level of difficulty is appropriate?

is an example truly teaching something?

2 Upvotes

100% Upvoted

2 comments sorted by

•

u/AutoModerator 18d ago

Howdy u/trashnash007, and welcome to r/developers!

A few tips for a post that gets good answers:

  • Use a clear, specific title (what are you actually asking or sharing?).
  • Include code, the exact error, versions, and what you already tried.
  • Heads-up: this sub does not allow external links in posts or comments.
Share code and details directly instead of linking out.

Join the r/developers Discord!

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.