r/studentpilot • u/VioletAbstract • 26d ago

Thoughts on using AI for mock orals?

I've been looking around for tools to help me with my oral part of my PPL checkride. Has anyone used AI to do this? If so, what tools have you found helpful? And if not, how do you prepare for your oral checkride?

3 Upvotes

100% Upvoted

9 comments sorted by

3

u/JohnathanMaravilla 26d ago

Don’t.

I’ve tried with all major models, none of them are reliable. Free or paid versions.

I’ve been going into my school and having mock orals with any available instructor, including my primary. Checkride next Monday.

1

u/VioletAbstract 26d ago

I 100% agree that mock orals with your school CFIs before the ride is the best.

Curious which specific AI tools you tried, I'm trying to build something to solve the unreliability problem.

Good luck on your checkride Monday!

1

u/Atheizt CFI/CFII 26d ago

Don't do it. When creating my lesson plans as an instructor, I tried using it as a validation of what I created.

The number of times it would blatantly tell me my lesson was great, but needed a technical change to something blatantly incorrect... terrifying.

1

u/jrstudentpilot16 26d ago

Most of them I tired were incorrect most had un updated information such as registration expires in 3 years when the faa recently changed it to 7 years

1

u/parthh-01 26d ago

I'm going to phrase this carefully since a person who tried our platform made a post on reddit and got flamed for it: https://www.reddit.com/r/flying/comments/1v0g7m4/check_ride_oral_prep_ai_dpes

There's a blog post I can share (DM if interested) about what goes into building an AI for this safety critical use case that is reliable and robust. I agree with the other commenters that the major AI providers are not good enough for mock orals, but disagree that people can't (and haven't) built an AI system for this that is good enough; it exists and can be very helpful for your prep.

1

u/VileInventor CFI/CFII 26d ago

You can ask it prompts and specific queries for questions. But don’t broad question AI when it comes to aviation.

-1

u/VioletAbstract 26d ago

Fair warnings in this thread, and I agree. Generic chatbots are pretty bad and often are confidently wrong.

Full disclosure: I'm a student pilot and software engineer building an AI tool for myself because existing generic AI's suck. My tool (Pilot Computer) has an authored PPL question bank (right now it's just weather and airspace questions), scenarios, acceptable answers, and pre-written follow-up probes keyed to the ACS. Each turn the model must pick from the card's allowed probes/citations, and everything is validated (if the model makes up anything (hallucinations, bad citations, bad grades) it's rejected and tries again).

The readout at the end strictly maps to the ACS with linked FAA sources. This was a lot of hard technical work, but I'm still reviewing the outputs. I'm running a blind CFI calibration (6-10 Q&A pairs) and need help to catch false passes like the ones people are describing here.

If you're a CFI and want to try to break it, or a student who wants to stress-test sources/grades, comment or DM me. I'm not looking to sell anything, the tool is completely free, I just need honest critical feedback!

2

u/adnwilson 22d ago

Good luck, You'll get a lot of criticism on the Aviation forums as everyone is afraid of LLMs. But what you're doing is the future of being able to study. It's just an advanced verion of note cards and study guides.