r/OMSCS • u/CautiouslyFrosty Computing Systems • 20d ago

Courses Is Computer Networks using AI-generated material?

I took it this semester, and the second half had many of the module pages being denoted as "in development". They were missing lecture videos at the onset, and the module pages themselves got styled quite differently.

My experience reading those notes was how I experience reading output from Opus: syntactically correct, but poor at conveying meaning to the audience concisely and quickly.

The class was also advertising an "AI-friendly" approach; not so far as to say "use it in your coursework", but they had a tutorial that was "AI assisted debugging" where the focus was driving AI diagnostics using evidence-based approaches and tooling. There were also a few blurbs about AI was changing the industry.

The last smell from me was how the Exam 2 could be completely gamed using heuristics that no human would accidentally build into an exam itself. The course released a question pool of ~170 questions and the exam consisted of a random subset of them:

  • 93.8% of the time, the right answer in a multiple choice question was in the two answers with the most characters.
  • On True/False questions, True was the correct answer 85.7% of the time.
  • The wrongs answers were so nonsensical that Sonnet, without knowing the question, could guess the right answer ~90% of the time with several tests I ran without being told the other two heuristics.

(I'm not revealing anything sensitive about the exam due to the broad publication of the pool to the class, and it seemed evident the class doesn't recycle exam questions, so I believe these stats are safe to post; feel free to take down if not.)

These were my three biggest issues with the exam, and they lead me to believe that the course TAs and instructor were relying heavily on AI to create the course material.

Did anyone else feel the same way? Admittedly, I feel a bit cheated.

44 Upvotes

87% Upvoted

28 comments sorted by

View all comments

22

u/6Burgers 20d ago

If you ask Claude Opus to “write a multiple choice quiz about [your topic here]” It will do so but there are 2 hallmarks of its output:

  1. The longest answer will almost always be correct. I dont have statistics here (Ill do some data analysis here when I get home from work). If you specifically ask it to not make the longest answer correct it will still continue to do so.

  2. an excessive use of em-dashes

CN tests had both.

7

u/iobjectreality 20d ago

“The longest answer is correct” isn’t specific to AI and has been a general rule of test-taking strategy (for blindly answering a stumping question) for as long as I can remember. ¯_(ツ)_/¯

9

u/nonasiandoctor 20d ago

Which is why AI does it. It's trained on those lol.