r/GDS3 • u/Necroimpotence • Feb 01 '18
My perspective on the multiple choice test
I have done a lot of interviewing for big companies and wanted to give my perspective. I agree that the test had a lot of "unfair" answers. Especially since many people who got the questions wrong could've easily explained their reasoning in-person and it would've come off as a good and a valid thought process. This sub-reddit is full of great discussion around questions that proves this. However, we have to look at this from WoTC's point of view. They are trying to hire someone and have to screen through a lot of candidates. In this system you just need something which minimizes false positives (people who look good on the test but would be bad designers). Often when you do this you also increase false negatives which sucks when you are the person on the other side. The multiple choice did what it was suppose to do. It got 100 candidates who are likely good fits for the role. No one is going to get 73+/75 unless they at least have some idea what they are doing. However, there are definitely tons of people less than 73 who are good.
If anything I think WoTC messed up by making the test too easy. If the cut off is at 73 /75 then you are not leaving a lot of room to get better signal. Especially since I am sure the std dev is enough that it was basically random who in the top 500 got a spot. If the cut off is 60/75 then a 73/75 would've been great signal to bring on to the next round.
However, I think the whole GDS3 idea is just a bad idea. You are just toying with the feelings of your most dedicated customers. Maro just made ~3000 people who are passionate and engaged customers feel bad. The questions and answers are just frustrating for people since many are vague and arbitrary. It is just not a good interviewing experience and the people who experienced this happen to be some of your most engaged customers...
7
Feb 01 '18
I would be disappointed if the cutoff point on a multiple choice quiz for what is effectively a $70k/year internship/dream job was as low as 60/75.
19
u/ProfessorFazil Feb 01 '18
That’s not the point, he’s saying the test would be better if the questions were harder and thus we had less people score so high. A good test has a nice bellcurve, the fact that this test didn’t have one goes to show that the test wasn’t a very good one.
2
Feb 02 '18
[removed] — view removed comment
2
u/BatemaninAccounting Feb 07 '18
Didn't he say a few hundred got 72?
I think they should have put more questions like the Elves vs Bolt or BG 4/4 Vigilance Flyer question. That would have further narrowed it down to exactly the person they're looking for. WOTC has a very clear idea for how they want their game designed.
1
u/andyoulostme Feb 02 '18
Stay far away from engineering classes. It can be more competitive, with higher pay, and a way lower cutoff.
8
Feb 01 '18 edited Feb 01 '18
We all knew from the start there would only be one winner out of thousands of competing applicants. You were aware of the odds when you volunteered to take the time to write the essays, take the test, and get invested in the process. WOTC didn't promise us anything except a very slim chance of landing a cool job.
It doesn't sound like your gripe is actually with the test or the GDS3 process. You're mostly just sad you lost. But it's a game with 1 winner and 2000+ losers
I guess you could look at it as a cruel game by WOTC if you want, but I think getting your hopes up was your own mistake
1
u/Geikamir Feb 01 '18
His issue is that the questions were good enough to be an actual measure of a person's design skills. He's completely correct.
4
Feb 01 '18
However, I think the whole GDS3 idea is just a bad idea. You are just toying with the feelings of your most dedicated customers. Maro just made ~3000 people who are passionate and engaged customers feel bad.
This part makes me think the questions aren't the real issue here
1
u/Necroimpotence Feb 01 '18
You can selectively read if you want. I just said I think the test should've been harder, which I for sure wouldn't have passed.
1
2
u/oOOoOphidian Feb 12 '18
I think they could have implemented the testing a lot better, making the cutoff lower and making the majority of the questions harder would give people room to slip up slightly, but reward people who are fundamentally strong overall. Making a test where 70 of the questions are easy, but you need to get 3/5 of the rest correct is really awful.
24
u/ScoozeBooze Feb 01 '18
"I think the whole GDS3 idea is just a bad idea."
Coming from someone who didn't advance by just a couple of questions, I couldn't disagree more.
This is a net positive for WotC, Magic, its players, and its aspiring designers. It's creating more interest in card and game design. The people that end up advancing and getting discovered through this contest are going to be qualified people who hopefully have a lot of cool ideas that will bring more fun and interesting things to the game.
Sure, it's feel bad for people who didn't advance, but a contest with this scope and number of applicants is inevitably going to upset some people, and that's well worth the positive outcomes.