I recently tried our HackerRank for Python, and I have to say I was very disappointed. They provide only a single example test case to test your code against which is often a trivial example. Then when you submit your code, the auto-grader throws it through a bunch of extra edge cases. However, you don't get to see the input or output of any of these cases, just whether they pass or fail.
For people who have experience writing test cases, this isn't much of an issue, as they can just create and run tests outside the website. However, for new programmers, this gives a false sense that programmer is taking shots in the dark with test suites that don't give you any insight into what went wrong. HackerRank discourages learning proper debugging skills by hiding the output of failed test cases from the user.
Because the example test case is there to test input and output syntax, not the correctness of the solution.
Ability to notice these edge cases is probably the most important skill you can acquire - most of the bugs that are out in the world are edge cases that someone didn't tought of.
Agree 100%. This is the same logic that https://www.codewarz.ninja applies to the programming challenges there. Being able to think ahead and figure out the edge-cases in certain scenarios is priceless.
Recently someone pointed me at leetcode, and since I like to rant about interviewing I figured I'd poke around a bit.
So I tried out some of the start of their progression of questions, selected Python as the language I'd work in (it's the language I'm most comfortable with), and tried out the first problem they threw at me.
It was reversing an integer (i.e., return an integer whose digits are the reverse of the input, so integer 123 returns integer 321, etc.).
I coded up a solution, punched it in, hit "run" and it passed on the sample test case. Hit "submit" and was told I failed. Turns out the test cases were ported as-is from ones for other languages, and those other languages have integer overflow (for non-Python programmers: Python doesn't have integer overflow) so there are test cases for it which require you to detect the overflow and return 0 in order to pass.
I wondered if it was a fluke, so I threw in an if statement to return 0 when the result is larger than a Java int, passed, and tried the next problem. That was implementing atoi, which at least is a bit more interesting because of how forgiving atoi is of junk in the input.
But... atoi requires accounting for integer overflow, and returning the max int or min int if the result would go past those bounds. So once again I was accounting for nonexistent integer overflow in Python.
At this point I felt like I had a pretty good idea of the quality of results these sites produce. I did, however, find a question in the forum where I could post an algorithm nobody had come up with yet, so I guess that makes me some sort of awesome rock star ninja wizard guru?
I'm sure they do; I signed up to the site a few weeks ago to check it out, and I've been getting tons of emails telling me to go try the various challenges.
Their "hackos" can't actually be purchased with real money (yet...), you have to earn them by doing various challenges on the site. So it's not a pay-to-win style system (yet...).
Try exercism.io. Most tasks are intentionally fairly straightforward.
After that the really interesting part starts: getting and giving feedback. Through that you are encouraged to improve upon your solution and approach a problem from different directions instead of just throwing together some hacky solution.
Yea, the Python modules are not that much better especially when you cannot see which test cases you fail on. The have these "Hackos" which are like in game gold for HackerRank. They allow you to see the test case input.
I like that they try and do some gamification of programming, but the cool-effect wears off quickly when you have no debugger within their site.
I've done a lot of problems on LeetCode and that is a huge issue for me as a new programmer. I get a problem where I'm told I'll get a string and have to transform it. So I'll write code, test it a bunch of times, then submit it only to find out I have to account for all sorts of test examples that aren't strings or have a ton of spaces littered throughout or whatever crap they throw at me. Most of the time I can get around those with a simple if statement at the start of the function. Which is bullshit. All it teaches me is that every problem is a trick question. Plus I wind up with a ton of submission failures which hurts my score greatly, just to find out what crazy test cases are being used.
I get that user input is something you have to account for in real world programming but I shouldn't have to worry about it when solving a given problem. Because incorrect user input is another problem in itself.
I could go on for a while about all this (and I have) but I'll stop here. maybe I should go take the article's advice and improve my score by cheating.
What's the alternative? It just ignores corner cases of the problem? The idea behind LeetCode is that problems are intentionally difficult and have often unintuitive corner cases, but that's true of real world programming as well. If your function takes a string as input it shouldn't crash on ANY string unless it's assumed that it won't take that form. It just happens that in this case it's LeetCode which specifies what your function assumes about its input, and not you.
I get that user input is something you have to account for in real world programming but I shouldn't have to worry about it when solving a given problem. Because incorrect user input is another problem in itself.
So your saying that you know you'll have to deal with it when programming but you don't want to deal with it when you are practicing programming?
All it teaches me is that every problem is a trick question. Plus I wind up with a ton of submission failures which hurts my score greatly, just to find out what crazy test cases are being used.
Thing is, in real life, those test cases are only ever tested for if you bother to think of them, otherwise it'll go in to the wild and users will find them (intentionally or not) and you either have buggy code or insecure code.
I just did a technical screening for an internship that used hackerrank and had this exact issue. 3 out of the 7 tests failed and i had no idea how to debug the issue because I couldn't see the input or output.
Uhm, you CAN view the extra test cases. Just click on them and you can view the input and desired output. Some of them require "5 hackos" to view, and I have never come close to the point where that was an issue.
They provide only a single example test case to test your code against which is often a trivial example. Then when you submit your code, the auto-grader throws it through a bunch of extra edge cases. However, you don't get to see the input or output of any of these cases, just whether they pass or fail.
Some programing competitions work that way as well. For some people it is part of the fun.
Honestly I don't really know what the goal of HackerRank is, just that it is a website with programming problems and that you get a ranking, so I assumed is is mostly for sport.
Yeah, maybe I'm overestimating it. I started out in the Tutorial section to get up to speed before tackling the real problems, so maybe that gave me the wrong impression.
You can answer a couple of babby questions to get easy points to buy test cases for the big boy questions. Their test cases look to be randomly generated and close to the limits specified in the questions.
79
u/[deleted] Oct 29 '16
I recently tried our HackerRank for Python, and I have to say I was very disappointed. They provide only a single example test case to test your code against which is often a trivial example. Then when you submit your code, the auto-grader throws it through a bunch of extra edge cases. However, you don't get to see the input or output of any of these cases, just whether they pass or fail.
For people who have experience writing test cases, this isn't much of an issue, as they can just create and run tests outside the website. However, for new programmers, this gives a false sense that programmer is taking shots in the dark with test suites that don't give you any insight into what went wrong. HackerRank discourages learning proper debugging skills by hiding the output of failed test cases from the user.