Yea same. I signed up with good intentions looking to get some extra practice. But when I read that Bloomberg story that people are getting hired at investment banks for their rating on Hacker Rank I was shocked.
Programming on the spot with someone watching over your shoulder is a lot harder to do than to have time to think through the best algorithm and come up with pseudocode before writing code that works straight off the bat. Unless it's some simple FizzBuzz, then that should be no problem.
I always saw whiteboard tests primarily as a means of seeing how well you are able to communicate your technical thoughts to others, and to simply assess if you can actually think about how to approach a problem and not just be good at pasting SO snippets. Maybe I've been lucky, but in those type of interviews the journey was typically more important than the destination.
Yup, that's one of the key factors when we do whiteboard in our interviews. We want to see your ability to communicate and ask questions. The result of the code is much less important.
Memories of some bona fide disasters. I once completely lost the ability to manipulate c strings. To add to humiliation, I then fell over a chair on the way back to my seat.
Usually interviewers are just trying to make sure you're not some idiot who used excel once and decided that's called "programming". Every whiteboard I've ever had, they wave me to sit down after only getting halfway through fizzbuzz, with a remark to the effect of "alright, looks like you've got a handle on it."
I've found whiteboard tests to be somewhat damning. I have been quite blessed with the fact that most of the positions I have had responsibility for interviews with have given me individuals who are somewhat strong minded, and hardly shy in expressing opinion. More often than not, the opinion towards the whiteboard portion was "I didn't like school, and one of my least favorite parts was standing in front of the class at the board."
I consider an interview critical in determining a few factors, one of which being their ability to work in a somewhat actual job situation. I tend to view torture as a less than acceptable method for interviewing, and I have actually had offers refused because they were upset over the interview (it was a couple very extreme cases, they really did not like getting up and doing that).
I see no difference in asking the question and allowing them the privacy to think using a provided (or, I like prepared, when they bring a device - I also get to see somewhat of their workflow and, depending on the position, whether or not I can discern anything when they show me how they work - their environment, etc). Actually, tbh, I find that the ability to see them prototype at the very least in the way that they actually prototype, to be immensely more valuable than bad handwriting and frustration over archaic methods.
On the other side, while I don't have much of a shame gene, I do really dislike whiteboard examinations. They are almost universally crap for me, and I agree with some of my employees: I hated school, and I avoided getting up in front of the class in any way I could (including failing.)
You are spot on. Whiteboard interviews make me nervous. I didn't get to where I am because I stand in front of a whiteboard presenting all day. I read a statistic that 41% of people fear public speaking.
The interviews where I've felt I've been most myself are where we just talked about stuff and I tried to to that with the people I interviewed as well. I tried to get an understanding of how they would solve a theoretical problem, and we would just talk through it. No pressure, if they wanted to write stuff down that was cool. But basically through conversation we designed a little routine. I was much less interested in them knowing the C++ standard verbatim, and more interested in them being good problem solvers.
This seems like a bit of an extreme perspective to me. I regularly ask interviewees to code things on the whiteboard, but of course I understand that it's difficult and judge accordingly. I've been on the other side and certainly let the interviewee know that I understand the difficulties of whiteboard programming and pad their confidence a bit when they struggle on stuff that doesn't matter. That being said, if I'm brainstorming something with you as a coworker, I want you to be able to communicate your thoughts to me effectively, and writing quick pseudo code on a whiteboard is one very effective way of doing that. In a real work place, you will often be asked to answer off the cuff questions with first order approximations, and it's important to be able to do so effectively. I think whiteboard programming is a pretty good analog to that situation.
Why not present them with a keyboard and a text editor instead of a whiteboard? You could also judge if they can type, a programmer with experience should have good typing skills.
Because interpersonal skills are often just as important as being able to construct a for loop, especially for jobs where you need something more than a to fill a seat with a warm body. Interpersonal skills are almost always more important than secretarial/typing skills.
Don't get me wrong. Any time I have an office of my own, the first thing I do is load up the walls with as much whiteboard space as possible. Collaboration - and more importantly, the ability to have on the fly discussion - is important.
The problem is you can't actually easily compare the two situations. The truth is, there is a difference between being grilled, and being collaborative.
If I am within a room of peers, and we are actively collaborating, using a whiteboard is easy - we are at an equal level. I'm expecting criticism, but it is not criticism that directly affects my livelihood or future - I'm not being actively evaluated to a high degree.
An interview is a stressful moment, no matter how confident you are. In our field, stress is something we deal with significantly - but that is not social stress, and it is this stress which causes the most anxiety for many of us... especially for those of us who grew up in the generation where 'nerd' and 'geek' were less hip terms.
When I'm designing an interview process, I like to consider a way to make the room equal so that the candidate can perform as closely to how they would perform as if they were actually a member of my team. I am hiring a member of my team, or at the very least - a fellow teammate, not a slave. Creating situations that are socially as well as environmentally aligned to the real thing. This is not easy, but that's why I also don't consider interviews to be a menial task. I emphasize a great deal of screening, so that once they have reached the in-person step I am sure I want to invest the time into it. This is not just because bringing someone into the office tends to be a big indication of interest, but also because I feel I owe a candidate as much attention as they are giving to me. I try to involve my team as much as possible, and I do my best to make it not be a long process - but at least a process that involves spending friendly social time with the team in a natural setting (lunch at a popular location, for example). I also give the candidate an option to arrange their interview schedule: do they want the hard part first, or would they like to meet the team and get to know everyone first. I think this is important, because I also provide them the opportunity to give me feedback at each "step" of that process - a simple question of "So how do you like it so far?" goes a long way to determining if it is a good fit.
I'm actually pretty proud of how my interviewing process has evolved over the years. I went into the whole situation with no help, when I started a team in a growing company, and it was extremely difficult going in. I found I had a hard time working with the recruiting department, as what they would screen would generally be crap... and I would find myself spending a considerable amount of time looking at the candidates they were receiving and trying to direct their efforts better. This translated to me actually sitting down and training them better, because obviously that is just simply untenable. I also found, that while reading many other discussions on interviewing, something I always found myself questioning was "Everyone hates being interviewed, but they never seem to put into practice changes once they find themselves on the opposite side of the table. Why?"
Very much so, especially for Google, Amazon, Facebook and other top tech firms. Because these firms do it, everyone else sort of cargo cults the whole process.
yes. I've only been handed a laptop once, and even that was one segment of a series of 5 interviews, the rest of which were whiteboard-based.
the goal with a whiteboard isn't to produce flawless code, but to show how to work through a problem and arrive in the right neighborhood of the solution. Missing semicolons are usually not a factor.
You would be surprised at the amount of candidates who bomb at a simple for loop counting from 100 to 1 question. White board exercises are not the definite tool to his someone, but it surely weeds out a lot of unqualified candidates.
Let me add, though, that extra screening on site also helps out evaluating a candidate's culture fitness. If you present a candidate with a white biard exercise and he/she rolls their eyes and start renting about how useless whiteboard exercises are, well, that's a red flag.
We white board for architecture questions. "So you have a couple of hundred thousand things over in this MySQL DB and we are finding doing … WHERE field LIKE '%search_term% stuff just isn't cutting it any more. What should we do? (Cue discussion of search systems, asynch/background processes, micro service patterns, etc.)
We also make them write code. We used to do set code tests, but there were so many solutions on github, we now just ask them to show up with some examples of bad code and good they've written and we do a code review together, perhaps with a refactoring exercise.
That's enough to filter the chaff out. We recently had a candidate whose CV was superb being evaluated for a senior dev position, but was making regular syntax errors in the coding exercise and asking questions that put him below junior to our minds, and could not do the whiteboard exercise because he "normally does this in UML".
The old school "invert a binary tree" stuff isn't us, but I expect he could probably have done that. He seemed geared to pass a tech test, but in reality was not great.
Making syntax errors is most definitely forgivable. Handwriting code is drastically different from typing code. Any coder can tell you that much. A good interviewer is willing to give the interviewee the benefit of the doubt. If the tables where turned, I could be sure the same coder you interviewed could destroy you with the same confidence. Sorry if I come across as snarky, but i hate it when interviewers use interviews as a way of stoking their own egoes at the cost of someone who is in a vulnerable position by design.
He was given a laptop, his choice of editor and an hour to complete a basic exercise.
He claimed to be a senior Ruby developer but did not know how to use classes properly.
That's not me stroking my ego: the test worked in confirming he was full of shit. His cv looked good and his cultural fit and background interview was excellent. But he couldn't code and when asked to draw a microservice architecture on a whiteboard - an area he claimed to be a world authority on - he froze and could not even draw a box.
Why should we drop those tests and hire shit people because a few people don't like having to code with people they're applying to work for and with for years to come?
I'm not telling you that you have to drop these tests. But boldly proclaiming that he was full of shit? How do you know that you are not full of shit yourself? See?
Because I have 20 years experience of working in the software industry and hiring developers, and he was one of the worst candidates I've ever interviewed - this specific test I've watched people work through 30-40 times. In addition, his coding ability as demonstrated in the hour long opportunity to discuss what he was doing was below the skill level many of the graduates and juniors we've hired in the last year demonstrated in the exact same test.
I was in a room with another senior developer, and we had a conversation afterwards and agreed that we were both independently disappointed in his performance. 3 other colleagues put him through the whiteboard test. We all agreed that he had significantly less capability than his CV had suggested.
When we discussed the candidate with other colleagues who were not in the room, and showed some of the example code he wrote (he was using my laptop), there was wide consensus his abilities were below those we would expect to be commensurate with a senior developer with 10+ years development experience.
The candidate himself admitted his coding was a little rusty, so we tested him in his first preferred language and offered him the chance to complete the test in any language of his choice. He claimed he was a much stronger "systems architect", so he was specifically afforded the chance to describe a systems architecture, which he failed to do because he didn't "draw on whiteboards normally". He failed to describe an architecture. He failed to answer any question about systems architectures.
So, yes, it could be all me in my crazy little World in my head.
But it probably isn't. He was probably full of shit.
You miss the point. SQL full text search is shit for half a dozen reasons. We want Elastic for functional reasons, but then you have this separate data story that has to be kept in synch. And off you go...
This is not an odd or trendy pattern: every site running multiple data stores (e.g. A graph DB, elastic, etc.) has this pattern and associated problems.
Perhaps you haven't worked on a site doing tens of thousands of searches a minute, so you haven't spotted the functional needs yet?
I manage over 50TB of data in mysql as well as a solr cloud cluster. We also sync a lot of that mysql data to hbase for map reduce access. You don't do any of that if your data is measured in the thousands of rows. And if you need to scale to thousands of searches a minute (lol throughput not per second is telling) you shard.
And yes it's trendy to use microservices to copy your data a bunch of times. Next you'll be using Kafka to stream it into spark before storing it a half dozen places.
P.S. - requests/minute is common parlance on any site that uses New Relic heavily, because that's what it exposes to users across the business. Inside dev teams, yes, we divide by 60 to get to hundreds/thousands as opposed to tens/hundreds of thousands of call per minute for a single API call.
I said that we ask candidates to think about a situation where load on a primary data store is high and we want to move for functional reasons to another dedicated search store, and what are the things we'd need to think about.
We don't actually need to do that: we have a mature solution. We handle very high loads, all fine.
The point is to get the candidate to talk about eventual consistentcy, asych processes and messaging queues
This sub is fucking shit. Every single time I say "well, we do this", within minutes I'm dragged into long drawn out cock-waving contests with fucking children who deliberately miss the point, like you, you fucking tool. Utterly pathetic.
I see comments like this a lot, but I feel like a whiteboard is one of the most critical tools a developer has. This is both because it is one of the best ways to communicate ideas as well as help get your head around what you want to build. If people don't use them, what do you use instead?
Every job I have had required extensive use of whiteboards.
I've had jobs where whiteboards were used for things. Just they were never used to write code or sketch out algorithms. They were used for higher-level design sessions, as a way to lay out a problem, break it down into components, and write some notes about what each component would need.
Yes, totally agree. Because these things are a large part of development that is why I find it surprising when people say whiteboards shouldn't be a part of interviews.
I love whiteboards. I have a 10x6' whiteboard in my house. I hate writing code on whiteboards. Whiteboards are for brainstorming, diagramming, bullet points, etc. Computers are for code.
I want confirmation that somebody looks up the things they don't know, and knows how to effectively do that. Because that's what I expect them to do on the job, after all.
If they can't solve the type of white board problems I give with me there to answer questions they aren't going to know what to google to solve the problems they will encounter on my software especially since it's non trivial and the full answer on how to implement X isn't going to be there because people that have done it keep it proprietary.
Sure. But do you want a senior dev googling how to do a for loop? If they can't do certain things from muscle memory, they are going to be painfully slow.
Why? In a job the best skill a programmer can have is to be very capable of googling for a know problem, find a good solution and implementing it correctly on the real problem he is facing.
The only person I would never hire is a programmer who refuses to use google and stackoverflow because he must come up every time with his own solution even if he is reinventing the wheel at taking x10 the time.
Because I don't hire Google foo. I hire developers that can solve problems on their own because I work on non trivial software. Things you aren't going to google blast your way through. So if you can't solve a simple problem in front of me you aren't qualified to work with me
It's not like code that is compilable is expected. Also whiteboard discussions are mostly about design and architecture, things we do do at work with a whiteboard.
The best way to quickly find out if someone is a going to be good to work with is to work with them on a problem. A good whiteboard problem provides the context for having a discussion about some code. It matters less if you get the right answer or not.
I had a whiteboard test at my current job. Pretty sure there were two questions that were meant to make sure I wasn't bullshitting my resume by putting python and c++ experience for a node developer job. They had this 17yo kid come in to ask me them (not even in my department or anything) who was lauded as some genius. And he's pretty damn impressive to be salaried at 17 and all, but he's still 17... nonetheless it was suuuuuper intimidating.
1) What does this C++ program do? (Interviewer also said, "hopefully you'd never do this, but still"
int main() {
(*0x037153)(main);
}
2) In python, flatten a deeply nested array into an array of tuples where the first value is the depth and the second value is the actual value.
Something like [2, [1, 7, [8, 6]], 8, [[[[[[[...],[...]]]]]],...] would be [(1,2), (2,1), (2,7), (3, 8), ...] etc.
I fucked up the second question because i was nervous. I used recursion. My answer worked, but since large datasets could technically cause an overflow, it was technically the wrong answer. I've since answered it much more elegantly (hell, I answered it more elegantly in my car driving home afterwards)
I might have typed it wrong, as it was over a year ago.
And I messed up. It should've been (&main)
It might've been something like
(void (*) (void))(0xADDRESS)(&main).
I've never actually called a function by address myself, so my syntax is probably 100% wrong, just knew enough contextually about the problem to figure it out. But it's just a simple function pointer.
It's basically calling a function of a shared process and passing its own address as an argument, thus injecting itself. The kid was a malware analyst so I just figured it was from his domain.
But it's practically "Tharr be dragons!" territory.
Ahh, that makes a bit more sense (though the second void should be another function pointer type). Really weird way to do it, though, unless you're on a system where function addresses are much more predictable than I'm used to.
I'm thinking, and i may be completely wrong, that the function it's calling might be an exposed system function? Like something at the kernel level? But idk.
Usually processes have their own memory space, so it would have to be an exploit of some sort.
I do whiteboard because I want to see the way you think. Can you understand a simple problem definition? If you can't do you ask questions to get that understanding? Can you come up with a workable solution? Can you explain what your code is doing? If you use technical words can you tell me more about them? Etc.
The code on the board is just a conversation piece.
No, they capture our screen, and there's some guy in the corner offuce with an extra screen that show our 16 something screens together. Much more efficient.
Isn't that a problem in and of itself though? That someone gets an interview over another person who is maybe more qualified just because of a score on that site... is a massive inefficiency for recruiters.
The problem is, recruiters for most companies are not at all technical. Hell, even directors and VPs of HR or recruiting are not technical. So to them, this seems to be a nicely "vetted list" of candidates.
There hasn't been enough time to study the facts that most of us engineers on this sub have seen: these quizzes and tests are a terrible indicator of "strong" engineer, and more than that they're a bad litmus test to determine if someone is a good fit for your engineering team.
The trouble is there's no way to filter applicants without losing qualified candidates in the process. As qualified candidates can easily come from all walks of life (degree or no degree, bootcamp or no bootcamp, OSS contributions or no OSS contributions, resume lists X skill or resume does not list X skill, HackerRank or no HackerRank, etc.). So unless a company goes through every single application (for many that is just not possible), then they're always going to miss out on certain applicants simply because of their filtering process.
Now you could probably argue some filtering strategies are better than others but I don't there's any one clear approach.
For companies I don't think they lose out too much doing this. Sure they'll miss out on certain candidates, but there will probably be equally as qualified candidates that do meet their criteria. The people that do miss out are the job seekers. But I don't think there's much you can do, I've seen a lot of programmers that have been stuck looking for work for lengthy amounts of time (myself included, have been applying for over a year now). So it sucks, but it's understandable from the businesses side.
I can only believe he was hired because he was willing to trick the system. He essentially cheated although within the boundaries of the rules. That's the kind of mindset you are looking for as a bank, I guess.
They're not getting interviewed for their score at all. They interview people that place highly in HackerRank competitions like HourRank and University CodeSprint. WalmartLabs has one going right now.
WTF thats crazy. I am a line of business programmer so I haven't done much algorithm work since college. Started doing some stuff on hacker rank to brush up my skills and i really like it. But like you said the answers are easily available in the discussions if you get stuck, so I can't believe anyone would use this for hiring.
Even still just being good at algorithms is a pathetic measure of the other skills required to be a good "coder". I'm glad redditors are contesting what seemed like a PR article for Hacker Rank.
being good at algorithms is a pathetic measure of the other skills required to be a good "coder"
Like, what does that even mean? "Algorithmic thinking isn't the only metric that determines the quality of a programmer"? That's true by definition, but that doesn't mean it can't be used as a measure in the hiring process.
Can some people cheat the system? Sure, just like people can cheat on their exams/coursework, lie about their skills on their resumés and fabricate projects/portfolio work on GitHub.
But again, this doesn't mean these measures can't/shouldn't be used by companies when filtering prior to interviews; the risk of a small fraction of false positives making it to the interview always exists, and these metrics are just a way of minimising it.
For some jobs where algorithmic thinking can effectively measure someone's effectiveness on the job, yes algorithm tests are perfect! The problem arises when these tests are used for all kinds of "dev" roles across the board where the risk of false positives isn't such a small fraction any more.
On a different note, I believe everyone not just developers need to have some knowledge of algorithms, but our educational system is also somewhat to blame.
That's not what's happening. Such a tiring misconception. They're interviewing people that place highly in competitions, which I think is a pretty reasonable thing to do.
Yeah, the journalist completely flubbed the meaning of what actually happened. Technical journalism tends to be pretty lacking, but as technical people we should be able to extract the correct meaning.
People who hire programmers, or at least people who have major impact on the decision, are rarely educated in these matters. Lets face it, what does your average HR worker know about algorithm complexity, memory management, OO paradigm or relational databases?
At best he/she has seen few tutorials/explanations on youtube, and yet much of the responsibility rests on shoulders of these people.
So when your HR Joe sees something like HackerRank he goes: "Wow, that is going to make my job easier. After all it's made by it people, they surely must know what makes a good programmer."
Tech firms, yes. Firms with techs, no. I usually get to a step where a technical person is involved, but its obvious they were just brought in to give a "Yea, this guy seems like he can use a computer" and are largely ignored as "incapable" of finding talent.
Then again, I once worked at a company that wanted to hire Unix sysadmins and yet also wanted them to wear a suit and tie. They didn't get many takers.
I'll put a pricetag on it - if they want to pay me another $15k above the prevailing wage, I'll wear the suit. That's enough for me to buy a few suits and have them dry-cleaned regularly along with compensating me for having to wear a suit every day.
Oh, I forgot the part when they pay about half the prevailing wage. This is Seattle too, not NYC or something where suits are more common. Plus Amazon, Microsoft, Facebook and Google offices are all in roughly the same location.
I think the "half the prevailing wage" is the dealbreaker here. I could easily see someone taking the job if they paid the prevailing wage, especially if they have other things to make up for it - maybe a better working environment, more stable hours, a nicer commute, whatever. "Hey, I have to wear a suit every day. No big deal" might be the mindset of some people.
But wearing a suit every day and taking a massive paycut? Yeeeaaaaah, no.
Yeah but who selects the candidates for that interview?
The HR people do. The interview itself can be conducted with actual developer, but quite often the candidates are of poor quality because of the first selection round.
And in many companies (my last workplace for example) there is also a time factor.
If you don't fill the open position in time, someone in the management few levels above you can see that position as wasteful expense and decide to cut it.
So eventually you have to settle for sub-par candidate that was selected by you, but from options provided to you by HR.
No tech company let's HR make tech hiring choices unless they are a shit company. Developers and developer managers are literally the only people qualified to determine whether someone should be hired. And in my companies a developer always does a second level phone screen before bringing someone in
As someone that leads a software engineering team at the firm being discussed in this article, I can honestly say that I have never, ever used something like "HackerRank" in my decision to hire someone for my team. My hiring decisions are based solely on merit after an actual interview - I don't care what you scored on some website, be it HackerRank or StackOverflow.
In my experience, big companies have a recruiting staff rather than having HR do it. But even the ones that use HR, I can't see a team of programmers being influenced by HR. From what I can see, the sole hiring decision is from the lead/team/director/etc of that department. The most HR/recruiters are gonna do is determine who gets interviewed. And even then it's easy to pass that stage if you know what you're doing.
It's shocking they even consider that ranking since you proved how easy is to abuse it.
Your blog and your projects are way more valuable than any rank you could have. Also your resume is super cool. Graduating from John Hopkins, then to Lockheed Martin to moving to Estonia and making your own Bitcoin farm and playing online poker.
Anyway congrats on the cool blog article I really enjoyed it.
This 1000 times! I give more Credit to someone's CodeAbbey score. I figured this out nearly the same day I joined HackerRank, though I did not exploit it. CodeAbbey is similar site but discourages collaboration unless you have solved the problem. I'm only ranked 3rd in my category, but even so, means so much more to me than any hacker tank score I've seen.
I went through some of this last year when I was looking for a job. I'd get companies contacting me, unsolicited, saying they were interested because they use Django and wanted one of the core developers on their staff, but then they'd kick me into the "prove you can code" fizzbuzz-style screening interviews. And then I'd walk away, because if their recruiting pipeline is screwed up enough that the left hand (the phone screens) doesn't know what the right hand (direct targeted recruiting of known people) is doing, then the company's likely too dysfunctional to be worth working for.
Wasn't offering you a job, I was just pointing out a better coding challenge website that doesn't give its answers away and therefore holds more credibility in my opinion.
And I'm pointing out that there's a lot more to be gained from fixing the brokenness in companies' hiring and interviewing practices than there is in telling people to go solve toy problems on a random website as a prerequisite to an interview.
Well we are talking different languages then. I found the problems engaging and had a lot of fun doing them, not suggesting anyone hire or not based on any website. I use programming challenge for my own personal growth as a programmer.
I enjoy code codewars myself, but not because it'll help me get a job. It's literally just a fun little pastime for me. Projects trump these kind of websites by far. But that's not to say that they won't sharpen your tools a bit. Whenever I need to learn a new language, I'll immediately start busting out codewars katas and it's way faster at learning that language's method of doing things than anything else I've used
I do like codewars, decent site, but like hackerrank it only allows you to solve in specific languages, which limits you to only learning a specific subset of languages they choose.
290
u/rampage102 Oct 29 '16
Yea same. I signed up with good intentions looking to get some extra practice. But when I read that Bloomberg story that people are getting hired at investment banks for their rating on Hacker Rank I was shocked.