The oth­er day, I joined a semi-viral tweet chain with I’ve been cod­ing since 1979 and I still have to look up java.lang.String meth­ods all the time. There were a bunch of pro­gram­mers do­ing this and I thought it con­sti­tut­ed amus­ing hu­mil­i­ty while al­so mak­ing a use­ful point: Re­mem­ber­ing the de­tails of any par­tic­u­lar API or al­go­rithm is ir­rel­e­van­t. Turns out I was part of a trend, see TheOut­line: Pro­gram­mers Are Con­fess­ing Their Cod­ing Sins to Protest a Bro­ken Job In­ter­view Pro­cess. Ex­cept for, that’s bull­shit; I still do white­boards at in­ter­views, and I don’t think the idea is bro­ken. (Al­so, they’re not sin­s.)

Does this mean I ask peo­ple to code a bub­ble sort? Or to show any ev­i­dence that they re­mem­ber the de­tails of any par­tic­u­lar API? Ob­vi­ous­ly not: As many have point­ed out, that’s what Wikipedi­a, Stack­Over­flow, and (most of­ten) my IDE’s auto-complete are for.

Th­ese days, when I’m in­ter­view­ing for senior-developer and development-manager po­si­tions at AWS, my ques­tions are of the form “Design X”, where X is some­thing like Twit­ter or SQS. The white­board is an ap­pro­pri­ate lev­el of de­tail; I want to look at the box­es and ar­rows they draw and see if they come up with a sane de­com­po­si­tion of the prob­lem, and if they spot where the hard parts are. Some­times, if there’s enough time, I might ask them to sketch in a bit of front-end code, but usu­al­ly not.

Back when I was at Google, I was most­ly in­ter­view­ing peo­ple for “Developer Advocate” po­si­tion­s, and a lot of peo­ple some­how got in­to the pro­cess with­out be­ing able to code at al­l. So, ear­ly on, I’d ask. “You’ve got a list of ob­ject­s, write some code to se­lect one of them at ran­dom. Any lan­guage, don’t wor­ry about syn­tax, as­sume the built-in ran­dom func­tion is good enough.”

That was ac­tu­al­ly a nice ques­tion: If you want­ed to dive a lit­tle deep­er, you could ask the can­di­date to sketch in unit test­s. And if you’re talk­ing to some­body super-technical, ask “Your code is in pro­duc­tion and some­times it’s throw­ing illegal-index ex­cep­tions un­der heavy load. What’s go­ing on and how do you fix it?” Just be­cause that’s a cool prob­lem, very real-life, and most peo­ple smile when they get it.

Ap­par­ent­ly DHH start­ed the trend by ad­mit­ting he couldn’t code a bub­ble sort on a white­board, and I think we can all agree that would be a to­tal­ly dorky in­ter­view ques­tion. But TheOutline’s piece goes fur­ther, al­leg­ing that white­board­ing is anti-diversity. See Aline Lern­er’s ex­cel­lent (and data-rich) You can’t fix di­ver­si­ty in tech with­out fix­ing the tech­ni­cal in­ter­view. It most­ly ar­gues that “logic puzzle” in­ter­view ques­tions are bull­shit, and I hearti­ly agree. By the way I got one of those in my in­ter­view day at Google, and an­oth­er at Ama­zon, and I blew them both.

At AWS, these days, I’m on a few hir­ing loop­s, and I don’t see any­one ask­ing log­ic puz­zles. But I have no idea what the state of the art is these days at the big tech com­pa­nies; are there any stud­ies?

I think and hope that the way I use the white­board doesn’t make me part of the di­ver­si­ty prob­lem, and I’ll be watch­ing out for da­ta on the sub­jec­t.



Contributions

Comment feed for ongoing:Comments feed

From: eerie quark doll (Mar 04 2017, at 12:09)

I've experienced Amazon and Google in recent times; logic puzzles at neither, but definitely white board coding of methods and classes.

On one hand, sure they have to do something; on the other hand, writing code on a whiteboard during an hour-long-interview is ill-suited to people who either don't write quickly, or whose design cycle usually features a first pass naïve implementation followed a second pass optimization after window and navel gaze.

I feel like if the candidate is obviously not incompetent through dialog and design discussions, has a personality that the interviewers get along with, and has a long successful working history, then that should suffice for a hire.

[link]

From: Paul Hoffman (Mar 04 2017, at 12:56)

"You’ve got a list of objects, write some code to select one of them at random."

"...you could ask the candidate to sketch in unit tests".

I'm fascinated what kind of unit tests you would expect for selecting from a list at random. :-)

[link]

From: Bob Monsour (Mar 05 2017, at 17:25)

I interviewed for a product manager role at Google back in 2015. Among the interviewers (all one-on-one), the engineer asked me to sketch out a bubble sort. I hadn't coded in a while, but recalled it conceptually. He seemed fine with it. One of the product managers threw a queueing theory question at me, in a particular context, I got through it ultimately, but stumbled a bit.

I didn't get the job, but I don't think it had anything to do with these two aspects. The two phone screens with senior product managers went well (2nd better than the first; but then I think that's why there was a 2nd).

I will never be sure if my age had anything to do with it (I'm your age) as I believe I still have a lot of energy and don't quite look my age.

I think whiteboards are a great tool for dialog and I think that the approach you describe makes a lot of sense, for both technical and product people.

[link]

From: you dont know me (Mar 07 2017, at 12:16)

you're missing the point, and possibly its because you havent been in the interviewing trenches (as a candidate) in a very long time.

I interviewed at Google about 18 months ago. Out of the 5 white-board interviews they conducted only 1 of them matches what you described (conceptual breakdown of a software design problem), and even that one still got pretty heavily involved in algorithm minutia once we talked through the high level design. here are the other 4 questions they asked me

1. implement (in syntactically correct code) binary tree sort for trees where each node contains a 3-tuple. Sort based only on the 2nd element of the 3-tuples.

2. implement (in syntactically correct code) an algorithm to determine if input string A is a substring of input string B.

3. implement (in syntactically correct code) a peer-to-peer payment settlement system where a group of N people who each have different amounts of money settle purchase orders for K items made by the various members of the group.

4. implement (in syntactically correct code) a game where Player 1 guesses letters such that the minimum number of letters is guessed to select (from a dictionary of ever word in English) a word chosen secretly by Player 2.

and no, they weren't interested in talking through the design concepts of the problem. passing the interviews meant having working code within 30 minutes. bonus points if it's also optimal in terms of big O time complexity.

this is what whiteboarding interviews are really like. being asked to implement Bubblesort would be a mercifully easy softball of a question.

just because you're not committing the worst offenses in your own interviews doesn't mean they aren't still widespread in practice. the worst part of this is how many companies get Google envy and decide that they should copy the Google interview technique. I interviewed for a second tier e-commerce company (food delivery industry) that conducted interviews just like Google, but they weren't doing massive scale systems or working on novel problems. They had a Ruby on Rails powered e-commerce website. It's all a sick joke.

[link]

From: Adrian Petrescu (Mar 07 2017, at 12:32)

@PaulHoffman: I imagine it looks something like "sample the function a lot of times and measure the distribution of objects you get back", along with the normal edge cases of small-to-empty lists, duplicate objects, etc. Then you can dive a bit deeper on how many samples you'd need, and how much variance to accept, etc.

Seems like exactly the sort of thing where unit testing it is non-trivial enough to be interesting.

[link]

From: Anonymous Opinion (Mar 07 2017, at 13:09)

Dunno if it counts on the diversity side but whiteboarding does certainly select against those with social anxiety and/or introversion. (Rather ironic for an industry where that is the popular stereotype for its members.)

FWIW, I've been in the industry for 20 years and I still get "whiteboard anxiety" attacks and bomb interviews.

[link]

From: John Cowan (Mar 08 2017, at 09:31)

I've done coding phone screens (using CollabEdit) and interviews with a whole bunch of intern candidates this year, and in each and every case I make it clear that I'm *not* a compiler. I don't give a damn about syntax errors or forgotten method names. I don't even care what programming language they use (as long as I know something about it).

Instead, I look at whether they explain what they are going to do before (or as) they do it, whether the code organization is coherent, and whether they get bogged down in complexity. I try to ensure that each interview is a learning experience, definitely for the candidate and preferably for me.

[link]

From: Gavin B. (Mar 17 2017, at 09:43)

A somewhat satirical take

https://aphyr.com/posts/340-acing-the-technical-interview

[link]

author · Dad · software · colophon · rights
picture of the day
March 04, 2017
· Technology (80 more)

By .

I am an employee
of Amazon.com, but
the opinions expressed here
are my own, and no other party
necessarily agrees with them.

A full disclosure of my
professional interests is
on the author page.