Some­times when we’re try­ing to hire a se­nior em­ploy­ee, I get asked to do a “sell call”, tell them what it’s like to work here. Since I’m com­ing up on three years and haven’t quit, I guess the sup­po­si­tion is that I’ll be pos­i­tive. Since these can­di­dates are out­siders and some of them don’t come to work for us, noth­ing I can say can be a se­cret. So why don’t I tell ev­ery­one?

Be­fore I dive in any fur­ther, ev­ery­thing here re­lates to AWS, not Ama­zon as a whole. It might be true of the re­tail side too, but I don’t know be­cause I don’t work there.

So, what is it like to work here?

Cus­tomers · That’s the big deal. AWS has over a mil­lion and you get to meet them all the time, and while there are lots of geek in­tro­verts who might not like that, I do. Any­how, lots of the cus­tomers are geek in­tro­verts too.

When you ship low-level hor­i­zon­tal general-purpose tech­nol­o­gy, there’s no way to pre­dict what peo­ple are go­ing to do with it. Which in prac­tice means that you’re go­ing to be shocked by the way it gets used. For an old guy like me it’s hi­lar­i­ous to hear the kid en­gi­neers look­ing at a bug re­port or fea­ture re­quest and say­ing “WTF, why would any­body want to do that?”

Me, I like com­put­er­s, and I like hear­ing about in­ter­est­ing things peo­ple do with them, and it makes me hap­py when the things they do work out well, es­pe­cial­ly when I helped build what they’re us­ing. I could tell you sto­ries that’d curl your hair (or straight­en your wavy lock­s) but they’re not mine to tel­l.

Any­how, if you like talk­ing to cus­tomers you’ll like work­ing here, and if not, prob­a­bly not.

Up­time · It mat­ters more than any­thing. Some of our ser­vices are cool­er than oth­er­s, but what I think cus­tomers care about most is con­fi­dence that the ser­vices, cool or bor­ing, will be there 24/7/365. What that means is that ev­ery­thing has to be au­to­mat­ed, and much of the most bril­liant en­gi­neer­ing at AWS, done by some of the smartest peo­ple, does its work be­hind the scenes where no­body will ev­er see it.

So if you’re the kind of per­son who, for ex­am­ple, thinks fig­ur­ing out a bet­ter way to au­to­mate de­tect­ing hot-spots in back-end clus­ters and re-routing traf­fic to cool things down is in­ter­est­ing work, then you’ll like work­ing here. If not, maybe not.

On­ly maybe not, be­cause we have lots of peo­ple who wran­gle JavaScript frame­works and mo­bile SDKs and build tools and database ker­nels all day. But op­er­a­tional au­toma­tion, that’s the heart of the mat­ter.

Be­ing document-driven · Go ahead and Google “Amazon six-pager” if you haven’t al­ready.

If you’re the kind of per­son who’s OK with spend­ing a lot of time con­struct­ing carefully-written nar­ra­tives, and be­ing in meet­ings that start with 20+ qui­et min­utes while ev­ery­one reads the nar­ra­tive, you’ll like work­ing here, and if not, def­i­nite­ly not. Dis­clo­sure: I’ve writ­ten over a mil­lion words on this blog so I’m se­ri­ous­ly pre­dis­posed to like this part.

There is one down­sid­e: Sup­pose you’re an au­thor of the doc­u­ment be­ing read. It can be si­mul­ta­ne­ous­ly crush­ing­ly bor­ing and twitch­ing­ly ner­vous while AWS’s se­nior lead­er­ship plows their way through your words in stony si­lence.

The ac­tu­al soft­ware · I don’t think any­one would be sur­prised. The ser­vices are usu­al­ly con­ven­tion­al REST­ful ser­vices, usu­al­ly com­posed of mul­ti­ple mi­croser­vices, usu­al­ly in main­stream pro­gram­ming lan­guages us­ing main­stream pro­gram­ming frame­work­s, We use con­ven­tion­al com­pil­ers and de­bug­gers and IDEs and frame­works for HTTP and unit test­ing and in­te­gra­tion test­ing and de­pen­den­cy in­jec­tion and so on.

Yes, there’s some se­cret sauce voodoo down in the in­fras­truc­ture that’s pret­ty mag­i­cal. But most of us don’t work on that most of the time.

What’s un­usu­al is the pro­por­tion of the code fo­cused on avail­abil­i­ty and da­ta dura­bil­i­ty. But you al­ready knew that. We al­so care a lot about do­ing bet­ter than O(N), be­cause N is typ­i­cal­ly so freak­ing huge at Ama­zon. I per­son­al­ly seem to end up work­ing most­ly on mes­sage pro­cess­ing of one kind or an­oth­er which is sort of stuck on O(N) in the num­ber of mes­sages, so we’re of­ten left with micro-optimizations.

You know what makes me hap­py? One lit­tle li­brary I wrote is cur­rent­ly be­ing used by sev­er­al dif­fer­ent teams to ef­fi­cient­ly do a use­ful thing on a million-ish mes­sages per sec­ond in ag­gre­gate. There aren’t that many places where your code gets that kind of op­por­tu­ni­ty.

Or­ga­ni­za­tion and cul­ture · The teams at AWS have a huge amount of in­de­pen­dence. If you’re on a team that’s try­ing to ship some­thing or op­er­ate some­thing or im­prove some­thing, that’s gen­er­al­ly great; no­body will get in your way. On the oth­er hand, if you’re try­ing to build some­thing that works across mul­ti­ple ser­vice team­s, it can give you grey hairs. But I al­ready had those; and it’s most­ly a fea­ture not a bug.

Cor­po­rate “culture” is a thing I find it hard to be ar­tic­u­late about. I find the ass­hole den­si­ty nonze­ro but low­er than av­er­age. We are try­ing to do very dif­fi­cult things and we fail a lot; Ma­ture tech­nol­o­gy com­pa­nies try make those teach­able mo­ments, and we do a pret­ty good job of that. There is [*­gasp*] pol­i­tic­s. I’m go­ing to ar­gue that the pro­por­tion of re­al­ly bad man­agers is un­usu­al­ly low; but that’s just an anec­dote, wouldn’t know how to mea­sure it.

My time rarely feels wast­ed.

Money? · If you work in a tech job at any large high-tech com­pa­ny, you’ll be well-paid by any rea­son­able stan­dard. Want more than just “well-paid”? The bru­tal truth is that the way high-tech com­pen­sa­tion is struc­tured, you’re mak­ing a bet on your employer’s share price. If it goes up re­al­ly a lot while you’re work­ing there, you’re gonna make out like a ban­dit, and if not, not. All these things are as true at Ama­zon as any­where else. Wel­come to twenty-first cen­tu­ry cap­i­tal­is­m.

What suck­s? · After I’ve talked through all this stuff, the per­son who might come to work for us wants to hear about the down­sid­e. Fair enough.

I guess the biggest one is that we’re not per­fect at op­er­a­tional au­toma­tion, so ev­ery­thing we fail to au­to­mate, ev­ery com­bi­na­tion of com­pute and net­work and stor­age fail­ure we weren’t smart enough to auto-remediate in ad­vance, has to be dealt with man­u­al­ly. Which means peo­ple have to be on-call and yeah, some­time the phone goes off af­ter mid­night.

The good news is, we’re get­ting bet­ter, learn­ing how to build au­toma­tion in at the core. Which means that the fresh new AWS ser­vices are bet­ter at self-maintaining and self-healing, so on-call sucks less. On the oth­er hand, the old­er ser­vices tend to have big­ger teams and thus less on-call time per per­son, so it evens out.

But there are some peo­ple who suf­fi­cient­ly dis­like the prospect of that oc­ca­sion­al wee-hours page that they just don’t want to stick around with us. Fair enough.

I dis­like our office-automation se­tup, but then I’m the kind of per­son who re­al­ly likes Google Docs and Cal­en­dar and Gmail, so maybe I’m an out­lier.

One more thing: Most jobs at AWS, you’ll nev­er be able to ex­plain to civil­ians what it is you do for a liv­ing. I gen­er­al­ly say “I help keep the In­ter­net running” and you know what? That’s not a to­tal lie.

On bal­ance · I could go on about di­ver­si­ty and work en­vi­ron­ments and ben­e­fits and pro­cess and pro­mo­tion and so on, but at the end of the day it’s just an­oth­er high-tech com­pa­ny, noth­ing sur­pris­ing.

I don’t re­al­ly need the mon­ey, but I haven’t quit.



Contributions

Comment feed for ongoing:Comments feed

From: hawkse (Oct 27 2017, at 10:30)

That six pager stuff sounds delightful. I'd be happy to get management to read even two Powerpoint slides as it is!

[link]

From: Chris Simmons (Oct 27 2017, at 12:05)

If all this sounds awesome but you don’t like the talking to customers piece, I’d recommend the retail side of he company. You still need to manage stakeholders and be customer obsessed, but from what I’ve seen AWS has much more direct customer interaction.

[link]

From: Robert de Forest (Oct 27 2017, at 13:34)

Well said. I've been at Amazon for five years, half in retail and half in AWS. I agree completely both with your representation of life at Amazon and your assessment of their relative priorities.

So there's at least two of us I guess?

[link]

From: Justin Nichols (Oct 30 2017, at 12:18)

So how many hours do you typically work in a week? This is not mentioned at all in your write-up and is usually one of the top things people complain about when they post about what it's like to work at Amazon.

[link]

From: Mike C (Oct 30 2017, at 14:17)

Well said, though I think you're understating how abysmal the "office" aspects are. You get an underpowered laptop and have to use email and calendar tools from the 1990s with it. Getting email and calendar access on your phone is non-trivial and non-reliable. You see your customers (who you *do* get to interact with a ton) using modern tools like Slack and Google Docs, but you're stuck in the stone ages when it comes to the tooling you're permitted to use for your work. I honestly think this would be a huge recruiting detractor, but thankfully it's not advertised all that much, so by the time a new employee finds out, they're already in.

[link]

From: LauraC (Oct 30 2017, at 21:39)

Interesting read and the first positive spin I've read or heard on working for Amazon. I worked for another retailer in Seattle who has a different set of problems. During my time in Seattle I met a large number of Amazon IT folks. All were very careful about what they said but they made it clear the place was a sweatshop. Not once did I ever hear people declare it was a great place to work or that they liked their job. Perhaps things have changed in the last couple of years. The fact that they still don't allow telecommuters is going to remain a turn off for many of us.

[link]

From: Tony (Oct 31 2017, at 06:34)

That actually sounds like fun. Confession time. One of the highlights of my job is that I get to interact with the Sales and customer service people here at the company. Most of them are nice outgoing people whose company I enjoy (for most part). One of my projects was making an internal website that was optimized for speed. So I got to work with friendly people and I got to performance tune something I wrote. A win on all counts :)

[link]

From: David Magda (Oct 31 2017, at 09:25)

An observation from Bryan Cantrill:

<blockquote>

All of these [fracking] leadership principles, in all of these organizations, where is integrity? God damn it, where’s integrity? Amazon has fourteen leadership principles, and integrity is not on it. Inexcusable. You’ve got one principle in your organization and its integrity. Right?

</blockquote>

https://www.youtube.com/watch?v=bNfAAQUQ_54&t=42m7s

He mentions Amazon specifically, but only as an example of a larger problem he sees in the tech industry.

[link]

From: John Cowan (Nov 01 2017, at 10:05)

When I was a kid engineer, the vendor's folks were always saying "Why would you want to do that?" and it pissed me and my boss off no end. Our reply was "We pay for it, we get to do what we want with it."

[link]

author · Dad · software · colophon · rights
picture of the day
October 26, 2017
· Business (108 fragments)
· · Internet (103 more)

By .

I am an employee
of Amazon.com, but
the opinions expressed here
are my own, and no other party
necessarily agrees with them.

A full disclosure of my
professional interests is
on the author page.