What
 · Technology
 · · Software

On Sharding · If you need to handle really a lot of traffic, there’s only one way to do it: sharding. Which is to say, splitting up the incoming requests among as many hosts (or Lambda functions, or message brokers, or data streams) as you need. Once you get this working you can handle an essentially unlimited request volume. Of course, you have to make choices on how you’re going to divide up the traffic among the shards. I’ve had intense exposure to the options since I came to work at AWS ...
 
Go Creeping In · I’ve seen the inside of the Google and Amazon tech stacks. There are common threads that run through them and also, I bet, through most BigTechCos. Here and there down the stack is a lot of C++ and vestigial remnants from earlier days, Perl or PHP or whatever. Out in front of humans, of course, JS. But in between, there are oceans and oceans of Java; to a remarkable degree, it runs the Internet. Except for, here and there, you find a small but steadily increasing proportion of Go ...
[5 comments]  
Tech Office Sketches · Herewith notes from the white-hot center of the Internet software profession. Maybe the reality these reflect prevails across other technology territories, but I wouldn’t know. In no particular order: ...
[10 comments]  
JSON Event Scheming · I’m pretty sure that event-driven software is already a big deal and is going to get bigger. Events, de facto, are JSON blobs, and in general we’d like to make them easier to consume in computer programs. I’ve written before about how it’s difficult to specify JSON dialects, and also about Schemaless message processing. It turns out there’s good news from the world of JSON Schema, but the problem is far from solved ...
[2 comments]  
Where Yegge’s Wrong · Recently, Steve Yegge published two rants, each of which has gone viral and moved the Internet conversation needle a bit. I liked both of them, thought they were generally well-argued and useful. Unfortunately, there are a few assertions of fact there that are just wrong, and may be damaging. So I thought that, first of all, I should encourage people to go read these pieces and, second, point out the things that are counterfactual ...
[6 comments]  
Happy New Year! · Best of 2018 to you. But this isn’t a real blog post, it’s a placeholder to debug a subtle Unicode bug in my comment system that doesn’t appear on my staging environment. *sigh* ...
[5 comments]  
How To Sell Bitcoins · In 2013, I bought a few Bitcoins from a dude in a coffee shop, paying with hundred-dollar bills. Later that year I sold enough to get my money back. Then I forgot about them, Bitcoin’s price gyrations occasionally registering in a corner of my mind. But earlier this month I decided to find out if the remainder could be turned into real actual money, and it turned out they could. Here’s how ...
[10 comments]  
The Last JSON Spec · The IETF just published RFC 8259 (also known as “STD 90”). Editor, yr humble servant. The legacy-ASCII full text is here and there’s a nice-looking HTML verion here. I think this is the last specification of JSON that anyone will ever publish ...
[7 comments]  
Unapocalyptic Software · The Atlantic published The Coming Software Apocalypse by James Somers, which is full of dire warnings and strong claims. Here’s one: Since the 1980s, the way programmers work and the tools they use have changed remarkably little. My first programming job was in 1979, I still construct software, and I can testify that that assertion is deeply wrong, as is much else in the piece ...
[12 comments]  
I Don’t Believe in Blockchain · There are conferences and foundations and consortia and keynotes; it’s the new hotness! But I looked into blockchain technologies carefully and I’ve ended up thinking it’s an overpromoted niche sideshow ...
[18 comments]  
Message Processing Styles · Recently I’m thinking about how we process messages in networked software ...
[4 comments]  
Susan and her SQL Problem · As usual, it all started out innocently enough. Susan [ed: names have been changed to protect privacy] had no way to meet the deadlines her bosses had set for her. Bob had recently and abruptly left the company, and Melissa was on an extended medical absence, leaving Susan to do the work of three people. That is, three people each trying to reconcile a few dozen 40,000+ row Excel spreadsheets representing the general ledger of the Fortune 1000 company they consulted for. She was about to brush off ever-chatty and annoying Michael from Compliance when, for once, he recognized the stress she was under and said something useful ...
[4 comments]  
Another JSON Schema Gripe · Recently I wrote of my disgruntlement with JSON Schema. Since then I’ve learned that its authors plan more work, and that there are several other efforts to build a schema facility for JSON. This note is just a complaint about a particular use-case, with the hope that it might inform these efforts ...
[2 comments]  
Specifying JSON · I find myself tasked with polishing and publishing a little custom JSON-encoded language. It’s harder than it ought to be ...
[18 comments]  
Getting the Picture · It’s like this: Averages are your enemy because they hide change. Making graphs is easy and cheap and sometimes they uncover secrets; more of us should do it more ...
[1 comment]  
RFC 7464, JSON Text Sequences · Here’s a nice little RFC describing a nice little trick that might even be useful. Short form: People like to write JSON into logfiles. Text sequences make reading them easier and more robust ...
[8 comments]  
OSCON Rear-view · I spent three days last week at OSCON 2014 and enjoyed it. I’m not actually sure what OSCON means any more, but I hope it finds a way to stay vital ...
[2 comments]  
Discouraged Developer · I’m a software guy; I like writing code. Over the last decade, my niche has been a modest amount of coding and a lot of writing about it. The size of my audience suggests that this is a reasonably useful combination. But I’m kind of discouraged and stuck at the moment. I’ll get over it, but the story’s worth sharing ...
[27 comments]  
Java Pain · In 2014, it’s not OK if it’s hard for a developer to run a simple program from the command line ...
[16 comments]  
What Programmers Do · I contributed a morsel of code, connective tissue linking two moderately-popular pieces of publicly-available software. The technology and culture that enable this? They’re the water and computer programmers are the fish, we can’t see it any more. By an accident of history I could this time; and want to write about it ...
[7 comments]  
JSON Redux AKA RFC7159 · The IETF has just revised its JSON spec; the new version is RFC7159 — that link is to the IETF’s traditional line-printer format, I’ve parked an HTML version at rfc7159.net for people who want to actually read the thing not just link to it. [Disclosure: I edited RFC7159.] ...
[5 comments]  
Software in 2014 · We’re at an inflection point in the practice of constructing software. Our tools are good, our server developers are happy, but when it comes to building client-side software, we really don’t know where we’re going or how to get there ...
[64 comments]  
I Sold Some Bitcoins · I held the flimsy scrap of printout up to the Bitcoin ATM’s scanner, tapped its screen, and ten crisp hundred-dollar bills shot into the delivery tray at the bottom. Maybe Bitcoin is real? ...
[5 comments]  
Tab Sweep · The tabs! They multiply like magnificently miscellaneous maggots! ...
 
What Does “App” Mean? · I’ve been Web-centric for a long time, and generally thought of what programmers build as “sites”. Then I was in Android and what everyone developed was “apps”. Now I’m in a generalist role and, uh, a little unclear as to how to refer, generally, to what software builders build ...
[13 comments]  
Golang Diaries II · I still haven’t written a thousand lines of Go; but what I have created does useful work and (considering I had to learn a language and a bunch of libraries along the way) didn’t burn that much time. Herewith another batch of programming-permanoob reportage ...
[9 comments]  
Golang Diaries I · Some of the most popular things on this blog have been diaries I’ve written as I explore my way through a new technology or hobby. I’m picking up the Go Programming Language, whose HQ is (significantly) at golang.org and which I’m going to refer to as “Golang” for reasons which should become obvious. Whatever you thing of Golang, it’s apparently got some momentum and some seriously smart people are working on and in it; so this may actually be of service to the community ...
[30 comments]  
findIDP Tech Choices · I’m building a service that uses a bunch of heuristics to, given an email address, figure out which Identity Provider (IDP) you should try to use to log that address in. I’m doing it in Go. Here’s why ...
[1 comment]  
Springtime Tab Sweep — Tech · Ouch, some of these tabs are old. Unifying theme: none ...
[5 comments]  
Polyglot · Or in full, the Vancouver Polyglot {un} Conference. I saw it coming, thought it looked cool and that I’d go, then I got copied on an internal conversation where someone suggested we should sponsor it. D’oh, good idea, why didn’t I think of it? So we are. So I’ll not only go, I’ll suggest an unconference session on my current Identity obsessions. It’s a cool location and they look like cool people; come on down!
 
I Bought Some Bitcoins · On Tuesday evening I gave an envelope full of hundred-dollar bills to a friendly long-haired young man I’d never met in an undistinguished coffee-shop in an undistinguished neighborhood. By the time I got home, the Bitcoins I’d bought were worth noticeably less than I paid ...
[7 comments]  
Best Practices · That Steve Yegge is really good at starting conversations. Everyone including Steve thinks his essays are way too long, but they’re insightful and erudite and funny. His most recent outing, Notes from the Mystery Machine Bus, is all of those things, and worth plowing through to the end. It’s also more or less completely wrong ...
[9 comments]  
Eclipse to IntelliJ · After two years I still loathe Eclipse, and my former love NetBeans is now from a bad neighborhood, and more and more people I respect are switching to IntelliJ Idea. So I thought I’d give it a try, since I’m just starting on a project that might grow large-ish. This is going to be a diary of the experience, heavily loaded with GoogleBait in case others encounter some of the same pain points that are inevitable in such a transition ...
[12 comments]  
Network App Macroeconomics · A friend of mine is working on a complicated publishing app; the data is XML, perfectly appropriate when your objects are documents. She told me they were thinking about automating some of the work by running XSLT transformations out there in the client with libxslt. I said “Well yeah, as long as the client’s a PC not a tablet”. The category of “things you can do on a PC but not a tablet” is interesting ...
[13 comments]  
Type-System Criteria · Starting some time around 2005, under the influence of Perl, Python, Erlang, and Ruby, I became convinced that application programs should be written in dynamically-typed languages. You get it built faster, there’s less code to maintain, and the bugs are no worse. I’ve felt negative not just about statically-typed tools in general, but about the Java programming language in particular. Living in the Android world has forced me to think about this more ...
[21 comments]  
Geeks Gather in Belgium · I mean the 2011 Devoxx conference, tenth in a series, held in Antwerp, otherwise famous for Rubens and diamonds ...
[6 comments]  
DMR, 1941—2011 · Some things we now know to be good ideas: ...
[61 comments]  
Chip Experience · Basically all the credit cards in Canada are now “chipped”, which is to say that there are visible microelectronics toward one end. To pay, you slip that chip into the reader, confirm the deal and enter your PIN. This allows for a surprising amount of variation in user-experience quality ...
[11 comments]  
Perma-noob · This week I’ve been learning how one would build a simple RESTful back-end for an Android app, using Sinatra on Rack on JRuby on Java on App Engine and its Datastore, by doing it. The app needs persistence and user-account authentication, among other things. It’s been stimulating, but I’m feeling wistful ...
[7 comments]  
“Web” vs. “Native” · Back in February of 2010 I interviewed for a new job. It was the typical Google hiring-process siege; I talked to six or eight people over the course of the day. At least half of them asked me “Native vs Web apps on mobile; what do you think?” ...
[37 comments]  
Three Mobile-Software Rules · These days, I spend quite a bit of time talking about how to write software for Android. I think three of the general rules are worth expanding on here because I’m increasingly convinced they apply to software in general, not just for mobile devices ...
[16 comments]  
Lightroom 3 · If I read the tea-leaves correctly, this is increasingly the tool of choice among serious photogs, although Aperture isn’t going away any time soon. The new release has been well-covered elsewhere; I don’t have too much more to add, but I do have a major gripe and a couple of pictures to illustrate the much-ballyhooed noise-reduction filters ...
[10 comments]  
Modeling Is Hard · In A post-mortem on the previous IT management revolution William Vambenepe writes, on the subject of standardization: “The first lesson is that protocols are easy and models are hard.” I agree about the relative difficulty, but think when it comes to interoperation, protocols are very difficult and shared models usually impossible. A couple of examples occur to me ...
[9 comments]  
Lightroom Ping-Pong · Here’s the short form in geek-speak: Apparently, you can use rsync to keep two computers running Lightroom in harmony. The long version is well, long, and digresses into Deep Vein Thrombosis and Olympic Table-Tennis qualification ...
[19 comments]  
Language Waves · I suppose I could have entitled this A General Model for Progress In Adoption of Popular Programming Languages. What happened was, I was composing a rant intended for use in an internal discussion of developer futures, and it dawned on me that there’s a repeating pattern in the waves of programming languages that manage to succeed in finding broad usage ...
[14 comments]  
Tab Sweep — Tech · This goes back weeks and weeks; I’ve been wide-finding and doing Sun stuff and the Web-watching has suffered ...
[6 comments]  
TestMaker · I got mail from Frank Cohen saying “We just released TestMaker 5.0, could you give us a plug?” Hey, why not; I don’t know the first thing about the software so this is not an endorsement, and the phrase “SOA Governance” gives me a mild wave of nausea, but Frank’s a good guy, a long-time proponent of dynamic languages on the JVM, and anyhow the software’s Open-Source. Hey Frank, does it talk REST?
[1 comment]  
Tech Tab Sweep · We’re all over the map today, from general theories of software development to low-level optimized bit-banging. Well, all over the software map, I guess ...
[2 comments]  
Not Much New Here · In recent talks, I’ve been using a graphic from State of the Computer Book Market, written for O’Reilly Radar by Mike Hendrickson, and it’s been preying on my mind. The more I think about it, and about the programming-languages landscape, the more I think that this picture isn’t changing much any time soon. The landscape is stable ...
[23 comments]  
Lightroom and Open Source · Over the last few years, I’ve become something of an open-source triumphalist, drifting to the conclusion that (on the engineering side) it’s the best way to build software and (on the business side) it’s a better way to monetize it. I have to confess that Adobe Lightroom has kind of shaken my convictions. Certain elements of its UI and design (for example, the crop/rotate tool, and the nondestructive editing paradigm) are qualitative steps forward in the state of the art. Furthermore, I can’t think of a single good business reason for Adobe to open-source it. I guess the conclusion is obvious: for the foreseeable future, both models of software building and marketing are going to march along; neither is doomed.
[7 comments]  
Whither IT? · There are a couple of browser tabs I’ve had open for at least a week now and they’ve been making me think and I think they’re related but I still don’t have a synthesis. The first is Bill de hÓra’s Matchstick Men, which says a bunch of smart things about WS-* and REST, but that’s not what resonates, it’s this: Critically the upkeep and maintenance of legacy systems has come to dominate business software spending. Most large enterprise IT divisions now have the equivalent of a pensions fund crisis, except that all the money is being spent on old systems instead of old people. The second is Nicholas Carr’s Citi whacks IT, from begins: In yet another sign of the vast amount of waste inherent in big-company IT operations... ...
[7 comments]  
Lightroom Fanboy · I really don’t often use this space to blather on about how good some piece of commercial client-side closed-source software is, but, well... Adobe Lightroom is a truly great piece of work. Not only is it just the thing for the serious photographer, I think it may have advanced the state of the GUI art ...
[9 comments]  
Obie Says · That would be Obie Fernandez; he offers Some of My Software Opinions, and I recommend it heartily. I agreed with almost all of them; enough that I fully intend to re-use them myself.
[2 comments]  
Defining Standards · In Pro Choice, Sam Ruby says: I prefer simpler definitions.  A standard is one that has multiple, inter-operable, independent implementations.  An open standard, at least in the software world, is one where at least one of those implementations is open source. You can sign me up for that, too.
[6 comments]  
Interview, with Snarls · James Gray of Linux Journal has published a lengthy email interview with me. Those who visit ongoing regularly won’t find much to surprise them; but I did take the chance to fulminate about dastardly DRM and Microsoft’s odious Office XML. And now that I think of it, I’ve been stingy with the polemics around here recently, maybe a little bland even; aren’t bloggers supposed to be ruthless attack puppies? It’s having a cute little girl baby around that does it I guess.
[2 comments]  
Cars and User Interfaces · I can remember, back in the day, when we were arguing about user interfaces we said “It should be as intuitive as a car is. Any experienced driver can get in a new, strange, car and figure it out in minutes.” Well, I’ve been using a bunch of different computers and driving a bunch of different cars lately, and I think we’re there. With a sad picture and some automotive micro-reviews ...
 
ORM Bien Phu · I thought the laugh line “Object-Relational Mapping is the Vietnam of Computer Science” was ancient, but Ted Neward claims that he made it up in 2004. Ted has written an immense, detailed, essay on the subject, The Vietnam of Computer Science, which, just to be thorough, includes a capsule history of the Vietnam conflict. This ought to be required reading for all Computer Science undergrads, so they’ll at least be forewarned before they stumble into their own private Southeast Asia. Bonus: in the comments, the first commenter asks “If ORM = Vietnam, does SOA = Iraq?”
 
Stiff’s Questions · A while ago I got an email out of the blue from Jarosław Rzeszótko, who’s from Poland I think, saying “I hope you don’t mind if I ask you a few questions about software.” Then I ran across it again while looking for something in my inbox on a plane ride, so I dashed off some answers; the person seemed pleasant and polite. He calls himself “Stiff” and his blog “Sztywny Blog”. Anyhow, he also heard back from Linus Torvalds, Dave Thomas, David Heinemeier Hansson, Steve Yegge, Peter Norvig, Guido van Rossum, and James Gosling. The answers to his questions are interesting.
 
On PHP · I should really buckle down and try writing a PHP app because, at the moment, I have an attitude problem. I know that IBM now officially loves it, and Tim O’Reilly’s been charting the upcurve in PHP book sales, and everyone’s saying that Oracle’s going to buy Zend. If you want your ears bent back, have a listen to Zend CEO Doron Gerstel; he’ll tell you that half the websites in the world are powered by PHP and that there are 2½ million developers and that the war is over and PHP won. So here’s my problem, based on my limited experience with PHP (deploying a couple of free apps to do this and that, and debugging a site for a non-technical friend here and there): all the PHP code I’ve seen in that experience has been messy, unmaintainable crap. Spaghetti SQL wrapped in spaghetti PHP wrapped in spaghetti HTML, replicated in slightly-varying form in dozens of places. Everyone agrees on PHP’s upsides: it’s written for the web, it’s easy to deploy and get running, and it’s pretty fast. Those are important advantages. And I’m sure that it’s possible to write clean, comprehensible, maintainable, PHP; only apparently it’s real easy not to. But PHP has competition, most obviously Rails; and don’t write the Java EE crowd off, they’re not stupid at all and they’re trying to learn the lessons that PHP is trying to teach. So PHP has earned everyone’s respect by getting where it is, and Sun should reach out to it more than we have. But in the big picture, it feels vulnerable to me. [Wow, I regret not having comments. There’s been some first-rate discussion in email and on other blogs. On this occasion, I’m going to create a virtual comment section by posting the good ones here.] [There is a new, good pro-PHP rant from Harry Fuecks, and with that I’m going to stop adding to this discussion, unless somebody says something strikingly new. Thanks everyone! I’ve added a brief Table of Contents to try to bring some order to the chaos.] ...
 
Ultra Sysadmin · I’ve got this Ultra 20 which among other things has a huge disk with room for lots of operating systems. I want to fool around a bit with Linux and Solaris Classic and GNU/Solaris, run some Bonnie numbers and also shake down my own stuff on all these different boxes. I expected pain, and I’ve already had some. This post will be my diary of the things that go wrong, and right ...
 
Carlyle on REST & O-O · You wouldn’t think there’d be much new to say on the impedance mismatch between the O-O & REST world-views, but Benjamin Carlyle has a very good piece on the subject, with lots of well-thought-out little side-trips. The blog’s called Sound advice and a glance suggests that there’s lots more there worth reading.
 
Long Weekend News · While most of North America was offline watching fireworks or whatever, the row of tabs across the top of my browser has been getting thicker and thicker with little news nuggets that I thought I ought to write about. Time to houseclean, so here are some interesting things you might have missed. Item: Eric Raymond says we don’t need the GPL any more. Item: Davanum Srinivas points out that OSS implementations of WS-Security may be impossible, foundering on the same rock that Sender-ID did. Item: The WaSP and Microsoft, sittin’ in a tree (and as Scoble notes, ice is observed forming in Hell). Item: Rico continues to pile up buzz.
 
Threads Redux · The June 12th On Threads piece got slashdotted (twenty thousand hits for a 2,300 word hard-tech piece, not bad), which provoked really interesting feedback from (among others) David Dagastine, Greg Wilson, and Ben Holm, along with pointers to some related work. All those pointers are worth following, and some of the points are worth a little more discussion ...
 
On Threads · Last week I attended a Sun “CMT Summit”, where CMT stands for “Chip Multi-Threading”; a roomful of really senior Sun people talking about the next wave of CPUs and what they mean. While much of the content was stuff I can’t talk about, I was left with a powerful feeling that there are some real important issues that the whole IT community needs to start thinking about now. I’ve written about this before, and of the many others who have too, I’m particularly impressed by Chris Rijk’s work. But I think it’s worthwhile to pull all this together into one place and do some calls to action, so here goes. [Ed. Note: Too long and too geeky for most.] [Update: This got slashdotted and I got some really smart feedback, thus this follow-up.] ...
 
Coding and Climbing · It had never occurred to me that writing software is like climbing Mt. Fuji, but Stephan Grell makes it all clear.
 
PhotoShop Elements 3 Review · Skipping to the conclusion: It’s good, if you’re using Elements already go upgrade. But along the way, there are thoughts on software’s lifespan and pricing. Plus, a thin black woman kissing a flower taller than herself ...
 
Software in the TLP Era · Flying over the Atlantic, I read all eight parts of Chris Rijk’s Thread Level Parallelism Design Decisions, and I wish a few more software geeks would go and read it. Herewith a few notes on software design in the era of Thread Level Parallelism ...
 
It’s the Libraries, Stupid · Via Jeff Dillon, some insightful words on programming in Java and in the C#/.NET/Mono ecosystem. I hadn’t thought about it that way.
 
Wondering About Vignette · I’d like to use the Web as a Really Smart Library—first time I’ve done that on this blog—because I’m looking for some user and programmer experience on Vignette. I haven’t been near their products for years, since the days of StoryServer, which was generally hated for bugginess, arcane complexity, and TCL-centricity. If anyone would like to share some Vignette content-management or portal experiences, on or off the record, I’d sure appreciate it, and I’d publish them here too (or not) on request.
 
author · Dad · software · colophon · rights
Random image, linked to its containing fragment

By

I am an employee of Amazon.com, but the opinions expressed here are my own, and no other party necessarily agrees with them.

A full disclosure of my professional interests is on the author page.