Today Dave Sifry of the excellent Technorati announced an API for the world. The API, as announced, is about as purely Webby a thing as you can imagine. Dave Winer pushed back, suggesting a more SOAP/XML-RPC kind of approach. This is maybe the single central issue in architecting Web apps right at the moment, so I think it's OK to take a few more whacks at the supine equine. Furthermore, I think the issue is simple enough that anyone who uses the web, not just geeks, ought to be able to understand it. So I've provided an introduction for the non-geeks who read ongoing, all three of them, and looked a little more closely at the Technorati situation.

Lightning Intro to the Issues · (People who know all this stuff can skip a few paragraphs here.)

Ways of Talking to a Web Server · If you're a piece of software that wants to talk to a Web Server (a browser, for example, but there are lots of others), you generally use the HTTP protocol, which supports all kinds of transactions, but the two most often used are called GET and POST.

GET is simple; you send a request encoded up in a URI to the server, and you get a package of info back from the server. This is what happens when you click on a hyperlink; nothing could be simpler, and a huge majority of all Web transactions are GETs. There are some rules that go with GET; it's not OK to use it ot make a purchase or trigger an explosive device or anything that changes the state of the world. It's for fetching information.

When you POST, you contact the server and send it a package of information, not just a URI, and get back a package of information. You should use POST if you want to do anything more than just retrieve information, for example buy or blow up something. So most times when you fill out a form on a Web site and press Submit, it's using a POST.

All this is straightforward and causes little angst as long as the Web is just browsers and pages. But a lot of people want to use the Web for more; to syndicate weblogs, run search engines, even to do large-scale B2B.

How do you fit this into the Web's GET/POST world? Well first off, you typically send machine-oriented XML rather than human-oriented HTML from the server. And if you ask for it with a POST, the package of information you send is usually XML too.

POST and GET, Does it Matter? · It matters. First of all, if you're going to do anything but just retrieve information, you have to use POST. And if all you're doing is retrieving information, you really really should use GET, even if you could use POST.

The reason is that the input to GET is a URI, that simple, wonderful little string that can be used to address anything on the Web, which is more or less everything in the world these days. That means you can email that URI around, or feed it to a search engine's spider, or bookmark it, or do a bunch of other things that probably haven't been invented yet.

It also means that things like the caches and proxies and so on that make the Web go fast can do their job and make the Web go fast.

REST · People (like me and the rest of the W3C TAG) who go around admonishing people to use GET are often said to be disciples of “REST”. REST is an acronym invented by Roy Fielding, a prominent geek who is chair of the Apache Foundation and chief author of the URI specification and so on. He invented it for his Ph.D. thesis, in which he tried to formalize how the Web works.

The Other Side · Since you know where I stand, you should take what I say about people on the other side of the issue with a grain of salt, but here goes anyhow. There are a lot of people who are used to thinking in terms of complicated data structures and complicated APIs to talk to them. These people tend to find the GET approach to life ludicrously simple, inflexible, and unsophisticated. They're entirely right; it's just that for the special case where you want to retrieve some information, the advantages of being able to just name it with a URI are so great (I believe) that they justify doing everything at the boneheadedly-simple level of GET transactions.

People who disagree tend to favor POST; they would like to encode a package of arguments to a complicated API, in some cases with the arguments being complicated data structures, and ship them off, and get some more data structures back.

Or not get them back—the Web Services visionaries imagine sending off a message that launches a cascade of asynchronously-queued long-lived transaction components being orchestrated through a constellation of distributed routers and value-adding agents. So far, this vision is way out there with the Semantic Web in terms of being a practical useful reality, and I'm fairly optimistic that both of them will deliver in their own time, and I'm pretty sure that neither is going to make it to first base without climbing in bed with the other, and furthermore I think that mixing baseball and sexual metaphors is just fine.

The conflict in world-views between the REST camp and the SOAP/XML-RPC camp was first highlighted by Paul Prescod, writing about Google's wandering path through the landscape, and his essay remains a first-rate overview of the issues.

SOAP and XML-RPC · There are two low-level protocols that you can use if you want to POST a chunk of XML off to a server and get another back. SOAP, which grew out of work by Dave Winer and Microsoft, is irritatingly overcomplicated and overequipped for what should be a simple task, but there are good implementations out there. XML-RPC, which is mostly Dave's, is irritatingly underquipped and oversimplified, even for what should be a simple job, but there are good implementations out there.

What Should Technorati Do? · Technorati is a web site that tracks hundreds of thousands of weblogs, specifically who's pointing to whom. Most weblog authors visit it regularly (not to say obsessively) to find out who's pointing to them, and would like the magic number of in-pointing weblogs to go up and up and up. (This is not hard to achieve as long as the number of weblogs keeps going up and up and up, but I digress).

So Dave Sifry of Technorati intelligently concluded that he should make the results available in XML, so that anyone who wants to could fetch them and do, well... anything. Let's pause for a moment of silence in Dave's honor, you da geek!

The way he proposes to do this is pure REST, he documents how you construct a URI to find out who's pointing to what, and he documents the XML he'll send back when you GET it, and that's all she wrote.

Dave Winer points out (correctly) that it would be easier for programmers with XML-RPC or SOAP toolkits to get at this if the interface were SOAP or XML-RPC via POST. He also points out (correctly) that it's really irritating that the URI has to include an “appkey.” (This is a Technorati ticket entitling you to use the service. There is a limit on the number of times per day you can use a ticket, unless of course you're willing to pay. On business grounds this makes all sorts of sense.)

Of course, I totally disagree with Dave (er Winer I mean, let's say DW and DS) on the first point. This is a pure get-me-some-info app, and yes, constructing URIs is a bit more work, but the result is that the Technorati results are first-class citizens of the Web and are represented by shortish strings, and if the Web hasn't yet taught us the magic of this combination, well we're never going to learn.

So I'd say grit your teeth and code up those URIs and bask in the satisfaction that you're helping make the Web a richer place. I don't know about you, but I owe the Web quite a few debts, and you know what? Every time I do a little extra work to try to improve it, it pays me back with interest.

Obviously, when you want to construct an app to send DS some money to use his excellent service, you'd want to POST via XML-RPC or equivalent.

I totally agree with DW on the second point, but it's tough to figure out the right solution. If I publish a URI with my appkey in it, then any time that gets passed around and used, then the usage goes on my account. I'm just not sure there is a good way to fit an appkey-based accounting scheme into the Web architecture, but I don't think we've really tried very hard yet, so I'm quite possibly wrong.

Also, there are other ways to go about chargeback and accounting at Websites; unfortunately for DS, most of them take more work to set up than the appkey approach.

From Here On In · You, dear readers, have ringside seats! Does the power of naming everything with URIs buy enough to compensate for the inconvenience of having to jam complex requests into them? Does the convenience of packaging up your request in XML buy enough to compensate for the loss of naming power? Is the future of the Web spelt S.W. or W.S. or both or neither? Stay tuned.

author · Dad · software · colophon · rights
picture of the day
May 12, 2003
· Technology (77 fragments)
· · Web (385 more)

By .

I am an employee
of, but
the opinions expressed here
are my own, and no other party
necessarily agrees with them.

A full disclosure of my
professional interests is
on the author page.