What
 · Technology
 · · Concurrency

Distributed Hardness · The oth­er day this, from Mathias Ver­raes, got thou­sands of retweet­s ...
[1 comment]  
Functional Programming Wisdom · I don’t of­ten ded­i­cate a blog en­try to just a link, but this one is im­por­tan­t. Im­por­tan­t, that is, if you’re a com­put­er pro­gram­mer; in par­tic­u­lar a pro­gram­mer who needs to make code run faster on ex­ist­ing real-world hard­ware. Which is a mi­nor­i­ty of a mi­nor­i­ty, since it ex­cludes most Web­folk whose servers are fast enough and clients are run­ning 90% idle time. But that mi­nor­i­ty re­al­ly needs to be think­ing about Func­tion­al Pro­gram­ming, and if you’re not 100% sure you know what that mean­s, you should drop ev­ery­thing and go read “Uncle Bob” Martin’s Func­tion­al Pro­gram­ming Ba­sics ...
[11 comments]  
WF2: That’s All, Folks · [This is part of the Wide Fin­der 2 se­ries.] This should be the fi­nal en­try, af­ter a cou­ple of years of si­lence. The re­sults can be read here, for as long as that ad­dress keeps work­ing. I’m glad I launched that pro­jec­t, and there is follow-on news; tak­ing ef­fect to­day in fac­t ...
[1 comment]  
Late Summer Tech Tab Sweep · Some of these pup­pies have been keep­ing a brows­er tab open since April. No the­me; rang­ing on the geek­i­ness scale from ex­treme to mostly-sociology ...
[2 comments]  
Concur.next — Hard-Core Clojure · Here’s re­al news: Alex Os­borne of Na­tion­al Li­brary of Aus­tralia, al­so known as @atos­borne, whom I first met on #clo­jure, took the Wide Fin­der bit be­tween his teeth and has post­ed a re­mark­able im­ple­men­ta­tion sto­ry: Widefind­er 2 with Clo­jure. Um, 8m4.663s! If you care about any as­pect of this stuff you re­al­ly ought to go read it now. Grab your­self a cof­fee or what­ev­er first; it’s not short ...
[1 comment]  
Concur.next · Are there any com­put­er pro­grams that you wish were faster? Time was, you could solve that prob­lem just by wait­ing; next year’s sys­tem would run them faster. No longer; Next year’s sys­tem will do more com­put­ing all right, but by giv­ing you more CPUs, run­ning at this year’s speed, to work with. So the on­ly way to make your pro­gram faster is to work with more CPUs. Bad news: this is hard. Good news: we have some re­al­ly promis­ing tech­nolo­gies to help make it less hard. Bad news: none of them are main­stream. But I’m bet­ting that will change ...
[29 comments]  
Concur.next & WF2 — Tuning Concurrent Clojure · I’ve been work­ing on, and writ­ing about, run­ning Clo­jure Wide Fin­der code. But I was nev­er sat­is­fied with the ab­so­lute per­for­mance num­ber­s. This is a write-up in some de­tail as to how I made the code faster and al­so slow­er, in­clud­ing lessons that might be use­ful to those work­ing on Clo­jure specif­i­cal­ly, con­cur­ren­cy more gen­er­al­ly, and with some in­ter­est­ing da­ta on Ja­va garbage col­lec­tion and JDK7 ...
[16 comments]  
Concur.next — Eleven Theses on Clojure · I’ve been bang­ing away on Clo­jure for a few days now, and while it would ob­vi­ous­ly take months of study and grind­ing through a big se­ri­ous real-world soft­ware project to be­come au­thor­i­ta­tive, I think that what I’ve learned is use­ful enough to share ...
[27 comments]  
Fortress · Since I’m spelunk­ing around the new-languages cav­erns these days, I re­al­ly ought to men­tion the long-ongoing and very in­ter­est­ing Fortress, brain-child of our own Guy Steele, who knows one or two things about de­sign­ing lan­guages ...
[5 comments]  
Concur.next — Idiomatic Clojure · I’m start­ing to wind down my Clo­jure re­search, but I’m feel­ing a lit­tle guilty about hav­ing ex­posed peo­ple to my klunky Lisp-newbie code, per­haps giv­ing a false im­pres­sion of how the lan­guage feel­s. So I’d like to show you what it looks like when it’s cre­at­ed by some­one who’s ac­tu­al­ly part of the tribe and thinks in it more na­tive­ly than I prob­a­bly ev­er will ...
[12 comments]  
Concur.next — Tab Sweep · Be­ing a bas­ket of concurrency-related morsels too short to stand alone and to long to tweet ...
[4 comments]  
Concur.next — No Free Lunch · In which the ac­tu­al costs of run­ning con­cur­rent­ly are ex­am­ined, and seem shock­ing­ly high ...
[12 comments]  
Concur.next — More Clojure I/O · I re­cent­ly wrote-up some Clojure-based Wide Fin­der work, in Par­al­lel I/O and Ref­er­ences. Driv­en most­ly by ideas from com­menter­s, I did some refac­tor­ing and knob-spinning. The re­sults are in­ter­im at best, and on US-Thanksgiving eve al­most nobody’s look­ing, but it’s good to get this stuff on the record ...
[4 comments]  
Concur.next — References · Th­e­se, “refs” for short, are one of the three tools of­fered by Clo­jure to make con­cur­ren­cy prac­ti­cal and man­age­able. Here­with a walk-through of code that us­es them to ac­com­plish a sim­ple task in a high­ly con­cur­rent fash­ion ...
[21 comments]  
Concur.next — Parallel I/O · Con­clu­sion first: It turns out that Clojure’s con­cur­ren­cy prim­i­tives al­low you, with a very mod­er­ate amount of un­com­pli­cat­ed code, to take ad­van­tage of par­al­lel hard­ware and out­per­form re­al­ly fast soft­ware when it doesn’t take such ad­van­tage ...
[9 comments]  
Clojure N00b Tips · Clo­jure is the new hot­ness among peo­ple who think the JVM is an in­ter­est­ing plat­form for post-Java lan­guages, and for peo­ple who think there’s still life in that ol’ Lisp beast, and for peo­ple who wor­ry about con­cur­ren­cy and state in the con­text of the mul­ti­core fu­ture. Over the last few days I’ve been severe­ly bipo­lar about Clo­jure, swing­ing from “way cool!” to “am I re­al­ly that stupid?” Here­with some getting-started tips for new­bies like me ...
[7 comments]  
Tail Call Amputation · This is per­haps a slight di­gres­sion; just an ex­tend­ed ex­pres­sion of plea­sure about Clojure’s re­cur state­men­t. It neat­ly squash­es a bee I’ve had in my bon­net for some years now; if it’s wrong to loathe “tail re­cur­sion optimization” then I don’t want to be right ...
[25 comments]  
Concur.next — Messaging · The more I look at Clo­jure, the more I think it’s a hero­ic at­tempt to Do The Right Thing, in fact All The Right Things, as we move to­ward the lots-of-not-particularly-fast-cores fu­ture. I’m still work­ing my head around Clojure’s con­cur­ren­cy prim­i­tives. We come to un­der­stand the things we don’t by con­trast with the things we do; so I’m find­ing con­trast be­tween the Clo­jure and Er­lang ap­proach­es to mes­sag­ing in­struc­tive ...
[28 comments]  
Concur.next — C vs. P · There are rip­ples spread­ing across the con­cur­ren­cy pond, some pro­voked by this se­ries, and there’s this one is­sue that keeps com­ing up: “Concurrency” vs. “Parallelism”. It is as­sert­ed that they’re not the same thing, and that this is im­por­tan­t ...
[12 comments]  
Concur.next — My Take · I’ve been try­ing to avoid ed­i­to­ri­al­iz­ing in this “Java-of-concurrency” se­ries. But since peo­ple are grum­bling about my bi­as­es, here are a few notes about how I see things at this (ear­ly, I hope) stage in the pro­cess ...
[9 comments]  
Concur.next — Crosstalk · This sub­ject seems to have hit a nerve, and there’s been out­stand­ing feed­back in the com­ments. Some of it makes a good case for changes in the se­ries ar­ti­cles, and oth­ers just need more at­ten­tion than I think they’ll get down in the com­ment sec­tion ...
[8 comments]  
Concur.next — The Laundry List · There are a lot of in­gre­di­ents that might or might not go in­to the win­ning for­mu­la that brings con­cur­rent pro­gram­ming to the main­stream. This is a very brief run-through of as many as I can think of ...
[34 comments]  
Concur.next — Java · This se­ries ar­gues that there’s an op­por­tu­ni­ty for some tech­nol­o­gy to be­come “The Ja­va of con­cur­rent programming”. For back­ground then, I’ll try to cap­ture the fac­tors that led to Java’s suc­cess as the first main­stream object-oriented plat­form for ap­pli­ca­tion de­vel­op­er­s ...
[10 comments]  
Meat-Grinder! · It’s days like these that make it fun work­ing for Sun. The new server’s of­fi­cial name is the T5440; they call it a “mid-range” box, but to me it looks like a mon­ster; count the num­bers for cores, thread­s, RAM, and so on. It’s as­tound­ing what you can fit in­to a 4U box these days ...
[2 comments]  
Tab Sweep — Technology · I’d kind of got­ten out of the habit of do­ing tab sweep­s, large­ly be­cause my Twit­ter feed is such a se­duc­tive place to drop in­ter­est­ing links. But as of now there are around 30 tabs open on my browser, each rep­re­sent­ing some­thing I thought was im­por­tant enough to think about and maybe write about. Some are over a month old. Some of them have been well-covered else­where. All I as­sert is that af­ter I read each one of the­se, I didn’t want to hit command-W to make that win­dow go away. Uni­fy­ing the­me? Sure­ly you jest ...
[1 comment]  
WF2: Midsummer Update · [This is part of the Wide Fin­der 2 se­ries.] We’re a few weeks in now, so I should pro­vide an up­date. Those who are re­al­ly in­ter­est­ed might want to join the Wide Fin­der group ...
[2 comments]  
WF2: Early Results · [This is part of the Wide Fin­der 2 se­ries.] The first se­ri­ous re­sults are in, and they’re eye-opening. The naive Ru­by ap­proach, re­mem­ber, burned some 25 hours. There are now four oth­er re­sults post­ed, with elapsed times of 8, 9, 15, and 17 min­utes. The write-ups are al­ready full of sur­pris­es, and I’m ex­pect­ing more.
[1 comment]  
WF2: Start Your Engines! · [This is part of the Wide Fin­der 2 se­ries.] I have now done the first “official” run of the naive Ru­by im­ple­men­ta­tion of the bench­mark. There is some dis­cus­sion of the code here. The bench­mark is de­scribed, and the naive Ru­by code pro­vid­ed, here. I’ve start­ed a re­sults page here. There are al­ready ten eleven oth­er peo­ple now with ac­counts on the Wide Fin­der ma­chine, and I know there’ve been re­sults that are huge­ly bet­ter than this first cut. Read on for a cou­ple of notes on this first run ...
[9 comments]  
WF2: People At Work · [This is part of the Wide Fin­der 2 se­ries.] I’m hap­py to re­port that I’ve giv­en out a bunch of ac­counts on the Wide Fin­der 2 ma­chine. I’ll ag­gre­gate links to others’ work in this en­try ...
[3 comments]  
WF2: First Pathetic Results · [This is part of the Wide Fin­der 2 se­ries.] I made a cou­ple of lit­tle changes to the Straw­man Bench­mark and let ’er rip on the big 45Gig da­ta set. The re­sults were mis­er­able, but in­struc­tive ...
[3 comments]  
WF2: The Benchmark · [This is part of the Wide Fin­der 2 se­ries.] A bunch of peo­ple have re­quest­ed the sam­ple data. Mean­while, over at the wi­ki, there’s a de­cent dis­cus­sion go­ing on of what bench­mark we should run ...
[5 comments]  
WF2: Benchmark Strawman · [This is part of the Wide Fin­der 2 se­ries.] I’ve just up­dat­ed the Wide Fin­der wi­ki with a straw-man bench­mark (see “Strawman”), with sam­ple Ru­by code and out­put. Argh... it takes Ru­by over 23 sec­onds to pro­cess just 100K lines. I hear talk of a So­lar­is/SPARC op­ti­mized ver­sion, got­ta track that down. Com­ments please on the bench­mark, then we can all start bash­ing our heads against the big file.
[15 comments]  
Wide Finder 2 · Last fal­l, I ran the Wide Fin­der Pro­ject. The re­sults were in­ter­est­ing, but in­com­plete; it was a re­al shoestring op­er­a­tion. I think this line of work is in­ter­est­ing, so I’m restart­ing it. I’ve got a new com­put­er and a new dataset, and any­one who’s in­ter­est­ed can play ...
[18 comments]  
Ruby News · This re­al­ly isn’t the place to come for Ru­by news. But that’s OK, be­cause I have the point­ers to where you should go. Plus, one of the news sto­ries is mak­ing me think “Smells like Erlang.” ...
[4 comments]  
New Computers · To­day, we at Sun had a serv­er an­nounce­men­t, and so did IBM. Get yer hot links & pix here ...
[3 comments]  
Hard Problems · I spent quite a bit of to­day at the O’Reilly 2008 Con­cur­ren­cy Sum­mit. It was a con­ge­nial crowd, but at the end of the day kind of a down­er, be­cause we have lots of hard con­cur­ren­cy prob­lems and not too many so­lu­tion­s. Any­how, two sub­jects that came up were REST (which is con­cur­rent at the largest pos­si­ble scale), and, un­sur­pris­ing­ly, Er­lang. And it struck me that they’re kind of like each oth­er ...
[4 comments]  
WF XIII: Logistical Pain · This is the thir­teenth progress re­port from the Wide Fin­der Pro­ject. It’s just scratch­pad to cat­a­logue all the prob­lems I’ve had get­ting con­tribut­ed code to work. Prob­a­bly not of gen­er­al in­ter­est, but an es­sen­tial part of a com­plete write-up ...
[13 comments]  
WF XI: Results · This is the eleventh progress re­port from the Wide Fin­der Pro­ject; I’ll use it as the re­sults ac­cu­mu­la­tor, up­dat­ing it in place even if I go on to write more on the sub­jec­t, which seems very like­ly. [Up­date: Your new lead­er: Per­l.] ...
[27 comments]  
The Wide Finder Project · In my Find­ing Things chap­ter of Beau­ti­ful Code, the first com­plete pro­gram is a lit­tle Ru­by script that reads the on­go­ing Apache log­file and fig­ures out which ar­ti­cles have been fetched the most. It’s a clas­sic ex­am­ple of the cul­ture, born in Awk, per­fect­ed in Per­l, of get­ting use­ful work done by com­bin­ing reg­u­lar ex­pres­sions and hash ta­bles. I want to fig­ure out how to write an equiv­a­lent pro­gram that runs fast on mod­ern CPUs with low clock rates but many cores; this is the Wide Fin­der pro­jec­t ...
[31 comments]  
WF XV: On Parallel I/O · This is the fif­teenth progress re­port from the Wide Fin­der Pro­ject; it’s fairly-uncooked mus­ing about par­al­lelism and I/O ...
[13 comments]  
WF XIV: My Opinion · This is the four­teenth progress re­port from the Wide Fin­der Pro­ject. I still have a bunch of Wide Fin­ders to run, and for the time be­ing I’ll keep try­ing to run what peo­ple send me; we can’t have too much da­ta about this prob­lem. But some con­clu­sions are start­ing to look un­avoid­able to me ...
[7 comments]  
WF XII: Discussion · This is the twelfth progress re­port from the Wide Fin­der Pro­ject. It ex­ists to host the ex­cel­lent dis­cus­sion so far from oth­er­s; see the com­ments ...
[17 comments]  
WF X: The Parade Continues · This is the tenth progress re­port from the Wide Fin­der Pro­ject. What with Ve­gas and the Com­mies, I’m be­hind on a lot of things, in­clud­ing the Wide Fin­der. This is an­oth­er en­try just to note oth­er people’s work, which I re­al­ly ab­so­lute­ly will buck­le down and run on the big iron and re­port back and show­er praise on the good ones and de­ri­sion on the big miss­es ...
[5 comments]  
WF IX: More, More, More · So, I got dis­tract­ed by a serv­er launch and a Ve­gas trip, but the Wide Fin­der im­ple­men­ta­tions keep rolling in ...
[4 comments]  
The T2 Servers · Th­ese T5x20 servers we’re an­nounc­ing to­day are a big deal. My bet is that they end up mak­ing Sun a lot of mon­ey; but on the way, they’re go­ing to bring the whole serv­er busi­ness (not just Sun’s piece of it) face to face with some re­al dis­rup­tion ...
 
Testing the T5120 · This was go­ing to be a Wide Fin­der Pro­ject progress re­port, but I end­ed up writ­ing so much about the serv­er that I’d bet­ter ded­i­cate an­oth­er frag­ment to the com­par­isons of all those im­ple­men­ta­tion­s; es­pe­cial­ly since there are still lots more im­ple­men­ta­tions to test. So this a hands-on re­port on a cou­ple of more-or-less pro­duc­tion T5120’s, the T2-based serv­er that’s be­ing an­nounced to­day. Head­li­nes: The chip is im­pres­sive but weird; as­tound­ing message-passing bench­mark num­ber­s; fight­ing the US DoD ...
[1 comment]  
WF VIII: Snapshot · This is the eighth progress re­port from the Wide Fin­der Pro­ject; a quick comment-light ag­gre­ga­tion of work that oth­er peo­ple have been do­ing in the space. I’ve man­aged to get ac­cess to an unan­nounced many-core serv­er and have some pre­lim­i­nary re­sults (sum­ma­ry: Vinoski’s in the lead­); I’ll pub­lish those in the, uh, very near fu­ture, when things are, uh, less unan­nounced ...
[5 comments]  
WF VI: The Goal · This is the sixth progress re­port from the Wide Fin­der Pro­ject, in which I try to paint a pic­ture of what the so­lu­tion should look like. The project re­al­ly has two goal­s. First, to es­tab­lish whether it’s pos­si­ble to write code that solves this prob­lem and runs much faster in a many-core en­vi­ron­men­t. Se­cond, to con­sid­er how that code should look to the pro­gram­mer who has to write it ...
[16 comments]  
WF VII: Other Voices · This is the sev­enth progress re­port from the Wide Fin­der Pro­ject, in which I re­port on the work of oth­ers and turn this in­to kind of a con­test. I’ll be back lat­er with one more whack at Er­lang, and then prob­a­bly move on to oth­er Wide Fin­der op­tion­s. [Up­date: If you’re work­ing on this but I missed you, sor­ry; holler at me and I’ll make sure you get cov­ered.] ...
[12 comments]  
WF IV: The Cascade · This is the fourth progress re­port from the Wide Fin­der Pro­ject. Fol­low­ing on my ear­li­er fum­bling at the con­trols of the Er­lang race-car, sev­er­al oth­ers joined the con­ver­sa­tion. For par­tic­u­lar­ly smart in­put, check out the Er­lang ques­tions mail­ing list and the already-linked-to Steve Vi­nos­ki piece, Tim Bray and Er­lang. Other voic­es have ques­tioned the project’s san­i­ty, and brought Scala and Haskell to bear on the prob­lem. But let’s push all that on the stack, and ask our­selves this ques­tion: what if pro­cess­es were free? An­swer­ing it turns out to be all sorts of fun; I seize the Er­lang con­trols back again, and steer it with a firm steady hand in­to a brick wall at top speed ...
[9 comments]  
WF V: Roundup · This is the fifth progress re­port from the Wide Fin­der Pro­ject; an ag­gre­ga­tion of what oth­er peo­ple have been say­ing ...
[5 comments]  
WF III: Lessons · This is the third progress re­port from the Wide Fin­der Pro­ject. Giv­en that I launched this brouha­ha late on a Fri­day, to­tal­ly the worst pos­si­ble time, I’m as­tound­ed at the in­ten­si­ty and qual­i­ty of the con­ver­sa­tion that’s go­ing on. I want to ad­dress two themes that have emerged, one of which seems stupid and the oth­er smart ...
[14 comments]  
WF II: Erlang Blues · This is the sec­ond progress re­port from the Wide Fin­der Pro­ject, and a follow-on from the first, Er­lang Ho! The one thing that Er­lang does right is so im­por­tant that I’d like to get past its prob­lem­s. So far I can’t ...
[30 comments]  
WF I: Erlang Ho! · This is the first progress re­port from the Wide Fin­der Pro­ject. Er­lang is the ob­vi­ous can­di­date for a Wide Fin­der im­ple­men­ta­tion. It may be decades old but it’s the new hot­ness, it’s got a PragBook (@A­ma­zon), I hear heavy breath­ing from se­ri­ous soft­ware geeks whichev­er way I lis­ten. So, let’s give it a whirl. [Warn­ing: Long and de­tailed, but the con­clu­sion comes first.] ...
[8 comments]  
Postmodern Errors · Today’s fash­ion­able pro­gram­ming lan­guages, in par­tic­u­lar Ruby, Python, and Er­lang, have some­thing in com­mon: re­al­ly lousy er­ror mes­sages. I guess we just got­ta suck it up and deal with it. But to­day I got some­thing re­al­ly, uh, spe­cial from Er­lang ...
[15 comments]  
Sideways Computing · On many-core, wast­ed cores, thread­s, pro­cess­es, trans­ac­tion­al mem­o­ry, and the E-word ...
[14 comments]  
Tab Sweep — Tech · To­day we have Ja­va yield­ing, thread rant­ing, REST lec­tur­ing, and iden­ti­ty in­sight ...
[6 comments]  
Thread Herrings · Over the last cou­ple of years, I’ve writ­ten lots here about con­cur­ren­cy, and its is­sues for de­vel­op­er­s. The prox­i­mate cause is that my em­ploy­er is push­ing the many-core CPU en­ve­lope hard­er than any­one. This last week there’s been an out­burst of dis­cus­sion: from Nat Tork­ing­ton, David Heine­meier Hans­son, and Phillip Toland. They’re all worth read­ing, and the prob­lem isn’t go­ing away. But I’m wor­ry­ing about it less all the time ...
[7 comments]  
Tab Sweep · Per­haps a lit­tle more all-over-the-map even than is usu­al: GPLv3 clar­i­ty, Func­tion­al Pearl­s, raina bird-writer, Ja­va cred­it­s, frame­work pro­gram­mer­s, and hack­ing my Canon ...
[4 comments]  
The London Illustrated News · I spent the week in Lon­don. Fun was had, pic­tures were tak­en, I learned things. Here­with il­lus­trat­ed notes on trans­porta­tion, en­er­gy, fi­nance tech­nol­o­gy, busi­nesslike drink­ing, women’s cloth­ing, Groovy, ex­cel­lent lamb-chop cur­ry, and a round red anoma­ly ...
[8 comments]  
Berkeley on Parallelism · Any­one who cares at all about tak­ing ad­van­tage of these nasty new mi­cro­pro­ces­sors that still fol­low Moore’s law but side­ways not just straight up ought to go and read The Land­scape of Par­al­lel Com­put­ing Re­search: A View from Berke­ley. As the ti­tle sug­gest­s, it’s an overview pa­per. Thank good­ness we have uni­ver­si­ties, so that smart peo­ple who know about this stuff can in­vest a few months in sur­vey­ing the land­scape and re­port back for the rest of us who are be­ing whipped around by the mar­ket mael­strom. Here­with way too much ap­plause and dis­agree­ment and ex­pan­sion ...
[5 comments]  
Clementson on Concurrency · That would be Bill Cle­mentson, in Con­cur­ren­t/Par­al­lel Pro­gram­ming - The Next Gen­er­a­tion. I’ve been work­ing on so much oth­er stuff that the concurrency’s kind of been crowd­ed out. Which isn’t good, be­cause the highly-parallel fu­ture hasn’t stopped get­ting closer, and I just haven’t heard that much ex­cit­ing con­cur­ren­cy news re­cent­ly. Ex­cept from Google, where MapRe­duce and Sawza­ll may be point­ing one of the ways for­ward. I ac­tu­al­ly did a lit­tle fool­ing around with Er­lang (damn, that is one heavy­weight in­stal­l) and there’s a lot to like, but I don’t think the world is ready to give up object-orientation. There’s low-hanging fruit out there, and lots of pieces of the so­lu­tion are in plain view, and we know where we’re try­ing to go: to a place where or­di­nary ap­pli­ca­tion programmers’ code nat­u­ral­ly and ef­fort­less­ly takes ad­van­tage of multi-core, multi-processor, clus­tered, or oth­er­wise par­al­lel hard­ware de­ploy­ments. Be­cause scal­ing out, rather than scal­ing up, is still the fu­ture.
 
Statelessness · Check out this sur­pris­ing piece: The un­bear­able light­ness of be­ing state­less, by Ariel Hen­del. He starts with wise words on as­ceti­cism and the busi­ness trav­el­er, moves through some­thing called “Logical Domains” which can serve as “surf bum domains”. It seems to make sense and I had to go back and read it again, be­cause it was good, even though at the end of the day he is try­ing to sell you a com­put­er.
 
Those Cruel Irish · Peo­ple in­side Sun were glee­ful­ly email­ing around Colm MacCárthaigh’s big Ni­a­gara bench­mark post and I was read­ing and found my­self laugh­ing out loud. The syn­op­sis is: it’s a big se­ri­ous bench­mark and the box did great, pret­ty well slaugh­ter­ing both a Dell Xeon and a Dell Ita­ni­um. But jeep­er­s, those Ir­ish dudes are heart­less, I’m sur­prised there weren’t smok­ing shards of cas­ing and sil­i­con on the floor. I think most Apache & *n*x geeks would find them­selves gasp­ing and snick­er­ing a bit at Colm’s write-up, but there’s some re­al wis­dom there too about filesys­tem and serv­er tun­ing and so on, al­though some of the tricks are def­i­nite­ly don’t-try-this-at-home. Any­how, here are some cute sam­ples:
“Also, in each case, the sys­tem was pret­ty much un­us­able by the time we were done!”
“... about 83,000 con­cur­rent downloads.”
[They man­aged to crash So­laris with the ex­per­i­men­tal event MPM]: “Then again, it was han­dling about 30,000 re­quests at the time, with no ac­cept mutex.”
“Of course, no serv­er should ev­er be al­lowed to get in­to that kind of in­sane territory.”
“Note: these are stupid val­ues for a real-world server... re­al­ly on­ly use­ful if you are do­ing some in­sane bench­mark­ing and testing.”
“...5718 re­quests per second.”
Hey Jonathan, let ’em keep the box. [Up­date: They’re keep­ing it.]

 
Niagara Day · You can’t pos­si­bly imag­ine the amount of work it’s tak­en to get here. Richard McDougall has put to­geth­er a Ni­a­gara Blog­ging Car­ni­val which is the right place to start if you’re the kind of per­son that the MSM (Main Stream Mar­ket­ing, that stands for) isn’t aimed at; i.e., not a CEO, CIO, or jour­nal­ist. My own per­son­al fa­vorite Ni­a­gara news­bites: Item: No­body gets 100% yield on their chip­s. I gath­er that for the Ni­a­garas that don’t turn out per­fec­t, we’ll sell ’em cheap­er as 7-core, 6-core, 4-core, or what­ev­er. Some of these con­figs might turn out to be the deal of the cen­tu­ry de­pend­ing how we price them. Item: They’re open-sourcing the hard­ware, too. I’m not sure ex­act­ly what that means in the big pic­ture, and the li­cens­ing is go­ing to mat­ter, but it’s cool. Item: Those eight cores, when one’s not busy, they stop it. No, they don’t idle-loop it, they stop it. Ob­vi­ous when you think of it. Item: When not to use the new stuff. Item: How the I/O works. Item: What makes chips wear out and fail? Lots of things, but es­pe­cial­ly heat; so low-wattage chips are RAS win­ners. Item: Max­i­mum geek-out! Last item: When you have Ja­va threads that map re­al close­ly on­to So­laris threads that map re­al close­ly on to hard­ware thread­s, and you al­so have a lot of well-implemented hard­ware thread­s, this is what hap­pens.
 
CMT Rumbles in the Distance · The par­ty line on our Ni­a­gara tech­nol­o­gy con­tin­ues to be “sometime late this year or ear­ly next year”. I’ve writ­ten lots on the con­cur­ren­cy is­sues that ar­rive along with Ni­a­gara (and ev­ery oth­er chip de­sign­er is head­ing down the same mul­ti­thread­ed path); if you want more, check out CMT is com­ing: Is your ap­pli­ca­tion ready? by Richard McDougal­l, with tons of point­ers to big, se­ri­ous pieces on the sub­jec­t, in­clud­ing one by Kun­le Oloko­tun, the man be­hind Ni­a­gara.
 
The Joy of Threads · I’ve had quite a bit to say here about how con­cur­rent soft­ware, which is get­ting more im­por­tant, re­mains bru­tal­ly difficult—beyond the reach, some say, of many ap­pli­ca­tion pro­gram­mer­s. I’m a lit­tle wor­ried about neg­a­tive spin, be­cause if you en­joy pro­gram­ming, you should give con­cur­ren­cy a try; some of us find it es­pe­cial­ly sat­is­fy­ing. I can re­mem­ber like yes­ter­day in the un­der­grad CS course when I first un­der­stood what a “process” was, and then a few years lat­er the same feel­ing when I re­al­ly got thread­s. Yeah, it’s tough; you’ll find your­self de­bug­ging by print state­men­t, and some­times with a compile-run-think cy­cle time mea­sured in min­utes. But when you have the com­put­er do­ing a bunch of things at on­ce, and they all fit to­geth­er and the right things hap­pen fast, well, that’s some pret­ty tasty brain can­dy. All this brought to mind dur­ing our re­cent long week­end in the English coun­tryside; it seemed en­tire­ly rea­son­able to me to sit in a qui­et cor­ner of the pub, or with a view of the ocean, and get a few of those compile-run-think cy­cles in. I can un­der­stand that not ev­ery­one feels this way, but to all the coders out there: this stuff is not on­ly good for your ca­reer, it can be its own re­ward.
 
Threads Redux · The June 12th On Threads piece got slash­dot­ted (twen­ty thou­sand hits for a 2,300 word hard-tech piece, not bad), which pro­voked re­al­ly in­ter­est­ing feed­back from (a­mong oth­er­s) David Da­gas­tine, Greg Wil­son, and Ben Holm, along with point­ers to some re­lat­ed work. All those point­ers are worth fol­low­ing, and some of the points are worth a lit­tle more dis­cus­sion ...
 
On Threads · Last week I at­tend­ed a Sun “CMT Summit”, where CMT stands for “Chip Multi-Threading”; a room­ful of re­al­ly se­nior Sun peo­ple talk­ing about the next wave of CPUs and what they mean. While much of the con­tent was stuff I can’t talk about, I was left with a pow­er­ful feel­ing that there are some re­al im­por­tant is­sues that the whole IT com­mu­ni­ty needs to start think­ing about now. I’ve writ­ten about this be­fore, and of the many oth­ers who have too, I’m par­tic­u­lar­ly im­pressed by Chris Rijk’s work. But I think it’s worth­while to pull all this to­geth­er in­to one place and do some calls to ac­tion, so here goes. [Ed. Note: Too long and too geeky for most.] [Up­date: This got slash­dot­ted and I got some re­al­ly smart feed­back, thus this follow-up.] ...
 
Laptops and Servers · It just dawned on me that in a desk-side box, heat is not a ma­jor prob­lem be­cause it’s got a room to cool it of­f, usu­al­ly with­out too many oth­er com­put­ers in it. Lap­top­s, how­ev­er, are like back-room “mainframe” servers in that heat is a big deal. In the one case you’re wor­ried about your users’ go­nads and in the oth­er your HVAC bud­get, but the prob­lem is the same. Right now, the heat bud­get is a big con­cern for the guys at Sun who de­sign our biggest server­s. It’s no se­cret that our through­put com­put­ing ini­tia­tive is part­ly about this: low­er clock speed­s, more cores, thread-level par­al­lelism. Am I pre­dict­ing that lap­tops with SPARC pro­ces­sors will be leap­ing off the shelves next quar­ter? I wouldn’t go that far. But I am smelling con­verg­ing de­sign spaces.
 
Software in the TLP Era · Fly­ing over the At­lantic, I read all eight parts of Chris Rijk’s Thread Lev­el Par­al­lelism De­sign De­ci­sions, and I wish a few more soft­ware geeks would go and read it. Here­with a few notes on soft­ware de­sign in the era of Thread Lev­el Par­al­lelis­m ...
 
author · Dad · software · colophon · rights
Random image, linked to its containing fragment

By .

I am an employee
of Amazon.com, but
the opinions expressed here
are my own, and no other party
necessarily agrees with them.

A full disclosure of my
professional interests is
on the author page.