This is the third progress report from the Wide Finder Project. Given that I launched this brouhaha late on a Friday, totally the worst possible time, I’m astounded at the intensity and quality of the conversation that’s going on. I want to address two themes that have emerged, one of which seems stupid and the other smart.
The Wrong Problem? · There’ve been comments and blogs along the lines of “WTF, trying to shake down Erlang using what amounts to a degenerate Awk script!!? This isn’t what Erlang is for! It’s all just I/O! Lines of text are so 1980’s!”
Nope; the further this goes, the more I think it’s a valid line of research. You know all those kazillions of Sun servers out there? Let me tell you, they’re not all running state-of-the-art Java-on-the-net apps. A huge proportion of all those cycles goes into Perl scripts and COBOL batches and C++ meat-grinders and FORTRAN number-crunching. Furthermore, if you look at the numbers from running my Perl and Ruby scripts, it’s obvious that they’re pegging both the CPU and I/O needles; so they’re nicely multidimensional in a simple way.
This is the kind of nasty unglamorous job that a lot of our customers run all the time to pay their rents, and this whole business is making a big bet on many-core computers, and it’s just not on to tell all those people that those are the wrong kind of jobs for the new iron.
The Good News · I first saw this on the Erlang Questions mailing list, from someone whose handle was Patrick: “The good news is speeding up the i/o in erlang should be easier than introducing better concurrency to another language.” Then there’s Patrick (the some one?) Logan saying the same thing in Postmodern I/O.
Not only will I kick the tires, I’ll try the code out on our unreleased T2-based many-core/many-thread monster. This is going to be fun.