I managed to ignore Atom for a few hours this week and get back to working on project Zeppelin, which leads to a few thoughts on object transmission, concurrency, Jython, and other stuff of possible interest to hands-on Javans.
Writing Objects Fast ·
A few weeks back, I
wrote about improving the
latency of client-server
Java Object network transmission from appalling to merely bad.
I had trouble getting it to run any better than bad, then
Mike Duigou made the
obvious suggestion that I jam a BufferedOutputStream in front of
the socket, d’oh, now
I can push a thousand objects a second through a decent network link.
The difference between this time and all the other times in my career I’ve made systems run faster by buffering output is, I didn’t have to write the software myself. This “Object-Oriented” stuff is gonna catch on.
Concurrent Pain · The way Zeppelin works is you’ve got a client that talks to a bunch of different servers, and in some cases it has to send a transaction to all of them. Stress-testing showed me the obvious, that doing this one by one was slow and stupid, so each client now keeps a bunch of pet threads around and when a transaction needs to go everywhere, you load it up and unleash all the threads, which write and read and report back, and you’re done. The efficiency went up enormously.
Once I’d debugged it, that is.
Concurrency, well y’know, it’s hard. After you’ve finished debugging it,
it’s obvious that if the parent says while (true) { load(transaction);
start(pets); waitfor(pets); } and the pet threads say
while (true) { waitfor(parent); send(); receive(); if(iAmTheLastPetToFinish)
{ tell(parent); } }, there’s this hole in the concurrency logic that you
only fall into on about the eight thousandth time around the loop, varying
wildly as a function of anything else the computer might be doing.
The first problem is that old programmers like me have a bunch of well-debugged design patterns in our heads which get us through most problems without thinking too much. But we don’t (well, I don’t anyhow) have nearly enough concurrent-system design patterns in there. Is there a book someone would like to recommend?
The second problem is that I haven’t figured out how to do Test-Driven Development of concurrent systems. I know how to write tests that catch an iterator that doesn’t iterate and numeric code that loses precision and a million other stupid things that every programmer does. But I don’t really know how to write tests that catch simple, bone-headed, obvious errors in concurrent systems. So I end up using a lot of print statements.
Dynamic Languages, Please · Another routine in the grotty underbelly of Zeppelin does a bunch of file/directory maintenance: create a shadow directory tree if it’s not there and copy a bunch of files selected by extension hither and yon. I’m sure programmers will still be writing this kind of code when I’m a hundred years in the grave.
Well, Java is really not the right tool. What would have been a few lines of straightforward Perl turned into half a dozen klunky-looking Java methods. If I’d been able to rely on Jython just being there and in my IDE, I would have come out way ahead. Fortunately, I think we’re going to fix that one...
