[This is part of the Wide Finder 2 series.] I’m happy to report that I’ve given out a bunch of accounts on the Wide Finder 2 machine. I’ll aggregate links to others’ work in this entry.
2008/06/02 · Ray Waldin has a bunch of posts: WideFinder in Scala, WiderFinder, WideFinder 2 Strawman implemented in Scala, Multicore WideFinding, and Multicore Sorter.
Alex Morega has Wide Finder, Wide Finder: Going Parallel, and Wide Finder: Many Cores.
Did I Miss You? · If you’re working on this let me know and I’ll keep adding links.
Contributions
Comment feed for ongoing:
From: Alex Morega (Jun 03 2008, at 03:06)
One more article: http://grep.ro/blog/2008/05/wide_finder_many_cores
[link]
From: MenTaLguY (Jun 03 2008, at 19:15)
I'd just like to caution folks about taking shortcuts in the reduce phase, for example collecting only the top N entries from each worker when you want to find the overall top N entries.
Let's say that we're using two workers to determine the top two entries. Here are the full results from our workers:
Worker 0:
{ bob: 9, dana: 8, laura: 4 }
Worker 1:
{ betty: 10, steve: 9, laura: 7 }
The overall ranking (with lexical ordering of names breaking ties) is:
{ laura: 11, betty: 10, bob: 9, steve: 9, dana: 8 }
And the top two entries are:
{ laura: 11, betty: 10 }
However, if you had only taken the top two entries from each worker, you will instead have gotten:
{ betty: 10, bob: 9 }
"laura" wasn't in the top two in either worker's list individually, but was actually top overall.
[link]
From: Erik Engbrecht (Jun 04 2008, at 05:42)
I'm working it but unfortunately haven't had time to blog about it...I decided with limited time coding was more useful than blogging.
I should have something by the end of the week.
[link]