Begin main content

MapReduce in Perl

Someone recently asked to get access to the OpenACS paypal-support package that I wrote and lives in my cvs repository. Unbenknownst to me, my recent firewall change blocked access to the cvs web interface, which I have now fixed.

While I was looking around my public cvs interface, I remembered that I had never quite finished my Perl implementation of Google's MapReduce infrastructure.

Since I'm too busy to get back onto it now I figure I may as well unleash it on you guys. Be warned that there's no documentation (that I remember) and that the current implementation only works on a single machine, although the structure is easily extendable to multiple machines by replacing the forking with some form of rpc.

My sample Perl implementation of MapReduce (for sufficiently weak meaning of the word 'implementation' ;)

Given that limitation it may also not quite work properly - I really don't remember what state I left it in. Nevertheless if you first read the MapReduce paper by Jeffrey Dean and Sanjay Ghemawat then the api and code should make sense.

If anyone finds this interesting please feel free to email me any questions. If enough people are interested I may be able to make the time to polish it off and write documentation!

11:36 PM, 17 Nov 2006 by Mark Aufflick Permalink | Comments (0)

XML

Blog Categories

software (41)
..cocoa (23)
  ..heads up 'tunes (5)
..ruby (6)
..lisp (4)
..perl (4)
..openacs (1)
mac (21)
embedded (2)
..microprocessor (2)
  ..avr (1)
electronics (3)
design (1)
photography (26)
..black and white (6)
..A day in Sydney (18)
..The Daily Shoot (6)
food (2)
Book Review (2)

Notifications

Icon of envelope Request notifications

Syndication Feed

XML

Recent Comments

  1. Mark Aufflick: Re: the go/Inbox go/Sent buttons
  2. Unregistered Visitor: How do make a button to jump to folder
  3. Unregistered Visitor: Note I've updated the gist
  4. Unregistered Visitor: umbrello is now an available port on macPorts
  5. Unregistered Visitor: Updated version on Github
  6. Unregistered Visitor: Modification request.
  7. Unregistered Visitor: Accents and labels with spaces
  8. Unregistered Visitor: Mel Kaye - additional info
  9. Unregistered Visitor: mmh
  10. Mark Aufflick: Thank you