Parallel list processing module for Erlang
While reading the erlang mailing list I came across a (fairly) new module plists containing parallel versions of the erlang lists module and a few throwins (like a mapreduce implementation). The functions operated just like the standard variants plus an optional parameter that allows you to tune how many parallel processes are run and whether to use remote nodes or not.
plists is a drop-in replacement for the Erlang module lists, making most list operations parallel. It can operate on each element in parallel, for IO-bound operations, on sublists in parallel, for taking advantage of multi-core machines with CPU-bound operations, and across erlang nodes, for parallizing inside a cluster. It handles errors and node failures. It can be configured, tuned, and tweaked to get optimal performance while minimizing overhead....
This module also include a simple mapreduce implementation, and the function runmany. All the other functions are implemented with runmany, which is as a generalization of parallel list operations.
Nice.
03:32 PM, 12 Dec 2007 by Mark Aufflick Permalink | Short Link







