Begin main content

Parallel sorting in Erlang

Of course super-light threads for parallel programming is Erlang's killer feature, so let's apply that to a simple merge sort. Erlang's spawning and message passing interface is fairly straightforward, but a lot of parallel tasks are simple start -> process -> return type tasks, so we can simplify our syntax even further by using some nice helper functions such as Joe Armstrong's parallel map pmap or Anonymous Matthew's parallel eval pareval (via Philip Robinson's blog).

Parallel map makes light work of a massively parallel merge sort:

-module(mergesort).
-export([msort/1]).

%% terminal cases for list of less than 2 entries

msort([]) ->
    [];
msort([A]) ->
    [A];

% split list into 2 & mergesort them

msort(L) ->
    Mid = trunc(length(L) / 2),
    {A, B} = lists:split(Mid, L),
    lists:umerge( pmap(fun(I) -> msort(I) end, [A, B]) ).
This gets really interesting in certain cases eg. where acquisition of the data is expensive but randomly accessible.

Instead of passing in a simple list, our sort function could accept a future that would promise to return any requested linear subset of the data. Assuming we know the total length of the source list, no data is needed until we reach the merge phase at the bottom of the recursion stack, and incrementally more data is required as we bubble back up to the final result.

In this example we are doing our network retrieval in parallel with our sorting (which itself is happening in parallel). If the remote data comes from multiple hosts we could parallelise the retrieval by using multiple port instances.

03:58 PM, 03 Dec 2007 by Mark Aufflick Permalink | Short Link | Comments (0)

XML

Blog Categories

software (39)
..cocoa (21)
  ..heads up 'tunes (5)
..ruby (6)
..lisp (3)
..perl (4)
..openacs (1)
mac (20)
embedded (2)
..microprocessor (2)
  ..avr (1)
electronics (3)
design (1)
photography (25)
..black and white (6)
..A day in Sydney (18)
..The Daily Shoot (6)
food (2)
Book Review (2)

Notifications

Icon of Envelope Request notifications

Syndication Feed

XML

Recent Comments

  1. Unregistered Visitor: An other Script
  2. Unregistered Visitor: A message in there?
  3. Unregistered Visitor: using Amazon S3
  4. Unregistered Visitor: Thank you ! Thank you ! Thank you !
  5. Unregistered Visitor: Umbrello on leopard
  6. Unregistered Visitor: Script gor generate for library
  7. Unregistered Visitor: Similar but different
  8. Unregistered Visitor: Thanks for fixing my problem!
  9. Unregistered Visitor: Pop up once the category is been defined
  10. Unregistered Visitor: smal amendment