slava-vishnyakov/map-reduce

MapReduce implementation

1.0.3 2018-04-17 18:05 UTC

Requires (Dev)

• phpunit/phpunit: ^6

MIT

README

To install:

composer require slava-vishnyakov/map-reduce

use \SlavaVishnyakov\MapReduce\MapReduceMemory;

$m = new MapReduceMemory(); // or MapReduceProcess
$m->send('a', 1);
$m->send('b', 1);
$m->send('a', 2);

foreach($m->iter() as $key => $groups) {
// yields
$key = 'a', $groups = [1,2]
$key = 'b', $groups = [1]

Also can be done via next() calls:

$m = new MapReduceMemory(); // or MapReduceProcess
$m->send('a', 1);
$m->send('b', 1);
$m->send('a', 2);
$m->send('a', new stdClass);

$m->next() ==> ['a', [1, 2, new stdClass]] 
$m->next() ==> ['b', [1]]
$m->next() ==> null

There are two implementations MapReduceMemory and MapReduceProcess.

The first does all sorting in memory, the second is for memory-hungry workloads, uses /usr/bin/sort to process basically unlimited amount of data.