PHP Classes

Forker PHP: Split tasks into multiple forked processes

Recommend this page to a friend!
  Info   View files View files (29)   DownloadInstall with Composer Download .zip   Reputation   Support forum   Blog    
Last Updated Ratings Unique User Downloads Download Rankings
2024-01-09 (4 days ago) RSS 2.0 feedNot enough user ratingsTotal: 350 This week: 1All time: 6,986 This week: 348Up
Version License PHP version Categories
forker 1.0.13MIT/X Consortium ...5.3PHP 5, Unix, Language
Description 

Author

This package can split tasks into multiple forked processes.

The main class takes an array of data to be processed as the whole task and creates child processes giving one element of the tasks array to be processed by each child process by calling given a given function that will process the respective task data.

It uses a separate storage object to store and retrieve task data to be shared between processes.

Currently it provides different storage classes that can store task data in memory as arrays, in files, or in a MemCached server.

Innovation Award
PHP Programming Innovation award nominee
May 2014
Number 4


Prize: PhpStorm IDE personal permanent license
Some CPU intensive tasks can be executed more efficiently if they are split into processes running in parallel. That way it takes advantage of multiple CPU cores that may be available.

This class can manage the split of processing of array data using forked processes to execute the same processing task in parallel to process each array element.

Manuel Lemos
Picture of Daniel Martinez
Name: Daniel Martinez <contact>
Classes: 3 packages by
Country: Spain Spain
Innovation award
Innovation award
Nominee: 2x

Recommendations

Threads PHP
php_fork

Details

Forker ------------ [![Build Status](https://travis-ci.org/danibrutal/Forker.svg?branch=master)](https://travis-ci.org/danibrutal/Forker) [![Latest Stable Version](https://poser.pugx.org/danibrutal/forker/v/stable.png)](https://packagist.org/packages/danibrutal/forker) ## Synopsis A structured , safe, and easiest way to perform tasks parallely in PHP. ## Code Example ```php <?php /************************************************** * Example: Retrieving the city-weather using external api * Usage : php examples/demo.api.weather.php * Storage: File **************************************************/ require 'vendor/autoload.php'; use Forker\Forker; use Forker\Storage\FileStorage; $allCitiesWeather = ""; $urlApiWeather = "http://api.openweathermap.org/data/2.5/weather?q=%s&mode=xml"; $myTasks = array( 'madrid' => sprintf($urlApiWeather, 'Madrid'), 'london' => sprintf($urlApiWeather, 'London'), 'new-york' => sprintf($urlApiWeather, 'NewYork'), 'barcelona' => sprintf($urlApiWeather, 'barcelona'), 'lisboa' => sprintf($urlApiWeather, 'lisboa'), 'iasi' => sprintf($urlApiWeather, 'iasi'), ); // a way to keep our data $storageSystem = new FileStorage; $numberOfSubTasks = 6; $forker = new Forker($storageSystem, $myTasks, $numberOfSubTasks); $forker->fork(function($city, $url, $emit) { echo "Retrieving weather in $city\n"; $contents = file_get_contents($url); $emit($city, $contents); }); $allCitiesWeather = $forker->fetch(); var_dump($allCitiesWeather); ``` ## Motivation Sometimes we have to work with a huge amount of data. [Lately](http://en.wikipedia.org/wiki/Big_data), more and more, and it's just no possible to work sequentially these times. For example, we have two cooks in a kitchen and a very big carrot. Well, we want our two workers not to be waiting for each other. So why we don't split the carrot in two parts so they can work each one with a part ? This way both cooks can work together, at the same time, and we will have our dinner soon ! Great huh ? So, the intention is to make an agile and encapsulated way to split a task in several child subtasks in parallel. ## Install Using composer: ``` "require": { "danibrutal/forker": "dev-master" } ``` ## API Reference You can check the API out [here](http://testdouble.es/Forker/API/); ## Creating your own StorageSystem: We follow here a TDD aproach so is extremely easy to develop a new system: 1º Create your own storage system following the StorageSystem interface's signature: ```php /** * @param key * @param value * @return bool */ public function store($key, $value); /** * @param key * @return value | false */ public function get($key); /** * @return array $tasks */ public function getStoredTasks(); /** * @return bool */ public function cleanUp(); ``` 2º Creates a test ```php <?php use Forker\Storage\ArrayStorage; require_once 'BaseStorageTest.php'; class ArrayStorageTest extends BaseStorageTest { protected function getSystemStorage() { return new ArrayStorage(); } } ``` Hard, huh? 3º Then, type phpunit so you can see 3 errors to solve: ``` There were 4 failures: 1) ArrayStorageTest::testWeCanGetASimpleStoredValue Failed asserting that null matches expected 'value'. 2) ArrayStorageTest::testWeCanSToreValues Failed asserting that null is true. 3) ArrayStorageTest::testIcanGetAllMyStoredTasks Failed asserting that a NULL is not empty. 4) ArrayStorageTest::testWeCanCleanUpAllPreviousTasks Failed asserting that a NULL is not empty. ``` 4º Just solve the errors. Create your implementation and you are done! Easy and funny :) ## Contributors Please, feel free to colaborate. Fork the project and check [issues][2]. We still have so much work ahead!. ## License MIT

  Files folder image Files  
File Role Description
Files folder imageexamples (4 files, 1 directory)
Files folder imagesrc (1 directory)
Files folder imagetests (1 file, 1 directory)
Accessible without login Plain text file .travis.yml Data Auxiliary data
Accessible without login Plain text file composer.json Data Auxiliary data
Accessible without login Plain text file LICENSE.md Data Auxiliary data
Accessible without login Plain text file phpunit.xml Data Auxiliary data
Accessible without login Plain text file README.md Data Auxiliary data

  Files folder image Files  /  examples  
File Role Description
Files folder imageMapReduce (4 files)
  Accessible without login Plain text file demo.api.weather.file-storage.php Example Example script
  Accessible without login Plain text file demo.api.weather.php Example Example script
  Accessible without login Plain text file demo.sum.php Example Example script
  Accessible without login Plain text file demo.timeout.php Example Example script

  Files folder image Files  /  examples  /  MapReduce  
File Role Description
  Accessible without login Plain text file demo.map-reduce.php Example Example script
  Accessible without login Plain text file quijote-1.txt Data Auxiliary data
  Accessible without login Plain text file quijote-2.txt Data Auxiliary data
  Accessible without login Plain text file quijote-3.txt Data Auxiliary data

  Files folder image Files  /  src  
File Role Description
Files folder imageForker (5 files, 2 directories)

  Files folder image Files  /  src  /  Forker  
File Role Description
Files folder imageException (2 files)
Files folder imageStorage (4 files)
  Plain text file Alarm.php Class Class source
  Plain text file ChilProcess.php Class Class source
  Plain text file Forker.php Class Class source
  Plain text file ISemaphore.php Class Class source
  Plain text file Semaphore.php Class Class source

  Files folder image Files  /  src  /  Forker  /  Exception  
File Role Description
  Plain text file ForkingErrorException.php Class Class source
  Plain text file StorageException.php Class Class source

  Files folder image Files  /  src  /  Forker  /  Storage  
File Role Description
  Plain text file ArrayStorage.php Class Class source
  Plain text file FileStorage.php Class Class source
  Plain text file MemcacheStorage.php Class Class source
  Plain text file StorageInterface.php Class Class source

  Files folder image Files  /  tests  
File Role Description
Files folder imageunit (1 file, 1 directory)
  Accessible without login Plain text file bootstrap.php Aux. Auxiliary script

  Files folder image Files  /  tests  /  unit  
File Role Description
Files folder imageStorage (3 files)
  Accessible without login Plain text file ForkerTest.php Test Unit test script

  Files folder image Files  /  tests  /  unit  /  Storage  
File Role Description
  Accessible without login Plain text file ArrayStorageTest.php Test Unit test script
  Accessible without login Plain text file BaseStorageTest.php Test Unit test script
  Accessible without login Plain text file FileStorageTest.php Test Unit test script

 Version Control Unique User Downloads Download Rankings  
 100%
Total:350
This week:1
All time:6,986
This week:348Up