Starting Long Server PHP Scripts and Monitor their Status Part 3: Running Multiple Parallel Tasks - Asynchronous Long Server Tasks package blog

Contents

Introduction

Do We Really Need a Multi-threaded Solution?

How to Run Many Long Tasks at the Same Time?

How to Control the Resources?

Starting Several Tasks

Opportunities to Improve the Solution

Conclusion

Introduction

In the first and second parts of this tutorial a solution was presented to run a single task on the server. However, the solution only allowed one long task to run at a given moment.

This limitation was imposed intentionally to prevent that too many long tasks running on the server would exhaust the server memory or take too much CPU, thus making the server too slow.

On the other hand, if you have a big server you may be able to run much more than one long task in parallel. This article tells you how to achieve that with the Asynchronous Long Server Task class.

Do We Really Need a Multi-threaded Solution?

Not really, to be accurate. The long tasks are actually run as Web server requests. Most Web servers are able to handle many simultaneous requests time, so we can take advantage of that feature to squeeze the server resources and run more long tasks at the same time.

How to Run Many Long Tasks at the Same Time?

The package for running one long task is the same to run many parallel tasks at the same time, but I created a few more PHP scripts that I am documenting in this part of the article. The files are:

• file-list-multi.php to display the new file list in the browser

• start-copy-multi.php to start the long task from the browser

• status-copy-multi.php to get the status of the tasks from the browser

• server/copy-file-multi.php implements the long server task

How to Control the Resources?

The same example presented in the second part of the article will be used in this part for the same purpose by changing the point of view.

We need now to exit the script as soon as possible and do not take care at all of the resources. In the previous part we were worried about to not crash the server and we queued the tasks in order to do them sequentially. In this part we will squeeze at maximum the resources and try to exit as soon as possible.

In the initial version of the package, in my development environment test it was impossible to run more than three tasks at the same time. If you increased the number of status files to more than 3 the consecutive requests you needed to wait for the previous tasks to finish. Later I tested the class on a production environment and did not observe any limits in the number of tasks.

The first version of the package was very simple, it did not use file locking. File locking is necessary to prevent that concurrent requests corrupt the status. The latest version uses locking on the status file using flock(), so it works well to queue many long tasks

Anyway, using different status files for parallel requests is a more robust solution because each task cannot cause corruption of the status files of other tasks and file locking is not necessary.

But we need to change the logic of our requests. Now we need to handle all the requests at the same time and let the server that processes them handle it with the available RAM and CPU resources.

Here is the JavaScript code that we use to start all tasks at once when the start task link is clicked.

$("#start-task").click(function(e){
    e.preventDefault();
    $(this).hide();
    for(var i=0; i<fileList.length; i++) {
        requestCopyOfFile(i);
    }
    return false;
})

function requestStatusOfFiles() { ... }

Starting Several Tasks

A different status file is used for each file copy task that is started, so the tasks do not interfere with each other. Keep in mind that actually all requests are handled by the same server PHP script and if they access to the same status file, it prevents the taking advantage of parallel processing. Therefor when each task uses its one status file it works better.

Here you can watch a video with a screencast of the example scripts page in action, as well the AJAX requests that are being sent to the server on the browser developer console.

Opportunities to Improve the Solution

One issue that we did not handle in this series of articles is the security of the solution when starting the server tasks. Keep in mind that our tasks are called from an URL and everybody who know its address and the format of the parameters can call it.

I am thinking that we could create a simple token that can be verified in both parts, as a pair of public-private keys. In the next days I will update the package to implement this improvement.

Another likely improvement that will be implemented is an option passed to the class constructor to determine the method that would be use to start the task.

In the beginning, the package was the curl external command program. Now it is using the PHP fsockopen function. Another possibility is to use the fopen function to avoid problems in PHP environments that have the fsockopen function disabled.

Conclusion

This part of the article presented a solution that allows to start more long server requests at the same time.

With this part I conclude this series of articles, despite there are several opportunities to improve the solution in different aspects.

If you like the article or you have questions about the solutions, post a comment here now.