10 Steps to properly do PHP Bug Tracking and Fixing as Fast as possible - Log watcher package blog

Contents

1. Test as Much as Possible Before in your Development Environment

2. Test in a Staging server Before you Send it to the Production Server

3. Separate your Code from Environment Configuration files

4. Use a Version Control System to Maintain Project files, Except Configuration files

5. Track PHP Errors with Assertion Condition Tests

6. Do Not Display PHP Errors on your Web pages

7. Send PHP Errors to an Error Log File

8. Add more Error Context Information to PHP Errors Logs

9. Monitor the PHP Error Log File to Quickly Fix Serious Bugs

10. Fix Your Bugs but Never Edit Code on the Production Server

More PHP Defensive Programming Practices

1. Test as Much as Possible Before in your Development Environment

This one should be obvious but it is amazing how many PHP developers do not do it.

The worst case is of those developers that do not use a development environment at all for their PHP applications. If you are not sure what is a development environment, that is your case.

A development environment is a setup that you should have in your local computer on which you do your PHP development as if you were in your production environment.

The development environment should be as much as possible like the production server. So you should run the same Web server and if possible the same operational system.

If there are differences between your development and production environments, these differences may cover bugs in your application that only show up in the production server.

In the ideal world you should have test scripts that test every aspect of your application. However, in the real world it is very hard or even impossible to test everything.

Furthermore, elaborating tests for all your code makes your development take much more time to finish. A good compromise is to elaborate tests for those 20% of your code that are responsible for the 80% of the features that cannot fail.

Anyway, regardless how much you test your code, there will always exist bugs in your code that will slip into production. This is why you need to add more safeguards to your development and deployment practices like those that follow.

2. Test in a Staging server Before you Send it to the Production Server

If your development environment cannot be similar enough to your production server, you can create a staging environment that is more similar to your production server.

A staging environment is often separate machine that replicates the production server but it runs on its own database and its own domain.

If you cannot afford setting up a separate machine, you may try creating a staging environment on the same machine as the production server, as long as it runs on the separate database and domain.

You may deploy your application updates to your production server only after you have tested your application in the staging environment . Chances are that you may have caught some bugs in your staging environment, so less bugs will ship in the version deployed in the production environment.

3. Separate your Code from Environment Configuration files

Having multiple environments to test your application is great but you may wonder, how will my application know in which environment it is running and how do I deal with different databases and site host names?

Those details that may be different in each environment should be defined in configuration files. One practical way to implement this is to create a configuration class for your application. It should have variables that define the values that vary from environment to environment.

 class configuration
 {
   /*
    * store error messages in case an error happens
    */
   public $error = '';

   /*
    * Path of the main application relative to the current directory
    */
   public $application_path = '.';

   /*
    * Host name of the site to compose absolute URLs
    */
   public $host = 'www.mysite.com';

   /*
    * name of the site database
    */
   public $database_name = 'production-database';

   /*
    * name of the account to access the database
    */
   public $database_user = '';

   /*
    * password of the account to access the database
    */
   public $database_password = '';

   /*
    * Initialize the configuration loading options from a local script
    */
   function Initialize()
   {
     $local_options = $this->application_path .
       '/configuration/'. 'local_options.php';
     if(!file_exists($local_options))
     {
       $this->error = 'the application configuration file '.
         $local_options.'does not exist';
       return false;
     }
     require($local_options);
     return true
   }

   function Fail()
   {
     error_log('Error: '.$this->error);
     exit;
   }
 };

Your application scripts may start like this. They will make the configuration class load option values from a script  named configuration/local.php that is inside your application directory.

 $options = new configuration;

 /*
  * Adjust the application path relative to the current script directory
  */
 $options->application_path = '.';

 if(!$options->Initialize())
   $options->Fail();

Now you can define different scripts for each of the environments (development, staging or production) setting the application options accordingly.

The configuration script path should be configuration/local.php . Each environment should have a different configuration script, which should look like this:

 $this->host = 'www.mysite.com';
 $this->database_host = 'some database host';
 $this->database_name = 'some database name';
 $this->database_user = 'some database user';
 $this->database_password = 'some database password';

4. Use a Version Control System to Maintain Project files, Except Configuration files

You should always use a version control system (VCS) to maintain your application source code, but keep in mind that the configuration/local.php script should never be committed to the VCS repository. The reason for this is that it is not a static file in your project. It is different depending on whether you are in the development, staging or production environment.

The default values for the options in the configuration class should be the ones to be used in the production environment, except for sensitive values that are related with the application security, like for instance the database access host name, user and password.

The reason for this is that only your staff that has access to the production server should be able to know those sensitive values. It would be dangerous if for some reason you need to fire a developer that knows how to access the databases in your production server. He could destroy your application database as revenge for getting fired.

5. Track PHP Errors with Assertion Condition Tests

Once you deploy to the production server, some bugs may still creep in. This is bad, but the reality is that it happens regularly. Do not blame yourself for not having done enough testing on the development and staging environments. You may be a good developer, but nobody is perfect. We are human, therefore our software will always have bugs.

Assuming that we will never be able to write 100% bug free software, the next thing you should do is to try to detect the symptoms of bugs to minimize the eventual damages caused by the bugs. You can do that by inserting some assertion code, I mean code that tests conditions that should never be true at run time.

Take a look at this example. It checks a condition that should never happen and makes the script exit before crashing, passing a meaningful error message to the configuration class above, so it can fail gracefully.

 if($y == 0)
 {
    $options->error = '$y is 0 near line '.__LINE__.' of '.__FILE__;
    $options->Fail();
 }
 $x = 1 / $y;
 

6. Do Not Display PHP Errors on your Web pages

This is fine but in real projects that you do not know where you have your bugs, it is hard to anticipate where you should place your assertion condition checks.

If the assertion code above is not present, the PHP code runs code that triggers a PHP warning.

These warnings are useful but for security reasons you should never show PHP runtime error messages to your users. Otherwise people with malicious intentions could use the information disclosed by the PHP error messages to abuse from your site somehow.

You can prevent that PHP displays errors on the your Web page output by having the following line in your php.ini configuration file:

 display_errors = Off

7. Send PHP Errors to an Error Log File

If you should not display PHP errors on the Web pages, you should be able to see them somehow. A better alternative is to send the errors to a error log file. This way you can watch what is going on without disclosing sensitive information to your site users.

You can enable PHP error logs adding a few configuration lines to your php.ini file. Additionally you also set other useful PHP options.

 error_reporting        = E_ALL
 log_errors             = On
 error_log              = /path/to/php_error_log
 log_errors_max_len     = 0
 ignore_repeated_errors = On
 ignore_repeated_source = Off
 report_memleaks        = On
 track_errors           = On
 html_errors            = Off

8. Add more Error Context Information to PHP Errors Logs

PHP error warnings may be useful to debug your code but sometimes they lack of context that could help understanding the problem faster.

For instance the error may have happened in a certain line of a function but if you do not know exactly what are the parameters passed to that function, nor the location of the code that called that function, you may have a hard time figuring why the error occurred.

You can enhance your PHP error logs with more error context information if you override the PHP default error log handler. You can do that using the PHP function set_error_handler. You can define a new function that will inspect the current PHP execution context and add more details to the PHP error log.

PHP error handlers are fine but they are not able to intercept fatal PHP errors, like for instance including a PHP file with an incorrect path, or creating an object of class with an incorrect name, etc..

A workaround for that limitation of PHP error handlers is to register PHP shutdown function to check if PHP ended the execution with a fatal error. That can be done with the PHP function register_shutdown_function.

You can do all this from your configuration class. Here is a more complete version of the class.

 class configuration
 {
   /*
    * store error messages in case an error happens
    */
   public $error = '';

   /*
    * Path of the main application relative to the current directory
    */
   public $application_path = '.';

   /*
    * Host name of the site to compose absolute URLs
    */
   public $host = 'www.mysite.com';

   /*
    * name of the site database
    */
   public $database_name = 'production-database';

   /*
    * name of the account to access the database
    */
   public $database_user = '';

   /*
    * password of the account to access the database
    */
   public $database_password = '';

   /*
    * A common error handler code for fatal and non-fatal errors
    */
   Function BaseErrorHandler($error, $message, $file, $line, $backtrace)
   {
     if($error & error_reporting())
     {
       $log=array();
       switch($error)
       {
         case E_ERROR:
         case E_USER_ERROR:
           $type='FATAL';
           break;
         case E_WARNING:
         case E_USER_WARNING:
           $type='WARNING';
           break;
         case E_NOTICE:
         case E_USER_NOTICE:
           $type='NOTICE';
           break;
         default:
           $type='Unknown error type ['.$error.']';
           break;
       }
       $log[]=str_repeat('_',75);
       $request_uri = GetEnv('REQUEST_URI');
       $agent = GetEnv('HTTP_USER_AGENT');
       $log[]=$type.': '. $message. ' in line '. $line. ' of file '. $file.
         ', PHP '. PHP_VERSION. ' ('.PHP_OS.')'.
         (strlen($request_uri) ? ' '.$request_uri : '').
         ((IsSet($_POST) && count($_POST)) ?
           ' POST='.serialize($_POST) : '').
         ((IsSet($_GET) && count($_GET)) ?
           ' GET='.serialize($_GET) : '').
         ((IsSet($_FILES) && count($_FILES)) ?
           ' FILES='.serialize($_FILES) : '').
         (strlen($agent) ? ' AGENT="'.$agent.'"' : '').
         (IsSet($_SERVER[ 'REQUEST_METHOD' ]) ?
           ' METHOD="'. $_SERVER['REQUEST_METHOD']. '"'.
         ($_SERVER['REQUEST_METHOD'] === 'POST' ?
           ' POST="'. serialize($_POST). '"' : '') : '');
       for($level=1;$level < count($backtrace);$level++)
       {
         $message='File: '. $backtrace[$level]['file'].
           ' Line: '. $backtrace[$level]['line']. ' Function: ';
         if(IsSet($backtrace[$level] ['class']))
           $message.='(class '. $backtrace[$level] ['class'].') ';
         if(IsSet($backtrace[$level] ['type']))
           $message.=$backtrace[$level] ['type'].' ';
         $message.=$backtrace[$level] ['function'].'(';
         if(IsSet($backtrace[$level] ['args']))
         {
           for($argument=0; $argument < count($backtrace[$level]['args']); $argument++)
           {
             if($argument>0)
               $message.=', ';
             if(GetType( $backtrace[$level] ['args'] [$argument] )=='object')
               $message.='class '.
                 get_class($backtrace[$level] ['args'] [$argument]);
             else
               $message.= serialize( $backtrace[$level] ['args'] [$argument]);
           }
         }
         $message.=')';
         $log[]=$message;
       }
       error_log(implode("\n\t",$log));
     }
     if($error==E_ERROR)
       exit(1);
   }

   /*
    * Error handler for non-fatal PHP errors
    */
   Function CommonErrorHandler($error, $message, $file, $line)
   {
     $backtrace=(function_exists('debug_backtrace') ?
       debug_backtrace() : array());
     BaseErrorHandler($error, $message, $file, $line, $backtrace);
   }

   /*
    * Error handler for fatal PHP errors
    */
   Function FatalErrorHandler()
   {
     $error = error_get_last();
     if(IsSet($error))
       BaseErrorHandler($error['type'], $error['message'], $error['file'],
         $error['line'], array());
   }

   /*
    * Initialize the configuration loading options from a local script
    */
   function Initialize()
   {
     set_error_handler(array($this, 'CommonErrorHandler'));
     register_shutdown_function( array($this, 'FatalErrorHandler'));
     $local_options = $this->application_path. '/configuration/'. 'local_options.php';
     if(!file_exists($local_options))
     {
       $this->error = 'the application configuration file '.
         $local_options.'does not exist';
       return false;
     }
     require($local_options);
     return true
   }

   function Fail()
   {
     error_log('Error: '.$this->error);
     exit;
   }
 };

9. Monitor the PHP Error Log File to Quickly Fix Serious Bugs 

Error logs are useful to discover bugs that may be causing PHP errors but you cannot spend all day looking at your PHP error log files.

It would be better if you could be notified when a PHP error occurs. That is the purpose of the Log Watcher class. It can monitor a log file and send an email message to you when new lines that are added to the log file.

You can use it for instance from a script started periodically by cron, lets say every 5 minutes, and make it track any new lines added to the PHP error log file.

Here is a simple example of a script to send email messages to a given email address using the Log Watcher class:

 $log_watcher=new log_watcher_class;

 /*
  *  Specify the file name of the log file to watch
  */
 $log_watcher->log_file_name = '/path/to/php_error_log';

 /*
  *  Specify who is going to receive the notifications about the new lines
  *  added to the log file
  */
 $log_watcher->watcher_name = 'Your name';
 $log_watcher->watcher_email = 'your@email.com';

 /*
  *  Open the log file and point it to the last line that was read or the
  *  end of the file if this the first time the log is opened
  */
 $log_watcher->OpenLog();

 /*
  *  Send the new log file lines, if any, in a message with the specified
  *  subject
  */
 $log_watcher->MailNews( 'PHP error log report' );

 /*
  *  Close the log file and store the last line position in the pointer
  *  file
  */
 $success=$log_watcher->CloseLog();

10. Fix Your Bugs but Never Edit Code on the Production Server

Once you get new alert emails with new PHP errors that are occurring in your production server, you should act promptly and analyze the information to fix any bugs as soon as possible. This will help you prevent that greater harm is caused by the bugs in your site.

Most bugs need to be fixed urgently. However you should always do it adequately.

One thing that you never should do is to access your production server and edit your code directly. The problem with this approach is that your attempt to fix the code may cause even greater harm. Unfortunately I see many less experienced developers doing this. If you do this, stop doing it now.

The right way to do it is to fix and test your code in the development environment. You should only update your code in production when you are certain that the fix really solves the problem and does not cause other problems. You should also pass your updates through the staging server first.

More PHP Defensive Programming Practices

All these practices described above are just part of a broader development approach called defensive programming. It reflects many years of attempts to improve development practices in such way that you prevent yourself from being affected by bad habits that can cause greater harm.

This means that I learned from my own mistakes to make my practices better and generally safer. I have written about this before in articles about PHP defensive programming and surviving PHP site traffic peaks.

Certainly these are not the only defensive programming practices you can apply in PHP. There are certainly more practices that were not covered here.

What about you do you use other defensive programming practices? Would you improve any of the suggested practices somehow? Post a comment now with your opinion on this.