Contents
Introduction
Queueing and Delivering
E-mail delivery methods
1. PHP mail() function
2. SMTP server relay
3. Sending urgent messages by doing direct delivery to the destination SMTP server
4. Sendmail program
5. Qmail, Postfix, Exim, etc..
6. Microsoft Exchange pickup folder
7. Putting all recipients in Bcc headers
8. External Web services
9. Caching message bodies
Conclusions
Introduction
The default way for sending e-mail in PHP is using the mail function. Besides other limitations, this may not be the fastest solution to send many messages at once, like for instance when you need to send a newsletter to many subscribers.
Lets analyse the factors that may influence the different ways of deliverying messages in PHP.
Queueing and Delivering
The first concept that you need to understand is that sending e-mail messages is a task usually split in two phases: queueing and delivering. Please take a look at the following diagram in order to understand better these concepts.
The delivery is the process of making a message reach the final destination, which is the message recipient mailbox. As you may see at the bottom left corner of this diagram, the PHP script that sends messages is the origin of a delivery. At the opposite corner of the diagram there is the recipient mailbox which is is the delivery destination.
The recipient mailbox is some kind of storage that keeps received messages waiting for the recipient to retrieve and read them. Often recipient mailboxes are not in the same computer as the person's computer to whom the message is intended.
That person may need to use a program that speaks a mail retrieval protocol, like POP3 or IMAP, to retrieve the message. But that is outside the scope of this article. You may want to read this other article if you are interested in processing incoming e-mail messages in PHP.
Ideally the PHP script would inject the message directly in the recipient's mailbox. However, mailboxes may be implemented in many non-standard ways. Therefore, the actual delivery that we will consider here consists only of sending the message to the destination SMTP server. From then on, the SMTP server takes care of storing the received messages in the recipients' mailboxes.
It is possible for a PHP script or any program that knows how to speak in the SMTP protocol to inject the messages in the destination SMTP server directly, as you may read below. However, that is not a simple process for several reasons.
If there is a connectivity problem between the origin computer network and the destination SMTP server network, the message cannot be delivered right away. Even when there are no connectivity problems, the SMTP server may temporarily reject the messages.
This is the case of many SMTP servers that employ grey lists to avoid taking so much spam. Such servers temporarily reject the messages received from a computer of a certain IP address, forcing it to retry later after a certain period of time when the messages are finally accepted.
These circumstances force e-mail delivery systems, also known as Mail Transfer Agents (MTA),to have a way to keep retrying to send the messages until they are accepted or rejected permanently. This process of retrying deliveries may usually last several days before the MTA gives up and drop the messages.
The common solution employed by most MTAs is to keep a local queue on which the messages are stored temporarily until they are delivered or dropped. This is the reason why queueing is usually the first phase of the whole message delivery.
E-mail delivery methods
As you may have realized by now, when we want to compare the speed of different delivery methods, in practice we are just evaluating how fast the messages are queued by the local MTA. Lets compare how the messages are queued using different delivery methods available to PHP scripts.
1. PHP mail() function
Lets start with the PHP mail() function. The way this function is implemented depends on the platform on which PHP is running.
If you are running on Linux or some other UNIX-like system, the PHP mail function passes the message to the sendmail program or an equivalent mail server program (Qmail, Postfix, Exim, etc..) that is running on the same machine on which PHP is running.
If you are running PHP on Windows, the mail function implicitly sends the message to an SMTP server that you need to configure in the php.ini file.
The base class of the MIME message composing and sending package (email_message_class) uses the PHP mail function. The following sections explain how each of the two methods the mail function uses work.
2. SMTP server relay
SMTP is a protocol mainly meant to transfer e-mail messages between the origin and destination computers. SMTP is also used to relay messages to an intermediate SMTP server. That server stores the messages temporarily and then tries to deliver the messages to the destination SMTP server.
SMTP server relays are also often used by customers of ISP to quickly forward the message they send. The relay server will try deliver the messages later, even after the ISP customers have already disconnected from the Internet.
SMTP server relays are also used to queue messages on systems that do not provide a more direct method to inject messages in the local mail server queue. That is the case of when you use the PHP mail function on a Windows machine.
When you do not have a mail server running on your PHP hosting machine, your PHP script may also have to relay messages to a remote SMTP server. You may also need to do that if you do not want to cause excessive load to your hosting server. In that case you may prefer to resort to a remote SMTP service, which may even be paid. Despite it may be very slow to queue your messages this way, at least it may be a solution for your problem.
Having to establish an TCP/IP connection to an SMTP server every time you want to send a message, is something that adds significant time overhead to the mail delivery. When you need to send newsletters to many recipients, the SMTP protocol overhead may be too high.
The smtp_message_class sub-class provides a way to avoid part of that overhead. It can simply leave connections opened between consecutive message deliveries. All you need to do is to call the SetBulkMail function before you start the deliveries. That hints the class to do whatever can be done to reduce the time it takes to perform multiple message deliveries, which in this case is to keep the SMTP connections opened.
3. Sending urgent messages by doing direct delivery to the destination SMTP server
Sometimes you need to send urgent messages that must be received by the recipients as fast as possible. In that case, it would be interesting to totally skip the local mail server queue.
The smtp_message_class sub-class provides a variable named direct_delivery that you can use to tell the class to connect to the destination SMTP server and send it the message directly.
As mentioned above this may be slow process. You may even have to fallback to other another delivery method when the message cannot be delivered right away. Still, it may be an useful solution to deliver messages to individual recipients very quickly.
4. Sendmail program
Sendmail is a very old mail server software package available since the days of the first UNIX systems. Nowadays, it is present in most Linux based distributions. However, it is often not the default mail server system.
By default, the sendmail program attempts to deliver the message to the destination SMTP server right away. This may take a while because the destination SMTP server may be taking the incoming accesses very slowly. In some cases this may be intentional. Some SMTP servers are configured to delay message acceptance to discourage the delivery of many spam messages.
Intentionally or not, it is usually not a good idea wait for sendmail to deliver the messages. It makes the calling PHP script wait a long time until it can move on and send the message to the next recipient. If you want to send a newsletter to many subscribers, this method may be very bad because it can make your PHP script wait many hours until the delivery to all recipients is finished.
Fortunately sendmail has an option to disable immediate delivery attempts. The messages are just dropped in the mail server queue when this option is used.
The sub-class sendmail_message_class has support to control this option. Just set the delivery_mode class variable to SENDMAIL_DELIVERY_QUEUE. Alternatively you can call the SetBulkMail function to hint the class to use the queuing mode.
5. Qmail, Postfix, Exim, etc..
There are several Sendmail compatible packages that implement the same functionality as sendmail, in some aspects with advantages. The most popular Sendmail replacement packages are Qmail, Postfix and Exim.
These packages provide a wrapper program named sendmail to assure the compatibility with the original sendmail package. This is why when you run PHP on Linux or any other UNIX-like system, you need to set the sendmail_path option of your php.ini file to configure how the mail function will send your messages.
Usually the sendmail emulation program just drop the messages in the mail server queue. So you do not need to worry with additional configuration to make it not attempt to deliver the messages to the destination SMTP server right away.
Since the sendmail compatible program is basically a wrapper around another program, it would save you some time if you would call the wrapped program right away. That is what the qmail_message_class sub-class does. It just calls the qmail-inject program directly, instead of calling sendmail program.
6. Microsoft Exchange pickup folder
Microsoft Exchange also provides a way to queue messages very quickly. Its mail server queue has a pickup folder on which you can create files with the content of the messages that you want to send. Later Microsoft Exchange will pick the message files and takes care of the deliveries without stalling the programs that are sending the messages.
The pickup_message_class sub-class can determine the path of the Microsoft Exchange message pickup folder. As long as PHP and Microsoft Exchange are running on the same machine, you can use this class to queue newsletter messages to many recipients.
7. External Web services
Nowadays there are several companies offering paid mail delivery services. This may be a good idea to avoid causing to much load to the machine on which you are running PHP.
While this may solve the load problem, this method of mail queuing tends to be as slow as relaying messages to remote SMTP services. When the performance of your mail queuing script matters, do not use this solution unless you do not have a choice.
8. Putting all recipients in Bcc headers
If you want to send the same message to many newsletter subscribers, queuing can be done very quickly by sending a single message with all the recipients' addresses in Bcc headers. Actually this is the solution used by many mailing list managers.
The problem with this solution is that the original recipient addresses do not appear in the message headers. Some mail services, notably Hotmail, will consider spam the messages sent to addresses that do not appear in any of the visible headers: To or Cc. Usually the users of those services need to white-list the sender addresses of those mailing lists. So, this may not be a good solution in general.
9. Caching message bodies
Part of the work of the whole message delivery process consists on composing the message that you want to send. If you want to send the same message to all subscribers of a newsletter or mailing list, the message body is the same. In that case you only need to change the recipient addresses in the To header.
The email_message_class base class has a variable named cache_body that you can use to tell it to store the message data in a variable when its sent for the first time. In subsequent deliveries the class will reuse the previously computed message body, thus skipping the whole process of encoding and assembly all the message parts.
The time that you save doing this may be significant, especially when you are sending messages to a large number of recipients. You can still change any headers, including the recipient addresses, as the headers section is a message part which is distinct from the body.
You can use this option when you do not need to personalize the message bodies for each recipient. Therefore, if the details that you were considering to personalize in the message body are not important, you may want to avoid personalization altogether to achieve maximum message delivery performance.
Conclusions
Now that you learned all the details of delivering e-mail to many users, it should be easy to understand that the fastest ways to send your messages are those that avoid doing needless repetitive efforts or use shortcuts to achieve the same purposes.
When it comes to queuing messages, nothing is faster than just dropping messages directly in a message queue directory for your server to pickup. That is something that can be done with Microsoft Exchange, as long as your PHP script is running on the same machine and it has the necessary folder access permissions.
Qmail, Postfix and other properly configured sendmail compatible programs that also drop the messages in the local message queue can be very fast too.
What is usually not so fast is to relay messages to an SMTP server, even when it is running on the same machine, due to SMTP protocol overhead. Still, if you really have to use this method, you can still save sometime by keeping your SMTP server connections opened between consecutive message deliveries.
The same goes for using remote Web services to providers of e-mail delivery services. If you have to queue individual messages for each recipient, it will take you a long time to queue the messages to all your newsletter subscribers.
Finally, whenever possible, avoid message body personalization, i.e. avoid sending distinct message contents to different subscribers of the same newsletter or mailing list. That will allow you to gain some time by using a message body cache.
Following these techniques should get you a lot of performance gains. Still if you know of other interesting optimization techniques, please post a comment telling about them.
