Author: Manuel Lemos
Viewers: 48
Last month viewers: 21
Categories: PHP Tutorials
If you ever tried to send e-mail messages that never seem to reach the destination, don't blame PHP limited built-in mail capabilities!
You may be victim of having your mail server blacklisted for SPAM that you never sent.
This article explains a situation that may cause innocent mail servers to be blacklisted very easily.
Other short news mention a PHP IDE survey, how to follow the latest package blog posts by RSS or on Twitter, and an upcoming update of the site privacy policy.
You may be victim of having your mail server blacklisted for SPAM that you never sent.
This article explains a situation that may cause innocent mail servers to be blacklisted very easily.
Other short news mention a PHP IDE survey, how to follow the latest package blog posts by RSS or on Twitter, and an upcoming update of the site privacy policy.
Contents
* PHP IDE survey
* Follow new package blog posts using RSS or Twitter
* Site privacy policy update
* Avoid being blacklisted innocently
Before the main subject of this post, as usual, here follows a small list of updates on news of interest.
* PHP IDE survey
CodeGear was the Borland division that created Delphi for PHP IDE. CodeGear was acquired by Embarcadero Technologies last year. They are now carrying a survey about PHP IDE and asked me to let you know, so you can provide feedback and help developing better PHP IDE suites.
This is an interesting survey even if you are not interested in using Delphi for PHP. It includes questions about PHP development and where to improve Delphi for PHP to support more and more PHP developers.
If you would like to help, they say it will not take you more than 15 minutes to fill the survey. Here you may find 4 different versions of the survey in different idioms.
English:
Japanese:
Portuguese:
Spanish:
* Follow new package blog posts using RSS or Twitter
As it was mentioned in the previous blog post, the PHPClasses site now allows class authors to post articles about their packages in blogs that exist separately for each package.
Each blog has its own RSS feed, but if you are interested in blog articles about any packages published in the site, there is now a page that aggregates all articles of all package blogs:
The numbers that appear next to each listed article are the numbers of the articles in their own blogs. So you may see apparently repeated article numbers.
If you want to follow these blogs, you can subscribe to these RSS feeds:
RSS 1.0
RSS 2.0
If you are a Twitter user, you can also follow the new package blog posts by following the site Twitter account: phpclasses .
It already aggregates several things like the latest classes published in the site, latest book reviews, latest job posts, latest main site and package blog posts.
* Site privacy policy update
As you may be aware, the PHPClasses site displays advertising in its pages that is served by the Google AdSense program. This is a program that places advertising relevant to the content of each page where the ads appear.
Like many other thousands of sites in the world, the PHPClasses site receives from Google AdSense part of the revenue that keeps a busy site like this financially sustainable.
To participate in the AdSense program the site must follow Google guidelines. These guidelines have just been updated. Now Google requires all sites to update our privacy policy to mention a new aspect that is going to be announced soon.
In a few days the site will exhibit advertising based on the interests of each user. This means that Google will keep track of the types of pages that you visit in all sites that participate in AdSense. This will be used to determine the topics related to the advertising that they will be shown.
For instance, if you visit pages about Web development like those of the PHPClasses site, you are more likely to see advertising about this topic on this and other sites that participate in AdSense.
Keep in mind that under no circumstances the PHPClasses site will share details of the registered user accounts with Google or any other site. That would violate the site privacy policy. It never happened before, it will not happen ever.
The way this works is that Google will set a cookie to identify each user and will use that to keep record of the interests of the user based on the visited pages.
Personally, I think this is a good idea because it will make advertising more interesting and relevant to the interests of each user. However, some users that are more concerned with privacy issues, may not appreciate the idea so much.
If you are not interested that Google keeps track of your interests based on the pages that you visit, you can opt out in the Google Ads Preferences page:
Anyway, since the PHPClasses site privacy needs to be updated to explain this, in a few days, logged users will be required to agree the new site privacy policy before you can proceed accessing the site as logged user.
* Avoid being blacklisted innocently
Just a few days ago I got a bounced e-mail message. The bounced message mentioned that it was not delivered because the site mail server IP address was blacklisted.
Some time ago I gave a talk on how to sending e-mail and avoid all the problems that may prevent that it reaches the destination. One of the things to avoid is being blacklisted. The slides of the talk are available here:
The presentation suggests that you check the OpenRBL site to find out whether your IP address is blacklisted in several places. Since the presentation was given, that site is no longer working for that purpose. So now you may use the Black List Alert site instead:
- SPAM traps
Being blacklist is odd because the site does not send spam nor its security was compromised. The bounce message pointed to a page of the UCE Protect service.
UCE Protect is a service that uses spam traps to list sites that send SPAM. Spam traps are fake e-mail addresses that are usually spread by honey pot sites to catch spammers that steal addresses from harvested site pages.
If spammers send messages to spam trap addresses they are caught sending unsolicited e-mail and the sender IP address becomes blacklisted.
PHPClasses is a reputed site that exists for almost 10 years and never sends spam. At most it sends useful newsletters or alert messages about new content, but only to users that voluntarily subscribe to the site.
So how come the site IP address got blacklisted? The fact that the IP address is not blacklisted anywhere else, made me wonder if this was not a problem specific to UCE Protect.
They recommend that we check the site mail server logs for messages sent about a given day and time when they got messages from the site mail server to their spam trap addresses.
After analyzing the mail server logs, it turned out that the server was sending replies to messages of certain domains that were indeed the domains used by UCE Protect spam trap e-mail addresses.
- Auto-replying to spam trap messages
The logs shown hundreds of incoming messages that used those spam trap domains as senders. Most messages were sent to invalid addresses, so they were immediately discarded.
The whole problem is that some of the messages were sent to valid addresses. Those addresses are used for instance to take requests from users that want to unsubscribe from the site newsletters.
There are no users subscribed using spam trap addresses. So, initially I thought it would be a good idea to ignore unsubscribe request messages from non-existing subscriber e-mail addresses. However, I realized that would be a bad idea.
What happens is that some users subscribe with addresses different from those that they regularly use to send their messages. Usually are equivalent addresses or addresses that forward messages to the user main address.
In this case, the site usually sends an automated reply message telling the user that there is no such subscriber address. That reply message also explains that the user needs to find out his real subscriber address by looking at the To: header of the newsletters that he no longer wants to receive.
This is pretty much the same way many millions of mailing lists work everywhere in the world, including those of Google Groups and Yahoo Groups. So basically all newsletter or mailing list servers are vulnerable to the risk of being blacklisted innocently.
- Avoiding sending messages to spam trap addresses
I was in a dilemma. Either I stopped sending automated replies to non-existing subscriber addresses or the site could remain blacklisted forever.
The immediate solution was to find out which are the spam trap domains of messages that were causing blacklisting and reject all incoming or outgoing messages to those domains.
This will work as a temporary solution, but I will not be able to automatically realize when new spam trap domains start being used.
I decided to contact the support people of UCE Protect service. After some e-mail exchanges they explained that they do not send messages with sender addresses set to spam traps.
The conclusion is that who was sending those messages were actual spammers, probably with the intention to confuse spam combat services like UCE Protect.
The problem is that the way UCE Protect works, it is very easy to blacklist innocent mail servers like of the PHPClasses site and millions of other servers everywhere that run auto-responders.
This is interesting because it seems spammers are trying to defeat spam combat services by turning sites that get innocently listed against UCE Protect and other services that work the same way.
- Using SPF to discard spam trap messages
Since the spam trap messages the site mail server was getting were forged, the only way to avoid them is to discard the messages, so they never get processed by auto-responders.
However, most sites do not know which are the spam trap domains. That information is not published because it would help spammers to clean the e-mail lists they harvested.
Another way to make spam trap messages be discarded automatically would be to use SPF standard (Sender Policy Framework).
This is a standard that lets domain owners specify a restricted list of addresses of mail servers that are allowed to send messages from their domains. Any messages sent by addresses not listed in SPF records should be discarded, as they are most likely spam.
The PHPClasses site has set an SPF record on the DNS to tell that messages from its domain can only come from its own server.
The problem is that the domains of spam traps used by UCE Protect do not have SPF records set. I tried to suggest their support people to set SPF records but I did not yet get a response on whether they will do it.
- Conclusion
If UCE Protect does not add SPF records to the spam trap domains, the only way to avoid this problem for now, is to manually add all known spam trap domains to your own server mail server blacklist.
The PHPClasses site uses qmail. So I only had to add the domains to qmail badmailfrom control file.
I think every site that has mail servers that handle mailing lists, newsletters or any other form of auto-responders need to add spam trap domains to their internal black lists to avoid being blacklisted by UCE Protect.
I am just not going to publicly tell which are the spam trap domains that I found out because that would help spammers. Anyway, if for some reason you get innocently blacklisted by UCE Protect, feel free to use the contact link at the bottom of the PHPClasses site pages to ask me.
For any other questions, please post a comment to this post here.
* PHP IDE survey
* Follow new package blog posts using RSS or Twitter
* Site privacy policy update
* Avoid being blacklisted innocently
Before the main subject of this post, as usual, here follows a small list of updates on news of interest.
* PHP IDE survey
CodeGear was the Borland division that created Delphi for PHP IDE. CodeGear was acquired by Embarcadero Technologies last year. They are now carrying a survey about PHP IDE and asked me to let you know, so you can provide feedback and help developing better PHP IDE suites.
This is an interesting survey even if you are not interested in using Delphi for PHP. It includes questions about PHP development and where to improve Delphi for PHP to support more and more PHP developers.
If you would like to help, they say it will not take you more than 15 minutes to fill the survey. Here you may find 4 different versions of the survey in different idioms.
English:
infopoll.net/live/surveys/s33343.ht ...Japanese:
video.codegear.com/pix/NickHodges/S ...Portuguese:
video.codegear.com/pix/NickHodges/S ...Spanish:
video.codegear.com/pix/NickHodges/S ...* Follow new package blog posts using RSS or Twitter
As it was mentioned in the previous blog post, the PHPClasses site now allows class authors to post articles about their packages in blogs that exist separately for each package.
phpclasses.org/blog/post/90-Blogs-f ...Each blog has its own RSS feed, but if you are interested in blog articles about any packages published in the site, there is now a page that aggregates all articles of all package blogs:
phpclasses.org/blog/package/all/The numbers that appear next to each listed article are the numbers of the articles in their own blogs. So you may see apparently repeated article numbers.
If you want to follow these blogs, you can subscribe to these RSS feeds:
RSS 1.0
phpclasses.org/blog/package/all/pos ...RSS 2.0
phpclasses.org/blog/package/all/pos ...If you are a Twitter user, you can also follow the new package blog posts by following the site Twitter account: phpclasses .
twitter.com/phpclassesIt already aggregates several things like the latest classes published in the site, latest book reviews, latest job posts, latest main site and package blog posts.
* Site privacy policy update
As you may be aware, the PHPClasses site displays advertising in its pages that is served by the Google AdSense program. This is a program that places advertising relevant to the content of each page where the ads appear.
google.com/adsenseLike many other thousands of sites in the world, the PHPClasses site receives from Google AdSense part of the revenue that keeps a busy site like this financially sustainable.
To participate in the AdSense program the site must follow Google guidelines. These guidelines have just been updated. Now Google requires all sites to update our privacy policy to mention a new aspect that is going to be announced soon.
In a few days the site will exhibit advertising based on the interests of each user. This means that Google will keep track of the types of pages that you visit in all sites that participate in AdSense. This will be used to determine the topics related to the advertising that they will be shown.
For instance, if you visit pages about Web development like those of the PHPClasses site, you are more likely to see advertising about this topic on this and other sites that participate in AdSense.
Keep in mind that under no circumstances the PHPClasses site will share details of the registered user accounts with Google or any other site. That would violate the site privacy policy. It never happened before, it will not happen ever.
The way this works is that Google will set a cookie to identify each user and will use that to keep record of the interests of the user based on the visited pages.
Personally, I think this is a good idea because it will make advertising more interesting and relevant to the interests of each user. However, some users that are more concerned with privacy issues, may not appreciate the idea so much.
If you are not interested that Google keeps track of your interests based on the pages that you visit, you can opt out in the Google Ads Preferences page:
google.com/ads/preferences/Anyway, since the PHPClasses site privacy needs to be updated to explain this, in a few days, logged users will be required to agree the new site privacy policy before you can proceed accessing the site as logged user.
* Avoid being blacklisted innocently
Just a few days ago I got a bounced e-mail message. The bounced message mentioned that it was not delivered because the site mail server IP address was blacklisted.
Some time ago I gave a talk on how to sending e-mail and avoid all the problems that may prevent that it reaches the destination. One of the things to avoid is being blacklisted. The slides of the talk are available here:
The presentation suggests that you check the OpenRBL site to find out whether your IP address is blacklisted in several places. Since the presentation was given, that site is no longer working for that purpose. So now you may use the Black List Alert site instead:
blacklistalert.org/- SPAM traps
Being blacklist is odd because the site does not send spam nor its security was compromised. The bounce message pointed to a page of the UCE Protect service.
uceprotect.net/UCE Protect is a service that uses spam traps to list sites that send SPAM. Spam traps are fake e-mail addresses that are usually spread by honey pot sites to catch spammers that steal addresses from harvested site pages.
If spammers send messages to spam trap addresses they are caught sending unsolicited e-mail and the sender IP address becomes blacklisted.
PHPClasses is a reputed site that exists for almost 10 years and never sends spam. At most it sends useful newsletters or alert messages about new content, but only to users that voluntarily subscribe to the site.
So how come the site IP address got blacklisted? The fact that the IP address is not blacklisted anywhere else, made me wonder if this was not a problem specific to UCE Protect.
They recommend that we check the site mail server logs for messages sent about a given day and time when they got messages from the site mail server to their spam trap addresses.
After analyzing the mail server logs, it turned out that the server was sending replies to messages of certain domains that were indeed the domains used by UCE Protect spam trap e-mail addresses.
- Auto-replying to spam trap messages
The logs shown hundreds of incoming messages that used those spam trap domains as senders. Most messages were sent to invalid addresses, so they were immediately discarded.
The whole problem is that some of the messages were sent to valid addresses. Those addresses are used for instance to take requests from users that want to unsubscribe from the site newsletters.
There are no users subscribed using spam trap addresses. So, initially I thought it would be a good idea to ignore unsubscribe request messages from non-existing subscriber e-mail addresses. However, I realized that would be a bad idea.
What happens is that some users subscribe with addresses different from those that they regularly use to send their messages. Usually are equivalent addresses or addresses that forward messages to the user main address.
In this case, the site usually sends an automated reply message telling the user that there is no such subscriber address. That reply message also explains that the user needs to find out his real subscriber address by looking at the To: header of the newsletters that he no longer wants to receive.
This is pretty much the same way many millions of mailing lists work everywhere in the world, including those of Google Groups and Yahoo Groups. So basically all newsletter or mailing list servers are vulnerable to the risk of being blacklisted innocently.
- Avoiding sending messages to spam trap addresses
I was in a dilemma. Either I stopped sending automated replies to non-existing subscriber addresses or the site could remain blacklisted forever.
The immediate solution was to find out which are the spam trap domains of messages that were causing blacklisting and reject all incoming or outgoing messages to those domains.
This will work as a temporary solution, but I will not be able to automatically realize when new spam trap domains start being used.
I decided to contact the support people of UCE Protect service. After some e-mail exchanges they explained that they do not send messages with sender addresses set to spam traps.
The conclusion is that who was sending those messages were actual spammers, probably with the intention to confuse spam combat services like UCE Protect.
The problem is that the way UCE Protect works, it is very easy to blacklist innocent mail servers like of the PHPClasses site and millions of other servers everywhere that run auto-responders.
This is interesting because it seems spammers are trying to defeat spam combat services by turning sites that get innocently listed against UCE Protect and other services that work the same way.
- Using SPF to discard spam trap messages
Since the spam trap messages the site mail server was getting were forged, the only way to avoid them is to discard the messages, so they never get processed by auto-responders.
However, most sites do not know which are the spam trap domains. That information is not published because it would help spammers to clean the e-mail lists they harvested.
Another way to make spam trap messages be discarded automatically would be to use SPF standard (Sender Policy Framework).
en.wikipedia.org/wiki/Sender_Policy ...This is a standard that lets domain owners specify a restricted list of addresses of mail servers that are allowed to send messages from their domains. Any messages sent by addresses not listed in SPF records should be discarded, as they are most likely spam.
The PHPClasses site has set an SPF record on the DNS to tell that messages from its domain can only come from its own server.
The problem is that the domains of spam traps used by UCE Protect do not have SPF records set. I tried to suggest their support people to set SPF records but I did not yet get a response on whether they will do it.
- Conclusion
If UCE Protect does not add SPF records to the spam trap domains, the only way to avoid this problem for now, is to manually add all known spam trap domains to your own server mail server blacklist.
The PHPClasses site uses qmail. So I only had to add the domains to qmail badmailfrom control file.
I think every site that has mail servers that handle mailing lists, newsletters or any other form of auto-responders need to add spam trap domains to their internal black lists to avoid being blacklisted by UCE Protect.
I am just not going to publicly tell which are the spam trap domains that I found out because that would help spammers. Anyway, if for some reason you get innocently blacklisted by UCE Protect, feel free to use the contact link at the bottom of the PHPClasses site pages to ask me.
For any other questions, please post a comment to this post here.
You need to be a registered user or login to post a comment
Login Immediately with your account on:
Comments:
7. Nonsense and Waste of time - Miklo (2014-04-07 04:51)
Nonsense and Waste of time... - 0 replies
Read the whole comment and replies
6. Avoid being blacklist... - Ingo Buse (2010-07-12 09:08)
you can say what you want... - 0 replies
Read the whole comment and replies
5. UCEPROTECT does nothing wrong - Thomas Berger (2010-04-18 23:11)
YOU send backscatter... - 10 replies
Read the whole comment and replies
2. SPF - Damian Williamson (2009-12-17 02:48)
SPF is to enable detection of invalid messages origin... - 5 replies
Read the whole comment and replies
4. confusion with spam - Michel COLLET (2009-04-18 00:34)
image URL with parameters or not?... - 1 reply
Read the whole comment and replies
1. Uceprotect - Halil Özgür (2009-04-01 17:55)
Spam fighters doesn't use spam fighting technologies :)... - 4 replies
Read the whole comment and replies
3. please help me - neghab (2009-04-01 08:48)
i send a newsletter... - 1 reply
Read the whole comment and replies
