|
mikescott - 2011-11-05 09:36:16
No change to my code, but I am not getting any data from Google - have they changed something?
Richard Keizer - 2011-11-05 15:45:26 - In reply to message 1 from mikescott
It looks like it needs updating:
if you fetching with a browser google returns different markup than fetching using fopen.
Problably google uses useragent checking and format according to that.
I think if you fetch using curl and set proper useragent things might work again.
Richard Keizer - 2011-11-05 16:39:57 - In reply to message 1 from mikescott
change the code in 'getHtmlCodeViaFopen' to:
function getHtmlCodeViaFopen($url){
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
$headers = array();
$headers[] = "Accept: text/html";
$headers[] = "User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:7.0.1) Gecko/20100101 Firefox/7.0.1";
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$r = curl_exec($ch);
return $r;
}
mikescott - 2011-11-05 18:16:51 - In reply to message 3 from Richard Keizer
Hi
Not accessing Google via browser. This is used to directly import into mySQL database from a PHP script that uses the class. The script has worked flawlessly for, oh, I don't know, a year(?), but the past couple of weeks, some of the 20 or so currencies are shown as having a zero rate - re-run the script, and another 'set' (possibly overlapping the previous 'set') of currencies are shown as having a zero rate - and for the past two days ALL the currencies are shown as having a zero rate.
The currencies are in common use - GBP, USD, EUR etc. etc.
I don't know that the mod that is proposed would be of any benefit, but I would appreciate guidance or feedback.
Richard Keizer - 2011-11-05 18:23:26 - In reply to message 4 from mikescott
the mod works.. try it..
Richard Keizer - 2011-11-05 18:27:39 - In reply to message 4 from mikescott
by sending the user-agent google thinks you are using a browser.
that's what makes the mod work.
mikescott - 2011-11-05 18:47:14 - In reply to message 6 from Richard Keizer
Hi Richard
At first I thought that this had failed, because I had incorporated a mod provided to me via private message earlier by the originator. That mod didn't improve matters, and even with you code it fails.
BUT... revert (and thank goodness for svn) to the original code, add your function, and hey, presto all is well!!
Curiosity forces me to ask whether your function is somewhat version-specific?
Many, many thanks.
Mike
Richard Keizer - 2011-11-05 19:26:05 - In reply to message 7 from mikescott
don't know... I just downloaded the current version and made the patch. I know it works with the version currently downloadable via this site, i'm not even aware of multiple versions.
mikescott - 2012-08-22 11:35:43 - In reply to message 8 from Richard Keizer
Hi there again. Is see that nearly a year has passed, and with your help I fixed the problem that I had encountered. Everything was hunky-dory, but in the past few days, the results retrieved from Google are all zeros for each of the 19 currencies that my script uses. I presume that Google has changed something, because the data is still available via Google - manually - but it is tiresome to have to update all the data manually every weekday!
Any guidance would be most appreciated. I can send my current code if that helps.
|