Darkstar
Reputation: 765
API requests with PHP slow loading time
I'm using PHP to get price data and volume data from several bitcoin exchanges but when you load the page it takes close to 20 seconds. How can I make the load time better? I think it has something to do with the curl.
<?php
function getData($url) {
$curl = curl_init();
curl_setopt($curl, CURLOPT_URL, $url);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
$rawData = curl_exec($curl);
curl_close($curl);
return json_decode($rawData, true);
}
//BTC Volume LocalBitcoins
$BTCVolumeLocal = getData('https://localbitcoins.com/bitcoinaverage/ticker-all-currencies/');
$LocalVolume = $BTCVolumeLocal["USD"]["volume_btc"];
//BTC Volume BTCE
$BTCVolumeBTCE = getData('https://btc-e.com/api/3/ticker/btc_usd');
$BTCEVolume = $BTCVolumeBTCE["btc_usd"]["vol_cur"];
//BTC Volume Bitstamp
$BTCVolumeStamp = getData('https://www.bitstamp.net/api/ticker/');
$StampVolume = $BTCVolumeStamp["volume"];
//BTC Volume Bitfinex
$BTCVolumeFinex = getData('https://api.bitfinex.com/v1/pubticker/btcusd');
$FinexVolume = $BTCVolumeFinex["volume"];
//BTC Volume OKCoin
$BTCVolumeOK = getData('https://www.okcoin.com/api/ticker.do?ok=1');
$OKCoinVolume = $BTCVolumeOK["ticker"]["vol"];
//BTC Volume LakeBTC
$BTCVolumeLake = getData('https://www.lakebtc.com/api_v1/ticker');
$LakeVolume = $BTCVolumeLake["USD"]["volume"];
//Totals the Volumes
$TotalVolume = $LakeVolume + $FinexVolume + $OKCoinVolume + $StampVolume + $BTCEVolume + $LocalVolume;
//Percents of Total Volume
$BTCEPercent = $BTCEVolume / $TotalVolume;
$StampPercent = $StampVolume / $TotalVolume;
$FinexPercent = $FinexVolume / $TotalVolume;
$OKPercent = $OKCoinVolume / $TotalVolume;
$LakePercent = $LakeVolume / $TotalVolume;
$LocalPercent = $LocalVolume / $TotalVolume;
//BTC Price BTCE
$BTCPriceBTCE = getData('https://btc-e.com/api/3/ticker/btc_usd');
$BTCEPrice = $BTCPriceBTCE["btc_usd"]["last"];
//BTC Price Bitstamp
$BTCPriceStamp = getData('https://www.bitstamp.net/api/ticker/');
$StampPrice = $BTCPriceStamp["last"];
//BTC Price Bitfinex
$BTCPriceFinex = getData('https://api.bitfinex.com/v1/pubticker/btcusd');
$FinexPrice = $BTCPriceFinex["last_price"];
//BTC Price OKCoin
$BTCPriceOK = getData('https://www.okcoin.com/api/ticker.do?ok=1');
$OKPrice = $BTCPriceOK["ticker"]["last"];
//BTC Price LakeBTC
$BTCPriceLake = getData('https://www.lakebtc.com/api_v1/ticker');
$LakePrice = $BTCPriceLake["USD"]["last"];
//BTC Price LocalBitcoins
$BTCPriceLocal = getData('https://localbitcoins.com/bitcoinaverage/ticker-all-currencies/');
$LocalPrice = $BTCPriceLocal["USD"]["avg_1h"];
//BTC Price * Percent
$BTCEPricePercent = $BTCEPrice * $BTCEPercent;
$StampPricePercent = $StampPrice * $StampPercent;
$FinexPricePercent = $FinexPrice * $FinexPercent;
$OKPricePercent = $OKPrice * $OKPercent;
$LakePricePercent = $LakePrice * $LakePercent;
$LocalPricePercent = $LocalPrice * $LocalPercent;
//Bitcoin Price
$bitcoinPrice = round($LakePricePercent + $OKPricePercent + $FinexPricePercent + $StampPricePercent + $BTCEPricePercent + $LocalPricePercent, 2);
?>
Upvotes: 0
Views: 6849
Answers (4)
Professor Abronsius
Reputation: 33823
I found that using ajax the results all came back reasonably quickly because the requests were running asynchronously and this is, so far, work in progress..... If others here can see the benefit I thought I perceived from this approach no doubt they will nail it.
The idea is a js function sends off a series of ajax requests to the php script that then sends the curl request to the various bitcoin urls given. Each request is setup to carry with it the fields to be returned from the request - the php script then drills down through the data and finds that info and returns to js.
Calculating the various percentages remains an issue as you don't necessarily know when all requests have finished. I guess promises might be useful?
Certainly using ajax allows the page to load quickly - then the results come back 'as and when' until finished.....
bitcoin.php
-----------
<?php
if( $_SERVER['REQUEST_METHOD']=='POST' ){
$cacert='c:\wwwroot\cacert.pem';
/* Two of the three params sent via ajax */
$url=$_POST['url'];
$fields=$_POST['fields'];
/* $fields might be a comma separated list of fields, or simply one - we want an array */
$params=( !empty( $fields ) && strstr( $fields, ',' ) ) ? explode( ',', $fields ) : (array)$fields;
$curl=curl_init( $url );
if( parse_url( $url, PHP_URL_SCHEME )=='https' ){
curl_setopt( $curl, CURLOPT_SSL_VERIFYPEER, FALSE );
curl_setopt( $curl, CURLOPT_SSL_VERIFYHOST, 2 );
curl_setopt( $curl, CURLOPT_CAINFO, realpath( $cacert ) );
}
curl_setopt( $curl, CURLOPT_URL, $url );
curl_setopt( $curl, CURLOPT_HEADER, false );
curl_setopt( $curl, CURLOPT_FRESH_CONNECT, true );
curl_setopt( $curl, CURLOPT_FORBID_REUSE, true );
curl_setopt( $curl, CURLOPT_HTTP_VERSION, CURL_HTTP_VERSION_1_1 );
curl_setopt( $curl, CURLOPT_CLOSEPOLICY, CURLCLOSEPOLICY_OLDEST );
curl_setopt( $curl, CURLOPT_BINARYTRANSFER, true );
curl_setopt( $curl, CURLOPT_AUTOREFERER, true );
curl_setopt( $curl, CURLOPT_CONNECTTIMEOUT, 10 );
curl_setopt( $curl, CURLOPT_RETURNTRANSFER, true );
curl_setopt( $curl, CURLOPT_USERAGENT, 'Bitcoin Fuzzer' );
$data=trim( curl_exec( $curl ) );
$status=curl_getinfo( $curl,CURLINFO_HTTP_CODE );
curl_close( $curl );
/* Deal with the response data */
if( $status==200 ){
$json=json_decode( $data );
/*
Hack:
there must be a better way to drill down dynamically through
an object structure but I was too tired to think straight
This simply echos back the response for javascript to deal with
but I guess one could make use of sessions to do the calculations
in php rather than js
*/
switch( count( $params ) ){
case 1:
$p1=$params[0];
print_r( $json->$p1 );
break;
case 2:
$p1=$params[0];
$p2=$params[1];
print_r( $json->$p1->$p2 );
break;
case 3:
$p1=$params[0];
$p2=$params[1];
$p3=$params[2];
print_r( $json->$p1->$p2->$p3 );
break;
}
}
}
?>
the html page - head:
<script type='text/javascript' charset='utf-8'>
/* simple ajax function */
function _ajax( url, options ){
var req=new XMLHttpRequest();
var callback=options.hasOwnProperty('callback') ? options.callback : false;
if( !callback ) return false;
var headers={
'Accept': "text/html, application/xml, application/json, text/javascript, "+"*"+"/"+"*"+"; charset=utf-8",
'Content-type': 'application/x-www-form-urlencoded',
'X-Requested-With': 'XMLHttpRequest'
};
var params=[];
if( options.hasOwnProperty('params') && typeof( options.params )=='object' ){
for( var n in options.params ) params.push( n + '=' + options.params[n] );
}
var args=options.hasOwnProperty('args') ? options.args : options;
req.onreadystatechange=function(){
if( req.readyState==4 ) {
if( req.status==200 ) options.callback.call( this, req.response, args );
else console.warn( 'Error: '+req.status+' status code returned' );
}
}
req.open( 'POST', url, true );
for( header in headers ) req.setRequestHeader( header, headers[ header ] );
req.send( params.join('&') );
}
var volumes=[];
var prices=[];
var tot_vols=0;
var tot_prices=0;
function bitcoin(){
var btc={
vols:{
'https:\/\/localbitcoins.com\/bitcoinaverage\/ticker-all-currencies\/':['USD','volume_btc'],
'https:\/\/btc-e.com\/api\/3\/ticker\/btc_usd':['btc_usd','vol_cur'],
'https:\/\/www.bitstamp.net\/api\/ticker\/':['volume'],
'https:\/\/api.bitfinex.com\/v1\/pubticker\/btcusd':['volume'],
'https:\/\/www.okcoin.com\/api\/ticker.do?ok=1':['ticker','vol'],
'https:\/\/www.lakebtc.com\/api_v1\/ticker':['USD','volume']
},
prices:{
'https:\/\/btc-e.com\/api\/3\/ticker\/btc_usd':['btc_usd','last'],
'https:\/\/www.bitstamp.net\/api\/ticker\/':['last'],
'https:\/\/api.bitfinex.com\/v1\/pubticker\/btcusd':['last_price'],
'https:\/\/www.okcoin.com\/api\/ticker.do?ok=1':['ticker','last'],
'https:\/\/www.lakebtc.com\/api_v1\/ticker':['USD','last'],
'https:\/\/localbitcoins.com\/bitcoinaverage\/ticker-all-currencies\/':['USD','avg_1h']
}
};
var url;
var vols=btc.vols;
var prices=btc.prices;
for( url in vols ){
getbitcoin.call( this, url, vols[url], 'volumes' );
}
for( url in prices ){
getbitcoin.call( this, url, vols[url], 'prices' );
}
}
function getbitcoin( url, fields, type ){
var options={
callback:cbbtc,
method:'POST',
params:{
'fields':fields,
'type':type,
'url':url
}
};
_ajax.call( this, '/test/bitcoin.php', options );
}
function cbbtc(r,o){
switch( o.params.type ){
case 'volumes':
tot_vols += parseFloat( r );
volumes.push( parseFloat( r ) );
document.getElementById('btc_vols').value=tot_vols;
break;
case 'prices':
tot_prices += parseFloat( r );
prices.push( parseFloat( r ) );
document.getElementById('btc_prices').value=tot_prices;
break;
}
}
/* launch the function when the page is ready */
document.addEventListener( 'DOMContentLoaded', bitcoin, false );
</script>
html page - body:
/* These simply receive values for testing */
<input type='text' id='btc_vols' />
<input type='text' id='btc_prices' />
Upvotes: 2
Rob Bailey
Reputation: 1787
Multithreading and/or caching, friend.
Multithreading in PHP is no tougher than in any other language. You can find instructions here: How can one use multi threading in PHP applications
You can use simple caching with Memcache or Redis, and there are all kinds of tutorials and helps on the web to use these to cache stuff in PHP.
Upvotes: 1
fico7489
Reputation: 8560
You have 12 service calls, and 20 seconds is real. I'm not familiar with your app but you can consider implementing some caching (Save downloaded data to files and later load from downloaded files).
$url = 'http://www.example.com';
$content = '';
$file_name = 'tmp/cache.txt';
if(file_exists($file_name))){
$content = file_get_contents($file_name);
}else{
$content = file_get_contents($url);
//or $content = getData($url);
file_put_contents($file_name, $content);
}
function getData($url) {
$curl = curl_init();
curl_setopt($curl, CURLOPT_URL, $url);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
$rawData = curl_exec($curl);
curl_close($curl);
return json_decode($rawData, true);
}
Upvotes: 2
Pampy
Reputation: 979
If the APIs don't respond fast enough and they are not under your control, you probably won't be able to change this.
Querying them in parallel might speed things up, but as mentioned in the comments, that's usually not that easy using PHP.
If it's just about the loading time of your page, you could cache the result of the API queries:
- Put the API requests in a cronjob, that's called automatically every x minutes. Depending on how fast the input data is changing, this can be as low as 1 minute (which unfortunately means you'll create a lot of traffic), or you could query it once an hour if that's okay for your calculation.
- Save the result to a local database or cache (MySQL, Redis, Memcache, ...)
- For your calculation only read from your local copy of the values, which will be a lot faster than querying the services every time
Upvotes: 5