CODEWITHSUNDEEP
phpxmlapi
Tyler Nichol
Tyler Nichol

Reputation: 655

Looping through large xml response then sending element back in another loop for response

There has to be a better way! This is my code:

checksession();
$restNew2 = new RESTConnector();


$urlNew2 = "https://localhost:9630/api/products/";
$restNew2->createRequest($urlNew2,"GET", null, $_SESSION['cookies'][0]);
$restNew2->sendRequest();


$responseNew2 = $restNew2->getResponse();
$xmlNew2 = new SimpleXMLElement($responseNew2);

foreach ($xmlNew2 as $purge){
    $id = (string)$purge->attributes()->id;

checksession();
$restNew = new RESTConnector();


$urlNew = "https://localhost:9630/api/products/".$id."/";
$restNew->createRequest($urlNew2,"GET", null, $_SESSION['cookies'][0]);
$restNew->sendRequest();


$responseNew = $restNew->getResponse();
$xmlNew[] = new SimpleXMLElement($responseNew);



}
$array = array();

foreach ($xmlNew as $purgeDet){
    $code[] = (string)$purgeDet->code;
    $classId[] = (string)$purgeDet->class['id'];
    for($i=0, $count = count($xmlNew); $i < $count; $i++) {
$array[$code[$i]]['classId'] = $classId[$i];
    }
}
print_r($array);

The first response gives me about 300,000 lines back so then I have to parse through that to grab the id of a product in order to send another request to get a full render of the product. It ends up sending like 40,000 requests. My hopes is to insert this into a mySQL database at night using cron so I can create reports on it the next day. Any ideas? Thanks!

Upvotes: 0

Views: 168

Answers (1)

prodigitalson
prodigitalson

Reputation: 60413

Id do it in chunks. Make the first API call and loop over it building a list of just ID's and store it. Id store them in a file with 1 per line... then you can the file line by line later instead of loading the whole array into buffer OR you coudl still read the file to an array... whichever ends up being the better solution.

Then read the file and loop through the IDs making your detail calls and storing them.

UPDATE:

You could also keep the grabbing of the IDs on the compiled side by using an xpath query.

$xmlNew2->xpath("//THE_ELEMENT_WITH_THE_ID/@id");

But i dunno if this is going to save you much because youd still need to loop over this... it just saves you have to call SimpleXMLElement::attributes on each node. But the result is still going to be array of attribute elements.

Upvotes: 1

Related Questions