CODEWITHSUNDEEP
ccurllibcurl
user179169
user179169

Reputation:

libcurl problem with getting data from curl_easy_perform()

This is probably a stupid question, but how do I actually get the site data returned by curl_easy_perform() into a variable that I can work with. When it executes, I just see it all flash by on the Terminal. I am using C, by the way.

Any ideas? Thanks.

EDIT: This is the code I'm using (I'm accessing the Twitter Streaming API, am I even doing it correctly?) 

int main(void)
{
  CURL *curl;
  CURLcode res;

  curl = curl_easy_init();
  if(curl) {
    curl_easy_setopt(curl, CURLOPT_URL, "http://stream.twitter.com/1/statuses/filter.json?track=http");
curl_easy_setopt(curl, CURLOPT_USERPWD, "JEggers2:password");
    res = curl_easy_perform(curl);

    /* always cleanup */ 
    curl_easy_cleanup(curl);


  }   
  return 0;
}

Upvotes: 7

Views: 15835

Answers (3)

Bayron Vazquez
Bayron Vazquez

Reputation: 371

The function writeDataOnStream has a void pointer called buffer this pointer hold all data (HTML CODE AND HEADERS)

#include <iostream>
#include <string.h>
#include <curl/curl.h>

using namespace std;

//ADD THIS FUNCTION: NOTE the Void pointer "buffer"
size_t writeDataOnStream(void * buffer, size_t size, size_t nbytes, void * stream){
    size_t bytes_written = fwrite( buffer, size, nbytes, (FILE *) stream);
    return bytes_written;
}

int main(void)
{
  CURL *curl;
  CURLcode res;

  curl = curl_easy_init();
  if(curl) {
     curl_easy_setopt(curl, CURLOPT_URL, "http://stream.twitter.com/1/statuses/filter.json?track=http");
     curl_easy_setopt(curl, CURLOPT_USERPWD, "JEggers2:password");

     //APPEND THIS
     curl_easy_setopt(curl, CURLOPT_HEADER, 1L);  //Enable Headers
     curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, writeDataOnStream);
     curl_easy_setopt(curl, CURLOPT_WRITEDATA, stdout);   //Print data in STDOUT
     //END

     res = curl_easy_perform(curl);
     curl_easy_cleanup(curl);      // always cleanup
  }   
  return 0;
}

CURLOPT_WRITEFUNCTION

CURLOPT_WRITEDATA

Upvotes: 0

Ruslan
Ruslan

Reputation: 19120

The other answer appears to be wrong in its usage of first and last parameters of callback_func (see the docs). Actual chunk of data you received is in the first parameter, ptr, while the pointer you pass with CURLOPT_WRITEDATA is the last parameter.

I've made a complete compilable example:

#include <stdlib.h>
#include <string.h>
#include <curl/curl.h>

size_t dataSize=0;
size_t curlWriteFunction(void* ptr, size_t size/*always==1*/,
                         size_t nmemb, void* userdata)
{
    char** stringToWrite=(char**)userdata;
    const char* input=(const char*)ptr;
    if(nmemb==0) return 0;
    if(!*stringToWrite)
        *stringToWrite=malloc(nmemb+1);
    else
        *stringToWrite=realloc(*stringToWrite, dataSize+nmemb+1);
    memcpy(*stringToWrite+dataSize, input, nmemb);
    dataSize+=nmemb;
    (*stringToWrite)[dataSize]='\0';
    return nmemb;
}

int main()
{
    char* data=0;
    CURL*const curl=curl_easy_init();
    if(!curl)
    {
        fprintf(stderr, "Failed to init curl");
        return 1;
    }
    curl_easy_setopt(curl, CURLOPT_URL, "https://www.google.com");
    curl_easy_setopt(curl, CURLOPT_WRITEDATA, &data);
    curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, &curlWriteFunction);
    if(curl_easy_perform(curl)!=CURLE_OK)
    {
        fprintf(stderr, "Failed to get web page\n");
        return 1;
    }
    curl_easy_cleanup(curl);

    if(!data)
    {
        fprintf(stderr, "Got no data\n");
        return 1;
    }

    printf("Page data:\n\n%s\n", data);
    free(data);
}

Upvotes: 4

cpx
cpx

Reputation: 17567

To get the data into string, you need to set up a write callback function:

curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, callback_func);

Also, the address of your string variable to receive the data:

curl_easy_setopt(curl, CURLOPT_WRITEDATA, &str)

Callback function would look like this:

size_t callback_func(void *ptr, size_t size, size_t count, void *stream)
{
      /* ptr - your string variable.
      stream - data chuck you received */

     printf("%.*s", size, (char*)stream);
}

Because you won't know the total size of data you'd be receiving so you would need to make pointer re-allocations to get it into a string.

Upvotes: 7

Related Questions