How can I get time taken from http request ESP32

Question

I am currently trying to get the time an HTTP request(downloading 1 file) needs to get done using ESP32 but the API has no option for it. I can set a timeout but I dont have the true time it needed to get done.

I tried to get the difference time after and before the perform function esp_err_t err = esp_http_client_perform(client); as so :

 struct timeval start, end;
 esp_http_client_config_t config = {
    .url = DOWNLOAD_FILE_URL,
    .event_handler = _http_event_handler,
 };
 esp_http_client_handle_t client = esp_http_client_init(&config);
 gettimeofday(&start, NULL);
 esp_err_t err = esp_http_client_perform(client);
 gettimeofday(&end, NULL);
 int64_t time_us = (((int64_t)end.tv_sec * 1000000L + (int64_t)end.tv_usec)-((int64_t)start.tv_sec * 1000000L + (int64_t)start.tv_usec));

But the value returned seems to be way too much for what I am doing compared (downloading a small file & I have done it with a curl on my Linux machine it took only 20ms) :

Time : 1305909 uS

I assume the esp_http_client_perform(client) function is probably doing many other things than just downloading the file and that is why I do not get only the time needed to download the file.

Do you have any idea what I can do to get only the time needed to download the file?

Thank you very much for your help.

Answer 1

Hi Mate I suggest to use the Network Time Protocol. It’s a standard Internet Protocol (IP) for synchronizing the computer clocks to some reference over a network. The protocol can be used to synchronize all networked devices to Coordinated Universal Time (UTC). NTP sets the clocks of computers to UTC, any local time zone offset or day light saving time offset is applied by the client. In this manner clients can synchronize to servers regardless of location and time zone differences. The server is using the User Datagram Protocol (UDP) on port 123. A client then transmits a request packet to a NTP server. In response to this request the NTP server sends a time stamp packet.

Please try this code:

#include "NTPClient.h"
#include "ESP8266WiFi.h"
#include "WiFiUdp.h"

const char *ssid = "***********";
const char *password = "***********";

const long utcOffsetInSeconds = 0;

char daysOfTheWeek[7][12] = {"Sunday", "Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday"};

// Define NTP Client to get time
WiFiUDP ntpUDP;
NTPClient timeClient(ntpUDP, "pool.ntp.org", utcOffsetInSeconds);

void setup(){
Serial.begin(115200);

WiFi.begin(ssid, password);

while ( WiFi.status() != WL_CONNECTED ) {
delay ( 500 );
Serial.print ( "." );
}

timeClient.begin();
}

void loop() {
timeClient.update();

Serial.print(daysOfTheWeek[timeClient.getDay()]);
Serial.print(", ");
Serial.print(timeClient.getHours());
Serial.print(":");
Serial.print(timeClient.getMinutes());
Serial.print(":");
Serial.println(timeClient.getSeconds());
//Serial.println(timeClient.getFormattedTime());

delay(1000);
}

Note: the line:

const long utcOffsetInSeconds = 0;

sets the UTC time (basically the Greenwich time with no light saving time offset), if you wish, set your own offset in seconds from the Greenwich time. Great article here: https://www.instructables.com/Getting-Time-From-Internet-Using-ESP8266-NTP-Clock/

If you get two readings and you make the difference between the two, you have the duration of the operation and, (very important since we are talking about network operations), the time reference for a following analysis.

I hope this helps All the best

Answer 2

Firstly, calling gettimeofday() and calculating the difference is IMHO too much effort for such a simple task. It's easier to call esp_timer_get_time() to get microseconds since boot and calculate durations based on that.

Secondly, the custom event handler which you've given your HTTP client (_http_event_handler) gets called for each relevant event in the HTTP download, so use that to see how long each step takes. I've converted to milliseconds because it seems the correct unit scale to measure download events.

uint64_t start_time;
esp_http_client_config_t config = {
    .url = DOWNLOAD_FILE_URL,
    .event_handler = _http_event_handler,
    .user_data = &start_time,
};
esp_http_client_handle_t client = esp_http_client_init(&config);
start_time = esp_timer_get_time();
esp_err_t err = esp_http_client_perform(client);
ESP_LOGI(TAG, "Total time %u ms", (uint32_t)((esp_timer_get_time() - start_time) / 1000));
...

esp_err_t _http_event_handle(esp_http_client_event_t *evt) {
    uint32_t duration_ms = (uint32_t) ((esp_timer_get_time() - *(uint64_t*)(evt->user_data)) / 1000);
    switch(evt->event_id) {
        case HTTP_EVENT_ERROR:
            ESP_LOGI(TAG, "HTTP_EVENT_ERROR %u ms", duration_ms);
            break;
        case HTTP_EVENT_ON_CONNECTED:
            ESP_LOGI(TAG, "HTTP_EVENT_ON_CONNECTED %u ms", duration_ms);
            break;
        case HTTP_EVENT_HEADER_SENT:
            ESP_LOGI(TAG, "HTTP_EVENT_HEADER_SENT %u ms", duration_ms);
            break;
        case HTTP_EVENT_ON_HEADER:
            ESP_LOGI(TAG, "HTTP_EVENT_ON_HEADER %u ms", duration_ms);
            printf("%.*s", evt->data_len, (char*)evt->data);
            break;
        case HTTP_EVENT_ON_DATA:
            ESP_LOGI(TAG, "HTTP_EVENT_ON_DATA, len=%d %u ms", evt->data_len, duration_ms);
            break;
        case HTTP_EVENT_ON_FINISH:
            ESP_LOGI(TAG, "HTTP_EVENT_ON_FINISH %u ms", duration_ms);
            break;
        case HTTP_EVENT_DISCONNECTED:
            ESP_LOGI(TAG, "HTTP_EVENT_DISCONNECTED %u ms", duration_ms);
            break;
    }
    return ESP_OK;
}

Finally, note that if you're doing an HTTPS download then the poor little ESP32 will take a significant time performing the TLS handshake crypto calculations.