CODEWITHSUNDEEP
arduinowebserveriotesp32
NickRyan
NickRyan

Reputation: 29

ESP32 - Best Way to Store Data Offline and Push when Network Connected

I am coding an offline, battery-powered esp32 to take periodic sensor readings and store them until a hotspot is found, in which it connects and pushes the data elsewhere. I am relatively new to esp32 and ask for suggestions on the best way to do this.

I was thinking of storing the reading and DateTime in SPIFFS memory and running a webserver that starts when a network is found, checking every minute or so. Since it is battery-powered, I would also like to deep sleep the board to save power. Does the setup() function run again when the board comes out of deep sleep or would I need to have my connectToWiFi function inside the loop?

Is this viable? And are there any better routes to take? I've seen things on asynchronous servers and using the esp32 as an access point that could maybe work. Is it best to download the file through a web server or send the file line by line through a free online database?

Upvotes: 2

Views: 3466

Answers (3)

legoblocks
legoblocks

Reputation: 595

As it sounds like you're just storing timestamps / data (not files), I would recommend using a cheap and low-power EEPROM chip.

Upvotes: 0

RemyHx
RemyHx

Reputation: 365

If an SD card is not an option (beware you don’t pull out the SD while writing!) I would write to SPIFFS or a direct write with esp_partition_write().

For the latter: if you use predefined structs with you sensor data (plus time etc) and you start the partition with a small table with the startvalue which has to be updated next time until the other (a mini fat) it’s easy to retrieve data (no fuzz with reading lines). Keep in mind that every time you wipe the flash the wear counts for that whole block! So if you accept old data to be ignored but present, this could dramatically reduce wear.

For example say you write:

Struct:

Uint8_t day;
Uint8_t month;
Uint8_t year; //year minus 2000, max 256
Uint8_t hour;
Uint8_t minutes;
Uint8_t seconds;
Uint8_t sensorMSB;
Uint8_t sensorLSB;

That’s 8 bytes.

The first struct (call it the mini fat):

Uint8_t firstToProcessMSB;
Uint8_t firstToProcessLSB;
Uint8_t amountToProcessMSB;
Uint8_t amountToProcessLSB;
Uint8_t ID0;
Uint8_t ID1;
Uint8_t ID2;
Uint8_t ID3;

Also eight bytes. For the ID you can use some values to recognize a well written partition. But that’s up to you. It’s only a suggestion!

In an 65.536 byte sized partition you can add 8192 (-1) bytes before you have to erase. Max 100.000 times…

When your device makes contact you read out the first bytes. Check if it’s ok, read the start byte. With fseek you step 8 bytes every hop to the end position and read all values in one time until you reached the end. If succesfull, you change the startposition to the end + 1 and only erase when things go tricky. (Not enough space)

It’s wise to do before the longest suspected amount of time to run out. Otherwise you will also lose data. Or you just make the partition bigger.

But if in this case you could write every minute for 5 days.

Upvotes: 0

romkey
romkey

Reputation: 7069

Deep sleep on the ESP32 is almost the equivalent of being power cycled - the CPU restarts, and any dynamic memory will have lost its contents. An Arduino program will enter setup() after deep sleep and will have to completely reinitialize everything the program needs to run.

There is a very small area (8Kbytes) of static memory associated with the real time clock (RTC) which is retained during deep sleep. You can directly reference variables stored there using a special decorator (RTC_DATA_ATTR) when you declare the variable.

For instance, you could use a variable stored in this area to count the number of times the CPU has slept and woken up.

RTC_DATA_ATTR uint64_t sleep_counter = 0;

void setup() {
  sleep_counter++;

  Serial.begin(115200);
  Serial.print("ESP32 has woken up ");
  Serial.print(sleep_counter);
  Serial.println(" times");
}

Beware that it's generally not safe to store objects in this area - you don't necessarily know whether they've allocated memory that won't persist during deep sleep. So storing a String in this memory won't work. Also storing a struct with pointers generally won't work as the pointers won't point to storage in this area.

Also beware that if the ESP32 loses power, RTC_DATA_ATTR will be wiped out.

The RTC static RAM also has the advantage of not costing as much power to write to as SPIFFS.

If you need more storage than this, SPIFFS is certainly an option. Beware that ESP32's generally use cheap NAND flash memory which is rated for a maximum of maybe 100,000 writes.

SPIFFS performs wear-leveling, which will help avoid writing to the same location in flash over and over again, but eventually it will still wear out. This isn't a problem for most projects but suppose you're writing to SPIFFS once a minute for two years - that's over a million writes. So if you're looking for persistent storage that's frequently written to over a very long time you might want to use a better quality of flash storage like an external SD card.

Upvotes: 2

Related Questions