CODEWITHSUNDEEP
pythonfastapipydantic
pgcudahy
pgcudahy

Reputation: 1601

Saving fastAPI POST to csv

I have a weather station that sends json data and want to make a fastAPI server to receive it and save it to disk. Currently I have

from typing import Optional
from fastapi import FastAPI
from pydantic import BaseModel

app = FastAPI()

class WeatherItem(BaseModel):
    wind_direction_raw: int
    rain_amount_raw: int
    timestamp: list = []
    elapsed_time: int
    wind_speed_raw: int
    message_id: int

@app.post("/station")
async def create_item(item: WeatherItem):
    return item

Which works well when I start it up with uvicorn main:app --host 192.168.1.151 --port 9494 and send test data from curl with

curl -X POST -H "Content-Type: application/json" --data '{"elapsed_time": 6245, "timestamp": [2020, 7, 26, 12, 2, 21, 6, 208], "wind_direction_raw": 108, "wind_speed_raw": 5, "message_id": 666, "rain_amount_raw": "0"}' http://192.168.1.151:9494/station

Now I need to save this data to disk. I think the simplest would be appending it to a csv file. But I can't figure out how to export a pydantic model to csv. Is there a straightforward way to do this or are other serialization methods preferred? I would like to analyze this data in R so I need it on disk in a fairly interchangeable format.

Upvotes: 0

Views: 2134

Answers (2)

pgcudahy
pgcudahy

Reputation: 1601

@Isabi has a good answer that got me on the right track and I've accepted it. But just to be complete I went with

import csv

...

@app.post("/station")
async def create_item(item: WeatherItem):

    write_path = "/home/pi/weather_station/data/weather_data_" + date.today().strftime("%Y-%m-%d") + ".csv"

    with open(write_path, "a") as f:
        writer = csv.writer(f)
        writer.writerow(dict(item).values())
    return item

Upvotes: 0

lsabi
lsabi

Reputation: 4456

As far as I'm aware, you have the following options:

Option 1

Append the straight values to the CSV file by enumerating the whole list of entries like this

with open("myFile.csv", "a") as f:
    f.write(f"{model.entry},{model.another_entry}")

(this is the blocking version)

It is to note that you can also create a dictionary and iterate over the values, append them, though it's difficult to preserve the order between entries..

Option 2

Load the file into memory with pandas, append to the DataFrame and then save the data to file

import pandas as pd

...

data = pd.read_csv("my_file.csv")
data_to_add = DataFrame.from_dict(my_model.dict())
data = data.append(data_to_add)
data.to_csv("my_file.csv", index=False)

Upvotes: 1

Related Questions