CODEWITHSUNDEEP
pythondatetimetime-seriesweather
Jonathon
Jonathon

Reputation: 271

Creating time series plot from large csv file

I have a large csv file with 3 months of temperature data. Column 1 is the month, column 2 is the day, column 3 is the hour, column 4 is minute, and column 5 is temperature. I am trying to graph all 3 months of temperature data with only the day/month listed on the x-axis. Here is what I have so far:

filename ='TemperatureFile.csv'
with open(filename) as f:
    reader = csv.reader(f)
    header_row = next(reader)

    month, day, hour, temperature = [], [], [], []
    for row in reader:
        mon=datetime.strptime(row[0],"%m")
        month.append(mon)
        date=datetime.strptime(row[1],"%d")
        day.append(date)
        hourx=datetime.strptime(row[2],"%H:%M")
        hour.append(hourx)
        temp = float(row[3])
        temperature.append(temp)

        datefinal=date.isoformat(month, day)


    #date.isoformat(months, days).isoformat()==    

    fig = plt.figure(dpi=128, figsize=(10,6))
    plt.plot(datefinal,temperature, c='red')

    plt.title("Temperatures", fontsize=20)
    plt.xlabel('Date', fontsize=16)
    plt.ylabel('Temperatures(F)', fontsize=16)

    plt.show()

I cannot figure out how to combine all of the month/day/hour/minute info so all the data plots and I cannot figure out how to place just month/day on the x-axis.

Upvotes: 0

Views: 1628

Answers (2)

R.Sharp
R.Sharp

Reputation: 316

I had to play around quite a bit just to get the code you presented to work.

Then I could create a single date from the 3 columns you have.

Finally I focus on the formatting of the plot.

It was 2 questions really.

filename ='TemperatureFile.csv'

with open(filename) as f:
    reader = csv.reader(f)
    header_row = next(reader)

    dates_list, temperature = [], []
    for row in reader:
        #Convert your 3 colums of strings into a sinlge datestring.
        datestring = ("{0:}/{1:} {2:}".format(*row))
        # Then convert that into a complete date object
        date_obj = datetime.datetime.strptime(datestring, "%m/%d %H:%M")
        # Get the temperature as before
        temp = float(row[3])
        # Add new date and temp to appropriate lists
        temperature.append(temp)
        dates_list.append(date_obj)

# Now have complete lists of date and temp.
# Next focus on the formatting of the plot.
myFmt = mdates.DateFormatter('%m/%d') # set the format to print the dates in.
months = mdates.MonthLocator()  # every month
days = mdates.DayLocator()   # every Day

fig = plt.figure(dpi=128, figsize=(10,6))
ax = fig.add_axes([0.1, 0.2, 0.85, 0.75])
plt.plot(dates_list,temperature, c='red')

# format the ticks
ax.xaxis.set_major_formatter(myFmt)    
ax.xaxis.set_major_locator(months)
ax.xaxis.set_minor_locator(days)

plt.title("Temperatures", fontsize=20)
plt.xlabel('Date', fontsize=16)
plt.ylabel('Temperatures(F)', fontsize=16)

plt.show()

For more information on how to format your plot, google or search SO for what you want to do and the word matplotlib. That's where I got my example from.

Upvotes: 1

Triggernometry
Triggernometry

Reputation: 583

Well, if you actually need to create a datetime from parts, you can do this:

dtms = []
temps = []

for row in reader:
    # split row into its 4 parts
    month, day, hour, temperature = row

    # create base datetime object with the current date/time
    dt = datetime.now()

    # replace current month
    dt = dt.replace(month=int(month))

    # replace current day
    dt = dt.replace(day=int(day))

    ## or, you could replace both at the same time
    #dt = dt.replace(month=int(month), day=int(day))

    # get date object from datetime
    dt = dt.date()

    # get time
    tm = datetime.strptime(hour, "%H:%M")

    # get time object from datetime
    tm = tm.time()

    # combine date and time
    dtm = datetime(dt, tm)

    # add datetime to list
    dtms.append(dtm)

    # add temperature to list
    temps.append(float(temperature))

However, it doesn't seem necessary in your case. It's actually much easier to just create one datetime object all at once, rather than creating the base datetime object and adding new parts:

dtms = []
temps = []

for row in reader:
    # split row into its 4 parts
    month, day, hour, temperature = row

    # concatenate columns into string
    # assuming the current year is the correct year, you may need to add additional logic to get the correct year
    # also assumes dateparts are properly zero-padded
    dtm = "{}-{}-{} {}".format(datetime.now().year, month, day, hour)

    # convert created string into datetime object
    dtm = datetime.strptime(dtm, "%Y-%m-%d %H:%M")

    # add datetime to list
    dtms.append(dtm)

    # add temperature to list
    temps.append(float(temperature))

Upvotes: 1

Related Questions