Error Python pandas: time data '20160101-000000' does not match format '%YYYY%mm%dd-%HH%MM%SS'

Question

I've searched SO and questions I've found do not seem to cover my case. I've followed code from https://gist.github.com/gjreda/7433f5f70299610d9b6b using directives from https://docs.python.org/3/library/datetime.html#strftime-and-strptime-behavior and wrote in Jupyter Python notebook:

import numpy as np
import pandas as pd
from datetime import datetime
to_datetime = lambda d: datetime.strptime(d, '%YYYY%mm%dd-%HH%MM%SS')
pd1 = pd.read_csv(r'c:\am\shared-2\topandas.txt',sep=':',header=None, names  = ['a','b','c','d'], parse_dates=[1], converters={'a': to_datetime}, index_col = 'a')
#pd2 = pd1.to_datetime (pd1.)
pd1

I've got error

ValueError: time data '20160101-000000' does not match format '%YYYY%mm%dd-%HH%MM%SS'

I'm looking into expression and do not see where I made mistake.

Full error below:

ValueError Traceback (most recent call last) in 1 from datetime import datetime 2 to_datetime = lambda d: datetime.strptime(d, '%YYYY%mm%dd-%HH%MM%SS') ----> 3 pd1 = pd.read_csv(r'c:\am\shared-2\topandas.txt',sep=':',header=None, names = ['a','b','c','d'], parse_dates=[1], converters={'a': to_datetime}) 4 #pd2 = pd1.to_datetime (pd1.) 5 pd1

C:\ProgramData\Anaconda3\lib\site-packages\pandas\io\parsers.py in parser_f(filepath_or_buffer, sep, delimiter, header, names, index_col, usecols, squeeze, prefix, mangle_dupe_cols, dtype, engine, converters, true_values, false_values, skipinitialspace, skiprows, skipfooter, nrows, na_values, keep_default_na, na_filter, verbose, skip_blank_lines, parse_dates, infer_datetime_format, keep_date_col, date_parser, dayfirst, iterator, chunksize, compression, thousands, decimal, lineterminator, quotechar, quoting, doublequote, escapechar, comment, encoding, dialect, tupleize_cols, error_bad_lines, warn_bad_lines, delim_whitespace, low_memory, memory_map, float_precision) 700 skip_blank_lines=skip_blank_lines) 701 --> 702 return _read(filepath_or_buffer, kwds) 703 704 parser_f.name = name

C:\ProgramData\Anaconda3\lib\site-packages\pandas\io\parsers.py in _read(filepath_or_buffer, kwds) 433 434 try: --> 435 data = parser.read(nrows) 436 finally: 437 parser.close()

C:\ProgramData\Anaconda3\lib\site-packages\pandas\io\parsers.py in read(self, nrows) 1137 def read(self, nrows=None): 1138
nrows = _validate_integer('nrows', nrows) -> 1139 ret = self._engine.read(nrows) 1140 1141 # May alter columns / col_dict

C:\ProgramData\Anaconda3\lib\site-packages\pandas\io\parsers.py in read(self, nrows) 1993 def read(self, nrows=None): 1994
try: -> 1995 data = self._reader.read(nrows) 1996 except StopIteration: 1997 if self._first_chunk:

pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader.read()

pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader._read_low_memory()

pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader._read_rows()

pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader._convert_column_data()

pandas/_libs/parsers.pyx in pandas._libs.parsers._apply_converter()

in (d) 1 from datetime import datetime ----> 2 to_datetime = lambda d: datetime.strptime(d, '%YYYY%mm%dd-%HH%MM%SS') 3 pd1 = pd.read_csv(r'c:\am\shared-2\topandas.txt',sep=':',header=None, names = ['a','b','c','d'], parse_dates=[1], converters={'a': to_datetime}) 4 #pd2 = pd1.to_datetime (pd1.) 5 pd1

C:\ProgramData\Anaconda3\lib_strptime.py in _strptime_datetime(cls, data_string, format) 575 """Return a class cls instance based on the input string and the 576 format string.""" --> 577 tt, fraction, gmtoff_fraction = _strptime(data_string, format) 578 tzname, gmtoff = tt[-2:] 579 args = tt[:6] + (fraction,)

C:\ProgramData\Anaconda3\lib_strptime.py in _strptime(data_string, format) 357 if not found: 358 raise ValueError("time data %r does not match format %r" % --> 359 (data_string, format)) 360 if len(data_string) != found.end(): 361 raise ValueError("unconverted data remains: %s" %

ValueError: time data '20160101-000000' does not match format '%YYYY%mm%dd-%HH%MM%SS'

Answer 1

Here it is necessary to change format to use only one letter, also check http://strftime.org/:

to_datetime = lambda d: datetime.strptime(d, '%Y%m%d-%H%M%S')

Another similar solution:

to_datetime = lambda d: pd.to_datetime(d, format='%Y%m%d-%H%M%S')

Answer 2

Your mistake is that %Y matches a 4-digit year - so by giving %YYYY you're actually matching "4-digit year, followed by three Ys". This holds for the others too.

Try the following format string instead: %Y%m%d-%H%M%S