CODEWITHSUNDEEP
pythonsql-serverpandaspyodbc
Silas
Silas

Reputation: 144

SQL Server DATE retrieved into pandas as a string

When I pull a "Date" variable from SQL Server into Python/Pandas, it comes through as an 'Object'. I have installed and tried several drivers (commented drivers tried shown in the code), each time with the same results:

import pandas as pd
import pyodbc

conn_str = (
    r'Driver={SQL Server Native Client 11.0};'
    # r'Driver={SQL Server Native Client 10.0};'
    # r'Driver={ODBC Driver 11 for SQL Server};'
    # r'Driver={ODBC Driver 13 for SQL Server};'
    # r'Driver={SQL Server};'
    r'Server=MyServer;'
    r'Database=MyDB;'
    r'Trusted_Connection=yes;'
    )

cnxn = pyodbc.connect(conn_str)

sql = (
    "Select cast('2017-08-19' as date) [DateVar]"
    ", cast('2017-08-19' as datetime) [DateTimeVar]"
    ", cast('2017-08-19' as datetime2) [DateTime2Var]"
    )

d2 = pd.read_sql(sql,cnxn)

cnxn.close()

print(d2.dtypes)

Returned result is:

DateVar                 object
DateTimeVar     datetime64[ns]
DateTime2Var    datetime64[ns]
dtype: object

I want that DateVar to be a datetime. Any ideas why this is happening??

Same issue as this guy: pyodbc returns SQL Server DATE fields as strings But the fix for him was to use {SQL Server Native Client 10.0} which I've installed and isn't working for me.

Version of SQL Server I'm connecting to is:

Microsoft SQL Server 2012 (SP3) (KB3072779) - 11.0.6020.0 (X64) 
Oct 20 2015 15:36:27 
Copyright (c) Microsoft Corporation
Enterprise Edition (64-bit) on Windows NT 6.1 <X64> (Build 7601: Service Pack 1)

UPDATES

1>

Based on Max's input, have tried sqlalchemy, but no luck, this still gives me a string back:

import sqlalchemy as sa
engine = sa.create_engine('mssql+pyodbc://MyDatabase/MyDB?driver=SQL+Server+Native+Client+10.0')
d2 = pd.read_sql(sql, engine)

2>

Based on Flipper's Q, have done this with just a Pyodbc cursor and it looks like the proper date data type is being returned in the cursor when using the Native Client 11.0:

(('DateVar', datetime.date, None, 10, 10, 0, True),
 ('DateTimeVar', datetime.datetime, None, 23, 23, 3, True),
 ('DateTime2Var', datetime.datetime, None, 27, 27, 7, True))

This would suggest the issue is in Pandas handling of the dtype datetime.date when loading into a dataframe.

Upvotes: 4

Views: 5544

Answers (2)

Bryan
Bryan

Reputation: 17703

Use the parse_dates parameter of pandas.read_sql to specify that DateVar column values are explicitly converted to datetime on dataframe load.

Updated original code snippet:

...
d2 = pd.read_sql(sql=sql,
                 con=cnxn,
                 # explicitly convert DATE type to datetime object
                 parse_dates=["DateVar"])

cnxn.close()

print(d2.dtypes)

Returns

DateVar         datetime64[ns]
DateTimeVar     datetime64[ns]
DateTime2Var    datetime64[ns]
dtype: object

Tested with pyodbc 4.0.17, pandas 0.20.3, and SQL Server 2014 on Windows.

Upvotes: 3

MaxU - stand with Ukraine
MaxU - stand with Ukraine

Reputation: 210922

Try to use SQLAlchemy as follows:

from sqlalchemy import create_engine

engine = create_engine("mssql+pyodbc://scott:tiger@myhost:port/databasename?driver=SQL+Server+Native+Client+10.0")

Upvotes: 0

Related Questions