CODEWITHSUNDEEP
pythonpython-2.7pandasjupyter-notebookexec
chowpay
chowpay

Reputation: 1687

pandas dataframe to csv export issue

Getting a strange error, In jupyter notes I can run:

df1.to_csv("{0}{1}.csv".format(report_path,db))

and my csv comes out fine.

When I try execute the same thing outside of jupyter by putting the above into a file an external file and running this:

    #imports requred to run querys
    import pandas as pd
    from pandas import DataFrame,Series
    import numpy as np
    from pyhive import presto
    import matplotlib.pyplot as plt
    import seaborn as sn    

 #run config file, which contains the query to generate the report
    def run_config(db):
        print args.an
        print ("config is = {0}".format(config))
        with open(config) as cfg:
            v = cfg.read()
        exec v

I get this error:

lib.write_csv_rows(self.data, ix, self.nlevels, self.cols, self.writer)
File "pandas/_libs/lib.pyx", line 1035, in pandas._libs.lib.write_csv_rows
UnicodeEncodeError: 'ascii' codec can't encode characters in position 8-11: ordinal not in range(128)

I'm fairly certain its something in my DF that is causing this error because other df don't have this issue. But I'm lost as how to fix or edit the code to capture this.

the fix

df1.to_csv("{0}{1}.csv".format(report_path,db), encoding='utf8-8')

Upvotes: 1

Views: 802

Answers (1)

Ami Tavory
Ami Tavory

Reputation: 76297

I would try changing the code to

df1.to_csv("{0}{1}.csv".format(report_path,db), encoding='utf8-8')
  1. It seems clear you're encountering characters which aren't ASCII
  2. My guess is that your Jupyter is running a different interpreter than your script
  3. For non-ASCII encoding, UTF8 is a good first bet; if it doesn't work, have a look at Unicode, Dammit.

Upvotes: 1

Related Questions