CODEWITHSUNDEEP
pythonpandasprecisionexport-to-csv
Zhang18
Zhang18

Reputation: 4930

pandas to_csv float format arbitrary precision without engineering formating

I have a pandas dataframe with arbitrary large and small values (whose magnitude is unknown a priori), say:

>>> import pandas as pd
>>> df = pd.DataFrame({'a' : [0.0000001, 2, 3], 'b' : [4, 5000000, 0.6]})

The default display will convert certain numbers to engineering format

>>> df
              a          b
0  1.000000e-07        4.0
1  2.000000e+00  5000000.0
2  3.000000e+00        0.6

which I don't care. My goal, however, is to write this to a csv file without the engineering format, while not printing unnecessary 0s either. The 2nd part of the requirement is to control file size since there are millions of rows.

For example, if I run df.to_csv(csv_file) then the file looks like

,a,b
0,1e-07,4.0
1,2.0,5000000.0
2,3.0,0.6

If I run df.to_csv(csv_file, float_format="%.7f") then it looks like (notice all the unnecessary 0s):

,a,b
0,0.0000001,4.0000000
1,2.0000000,5000000.0000000
2,3.0000000,0.6000000

What I'd like to have in the output file is:

,a,b
0,0.0000001,4.0
1,2.0,5000000.0
2,3.0,0.6

Is there a simple way to achieve that?

Upvotes: 0

Views: 1133

Answers (1)

Max Power
Max Power

Reputation: 8954

import pandas as pd   
df = pd.DataFrame({'a' : [0.0000001, 2, 3], 'b' : [4, 5000000, 0.6]})   


def export_formatted(df, csv_path, cols=None):

    # By default, format all columns in df
    if cols==None:
        cols = df.columns

    # Change columns to strings with 0's stripped as desired
    for c in cols:
        df[c] = df[c].map('{:,.15f}'.format).str.rstrip('0')

    # export
    df.to_csv(csv_path)

export_formatted(df, 'stack_overflow_scratch2.csv')

provided what I think you want (below):

    a               b
0   0.0000001       4
1   2               5000000
2   3               0.6

Upvotes: 1

Related Questions