CODEWITHSUNDEEP
pythonpython-2.7numpyarcpy
Learner
Learner

Reputation: 5292

Numpy savetxt error

Below is the code I am struggling with as a beginner:

commonfields = arcpy.ListFields(gt_pnts)
names = [f.name for f in commonfields]
needed_names = names[3:]

gt_pnts_arr = arcpy.da.FeatureClassToNumPyArray(gt_pnts, needed_names)
gt_pnts_arr = gt_pnts_arr.reshape((gt_pnts_arr.shape[0],1))
eq_smpled_pnts_arr = arcpy.da.FeatureClassToNumPyArray(eq_smpled_pnts, needed_names)
eq_smpled_pnts_arr = eq_smpled_pnts_arr.reshape((eq_smpled_pnts_arr.shape[0],1))
master_table = np.concatenate((gt_pnts_arr, eq_smpled_pnts_arr), axis=0)
np.savetxt(outputcsvfilename,master_table, fmt="%.1f")

Error is as below:

TypeError: float argument required, not numpy.void

I searched and found this exception raised when datatype is not defined. But when I test as below I am in the dilemma-

>>>master_table.dtype.descr
>>>[('grid_code', '<i4'), ('b1_Clip_ProximityRaster1', '<f4'), ('b2_Clip_ProximityRaster1', '<f4'), ('b3_Clip_ProximityRaster1', '<f4'), ('b4_Clip_ProximityRaster1', '<f4'), ('b5_Clip_ProximityRaster1', '<f4'), ('b6_Clip_ProximityRaster1', '<f4'), ('b7_Clip_ProximityRaster1', '<f4'), ('b8_Clip_ProximityRaster1', '<f4'), ('b9_Clip_ProximityRaster1', '<f4'), ('b10_Clip_ProximityRaster1', '<f4'), ('b11_Clip_ProximityRaster1', '<f4'), ('b12_Clip_ProximityRaster1', '<f4'), ('b13_Clip_ProximityRaster1', '<f4'), ('resp', '<U2')]
>>>master_table
array([[ (13, 13.0, 3810.0, 3810.0, 1982.952392578125, 3873.71923828125, 34869.9140625, 5483.3564453125, 7272.138671875, 4409.591796875, 872.0665283203125, 36238.62109375, 4441.62109375, 6775.2861328125, u'1')],
       [ (1, 1.0, 3601.99951171875, 3603.12353515625, 1626.9295654296875, 3725.922607421875, 34595.9453125, 5810.5595703125, 7592.90478515625, 4476.0361328125, 576.2811889648438, 36462.984375, 4499.0, 7164.47509765625, u'1')],
       [ (13, 13.0, 3721.93505859375, 3723.02294921875, 1642.3458251953125, 3842.928466796875, 34713.43359375, 5702.3681640625, 7597.17041015625, 4562.07177734375, 657.9513549804688, 36343.12890625, 4586.9599609375, 7111.0126953125, u'1')],.....................
>>>master_table.shape
>>>(50, 1)
>>>gt_pnts_arr.shape
>>>(25, 1)

Even I can not load this master_table into pandas dataframe as below-

df = pd.DataFrame(data=master_table[1:,1:], index=master_table[1:,0],columns=master_table[0,1:])

My table datatypes: There are 13 colums and 50 rows in the master_table.First and last column datatype is respectively integer and integer but all other(11) data type is float.

Upvotes: 0

Views: 2612

Answers (1)

hpaulj
hpaulj

Reputation: 231385

Recreating your data (thaxs for enough information), I make a data array

In [33]: data.shape
Out[33]: (3, 1)
In [34]: len(data.dtype.fields)

Trying savetxt:

In [28]: np.savetxt('test.csv',data, fmt="%.1f")
-> 1158   fh.write(asbytes(format % tuple(row) + newline))
   1159      except TypeError:

TypeError: cannot convert to a float; scalar object is not a number

and a secondary error:

TypeError: Mismatch between array dtype .... and format specifier ('%.1f')

savetxt iterates on data

for row in data:
    f.write(fmt%tuple(row))

But with this shape row is a (1,) array.

But if I get rid of the unnecessary 2nd dimension (size 1)

In [37]: np.savetxt('test.csv',data[:,0], fmt="%.1f")
TypeError: a float is required

and

TypeError: Mismatch between array dtype .... and format specifier ('%.1f %.1f %.1f %.1f %.1f %.1f %.1f %.1f %.1f %.1f %.1f %.1f %.1f %.1f %.1f')

Note that is has taken your fmt, and replicated it to match the number of fields. Copy that last format specifier, and tweak it to work with the final string column

In [38]: fmt='%.1f %.1f %.1f %.1f %.1f %.1f %.1f %.1f %.1f %.1f %.1f %.1f %.1f %.1f %s'

Now it works:

In [39]: np.savetxt('test.csv',data[:,0], fmt=fmt)
In [40]: cat test.csv
13.0 13.0 3810.0 3810.0 1983.0 3873.7 34869.9 5483.4 7272.1 4409.6 872.1 36238.6 4441.6 6775.3 1
1.0 1.0 3602.0 3603.1 1626.9 3725.9 34595.9 5810.6 7592.9 4476.0 576.3 36463.0 4499.0 7164.5 1
13.0 13.0 3721.9 3723.0 1642.3 3842.9 34713.4 5702.4 7597.2 4562.1 658.0 36343.1 4587.0 7111.0 1

So there are 2 corrections - a structured array has to be 1d to work with savetxt - rows and fields. And the fmt has to work with the dtype. In this case you have to handle the string field.

If the last field had been numeric (e.g. int) as opposed to U2, the data[:,0] would have worked with the simple %.1f fmt. The initial int field saves just fine (though the fmt could also be tweaked for that).

Upvotes: 1

Related Questions