CODEWITHSUNDEEP
pythondatabasehdf5pytables
jtorca
jtorca

Reputation: 1551

Append all rows from one table to another using Pytables

Let's take for example the following:

import tables
import numpy as np

# Two Example Tables
hfile = tables.open_file('myfile.h5', 'a')
data1 = np.ones((3, 2))
data1.dtype = [('a', float), ('b', float)]
data2 = np.zeros((3, 2))
data2.dtype = [('a', float), ('b', float)]

table1 = hfile.create_table(where='/', name='table1', obj=data1)
table2 = hfile.create_table(where='/', name='table2', obj=data2)

# Appending
table1.append(table2.read())
table2.remove()

hfile.flush()
hfile.close()

Is there a better way to do this on-disk?

One solution is to iterate through rows:

for r in table2.iterrows():
    table1.append([r[:]])

The latter seems to bulky and the former draws the whole second table into memory before appending. I'd rather do something like:

table2.append_where(dstTable=table1)

But this function only works with a condition so I would need one that always evaluates to true to get the whole table. Surely there must be a better way.

Upvotes: 1

Views: 1343

Answers (2)

153957
153957

Reputation: 430

I created a pull request for PyTables to make the condition optional, as @jtorca requested. Given the support voiced by one of the maintainers it is likely to be accepted and included in a future version of PyTables.

Upvotes: 0

Anthony Scopatz
Anthony Scopatz

Reputation: 3637

I think that append_where() with a trivial condition like 'True' probably is the best solution.

Upvotes: 1

Related Questions