CODEWITHSUNDEEP
hdf5julia
Jim Garrison
Jim Garrison

Reputation: 4276

How can I read a complex HDF5 array in Julia?

I have many HDF5 datasets containing complex number arrays, which I have created using Python and h5py. For example:

import numpy, h5py
with h5py.File("test.h5", "w") as f:
    f["mat"] = numpy.array([1.0 + .5j, 2.0 - 1.0j], dtype=complex)

HDF5 has no native concept of complex numbers, so h5py stores them as a compound data type, with fields "r" and "i" for the real and imaginary parts.

How can I load such arrays of complex numbers in Julia, using HDF5.jl?

EDIT: The obvious attempt

using HDF5
h5open("test.h5", "r") do fd
    println(read(fd, "mat"))
end

returns a cryptic response:

HDF5Compound(Uint8[0,0,0,0,0,0,240,63,0,0,0,0,0,0,224,63,0,0,0,0,0,0,0,64,0,0,0,0,0,0,240,191],Type[Float64,Float64],ASCIIString["r","i"],Uint64[0,8])

Upvotes: 4

Views: 2021

Answers (3)

Jeremy
Jeremy

Reputation: 631

In Julia 0.6 you can do the following. As long as you have the HDF5 module and DataFrames already installed this example is immediately executable because the example HDF5 file comes with HDF5.jl. In all likelihood it only works on common types. I haven't tested it beyond the example file as I'm still trying to figure out how to write/create compound tables from Julia.

using HDF5
using DataFrames

# Compound Table Read
d = h5read(joinpath(Pkg.dir("HDF5"),"test","test_files","compound.h5"),"/data")

# Convert d to a dataframe, D
types = [typeof(i) for i in d[1].data] # Data type list
names_HDF5 = [Symbol(i) for i in d[1].membername] # Column name list
D = DataFrame(types,names_HDF5,length(d)) # Preallocate the array
rows = length(d) # Number of rows
cols = length(d[1].data) # Number of columns
for i=1:rows
    for j=1:cols
        D[i,j] = d[i].data[j] # Save each element to the preallocated dataframe
    end
end

d is a vector of table rows. Each element is of type HD5FCompound which each have three fields: data, membername, and membertype.

Upvotes: 0

rickhg12hs
rickhg12hs

Reputation: 11932

As @simonster pointed out, there is a fast and safe way to do this.

If you had written:

a = read(fd, "mat"))

then the complex vector that you want is simply:

cx_vec = reinterpret(Complex{Float64}, a.data)

Upvotes: 3

Jim Garrison
Jim Garrison

Reputation: 4276

I hadn't thought of this before, but one solution is simply to use h5py with PyCall:

using PyCall
@pyimport h5py

f = h5py.File("test.h5", "r")
mat = get(get(f, "mat"), pybuiltin("Ellipsis"))
f[:close]()

println(mat)

Upvotes: 2

Related Questions