CODEWITHSUNDEEP
pythonpython-2.7image-processingkeras
Revati Joshi
Revati Joshi

Reputation: 183

keras zca_whitening - no error, no output generated

While using zca_whitening, my code gets stuck somewhere, neither it shows any error nor the output. When i skip the zca_whitening and apply other transformations, the code runs perfectly. I am attaching the code snippet here. Pl help me if I am doing anything wrong here :

datagen = ImageDataGenerator(zca_whitening=True)
datagen.fit(x_train)

where >> x_train is the set of training images (dim = 50 x 64 x 64 x 3) . After running datagen.fit, the code shows no further output or error, seems to have the process running for an infinite time period.

on the other hand these transformations work perfectly :

datagen = ImageDataGenerator(rotation_range=0.90,
    width_shift_range=0.2,
    height_shift_range=0.2,
    fill_mode='nearest',
    horizontal_flip=True,
    vertical_flip=True)
datagen.fit(x_train)

Is there anything I am missing here?

Upvotes: 1

Views: 734

Answers (2)

NeStack
NeStack

Reputation: 2012

I disagree with @Avjiit that the problem is due to wrong shape. The problem is common and as confirmed by the keras main contributors the problem is that the zca-calculation takes very, very long, because it uses numpy.linalg.svd() which is computationally heavy even on matrices (n*m*3), n~m~100.

There are some approaches to calculate a quick approximation of the svd, like randomized svd by scikitlearn, truncated svd by irlb, Lanczos method svd, but they are not always numerically stable.

I think I have found another very simple method, that is very quick and gives exactly the same results as the standard method! It helps in case you have a data matrix with shape (m x n), where m is much smaller than n - e.g. you have much less images (m~1000) than number of pixels (n~100 x 100 x 3 = 30000 pixel). In such a case keras would calculate the linalg.svd(sigma), with sigma.shape=(30000,30000), which is computationally too exhausting and takes forever. But in a good approximation, instead of calculating the svd on a (n x n) matrix you could calculate it on a (m x m) matrix, by just rotating the input data X or flipping the order of the sigma calculation to look like sigma = np.dot(flat_x, flat_x.T) / flat_x.shape[0]. With this approach the calculation takes only ca. 10 sec, if m~1000 and n~30000. The good thing is that the eigenvectors of the linalg.svd(sigma) are the same in both cases up to a factor, see here, slide 30. You could test this on a dataset of yours or on the inbuilt cifer set with from keras.datasets import cifar10.

The mathematical justification can be found here and here. For visuality I share these images:

enter image description here enter image description here

And below is the modified keras code for creating the zca calculations. You could use it to make modifications into your keras zca code, located here. The modifications I made are to put if-clauses for transposing the data matrix, when needed:

from keras.datasets import cifar10
import numpy as np
from scipy import linalg

(X_train, y_train), (X_test, y_test) = cifar10.load_data()
X = X_train[:1000]
flat_x = np.reshape(x, (x.shape[0], x.shape[1] * x.shape[2] * x.shape[3]))

# normalize x:
flat_x = flat_x / 255.
flat_x = flat_x - flat_x.mean(axis=0)

# CHANGES HAPPEN BELOW.
# if m>n execute the svd as usual
if flat_x.shape[0] => flat_x.shape[1]:
    sigma = np.dot(flat_x.T, flat_x) / flat_x.shape[0]
    u, s, _ = linalg.svd(sigma)
# and if m<n do the trnaspose trick
if flat_x.shape[0] < flat_x.shape[1]:
    sigma = np.dot(flat_x, flat_x.T) / flat_x.shape[0]
    u, s, _ = linalg.svd(sigma)
    u = np.dot(flat_x.T, u) / np.sqrt(s*flat_x.shape[0])

s_inv = 1. / np.sqrt(s[np.newaxis] + 0.1) # the 0.1 is the epsilon value for zca
principal_components = (u * s_inv).dot(u.T)
whitex = np.dot(flat_x, principal_components)

Upvotes: 0

Avijit Dasgupta
Avijit Dasgupta

Reputation: 2065

Modify x_train to have shape (3, 64, 64).

You can do that by using the following code: x_train = x_train.transpose((2,1,0))

This is mainly due to the switching between theano and tensorflow backend. Check dim_order in Keras documentation.

Upvotes: 2

Related Questions