CODEWITHSUNDEEP
javaautoencoderdeeplearning4jdimensionality-reductiondl4j
code monkey
code monkey

Reputation: 1

Autoencoder for dimesionality reduction in DL4J

I'm trying to write an autoencoder for dimensionality reduction in DL4J, but all the autoencoder examples I can find for DL4J are for outlier detection.

https://deeplearning4j.konduit.ai/v/en-1.0.0-beta6/getting-started/tutorials/basic-autoencoder

Does anybody have an example for dimensionality reduction in DL4J or how I can load just the encoding part of the multi layer network for evaluation after the training on the full encoder/decoder stack has been completed?

Any help or pointers are greatly appreciated.

Thanks

Upvotes: -1

Views: 63

Answers (1)

Adam Gibson
Adam Gibson

Reputation: 3205

I would use the transfer learning api for that. You can freeze layers as well as remove and add new layers. That should help with freezing or adding/removing what you need.

In your case with the feed forward you would need to know the the index of the layer(s) to remove to only have the encoder after you're done training. Some examples can be found here: https://github.com/eclipse/deeplearning4j-examples/tree/master/dl4j-examples/src/main/java/org/deeplearning4j/examples/advanced/features/transferlearning

From the examples repo there here's a simpler example of setting up a FineTuneConfiguration:

ComputationGraph vgg16 = (ComputationGraph) zooModel.initPretrained();
        log.info(vgg16.summary());

        //Decide on a fine tune configuration to use.
        //In cases where there already exists a setting the fine tune setting will
        //  override the setting for all layers that are not "frozen".
        FineTuneConfiguration fineTuneConf = new FineTuneConfiguration.Builder()
            .updater(new Nesterovs(5e-5))
            .seed(seed)
            .build();

        //Construct a new model with the intended architecture and print summary
        ComputationGraph vgg16Transfer = new TransferLearning.GraphBuilder(vgg16)
            .fineTuneConfiguration(fineTuneConf)
            .setFeatureExtractor(featureExtractionLayer) //the specified layer and below are "frozen"
            .removeVertexKeepConnections("predictions") //replace the functionality of the final vertex
            .addLayer("predictions",
                new OutputLayer.Builder(LossFunctions.LossFunction.NEGATIVELOGLIKELIHOOD)
                    .nIn(4096).nOut(numClasses)
                    .weightInit(new NormalDistribution(0,0.2*(2.0/(4096+numClasses)))) //This weight init dist gave better results than Xavier
                    .activation(Activation.SOFTMAX).build(),
                "fc2")
            .build();

This is changing a computation graph and only changing the last layer. Note most of the examples are focused on simpler use cases like changing the last layer. Feel free to post more questions if you have them. Hopefully this helps as a starting point.

Upvotes: 0

Related Questions