Neural network and XOR as a classification

Question

I read somewhere that mean squared error loss is good for regression, and cross entropy loss for classification.

When I tried to train XOR as a classification problem with cross entropy loss, network failed to converge.

My setting:

• Network is 2-2-2

• First output is signaling 0 and second 1 (so two classes of inputs).

• Cross entropy is used for calculating error in output layer of network instead of mean squared error.

• as a activation function, Im using logsig.

Apparently, Im missing something, where is my mistake ?

Answer 1

Here's an implementation of this network in Mathematica:

net = NetChain[{2, Tanh, 2, Tanh, 1, LogisticSigmoid}, "Input" -> {2}];

eps = 0.01;
data = {{0, 0} -> {eps}, {1, 0} -> {1 - eps}, {0, 1} -> {1 - eps}, {1,
      1} -> {eps}};

trained = 
 NetTrain[net, data, CrossEntropyLossLayer["Binary"], 
  MaxTrainingRounds -> Quantity[5, "Minutes"], TargetDevice -> "GPU"]

Which converges after a few thousand rounds. So, I don't think you're missing anything - there's probably a bug in your library