Can Neural Networks Output Numbers > 1?

Question

Usually when training a Network to decide between two values from a given data 2 things I have noticed.

1.) Two output neurons per decision are used and by favoring the neuron with the higher value a +ve decision is made.

2.) A single neuron per decision against a certain threshold say 0.5 hence a +ve decision is made by checking if the neuron outputs a value >= 0.5

Say one wants the network to make predictions based on the input data such as predicting the centroid of a shape from a given image each time what can one do other than making the output neurons the possible number of outcomes(which is definitely the entire number of pixels x 2 due to X,Y value pairs possible).

Specifically How can one make the output neurons fire e.g. (432,50) as the centroid in this case say the possible centroid (X,Y) coordinates are from 0 to 450 [ in other words can artificial neurons be modeled to behave like the Grandmother Cell if not then why ] ?

Answer 1

You can scale your [0,1] output to the range you want.

If you want your output to be X ∈ [0,500] and Y ∈ [0,500], you can have 2 outputs that both outputs [0,1] and multiply each result by 500.

You should also scale your input data to some uniform range. Some functions require that the input is in a specific range (e.g. [0,1]) and, for example, having some input sets in [0,10] and some in [0,1000] may confuse the network or having different features in different ranges could make it harder for the network to weight the features appropriately.

If your output is in the same range as your input, determining by how much to scale the input will also tell you by how much you need scale the output.

Answer 2

You're question contains two components

1.) can a neural network learn concepts like GrandMa. The answer is yes, however, the approach taken is typically different from what you have described. The branch of ML is called representation learning. In simple terms the typical concept is as follows: A vector is used to describe all concepts. Each entry of the vector corresponds to a neuron of the neural network. However, concepts do not directly map onto individual neurons. One could enforce that, but it wouldn't make sense.

2.) Can neural networks predict numbers. Indeed like in the other answer you can use output normalization -- this is map the target values on the number range 0-1 and denormalise afterwards. However, this has the drawback, that it will take a very long time until your neural network converges to the extreme values. An alternative and much easier approach is to use a neural network for regression. The idea is basically to not use a squashing function like relu in the output layer, but for example a linear activation function. The following function for example predicts two columns in iris using Keras

iris = load_iris()
idf = pd.DataFrame(data= np.c_[iris['data'], iris['target']],
                     columns= iris['feature_names'] + ['target'])

X = idf[["sepal length (cm)","sepal width (cm)","target"]] #this is ugly including class not ordinal
Y = idf[["petal length (cm)","petal width (cm)"]]
model = Sequential()
model.add(Dense(8, input_dim=3,
                activation='relu'))
model.add(Dense(16, activation='softmax'))
model.add(Dense(2, activation='linear'))
model.compile(loss='mse', optimizer='adam', metrics=['mae'])
model.fit(X.values, Y.values, epochs=1000, verbose=1)
Yp = model.predict(X.values)
#print([Y.iloc[:,0],Y.iloc[:,1],Y.iloc[:,0]-Yp[:,0],Y.iloc[:,1]-Yp[:,1]])
plt.scatter(Y.iloc[:,0].values,
                        Y.iloc[:,1].values)
for (x,y,dx,dy) in  zip(Y.iloc[:,0].values,
                        Y.iloc[:,1].values,
                        (Y.iloc[:,0].values-Yp[:,0]),
                        (Y.iloc[:,1].values-Yp[:,1])):
    #print(str(x)+" "+str(y)+" "+str(dx)+" "+str(dy))
    plt.arrow(x,y,dx,dy)
plt.show()