CODEWITHSUNDEEP
machine-learningtensorflowneural-networkdeep-learning
Wei Liu
Wei Liu

Reputation: 1014

Neural network: constraint certain input nodes contribution to certain output nodes

I want to use neural network to learn the mapping of a input vector and a output vector. The physics of the problem has constraints such that certain input nodes only have influence on certain output nodes. I want to use that constraint in the training.

If I formulate the NN as a directed graph, I imagine the paths from certain input nodes to output nodes are 'blocked', and the error message should not back propagate through such paths. For example, in the figure below, I show a NN with 2 input and 2 output nodes. Input node 1 should not have any influence to output 4, so any path from node 1 to 4 (as shown in dashed lines) should not have back-prop.

I could not simply set some edge/weight to zero to satisfy the constraints, because the constraints are on paths, not on a single edge/weight.

I appreciate anyone share thoughts and experience on this issue. Maybe this is a well-thought problem but I haven't found anything after hard searching.

A simple neural network with 2 inputs and 2 outputs

Upvotes: 3

Views: 555

Answers (1)

Maxim
Maxim

Reputation: 53758

Interesting case. I'm afraid neural networks don't work like this. Layers are considered independent: forward and back passes flow through all available connections and each layer doesn't know how the current tensor was accumulated.

Your choice in terms of architecture is to block individual connections, something like DropConnect, but without randomness, if that's possible.

You can also consider separate networks for each output, e.g. one network, in which (1, 2) predicts 3, and another network, in which 2 predicts 4. This way, you force your constraints, but lose sharing weights between different networks, which is not ideal.

Another option: I can imagine that you can augment the dataset so that the networks actually learns that certain inputs are not affecting certain outputs. Depending on your actual problem, this may be time consuming, but at least in theory it may work: for a given input/output pair (1, 2) -> (3, 4) you can add several additional pairs (1*, 2) -> (3*, 4) to show that changing 1 affects the first output 3*, but not the second 4.

Upvotes: 2

Related Questions