CODEWITHSUNDEEP
pythontensorflowneural-networkartificial-intelligencereinforcement-learning
Alexandre Krul
Alexandre Krul

Reputation: 55

Why do q_net has this number of input layers?

I was following this tutorial (https://www.tensorflow.org/agents/tutorials/1_dqn_tutorial?hl=en) on how to implement a Deep Q Network algorithm with TF Agents to solve the Cart Pole using RL.

I create the q_net:

fc_layer_params = (100,)

q_net = q_network.QNetwork(
    train_env.observation_spec(),
    train_env.action_spec(),
    fc_layer_params=fc_layer_params)

And when i use q_net.summary() it shows that the network has 500 input layers:

    Model: "QNetwork"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
EncodingNetwork (EncodingNet multiple                  500       
_________________________________________________________________
dense_1 (Dense)              multiple                  202       
=================================================================
Total params: 702
Trainable params: 702
Non-trainable params: 0
_________________________________________________________________
time: 3.63 ms (started: 2021-01-16 13:44:09 +00:00)

I would like to know why the value of input layers are 500 if for the cart pole environment we have the observation_spec and action_spec as:

Observation Spec:
BoundedArraySpec(shape=(4,), dtype=dtype('float32'), name='observation', minimum=[-4.8000002e+00 -3.4028235e+38 -4.1887903e-01 -3.4028235e+38], maximum=[4.8000002e+00 3.4028235e+38 4.1887903e-01 3.4028235e+38])

Action Spec:
BoundedTensorSpec(shape=(), dtype=tf.int64, name='action', minimum=array(0), maximum=array(1))time: 5.24 ms (started: 2021-01-16 13:48:27 +00:00)

This problem has a maximum time step per episode of 200, shouldn't the input layer be 200?

Upvotes: 5

Views: 448

Answers (1)

Federico Malerba
Federico Malerba

Reputation: 815

500 is the number of parameters. If you have 4 input nodes and 100 first layer nodes in a Dense layer, then you have 4x100 weights and 100 biases totalling 500 parameters.

Further explanation is here where they give the formula (equivalent to my computations above)

output_size * (input_size + 1) == number_parameters

Upvotes: 3

Related Questions