Reputation: 23
I did a small proof-of-concept to know whether or not Dense layer in Keras supports Masking. Below is my code:-
import keras
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense,Masking,Flatten
import numpy as np
input_shape = (125,)
model = Sequential()
model.add(Masking(mask_value=-1,input_shape=input_shape))
model.add(Dense(16, activation='relu', kernel_initializer='he_uniform'))
model.add(Dense(8, activation='relu', kernel_initializer='he_uniform'))
model.add(Flatten())
model.add(Dense(1, activation='linear'))
model.compile(loss='mean_absolute_error', optimizer='adam', metrics=['mean_absolute_error'])
history = model.fit(x_train, y_train, epochs=5, batch_size=1, verbose=1)
My dataset contains only 2 samples, each having 125 features, which can be seen below:-
[[ 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14.
15. 16. 17. 18. 19. 20. 21. 22. 23. 24. 25. 26. 27. -1.
-1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1.
-1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1.
-1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1.
-1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1.
-1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1.
-1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1.
-1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1.]
[ 82. 83. 84. 85. 86. 87. 88. 89. 90. 91. 92. 93. 94. 95.
96. 97. 98. 99. 100. 101. 102. 103. 104. 105. 106. 107. 108. 109.
110. 111. 112. 113. 114. 115. 116. 117. 118. 119. 120. 121. 122. 123.
124. 125. 126. 127. 128. 129. 130. 131. 132. 133. 134. 135. 136. 137.
138. 139. 140. 141. 142. 143. 144. 145. 146. 147. 148. 149. 150. 151.
152. 153. 154. 155. 156. 157. 158. 159. 160. 161. 162. -1. -1. -1.
-1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1.
-1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1.
-1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1.]]
With Masking layer, I want Dense layer to ignore all the '-1' values while training the network. Hence I wrote model.add(Masking(mask_value=-1,input_shape=input_shape))
.
I also headed over to Keras Github code (https://github.com/keras-team/keras/blob/master/keras/layers/core.py) for Dense layer and found that Dense layer class has self.supports_masking = True
but I am not sure that it propagates the mask to other layers in the network.
So, my questions are as follows:-
dense_18 doesn't support masking
error, does this mean that it does support Masking?Any help would be highly appreciated. Thanks...
Upvotes: 2
Views: 2836
Reputation: 56
Not sure if you ever received an answer, but in case you haven't, you might look at this: Masking Guide
I was just going to look to see if Dense supports masking also. Here is the relevant quote from the linked guide:
If you have a custom layer that does not modify the time dimension, and if you want it to be able to propagate the current input mask, you should set
self.supports_masking = True
in the layer constructor. In this case, the default behavior ofcompute_mask()
is to just pass the current mask through.
This to me, says that Dense will propagate the mask.
Upvotes: 1