Error "DimensionMismatch("A has dimensions (83,5) but B has dimensions (83,5)")" when using Flux package in Julia

Question

I have encountered a problem where it says my dimension mismatch and I am not sure how to go about solving it. I tried making them the same but yet to no avail, it gives me the same error...Here are the Matrices I am using

X>83×5 Matrix{Float32}:
  0.685468    2.71934    -1.3916    -1.64212    -2.46184
  1.9476     -0.368776   -0.706665   0.552662    0.0840651
 -0.896637   -0.774699    0.320741  -0.167762    0.471714
  1.07655    -0.0556672   0.320741   1.69273     0.074243
  0.836567   -0.590463    0.663209  -0.36163     0.469094
 -1.13662     0.620122   -0.706665   1.21484    -1.04679
  1.1921     -0.382114    0.320741  -0.0105933   0.854123
  ⋮                                             
  0.0544035   0.864285   -0.364196  -0.442209   -1.37747
 -0.372231    2.37295    -1.3916    -2.17745    -1.94192
 -1.09218    -0.786059    1.69061    0.437777    0.630833
  0.0810681  -0.548226    1.00568    0.828704    0.418674
 -0.869972   -0.774699    0.320741  -0.167762    0.471714
 -1.01218    -0.786059    1.69061    0.437777    0.630833



Y>83 elments
Vector{Float64}
0.000282416
0.309485
0.676214
1.21552
0.374202
-0.416781
0.172861
0.633069
-1.37315
-1.13586
0.251959
1.45282
0.100953
-1.01361
-0.258585
0.906318
2.31571
-0.0356714
0.316676
0.704977
more
-0.0140991
-0.344874
0.331057
-0.474308
0.0146639
1.74045
-0.0860067
-1.028
-1.34439

The error comes when I run this specific line of code:

Flux.train!(loss(x,y),params(ms),data ,opt)

error:

DimensionMismatch("A has dimensions (83,5) but B has dimensions (83,5)")
gemm_wrapper!(::Matrix{Float64}, ::Char, ::Char, ::Matrix{Float64}, ::Matrix{Float64}, ::LinearAlgebra.MulAddMul{true, true, Bool, Bool})@matmul.jl:643
mul!@matmul.jl:169[inlined]
mul!@matmul.jl:275[inlined]
*@matmul.jl:160[inlined]
(::Flux.Dense{typeof(NNlib.relu), Matrix{Float32}, Vector{Float32}})(::Matrix{Float64})@basic.jl:158
applychain@basic.jl:47[inlined]
(::Flux.Chain{Tuple{Flux.Dense{typeof(NNlib.relu), Matrix{Float32}, Vector{Float32}}, Flux.Dense{typeof(NNlib.relu), Matrix{Float32}, Vector{Float32}}, Flux.Dense{typeof(NNlib.relu), Matrix{Float32}, Vector{Float32}}}})(::Matrix{Float64})@basic.jl:49
loss(::Matrix{Float64}, ::Vector{Float64})@Other: 2
top-level scope@Local: 2[inlined]

The Model Architecture I am trying to use is a multi-layer perceptron and I have 5 input features and 1 output and I believe this is where my problem also is + I am using the Flux package>

ms = Chain(
  Dense(83,5, relu),
  Dense(5,83, relu),
  Dense(5,83,relu),
)  

This is how I defined my loss function:

loss(x,y)= Flux.mse(ms(x),y)

Could anyone please give me some form of guidance or a solution to fix this.

Answer 1

In this cases, your best option is to split your neural network to understand what is happening:

# I will create just X and y
X = rand(Float32, (83, 5))
y = rand(Float64, 83)

Denses take 3 parameters: input size, output size and activation function. The first one says how many variables per observation we have, and the output size how many variables per observations should be output-ed.

If the input (X matrix) is 83x5, we have 5 observations with 83 variables, so when computing a Dense(83, 5, relu), we are saying: for each observation, take 5 output variables. Check this out:

julia> Dense(83, 5, relu)(X)
Tracked 5×5 Matrix{Float32}:
 1.21155   0.416421   1.22945   0.300055  0.0
 0.469003  0.0637476  0.26137   0.377286  0.323375
 0.0       0.0        0.0       0.0       0.0
 0.0       0.0        0.091296  0.0       0.341985
 0.0       0.0242414  0.0       0.0       0.0

Note that the output size is 5x5, i.e., for each of the 5 observations, we have now 5 new variables. The next layer:

julia> Dense(5, 83, relu)(Dense(83, 5, relu)(X))
Tracked 83×5 Matrix{Float32}:
...

We asked for a 83 variables for each observations, so for each of the 5 observations, it is what we got. However, your last layer is declared as Dense(5, 83, relu), which is expecting a number X of observations but with 5 variables each, however, as discussed, we have 83.

To solve the error you just need to do something like:

Chain(
  Dense(input_size,output_size1, relu),
  Dense(output_size1, output_size2, relu),
  Dense(output_size2, output_size3, relu),
)