Is the output linear layer parameter of the MultiHeadAttention class incorrectly set in mha.py file? in_features should be heads*d_k?
Activity
LXXiaogege commentedon May 2, 2023
The get_positional_encoding method of position encoder generates an error when d_model is set to odd
vpj commentedon Jun 30, 2023
Our implementation assumes that heads * d_k = d_model. Need to change that