You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
As an initial test to verify the transformer works i'm training it to sort a sequence of 6 numbers with 3 digits in order of size. It usually gets accuracy ~ 99% on this after less than 10 epochs. I turns out that removing the MLP layer has almost no effect on the result so we should devise a harder test to ensure that we're testing both the MLP and the Attention layer.
The text was updated successfully, but these errors were encountered:
As an initial test to verify the transformer works i'm training it to sort a sequence of 6 numbers with 3 digits in order of size. It usually gets accuracy ~ 99% on this after less than 10 epochs. I turns out that removing the MLP layer has almost no effect on the result so we should devise a harder test to ensure that we're testing both the MLP and the Attention layer.
The text was updated successfully, but these errors were encountered: