makemore-language-model

a language model that create names. inspired by Andrej Karpathy

The model get enhanced as per the following implimentations

1. Biagram - basic probability based model
   • implemented in the [Language_Models]_makemore.ipynb
2. MLP: A Neural Probabilistic Language Model
   • implemented in the [Language_Models]_makemore_MLP.ipynb
   • "Kaiming init" paper: https://arxiv.org/abs/1502.01852
   • BatchNorm paper: https://arxiv.org/abs/1502.03167
   • Good paper illustrating some of the problems with batchnorm in practice: https://arxiv.org/abs/2105.07576
3. CN: Paper by deepmind
   • In progress
4. RNN: Recurrent neural network based language model
   • In progress
5. LSTM: Generating Sequences With Recurrent Neural Networks
   • In progress
6. GRU: On the Properties of Neural Machine Translation: Encoder-Decoder Approaches
   • In progress
7. Transformer Attention Is All You Need
   • In progress