DOCS: Update README for transformer and torch lightning (callback reading)

Author: MKaczkow

Diff for: theory/transformer/README.md

@@ -1,17 +1,21 @@
-### Useful links:
+# Transformer
+
+## Useful links:
 * https://towardsdatascience.com/transformers-141e32e69591
 * https://medium.com/inside-machine-learning/what-is-a-transformer-d07dd1fbec04  
 * https://arxiv.org/abs/1706.03762 
 * https://www.youtube.com/watch?v=rURRYI66E54&ab_channel=Computerphile 
 * https://www.youtube.com/watch?v=U0s0f995w14&ab_channel=AladdinPersson 
 
-### Multi-Head Attention
+## Multi-Head Attention
 * values (V)
 * keys (K) 
 * queries (Q)
 
-### Formulas
-attention(Q, K, V) = softmax((Q*K**T) / (sqrt(d_k)) * V
+## Formulas
+```math
+attention(Q, K, V) = softmax(Q*K**T) / (sqrt(d_k)) * V
+```
 
-### More complex ideas / words / stuff
+## More complex ideas / words / stuff
 * permutationally invariant

Diff for: tools/pytorch_lightning/README.md

@@ -0,0 +1,17 @@
+# PyTorch Lightning
+
+## General
+* separating research code, engineering code and add-ons
+
+## Architecture
+* `Trainer` -> engineering code
+* `LightningModule` -> research code
+* `Callback` -> add-ons
+
+## Callbacks
+* `callback` is really a general term from CS, which means pluging-in some code to be executed at arbitrary point in the code, which means arbitrary time and state of code execution
+* in `lightning` some functionalities, like `early stopping`, `model checkpointing`, `logging`, `learning rate scheduling`, etc. are implemented as `callbacks`
+
+## References
+* [part of yt tutorial on callbacks](https://www.youtube.com/watch?v=Wcze6oGch1g)
+* [docs, obviously](https://pytorch-lightning.readthedocs.io/en/1.0.8/callbacks.html)