Stay tuned.

References:

---

1. Transformers explained: https://e2eml.school/transformers.html ↩

2. Positional Embeddings: https://kazemnejad.com/blog/transformer_architecture_positional_encoding/ ↩

3. Positional Embeddings: https://timodenk.com/blog/linear-relationships-in-the-transformers-positional-encoding/ ↩

4. POS Summation v/s Concat: https://github.com/tensorflow/tensor2tensor/issues/1591 ↩