✨ Awesome List:

• PositionalEncoding:
  • https://e2eml.school/transformers.html
  • https://github.com/tensorflow/tensor2tensor/issues/1591
  • https://kazemnejad.com/blog/transformer_architecture_positional_encoding/
• BatchNorm:
  • https://colab.research.google.com/drive/1Sw2GXJmylz9DvtPaBfJIeutoXNISelZy?usp=sharing
  • https://e2eml.school/batch_normalization.html
  • How BatchNorm Helps?: https://arxiv.org/pdf/1805.11604.pdf
  • Understanding BatchNorm: https://proceedings.neurips.cc/paper/2018/file/36072923bfc3cf47745d704feb489480-Paper.pdf
  • Batch-ReNorm: https://arxiv.org/pdf/1702.03275.pdf
• StreamingNorm: https://arxiv.org/pdf/1610.06160.pdf
• Online Norm: https://proceedings.neurips.cc/paper/2019/file/cb3ce9b06932da6faaa7fc70d5b5d2f4-Paper.pdf
• How optimization works:
  • https://e2eml.school/how_optimization_works_1.html
  • https://e2eml.school/how_optimization_works_2.html
• Optimizing a linear model: https://e2eml.school/how_optimization_works_3.html
• Optimizing complex models:vhttps://e2eml.school/how_optimization_works_4.html
• EINSUM:
  • https://obilaniu6266h16.wordpress.com/2016/02/04/einstein-summation-in-numpy/
  • https://rockt.github.io/2018/04/30/einsum
  • https://ajcr.net/Basic-guide-to-einsum/
  • einops: https://einops.rocks/
  • einops EXAMPLES: http://einops.rocks/pytorch-examples.html
  • EINSUM / einops / PyTorch: https://theaisummer.com/einsum-attention/
• Linear Regression: https://machinelearningcompass.com/machine_learning_models/linear_regression/
• Types of Convolutions: https://towardsdatascience.com/types-of-convolutions-in-deep-learning-717013397f4d
• 3D Graph Scene: https://3dscenegraph.stanford.edu

External code refernce

    elif n == 1:
        return [0]

    fib = [0, 1]
    for i in range(2, n):
        fib.append(fib[i-1] + fib[i-2])