Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity

Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity
Share:

Download MP3


Similar Tracks

Feedback Transformers: Addressing Some Limitations of Transformers with Feedback Memory (Explained) Yannic Kilcher
Sparse Expert Models (Switch Transformers, GLAM, and more... w/ the Authors) Yannic Kilcher
TRILLION Parameter Models Are Here Edan Meyer
Understanding Mixture of Experts Trelis Research
The Map of Quantum Computing - Quantum Computing Explained Domain of Science
TransGAN: Two Transformers Can Make One Strong GAN (Machine Learning Research Paper Explained) Yannic Kilcher
Reformer: The Efficient Transformer Yannic Kilcher
Linear Transformers Are Secretly Fast Weight Memory Systems (Machine Learning Paper Explained) Yannic Kilcher
Gradient descent, how neural networks learn | DL2 3Blue1Brown
Big Bird: Transformers for Longer Sequences (Paper Explained) Yannic Kilcher
Stanford CS25: V1 I Mixture of Experts (MoE) paradigm and the Switch Transformer Stanford Online
OpenAI CLIP: ConnectingText and Images (Paper Explained) Yannic Kilcher
Nyströmformer: A Nyström-Based Algorithm for Approximating Self-Attention (AI Paper Explained) Yannic Kilcher
Decoder-Only Transformers, ChatGPTs specific Transformer, Clearly Explained!!! StatQuest with Josh Starmer
Pretrained Transformers as Universal Computation Engines (Machine Learning Research Paper Explained) Yannic Kilcher
What is Mixture of Experts? IBM Technology
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Yannic Kilcher
Large Language Models (LLMs) - Everything You NEED To Know Matthew Berman
Let's build GPT: from scratch, in code, spelled out. Andrej Karpathy
Illustrated Guide to Transformers Neural Network: A step by step explanation The AI Hacker