Topic
Gpt 2
2 episodes
-
Serving Multiple Users at Once: How Continuous Batching Keeps LLM Inference Efficient MachineLearningMastery
Continuous batching is a scheduling technique that keeps LLM inference servers from wasting GPU cycles on padding. Instead of forcing short requests to wait for long ones in a fixed batch, continuous batching frees up slots the moment a request finishes and admits new work immediately, eliminating idle padding tokens and improving throughput.
-
6 Things I Learned Building LLMs From Scratch That No Tutorial Teaches You | Towards Data Science
Justy and Cody dig into what actually changes when you stop calling an LLM API and start building pieces yourself: why fine-tuning tricks like RsLoRA matter, why RoPE won, where weight tying still makes sense, why Pre-LN became the default, and how KV cache buys speed by spending memory.