Exploring Next
Exploring Next — Ep 221 w/ Justy & Cody — LLM2Vec-Gen: Generative Embeddings from Large Language Models
Episode 221 explores LLM2Vec-Gen, a breakthrough approach that creates embeddings by learning to represent what a language model would generate, rather than encoding the input. Instead of traditional contrastive learning, this method adds special tokens that capture the model's potential response, achieving state-of-the-art results while maintaining safety alignment and reasoning capabilities.