Lessons learned from scaling large language models in production
41 min

Lessons learned from scaling large language models in production

MLOps World - MLOps World & Generative AI World 2024
Matt Squire
ray-serve large-language-models llm rag mlops gpu performance-optimization inference scaling python fastapi kubernetes vm vector-database