Monitor GenAI apps with Firebase Genkit
Introduction
Want to master the art of keeping Firebase Genkit LLM applications alive and thriving in the real world? This hackathon puts you in the driver’s seat of production monitoring for Large Language Model (LLM) powered applications. Assuming the role of a Site Reliability Engineer (SRE) on a fictional Movie Guru app team, you will use Firebase Genkit Monitoring to tackle the critical challenges of ensuring LLMs perform flawlessly in production. You’ll dive deep into troubleshooting live issues, optimizing performance bottlenecks, and guaranteeing a smooth user experience for a movie recommendation app.
Why is this crucial? Because in the age of AI, those who can effectively monitor and manage LLM applications in production are the ones who will build the future. The GenAI monitoring, debugging, and optimization skills you’ll gain are universally applicable for a modern SRE and are transferable to a wide range of systems beyond Genkit Monitoring.
“GenAI App Development with Genkit” is a recommended pre-requisite.
Learning Objectives
- Understand your GenAI app health in production.
- Troubleshoot different types of GenAI issues that might arise like model limitations, latency, etc
- Find and address issues associated with the quality of your generated content
Challenges
- Challenge 1: Set up your environment
- Challenge 2: Explore Firebase Genkit Monitoring
- Challenge 3: Troubleshoot failures
- Challenge 4: Improve performance
- Challenge 5: Improve search quality
Contributors
- Manasa Kandula
- Esther Lloyd
- Cleo Schneider
- Polina Govorkova