Crash Course in AI: Formula E Edition
Introduction
Given the close proximity racing, varying track conditions, and high speeds of electric cars, incidents in Formula E are virtually inevitable. The series’ aggressive, unpredictable nature combined with street circuits often leads to crashes and collisions.
In this gHack we’ll analyze Formula E CCTV footage and telemetry data to identify crashes and determine the drivers involved using Gemini.
We’ll start with loading the video files into BigQuery. Following this, video embeddings will be generated, enabling RAG-based semantic search to pinpoint the precise timestamp of any potential crash. Finally, telemetry data will be ingested, filtered, and aggregated around the crash timestamp to identify the drivers involved in the crash.
Learning Objectives
During the process we’ll learn about
- BigQuery Object tables
- Multimodal embeddings & Vector search in BigQuery
- Retrieval Augmented Generation (RAG)
- BigQuery for analyzing tabular data
Challenges
- Challenge 1: Getting in gear
- Challenge 2: Formula E-mbed
- Challenge 3: Formula E RAG-ing
- Challenge 4: Telemetry to the rescue!
Prerequisites
- Basic knowledge of GCP
- Basic knowledge of Python
- Basic knowledge of SQL
- Access to a GCP environment
Contributors
- Murat Eken
- Michelle Liu
- Deb Lee
- Gino Filicetti