Challenge 5: Semantic search

Introduction

In this challenge we’ll use reverse ETL to transfer the product descriptions and embeddings over to Spanner. Once the information is available in Spanner we’ll implement semantic search through Spanner’s embeddings capabilities.

Description

Before we start copying the data from BigQuery to Spanner, let’s complete the product descriptions for the whole table. We’ve prepared a Worfklow called prep-semantic-search, look it up, execute it with the parameter embeddings_model_name value set to the name of the embeddings model that was created in the previous challenge. Once this workflow succesfully completes, all products should have descriptions and embeddings.

Now, add the following two columns product_description of type STRING(MAX) and product_description_embeddings of type ARRAY<FLOAT64> to the products table in Spanner. Export the data from BigQuery to Spanner through reverse ETL, and create the same embeddings model in Spanner.

Using the embeddings model in Spanner do a search for the query Luxury items for women and return the top 5 items.

Success Criteria

There are two new columns in the Spanner products table: product_descriptions and product_description_embeddings.
The column product_description contains the LLM generated product descriptions for all products.
The column product_description_embbedings contains the embeddings for the product descriptions for all products.

The semantic search for Luxury items for women on Spanner returns the following products:

id	name	department
13981	Super Soft Black 3m Insulated Leather Women’s Gloves	Women
13896	Black Fleur Di Lis Rhinestone Western Bling Belt	Women
13869	WARMEN Luxury Gift Women Genuine Nappa Soft Leather Lined Long Elbow Gloves	Women
13967	Women’s Italian Leather Gloves Lined in Cashmere in Many Colors. Simple By Solo Classe	Women
14085	Luxury Lane Women’s Cashmere Lined Lambskin Leather Long Gloves in Black Chocolate or Tobacco	Women

Learning Resources

Previous Challenge Next Challenge