Challenge 3: Create vector searchable queries

Introduction

If a user tells the Movie Guru chatbot that “I am in the mood for a nice drama film”, you cannot take this statement and directly query the vector database. Often, the user doesn’t make clear and consice statements, and sometimes the context of a longer conversation is required to understand what the user is looking for. Lets analyse this conversation for example:

User: Hi

Chatbot: Hi. How can I help you.

User: I feel like watching a movie. Have any recommendations?

Chatbot: Yes. I have many movies in my database. I have comedy films, action films, romance films and many others. Do you know what you re looking for?

User: Yes. the first type I think.

The Movie Guru app then needs to understand from this context that the user is looking for a comedy film, and use this as a search query. The goal of this challenge is to take a conversation history, and the latest user statement, and transform it into a vector searchable query.

This challenge has two parts:

Craft the Prompt: You’ll engineer a prompt to guide the AI in understanding user queries and extracting key information.
Integrate into a Flow: You’ll then embed this prompt within a Genkit flow. Flows provide a structured way to interact with the AI, ensuring reliable outputs and error handling. This involves querying the model, processing the response, and formatting it for use in the Movie Guru application. In the previous challenge, we included the code for the flow for you (you just needed to write th prompt). In this challenge, you’ll need to write it yourself. The challenge also includes setting the correct input and output types for the flow.

Note: Think of the relationship between flows and prompts like this; the prompt is the recipe, and the flow is the chef who executes it and serves the final dish.

We want the model to take a user’s statement, the conversation history and extract the following:

Transformed query: The query that will be sent to the vector database to search for relevant documents.
User Intent: The intent of the user’s latest statement. Did the user issue a greeting to the chatbot (GREET), end the conversation (END_CONVERSATION), make a request to the chatbot (REQUEST), respond to the chatbot’s question (RESPONSE), ackowledge a chatbot’s statement (ACKNOWLEDGE), or is it unclear (UNCLEAR)? The reason we do this is to prevent a call to the vector DB if the user is not searching for anything. The application only performs a search if the intent is REQUEST or RESPONSE.
Optional Justification: Overall explanation of the model’s response. This will help you understand why the model made its suggestions and help you debug and improve your prompt.

Note: We can improve the testability of our model by augmenting its response with strongly typed outputs (those with a limited range of possible values like the enum User Intent). This is because automatically validating free-form responses, like the Transformed query, is challenging due to the inherent variability and non-deterministic nature of the output. Even a short Transformed query can have many variations (e.g., “horror films,” “horror movies,” “horror,” or “films horror”). However, by including additional outputs (with a restricted set of possible values, such as booleans, enums, or integers), we provide more concrete data points for our tests, ultimately leading to more robust and reliable validation of the model’s performance.

You need to perform the following steps:

Create a prompt that outputs the information mentioned above. The model takes in a user’s query, the conversation history, and the user’s profile information (long lasting likes or disklikes).
Update the prompt in the codebase (look at instructions in GoLang or TS) to see how.
Use the Genkit UI (see steps below) to test the response of the model and make sure it returns what you expect.
After the prompt does what you expect, then update the flow to use the prompt and return an output of the type QueryTransformFlowOutput

You can do this with GoLang or TypeScript. Refer to the specific sections on how to continue.

Description

GoLang Query Transform Flow

Pre-requisites

Make sure the Genkit UI is up and running at http://localhost:4002

Challenge-steps

Go to chat_server_go/cmd/standaloneFlows/main.go. You should see code that looks like this in the method getPrompts().

    
 queryTransformPrompt :=
 `
 This is the user profile. This expresses their long-term likes and dislikes:
 {{userProfile}} 

 This is the history of the conversation with the user so far:
 {{history}} 
 
 This is the last message the user sent. Use this to understand the user's intent:
 {{userMessage}}
 Translate the user's message into a different language of your choice.
 `
    

Keep this file (main.go) open in the editor. You will be editing the prompt here, and testing it in the genkit UI.
From the Genkit UI, go to Prompts/dotprompt/queryTransformFlow. If you choose to work with the flow directly go to Flows/queryTransformFlow (you cannot tweak the model parameters here, only the inputs).

You should see an empty input to the prompt that looks like this:

 {
     "history": [
         {
             "role": "",
             "content": ""
         }
     ],
     "userProfile": {
         "likes": { "actors":[""], "directors":[""], "genres":[], "others":[""]},
         "dislikes": {"actors":[""], "directors":[""], "genres":[], "others":[""]}
     },
     "userMessage": ""
 }

You should also see a prompt in the prompt view (the same prompt in main.go) below. You need to edit this prompt in main.go but can test it out by changing the input, model and other params in the UI.
Test it out: Add a userMessage “I want to watch a movie”, and leave the rest empty and click on RUN.
The model should respond by translating this into a random language (this is what the prompt asks it to do).
You need to rewrite the prompt (in main.go) and test the model’s outputs for various inputs such that it does what it is required to do (refer to the goal of challenge 2). Edit the prompt in main.go and save the file. The updated prompt should show up in the UI. If it doesn’t, just refresh the Genkit UI page. You can also play around with the model parameters.

After you get your prompt working, it’s now time to get implement the flow. Navigate to chat_server_go/cmd/standaloneFlows/queryTransform.go. You should see something that looks like this (code snippet below). What you see is that we define the dotprompt and specify the input and output format for the dotprompt. The prompt is however never invoked. We create an empty queryTransformFlowOutput and this will always result in the default output. You need to invoke the prompt and have the model generate an output for this.

 func GetQueryTransformFlow(ctx context.Context, model ai.Model, prompt string) (*genkit.Flow[*QueryTransformFlowInput, *QueryTransformFlowOutput, struct{}], error) {
    
 // Defining the dotPrompt
  queryTransformPrompt, err := dotprompt.Define("queryTransformFlow",
   prompt, // the prompt you created earlier is passed along as a variable
    
   dotprompt.Config{
    Model:        model,
    InputSchema:  jsonschema.Reflect(QueryTransformFlowInput{}),
    OutputSchema: jsonschema.Reflect(QueryTransformFlowOutput{}),
    OutputFormat: ai.OutputFormatJSON,
    GenerationConfig: &ai.GenerationCommonConfig{
     Temperature: 0.5,
    },
   },
 )
 if err != nil {
     return nil, err
 }

 // Defining the flow
 queryTransformFlow := genkit.DefineFlow("queryTransformFlow", func(ctx context.Context, input *QueryTransformFlowInput) (*QueryTransformFlowOutput, error) {
  
 // Create default output
 queryTransformFlowOutput := &QueryTransformFlowOutput{
 ModelOutputMetadata: &types.ModelOutputMetadata{
     SafetyIssue:   false,
     Justification: "",
 },
 TransformedQuery: "",
 Intent:           types.USERINTENT(types.UNCLEAR),
 }
    
 // Missing flow invocation code
    
 // We're directly returning the default output
 return queryTransformFlowOutput, nil
 })
 ~~~~
 return queryTransformFlow, nil
 }

If you try to invoke the flow in Genkit UI (Flows/queryTransformFlow) You should get an output something that looks like this as the flow is returning a default empty output.
```
 {
     "result": {
     "transformedQuery":"",
     "userIntent":"UNCLEAR",
     "justification":"",
     }
 }
```

But, once you implement the necessary code (and prompt), you should see something like this when you ask for movie recommendations.

 {
     "result": {
     "transformedQuery":"movie",
     "userIntent":"REQUEST",
     "justification":"The user's request is simple and lacks specifics.  Since the user profile provides no likes or dislikes, the transformed query will reflect the user's general request for a movie to watch.  No additional information is added because there is no context to refine the search.",
     }
 }

TypeScript Query Transform Flow

Pre-requisites

Make sure the Genkit UI is up and running at http://localhost:4003

Challenge-steps

Go to js/flows-js/src/prompts.ts. You should see code that looks like this in the method getPrompts().

    
 export const QueryTransformPromptText = `
 Here are the inputs:
 * userProfile: (May be empty)
     * likes: 
         * actors: {{#each userProfile.likes.actors}}{{this}}, {{~/each}}
         * directors: {{#each userProfile.likes.directors}}{{this}}, {{~/each}}
         * genres: {{#each userProfile.likes.genres}}{{this}}, {{~/each}}
         * others: {{#each userProfile.likes.others}}{{this}}, {{~/each}}
     * dislikes: 
         * actors: {{#each userProfile.dislikes.actors}}{{this}}, {{~/each}}
         * directors: {{#each userProfile.dislikes.directors}}{{this}}, {{~/each}}
         * genres: {{#each userProfile.dislikes.genres}}{{this}}, {{~/each}}
         * others: {{#each userProfile.dislikes.others}}{{this}}, {{~/each}}
 * userMessage: {{userMessage}}
 * history: (May be empty)
     {{#each history}}{{this.role}}: {{this.content}}{{~/each}}
 `
    

Note: When using Genkit dotprompts with Typescript, any elements of type zod.array() need to be rolled out in the prompt, else the object is not passed along to the model. While in GoLang, you can send the entire object as a single entity.

Keep this file open in the editor. You will be editing the prompt here, and testing it in the genkit UI.
From the Genkit UI, go to Prompts/queryTransformFlow.

You should see an empty input to the prompt that looks like this:

 {
 "history": [
     {
         "role": "",
         "content": ""
     }
 ],
 "userProfile": {
     "likes": {
         "actors": [""], "directors": [""], "genres": [""], "others":  [""]
     },
     "dislikes": {
     "actors": [""], "directors": [""], "genres": [""], "others":  [""]
     }
 },
 "userMessage": ""
 }

You should also see a prompt (the same prompt in prompt.go) below. You need to edit this prompt in the file but can test it out by changing the input, model and other params in the UI.
Test it out: Add a userMessage “I want to watch a movie”, and leave the rest empty and click on RUN.
The model should respond by saying something like this (don’t expect the exact same output). This is clearly nonsensical as a “I want to watch a movie” is not a sensible vector db query. The model is just retrofitting the output to match the output schema we’ve suggested (see queryTransformFlow.ts, we define an output schema) and trying to infer some semi-sensible outputs.
```
 {
     "transformedQuery": "I want to watch a movie",
     "userIntent": "REQUEST",
     "justification": "The user is requesting to watch a movie."
 }
```
You need to rewrite the prompt and test the model’s outputs such that it does what it is required to do (refer to the goal of challenge 2). Edit the prompt in prompts.ts and save the file. The updated prompt should show up in the UI. If it doesn’t, just refresh the UI. You can also play around with the model parameters.

After you get your prompt working, it’s now time to get implement the flow. Navigate to js/flows-js/src/queryTransformFlow.ts. You should see something that looks like this. What you see is that we define the dotprompt and specify the input and output format for the dotprompt. The prompt is however never invoked in a flow. We create an empty queryTransformFlowOutput and this will always result in the default output. You need to invoke the flow and have the model generate an output for this.

 // defining the dotPrompt
 export const QueryTransformPrompt = defineDotprompt(
 {
     name: 'queryTransformFlow',
     model: gemini15Flash,
     input: {
         schema: QueryTransformFlowInputSchema,
     },
     output: {
         format: 'json',
         schema: QueryTransformFlowOutputSchema,
     },  
 }, 
 QueryTransformPromptText // the prompt you created earlier is passed along as a variable
 )
        
 // defining the flow
 export const QueryTransformFlow = defineFlow(
 {
     name: 'queryTransformFlow',
     inputSchema: z.string(), // what should this be?
     outputSchema: z.string(), // what should this be?
     },
     async (input) => {
     // Missing flow invocation
        
     // Just returning hello world
     return "Hello World"
 }
 );

If you try to invoke the flow in Genkit UI (Flows/queryTransformFlow), you’ll notice that the input format for the flow is different from the prompt. The flow just expects a string. You need to fix the code in the challenge to change the input type from string to the required input type, so that the prompt and flow take the same input type. The output will just say “Hello World”. You should get an output something that looks like this:
```
 "Hello World"
```

But, once you implement the necessary code (and prompt), you should see something like this (if the userMessage is “I want to watch a movie”).

 {
     "result": {
     "transformedQuery":"movie",
     "userIntent":"REQUEST",
     "justification":"The user's request is simple and lacks specifics. Since the user profile provides no likes or dislikes, the transformed query will reflect the user's general request for a movie to watch.  No additional information is added because there is no context to refine the search.",
     }
 }

Success Criteria

Note: What to do if you’ve made the necessary change in the code files and still see weird output in the UI? Changing the code in the code files should automatically refresh it in the UI. Sometimes, however, genkit fails to autoupdate the prompt/flow in the UI after you’ve made the change in code. Hitting refresh on the browser (afer you’ve made and saved the code change) and reloading the UI page should fix it.

The model should be able to extract the user’s intent from the message and a meaningful query.

The model doesn’t return a transformed query when the user is just greeting them. The input of:

 {
     "history": [
         {
             "role": "agent",
             "content": "How can I help you today"
         },
         {
             "role": "user",
             "content": "Hi"
         }
     ],
     "userProfile": {
         "likes": { "actors":[], "directors":[], "genres":[], "others":[]},
         "dislikes": {"actors":[], "directors":[], "genres":[], "others":[]}
           
     },
     "userMessage": "Hi"
 }