1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "lTx8eQlc3cP-"
},
"source": [
"##### Copyright 2025 Google LLC."
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"cellView": "form",
"id": "4HZoi8yf4GEU"
},
"outputs": [],
"source": [
"# @title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
"# you may not use this file except in compliance with the License.\n",
"# You may obtain a copy of the License at\n",
"#\n",
"# https://www.apache.org/licenses/LICENSE-2.0\n",
"#\n",
"# Unless required by applicable law or agreed to in writing, software\n",
"# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
"# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
"# See the License for the specific language governing permissions and\n",
"# limitations under the License."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "M9I7LG483nXB"
},
"source": [
"# Gemini API: Similarity Search using Qdrant"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "awKO767lQIWh"
},
"source": [
"<a target=\"_blank\" href=\"https://colab.research.google.com/github/google-gemini/cookbook/blob/main/examples/qdrant/Qdrant_similarity_search.ipynb\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" height=30/></a>"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "bbf4f2c17530"
},
"source": [
"<!-- Princing warning Badge -->\n",
"<table>\n",
" <tr>\n",
" <!-- Emoji -->\n",
" <td bgcolor=\"#f5949e\">\n",
" <font size=30>⚠️</font>\n",
" </td>\n",
" <!-- Text Content Cell -->\n",
" <td bgcolor=\"#f5949e\">\n",
" <h3><font color=black>This notebook requires paid tier rate limits to run properly.<br> \n",
"(cf. <a href=\"https://ai.google.dev/pricing#veo2\"><font color='#217bfe'>pricing</font></a> for more details).</font></h3>\n",
" </td>\n",
" </tr>\n",
"</table>"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "b1xoF_bU4NCP"
},
"source": [
"## Overview"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "CWedABji6bXJ"
},
"source": [
"The [Gemini API](https://ai.google.dev/models/gemini) provides access to a family of generative AI models for generating content and solving problems. These models are designed and trained to handle both text and images as input.\n",
"\n",
"[Qdrant](https://qdrant.tech/) is a vector similarity search engine that offers an easy-to-use API for managing, storing, and searching vectors, with an additional payload. It is a production-ready service.\n",
"\n",
"In this notebook, you'll learn how to perform a similarity search on data from a website with the help of Gemini API and Qdrant."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "dIAarGkG8VwC"
},
"source": [
"## Setup\n",
"\n",
"First, you must install the packages and set the necessary environment variables.\n",
"\n",
"### Installation"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "70wYOKUC8q1m"
},
"source": [
"Install google's python client SDK for the Gemini API, `google-genai`. Next, install Qdrant's Python client SDK, `qdrant-client`."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "mnQbBnA1GKha"
},
"outputs": [],
"source": [
"%pip install -q \"google-genai>=1.0.0\"\n",
"%pip install -q protobuf==4.25.1 qdrant-client[fastembed]"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "eomJzCa6lb90"
},
"source": [
"## Configure your API key\n",
"\n",
"To run the following cell, your API key must be stored it in a Colab Secret named `GOOGLE_API_KEY`. If you don't already have an API key, or you're not sure how to create a Colab Secret, see [Authentication](https://github.com/google-gemini/cookbook/blob/main/quickstarts/Authentication.ipynb) for an example."
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"id": "v-JZzORUpVR2"
},
"outputs": [],
"source": [
"from google.colab import userdata\n",
"from google import genai\n",
"\n",
"GEMINI_API_KEY=userdata.get('GEMINI_API_KEY')\n",
"client = genai.Client(api_key=GEMINI_API_KEY)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "-go1BAF-_GtV"
},
"source": [
"## Basic steps\n",
"\n",
"Semantic search is the process using which search engines interpret and match keywords to a user's intent in organic search results. It goes beyond surface-level keyword matching. It uses the meaning of words, phrases, and context using advanced algorithms resulting in more relevant and user-friendly search experiences.\n",
"\n",
"Semantic searches rely on vector embeddings which can best match the user query to the most similar result.\n",
"\n",
"In this tutorial, you'll implement the three main components of semantic search:\n",
"\n",
"1. Create an index\n",
"\n",
" Create and store the index for the data in the Qdrant vector store. You will use a Gemini API embedding model to create embedding vectors that can be stored in the Qdrant vector store.\n",
"\n",
"2. Query the index\n",
"\n",
" Query the index using a query string to return the top `n` neighbors of the query.\n",
"\n",
"You'll learn more about these stages in the upcoming sections while implementing the application."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "0egnCR92JKsj"
},
"source": [
"## Import the required libraries"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"id": "LfJN5QosJQqD"
},
"outputs": [],
"source": [
"from bs4 import BeautifulSoup\n",
"from qdrant_client import models, QdrantClient\n",
"from urllib.request import urlopen"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "lL7J7BtyJsNQ"
},
"source": [
"## 1. Create an index\n",
"\n",
"In this stage, you will perform the following steps:\n",
"\n",
"1. Read and parse the website data using Python's BeautifulSoup library.\n",
"\n",
"2. Create embeddings of the website data.\n",
"\n",
"3. Store the embeddings in Qdrant's vector database.\n",
" \n",
" Qdrant is a vector similarity search engine. Along with a convenient API to store, search, and manage points(i.e. vectors), it also provides an option to add an additional payload. The payloads are essentially extra bits of data that you can utilize to refine your search and obtain relevant information that you can then share with your users."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "kFlGmkKbRebP"
},
"source": [
"### Read and parse the website data\n",
"\n",
"To read the website data as text, you will use the `BeautifulSoup` library from Python."
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"id": "oMs-ux1gtxOa"
},
"outputs": [],
"source": [
"url = \"https://blog.google/outreach-initiatives/sustainability/\"\\\n",
" \"report-ai-sustainability-google-cop28/\"\n",
"html = urlopen(url).read()\n",
"soup = BeautifulSoup(html, features=\"html.parser\")\n",
"\n",
"# Remove all script and style elements\n",
"for script in soup([\"script\", \"style\"]):\n",
" script.extract() # Self-destruct\n",
"\n",
"# Get the text\n",
"text_content = soup.get_text()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "toC03rqUSfom"
},
"source": [
"If you only want to select a specific portion of the website data to add context to the prompt, you can use regex, text slicing, or text splitting.\n",
"\n",
"In this example, you'll use Python's `split()` function to extract the required portion of the text."
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"id": "cHJq059duxj7"
},
"outputs": [],
"source": [
"# The text content between the substrings \"Later this month at COP28\" to\n",
"# \"POSTED IN:\" is relevant for this tutorial. You can use Python's `split()`\n",
"# to select the required content.\n",
"text_content_1 = text_content.split(\"Later this month at COP28\",1)[1]\n",
"final_text = text_content_1.split(\"POSTED IN:\",1)[0]\n",
"\n",
"texts = final_text.split(\".\")\n",
"\n",
"documents = []\n",
"\n",
"# Convert text into a chunk of 3 sentences.\n",
"for i in range(0, len(texts), 3):\n",
" documents.append({\"content\": \" \".join(texts[i:i+3])})"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "-CVPdm0h6aTd"
},
"source": [
"### Initialize the embedding model\n",
"\n",
"To create the embeddings from the website data, you'll use the **gemini-embedding-001** model, which supports creating embeddings from text.\n",
"\n",
"To use the embedding model, you have to use the `embed_content` function from the `google-genai` package. To learn more about the embedding model, read the [model documentation](https://ai.google.dev/gemini-api/docs/embeddings).\n",
"\n",
"One of the arguments passed to the embedding function is `task_type`. Specifying the `task_type` parameter ensures the model produces appropriate embeddingsfor the expected task and inputs. It is a string that can take on one of the following values:\n",
"\n",
"| task_type\t | Description |\n",
"|---|---|\n",
"| `RETRIEVAL_QUERY` | Specifies the given text is a query in a search or retrieval setting. |\n",
"| `RETRIEVAL_DOCUMENT` | Specifies the given text is a document in a search or retrieval setting. | \n",
"| `SEMANTIC_SIMILARITY` | Specifies the given text will be used for Semantic Textual Similarity (STS). | \n",
"| `CLASSIFICATION` | Specifies that the embeddings will be used for classification. |\n",
"| `CLUSTERING` | Specifies that the embeddings will be used for clustering. |"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"id": "EeSW5NOBTS0a"
},
"outputs": [],
"source": [
"from google.genai import types\n",
"\n",
"# Select embedding model\n",
"MODEL_ID = \"gemini-embedding-001\" # @param [\"gemini-embedding-001\", \"text-embedding-004\"] {\"allow-input\": true, \"isTemplate\": true}\n",
"\n",
"\n",
"# Function to convert text to embeddings\n",
"def make_embed_text_fn(text, model=MODEL_ID,\n",
" task_type=\"retrieval_document\"):\n",
" embedding = client.models.embed_content(\n",
" model=model,\n",
" contents=text,\n",
" config=types.EmbedContentConfig(\n",
" task_type=task_type,\n",
" )\n",
" )\n",
" return embedding.embeddings[0].values"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "9ajByZTxVXal"
},
"source": [
"### Store the data using Qdrant\n",
"\n",
" Next, you'll store the embeddings of the website data in Qdrant's vector store.\n",
"\n",
" First, you have to initiate a Qdrant client by creating an instance of `QdrantClient`. In this tutorial, you will store the embeddings in memory. To create an in-memory Qdrant client specify `:memory:` for the `location` argument of the `QdrantClient` class initializer. You can read more about the different types of storage in Qdrant in the [storage reference guide](https://qdrant.tech/documentation/concepts/storage/).\n",
"\n",
"After initializing the client, you have to create a Qdrant collection using the `recreate_collection` function of `QdrantClient`. You can specify your vector configuration inside the `recreate_collection` function. Pass an instance of `VectorParams` with the `size` set to `768` to match the embedding model and `distance` set to cosine.\n",
"\n",
"**Note**: Since you will run the script several times during your experiments, `recreate_collection` is appropriate for this tutorial. `recreate_collection` will first try to remove an existing collection with the same name."
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {
"id": "pnURtmtZTImC"
},
"outputs": [
{
"data": {
"text/plain": [
"True"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Initialize Qdrant client.\n",
"qdrant = QdrantClient(\":memory:\")\n",
"\n",
"# Create a collection named \"GeminiCollection\".\n",
"qdrant.create_collection(\n",
" collection_name=\"GeminiCollection\",\n",
" vectors_config=models.VectorParams(\n",
" size=3072, # Vector size of `gemini-embedding-001`\n",
" distance=models.Distance.COSINE,\n",
" ),\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "YQu7FIhLeC0O"
},
"source": [
"You will now insert the `documents` you parsed from the website data into the Qdrant collection you created earlier and index them using the `upsert` function of `QdrantClient`.\n",
"\n",
"The `upsert` function takes the data to be stored and indexed as an array of `PointsStruct`s.\n",
"\n",
"Points are the main entity in Qdrant operations. A point is a record consisting of a vector and an optional payload. You can perform a similarity search among the points in one collection. Read more about points in [Qdrant's points documentation](https://qdrant.tech/documentation/concepts/points/).\n",
"\n",
"You'll create an array of points by enumerating over the documents you prepared earlier from the website data."
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {
"id": "uOqivudxSyR9"
},
"outputs": [
{
"data": {
"text/plain": [
"UpdateResult(operation_id=0, status=<UpdateStatus.COMPLETED: 'completed'>)"
]
},
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Qdrant uses batch loading of points to optimize performance.\n",
"# You can create a batch in two ways - record-oriented and column-oriented.\n",
"# Here you are using the record-oriented approach.\n",
"\n",
"qdrant.upsert(\n",
" collection_name=\"GeminiCollection\",\n",
" points=[\n",
" # Use PointStruct function to intialize the point.\n",
" models.PointStruct(\n",
" # Use `make_embed_text_fn` to convert text to embeddings.\n",
" # Pass the same data as payload for a refined search.\n",
" id=idx, vector=make_embed_text_fn(doc[\"content\"]), payload = doc\n",
" )\n",
" for idx, doc in enumerate(documents)\n",
" ]\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "JdVrKZZ0cTkV"
},
"source": [
"## 2. Query the index\n",
"\n",
"You'll now query the Qdrant index you created earlier with a question related to the data contained in the website documents.\n",
"To query the index, you have to mention the collection name and the query vector. The query vector should be first converted to an embedding vector using the Gemini API embedding model you leveraged to create embedding vectors for the website data. Use the `make_embed_text_fn` you defined earlier for creating an embedding vector from your query. Since you are embedding a query string that is being used to search `retrieval_document` embeddings, the `task_type` must be set to `retrieval_query`."
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {
"id": "6LQVKNfMTyOx"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"score: 0.8047714249217505 - content: Already, it is starting to address climate challenges in three key areas: providing people and organizations with better information to make more sustainable choices, delivering improved predictions to help adapt to climate change, and finding recommendations to optimize climate action for high-impact applications Here’s a look at how, at Google, we’ve used AI to address climate challenges:Providing helpful information: People are looking for information to reduce their environmental footprint Fuel-efficient routing in Google Maps uses AI to suggest routes that have fewer hills, less traffic, and constant speeds with the same or similar ETA\n",
"score: 0.7745056851050607 - content: Managing the environmental impact of AIWhile scaling these applications of AI and finding new ways to use it to accelerate climate action is crucial, we need to build AI responsibly and manage the environmental impact associated with it As AI is at an inflection point, predicting the future growth of energy use and emissions from AI compute in our data centers is challenging Historically, data center energy consumption has grown much more slowly than demand for computing power\n",
"score: 0.7709980934840739 - content: Promoting environmentally and socially responsible deployment of AI Together, we can boldly and responsibly develop more tools and products that harness the power of AI to accelerate the climate progress we need \n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"/tmp/ipykernel_47101/1026465469.py:1: DeprecationWarning: `search` method is deprecated and will be removed in the future. Use `query_points` instead.\n",
" hits = qdrant.search(\n"
]
}
],
"source": [
"hits = qdrant.search(\n",
" collection_name=\"GeminiCollection\",\n",
" query_vector=make_embed_text_fn(\"How can AI address climate challenges?\",\n",
" task_type=\"retrieval_query\"),\n",
" limit=3,\n",
")\n",
"for hit in hits:\n",
" print(\"score:\", hit.score, \"- content:\", hit.payload.get(\"content\").replace(\"\\n\", \"\"))"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Tt1wSSMIxsf2"
},
"source": [
"## Conclusion\n",
"\n",
"That's it. You have successfully performed a similarity search using Qdrant with the help of a Gemini API embedding model."
]
}
],
"metadata": {
"colab": {
"name": "Qdrant_similarity_search.ipynb",
"toc_visible": true
},
"kernelspec": {
"display_name": "Python 3",
"name": "python3"
}
},
"nbformat": 4,
"nbformat_minor": 0
}