Skip to content

Commit

Permalink
Merge branch 'main' into feat/adds-mvi-limits
Browse files Browse the repository at this point in the history
  • Loading branch information
@eaddingtonwhite
eaddingtonwhite committed Oct 6, 2023
2 parents ede9ebd + 17644c0 commit b218f3c
Show file tree
Hide file tree
Showing 6 changed files with 305 additions and 69 deletions.
151 changes: 83 additions & 68 deletions docs/vector-index/develop/sdks/python/cheat-sheet.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,15 +12,10 @@ If you need to get going quickly with Python and Momento Vector Index, this page

![An image of a python on a pile of books searching through them like a vector index.](@site/static/img/vector-index/python-mvi-cheat-sheet.jpg)

:::tip

If you combine all of the functions on this page into one python file, you'd have a central collection of functions you can import and call from other python code. In addition, if you are using this code in production you might look to replace the print() calls with ones using the logging library (`import logging`) in python. [Click here](@site/static/code/cheat-sheets/MomentoBasics.py) to see the class file with all definitions in it.
## Import libraries and instantiate a vector index client

:::

## Import libraries and connect to return a PreviewVectorIndexClient object

This code sets up the class with the necessary imports, the class definition, and the PreviewVectorIndexClient instantiation that each of the other functions will need to call.
This code sets up the class with the necessary imports, the class definition, and the `PreviewVectorIndexClient` instantiation that each of the other functions will need to call.

```python
from momento import (
Expand All @@ -29,108 +24,128 @@ from momento import (
VectorIndexConfigurations,
)

from momento.requests.vector_index import Item
from momento.requests.vector_index import (
ALL_METADATA,
Item,
SimilarityMetric
)
from momento.responses.vector_index import (
AddItemBatch,
UpsertItemBatch,
CreateIndex,
DeleteIndex,
DeleteItemBatch,
ListIndexes,
Search,
)

def create_vector_index_client():
momento_auth_token = CredentialProvider.from_environment_variable('MOMENTO_AUTH_TOKEN')
config = {
'configuration': VectorIndexConfigurations.Default.latest(),
'credential_provider': momento_auth_token,
}
return PreviewVectorIndexClient(**config)
client = PreviewVectorIndexClient(
configuration=VectorIndexConfigurations.Default.latest(),
credential_provider=CredentialProvider.from_environment_variable('MOMENTO_API_KEY')
)
```

The following examples assume that you have already instantiated a `PreviewVectorIndexClient` as shown above.

## Create a new index in Momento Vector Index

Use this function to create a new index in your account.
Use this snippet to create a new index in your account. The `similarity_metric` parameter is optional and defaults to `SimilarityMetric.COSINE_SIMILARITY`.

```python
def create_index(client, index_name: str) -> None:
print("Creating index with name " + index_name)
create_index_response = client.create_index(index_name, num_dimensions=2)
if isinstance(create_index_response, CreateIndex.Success):
print("Index with name " + index_name + " successfully created!")
elif isinstance(create_index_response, CreateIndex.IndexAlreadyExists):
print("Index with name " + index_name + " already exists")
elif isinstance(create_index_response, CreateIndex.Error):
raise(Exception("Error while creating index " + create_index_response.message))
index_name = "my_index"
# The number of dimensions in your vectors
num_dimensions = 2
similarity_metric = SimilarityMetric.COSINE_SIMILARITY

create_index_response = client.create_index(index_name, num_dimensions, similarity_metric)
if isinstance(create_index_response, CreateIndex.Success):
print(f"Index with name {index_name!r} successfully created!")
elif isinstance(create_index_response, CreateIndex.IndexAlreadyExists):
print(f"Index with name {index_name!r} already exists")
elif isinstance(create_index_response, CreateIndex.Error):
print(f"Error while creating index: {create_index_response.message}")
```

## Get list of existing indexes in your account

In this example, we use the client function above to get a client object and then use it to list all of the indexes for this account.
In this example, we list the indexes in your account.

```python
def list_indexes(client) -> None:
print("Listing indexes")
list_indexes_response = client.list_indexes()
if isinstance(list_indexes_response, ListIndexes.Success):
for index in list_indexes_response.index_names:
print("Account has an index with name " + index)
elif isinstance(list_indexes_response, ListIndexes.Error):
print(Exception("Error while listing indexes " + list_indexes_response.message))
list_indexes_response = client.list_indexes()
if isinstance(list_indexes_response, ListIndexes.Success):
for index in list_indexes_response.index_names:
print(f"Account has an index with name: {index!r}")
elif isinstance(list_indexes_response, ListIndexes.Error):
print(f"Error while listing indexes: {list_indexes_response.message}")
```

## Write an item batch to the index
## Write a batch of items to the index

A simple example of doing an `add_item_batch` operation.
A simple example of doing an `upsert_item_batch` operation. This operation will insert the items if they don't exist, or replace them if they do.

```python
def add_items(client, index_name: str):
items = [
index_name = "my_index"
items = [
Item(id="test_item_1", vector=[1.0, 2.0], metadata={"key1": "value1"}),
Item(id="test_item_2", vector=[3.0, 4.0], metadata={"key2": "value2"}),
Item(id="test_item_3", vector=[5.0, 6.0], metadata={"key1": "value3", "key3": "value3"}),
]
print("Adding items " + str(items))
add_response = client.add_item_batch(
index_name,
items=items,
)
if isinstance(add_response, AddItemBatch.Success):
print("Successfully added items")
elif isinstance(add_response, AddItemBatch.Error):
raise(Exception("Error while adding items to index " + index_name + " " + add_response.message))
]

print(f"Upserting items {items}")
upsert_response = client.upsert_item_batch(
index_name,
items=items,
)
if isinstance(upsert_response, UpsertItemBatch.Success):
print("Successfully upserted items")
elif isinstance(upsert_response, UpsertItemBatch.Error):
print(f"Error while adding items to index {index_name!r}: {upsert_response.message}")
```

## Searching the index

This is an example of a search operation to get the topK items from the index matching the `query_vector`.
This is an example of a search operation to get the top-k items from the index matching the `query_vector`. The `metadata_fields` parameter is optional and can be used to specify which metadata fields to return in the response.

Here we use a `query_vector` of `[1.0, 2.0]` and ask for the top 2 results.


```python
def search(client, index_name: str):
query_vector = [1.0, 2.0]
top_k = 2
print("Searching index " + index_name + " with query_vector " + str(query_vector) + " and top " + str(top_k) + " elements")
search_response = client.search(index_name, query_vector=query_vector, top_k=top_k)
if isinstance(search_response, Search.Success):
print("Search succeeded with " + str(len(search_response.hits)) + " matches")
elif isinstance(search_response, Search.Error):
raise(Exception("Error while searching on index " + index_name + " " + search_response.message))
index_name = "my_index"
query_vector = [1.0, 2.0]
top_k = 2

print(f"Searching index {index_name} with query_vector {query_vector} and top {top_k} elements")
search_response = client.search(index_name, query_vector=query_vector, top_k=top_k, metadata_fields=ALL_METADATA)
if isinstance(search_response, Search.Success):
print(f"Search succeeded with {len(search_response.hits)} matches")
elif isinstance(search_response, Search.Error):
print(f"Error while searching on index {index_name}: {search_response.message}")
```

## Deleting items from the index

An example of deleting the items from an index using `delete_item_batch`.

```python
def delete_items(client, index_name: str):
item_ids_to_delete = ["test_item_1", "test_item_3"]
delete_response = client.delete_item_batch(index_name, ids=item_ids_to_delete)
if isinstance(delete_response, DeleteItemBatch.Success):
print("Successfully deleted items")
elif isinstance(delete_response, DeleteItemBatch.Error):
raise(Exception("Error while deleting items " + delete_response.message))
index_name = "my_index"
item_ids_to_delete = ["test_item_1", "test_item_3"]

delete_response = client.delete_item_batch(index_name, ids=item_ids_to_delete)
if isinstance(delete_response, DeleteItemBatch.Success):
print("Successfully deleted items")
elif isinstance(delete_response, DeleteItemBatch.Error):
print(f"Error while deleting items {delete_response.message}")
```

## Usage notes
## Deleting an index

An example of deleting an index using `delete_index`.

For any of these functions, call the `create_vector_index_client()` function which returns a `PreviewVectorIndexClient` object. Then pass that object into subsequent functions. This way, calls are more efficient as they reuse the `PreviewVectorIndexClient` for multiple calls to Momento.
```python
index_name = "my_index"

delete_response = client.delete_index(index_name)
if isinstance(delete_response, DeleteIndex.Success):
print("Successfully deleted index")
elif isinstance(delete_response, DeleteIndex.Error):
print(f"Error while deleting index {delete_response.message}")
```
2 changes: 1 addition & 1 deletion docs/vector-index/develop/sdks/python/index.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,6 @@ The source code can be found on GitHub: [momentohq/client-sdk-python](https://gi

## Resources

- [Python SDK Examples](https://github.com/momentohq/client-sdk-python/blob/main/examples/README.md): working example projects that illustrate how to use the Python SDK
- [Python SDK Examples](https://github.com/momentohq/client-sdk-python/blob/main/examples/README.md): working example projects that illustrate how to use the Python SDK with Momento Vector Index.
- COMING SOON: Observability: Logging and Client-side Metrics with the Python SDK
- COMING SOON: Taking your code to prod: Configuration and Error handling in the Python SDK
32 changes: 32 additions & 0 deletions i18n/ja/docusaurus-plugin-content-docs/current/vector-index/develop/index.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
---
sidebar_position: 2
sidebar_label: Develop
sidebar_class_name: "sidebar-item-api-reference"
title: Developing applications with Momento SDKs
description: Learn the core concepts for getting started with the Momento SDKs.
---

import { SdkExampleTabs } from "@site/src/components/SdkExampleTabs";
// This import is necessary even though it looks like it's un-used; The inject-example-code-snippet
// plugin will transform instances of SdkExampleTabs to SdkExampleTabsImpl
import { SdkExampleTabsImpl } from "@site/src/components/SdkExampleTabsImpl";

# Developing applications with Momento SDKs

Welcome! This page provides information about common constructs you will need in order to assemble Momento clients in all of our SDKs. This page covers how to provide your Momento credentials (called api keys), how to configure your client, and some basic information about error handling and production readiness.

![a technical illustration on a white background depicting the intersection of speed, ease of use, and security.](@site/static/img/city.jpg)

## Constructing a Preview Momento Vector Index client

Work in progress.

## Instantiating credential providers using Momento api keys

You need to provide a Momento auth token when instantiating a Momento client. If you don't have one yet, you can get one from the [Momento Web Console](https://console.gomomento.com/). Once you have a token, provide it to Momento SDKs when you create an instance of `CredentialProvider`. There are convenient factory methods provided to construct a `CredentialProvider` object, either from an environment variable or from a String. Below is an example of how to instantiate `CredentialProvider` from an environment variable:

<SdkExampleTabs snippetId={'API_CredentialProviderFromEnvVar'} />

If you're storing your Momento auth token in a secret manager such as [AWS Secret Manager](https://aws.amazon.com/secrets-manager/), [GCP Secret Manager](https://cloud.google.com/secret-manager), or a local config file, you must first retrieve the credentials from there and then instantiate a `CredentialProvider` from a string, like this:

<SdkExampleTabs snippetId={'API_CredentialProviderFromString'} />
10 changes: 10 additions & 0 deletions i18n/ja/docusaurus-plugin-content-docs/current/vector-index/develop/sdks/_category_.json
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
{
"label": "SDKs",
"position": 7,
"collapsible": true,
"collapsed": true,
"link": {
"description": "Momento SDKs, available in all of your favorite programming languages!"
},
"className": "sidebar-item-api-reference"
}
Loading

0 comments on commit b218f3c

Please sign in to comment.