camel.storages.vectordb_storages package

abstract status() → VectorDBStatus[source]#

Returns status of the vector database.

Returns:: The vector database status.
Return type:: VectorDBStatus

class camel.storages.vectordb_storages.base.VectorDBQuery(query_vector: List[float], top_k: int)[source]#

Bases: BaseModel

Represents a query to a vector database.

query_vector#

The numerical representation of the query vector.

Type:: List[float]

top_k#

The number of top similar vectors to retrieve from the database. (default: 1)

Type:: int, optional

model_config: ClassVar[ConfigDict] = {}#: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

query_vector: List[float]#: The numerical representation of the query vector.

top_k: int#: The number of top similar vectors to retrieve from the database.

class camel.storages.vectordb_storages.base.VectorDBQueryResult(*, record: VectorRecord, similarity: float)[source]#

Bases: BaseModel

Encapsulates the result of a query against a vector database.

record#

The target vector record.

Type:: VectorRecord

similarity#

The similarity score between the query vector and the record.

Type:: float

classmethod create(similarity: float, vector: List[float], id: str, payload: Dict[str, Any] | None = None) → VectorDBQueryResult[source]#: A class method to construct a VectorDBQueryResult instance.

model_config: ClassVar[ConfigDict] = {}#: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

record: VectorRecord#

similarity: float#

class camel.storages.vectordb_storages.base.VectorDBStatus(*, vector_dim: int, vector_count: int)[source]#

Bases: BaseModel

Vector database status.

vector_dim#

The dimension of stored vectors.

Type:: int

vector_count#

The number of stored vectors.

Type:: int

model_config: ClassVar[ConfigDict] = {}#: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

vector_count: int#

vector_dim: int#

class camel.storages.vectordb_storages.base.VectorRecord(*, vector: ~typing.List[float], id: str = <factory>, payload: ~typing.Dict[str, ~typing.Any] | None = None)[source]#

Bases: BaseModel

Encapsulates information about a vector’s unique identifier and its payload, which is primarily used as a data transfer object when saving to vector storage.

vector#

The numerical representation of the vector.

Type:: List[float]

id#

A unique identifier for the vector. If not provided, an random uuid will be assigned.

Type:: str, optional

payload#

Any additional metadata or information related to the vector. (default: None)

Type:: Optional[Dict[str, Any]], optional

id: str#

model_config: ClassVar[ConfigDict] = {}#: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

payload: Dict[str, Any] | None#

vector: List[float]#

camel.storages.vectordb_storages.milvus module#

class camel.storages.vectordb_storages.milvus.MilvusStorage(vector_dim: int, url_and_api_key: Tuple[str, str], collection_name: str | None = None, **kwargs: Any)[source]#

An implementation of the BaseVectorStorage for interacting with Milvus, a cloud-native vector search engine.

The detailed information about Milvus is available at: Milvus

Parameters:

vector_dim (int) – The dimension of storing vectors.
url_and_api_key (Tuple[str, str]) – Tuple containing the URL and API key for connecting to a remote Milvus instance. URL maps to Milvus uri concept, typically “endpoint:port”. API key maps to Milvus token concept, for self-hosted it’s “username:pwd”, for Zilliz Cloud (fully-managed Milvus) it’s API Key.
collection_name (Optional[str], optional) – Name for the collection in the Milvus. If not provided, set it to the current time with iso format. (default: None)
**kwargs (Any) – Additional keyword arguments for initializing MilvusClient.

Raises:

ImportError – If pymilvus package is not installed.

add(records: List[VectorRecord], **kwargs) → None[source]#

Adds a list of vectors to the specified collection.

Parameters:

records (List[VectorRecord]) – List of vectors to be added.
**kwargs (Any) – Additional keyword arguments pass to insert.

Raises:

RuntimeError – If there was an error in the addition process.

clear() → None[source]#: Removes all vectors from the Milvus collection. This method deletes the existing collection and then recreates it with the same schema to effectively remove all stored vectors.

property client: Any#

Provides direct access to the Milvus client. This property allows for direct interactions with the Milvus client for operations that are not covered by the MilvusStorage class.

Returns:: The Milvus client instance.
Return type:: Any

delete(ids: List[str], **kwargs: Any) → None[source]#

Deletes a list of vectors identified by their IDs from the storage. If unsure of ids you can first query the collection to grab the corresponding data.

Parameters:

ids (List[str]) – List of unique identifiers for the vectors to be deleted.
**kwargs (Any) – Additional keyword arguments passed to delete.

Raises:

RuntimeError – If there is an error during the deletion process.

load() → None[source]#: Load the collection hosted on cloud service.

query(query: VectorDBQuery, **kwargs: Any) → List[VectorDBQueryResult][source]#

Searches for similar vectors in the storage based on the provided query.

Parameters:

query (VectorDBQuery) – The query object containing the search vector and the number of top similar vectors to retrieve.
**kwargs (Any) – Additional keyword arguments passed to search.

Returns:

A list of vectors retrieved from the: storage based on similarity to the query vector.

Return type:

status() → VectorDBStatus[source]#

Retrieves the current status of the Milvus collection. This method provides information about the collection, including its vector dimensionality and the total number of vectors stored.

Returns:

An object containing information about the: collection’s status.

Return type:

camel.storages.vectordb_storages.qdrant module#

class camel.storages.vectordb_storages.tidb.EnumEncoder(*, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, sort_keys=False, indent=None, separators=None, default=None)[source]#

Bases: JSONEncoder

default(obj)[source]#

Implement this method in a subclass such that it returns a serializable object for o, or calls the base implementation (to raise a TypeError).

For example, to support arbitrary iterators, you could implement default like this:

def default(self, o):
    try:
        iterable = iter(o)
    except TypeError:
        pass
    else:
        return list(iterable)
    # Let the base class default method raise the TypeError
    return JSONEncoder.default(self, o)

class camel.storages.vectordb_storages.tidb.TiDBStorage(vector_dim: int, collection_name: str | None = None, url_and_api_key: Tuple[str, str] | str | None = None, **kwargs: Any)[source]#

An implementation of the BaseVectorStorage for interacting with TiDB.

The detailed information about TiDB is available at: TiDB Vector Search

Parameters:

vector_dim (int) – The dimension of storing vectors.
url_and_api_key (Optional[Union[Tuple[str, str], str]]) –

A tuple
containing the database url and API key for connecting to a TiDB cluster. The URL should be in the format:

”mysql+pymysql://<username>:<password>@<host>:<port>/<db_name>”. TiDB will not use the API Key, but retains the definition for interface compatible.
collection_name (Optional[str]) – Name of the collection. The collection name will be used as the table name in TiDB. If not provided, set it to the current time with iso format.
**kwargs (Any) – Additional keyword arguments for initializing TiDB connection.

Raises:

ImportError – If pytidb package is not installed.

add(records: List[VectorRecord], **kwargs) → None[source]#

Adds a list of vectors to the specified table.

Parameters:

records (List[VectorRecord]) – List of vectors to be added.
**kwargs (Any) – Additional keyword arguments pass to insert.

Raises:

RuntimeError – If there was an error in the addition process.

clear() → None[source]#: Removes all vectors from the TiDB table. This method deletes the existing table and then recreates it with the same schema to effectively remove all stored vectors.

property client: TiDBClient#

Provides direct access to the TiDB client.

Returns:: The TiDB client instance.
Return type:: Any

delete(ids: List[str], **kwargs: Any) → None[source]#

Deletes a list of vectors identified by their IDs from the storage.

Parameters:

ids (List[str]) – List of unique identifiers for the vectors to be deleted.
**kwargs (Any) – Additional keyword arguments passed to delete.

Raises:

RuntimeError – If there is an error during the deletion process.

load() → None[source]#: Load the collection hosted on cloud service.

query(query: VectorDBQuery, **kwargs: Any) → List[VectorDBQueryResult][source]#

Searches for similar vectors in the storage based on the provided query.

Parameters:

query (VectorDBQuery) – The query object containing the search vector and the number of top similar vectors to retrieve.
**kwargs (Any) – Additional keyword arguments passed to search.

Returns:

A list of vectors retrieved from the: storage based on similarity to the query vector.

Return type:

status() → VectorDBStatus[source]#

Retrieves the current status of the TiDB table.

Returns:

An object containing information about the: table’s status.

Return type:

camel.storages.vectordb_storages.tidb module#

class camel.storages.vectordb_storages.qdrant.QdrantStorage(vector_dim: int, collection_name: str | None = None, url_and_api_key: Tuple[str, str] | None = None, path: str | None = None, distance: VectorDistance = VectorDistance.COSINE, delete_collection_on_del: bool = False, **kwargs: Any)[source]#

An implementation of the BaseVectorStorage for interacting with Qdrant, a vector search engine.

The detailed information about Qdrant is available at: Qdrant

Parameters:

vector_dim (int) – The dimension of storing vectors.
collection_name (Optional[str], optional) – Name for the collection in the Qdrant. If not provided, set it to the current time with iso format. (default: None)
url_and_api_key (Optional[Tuple[str, str]], optional) – Tuple containing the URL and API key for connecting to a remote Qdrant instance. (default: None)
path (Optional[str], optional) – Path to a directory for initializing a local Qdrant client. (default: None)
distance (VectorDistance, optional) – The distance metric for vector comparison (default: VectorDistance.COSINE)
delete_collection_on_del (bool, optional) – Flag to determine if the collection should be deleted upon object destruction. (default: False)
**kwargs (Any) – Additional keyword arguments for initializing QdrantClient.

Notes

If url_and_api_key is provided, it takes priority and the client will attempt to connect to the remote Qdrant instance using the URL endpoint.
If url_and_api_key is not provided and path is given, the client will use the local path to initialize Qdrant.
If neither url_and_api_key nor path is provided, the client will be initialized with an in-memory storage (“:memory:”).

add(records: List[VectorRecord], **kwargs) → None[source]#

Adds a list of vectors to the specified collection.

Parameters:

vectors (List[VectorRecord]) – List of vectors to be added.
**kwargs (Any) – Additional keyword arguments.

Raises:

RuntimeError – If there was an error in the addition process.

clear() → None[source]#: Remove all vectors from the storage.

property client: QdrantClient#: Provides access to the underlying vector database client.

close_client(**kwargs)[source]#: Closes the client connection to the Qdrant storage.

delete(ids: List[str] | None = None, payload_filter: Dict[str, Any] | None = None, **kwargs: Any) → None[source]#

Deletes points from the collection based on either IDs or payload filters.

Parameters:

ids (Optional[List[str]], optional) – List of unique identifiers for the vectors to be deleted.
payload_filter (Optional[Dict[str, Any]], optional) – A filter for the payload to delete points matching specific conditions. If ids is provided, payload_filter will be ignored unless both are combined explicitly.
**kwargs (Any) – Additional keyword arguments pass to QdrantClient. delete.

Examples

>>> # Delete points with IDs "1", "2", and "3"
>>> storage.delete(ids=["1", "2", "3"])
>>> # Delete points with payload filter
>>> storage.delete(payload_filter={"name": "Alice"})

Raises:

ValueError – If neither ids nor payload_filter is provided.
RuntimeError – If there is an error during the deletion process.

Notes

If ids is provided, the points with these IDs will be deleted
directly, and the payload_filter will be ignored.
If ids is not provided but payload_filter is, then points
matching the payload_filter will be deleted.

delete_collection() → None[source]#: Deletes the entire collection in the Qdrant storage.

load() → None[source]#: Load the collection hosted on cloud service.

query(query: VectorDBQuery, filter_conditions: Dict[str, Any] | None = None, **kwargs: Any) → List[VectorDBQueryResult][source]#

Searches for similar vectors in the storage based on the provided query.

Parameters:

query (VectorDBQuery) – The query object containing the search vector and the number of top similar vectors to retrieve.
filter_conditions (Optional[Dict[str, Any]], optional) – A dictionary specifying conditions to filter the query results.
**kwargs (Any) – Additional keyword arguments.

Returns:

A list of vectors retrieved from the: storage based on similarity to the query vector.

Return type:

status() → VectorDBStatus[source]#

Returns status of the vector database.

Returns:: The vector database status.
Return type:: VectorDBStatus

update_payload(ids: List[str], payload: Dict[str, Any], **kwargs: Any) → None[source]#

Updates the payload of the vectors identified by their IDs.

Parameters:

ids (List[str]) – List of unique identifiers for the vectors to be updated.
payload (Dict[str, Any]) – List of payloads to be updated.
**kwargs (Any) – Additional keyword arguments.

Raises:

RuntimeError – If there is an error during the update process.

camel.storages.vectordb_storages.faiss module#

class camel.storages.vectordb_storages.faiss.FaissStorage(vector_dim: int, index_type: str = 'Flat', collection_name: str | None = None, storage_path: str | None = None, distance: VectorDistance = VectorDistance.COSINE, nlist: int = 100, m: int = 16, **kwargs: Any)[source]#

An implementation of the BaseVectorStorage using FAISS, Facebook AI’s Similarity Search library for efficient vector search.

The detailed information about FAISS is available at: FAISS

Parameters:

vector_dim (int) – The dimension of storing vectors.
index_type (str, optional) – Type of FAISS index to create. Options include ‘Flat’, ‘IVF’, ‘HNSW’, etc. (default: 'Flat')
collection_name (Optional[str], optional) – Name for the collection. If not provided, set it to the current time with iso format. (default: None)
storage_path (Optional[str], optional) – Path to directory where the index will be stored. If None, index will only exist in memory. (default: None)
distance (VectorDistance, optional) – The distance metric for vector comparison (default: VectorDistance.COSINE)
nlist (int, optional) – Number of cluster centroids for IVF indexes. Only used if index_type includes ‘IVF’. (default: 100)
m (int, optional) – HNSW parameter. Number of connections per node. Only used if index_type includes ‘HNSW’. (default: 16)
**kwargs (Any) – Additional keyword arguments.

Notes

FAISS offers various index types optimized for different use cases: - ‘Flat’: Exact search, but slowest for large datasets - ‘IVF’: Inverted file index, good balance of speed and recall - ‘HNSW’: Hierarchical Navigable Small World, fast with high recall - ‘PQ’: Product Quantization for memory-efficient storage
The choice of index should be based on your specific requirements for search speed, memory usage, and accuracy.

add(records: List[VectorRecord], **kwargs) → None[source]#

Adds a list of vectors to the index.

Parameters:

records (List[VectorRecord]) – List of vector records to be added.
**kwargs (Any) – Additional keyword arguments.

Raises:

RuntimeError – If there was an error in the addition process.

clear() → None[source]#: Remove all vectors from the storage.

property client: Any#: Provides access to the underlying FAISS client.

delete(ids: List[str] | None = None, payload_filter: Dict[str, Any] | None = None, **kwargs: Any) → None[source]#

Deletes vectors from the index based on either IDs or payload filters.

Parameters:

ids (Optional[List[str]], optional) – List of unique identifiers for the vectors to be deleted.
payload_filter (Optional[Dict[str, Any]], optional) – A filter for the payload to delete points matching specific conditions.
**kwargs (Any) – Additional keyword arguments.

Raises:

ValueError – If neither ids nor payload_filter is provided.
RuntimeError – If the FAISS index does not support removal.

Notes

FAISS does not support efficient single vector removal for most index types. This implementation recreates the index without the deleted vectors, which can be inefficient for large datasets.
If both ids and payload_filter are provided, both filters will be applied (vectors matching either will be deleted).

delete_collection() → None[source]#: Deletes the entire collection (index and metadata).

load() → None[source]#: Load the index from disk if storage_path is provided.

query(query: VectorDBQuery, filter_conditions: Dict[str, Any] | None = None, **kwargs: Any) → List[VectorDBQueryResult][source]#

Searches for similar vectors in the storage based on the provided query.

Parameters:

query (VectorDBQuery) – The query object containing the search vector and the number of top similar vectors to retrieve.
filter_conditions (Optional[Dict[str, Any]], optional) – A dictionary specifying conditions to filter the query results.
**kwargs (Any) – Additional keyword arguments.

Returns:

A list of query results ordered by: similarity.

Return type:

status() → VectorDBStatus[source]#

Returns the status of the vector database.

Returns:: Current status of the vector database.
Return type:: VectorDBStatus

update_payload(ids: List[str], payload: Dict[str, Any], **kwargs: Any) → None[source]#

Updates the payload of the vectors identified by their IDs.

Parameters:

ids (List[str]) – List of unique identifiers for the vectors to be updated.
payload (Dict[str, Any]) – Payload to be updated for all specified IDs.
**kwargs (Any) – Additional keyword arguments.

Raises:

KeyError – If any of the provided IDs does not exist in the index.

Module contents#

class camel.storages.vectordb_storages.BaseVectorStorage[source]#

Bases: ABC

An abstract base class for vector storage systems.

abstract add(records: List[VectorRecord], **kwargs: Any) → None[source]#

Saves a list of vector records to the storage.

Parameters:

records (List[VectorRecord]) – List of vector records to be saved.
**kwargs (Any) – Additional keyword arguments.

Raises:

RuntimeError – If there is an error during the saving process.

abstract clear() → None[source]#: Remove all vectors from the storage.

abstract property client: Any#: Provides access to the underlying vector database client.

abstract delete(ids: List[str], **kwargs: Any) → None[source]#

Deletes a list of vectors identified by their IDs from the storage.

Parameters:

ids (List[str]) – List of unique identifiers for the vectors to be deleted.
**kwargs (Any) – Additional keyword arguments.

Raises:

RuntimeError – If there is an error during the deletion process.

get_payloads_by_vector(vector: List[float], top_k: int) → List[Dict[str, Any]][source]#

Returns payloads of top k vector records that closest to the given vector.

This function is a wrapper of BaseVectorStorage.query.

Parameters:

vector (List[float]) – The search vector.
top_k (int) – The number of top similar vectors.

Returns:

A list of vector payloads retrieved: from the storage based on similarity to the query vector.

Return type:

List[List[Dict[str, Any]]]

abstract load() → None[source]#: Load the collection hosted on cloud service.

abstract query(query: VectorDBQuery, **kwargs: Any) → List[VectorDBQueryResult][source]#

Searches for similar vectors in the storage based on the provided query.

Parameters:

query (VectorDBQuery) – The query object containing the search vector and the number of top similar vectors to retrieve.
**kwargs (Any) – Additional keyword arguments.

Returns:

A list of vectors retrieved from the: storage based on similarity to the query vector.

Return type:

abstract status() → VectorDBStatus[source]#

Returns status of the vector database.

Returns:: The vector database status.
Return type:: VectorDBStatus

class camel.storages.vectordb_storages.FaissStorage(vector_dim: int, index_type: str = 'Flat', collection_name: str | None = None, storage_path: str | None = None, distance: VectorDistance = VectorDistance.COSINE, nlist: int = 100, m: int = 16, **kwargs: Any)[source]#

An implementation of the BaseVectorStorage using FAISS, Facebook AI’s Similarity Search library for efficient vector search.

The detailed information about FAISS is available at: FAISS

Parameters:

vector_dim (int) – The dimension of storing vectors.
index_type (str, optional) – Type of FAISS index to create. Options include ‘Flat’, ‘IVF’, ‘HNSW’, etc. (default: 'Flat')
collection_name (Optional[str], optional) – Name for the collection. If not provided, set it to the current time with iso format. (default: None)
storage_path (Optional[str], optional) – Path to directory where the index will be stored. If None, index will only exist in memory. (default: None)
distance (VectorDistance, optional) – The distance metric for vector comparison (default: VectorDistance.COSINE)
nlist (int, optional) – Number of cluster centroids for IVF indexes. Only used if index_type includes ‘IVF’. (default: 100)
m (int, optional) – HNSW parameter. Number of connections per node. Only used if index_type includes ‘HNSW’. (default: 16)
**kwargs (Any) – Additional keyword arguments.

Notes

FAISS offers various index types optimized for different use cases: - ‘Flat’: Exact search, but slowest for large datasets - ‘IVF’: Inverted file index, good balance of speed and recall - ‘HNSW’: Hierarchical Navigable Small World, fast with high recall - ‘PQ’: Product Quantization for memory-efficient storage
The choice of index should be based on your specific requirements for search speed, memory usage, and accuracy.

add(records: List[VectorRecord], **kwargs) → None[source]#

Adds a list of vectors to the index.

Parameters:

records (List[VectorRecord]) – List of vector records to be added.
**kwargs (Any) – Additional keyword arguments.

Raises:

RuntimeError – If there was an error in the addition process.

clear() → None[source]#: Remove all vectors from the storage.

property client: Any#: Provides access to the underlying FAISS client.

delete(ids: List[str] | None = None, payload_filter: Dict[str, Any] | None = None, **kwargs: Any) → None[source]#

Deletes vectors from the index based on either IDs or payload filters.

Parameters:

ids (Optional[List[str]], optional) – List of unique identifiers for the vectors to be deleted.
payload_filter (Optional[Dict[str, Any]], optional) – A filter for the payload to delete points matching specific conditions.
**kwargs (Any) – Additional keyword arguments.

Raises:

ValueError – If neither ids nor payload_filter is provided.
RuntimeError – If the FAISS index does not support removal.

Notes

FAISS does not support efficient single vector removal for most index types. This implementation recreates the index without the deleted vectors, which can be inefficient for large datasets.
If both ids and payload_filter are provided, both filters will be applied (vectors matching either will be deleted).

delete_collection() → None[source]#: Deletes the entire collection (index and metadata).

load() → None[source]#: Load the index from disk if storage_path is provided.

query(query: VectorDBQuery, filter_conditions: Dict[str, Any] | None = None, **kwargs: Any) → List[VectorDBQueryResult][source]#

Searches for similar vectors in the storage based on the provided query.

Parameters:

query (VectorDBQuery) – The query object containing the search vector and the number of top similar vectors to retrieve.
filter_conditions (Optional[Dict[str, Any]], optional) – A dictionary specifying conditions to filter the query results.
**kwargs (Any) – Additional keyword arguments.

Returns:

A list of query results ordered by: similarity.

Return type:

status() → VectorDBStatus[source]#

Returns the status of the vector database.

Returns:: Current status of the vector database.
Return type:: VectorDBStatus

update_payload(ids: List[str], payload: Dict[str, Any], **kwargs: Any) → None[source]#

Updates the payload of the vectors identified by their IDs.

Parameters:

ids (List[str]) – List of unique identifiers for the vectors to be updated.
payload (Dict[str, Any]) – Payload to be updated for all specified IDs.
**kwargs (Any) – Additional keyword arguments.

Raises:

KeyError – If any of the provided IDs does not exist in the index.

class camel.storages.vectordb_storages.MilvusStorage(vector_dim: int, url_and_api_key: Tuple[str, str], collection_name: str | None = None, **kwargs: Any)[source]#

An implementation of the BaseVectorStorage for interacting with Milvus, a cloud-native vector search engine.

The detailed information about Milvus is available at: Milvus

Parameters:

vector_dim (int) – The dimension of storing vectors.
url_and_api_key (Tuple[str, str]) – Tuple containing the URL and API key for connecting to a remote Milvus instance. URL maps to Milvus uri concept, typically “endpoint:port”. API key maps to Milvus token concept, for self-hosted it’s “username:pwd”, for Zilliz Cloud (fully-managed Milvus) it’s API Key.
collection_name (Optional[str], optional) – Name for the collection in the Milvus. If not provided, set it to the current time with iso format. (default: None)
**kwargs (Any) – Additional keyword arguments for initializing MilvusClient.

Raises:

ImportError – If pymilvus package is not installed.

add(records: List[VectorRecord], **kwargs) → None[source]#

Adds a list of vectors to the specified collection.

Parameters:

records (List[VectorRecord]) – List of vectors to be added.
**kwargs (Any) – Additional keyword arguments pass to insert.

Raises:

RuntimeError – If there was an error in the addition process.

clear() → None[source]#: Removes all vectors from the Milvus collection. This method deletes the existing collection and then recreates it with the same schema to effectively remove all stored vectors.

property client: Any#

Provides direct access to the Milvus client. This property allows for direct interactions with the Milvus client for operations that are not covered by the MilvusStorage class.

Returns:: The Milvus client instance.
Return type:: Any

delete(ids: List[str], **kwargs: Any) → None[source]#

Deletes a list of vectors identified by their IDs from the storage. If unsure of ids you can first query the collection to grab the corresponding data.

Parameters:

ids (List[str]) – List of unique identifiers for the vectors to be deleted.
**kwargs (Any) – Additional keyword arguments passed to delete.

Raises:

RuntimeError – If there is an error during the deletion process.

load() → None[source]#: Load the collection hosted on cloud service.

query(query: VectorDBQuery, **kwargs: Any) → List[VectorDBQueryResult][source]#

Searches for similar vectors in the storage based on the provided query.

Parameters:

query (VectorDBQuery) – The query object containing the search vector and the number of top similar vectors to retrieve.
**kwargs (Any) – Additional keyword arguments passed to search.

Returns:

A list of vectors retrieved from the: storage based on similarity to the query vector.

Return type:

status() → VectorDBStatus[source]#

Retrieves the current status of the Milvus collection. This method provides information about the collection, including its vector dimensionality and the total number of vectors stored.

Returns:

An object containing information about the: collection’s status.

Return type:

class camel.storages.vectordb_storages.OceanBaseStorage(vector_dim: int, table_name: str, uri: str = '127.0.0.1:2881', user: str = 'root@test', password: str = '', db_name: str = 'test', distance: Literal['l2', 'cosine'] = 'l2', delete_table_on_del: bool = False, **kwargs: Any)[source]#

An implementation of the BaseVectorStorage for interacting with OceanBase Vector Database.

Parameters:

vector_dim (int) – The dimension of storing vectors.
table_name (str) – Name for the table in OceanBase.
uri (str) – Connection URI for OceanBase (host:port). (default: "127.0.0.1:2881")
user (str) – Username for connecting to OceanBase. (default: "root@test")
password (str) – Password for the user. (default: "")
db_name (str) – Database name in OceanBase. (default: "test")
distance (Literal["l2", "cosine"], optional) – The distance metric for vector comparison. Options: “l2”, “cosine”. (default: "l2")
delete_table_on_del (bool, optional) – Flag to determine if the table should be deleted upon object destruction. (default: False)
**kwargs (Any) – Additional keyword arguments for initializing ObVecClient.

Raises:

ImportError – If pyobvector package is not installed.

add(records: List[VectorRecord], batch_size: int = 100, **kwargs: Any) → None[source]#

Saves a list of vector records to the storage.

Parameters:

records (List[VectorRecord]) – List of vector records to be saved.
batch_size (int) – Number of records to insert each batch. Larger batches are more efficient but use more memory. (default: 100)
**kwargs (Any) – Additional keyword arguments.

Raises:

RuntimeError – If there is an error during the saving process.
ValueError – If any vector dimension doesn’t match vector_dim.

clear() → None[source]#: Remove all vectors from the storage.

property client: ObVecClient#: Provides access to underlying OceanBase vector database client.

delete(ids: List[str], **kwargs: Any) → None[source]#

Deletes a list of vectors identified by their IDs from the storage.

Parameters:

ids (List[str]) – List of unique identifiers for the vectors to be deleted.
**kwargs (Any) – Additional keyword arguments.

Raises:

RuntimeError – If there is an error during the deletion process.

load() → None[source]#: Load the collection hosted on cloud service.

query(query: VectorDBQuery, **kwargs: Any) → List[VectorDBQueryResult][source]#

Searches for similar vectors in the storage based on the provided query.

Parameters:

query (VectorDBQuery) – The query object containing the search vector and the number of top similar vectors to retrieve.
**kwargs (Any) – Additional keyword arguments.

Returns:

A list of vectors retrieved from the: storage based on similarity to the query vector.

Return type:

Raises:

RuntimeError – If there is an error during the query process.
ValueError – If the query vector dimension does not match the storage dimension.

status() → VectorDBStatus[source]#

Returns status of the vector database.

Returns:: The vector database status.
Return type:: VectorDBStatus

class camel.storages.vectordb_storages.QdrantStorage(vector_dim: int, collection_name: str | None = None, url_and_api_key: Tuple[str, str] | None = None, path: str | None = None, distance: VectorDistance = VectorDistance.COSINE, delete_collection_on_del: bool = False, **kwargs: Any)[source]#

An implementation of the BaseVectorStorage for interacting with Qdrant, a vector search engine.

The detailed information about Qdrant is available at: Qdrant

Parameters:

vector_dim (int) – The dimension of storing vectors.
collection_name (Optional[str], optional) – Name for the collection in the Qdrant. If not provided, set it to the current time with iso format. (default: None)
url_and_api_key (Optional[Tuple[str, str]], optional) – Tuple containing the URL and API key for connecting to a remote Qdrant instance. (default: None)
path (Optional[str], optional) – Path to a directory for initializing a local Qdrant client. (default: None)
distance (VectorDistance, optional) – The distance metric for vector comparison (default: VectorDistance.COSINE)
delete_collection_on_del (bool, optional) – Flag to determine if the collection should be deleted upon object destruction. (default: False)
**kwargs (Any) – Additional keyword arguments for initializing QdrantClient.

Notes

If url_and_api_key is provided, it takes priority and the client will attempt to connect to the remote Qdrant instance using the URL endpoint.
If url_and_api_key is not provided and path is given, the client will use the local path to initialize Qdrant.
If neither url_and_api_key nor path is provided, the client will be initialized with an in-memory storage (“:memory:”).

add(records: List[VectorRecord], **kwargs) → None[source]#

Adds a list of vectors to the specified collection.

Parameters:

vectors (List[VectorRecord]) – List of vectors to be added.
**kwargs (Any) – Additional keyword arguments.

Raises:

RuntimeError – If there was an error in the addition process.

clear() → None[source]#: Remove all vectors from the storage.

property client: QdrantClient#: Provides access to the underlying vector database client.

close_client(**kwargs)[source]#: Closes the client connection to the Qdrant storage.

delete(ids: List[str] | None = None, payload_filter: Dict[str, Any] | None = None, **kwargs: Any) → None[source]#

Deletes points from the collection based on either IDs or payload filters.

Parameters:

ids (Optional[List[str]], optional) – List of unique identifiers for the vectors to be deleted.
payload_filter (Optional[Dict[str, Any]], optional) – A filter for the payload to delete points matching specific conditions. If ids is provided, payload_filter will be ignored unless both are combined explicitly.
**kwargs (Any) – Additional keyword arguments pass to QdrantClient. delete.

Examples

>>> # Delete points with IDs "1", "2", and "3"
>>> storage.delete(ids=["1", "2", "3"])
>>> # Delete points with payload filter
>>> storage.delete(payload_filter={"name": "Alice"})

Raises:

ValueError – If neither ids nor payload_filter is provided.
RuntimeError – If there is an error during the deletion process.

Notes

If ids is provided, the points with these IDs will be deleted
directly, and the payload_filter will be ignored.
If ids is not provided but payload_filter is, then points
matching the payload_filter will be deleted.

delete_collection() → None[source]#: Deletes the entire collection in the Qdrant storage.

load() → None[source]#: Load the collection hosted on cloud service.

query(query: VectorDBQuery, filter_conditions: Dict[str, Any] | None = None, **kwargs: Any) → List[VectorDBQueryResult][source]#

Searches for similar vectors in the storage based on the provided query.

Parameters:

query (VectorDBQuery) – The query object containing the search vector and the number of top similar vectors to retrieve.
filter_conditions (Optional[Dict[str, Any]], optional) – A dictionary specifying conditions to filter the query results.
**kwargs (Any) – Additional keyword arguments.

Returns:

A list of vectors retrieved from the: storage based on similarity to the query vector.

Return type:

status() → VectorDBStatus[source]#

Returns status of the vector database.

Returns:: The vector database status.
Return type:: VectorDBStatus

update_payload(ids: List[str], payload: Dict[str, Any], **kwargs: Any) → None[source]#

Updates the payload of the vectors identified by their IDs.

Parameters:

ids (List[str]) – List of unique identifiers for the vectors to be updated.
payload (Dict[str, Any]) – List of payloads to be updated.
**kwargs (Any) – Additional keyword arguments.

Raises:

RuntimeError – If there is an error during the update process.

class camel.storages.vectordb_storages.TiDBStorage(vector_dim: int, collection_name: str | None = None, url_and_api_key: Tuple[str, str] | str | None = None, **kwargs: Any)[source]#

An implementation of the BaseVectorStorage for interacting with TiDB.

The detailed information about TiDB is available at: TiDB Vector Search

Parameters:

vector_dim (int) – The dimension of storing vectors.
url_and_api_key (Optional[Union[Tuple[str, str], str]]) –

A tuple
containing the database url and API key for connecting to a TiDB cluster. The URL should be in the format:

”mysql+pymysql://<username>:<password>@<host>:<port>/<db_name>”. TiDB will not use the API Key, but retains the definition for interface compatible.
collection_name (Optional[str]) – Name of the collection. The collection name will be used as the table name in TiDB. If not provided, set it to the current time with iso format.
**kwargs (Any) – Additional keyword arguments for initializing TiDB connection.

Raises:

ImportError – If pytidb package is not installed.

add(records: List[VectorRecord], **kwargs) → None[source]#

Adds a list of vectors to the specified table.

Parameters:

records (List[VectorRecord]) – List of vectors to be added.
**kwargs (Any) – Additional keyword arguments pass to insert.

Raises:

RuntimeError – If there was an error in the addition process.

clear() → None[source]#: Removes all vectors from the TiDB table. This method deletes the existing table and then recreates it with the same schema to effectively remove all stored vectors.

property client: TiDBClient#

Provides direct access to the TiDB client.

Returns:: The TiDB client instance.
Return type:: Any

delete(ids: List[str], **kwargs: Any) → None[source]#

Deletes a list of vectors identified by their IDs from the storage.

Parameters:

ids (List[str]) – List of unique identifiers for the vectors to be deleted.
**kwargs (Any) – Additional keyword arguments passed to delete.

Raises:

RuntimeError – If there is an error during the deletion process.

load() → None[source]#: Load the collection hosted on cloud service.

query(query: VectorDBQuery, **kwargs: Any) → List[VectorDBQueryResult][source]#

Searches for similar vectors in the storage based on the provided query.

Parameters:

query (VectorDBQuery) – The query object containing the search vector and the number of top similar vectors to retrieve.
**kwargs (Any) – Additional keyword arguments passed to search.

Returns:

A list of vectors retrieved from the: storage based on similarity to the query vector.

Return type:

status() → VectorDBStatus[source]#

Retrieves the current status of the TiDB table.

Returns:

An object containing information about the: table’s status.

Return type: