langchain.cache.MomentoCache

class langchain.cache.MomentoCache(cache_client: momento.CacheClient, cache_name: str, *, ttl: Optional[timedelta] = None, ensure_cache_exists: bool = True)[source]

Cache that uses Momento as a backend. See https://gomomento.com/

Instantiate a prompt cache using Momento as a backend.

Note: to instantiate the cache client passed to MomentoCache, you must have a Momento account. See https://gomomento.com/.

Parameters
  • cache_client (CacheClient) – The Momento cache client.

  • cache_name (str) – The name of the cache to use to store the data.

  • ttl (Optional[timedelta], optional) – The time to live for the cache items. Defaults to None, ie use the client default TTL.

  • ensure_cache_exists (bool, optional) – Create the cache if it doesn’t exist. Defaults to True.

Raises
  • ImportError – Momento python package is not installed.

  • TypeError – cache_client is not of type momento.CacheClientObject

  • ValueError – ttl is non-null and non-negative

Methods

__init__(cache_client, cache_name, *[, ttl, ...])

Instantiate a prompt cache using Momento as a backend.

clear(**kwargs)

Clear the cache.

from_client_params(cache_name, ttl, *[, ...])

Construct cache from CacheClient parameters.

lookup(prompt, llm_string)

Lookup llm generations in cache by prompt and associated model and settings.

update(prompt, llm_string, return_val)

Store llm generations in cache.

__init__(cache_client: momento.CacheClient, cache_name: str, *, ttl: Optional[timedelta] = None, ensure_cache_exists: bool = True)[source]

Instantiate a prompt cache using Momento as a backend.

Note: to instantiate the cache client passed to MomentoCache, you must have a Momento account. See https://gomomento.com/.

Parameters
  • cache_client (CacheClient) – The Momento cache client.

  • cache_name (str) – The name of the cache to use to store the data.

  • ttl (Optional[timedelta], optional) – The time to live for the cache items. Defaults to None, ie use the client default TTL.

  • ensure_cache_exists (bool, optional) – Create the cache if it doesn’t exist. Defaults to True.

Raises
  • ImportError – Momento python package is not installed.

  • TypeError – cache_client is not of type momento.CacheClientObject

  • ValueError – ttl is non-null and non-negative

clear(**kwargs: Any) None[source]

Clear the cache.

Raises

SdkException – Momento service or network error

classmethod from_client_params(cache_name: str, ttl: timedelta, *, configuration: Optional[momento.config.Configuration] = None, api_key: Optional[str] = None, auth_token: Optional[str] = None, **kwargs: Any) MomentoCache[source]

Construct cache from CacheClient parameters.

lookup(prompt: str, llm_string: str) Optional[Sequence[Generation]][source]

Lookup llm generations in cache by prompt and associated model and settings.

Parameters
  • prompt (str) – The prompt run through the language model.

  • llm_string (str) – The language model version and settings.

Raises

SdkException – Momento service or network error

Returns

A list of language model generations.

Return type

Optional[RETURN_VAL_TYPE]

update(prompt: str, llm_string: str, return_val: Sequence[Generation]) None[source]

Store llm generations in cache.

Parameters
  • prompt (str) – The prompt run through the language model.

  • llm_string (str) – The language model string.

  • return_val (RETURN_VAL_TYPE) – A list of language model generations.

Raises
  • SdkException – Momento service or network error

  • Exception – Unexpected response

Examples using MomentoCache