langchain_community.embeddings.llamacpp.LlamaCppEmbeddings

class langchain_community.embeddings.llamacpp.LlamaCppEmbeddings

Bases: BaseModel, Embeddings

llama.cpp embedding models.

To use, you should have the llama-cpp-python library installed, and provide the path to the Llama model as a named parameter to the constructor. Check out: https://github.com/abetlen/llama-cpp-python

Example

from langchain_community.embeddings import LlamaCppEmbeddings
llama = LlamaCppEmbeddings(model_path="/path/to/model.bin")

Create a new model by parsing and validating input data from keyword arguments.

Raises ValidationError if the input data cannot be parsed to form a valid model.

param device: Optional[str] = None

Device type to use and pass to the model

param f16_kv: bool = False

Use half-precision for key/value cache.

param logits_all: bool = False

Return logits for all tokens, not just the last token.

param model_path: str [Required]

param n_batch: Optional[int] = 512

Number of tokens to process in parallel. Should be a number between 1 and n_ctx.

param n_ctx: int = 512

Token context window.

param n_gpu_layers: Optional[int] = None

Number of layers to be loaded into gpu memory. Default None.

param n_parts: int = -1

Number of parts to split the model into. If -1, the number of parts is automatically determined.

param n_threads: Optional[int] = None

Number of threads to use. If None, the number of threads is automatically determined.

param seed: int = -1

Seed. If -1, a random seed is used.

param use_mlock: bool = False

Force system to keep model in RAM.

param verbose: bool = True

Print verbose output to stderr.

param vocab_only: bool = False

Only load the vocabulary, no weights.

async aembed_documents(texts: List[str]) → List[List[float]]

Asynchronous Embed search docs.

Parameters

texts (List[str]) – List of text to embed.

Returns

List of embeddings.

Return type

List[List[float]]

async aembed_query(text: str) → List[float]

Asynchronous Embed query text.

Parameters

text (str) – Text to embed.

Returns

Embedding.

Return type

List[float]

embed_documents(texts: List[str]) → List[List[float]]

Embed a list of documents using the Llama model.

Parameters

texts (List[str]) – The list of texts to embed.

Returns

List of embeddings, one for each text.

Return type

List[List[float]]

embed_query(text: str) → List[float]

Embed a query using the Llama model.

Parameters

text (str) – The text to embed.

Returns

Embeddings for the text.

Return type

List[float]

Examples using LlamaCppEmbeddings

• Llama-cpp

• Llama.cpp