langchain_community.document_loaders.text.TextLoader

class langchain_community.document_loaders.text.TextLoader(file_path: Union[str, Path], encoding: Optional[str] = None, autodetect_encoding: bool = False)

Load text file.

Parameters

• file_path (Union[str, Path]) – Path to the file to load.

• encoding (Optional[str]) – File encoding to use. If None, the file will be loaded

• encoding. (with the default system) –

• autodetect_encoding (bool) – Whether to try to autodetect the file encoding if the specified encoding fails.

Initialize with file path.

Methods

__init__(file_path[, encoding, ...])	Initialize with file path.
alazy_load()	A lazy loader for Documents.
aload()	Load data into Document objects.
lazy_load()	Load from file path.
load()	Load data into Document objects.
load_and_split([text_splitter])	Load Documents and split into chunks.

__init__(file_path: Union[str, Path], encoding: Optional[str] = None, autodetect_encoding: bool = False)

Initialize with file path.

Parameters

• file_path (Union[str, Path]) –

• encoding (Optional[str]) –

• autodetect_encoding (bool) –

async alazy_load() → AsyncIterator[Document]

A lazy loader for Documents.

Return type

AsyncIterator[Document]

async aload() → List[Document]

Load data into Document objects.

Return type

List[Document]

lazy_load() → Iterator[Document]

Load from file path.

Return type

Iterator[Document]

load() → List[Document]

Load data into Document objects.

Return type

List[Document]

load_and_split(text_splitter: Optional[TextSplitter] = None) → List[Document]

Load Documents and split into chunks. Chunks are returned as Documents.

Do not override this method. It should be considered to be deprecated!

Parameters

text_splitter (Optional[TextSplitter]) – TextSplitter instance to use for splitting documents. Defaults to RecursiveCharacterTextSplitter.

Returns

List of Documents.

Return type

List[Document]

Examples using TextLoader

• # Automatically restart kernel after installs so that your environment can access the new packages

• # if you plan to use bson serialization, install also:

• %pip install –upgrade –quiet surrealdb langchain langchain-community

• 1. Vanilla RAG {#vanilla-rag-1}

• Create a bes instance and index docs.

• Create collection if running for the first time. If the collection

• Dependencies {#dependencies}

• Document loaders

• DocumentDB connection string

• Establishing a connection to the database is facilitated through the singlestoredb Python connector.

• File Directory

• Get an OpenAI token: https://platform.openai.com/account/api-keys

• Helper function for printing docs

• Here we useimport getpass

• Here we want to make sure the answer is minimally relevant

• If using the default Docker installation, use this instantiation instead:

• Load in document to retrieve over

• Load the document and split it into chunks

• Metadata {#metadata}

• OR

• OR (depending on Python version)

• Option 1: use an OpenAI account

• Pip install necessary package

• Pip install necessary package {#pip-install-necessary-package}

• Pip install necessary packages

• Refresh is required for server use

• Retrievers

• Run tests with shell:

• Set up the OpenAI Environment Variables

• Setup {#setup}

• Swapping the `ByteStore` {#swapping-the-bytestore}

• The vectorstore to use to index the child chunks

• This text splitter is used to create the child documents

• This will only get documents for Ankush

• Use Meilisearch vector store to store texts & associated embeddings as vector

• VDMS

• Vector stores

• Wait until the cluster is ready for use.

• You need to install sqlite-vss as a dependency.

• add texts

• analyticdb.md

• atlas.md

• awadb.md

• azure_ai_search.md

• baiduvectordb.md

• clickhouse.md

• conversational_retrieval_agents.md

• cosine: distance metric

• create cluster and add texts

• databricks_vector_search.md

• default metric is angular

• docs[0].metadata[“id”] == “id:testapp:testapp::32”

• drop first if index already exists

• duckdb.md

• ecloud_vector_search.md

• epsilla.md

• for example

• from langchain_community.embeddings.openai import OpenAIEmbeddings

• hologres.md

• import

• initialize marqo

• install package

• lancedb.md

• openai

• or install latest:

• or shorter

• os.environ[“LANGCHAIN_TRACING_V2”] = “true”

• output length: 4

• pinecone.md

• qdrant.md

• replace

• scann.md

• tigris.md

• tiledb.md

• typesense.md

• use directly a `where_str` to delete

• usearch.md

• vectorstore.md

• vikingdb.md

• with pip

• xata.md