langchain_community.document_loaders.unstructured
.UnstructuredAPIFileIOLoader¶
- class langchain_community.document_loaders.unstructured.UnstructuredAPIFileIOLoader(file: Union[IO[bytes], Sequence[IO[bytes]]], *, mode: str = 'single', url: str = 'https://api.unstructuredapp.io/general/v0/general', api_key: str = '', **unstructured_kwargs: Any)[source]¶
Deprecated since version 0.2.8: Use
langchain_unstructured.UnstructuredLoader
instead.Send file-like objects with unstructured-client sdk to the Unstructured API.
By default, the loader makes a call to the hosted Unstructured API. If you are running the unstructured API locally, you can change the API rule by passing in the url parameter when you initialize the loader. The hosted Unstructured API requires an API key. See the links below to learn more about our API offerings and get an API key.
You can run the loader in different modes: “single”, “elements”, and “paged”. The default “single” mode will return a single langchain Document object. If you use “elements” mode, the unstructured library will split the document into elements such as Title and NarrativeText and return those as individual langchain Document objects. In addition to these post-processing modes (which are specific to the LangChain Loaders), Unstructured has its own “chunking” parameters for post-processing elements into more useful chunks for uses cases such as Retrieval Augmented Generation (RAG). You can pass in additional unstructured kwargs to configure different unstructured settings.
Examples
from langchain_community.document_loaders import UnstructuredAPIFileLoader
- with open(“example.pdf”, “rb”) as f:
- loader = UnstructuredAPIFileIOLoader(
f, mode=”elements”, strategy=”fast”, api_key=”MY_API_KEY”,
) docs = loader.load()
References
https://docs.unstructured.io/api-reference/api-services/sdk https://docs.unstructured.io/api-reference/api-services/overview https://docs.unstructured.io/open-source/core-functionality/partitioning https://docs.unstructured.io/open-source/core-functionality/chunking
Initialize with file path.
Methods
__init__
(file, *[, mode, url, api_key])Initialize with file path.
A lazy loader for Documents.
aload
()Load data into Document objects.
Load file.
load
()Load data into Document objects.
load_and_split
([text_splitter])Load Documents and split into chunks.
- Parameters
file (Union[IO[bytes], Sequence[IO[bytes]]]) –
mode (str) –
url (str) –
api_key (str) –
unstructured_kwargs (Any) –
- __init__(file: Union[IO[bytes], Sequence[IO[bytes]]], *, mode: str = 'single', url: str = 'https://api.unstructuredapp.io/general/v0/general', api_key: str = '', **unstructured_kwargs: Any)[source]¶
Initialize with file path.
- Parameters
file (Union[IO[bytes], Sequence[IO[bytes]]]) –
mode (str) –
url (str) –
api_key (str) –
unstructured_kwargs (Any) –
- async alazy_load() AsyncIterator[Document] ¶
A lazy loader for Documents.
- Return type
AsyncIterator[Document]
- load_and_split(text_splitter: Optional[TextSplitter] = None) List[Document] ¶
Load Documents and split into chunks. Chunks are returned as Documents.
Do not override this method. It should be considered to be deprecated!
- Parameters
text_splitter (Optional[TextSplitter]) – TextSplitter instance to use for splitting documents. Defaults to RecursiveCharacterTextSplitter.
- Returns
List of Documents.
- Return type
List[Document]