langchain_community.document_loaders.airbyte
.AirbyteCDKLoader¶
- class langchain_community.document_loaders.airbyte.AirbyteCDKLoader(config: Mapping[str, Any], source_class: Any, stream_name: str, record_handler: Optional[Callable[[Any, Optional[str]], Document]] = None, state: Optional[Any] = None)[source]¶
Load with an Airbyte source connector implemented using the CDK.
Initializes the loader.
- Parameters
config (Mapping[str, Any]) – The config to pass to the source connector.
source_class (Any) – The source connector class.
stream_name (str) – The name of the stream to load.
record_handler (Optional[Callable[[Any, Optional[str]], Document]]) – A function that takes in a record and an optional id and returns a Document. If None, the record will be used as the document. Defaults to None.
state (Optional[Any]) – The state to pass to the source connector. Defaults to None.
Attributes
last_state
Methods
__init__
(config, source_class, stream_name)Initializes the loader.
A lazy loader for Documents.
aload
()Load data into Document objects.
A lazy loader for Documents.
load
()Load data into Document objects.
load_and_split
([text_splitter])Load Documents and split into chunks.
- __init__(config: Mapping[str, Any], source_class: Any, stream_name: str, record_handler: Optional[Callable[[Any, Optional[str]], Document]] = None, state: Optional[Any] = None) None [source]¶
Initializes the loader.
- Parameters
config (Mapping[str, Any]) – The config to pass to the source connector.
source_class (Any) – The source connector class.
stream_name (str) – The name of the stream to load.
record_handler (Optional[Callable[[Any, Optional[str]], Document]]) – A function that takes in a record and an optional id and returns a Document. If None, the record will be used as the document. Defaults to None.
state (Optional[Any]) – The state to pass to the source connector. Defaults to None.
- Return type
None
- async alazy_load() AsyncIterator[Document] ¶
A lazy loader for Documents.
- Return type
AsyncIterator[Document]
- lazy_load() Iterator[Document] [source]¶
A lazy loader for Documents.
- Return type
Iterator[Document]
- load_and_split(text_splitter: Optional[TextSplitter] = None) List[Document] ¶
Load Documents and split into chunks. Chunks are returned as Documents.
Do not override this method. It should be considered to be deprecated!
- Parameters
text_splitter (Optional[TextSplitter]) – TextSplitter instance to use for splitting documents. Defaults to RecursiveCharacterTextSplitter.
- Returns
List of Documents.
- Return type
List[Document]