langchain_community.utilities.pebblo.generate_size_based_batches¶

langchain_community.utilities.pebblo.generate_size_based_batches(docs: List[Document], max_batch_size: int = 102400) → List[List[Document]][source]¶

Generate batches of documents based on page_content size. :param docs: List of documents to be batched. :param max_batch_size: Maximum size of each batch in bytes. Defaults to 100*1024(100KB)

Returns

List of batches of documents

Return type

List[List[Document]]

Parameters

  • docs (List[Document]) –

  • max_batch_size (int) –