Class RecursiveCharacterTextSplitter

Hierarchy (view full)

TextSplitter
- RecursiveCharacterTextSplitter
  - MarkdownTextSplitter
  - LatexTextSplitter

Implements

RecursiveCharacterTextSplitterParams

Index

Constructors

Properties

chunkOverlap chunkSize keepSeparator lengthFunction separators

Methods

createDocuments mergeSplits splitDocuments splitText transformDocuments fromLanguage getSeparatorsForLanguage splitOnSeparator

Constructors

constructor

new RecursiveCharacterTextSplitter(fields?): RecursiveCharacterTextSplitter
Parameters
- Optional fields: Partial<RecursiveCharacterTextSplitterParams>
Returns RecursiveCharacterTextSplitter
Overrides TextSplitter.constructor
- Defined in libs/langchain-textsplitters/src/text_splitter.ts:293

Properties

chunkOverlap

chunkOverlap: number = 200

chunkSize

chunkSize: number = 1000

keepSeparator

keepSeparator: boolean = false

lengthFunction

lengthFunction: ((text) => number) | ((text) => Promise<number>)

Type declaration

- (text): number
- Parameters
  - text: string
  Returns number

Type declaration

- (text): Promise<number>
- Parameters
  - text: string
  Returns Promise<number>

separators

separators: string[] = ...

Methods

createDocuments

createDocuments(texts, metadatas?, chunkHeaderOptions?): Promise<Document<Record<string, any>>[]>
Parameters
- texts: string[]
- metadatas: Record<string, any>[] = []
- chunkHeaderOptions: TextSplitterChunkHeaderOptions = {}
Returns Promise<Document<Record<string, any>>[]>
Inherited from TextSplitter.createDocuments
- Defined in libs/langchain-textsplitters/src/text_splitter.ts:75

mergeSplits

mergeSplits(splits, separator): Promise<string[]>
Parameters
- splits: string[]
- separator: string
Returns Promise<string[]>
Inherited from TextSplitter.mergeSplits
- Defined in libs/langchain-textsplitters/src/text_splitter.ts:184

splitDocuments

splitDocuments(documents, chunkHeaderOptions?): Promise<Document<Record<string, any>>[]>
Parameters
- documents: Document<Record<string, any>>[]
- chunkHeaderOptions: TextSplitterChunkHeaderOptions = {}
Returns Promise<Document<Record<string, any>>[]>
Inherited from TextSplitter.splitDocuments
- Defined in libs/langchain-textsplitters/src/text_splitter.ts:167

splitText

splitText(text): Promise<string[]>
Parameters
- text: string
Returns Promise<string[]>
Overrides TextSplitter.splitText
- Defined in libs/langchain-textsplitters/src/text_splitter.ts:348

transformDocuments

transformDocuments(documents, chunkHeaderOptions?): Promise<Document<Record<string, any>>[]>
Parameters
- documents: Document<Record<string, any>>[]
- chunkHeaderOptions: TextSplitterChunkHeaderOptions = {}
Returns Promise<Document<Record<string, any>>[]>
Inherited from TextSplitter.transformDocuments
- Defined in libs/langchain-textsplitters/src/text_splitter.ts:48

`Static` fromLanguage

fromLanguage(language, options?): RecursiveCharacterTextSplitter
Parameters
- language: "html" | "cpp" | "go" | "java" | "js" | "php" | "proto" | "python" | "rst" | "ruby" | "rust" | "scala" | "swift" | "markdown" | "latex" | "sol"
- Optional options: Partial<RecursiveCharacterTextSplitterParams>
Returns RecursiveCharacterTextSplitter
- Defined in libs/langchain-textsplitters/src/text_splitter.ts:352

`Static` getSeparatorsForLanguage

getSeparatorsForLanguage(language): string[]
Parameters
- language: "html" | "cpp" | "go" | "java" | "js" | "php" | "proto" | "python" | "rst" | "ruby" | "rust" | "scala" | "swift" | "markdown" | "latex" | "sol"
Returns string[]
- Defined in libs/langchain-textsplitters/src/text_splitter.ts:363

`Protected` splitOnSeparator

splitOnSeparator(text, separator): string[]
Parameters
- text: string
- separator: string
Returns string[]
Inherited from TextSplitter.splitOnSeparator
- Defined in libs/langchain-textsplitters/src/text_splitter.ts:57

Generated using TypeDoc