-
Notifications
You must be signed in to change notification settings - Fork 2.3k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge branch 'main' of https://github.com/langchain-ai/langchainjs in…
…to nc/31jul/runnable-abort-signal
- Loading branch information
Showing
85 changed files
with
7,845 additions
and
1,438 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -22,14 +22,21 @@ | |
| "zod": "npm:/zod", | ||
| "zod-to-json-schema": "npm:/zod-to-json-schema", | ||
| "node-llama-cpp": "npm:/node-llama-cpp", | ||
| "ml-distance": "npm:/ml-distance", | ||
| "pdf-parse": "npm:/pdf-parse", | ||
| "peggy": "npm:/peggy", | ||
| "readline": "https://deno.land/x/[email protected]/mod.ts", | ||
| "uuid": "npm:/uuid", | ||
| "youtubei.js": "npm:/youtubei.js", | ||
| "youtube-transcript": "npm:/youtube-transcript", | ||
| "neo4j-driver": "npm:/neo4j-driver", | ||
| "axios": "npm:/axios" | ||
| "axios": "npm:/axios", | ||
| "@mendable/firecrawl-js": "npm:/@mendable/firecrawl-js", | ||
| "@aws-crypto/sha256-js": "npm:/@aws-crypto/sha256-js", | ||
| "@aws-sdk/credential-provider-node": "npm:/@aws-sdk/credential-provider-node", | ||
| "@smithy/protocol-http": "npm:/@smithy/protocol-http", | ||
| "@smithy/signature-v4": "npm:/@smithy/signature-v4", | ||
| "@smithy/eventstream-codec": "npm:/@smithy/eventstream-codec", | ||
| "@smithy/util-utf8": "npm:/@smithy/util-utf8", | ||
| "@aws-sdk/types": "npm:/@aws-sdk/types" | ||
| } | ||
| } | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
226 changes: 226 additions & 0 deletions
226
docs/core_docs/docs/integrations/document_loaders/file_loaders/csv.ipynb
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,226 @@ | ||
| { | ||
| "cells": [ | ||
| { | ||
| "cell_type": "raw", | ||
| "metadata": { | ||
| "vscode": { | ||
| "languageId": "raw" | ||
| } | ||
| }, | ||
| "source": [ | ||
| "---\n", | ||
| "sidebar_label: CSV\n", | ||
| "sidebar_class_name: node-only\n", | ||
| "---" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "# CSVLoader\n", | ||
| "\n", | ||
| "```{=mdx}\n", | ||
| "\n", | ||
| ":::tip Compatibility\n", | ||
| "\n", | ||
| "Only available on Node.js.\n", | ||
| "\n", | ||
| ":::\n", | ||
| "\n", | ||
| "```\n", | ||
| "\n", | ||
| "This notebook provides a quick overview for getting started with `CSVLoader` [document loaders](/docs/concepts/#document-loaders). For detailed documentation of all `CSVLoader` features and configurations head to the [API reference](https://api.js.langchain.com/classes/langchain_community_document_loaders_fs_csv.CSVLoader.html).\n", | ||
| "\n", | ||
| "This example goes over how to load data from CSV files. The second argument is the `column` name to extract from the CSV file. One document will be created for each row in the CSV file. When `column` is not specified, each row is converted into a key/value pair with each key/value pair outputted to a new line in the document's `pageContent`. When `column` is specified, one document is created for each row, and the value of the specified column is used as the document's `pageContent`.\n", | ||
| "\n", | ||
| "## Overview\n", | ||
| "### Integration details\n", | ||
| "\n", | ||
| "| Class | Package | Compatibility | Local | [PY support](https://python.langchain.com/docs/integrations/document_loaders/csv)| \n", | ||
| "| :--- | :--- | :---: | :---: | :---: |\n", | ||
| "| [CSVLoader](https://api.js.langchain.com/classes/langchain_community_document_loaders_fs_csv.CSVLoader.html) | [@langchain/community](https://api.js.langchain.com/modules/langchain_community_document_loaders_fs_csv.html) | Node-only | ✅ | ✅ |\n", | ||
| "\n", | ||
| "## Setup\n", | ||
| "\n", | ||
| "To access `CSVLoader` document loader you'll need to install the `@langchain/community` integration, along with the `d3-dsv@2` peer dependency.\n", | ||
| "\n", | ||
| "### Installation\n", | ||
| "\n", | ||
| "The LangChain CSVLoader integration lives in the `@langchain/community` integration package.\n", | ||
| "\n", | ||
| "```{=mdx}\n", | ||
| "import IntegrationInstallTooltip from \"@mdx_components/integration_install_tooltip.mdx\";\n", | ||
| "import Npm2Yarn from \"@theme/Npm2Yarn\";\n", | ||
| "\n", | ||
| "<IntegrationInstallTooltip></IntegrationInstallTooltip>\n", | ||
| "\n", | ||
| "<Npm2Yarn>\n", | ||
| " @langchain/community d3-dsv@2\n", | ||
| "</Npm2Yarn>\n", | ||
| "\n", | ||
| "```" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "## Instantiation\n", | ||
| "\n", | ||
| "Now we can instantiate our model object and load documents:" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": 8, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "import { CSVLoader } from \"@langchain/community/document_loaders/fs/csv\"\n", | ||
| "\n", | ||
| "const exampleCsvPath = \"../../../../../../langchain/src/document_loaders/tests/example_data/example_separator.csv\";\n", | ||
| "\n", | ||
| "const loader = new CSVLoader(exampleCsvPath)" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "## Load" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": 6, | ||
| "metadata": {}, | ||
| "outputs": [ | ||
| { | ||
| "name": "stdout", | ||
| "output_type": "stream", | ||
| "text": [ | ||
| "Document {\n", | ||
| " pageContent: 'id｜html: 1｜\"<i>Corruption discovered at the core of the Banking Clan!</i>\"',\n", | ||
| " metadata: {\n", | ||
| " source: '../../../../../../langchain/src/document_loaders/tests/example_data/example_separator.csv',\n", | ||
| " line: 1\n", | ||
| " },\n", | ||
| " id: undefined\n", | ||
| "}\n" | ||
| ] | ||
| } | ||
| ], | ||
| "source": [ | ||
| "const docs = await loader.load()\n", | ||
| "docs[0]" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": 7, | ||
| "metadata": {}, | ||
| "outputs": [ | ||
| { | ||
| "name": "stdout", | ||
| "output_type": "stream", | ||
| "text": [ | ||
| "{\n", | ||
| " source: '../../../../../../langchain/src/document_loaders/tests/example_data/example_separator.csv',\n", | ||
| " line: 1\n", | ||
| "}\n" | ||
| ] | ||
| } | ||
| ], | ||
| "source": [ | ||
| "console.log(docs[0].metadata)" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "## Usage, extracting a single column\n", | ||
| "\n", | ||
| "Example CSV file:\n", | ||
| "\n", | ||
| "```csv\n", | ||
| "id｜html\n", | ||
| "1｜\"<i>Corruption discovered at the core of the Banking Clan!</i>\"\n", | ||
| "2｜\"<i>Reunited, Rush Clovis and Senator Amidala</i>\"\n", | ||
| "3｜\"<i>discover the full extent of the deception.</i>\"\n", | ||
| "4｜\"<i>Anakin Skywalker is sent to the rescue!</i>\"\n", | ||
| "```" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": 14, | ||
| "metadata": {}, | ||
| "outputs": [ | ||
| { | ||
| "name": "stdout", | ||
| "output_type": "stream", | ||
| "text": [ | ||
| "Document {\n", | ||
| " pageContent: '<i>Corruption discovered at the core of the Banking Clan!</i>',\n", | ||
| " metadata: {\n", | ||
| " source: '../../../../../../langchain/src/document_loaders/tests/example_data/example_separator.csv',\n", | ||
| " line: 1\n", | ||
| " },\n", | ||
| " id: undefined\n", | ||
| "}\n" | ||
| ] | ||
| } | ||
| ], | ||
| "source": [ | ||
| "import { CSVLoader } from \"@langchain/community/document_loaders/fs/csv\";\n", | ||
| "\n", | ||
| "const singleColumnLoader = new CSVLoader(\n", | ||
| " exampleCsvPath,\n", | ||
| " {\n", | ||
| " column: \"html\",\n", | ||
| " separator:\"｜\"\n", | ||
| " }\n", | ||
| ");\n", | ||
| "\n", | ||
| "const singleColumnDocs = await singleColumnLoader.load();\n", | ||
| "console.log(singleColumnDocs[0]);" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "## API reference\n", | ||
| "\n", | ||
| "For detailed documentation of all CSVLoader features and configurations head to the API reference: https://api.js.langchain.com/classes/langchain_community_document_loaders_fs_csv.CSVLoader.html" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [] | ||
| } | ||
| ], | ||
| "metadata": { | ||
| "kernelspec": { | ||
| "display_name": "TypeScript", | ||
| "language": "typescript", | ||
| "name": "tslab" | ||
| }, | ||
| "language_info": { | ||
| "codemirror_mode": { | ||
| "mode": "typescript", | ||
| "name": "javascript", | ||
| "typescript": true | ||
| }, | ||
| "file_extension": ".ts", | ||
| "mimetype": "text/typescript", | ||
| "name": "typescript", | ||
| "version": "3.7.2" | ||
| } | ||
| }, | ||
| "nbformat": 4, | ||
| "nbformat_minor": 4 | ||
| } |
Oops, something went wrong.