Skip to content

Commit

Permalink
Added docs and reformatted the code
Browse files Browse the repository at this point in the history
  • Loading branch information
Minjun Kim authored and AhmadHakim2004 committed Nov 30, 2024
1 parent bd1d11f commit 804a7db
Show file tree
Hide file tree
Showing 11 changed files with 766 additions and 736 deletions.
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
---
hide_table_of_contents: true
---

# Oracle AI

This example goes over how to load documents using Oracle AI Vector Search.

## Setup

You'll need to install the [oracledb](https://www.npmjs.com/package/oracledb) package:

```bash npm2yarn
npm install @langchain/community @langchain/core oracledb
```

## Usage

### Connect to Oracle Database
You'll need to provide the username, password, hostname and service_name:

```typescript
import oracledb from 'oracledb';

let connection: oracledb.Connection;

// Replace the placeholders with your information
const username = "<username>";
const password = "<password>";
const dsn = "<hostname>/<service_name>";

try {
connection = await oracledb.getConnection({
user: username,
password:password,
connectString: dsn
});
console.log("Connection Successful");
} catch (err) {
console.error('Connection failed:', err);
throw err;
}
```

### Load Documents
As for loading documents, you have 3 options:
- Loading a local file.
- Loading from a local directory.
- Loading from the Oracle Database.

When loading from the Oracle Database, you must provide the table's name, owner's name, and the name of the column to load. Optionally, you can provide extra column names to be included in the returned documents' metadata:

```typescript
import { OracleDocLoader, OracleLoadFromType } from "@langchain/community/document_loaders/web/oracleai";

/*
// Loading a local file (replace <filepath> with the path of the file you want to load.)
const loader = new OracleDocLoader(connection, <filepath>, OracleLoadFromType.FILE);
// Loading from a local directory (replace <dirpath> with the path of the directory you want to load from.)
const loader = new OracleDocLoader(connection, <dirpath>, OracleLoadFromType.DIR);
*/

// Loading from Oracle Database table (replace the placeholders with your information, optionally add a [metadata_cols] parameter to include columns as metadata.)
const loader = new OracleDocLoader(connection, <tablename>, OracleLoadFromType.TABLE, <owner_name>, <colname>);

// Load the docs
const docs = loader.load();
console.log("Number of docs loaded:", docs.length);
console.log("Document-0:", docs[0].page_content); // content
```

40 changes: 40 additions & 0 deletions examples/src/document_loaders/oracleai.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
import oracledb from 'oracledb';
import { OracleDocLoader, OracleLoadFromType } from "@langchain/community/document_loaders/web/oracleai";

let connection: oracledb.Connection;

// Replace the placeholders with your information
const username = "<username>";
const pwd = "<password>";
const dsn = "<hostname>/<service_name>";

try {
connection = await oracledb.getConnection({
user: username,
password: pwd,
connectString: dsn
});
console.log("Connection Successful");
} catch (err) {
console.error('Connection failed:', err);
throw err;
}

// Loading a local file (replace <filepath> with the path of the file you want to load.)
const loader = new OracleDocLoader(connection, "src/document_loaders/example_data/bitcoin.pdf", OracleLoadFromType.FILE);

/*
// Loading from a local directory (replace <dirpath> with the path of the directory you want to load from.)
const loader = new OracleDocLoader(connection, <dirpath>, OracleLoadFromType.DIR);
// Loading from Oracle Database table (replace the placeholders with your information, optionally add a [metadata_cols] parameter to include columns as metadata.)
const loader = new OracleDocLoader(connection, <tablename>, OracleLoadFromType.TABLE, <owner_name>, <colname>);
*/

// Load the docs
const docs = loader.load();
console.log("Number of docs loaded:", docs.length);
console.log("Document-0:", docs[0].page_content); // content


4 changes: 4 additions & 0 deletions libs/langchain-community/.gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -922,6 +922,10 @@ document_loaders/web/notionapi.cjs
document_loaders/web/notionapi.js
document_loaders/web/notionapi.d.ts
document_loaders/web/notionapi.d.cts
document_loaders/web/oracleai.cjs
document_loaders/web/oracleai.js
document_loaders/web/oracleai.d.ts
document_loaders/web/oracleai.d.cts
document_loaders/web/pdf.cjs
document_loaders/web/pdf.js
document_loaders/web/pdf.d.ts
Expand Down
2 changes: 2 additions & 0 deletions libs/langchain-community/langchain.config.js
Original file line number Diff line number Diff line change
Expand Up @@ -286,6 +286,7 @@ export const config = {
"document_loaders/web/github": "document_loaders/web/github",
"document_loaders/web/taskade": "document_loaders/web/taskade",
"document_loaders/web/notionapi": "document_loaders/web/notionapi",
"document_loaders/web/oracleai": "document_loaders/web/oracleai",
"document_loaders/web/pdf": "document_loaders/web/pdf",
"document_loaders/web/recursive_url": "document_loaders/web/recursive_url",
"document_loaders/web/s3": "document_loaders/web/s3",
Expand Down Expand Up @@ -505,6 +506,7 @@ export const config = {
"document_loaders/web/pdf",
"document_loaders/web/taskade",
"document_loaders/web/notionapi",
"document_loaders/web/oracleai",
"document_loaders/web/recursive_url",
"document_loaders/web/s3",
"document_loaders/web/sitemap",
Expand Down
21 changes: 18 additions & 3 deletions libs/langchain-community/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -39,9 +39,11 @@
"binary-extensions": "^2.2.0",
"expr-eval": "^2.0.2",
"flat": "^5.0.2",
"htmlparser2": "^9.1.0",
"js-yaml": "^4.1.0",
"langchain": ">=0.2.3 <0.3.0 || >=0.3.4 <0.4.0",
"langsmith": "^0.2.0",
"oracledb": "^6.7.0",
"uuid": "^10.0.0",
"zod": "^3.22.3",
"zod-to-json-schema": "^3.22.5"
Expand Down Expand Up @@ -116,11 +118,11 @@
"@types/d3-dsv": "^3.0.7",
"@types/flat": "^5.0.2",
"@types/html-to-text": "^9",
"@types/jest": "^29.5.14",
"@types/jsdom": "^21.1.1",
"@types/jsonwebtoken": "^9",
"@types/lodash": "^4",
"@types/mozilla-readability": "^0.2.1",
"@types/oracledb": "^6",
"@types/pdf-parse": "^1.1.1",
"@types/pg": "^8.11.0",
"@types/pg-copy-streams": "^1.2.2",
Expand Down Expand Up @@ -179,7 +181,7 @@
"interface-datastore": "^8.2.11",
"ioredis": "^5.3.2",
"it-all": "^3.0.4",
"jest": "^29.7.0",
"jest": "^29.5.0",
"jest-environment-node": "^29.6.4",
"jsdom": "^22.1.0",
"jsonwebtoken": "^9.0.2",
Expand Down Expand Up @@ -209,7 +211,7 @@
"rollup": "^3.19.1",
"sonix-speech-recognition": "^2.1.1",
"srt-parser-2": "^1.2.3",
"ts-jest": "^29.2.5",
"ts-jest": "^29.1.0",
"typeorm": "^0.3.20",
"typescript": "~5.1.6",
"typesense": "^1.5.3",
Expand Down Expand Up @@ -2792,6 +2794,15 @@
"import": "./document_loaders/web/notionapi.js",
"require": "./document_loaders/web/notionapi.cjs"
},
"./document_loaders/web/oracleai": {
"types": {
"import": "./document_loaders/web/oracleai.d.ts",
"require": "./document_loaders/web/oracleai.d.cts",
"default": "./document_loaders/web/oracleai.d.ts"
},
"import": "./document_loaders/web/oracleai.js",
"require": "./document_loaders/web/oracleai.cjs"
},
"./document_loaders/web/pdf": {
"types": {
"import": "./document_loaders/web/pdf.d.ts",
Expand Down Expand Up @@ -4026,6 +4037,10 @@
"document_loaders/web/notionapi.js",
"document_loaders/web/notionapi.d.ts",
"document_loaders/web/notionapi.d.cts",
"document_loaders/web/oracleai.cjs",
"document_loaders/web/oracleai.js",
"document_loaders/web/oracleai.d.ts",
"document_loaders/web/oracleai.d.cts",
"document_loaders/web/pdf.cjs",
"document_loaders/web/pdf.js",
"document_loaders/web/pdf.d.ts",
Expand Down
Original file line number Diff line number Diff line change
@@ -1,25 +1,28 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="keywords">
<meta>
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="keywords" />
<meta />
<title>Sample HTML Page</title>
</head>
<body>
</head>
<body>
<header>
<h1>Welcome to My Sample HTML Page</h1>
<h1>Welcome to My Sample HTML Page</h1>
</header>

<main>
<h2>Introduction</h2>
<p>This is a small HTML file with a header, main content section, and a footer.</p>
<p>Feel free to modify and experiment with the code!</p>
<h2>Introduction</h2>
<p>
This is a small HTML file with a header, main content section, and a
footer.
</p>
<p>Feel free to modify and experiment with the code!</p>
</main>

<footer>
<p>Footer Content - &copy; 2024</p>
<p>Footer Content - &copy; 2024</p>
</footer>
</body>
</html>
</body>
</html>
Loading

0 comments on commit 804a7db

Please sign in to comment.