Merge remote-tracking branch 'origin/AYSZ' into AYSL

applications/node/lanchain_writing_assistant/README.md

@@ -1,13 +1,132 @@
-# writing assistant using lanchain.js and LanceDB
+**AI Writing Assistant with LangChain.js and LanceDB**
+======================================================
 
-![](https://github.com/lancedb/assets/blob/main/recipes/demo_vid_langchain_integration-ezgif.com-video-to-gif-converter.gif)
+This project demonstrates an **AI-powered writing assistant** built using **LangChain.js** and **LanceDB**. The assistant utilizes **OpenAI's API** for language generation and LanceDB for efficient vector storage and retrieval, making it both scalable and customizable.
 
-# add your open_ai_key (run this command in your terminal)
- export OPENAI_API_KEY= your_key
+* * *
 
-# command to run on your local machine
+**Features**
+------------
 
-npm run dev
-# To use your own data. go to src>Backend>dataSourceFiles and replace the .txt file
+*   🔍 **Customizable Data**: Replace the default dataset with your own text files to tailor the assistant for domain-specific applications.
+*   ⚡ **Efficient Retrieval**: Uses LanceDB to ensure low-latency and high-accuracy content retrieval.
+*   🌐 **Scalable Architecture**: Built with modular components, making it easy to extend functionality or integrate with other tools.
 
-Open [http://localhost:3000](http://localhost:3000) with your browser to see the result.
+* * *
+
+**Demo**
+--------
+
+![LangChain.js + LanceDB Demo](https://github.com/lancedb/assets/blob/main/recipes/demo_vid_langchain_integration-ezgif.com-video-to-gif-converter.gif)
+
+* * *
+
+**Getting Started**
+-------------------
+
+### **1\. Prerequisites**
+
+Ensure the following are installed:
+
+*   Node.js (v20 or above)
+*   NPM or Yarn
+*   A valid [OpenAI API Key](https://platform.openai.com/signup)
+
+* * *
+
+### **2\. Installation**
+
+Clone the repository and install the required dependencies:
+
+`git clone`
+
+`cd`
+
+`npm install` 
+
+* * *
+
+### **3\. Configuration**
+
+Set your OpenAI API key in your environment:
+
+`export OPENAI_API_KEY=your_openai_key` 
+
+Alternatively, create a `.env` file in the project root:
+
+
+`OPENAI_API_KEY=your_openai_key` 
+
+* * *
+
+### **4\. Customizing the Dataset**
+
+To use your own data:
+
+1.  Navigate to the directory:
+
+
+    `src/Backend/dataSourceFiles` 
+
+2.  Replace the default `.txt` file with your own text file.
+3.  Restart the server to apply changes.
+
+* * *
+
+### **5\. Running the Application**
+
+Start the development server:
+
+`npm run dev` 
+
+Access the app in your browser at:
+
+`http://localhost:3000` 
+
+* * *
+
+**Technical Details**
+---------------------
+
+### **How It Works**
+
+1.  **Data Embedding**
+
+    *   Text data from the `.txt` file is processed and converted into embeddings using OpenAI's embedding models.
+2.  **Vector Storage and Retrieval**
+
+    *   LanceDB stores and indexes the embeddings for fast similarity-based searches.
+    *   When a user inputs a query, the system retrieves relevant text segments based on vector similarity.
+3.  **Response Generation**
+
+    *   LangChain.js orchestrates the interaction with OpenAI's API, using the retrieved context to generate meaningful responses.
+
+* * *
+
+**Commands**
+------------
+
+*   **Start the App**:
+
+    `npm run dev` 
+
+*   **Start the server only**:
+
+    `npm run server`
+
+*   **Replace Data**:  
+    Place your `.txt` file in the `src/Backend/dataSourceFiles` directory and restart the server.
+
+
+* * *
+
+**Future Enhancements**
+-----------------------
+
+*   Support for additional file formats (e.g., `.csv`, `.json`).
+*   Deployment to cloud platforms (e.g., Vercel, AWS).
+*   A user-friendly interface for uploading and managing datasets.
+
+* * *
+
+Feel free to fork, contribute, and customize the project to suit your needs! 🎉

applications/node/lanchain_writing_assistant/package.json

@@ -4,6 +4,7 @@
   "private": true,
   "dependencies": {
     "@heroicons/react": "^2.1.5",
+    "@lancedb/lancedb": "^0.12.0",
     "@langchain/community": "^0.3.1",
     "@langchain/openai": "^0.3.0",
     "@testing-library/jest-dom": "^5.17.0",

applications/node/lanchain_writing_assistant/src/Backend/langchainProcessor.mjs

@@ -20,7 +20,6 @@ let retriever;
 
 export async function runLangChainProcess() {
   try {
-    console.log("called")
     const loader = new TextLoader("src/Backend/dataSourceFiles/data.txt");
     const rawDocs = await loader.load();
     const splitter = new RecursiveCharacterTextSplitter({

applications/node/lanchain_writing_assistant/src/components/TextEditor.js

@@ -108,7 +108,7 @@ function TextEditor({ onProcess, onLoading }) {
         </h2>
         <ReactQuill
           value={input}
-          className = "h-[75vh] w-full bg-white rounded shadow-md rounded-lg border-2 border-gray-300 my-6 bg-white shadow-lg transition-shadow hover:shadow-2xl suggestion-box"
+          className = "h-[75vh] bg-white rounded shadow-md rounded-lg border-2 border-gray-300 my-6 bg-white shadow-lg transition-shadow hover:shadow-2xl suggestion-box mr-4"
           onChange={handleChange}
           theme="snow"
           placeholder="Start typing..."

applications/node/sentance_auto_complete/README.md

@@ -1,21 +1,119 @@
-# Real Time autocomplete suggestions using lanchain.js and LanceDB
-#  upload your data source and start typeing your article and you will be suggested with relevent relea time  suggestions that will act as auto complete for your article sentance
-# you can also switch between gpt models.
+**Real-Time Autocomplete Suggestions with LangChain.js and LanceDB**
 
-![](https://github.com/lancedb/assets/blob/main/recipes/sentance_Auto_complete.gif)
+An AI-powered **real-time autocomplete suggestion tool** built with **LangChain.js** and **LanceDB**. Upload your data source, start typing your article, and receive **relevant sentence suggestions** in real-time. This tool dynamically predicts and completes your sentences, enhancing your writing flow. Additionally, you can seamlessly switch between different **GPT models** for varied outputs.
 
-# add your open_ai_key (run this command in your terminal)
- export OPENAI_API_KEY= your_key
+**Demo**
+--------
 
-# node version  above 20 required
-# to run only server use
+![Real-Time Autocomplete Demo](https://github.com/lancedb/assets/blob/main/recipes/sentance_Auto_complete.gif)
 
-npm install
-npm run server
-# command to run on your local machine
+**Features**
+------------
 
-npm install
-npm run dev
-# To use your own data. go to src>Backend>dataSourceFiles and replace the .txt file or you can upload.
+*   ✍️ **Real-Time Suggestions**: Contextual, sentence-level suggestions to speed up your writing process.
+*   📂 **Customizable Data Source**: Use your own datasets by uploading text files or replacing the default source.
+*   🌟 **Switchable Models**: Easily toggle between GPT models for varied response styles.
 
-Open [http://localhost:3000](http://localhost:3000) with your browser to see the result.
+**Getting Started**
+-------------------
+
+### **1\. Prerequisites**
+
+Ensure you have:
+
+*   **Node.js** version **20+** installed.
+*   A valid [OpenAI API Key](https://platform.openai.com/signup).
+
+### **2\. Installation**
+
+#### Clone the Repository and Install Dependencies:
+
+
+`git clone`
+
+`cd `
+
+`npm install` 
+
+### **3\. Configuration**
+
+Set your OpenAI API key in your environment:
+
+`export OPENAI_API_KEY=your_openai_key` 
+
+Alternatively, you can create a `.env` file in the project root:
+
+`OPENAI_API_KEY=your_openai_key` 
+
+### **4\. Running the Server Only**
+
+To run just the backend server:
+
+`npm run server` 
+
+
+### **5\. Running the Full Application**
+
+To run the full application locally:
+
+`npm run dev` 
+
+Access the app in your browser at:
+
+`http://localhost:3000` 
+
+
+**Customizing the Dataset**
+---------------------------
+
+### Upload or Replace the Data Source:
+
+1.  Navigate to:
+
+    plaintext
+
+    Copy code
+
+    `src/Backend/dataSourceFiles` 
+
+2.  Replace the default `.txt` file with your custom text file **or** use the file upload functionality in the app.
+3.  Restart the server to load the new dataset.
+
+**Technical Overview**
+
+### **How It Works**
+
+1.  **Data Preprocessing and Embedding**
+
+    *   Input data is tokenized and converted into embeddings using OpenAI's models.
+2.  **Vector Search and Retrieval**
+
+    *   LanceDB efficiently retrieves the most relevant text snippets based on similarity to the user's input.
+3.  **Real-Time Sentence Suggestions**
+
+    *   LangChain.js dynamically generates sentence-level completions tailored to the current context.
+4.  **Model Switching**
+
+    *   Easily toggle between GPT models in the application for diverse suggestions.
+
+**Commands**
+
+*   **Start the Backend Server**:
+
+    `npm run server` 
+
+*   **Run the Full Application**:
+
+    `npm run dev` 
+
+*   **Replace or Upload Data**:  
+    Place your `.txt` file in the `src/Backend/dataSourceFiles` directory or use the file upload feature in the app.
+
+**Future Enhancements**
+-----------------------
+
+*   Support for more file formats, such as `.csv` and `.json`.
+*   Interactive UI for dataset uploads and management.
+*   Deployable versions for cloud platforms like AWS, Vercel, or Netlify.
+
+Feel free to fork, contribute, and customize this project for your unique requirements! 🚀

applications/node/sentance_auto_complete/src/Backend/langchainProcessor.mjs

@@ -22,7 +22,7 @@ let retriever;
 const llm = new OpenAI({
   temperature: 0.7, // Adjust based on how creative/responsive you want the LLM to be
   modelName: 'gpt-4o-mini', // Choose the model you're using, e.g., gpt - 4 or gpt - 3.5
-  apiKey: 'process.env.OPENAI_API_KEY', // Replace with your API key or use dotenv to load it
+  apiKey: process.env.OPENAI_API_KEY, // Replace with your API key or use dotenv to load it
 });
 
 export async function runLangChainProcess(rawDocsUploaded) {

applications/node/sentance_auto_complete/src/components/TextEditor.js

@@ -120,7 +120,7 @@ function TextEditor({ onProcess, onLoading }) {
         </h2>
         <ReactQuill
           value={input}
-          className = "h-[75vh] w-full bg-white rounded shadow-[0_4px_15px_rgba(0,0,0,0.8)] my-6 suggestion-box"
+          className = "h-[75vh] bg-white rounded shadow-[0_4px_15px_rgba(0,0,0,0.8)] my-6 suggestion-box mr-4"
           onChange={handleChange}
           theme="snow"
           placeholder="Start typing..."