Merge branch 'sphinx' of https://github.com/arjbingly/Capstone_5 into…

… sphinx
arjbingly · Apr 17, 2024 · 39a8c0e · 39a8c0e
2 parents 14ede10 + 614ddad
commit 39a8c0e
Show file tree

Hide file tree

Showing 8 changed files with 308 additions and 12 deletions.
diff --git a/cookbook/Basic-RAG/README.md b/cookbook/Basic-RAG/README.md
@@ -3,7 +3,7 @@
 Welcome to the Basic RAG Cookbook! This repository is dedicated to showcasing how to utilize the Retrieval-Augmented
 Generation (RAG) model for various applications using custom and few-shot prompts. For in depth understanding of RAG
 pipelines, chains, and prompts
-check [RAG-Piplines.md](https://github.com/arjbingly/Capstone_5/blob/main/projects/Basic-RAG/RAG-Piplines.md)
+check [RAG-Pipelines.md](https://github.com/arjbingly/Capstone_5/blob/main/cookbook/Basic-RAG/RAG-PIPELINES.md)
 
 ### Contents:
 

diff --git a/documentation/Sphinx - How to get started.md b/documentation/Sphinx - How to get started.md
@@ -0,0 +1,150 @@
+# Sphinx - How to get started?
+
+## Prereqs
+
+- You have proper documentation for your code. This includes doc strings.
+- Make sure that your doc strings follow a standard, eg. PEP, Google, Numpy, etc.
+- Highly recommended to use a linter for both your code and docs. I use Ruff.
+
+## 1. Install Sphinx
+
+*refer: https://www.sphinx-doc.org/en/master/usage/installation.html*
+
+- **for Ubuntu/Debian** : `sudo apt install python3-sphinx`
+
+## 1.1 Folder Structure (Recommended)
+
+I like my docs as a separate folder in `src`. Therefore I create a new directory `docs`. Move here for the next step.
+
+```
+├── src
+│   ├── build
+│   ├── package_dir
+│   │   ├── __init__.py
+│   │   ├── module_1.py
+│   │   ├── module_2.py
+│   │   ├── subpackage_dir
+│   │   │   ├── __init__.py
+│   │   │   ├── module_1.py
+│   │   │   ├── module_2.py
+├── other directories and files not documented
+│   ├── css
+│   │   ├── **/*.css
+│   ├── images
+│   ├── js
+│   ├── index.html
+├── pyproject.toml
+├── requirements.txt
+├── package-lock.json
+└── .gitignore
+
+```
+
+## 2. Quickstart
+
+Run:
+
+```
+sphinx-quickstart
+```
+
+This will ask you the following:
+
+1. **Do you wanna separate your build and source?** N, Since I recommend a folder structure with a separate docs
+   directory.
+2. Name of Project
+3. Author Name
+4. Project Release
+5. Project Language
+
+If run successfully this will create a `conf.py` file, the make files and folders like `_build`, `_static`,
+and `_template`.
+
+## 3. Extract docs
+
+Move to `src` dir.  
+Run:
+
+```
+sphinx-apidoc -o docs package_dir
+``` 
+
+Note that: here `-o` flag is the output folder, in our structure it is the `docs` directory, and `package_dir` is the
+directory with all the code you want to document.
+
+This will create `.rst` files for each Python module.
+
+## 4. Conf.py changes
+
+The `conf.py` file is in `docs` folder.
+You can see all the details you specified in the `quickstart` here.
+
+## 4.1 OS Syspath Changes
+
+Add the following to the beginning of `conf.py`.
+
+```
+import os
+import sys
+sys.path.insert(0, os.path.abspath("../package_dir"))
+``` 
+
+*Note that this assumes the above-mentioned folder structure, if you have a different structure, make sure to point it
+to the source code. *
+
+## 4.3 Extensions
+
+Sphinx has a lot of useful extensions. These should be added to the `extensions` tag. Some of the extensions I use are:
+
+1. `sphinx.ext.autodoc` - This extension automatically takes doc strings.
+2. `sphinx.ext.linkcode` (Optional) - This extension provides a link to the GitHub code. *(Note that this extension
+   requires other configs. Refer to Sphinx extension documentation for more details.)*
+3. `sphinx.ext.viewcode` (Optional) - This extension is similar to the above extension, instead of linking the code to
+   GitHub, it displays the code in a static webpage.
+4. `sphinx.ext.napoleon` (Optional) - Since we use Google doc-string standard, this is an essential extension.
+
+## 4.2 Theme (Optional)
+
+I use a Sphinx theme, which can be installed by running,
+
+``` 
+pip install sphinx-rtd-theme 
+```
+
+Change the `html_theme` tag in the `conf.py` to `sphinx_rtd_theme`.
+
+You can find more themes at various sources like `www.sphinx-themes.org/`,
+`https://sphinxthemes.com`, etc.
+
+## 5 Building the docs
+
+To finally generate the docs run the following command from the `docs` directory.
+
+```
+make html
+```
+
+This will create a `build` directory, where you can find the html files. Opening the `index.html` shows you the homepage
+of your docs.
+
+## 6.1 Adding other pages (Optional)
+
+To add other pages to your sphinx website, you just have to create `.rst` reStructuredText files in the appropriate
+location and add them to your `index.rst` or to the `toctree` of a file already mentioned in `index.rst`.
+
+For more instructions on defining document structure refer
+[Defining Docuement Structure](https://www.sphinx-doc.org/en/master/usage/quickstart.html#defining-document-structure)
+
+For instructions on how to format reStructuredText refer to
+[reStructuredText Basics](https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html)
+
+## 6.2 Adding examples (Optional)
+
+TODO
+
+- [https://sphinx-gallery.github.io/stable/index.html]()
+
+### References
+
+- [https://www.sphinx-doc.org/en/master/usage/quickstart.html]()
+- [https://samnicholls.net/2016/06/15/how-to-sphinx-readthedocs/]()
diff --git a/llm_quantize/README.md b/llm_quantize/README.md
@@ -1,15 +1,14 @@
 ## Model Quantization
 
-This module provides interactive way to quantize your model.
-To quantize model:  
+This module provides an interactive way to quantize your model.
+To quantize model, run:  
 `python -m grag.quantize.quantize`
 
-After running the above command user will be prompted with following:
+After running the above command, user will be prompted with the following:
 
-- Path where user want to clone [llama.cpp](!https://github.com/ggerganov/llama.cpp) repo.
+- Path where user wants to clone [llama.cpp](!https://github.com/ggerganov/llama.cpp) repo
 - If user wants us to download model from [HuggingFace](!https://huggingface.co/models) or user has model downloaded
-  locally.
-- For former, user will be prompted to provide repo path from HuggingFace.
-- In case of later, user will be instructed to copy the model and input the name of model directory.
-- Finally, user will be prompted to enter quantization (recommended Q5_K_M or Q4_K_M, etc.). Check
-  more [here](!https://github.com/ggerganov/llama.cpp/blob/master/examples/quantize/quantize.cpp#L19).
+  locally
+  - For the former, user will be prompted to provide repo path from HuggingFace
+  - For the latter, user will be instructed to copy the model and input the name of model directory
+- Finally, user will be prompted to enter quantization (recommended Q5_K_M or Q4_K_M, etc.). For more details, check [here](!https://github.com/ggerganov/llama.cpp/blob/master/examples/quantize/quantize.cpp#L19).
diff --git a/src/docs/_build/doctrees/get_started.introduction.doctree b/src/docs/_build/doctrees/get_started.introduction.doctree
diff --git a/src/docs/_build/html/_sources/get_started.introduction.rst.txt b/src/docs/_build/html/_sources/get_started.introduction.rst.txt
@@ -0,0 +1,8 @@
+GRAG Overview
+=============
+
+GRAG provides an implementation of Retrieval-Augmented Generation that is completely open-sourced.
+
+Retrieval-Augmented Generation
+###################
+
diff --git a/src/docs/_build/html/get_started.introduction.html b/src/docs/_build/html/get_started.introduction.html
@@ -0,0 +1,131 @@
+<!DOCTYPE html>
+<html class="writer-html5" lang="en" data-content_root="./">
+<head>
+  <meta charset="utf-8" /><meta name="viewport" content="width=device-width, initial-scale=1" />
+
+  <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+  <title>GRAG Overview &mdash; GRAG 0.0.1 documentation</title>
+      <link rel="stylesheet" type="text/css" href="_static/pygments.css?v=80d5e7a1" />
+      <link rel="stylesheet" type="text/css" href="_static/css/theme.css?v=19f00094" />
+      <link rel="stylesheet" type="text/css" href="_static/sg_gallery.css?v=61a4c737" />
+      <link rel="stylesheet" type="text/css" href="_static/sg_gallery-binder.css?v=f4aeca0c" />
+      <link rel="stylesheet" type="text/css" href="_static/sg_gallery-dataframe.css?v=2082cf3c" />
+      <link rel="stylesheet" type="text/css" href="_static/sg_gallery-rendered-html.css?v=1277b6f3" />
+
+
+  <!--[if lt IE 9]>
+    <script src="_static/js/html5shiv.min.js"></script>
+  <![endif]-->
+
+        <script src="_static/jquery.js?v=5d32c60e"></script>
+        <script src="_static/_sphinx_javascript_frameworks_compat.js?v=2cd50e6c"></script>
+        <script src="_static/documentation_options.js?v=d45e8c67"></script>
+        <script src="_static/doctools.js?v=888ff710"></script>
+        <script src="_static/sphinx_highlight.js?v=dc90522c"></script>
+    <script src="_static/js/theme.js"></script>
+    <link rel="index" title="Index" href="genindex.html" />
+    <link rel="search" title="Search" href="search.html" />
+    <link rel="next" title="Installation" href="get_started.installation.html" />
+    <link rel="prev" title="Get Started" href="get_started.html" /> 
+</head>
+
+<body class="wy-body-for-nav"> 
+  <div class="wy-grid-for-nav">
+    <nav data-toggle="wy-nav-shift" class="wy-nav-side">
+      <div class="wy-side-scroll">
+        <div class="wy-side-nav-search" >
+
+
+
+          <a href="index.html" class="icon icon-home">
+            GRAG
+          </a>
+<div role="search">
+  <form id="rtd-search-form" class="wy-form" action="search.html" method="get">
+    <input type="text" name="q" placeholder="Search docs" aria-label="Search docs" />
+    <input type="hidden" name="check_keywords" value="yes" />
+    <input type="hidden" name="area" value="default" />
+  </form>
+</div>
+        </div><div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="Navigation menu">
+              <p class="caption" role="heading"><span class="caption-text">Contents:</span></p>
+<ul class="current">
+<li class="toctree-l1 current"><a class="reference internal" href="get_started.html">Get Started</a><ul class="current">
+<li class="toctree-l2 current"><a class="current reference internal" href="#">GRAG Overview</a><ul>
+<li class="toctree-l3"><a class="reference internal" href="#retrieval-augmented-generation">Retrieval-Augmented Generation</a></li>
+</ul>
+</li>
+<li class="toctree-l2"><a class="reference internal" href="get_started.installation.html">Installation</a></li>
+<li class="toctree-l2"><a class="reference internal" href="get_started.llms.html">LLMs</a></li>
+<li class="toctree-l2"><a class="reference internal" href="get_started.vectordb.html">Vector Stores</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal" href="grag.html">GRAG</a></li>
+<li class="toctree-l1"><a class="reference internal" href="auto_examples_index.html">Cookbooks</a></li>
+</ul>
+
+        </div>
+      </div>
+    </nav>
+
+    <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap"><nav class="wy-nav-top" aria-label="Mobile navigation menu" >
+          <i data-toggle="wy-nav-top" class="fa fa-bars"></i>
+          <a href="index.html">GRAG</a>
+      </nav>
+
+      <div class="wy-nav-content">
+        <div class="rst-content">
+          <div role="navigation" aria-label="Page navigation">
+  <ul class="wy-breadcrumbs">
+      <li><a href="index.html" class="icon icon-home" aria-label="Home"></a></li>
+          <li class="breadcrumb-item"><a href="get_started.html">Get Started</a></li>
+      <li class="breadcrumb-item active">GRAG Overview</li>
+      <li class="wy-breadcrumbs-aside">
+              <a href="https://github.com/arjbingly/Capstone_5/blob/main/src/get_started.introduction.rst" class="fa fa-github"> Edit on GitHub</a>
+      </li>
+  </ul>
+  <hr/>
+</div>
+          <div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
+           <div itemprop="articleBody">
+
+  <section id="grag-overview">
+<h1>GRAG Overview<a class="headerlink" href="#grag-overview" title="Link to this heading"></a></h1>
+<p>GRAG provides an implementation of Retrieval-Augmented Generation that is completely open-sourced.</p>
+<section id="retrieval-augmented-generation">
+<h2>Retrieval-Augmented Generation<a class="headerlink" href="#retrieval-augmented-generation" title="Link to this heading"></a></h2>
+</section>
+</section>
+
+
+           </div>
+          </div>
+          <footer><div class="rst-footer-buttons" role="navigation" aria-label="Footer">
+        <a href="get_started.html" class="btn btn-neutral float-left" title="Get Started" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left" aria-hidden="true"></span> Previous</a>
+        <a href="get_started.installation.html" class="btn btn-neutral float-right" title="Installation" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right" aria-hidden="true"></span></a>
+    </div>
+
+  <hr/>
+
+  <div role="contentinfo">
+    <p>&#169; Copyright 2024, Arjun Bingly, Sanchit Vijay, Erica Pham, Kunal Inglunkar.</p>
+  </div>
+
+  Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
+    <a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
+    provided by <a href="https://readthedocs.org">Read the Docs</a>.
+
+
+</footer>
+        </div>
+      </div>
+    </section>
+  </div>
+  <script>
+      jQuery(function () {
+          SphinxRtdTheme.Navigation.enable(true);
+      });
+  </script> 
+
+</body>
+</html>
diff --git a/src/docs/conf.py b/src/docs/conf.py
@@ -18,8 +18,8 @@
 # -- Project information -----------------------------------------------------
 
 project = 'GRAG'
-copyright = '2024, Arjun Bingly, Sanchit Vijay, Erica Pham, Kunal Inglunkar'
-author = 'Arjun Bingly, Sanchit Vijay, Erica Pham, Kunal Inglunkar'
+copyright = '2024, Arjun Bingly, Sanchit Vijay, Erika Pham, Kunal Inglunkar'
+author = 'Arjun Bingly, Sanchit Vijay, Erika Pham, Kunal Inglunkar'
 
 # The full version, including alpha/beta/rc tags
 release = '0.0.1'

diff --git a/src/docs/get_started.introduction.rst b/src/docs/get_started.introduction.rst
@@ -0,0 +1,8 @@
+GRAG Overview
+=============
+
+GRAG provides an implementation of Retrieval-Augmented Generation that is completely open-sourced.
+
+Retrieval-Augmented Generation
+###################
+