-
Notifications
You must be signed in to change notification settings - Fork 1
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
45 changed files
with
905 additions
and
131 deletions.
There are no files selected for viewing
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file modified
BIN
-441 Bytes
(100%)
src/docs/_build/doctrees/grag.components.vectordb.doctree
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,4 +1,4 @@ | ||
# Sphinx build info version 1 | ||
# This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done. | ||
config: eef517887ca30e256065416dae71ba9d | ||
config: 36fd2c4ce0764bf658f7515071f6fa4f | ||
tags: 645f666f9bcd5a90fca523b33c5a78b7 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,40 @@ | ||
LLMs | ||
===== | ||
|
||
GRAG offers two ways to run LLMs locally, | ||
|
||
1. LlamaCPP | ||
2. HuggingFace | ||
|
||
To run LLMs using HuggingFace | ||
############################# | ||
This is the easiest way to get started but does not offer as much | ||
flexibility. | ||
If using a config file (*config.ini*), just change the `model_name` to | ||
to the HuggingFace repo id. *Note that if the models are gated, make sure to | ||
provide an auth token* | ||
|
||
To run LLMs using LlamaCPP | ||
############################# | ||
To run models using LlamaCPP, make sure to have a `.gguf` model file. | ||
Smaller models offer this, but the recommended method is to quantize | ||
larger models or just download the quantized models. | ||
|
||
How to quantize models. | ||
************************ | ||
1. Steps to start with llama.cpp: | ||
|
||
* Clone the `llama.cpp <https://github.com/ggerganov/llama.cpp>`_ repository. | ||
``git clone https://github.com/ggerganov/llama.cpp.git`` | ||
* Change directory to `llama.cpp` using `cd llama.cpp` | ||
* To inference using GPU, which is necessary for mose models. | ||
* Make sure you have CUDA installed (check using ``nvcc --version``) | ||
* Follow steps from the `llama.cpp documentation <https://github.com/ggerganov/llama.cpp?tab=readme-ov-file#cublas>`_. | ||
|
||
*Note: While inferencing if model is not utilizing GPU check the `BLAS=1` in the outputs and* | ||
*if it is not then try reinstalling using*:: | ||
|
||
CMAKE_ARGS="-DLLAMA_CUBLAS=on" pip install llama-cpp-python --upgrade --force-reinstall --no-cache-dir | ||
|
||
*or follow the solution provided by* | ||
*`this Stack Overflow post <https://stackoverflow.com/questions/76963311/llama-cpp-python-not-using-nvidia-gpu-cuda>`_* |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,16 @@ | ||
Get Started | ||
=============== | ||
.. toctree:: | ||
:maxdepth: 2 | ||
|
||
get_started.llms | ||
|
||
|
||
To install the package. | ||
*Since we are just in the development phase we have not published to pypi yet.* | ||
|
||
* ``git clone`` the repository | ||
* ``pip install .`` from the repository | ||
* *For Developers*: ``pip install -e .`` | ||
|
||
Moreover, further customization can be made on the config file, `src/config.ini`. |
17 changes: 17 additions & 0 deletions
17
src/docs/_build/html/_sources/getting_started.llms.rst.txt
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,17 @@ | ||
LLMs | ||
===== | ||
|
||
GRAG offers two ways to run LLMs locally, | ||
1. LlamaCPP | ||
2. HuggingFace | ||
|
||
To run LLMs using LlamaCPP | ||
############################# | ||
|
||
To run LLMs using HuggingFace | ||
############################# | ||
This is the easiest way to get started but does not offer as much | ||
flexibility. | ||
If using a config file (*config.ini*), just change the `model_name` to | ||
to the HuggingFace repo id. *Note that if the models are gated, make sure to | ||
provide an auth token* |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,16 @@ | ||
Getting Started | ||
=============== | ||
.. toctree:: | ||
:maxdepth: 2 | ||
|
||
getting_started.llms | ||
|
||
|
||
To install the package. | ||
*Since we are just in the development phase we have not published to pypi yet.* | ||
|
||
* `git clone` the repository | ||
* `pip install .` from the repository | ||
* *For Developers*: `pip install -e .` | ||
|
||
Moreover, further customization can be made on the config file, `src/config.ini`. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,4 +1,4 @@ | ||
grag.components package | ||
Components | ||
======================= | ||
|
||
Subpackages | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,9 +1,6 @@ | ||
grag package | ||
GRAG | ||
============ | ||
|
||
Subpackages | ||
----------- | ||
|
||
.. toctree:: | ||
:maxdepth: 4 | ||
|
||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,132 @@ | ||
<!DOCTYPE html> | ||
<html class="writer-html5" lang="en" data-content_root="./"> | ||
<head> | ||
<meta charset="utf-8" /><meta name="viewport" content="width=device-width, initial-scale=1" /> | ||
|
||
<meta name="viewport" content="width=device-width, initial-scale=1.0" /> | ||
<title>Get Started — GRAG 0.0.1 documentation</title> | ||
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=80d5e7a1" /> | ||
<link rel="stylesheet" type="text/css" href="_static/css/theme.css?v=19f00094" /> | ||
|
||
|
||
<!--[if lt IE 9]> | ||
<script src="_static/js/html5shiv.min.js"></script> | ||
<![endif]--> | ||
|
||
<script src="_static/jquery.js?v=5d32c60e"></script> | ||
<script src="_static/_sphinx_javascript_frameworks_compat.js?v=2cd50e6c"></script> | ||
<script src="_static/documentation_options.js?v=d45e8c67"></script> | ||
<script src="_static/doctools.js?v=888ff710"></script> | ||
<script src="_static/sphinx_highlight.js?v=dc90522c"></script> | ||
<script src="_static/js/theme.js"></script> | ||
<link rel="index" title="Index" href="genindex.html" /> | ||
<link rel="search" title="Search" href="search.html" /> | ||
<link rel="next" title="LLMs" href="get_started.llms.html" /> | ||
<link rel="prev" title="Welcome to GRAG’s documentation!" href="index.html" /> | ||
</head> | ||
|
||
<body class="wy-body-for-nav"> | ||
<div class="wy-grid-for-nav"> | ||
<nav data-toggle="wy-nav-shift" class="wy-nav-side"> | ||
<div class="wy-side-scroll"> | ||
<div class="wy-side-nav-search" > | ||
|
||
|
||
|
||
<a href="index.html" class="icon icon-home"> | ||
GRAG | ||
</a> | ||
<div role="search"> | ||
<form id="rtd-search-form" class="wy-form" action="search.html" method="get"> | ||
<input type="text" name="q" placeholder="Search docs" aria-label="Search docs" /> | ||
<input type="hidden" name="check_keywords" value="yes" /> | ||
<input type="hidden" name="area" value="default" /> | ||
</form> | ||
</div> | ||
</div><div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="Navigation menu"> | ||
<p class="caption" role="heading"><span class="caption-text">Contents:</span></p> | ||
<ul class="current"> | ||
<li class="toctree-l1 current"><a class="current reference internal" href="#">Get Started</a><ul> | ||
<li class="toctree-l2"><a class="reference internal" href="get_started.llms.html">LLMs</a></li> | ||
</ul> | ||
</li> | ||
<li class="toctree-l1"><a class="reference internal" href="grag.html">GRAG</a></li> | ||
</ul> | ||
|
||
</div> | ||
</div> | ||
</nav> | ||
|
||
<section data-toggle="wy-nav-shift" class="wy-nav-content-wrap"><nav class="wy-nav-top" aria-label="Mobile navigation menu" > | ||
<i data-toggle="wy-nav-top" class="fa fa-bars"></i> | ||
<a href="index.html">GRAG</a> | ||
</nav> | ||
|
||
<div class="wy-nav-content"> | ||
<div class="rst-content"> | ||
<div role="navigation" aria-label="Page navigation"> | ||
<ul class="wy-breadcrumbs"> | ||
<li><a href="index.html" class="icon icon-home" aria-label="Home"></a></li> | ||
<li class="breadcrumb-item active">Get Started</li> | ||
<li class="wy-breadcrumbs-aside"> | ||
<a href="https://github.com/arjbingly/Capstone_5/blob/main/src/get_started.rst" class="fa fa-github"> Edit on GitHub</a> | ||
</li> | ||
</ul> | ||
<hr/> | ||
</div> | ||
<div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article"> | ||
<div itemprop="articleBody"> | ||
|
||
<section id="get-started"> | ||
<h1>Get Started<a class="headerlink" href="#get-started" title="Link to this heading"></a></h1> | ||
<div class="toctree-wrapper compound"> | ||
<ul> | ||
<li class="toctree-l1"><a class="reference internal" href="get_started.llms.html">LLMs</a><ul> | ||
<li class="toctree-l2"><a class="reference internal" href="get_started.llms.html#to-run-llms-using-huggingface">To run LLMs using HuggingFace</a></li> | ||
<li class="toctree-l2"><a class="reference internal" href="get_started.llms.html#to-run-llms-using-llamacpp">To run LLMs using LlamaCPP</a></li> | ||
</ul> | ||
</li> | ||
</ul> | ||
</div> | ||
<p>To install the package. | ||
<em>Since we are just in the development phase we have not published to pypi yet.</em></p> | ||
<ul class="simple"> | ||
<li><p><code class="docutils literal notranslate"><span class="pre">git</span> <span class="pre">clone</span></code> the repository</p></li> | ||
<li><p><code class="docutils literal notranslate"><span class="pre">pip</span> <span class="pre">install</span> <span class="pre">.</span></code> from the repository</p></li> | ||
<li><p><em>For Developers</em>: <code class="docutils literal notranslate"><span class="pre">pip</span> <span class="pre">install</span> <span class="pre">-e</span> <span class="pre">.</span></code></p></li> | ||
</ul> | ||
<p>Moreover, further customization can be made on the config file, <cite>src/config.ini</cite>.</p> | ||
</section> | ||
|
||
|
||
</div> | ||
</div> | ||
<footer><div class="rst-footer-buttons" role="navigation" aria-label="Footer"> | ||
<a href="index.html" class="btn btn-neutral float-left" title="Welcome to GRAG’s documentation!" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left" aria-hidden="true"></span> Previous</a> | ||
<a href="get_started.llms.html" class="btn btn-neutral float-right" title="LLMs" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right" aria-hidden="true"></span></a> | ||
</div> | ||
|
||
<hr/> | ||
|
||
<div role="contentinfo"> | ||
<p>© Copyright 2024, Arjun Bingly, Sanchit Vijay, Erica Pham, Kunal Inglunkar.</p> | ||
</div> | ||
|
||
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a | ||
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a> | ||
provided by <a href="https://readthedocs.org">Read the Docs</a>. | ||
|
||
|
||
</footer> | ||
</div> | ||
</div> | ||
</section> | ||
</div> | ||
<script> | ||
jQuery(function () { | ||
SphinxRtdTheme.Navigation.enable(true); | ||
}); | ||
</script> | ||
|
||
</body> | ||
</html> |
Oops, something went wrong.