Add support for Palm, Claude-2, Cohere, Llama2, CodeLlama (100+LLMs)

[ishaan-jaff]

This PR adds support for the above mentioned LLMs using LiteLLM https://github.com/BerriAI/litellm/
LiteLLM is a lightweight package to simplify LLM API calls - use any llm as a drop in replacement for gpt-3.5-turbo.

Example

from litellm import completion

## set ENV variables
os.environ["OPENAI_API_KEY"] = "openai key"
os.environ["COHERE_API_KEY"] = "cohere key"

messages = [{ "content": "Hello, how are you?","role": "user"}]

# openai call
response = completion(model="gpt-3.5-turbo", messages=messages)

# cohere call
response = completion(model="command-nightly", messages)

# anthropic call
response = completion(model="claude-instant-1", messages=messages)

[ishaan-jaff]

@gaurav274 @jarulraj can i get a review on this pr ?

happy to add docs/testing if this initial commit looks good

[github-actions]

👋 Hello @ishaan-jaff, thanks for submitting a EVA DB PR 🙏 To allow your work to be integrated as seamlessly as possible, we advise you to:

• ✅ Verify that your PR is up-to-date with georgia-tech-db/eva master branch. If your PR is behind you can update your code by clicking the 'Update branch' button or by running git pull and git merge master locally.
• ✅ Verify that all EVA DB Continuous Integration (CI) checks are passing.
• ✅ Reduce changes to the absolute minimum required for your bug fix or feature addition.

[gaurav274]

@gaurav274 @jarulraj can i get a review on this pr ?

happy to add docs/testing if this initial commit looks good

@ishaan-jaff Thanks for working on the integration. Absolutely, happy to review.

[xzdandy]

Does litellm subsumes the openai? Or we still need both dependency?

[ishaan-jaff]

litellm has openai as a dependency

[xzdandy]

It seems we can not find a version of litellm that works with the rest of the system.

ERROR: Ignored the following versions that require a different python version: 1.9.5 Requires-Python >=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, <3.7
ERROR: Could not find a version that satisfies the requirement litellm>=0.500; extra == "dev" (from evadb[dev,ludwig,qdrant]) (from versions: 0.1.0, 0.1.1, 0.1.2, 0.1.3, 0.1.31, 0.1.32, 0.1.34, 0.1.201, 0.1.202, 0.1.203, 0.1.204, 0.1.205, 0.1.206, 0.1.207, 0.1.208, 0.1.209, 0.1.210, 0.1.211, 0.1.212, 0.1.213, 0.1.214, 0.1.215, 0.1.216, 0.1.217, 0.1.218, 0.1.219, 0.1.220, 0.1.221, 0.1.222, 0.1.223, 0.1.224, 0.1.225, 0.1.226, 0.1.227, 0.1.228, 0.1.229, 0.1.230, 0.1.231, 0.1.232, 0.1.233, 0.1.234, 0.1.235, 0.1.236, 0.1.237, 0.1.238, 0.1.330, 0.1.331, 0.1.341, 0.1.343, 0.1.345, 0.1.347, 0.1.348, 0.1.349, 0.1.351, 0.1.352, 0.1.353, 0.1.354, 0.1.356, 0.1.360, 0.1.361, 0.1.362, 0.1.363, 0.1.364, 0.1.365, 0.1.366, 0.1.367, 0.1.368, 0.1.369, 0.1.370, 0.1.371, 0.1.372, 0.1.373, 0.1.375, 0.1.376, 0.1.379, 0.1.380, 0.1.381, 0.1.383, 0.1.384, 0.1.385, 0.1.386, 0.1.387, 0.1.388, 0.1.389, 0.1.392, 0.1.393, 0.1.394, 0.1.398, 0.1.399, 0.1.400, 0.1.401, 0.1.402, 0.1.403, 0.1.404, 0.1.405, 0.1.408, 0.1.410, 0.1.411, 0.1.412, 0.1.415, 0.1.419, 0.1.420, 0.1.421, 0.1.422, 0.1.424, 0.1.425, 0.1.426, 0.1.429, 0.1.432, 0.1.433, 0.1.434, 0.1.435, 0.1.436, 0.1.437, 0.1.438, 0.1.439, 0.1.440, 0.1.441, 0.1.442, 0.1.443, 0.1.444, 0.1.445, 0.1.446, 0.1.447, 0.1.448, 0.1.449, 0.1.450, 0.1.451, 0.1.452, 0.1.456, 0.1.457, 0.1.459, 0.1.460, 0.1.461, 0.1.464, 0.1.465, 0.1.475, 0.1.477, 0.1.479, 0.1.480, 0.1.481, 0.1.482, 0.1.486, 0.1.487, 0.1.488, 0.1.490, 0.1.491, 0.1.492, 0.1.493, 0.1.494, 0.1.495, 0.1.497, 0.1.500, 0.1.501, 0.1.504, 0.1.507, 0.1.508, 0.1.509, 0.1.510, 0.1.511, 0.1.512, 0.1.516, 0.1.517, 0.1.518, 0.1.520, 0.1.525, 0.1.530, 0.1.531, 0.1.533, 0.1.535, 0.1.536, 0.1.537, 0.1.538, 0.1.544, 0.1.546, 0.1.547, 0.1.548, 0.1.549, 0.1.550, 0.1.551, 0.1.552, 0.1.553, 0.1.554, 0.1.555, 0.1.556, 0.1.2291)
ERROR: No matching distribution found for litellm>=0.500; extra == "dev"

[ishaan-jaff]

@xzdandy sorry for that bumped the version to 0.1.561 which is our latest version. can you try installing that and see if it works?

[gaurav274]

It seems we can not find a version of litellm that works with the rest of the system.

ERROR: Ignored the following versions that require a different python version: 1.9.5 Requires-Python >=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, <3.7
ERROR: Could not find a version that satisfies the requirement litellm>=0.500; extra == "dev" (from evadb[dev,ludwig,qdrant]) (from versions: 0.1.0, 0.1.1, 0.1.2, 0.1.3, 0.1.31, 0.1.32, 0.1.34, 0.1.201, 0.1.202, 0.1.203, 0.1.204, 0.1.205, 0.1.206, 0.1.207, 0.1.208, 0.1.209, 0.1.210, 0.1.211, 0.1.212, 0.1.213, 0.1.214, 0.1.215, 0.1.216, 0.1.217, 0.1.218, 0.1.219, 0.1.220, 0.1.221, 0.1.222, 0.1.223, 0.1.224, 0.1.225, 0.1.226, 0.1.227, 0.1.228, 0.1.229, 0.1.230, 0.1.231, 0.1.232, 0.1.233, 0.1.234, 0.1.235, 0.1.236, 0.1.237, 0.1.238, 0.1.330, 0.1.331, 0.1.341, 0.1.343, 0.1.345, 0.1.347, 0.1.348, 0.1.349, 0.1.351, 0.1.352, 0.1.353, 0.1.354, 0.1.356, 0.1.360, 0.1.361, 0.1.362, 0.1.363, 0.1.364, 0.1.365, 0.1.366, 0.1.367, 0.1.368, 0.1.369, 0.1.370, 0.1.371, 0.1.372, 0.1.373, 0.1.375, 0.1.376, 0.1.379, 0.1.380, 0.1.381, 0.1.383, 0.1.384, 0.1.385, 0.1.386, 0.1.387, 0.1.388, 0.1.389, 0.1.392, 0.1.393, 0.1.394, 0.1.398, 0.1.399, 0.1.400, 0.1.401, 0.1.402, 0.1.403, 0.1.404, 0.1.405, 0.1.408, 0.1.410, 0.1.411, 0.1.412, 0.1.415, 0.1.419, 0.1.420, 0.1.421, 0.1.422, 0.1.424, 0.1.425, 0.1.426, 0.1.429, 0.1.432, 0.1.433, 0.1.434, 0.1.435, 0.1.436, 0.1.437, 0.1.438, 0.1.439, 0.1.440, 0.1.441, 0.1.442, 0.1.443, 0.1.444, 0.1.445, 0.1.446, 0.1.447, 0.1.448, 0.1.449, 0.1.450, 0.1.451, 0.1.452, 0.1.456, 0.1.457, 0.1.459, 0.1.460, 0.1.461, 0.1.464, 0.1.465, 0.1.475, 0.1.477, 0.1.479, 0.1.480, 0.1.481, 0.1.482, 0.1.486, 0.1.487, 0.1.488, 0.1.490, 0.1.491, 0.1.492, 0.1.493, 0.1.494, 0.1.495, 0.1.497, 0.1.500, 0.1.501, 0.1.504, 0.1.507, 0.1.508, 0.1.509, 0.1.510, 0.1.511, 0.1.512, 0.1.516, 0.1.517, 0.1.518, 0.1.520, 0.1.525, 0.1.530, 0.1.531, 0.1.533, 0.1.535, 0.1.536, 0.1.537, 0.1.538, 0.1.544, 0.1.546, 0.1.547, 0.1.548, 0.1.549, 0.1.550, 0.1.551, 0.1.552, 0.1.553, 0.1.554, 0.1.555, 0.1.556, 0.1.2291)
ERROR: No matching distribution found for litellm>=0.500; extra == "dev"

@xzdandy Are we planning to remove openai dependency from evadb? How about we add a new UDF using litellm?

[xzdandy]

It seems we can not find a version of litellm that works with the rest of the system.

ERROR: Ignored the following versions that require a different python version: 1.9.5 Requires-Python >=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, <3.7
ERROR: Could not find a version that satisfies the requirement litellm>=0.500; extra == "dev" (from evadb[dev,ludwig,qdrant]) (from versions: 0.1.0, 0.1.1, 0.1.2, 0.1.3, 0.1.31, 0.1.32, 0.1.34, 0.1.201, 0.1.202, 0.1.203, 0.1.204, 0.1.205, 0.1.206, 0.1.207, 0.1.208, 0.1.209, 0.1.210, 0.1.211, 0.1.212, 0.1.213, 0.1.214, 0.1.215, 0.1.216, 0.1.217, 0.1.218, 0.1.219, 0.1.220, 0.1.221, 0.1.222, 0.1.223, 0.1.224, 0.1.225, 0.1.226, 0.1.227, 0.1.228, 0.1.229, 0.1.230, 0.1.231, 0.1.232, 0.1.233, 0.1.234, 0.1.235, 0.1.236, 0.1.237, 0.1.238, 0.1.330, 0.1.331, 0.1.341, 0.1.343, 0.1.345, 0.1.347, 0.1.348, 0.1.349, 0.1.351, 0.1.352, 0.1.353, 0.1.354, 0.1.356, 0.1.360, 0.1.361, 0.1.362, 0.1.363, 0.1.364, 0.1.365, 0.1.366, 0.1.367, 0.1.368, 0.1.369, 0.1.370, 0.1.371, 0.1.372, 0.1.373, 0.1.375, 0.1.376, 0.1.379, 0.1.380, 0.1.381, 0.1.383, 0.1.384, 0.1.385, 0.1.386, 0.1.387, 0.1.388, 0.1.389, 0.1.392, 0.1.393, 0.1.394, 0.1.398, 0.1.399, 0.1.400, 0.1.401, 0.1.402, 0.1.403, 0.1.404, 0.1.405, 0.1.408, 0.1.410, 0.1.411, 0.1.412, 0.1.415, 0.1.419, 0.1.420, 0.1.421, 0.1.422, 0.1.424, 0.1.425, 0.1.426, 0.1.429, 0.1.432, 0.1.433, 0.1.434, 0.1.435, 0.1.436, 0.1.437, 0.1.438, 0.1.439, 0.1.440, 0.1.441, 0.1.442, 0.1.443, 0.1.444, 0.1.445, 0.1.446, 0.1.447, 0.1.448, 0.1.449, 0.1.450, 0.1.451, 0.1.452, 0.1.456, 0.1.457, 0.1.459, 0.1.460, 0.1.461, 0.1.464, 0.1.465, 0.1.475, 0.1.477, 0.1.479, 0.1.480, 0.1.481, 0.1.482, 0.1.486, 0.1.487, 0.1.488, 0.1.490, 0.1.491, 0.1.492, 0.1.493, 0.1.494, 0.1.495, 0.1.497, 0.1.500, 0.1.501, 0.1.504, 0.1.507, 0.1.508, 0.1.509, 0.1.510, 0.1.511, 0.1.512, 0.1.516, 0.1.517, 0.1.518, 0.1.520, 0.1.525, 0.1.530, 0.1.531, 0.1.533, 0.1.535, 0.1.536, 0.1.537, 0.1.538, 0.1.544, 0.1.546, 0.1.547, 0.1.548, 0.1.549, 0.1.550, 0.1.551, 0.1.552, 0.1.553, 0.1.554, 0.1.555, 0.1.556, 0.1.2291)
ERROR: No matching distribution found for litellm>=0.500; extra == "dev"

@xzdandy Are we planning to remove openai dependency from evadb? How about we add a new UDF using litellm?

I think we can remove the explicit openai dependency, given litellm has openai dependency. Maybe we add a new type of function?

CREATE FUNCTION openai 
TYPE litellm
MODEL "gpt-3.5-turbo";

[jarulraj]

Yes, @xzdandy. Let's do this. We had a meeting with @ishaan-jaff earlier today.