From bcf4dd86d8cd00d4f8d5b67205015320bd69cce2 Mon Sep 17 00:00:00 2001 From: Dustin Loring Date: Fri, 29 Nov 2024 12:34:46 -0500 Subject: [PATCH] split up the README file --- Adding-LLMs.md | 13 ++++ CONTRIBUTING.md | 6 +- Docker.md | 49 ++++++++++++ FAQ.md | 42 +++++++++++ README.md | 197 +++++++----------------------------------------- Requested.md | 44 +++++++++++ 6 files changed, 181 insertions(+), 170 deletions(-) create mode 100644 Adding-LLMs.md create mode 100644 Docker.md create mode 100644 FAQ.md create mode 100644 Requested.md diff --git a/Adding-LLMs.md b/Adding-LLMs.md new file mode 100644 index 000000000..20cf4e9f1 --- /dev/null +++ b/Adding-LLMs.md @@ -0,0 +1,13 @@ +[![Bolt.new: AI-Powered Full-Stack Web Development in the Browser](./public/social_preview_index.jpg)](https://bolt.new) + +# Bolt.new Fork by Cole Medin - oTToDev + +This fork of Bolt.new (oTToDev) allows you to choose the LLM that you use for each prompt! Currently, you can use OpenAI, Anthropic, Ollama, OpenRouter, Gemini, LMStudio, Mistral, xAI, HuggingFace, DeepSeek, or Groq models - and it is easily extended to use any other model supported by the Vercel AI SDK! See the instructions below for running this locally and extending it to include more models. + +## Adding New LLMs: + +To make new LLMs available to use in this version of Bolt.new, head on over to `app/utils/constants.ts` and find the constant MODEL_LIST. Each element in this array is an object that has the model ID for the name (get this from the provider's API documentation), a label for the frontend model dropdown, and the provider. + +By default, Anthropic, OpenAI, Groq, and Ollama are implemented as providers, but the YouTube video for this repo covers how to extend this to work with more providers if you wish! + +When you add a new model to the MODEL_LIST array, it will immediately be available to use when you run the app locally or reload it. For Ollama models, make sure you have the model installed already before trying to use it here! diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 68215a289..87886fcd3 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,4 +1,8 @@ -# Contributing to oTToDev +[![Bolt.new: AI-Powered Full-Stack Web Development in the Browser](./public/social_preview_index.jpg)](https://bolt.new) + +# Bolt.new Fork by Cole Medin - oTToDev + +## Contributing to oTToDev First off, thank you for considering contributing to oTToDev! This fork aims to expand the capabilities of the original project by integrating multiple LLM providers and enhancing functionality. Every contribution helps make oTToDev a better tool for developers worldwide. diff --git a/Docker.md b/Docker.md new file mode 100644 index 000000000..6143ae95e --- /dev/null +++ b/Docker.md @@ -0,0 +1,49 @@ +[![Bolt.new: AI-Powered Full-Stack Web Development in the Browser](./public/social_preview_index.jpg)](https://bolt.new) + +# Bolt.new Fork by Cole Medin - oTToDev + +## Run with Docker + +Prerequisites: + +Git and Node.js as mentioned in our [README](README.md), as well as Docker: https://www.docker.com/ + +### 1a. Using Helper Scripts + +NPM scripts are provided for convenient building: + +```bash +# Development build +npm run dockerbuild + +# Production build +npm run dockerbuild:prod +``` + +### 1b. Direct Docker Build Commands (alternative to using NPM scripts) + +You can use Docker's target feature to specify the build environment instead of using NPM scripts if you wish: + +```bash +# Development build +docker build . --target bolt-ai-development + +# Production build +docker build . --target bolt-ai-production +``` + +### 2. Docker Compose with Profiles to Run the Container + +Use Docker Compose profiles to manage different environments: + +```bash +# Development environment +docker-compose --profile development up + +# Production environment +docker-compose --profile production up +``` + +When you run the Docker Compose command with the development profile, any changes you +make on your machine to the code will automatically be reflected in the site running +on the container (i.e. hot reloading still applies!). \ No newline at end of file diff --git a/FAQ.md b/FAQ.md new file mode 100644 index 000000000..3f74fa8ab --- /dev/null +++ b/FAQ.md @@ -0,0 +1,42 @@ +[![Bolt.new: AI-Powered Full-Stack Web Development in the Browser](./public/social_preview_index.jpg)](https://bolt.new) + +# Bolt.new Fork by Cole Medin - oTToDev + +## FAQ + +### Do you plan on merging oTToDev back into the official Bolt.new repo? + +More news coming on this coming early next month - stay tuned! + +### What are the future plans for oTToDev? + +[Check out our Roadmap here!](https://roadmap.sh/r/ottodev-roadmap-2ovzo) + +Lot more updates to this roadmap coming soon! + +### Why are there so many open issues/pull requests? + +oTToDev was started simply to showcase how to edit an open source project and to do something cool with local LLMs on my (@ColeMedin) YouTube channel! However, it quickly +grew into a massive community project that I am working hard to keep up with the demand of by forming a team of maintainers and getting as many people involved as I can. +That effort is going well and all of our maintainers are ABSOLUTE rockstars, but it still takes time to organize everything so we can efficiently get through all +the issues and PRs. But rest assured, we are working hard and even working on some partnerships behind the scenes to really help this project take off! + +### How do local LLMs fair compared to larger models like Claude 3.5 Sonnet for oTToDev/Bolt.new? + +As much as the gap is quickly closing between open source and massive close source models, you’re still going to get the best results with the very large models like GPT-4o, Claude 3.5 Sonnet, and DeepSeek Coder V2 236b. This is one of the big tasks we have at hand - figuring out how to prompt better, use agents, and improve the platform as a whole to make it work better for even the smaller local LLMs! + +### I'm getting the error: "There was an error processing this request" + +If you see this error within oTToDev, that is just the application telling you there is a problem at a high level, and this could mean a number of different things. To find the actual error, please check BOTH the terminal where you started the application (with Docker or pnpm) and the developer console in the browser. For most browsers, you can access the developer console by pressing F12 or right clicking anywhere in the browser and selecting “Inspect”. Then go to the “console” tab in the top right. + +### I'm getting the error: "x-api-key header missing" + +We have seen this error a couple times and for some reason just restarting the Docker container has fixed it. This seems to be Ollama specific. Another thing to try is try to run oTToDev with Docker or pnpm, whichever you didn’t run first. We are still on the hunt for why this happens once and a while! + +### I'm getting a blank preview when oTToDev runs my app! + +We promise you that we are constantly testing new PRs coming into oTToDev and the preview is core functionality, so the application is not broken! When you get a blank preview or don’t get a preview, this is generally because the LLM hallucinated bad code or incorrect commands. We are working on making this more transparent so it is obvious. Sometimes the error will appear in developer console too so check that as well. + +### Everything works but the results are bad + +This goes to the point above about how local LLMs are getting very powerful but you still are going to see better (sometimes much better) results with the largest LLMs like GPT-4o, Claude 3.5 Sonnet, and DeepSeek Coder V2 236b. If you are using smaller LLMs like Qwen-2.5-Coder, consider it more experimental and educational at this point. It can build smaller applications really well, which is super impressive for a local LLM, but for larger scale applications you want to use the larger LLMs still! diff --git a/README.md b/README.md index 0127dd0fa..95868b6a4 100644 --- a/README.md +++ b/README.md @@ -4,52 +4,13 @@ This fork of Bolt.new (oTToDev) allows you to choose the LLM that you use for each prompt! Currently, you can use OpenAI, Anthropic, Ollama, OpenRouter, Gemini, LMStudio, Mistral, xAI, HuggingFace, DeepSeek, or Groq models - and it is easily extended to use any other model supported by the Vercel AI SDK! See the instructions below for running this locally and extending it to include more models. -Join the community for oTToDev! - -https://thinktank.ottomator.ai - -# Requested Additions to this Fork - Feel Free to Contribute!! - -- ✅ OpenRouter Integration (@coleam00) -- ✅ Gemini Integration (@jonathands) -- ✅ Autogenerate Ollama models from what is downloaded (@yunatamos) -- ✅ Filter models by provider (@jasonm23) -- ✅ Download project as ZIP (@fabwaseem) -- ✅ Improvements to the main Bolt.new prompt in `app\lib\.server\llm\prompts.ts` (@kofi-bhr) -- ✅ DeepSeek API Integration (@zenith110) -- ✅ Mistral API Integration (@ArulGandhi) -- ✅ "Open AI Like" API Integration (@ZerxZ) -- ✅ Ability to sync files (one way sync) to local folder (@muzafferkadir) -- ✅ Containerize the application with Docker for easy installation (@aaronbolton) -- ✅ Publish projects directly to GitHub (@goncaloalves) -- ✅ Ability to enter API keys in the UI (@ali00209) -- ✅ xAI Grok Beta Integration (@milutinke) -- ✅ LM Studio Integration (@karrot0) -- ✅ HuggingFace Integration (@ahsan3219) -- ✅ Bolt terminal to see the output of LLM run commands (@thecodacus) -- ✅ Streaming of code output (@thecodacus) -- ✅ Ability to revert code to earlier version (@wonderwhy-er) -- ✅ Cohere Integration (@hasanraiyan) -- ✅ Dynamic model max token length (@hasanraiyan) -- ⬜ **HIGH PRIORITY** - Prevent Bolt from rewriting files as often (file locking and diffs) -- ⬜ **HIGH PRIORITY** - Better prompting for smaller LLMs (code window sometimes doesn't start) -- ⬜ **HIGH PRIORITY** - Load local projects into the app -- ⬜ **HIGH PRIORITY** - Attach images to prompts -- ⬜ **HIGH PRIORITY** - Run agents in the backend as opposed to a single model call -- ⬜ Mobile friendly -- ⬜ Together Integration -- ⬜ Azure Open AI API Integration -- ⬜ Perplexity Integration -- ⬜ Vertex AI Integration -- ⬜ Deploy directly to Vercel/Netlify/other similar platforms -- ⬜ Prompt caching -- ⬜ Better prompt enhancing -- ⬜ Have LLM plan the project in a MD file for better results/transparency -- ⬜ VSCode Integration with git-like confirmations -- ⬜ Upload documents for knowledge - UI design templates, a code base to reference coding style, etc. -- ⬜ Voice prompting - -# Bolt.new: AI-Powered Full-Stack Web Development in the Browser + +## Join the community for oTToDev! + +[Please join our community here to stay update with the latest!](https://thinktank.ottomator.ai) + +## Bolt.new: AI-Powered Full-Stack Web Development in the Browser + Bolt.new is an AI-powered web development agent that allows you to prompt, run, edit, and deploy full-stack applications directly from your browser—no local setup required. If you're here to build your own AI-powered web dev agent using the Bolt open source codebase, [click here to get started!](./CONTRIBUTING.md) @@ -70,7 +31,7 @@ Whether you’re an experienced developer, a PM, or a designer, Bolt.new allows For developers interested in building their own AI-powered development tools with WebContainers, check out the open-source Bolt codebase in this repo! -## Setup +## Prerequisites Many of you are new users to installing software from Github. If you have any installation troubles reach out and submit an "issue" using the links above, or feel free to enhance this documentation by forking, editing the instructions, and doing a pull request. @@ -88,15 +49,15 @@ echo $PATH . If you see usr/local/bin in the output then you're good to go. -3. Clone the repository (if you haven't already) by opening a Terminal window (or CMD with admin permissions) and then typing in this: +## Setup + +1. Clone the repository (if you haven't already) by opening a Terminal window (or CMD with admin permissions) and then typing in this: ``` git clone https://github.com/coleam00/bolt.new-any-llm.git ``` -3. Rename .env.example to .env.local and add your LLM API keys. You will find this file on a Mac at "[your name]/bold.new-any-llm/.env.example". For Windows and Linux the path will be similar. - -![image](https://github.com/user-attachments/assets/7e6a532c-2268-401f-8310-e8d20c731328) +2. Rename .env.example to .env.local and add your LLM API keys. You will find this file on a Mac at "[your name]/bold.new-any-llm/.env.example". For Windows and Linux the path will be similar. If you can't see the file indicated above, its likely you can't view hidden files. On Mac, open a Terminal window and enter this command below. On Windows, you will see the hidden files option in File Explorer Settings. A quick Google search will help you if you are stuck here. @@ -126,53 +87,7 @@ VITE_LOG_LEVEL=debug **Important**: Never commit your `.env.local` file to version control. It's already included in .gitignore. -## Run with Docker - -Prerequisites: - -Git and Node.js as mentioned above, as well as Docker: https://www.docker.com/ - -### 1a. Using Helper Scripts - -NPM scripts are provided for convenient building: - -```bash -# Development build -npm run dockerbuild - -# Production build -npm run dockerbuild:prod -``` - -### 1b. Direct Docker Build Commands (alternative to using NPM scripts) - -You can use Docker's target feature to specify the build environment instead of using NPM scripts if you wish: - -```bash -# Development build -docker build . --target bolt-ai-development - -# Production build -docker build . --target bolt-ai-production -``` - -### 2. Docker Compose with Profiles to Run the Container - -Use Docker Compose profiles to manage different environments: - -```bash -# Development environment -docker-compose --profile development up - -# Production environment -docker-compose --profile production up -``` - -When you run the Docker Compose command with the development profile, any changes you -make on your machine to the code will automatically be reflected in the site running -on the container (i.e. hot reloading still applies!). - -## Run Without Docker +## Run 1. Install dependencies using Terminal (or CMD in Windows with admin permissions): @@ -192,38 +107,27 @@ sudo npm install -g pnpm pnpm run dev ``` -## Super Important Note on Running Ollama Models - -Ollama models by default only have 2048 tokens for their context window. Even for large models that can easily handle way more. -This is not a large enough window to handle the Bolt.new/oTToDev prompt! You have to create a version of any model you want -to use where you specify a larger context window. Luckily it's super easy to do that. - -All you have to do is: +## Development -- Create a file called "Modelfile" (no file extension) anywhere on your computer -- Put in the two lines: +To start the development server: -``` -FROM [Ollama model ID such as qwen2.5-coder:7b] -PARAMETER num_ctx 32768 +```bash +pnpm run dev ``` -- Run the command: +This will start the Remix Vite development server. You will need Google Chrome Canary to run this locally if you use Chrome! It's an easy install and a good browser for web development anyway. -``` -ollama create -f Modelfile [your new model ID, can be whatever you want (example: qwen2.5-coder-extra-ctx:7b)] -``` +### Run With Docker -Now you have a new Ollama model that isn't heavily limited in the context length like Ollama models are by default for some reason. -You'll see this new model in the list of Ollama models along with all the others you pulled! +[Please check out our dedicated page for running oTToDev in docker here!](Docker.md) -## Adding New LLMs: +### How do I contribute to oTToDev? -To make new LLMs available to use in this version of Bolt.new, head on over to `app/utils/constants.ts` and find the constant MODEL_LIST. Each element in this array is an object that has the model ID for the name (get this from the provider's API documentation), a label for the frontend model dropdown, and the provider. +[Please check out our dedicated page for contributing to oTToDev here!](CONTRIBUTING.md) -By default, Anthropic, OpenAI, Groq, and Ollama are implemented as providers, but the YouTube video for this repo covers how to extend this to work with more providers if you wish! +## Adding New LLMs: -When you add a new model to the MODEL_LIST array, it will immediately be available to use when you run the app locally or reload it. For Ollama models, make sure you have the model installed already before trying to use it here! +[Please check out our dedicated page for adding LLM's to oTToDev here!](Adding-LLMs.md) ## Available Scripts @@ -236,18 +140,6 @@ When you add a new model to the MODEL_LIST array, it will immediately be availab - `pnpm run typegen`: Generates TypeScript types using Wrangler. - `pnpm run deploy`: Builds the project and deploys it to Cloudflare Pages. -## Development - -To start the development server: - -```bash -pnpm run dev -``` - -This will start the Remix Vite development server. You will need Google Chrome Canary to run this locally if you use Chrome! It's an easy install and a good browser for web development anyway. - -## FAQ - ### How do I get the best results with oTToDev? - **Be specific about your stack**: If you want to use specific frameworks or libraries (like Astro, Tailwind, ShadCN, or any other popular JavaScript framework), mention them in your initial prompt to ensure Bolt scaffolds the project accordingly. @@ -258,43 +150,10 @@ This will start the Remix Vite development server. You will need Google Chrome C - **Batch simple instructions**: Save time by combining simple instructions into one message. For example, you can ask oTToDev to change the color scheme, add mobile responsiveness, and restart the dev server, all in one go saving you time and reducing API credit consumption significantly. -### How do I contribute to oTToDev? - -[Please check out our dedicated page for contributing to oTToDev here!](CONTRIBUTING.md) - -### Do you plan on merging oTToDev back into the official Bolt.new repo? - -More news coming on this coming early next month - stay tuned! - -### What are the future plans for oTToDev? - -[Check out our Roadmap here!](https://roadmap.sh/r/ottodev-roadmap-2ovzo) - -Lot more updates to this roadmap coming soon! - -### Why are there so many open issues/pull requests? - -oTToDev was started simply to showcase how to edit an open source project and to do something cool with local LLMs on my (@ColeMedin) YouTube channel! However, it quickly -grew into a massive community project that I am working hard to keep up with the demand of by forming a team of maintainers and getting as many people involved as I can. -That effort is going well and all of our maintainers are ABSOLUTE rockstars, but it still takes time to organize everything so we can efficiently get through all -the issues and PRs. But rest assured, we are working hard and even working on some partnerships behind the scenes to really help this project take off! - -### How do local LLMs fair compared to larger models like Claude 3.5 Sonnet for oTToDev/Bolt.new? - -As much as the gap is quickly closing between open source and massive close source models, you’re still going to get the best results with the very large models like GPT-4o, Claude 3.5 Sonnet, and DeepSeek Coder V2 236b. This is one of the big tasks we have at hand - figuring out how to prompt better, use agents, and improve the platform as a whole to make it work better for even the smaller local LLMs! - -### I'm getting the error: "There was an error processing this request" - -If you see this error within oTToDev, that is just the application telling you there is a problem at a high level, and this could mean a number of different things. To find the actual error, please check BOTH the terminal where you started the application (with Docker or pnpm) and the developer console in the browser. For most browsers, you can access the developer console by pressing F12 or right clicking anywhere in the browser and selecting “Inspect”. Then go to the “console” tab in the top right. - -### I'm getting the error: "x-api-key header missing" - -We have seen this error a couple times and for some reason just restarting the Docker container has fixed it. This seems to be Ollama specific. Another thing to try is try to run oTToDev with Docker or pnpm, whichever you didn’t run first. We are still on the hunt for why this happens once and a while! - -### I'm getting a blank preview when oTToDev runs my app! +### Requested Additions - Feel Free to Contribute!! -We promise you that we are constantly testing new PRs coming into oTToDev and the preview is core functionality, so the application is not broken! When you get a blank preview or don’t get a preview, this is generally because the LLM hallucinated bad code or incorrect commands. We are working on making this more transparent so it is obvious. Sometimes the error will appear in developer console too so check that as well. +[Please check out our dedicated page for requesed additions here!](Requested.md) -### Everything works but the results are bad +### FAQ -This goes to the point above about how local LLMs are getting very powerful but you still are going to see better (sometimes much better) results with the largest LLMs like GPT-4o, Claude 3.5 Sonnet, and DeepSeek Coder V2 236b. If you are using smaller LLMs like Qwen-2.5-Coder, consider it more experimental and educational at this point. It can build smaller applications really well, which is super impressive for a local LLM, but for larger scale applications you want to use the larger LLMs still! +[Please check out our dedicated page for FAQ's for oTToDev here!](FAQ.md) \ No newline at end of file diff --git a/Requested.md b/Requested.md new file mode 100644 index 000000000..81bffa580 --- /dev/null +++ b/Requested.md @@ -0,0 +1,44 @@ +[![Bolt.new: AI-Powered Full-Stack Web Development in the Browser](./public/social_preview_index.jpg)](https://bolt.new) + +# Bolt.new Fork by Cole Medin - oTToDev + +## Requested Additions - Feel Free to Contribute!! + +- ✅ OpenRouter Integration (@coleam00) +- ✅ Gemini Integration (@jonathands) +- ✅ Autogenerate Ollama models from what is downloaded (@yunatamos) +- ✅ Filter models by provider (@jasonm23) +- ✅ Download project as ZIP (@fabwaseem) +- ✅ Improvements to the main Bolt.new prompt in `app\lib\.server\llm\prompts.ts` (@kofi-bhr) +- ✅ DeepSeek API Integration (@zenith110) +- ✅ Mistral API Integration (@ArulGandhi) +- ✅ "Open AI Like" API Integration (@ZerxZ) +- ✅ Ability to sync files (one way sync) to local folder (@muzafferkadir) +- ✅ Containerize the application with Docker for easy installation (@aaronbolton) +- ✅ Publish projects directly to GitHub (@goncaloalves) +- ✅ Ability to enter API keys in the UI (@ali00209) +- ✅ xAI Grok Beta Integration (@milutinke) +- ✅ LM Studio Integration (@karrot0) +- ✅ HuggingFace Integration (@ahsan3219) +- ✅ Bolt terminal to see the output of LLM run commands (@thecodacus) +- ✅ Streaming of code output (@thecodacus) +- ✅ Ability to revert code to earlier version (@wonderwhy-er) +- ✅ Cohere Integration (@hasanraiyan) +- ✅ Dynamic model max token length (@hasanraiyan) +- ⬜ **HIGH PRIORITY** - Prevent Bolt from rewriting files as often (file locking and diffs) +- ⬜ **HIGH PRIORITY** - Better prompting for smaller LLMs (code window sometimes doesn't start) +- ⬜ **HIGH PRIORITY** - Load local projects into the app +- ⬜ **HIGH PRIORITY** - Attach images to prompts +- ⬜ **HIGH PRIORITY** - Run agents in the backend as opposed to a single model call +- ⬜ Mobile friendly +- ⬜ Together Integration +- ⬜ Azure Open AI API Integration +- ⬜ Perplexity Integration +- ⬜ Vertex AI Integration +- ⬜ Deploy directly to Vercel/Netlify/other similar platforms +- ⬜ Prompt caching +- ⬜ Better prompt enhancing +- ⬜ Have LLM plan the project in a MD file for better results/transparency +- ⬜ VSCode Integration with git-like confirmations +- ⬜ Upload documents for knowledge - UI design templates, a code base to reference coding style, etc. +- ⬜ Voice prompting \ No newline at end of file