Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Correct docker network issue in docker-compose.yaml #658

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

TrentSaggus
Copy link

Updated Docker Compose file to fix an error which was preventing the docker container from seeing other containers and local llm's through ollama and LMstudio on linux.

With the old variation docker was creating a network specifically for this container which did not have access to localhost or other containers. This change forces the container to use the 'host' docker network allowing access to the bare metal llms.

If this conflicts with the way docker works on Windows or Mac then maybe comment it out and add a comment to uncomment if having the above noted issues???

Please note, this is my first ever pull request. If i've gotten something wrong please let me know so i can fix it for next time.

Updated Docker Compose file to fix an error which was preventing the docker container from seeing other containers and local llm's through ollama and LMstudio on linux.

With the old variation docker was creating a network specifically for this container which did not have access to localhost or other containers. This change forces the container to use the 'host' docker network allowing access to the bare metal llms.
@Felitendo
Copy link

This does indeed fix the discoverability, but now you can't change the port anymore. Consider adding an Environment Variable that is called "WEBUI_PORT" that can change the internal port.

@thecodacus
Copy link
Collaborator

this is one way to fix it.. but yes you cannot change the port now.. and if the host system is using that port then this will not work.
try to modify the command to accept arguments command: pnpm run dockerstart
you might need to check the package.json file to see the full command of dockerstart

if this is resolved I will merge it happily

@hillct
Copy link

hillct commented Dec 23, 2024

I would agree that switching to host based networking is definitely not the best practice here, as it would introduce too many new deployment limitations.

The actual problem here is one of documentation. I designed and intended the docker configuration to use bridged networking as this is the most flexible and allows the most dense co-location on bare metal hardware.

@TrentSaggus please check to make sure that your docker installation on your Linux host is up to date as you will need at least 20.10.0 and utilize the docker bridge hostname host.docker.internal for accessing services on the bare metal hardware.

host.docker.internal is supported on Docker Desktop for macOS and Windows, and on Linux with Docker versions 20.10.0 and above.

Also, for what it's worth, it is best practice, particularly on Linux to utilize the Nvidia container runtime so as to encapsulate services utilizing cuda devices for inference, training or other GPU centric tasks. The Nvidia container runtime essentially just exposes GPU devices to the container instances so you continue to get the benefits of containerization while being able to operate GPU centric services. https://developer.nvidia.com/container-runtime

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants