Unable to set Client host, httpx.ConnectError

[tdimitch]

Hi there!

I'm trying to run an instance of ollama Client and set the host. As shown in the documentation (https://github.com/ollama/ollama-python?tab=readme-ov-file#custom-client) I see the code should work, but prociding any host but the OLLAMA_HOST environment variable (or 11434 as default if OLLAMA_HOST does not exist), gives me
httpx.ConnectError: [Errno 111] Connection refused

I put the code of an interactive session:

Python 3.10.12 (main, Sep 11 2024, 15:47:36) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.

client = Client(host='http://localhost:11435')
response = client.chat(model='llama3.1', messages=[
... {
... 'role': 'user',
... 'content': 'Why is the sky blue?',
... },
... ])

Traceback (most recent call last):
  File "/home/bermudez/.local/lib/python3.10/site-packages/httpx/_transports/default.py", line 72, in map_httpcore_exceptions
    yield
  File "/home/bermudez/.local/lib/python3.10/site-packages/httpx/_transports/default.py", line 236, in handle_request
    resp = self._pool.handle_request(req)
  File "/home/bermudez/.local/lib/python3.10/site-packages/httpcore/_sync/connection_pool.py", line 216, in handle_request
    raise exc from None
  File "/home/bermudez/.local/lib/python3.10/site-packages/httpcore/_sync/connection_pool.py", line 196, in handle_request
    response = connection.handle_request(
  File "/home/bermudez/.local/lib/python3.10/site-packages/httpcore/_sync/connection.py", line 99, in handle_request
    raise exc
  File "/home/bermudez/.local/lib/python3.10/site-packages/httpcore/_sync/connection.py", line 76, in handle_request
    stream = self._connect(request)
  File "/home/bermudez/.local/lib/python3.10/site-packages/httpcore/_sync/connection.py", line 122, in _connect
    stream = self._network_backend.connect_tcp(**kwargs)
  File "/home/bermudez/.local/lib/python3.10/site-packages/httpcore/_backends/sync.py", line 205, in connect_tcp
    with map_exceptions(exc_map):
  File "/usr/lib/python3.10/contextlib.py", line 153, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/home/bermudez/.local/lib/python3.10/site-packages/httpcore/_exceptions.py", line 14, in map_exceptions
    raise to_exc(exc) from exc
httpcore.ConnectError: [Errno 111] Connection refused

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/bermudez/.local/lib/python3.10/site-packages/ollama/_client.py", line 236, in chat
    return self._request_stream(
  File "/home/bermudez/.local/lib/python3.10/site-packages/ollama/_client.py", line 99, in _request_stream
    return self._stream(*args, **kwargs) if stream else self._request(*args, **kwargs).json()
  File "/home/bermudez/.local/lib/python3.10/site-packages/ollama/_client.py", line 70, in _request
    response = self._client.request(method, url, **kwargs)
  File "/home/bermudez/.local/lib/python3.10/site-packages/httpx/_client.py", line 837, in request
    return self.send(request, auth=auth, follow_redirects=follow_redirects)
  File "/home/bermudez/.local/lib/python3.10/site-packages/httpx/_client.py", line 926, in send
    response = self._send_handling_auth(
  File "/home/bermudez/.local/lib/python3.10/site-packages/httpx/_client.py", line 954, in _send_handling_auth
    response = self._send_handling_redirects(
  File "/home/bermudez/.local/lib/python3.10/site-packages/httpx/_client.py", line 991, in _send_handling_redirects
    response = self._send_single_request(request)
  File "/home/bermudez/.local/lib/python3.10/site-packages/httpx/_client.py", line 1027, in _send_single_request
    response = transport.handle_request(request)
  File "/home/bermudez/.local/lib/python3.10/site-packages/httpx/_transports/default.py", line 235, in handle_request
    with map_httpcore_exceptions():
  File "/usr/lib/python3.10/contextlib.py", line 153, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/home/bermudez/.local/lib/python3.10/site-packages/httpx/_transports/default.py", line 89, in map_httpcore_exceptions
    raise mapped_exc(message) from exc
httpx.ConnectError: [Errno 111] Connection refused

After reading about ollama, I tried to change the OLLAMA_HOST env variable:

import os
print(os.environ.get('OLLAMA_HOST'))
None
os.environ['OLLAMA_HOST'] = "http://localhost:11435"
and same error again, which I kind of expected, since the ollama library should handle this via the host parameter for Client.

The same happens in a Windows machine, but in this case the error is a bit different:

Errno 10061 : No connection could be made because the target machine actively refused it

[frieda-huang]

Same issue. In addition, when I run the script, it sometimes works and sometimes doesn't.

[TaosLezz]

You need to check message "Ollama is running" in http://localhost:11434/. If you don't have this message, you need to open cmd and run the command ollama run "yourmodel"

[RXZAN]

I also meet the same issue.But I run ollama service in server side , when I curl http://my_server_ip:11434/ in my local machine, it can show "Ollama is running"

[ParthSareen]

Hey @tdimitch - this seems to be a connection issue between the deployed Ollama instance and the python code. Is this all running locally or is there a cloud-deployed instance?