Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] s3gw-ui: unable to access UI due to admin ops verifying cert #737

Closed
jecluis opened this issue Oct 4, 2023 · 0 comments · Fixed by s3gw-tech/s3gw-ui#269
Closed

[BUG] s3gw-ui: unable to access UI due to admin ops verifying cert #737

jecluis opened this issue Oct 4, 2023 · 0 comments · Fixed by s3gw-tech/s3gw-ui#269
Assignees
@jecluis
Labels
area/ui User Interface kind/bug Something isn't working priority/0 Needs to go into the next release or force a patch
Milestone
v0.22.0

Comments

@jecluis
Copy link
Contributor

jecluis commented Oct 4, 2023

The s3gw-ui backend is verifying certificates when sending admin ops api requests to the s3gw service. This is not a problem on local clusters not relying on https, nor or clusters with properly signed certificates (e.g., let's encrypt), but for self-signed certificates this becomes a problem.

This can easily be reproduced by deploying s3gw's helm chart, accessing the UI, and trying to login. How this was not found out during testing is weird, but I guess this proves that we need more resilient integration testing.

The UI will show a rather cryptic error:

image

While the logs will essentially be a big backtrace:

INFO:     Started server process [1]            
INFO:     Waiting for application startup.                                                                                     
INFO:     2023-10-04 06:11:53 -- s3gw_ui_backend -- Starting s3gw-ui backend                                                   
INFO:     2023-10-04 06:11:53 -- config -- Servicing s3gw at https://s3gw-lolwhat-s3gw.172.20.20.5.omg.howdoi.website          
INFO:     2023-10-04 06:11:53 -- on -- Application startup complete.                                                           
INFO:     2023-10-04 06:11:53 -- on -- Application startup complete.                                                           
INFO:     2023-10-04 06:11:53 -- server -- Uvicorn running on http://0.0.0.0:8080 (Press CTRL+C to quit)
INFO:     2023-10-04 06:11:53 -- server -- Uvicorn running on http://0.0.0.0:8080 (Press CTRL+C to quit) 
INFO:     2023-10-04 06:12:28 -- h11_impl -- 10.42.0.8:53356 - "GET / HTTP/1.1" 200                                            
INFO:     2023-10-04 06:12:28 -- h11_impl -- 10.42.0.8:53356 - "GET / HTTP/1.1" 200                       
INFO:     2023-10-04 06:12:28 -- h11_impl -- 10.42.0.8:53356 - "GET /runtime.d615879041906b88.js HTTP/1.1" 200
INFO:     2023-10-04 06:12:28 -- h11_impl -- 10.42.0.8:53356 - "GET /runtime.d615879041906b88.js HTTP/1.1" 200                 
INFO:     2023-10-04 06:12:28 -- h11_impl -- 10.42.0.8:39290 - "GET /polyfills.b9a16c87a52c6b7a.js HTTP/1.1" 200               
INFO:     2023-10-04 06:12:28 -- h11_impl -- 10.42.0.8:39290 - "GET /polyfills.b9a16c87a52c6b7a.js HTTP/1.1" 200               
INFO:     2023-10-04 06:12:28 -- h11_impl -- 10.42.0.8:39290 - "GET /main.7c0f5c0f1cdde25a.js HTTP/1.1" 200                    
INFO:     2023-10-04 06:12:28 -- h11_impl -- 10.42.0.8:39290 - "GET /main.7c0f5c0f1cdde25a.js HTTP/1.1" 200
INFO:     2023-10-04 06:12:28 -- h11_impl -- 10.42.0.8:39290 - "GET /styles.13e1dfb63f66b3b1.css HTTP/1.1" 200
INFO:     2023-10-04 06:12:28 -- h11_impl -- 10.42.0.8:39290 - "GET /styles.13e1dfb63f66b3b1.css HTTP/1.1" 200
INFO:     2023-10-04 06:12:28 -- h11_impl -- 10.42.0.8:53356 - "GET /lato-latin-400-normal.b7ffde2383bb16ba.woff2 HTTP/1.1" 200
INFO:     2023-10-04 06:12:28 -- h11_impl -- 10.42.0.8:53356 - "GET /lato-latin-400-normal.b7ffde2383bb16ba.woff2 HTTP/1.1" 200
INFO:     2023-10-04 06:12:29 -- h11_impl -- 10.42.0.8:39290 - "GET /assets/app.config.json HTTP/1.1" 200                      
INFO:     2023-10-04 06:12:29 -- h11_impl -- 10.42.0.8:39290 - "GET /assets/app.config.json HTTP/1.1" 200                      
INFO:     2023-10-04 06:12:29 -- h11_impl -- 10.42.0.8:53356 - "GET /assets/i18n/en_US.json HTTP/1.1" 200                      
INFO:     2023-10-04 06:12:29 -- h11_impl -- 10.42.0.8:53356 - "GET /assets/i18n/en_US.json HTTP/1.1" 200                      
INFO:     2023-10-04 06:12:29 -- h11_impl -- 10.42.0.8:53356 - "GET /assets/images/login-landscape.svg HTTP/1.1" 200           
INFO:     2023-10-04 06:12:29 -- h11_impl -- 10.42.0.8:53356 - "GET /assets/images/login-landscape.svg HTTP/1.1" 200
INFO:     2023-10-04 06:12:29 -- h11_impl -- 10.42.0.8:39290 - "GET /materialdesignicons-webfont.68358e875826bc5f.woff2?v=7.2.9
6 HTTP/1.1" 200                                                                                                                
INFO:     2023-10-04 06:12:29 -- h11_impl -- 10.42.0.8:39290 - "GET /materialdesignicons-webfont.68358e875826bc5f.woff2?v=7.2.9
6 HTTP/1.1" 200                                                                                                                
INFO:     2023-10-04 06:12:29 -- h11_impl -- 10.42.0.8:39290 - "GET /favicon_180x180.png HTTP/1.1" 200                         
INFO:     2023-10-04 06:12:29 -- h11_impl -- 10.42.0.8:39290 - "GET /favicon_180x180.png HTTP/1.1" 200                         
INFO:     2023-10-04 06:12:29 -- h11_impl -- 10.42.0.8:53356 - "GET /favicon.svg HTTP/1.1" 200
INFO:     2023-10-04 06:12:29 -- h11_impl -- 10.42.0.8:53356 - "GET /favicon.svg HTTP/1.1" 200                        
INFO:     2023-10-04 06:12:33 -- h11_impl -- 10.42.0.8:53356 - "GET /auth/authenticate HTTP/1.1" 500
INFO:     2023-10-04 06:12:33 -- h11_impl -- 10.42.0.8:53356 - "GET /auth/authenticate HTTP/1.1" 500                           
ERROR:    2023-10-04 06:12:33 -- h11_impl -- Exception in ASGI application                                                     
Traceback (most recent call last):                                                                                             
  File "/usr/local/lib/python3.11/site-packages/uvicorn/protocols/http/h11_impl.py", line 429, in run_asgi                     
    result = await app(  # type: ignore[func-returns-value]                                                                    
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                    
  File "/usr/local/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 78, in __call__                     
    return await self.app(scope, receive, send)                                                                                
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                
  File "/usr/local/lib/python3.11/site-packages/fastapi/applications.py", line 289, in __call__
    await super().__call__(scope, receive, send)                                                                               
  File "/usr/local/lib/python3.11/site-packages/starlette/applications.py", line 122, in __call__                              
    await self.middleware_stack(scope, receive, send)                                                                          
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 184, in __call__                         
    raise exc                                                                                                                  
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 162, in __call__
    await self.app(scope, receive, _send)                                                                                      
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 79, in __call__                      
    raise exc                                                                                                                  
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 68, in __call__                      
    await self.app(scope, receive, sender)                                                                                     
  File "/usr/local/lib/python3.11/site-packages/fastapi/middleware/asyncexitstack.py", line 20, in __call__                    
    raise e
  File "/usr/local/lib/python3.11/site-packages/fastapi/middleware/asyncexitstack.py", line 17, in __call__                    
    await self.app(scope, receive, send)                                                                                       
  File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 718, in __call__
    await route.handle(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 443, in handle
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/fastapi/applications.py", line 289, in __call__
    await super().__call__(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/starlette/applications.py", line 122, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 184, in __call__
    raise exc
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 79, in __call__
    raise exc
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 68, in __call__
    await self.app(scope, receive, sender)
  File "/usr/local/lib/python3.11/site-packages/fastapi/middleware/asyncexitstack.py", line 20, in __call__
    raise e
  File "/usr/local/lib/python3.11/site-packages/fastapi/middleware/asyncexitstack.py", line 17, in __call__
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 718, in __call__
    await route.handle(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 276, in handle
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 66, in app
    response = await func(request)
               ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 273, in app
    raw_response = await run_endpoint_function(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 190, in run_endpoint_function
    return await dependant.call(**values)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/srv/backend/api/auth.py", line 43, in authenticate
    await admin_ops_users.get_user_info(
  File "/srv/backend/admin_ops/users.py", line 53, in get_user_info
    res = await do_request(
          ^^^^^^^^^^^^^^^^^
  File "/srv/backend/admin_ops/__init__.py", line 93, in do_request
    return await send_request(req)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/srv/backend/admin_ops/__init__.py", line 66, in send_request
    res: httpx.Response = await client.send(req)
                          ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/httpx/_client.py", line 1617, in send
    response = await self._send_handling_auth(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/httpx/_client.py", line 1645, in _send_handling_auth
    response = await self._send_handling_redirects(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/httpx/_client.py", line 1682, in _send_handling_redirects
    response = await self._send_single_request(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/httpx/_client.py", line 1719, in _send_single_request
    response = await transport.handle_async_request(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/httpx/_transports/default.py", line 353, in handle_async_request
    resp = await self._pool.handle_async_request(req)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/httpcore/_async/connection_pool.py", line 262, in handle_async_request
    raise exc
  File "/usr/local/lib/python3.11/site-packages/httpcore/_async/connection_pool.py", line 245, in handle_async_request
    response = await connection.handle_async_request(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/httpcore/_async/connection.py", line 92, in handle_async_request
    raise exc
  File "/usr/local/lib/python3.11/site-packages/httpcore/_async/connection.py", line 69, in handle_async_request
    stream = await self._connect(request)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/httpcore/_async/connection.py", line 149, in _connect
    stream = await stream.start_tls(**kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/httpcore/_backends/anyio.py", line 78, in start_tls
    raise exc
  File "/usr/local/lib/python3.11/site-packages/httpcore/_backends/anyio.py", line 69, in start_tls
    ssl_stream = await anyio.streams.tls.TLSStream.wrap(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/anyio/streams/tls.py", line 122, in wrap
    await wrapper._call_sslobject_method(ssl_object.do_handshake)
  File "/usr/local/lib/python3.11/site-packages/anyio/streams/tls.py", line 130, in _call_sslobject_method
    result = func(*args)
             ^^^^^^^^^^^
  File "/usr/local/lib/python3.11/ssl.py", line 979, in do_handshake
    self._sslobj.do_handshake() 
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate (_ssl.c:1006)
@jecluis jecluis added kind/bug Something isn't working priority/0 Needs to go into the next release or force a patch area/ui User Interface labels Oct 4, 2023
@jecluis jecluis added this to the v0.22.0 milestone Oct 4, 2023
@jecluis jecluis self-assigned this Oct 4, 2023
@jecluis jecluis added this to S3GW Oct 4, 2023
@github-project-automation github-project-automation bot moved this to Backlog in S3GW Oct 4, 2023
@github-actions github-actions bot added the triage/waiting Waiting for triage label Oct 4, 2023
@jecluis jecluis moved this from Backlog to In Progress 🏗️ in S3GW Oct 4, 2023
@jecluis jecluis removed the triage/waiting Waiting for triage label Oct 4, 2023
jecluis referenced this issue in jecluis/s3gw-ui Oct 4, 2023
@jecluis
backend/admin_ops: don't verify certs on send request
4a59061 
Fixes: aquarist-labs/s3gw#737

Signed-off-by: Joao Eduardo Luis <[email protected]>
This was referenced Oct 4, 2023
Merged
Open
@github-project-automation github-project-automation bot moved this from In Progress 🏗️ to Done in S3GW Oct 5, 2023
@vmoutoussamy vmoutoussamy mentioned this issue Nov 15, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jecluis jecluis

Labels
area/ui User Interface kind/bug Something isn't working priority/0 Needs to go into the next release or force a patch
Projects
None yet
Milestone
v0.22.0
Development

Successfully merging a pull request may close this issue.

backend/admin_ops: don't verify certs on send request jecluis/s3gw-ui
1 participant
@jecluis