Access API Server 502 after using HTTPS proxy

[wisheen]

General remarks

This form is to report bugs. For general usage questions refer to our Slack channel
KubeSphere-users

Describe the bug
Using Nginx as a reverse proxy, when configured for HTTPS access, All /api/xx requests report 502 errors， But /kapis/xxx are fine。Using ip:30880 is also correct。

For UI issues please also add a screenshot that shows the issue.

Versions used(KubeSphere/Kubernetes)
KubeSphere: 2.1.0
Kubernetes: (If KubeSphere installer used, you can skip this)

Environment
How many nodes and their hardware configuration:
3 masters: 2cpu/4g
3 nodes: 4cpu/16g

(and other info are welcomed to help us debugging)

To Reproduce
Steps to reproduce the behavior:

Expected behavior
A clear and concise description of what you expected to happen.
The console supports HTTPS proxy access

[leoendless]

@zryfish Please help to resolve the problem.

[rayzhou2017]

Please check if it still exists in 3.0.0 @zryfish

[zryfish]

Can you give more specific details? What's ther error message, what's your proxy configuration looks like?

[wisheen]

Nginx settings:

server {
    listen 443 ssl;
    server_name  kubesphere.xxxx.com;
    ssl_certificate /etc/nginx/certs/xxxx.com.pem;
    ssl_certificate_key /etc/nginx/certs/xxxx.com.key;
    add_header Strict-Transport-Security "max-age=0; includeSubDomains" always;
    resolver 100.100.2.138 100.100.2.136;

    location / {
        #dyups_shm_zone_size 10MB;
        proxy_redirect      off;
        proxy_http_version  1.1;
        proxy_set_header    Host             $host;
        proxy_set_header    X-Real-IP        $remote_addr;
        proxy_set_header    X-Forwarded-For  $proxy_add_x_forwarded_for;
        proxy_set_header    X-Forwarded-Proto  $scheme;
        proxy_set_header    X-Nginx-Proxy true;
        proxy_next_upstream error timeout invalid_header http_500 http_502 http_503 http_504;
        proxy_max_temp_file_size 0;
        proxy_connect_timeout      130;
        proxy_send_timeout         130;
        proxy_read_timeout         130;
        proxy_buffer_size          4k;
        proxy_buffers              4 32k;
        proxy_busy_buffers_size    64k;
        proxy_temp_file_write_size 64k;
        proxy_buffering            off;
        proxy_set_header    Upgrade $http_upgrade;
        proxy_set_header    Connection "upgrade";

        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_redirect http:// $scheme://;

        proxy_ssl_verify off;
        proxy_ssl_session_reuse on;

        proxy_pass  http://192.168.0.95:30880;
    }

}

Request：
https://kubesphere.xxx.com/infrastructure/nodes/master1

Request URL: https://kubesphere.xxxx.com/api/v1/nodes/master1
Request Method: GET
Status Code: 502 Bad Gateway
Remote Address: 192.168.0.95:443
Referrer Policy: no-referrer-when-downgrade
Connection: keep-alive
Content-Length: 16
Content-Type: text/plain; charset=utf-8
Date: Thu, 23 Jul 2020 14:17:05 GMT
Server: openresty/1.15.8.2
Strict-Transport-Security: max-age=0; includeSubDomains
Vary: Accept-Encoding
x-content-type-options: nosniff
Accept: /
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9,zh-CN;q=0.8,zh;q=0.7
Connection: keep-alive
content-type: application/json
Cookie: lang=en; currentUser=admin; kubesphere:sess.sig=qiwJX8EP9SGEndmzkCtbgtC4m0k; token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJlbWFpbCI6ImFkbWluQGt1YmVzcGhlcmUuaW8iLCJpYXQiOjE1OTU1MTM2OTMsInVzZXJuYW1lIjoiYWRtaW4ifQ.ji1Wvgv5dU9xqiDG2vkwbgCT-qMlqTfyvjO28zmTk6o
Host: kubesphere.xxxx.com
Referer: https://kubesphere.xxxx.com/infrastructure/nodes/master1
Sec-Fetch-Dest: empty
Sec-Fetch-Mode: cors
Sec-Fetch-Site: same-origin
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/83.0.4103.61 Chrome/83.0.4103.61 Safari/537.36

[wisheen]

The old version 2.0.2 is correct

[wisheen]

It's problem with websocket setttings.

delete sesetion:

    server {
        # proxy_set_header    Upgrade $http_upgrade;
        # proxy_set_header    Connection "upgrade";
    }

or:

    map $http_upgrade $connection_upgrade {
        default upgrade;
        ''      close;
    }

    server {
        listen 443 ssl;
        ...
        proxy_set_header    Upgrade $http_upgrade;
        proxy_set_header    Connection $connection_upgrade;
       ...
    }

[AnsNid]

proxy_ssl_server_name on;