Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Connection issues w/ 1.3.4 #29

Open
sboutros opened this issue Nov 15, 2024 · 12 comments
Open

Connection issues w/ 1.3.4 #29

sboutros opened this issue Nov 15, 2024 · 12 comments

Comments

@sboutros
Copy link

sboutros commented Nov 15, 2024 •
edited
Loading

Long time user - noticed a few nodes down that were auto-updated to v1.3.x
No key or config changes on either side.

Failures in the logs point to incorrect/missing keys.
In the logs/prepared ssh command I see "-i /data/ssh_keys/autossh_rsa_key" but that directory/file does not exist.

What am I missing, considering it was previously working.
I just ran a force_keygen and it does indicate that dir/file should exist yet it does not.
"Your identification has been saved in /data/ssh_keys/autossh_rsa_key"
I do have a /data/host_keys directory but that's it.

I've also uninstalled/reinstalled and manually created /data/ssh_keys/ + force_keygen - no luck.

Core = 2024.9.3
Supervisor = 2024.11.3
Operating System = 13.1
Frontend = 20240909.1

Seeting this at the end of each connection try in the logs:
debug1: remote forward failure for: listen 127.0.0.1:8123, connect 172.30.32.1:8123
Error: remote port forwarding failed for listen port 8123
@ThomDietrich
Copy link
Owner

Hello @sboutros,
Sorry to hear! The major change in 1.3.x was the remote_forwarding refactor, plus some additional changes to ensure and keep proper SSH connection+forwarding.

3212fc8...master

There were no changes to the SSH keys. I can't think of a reason why this would be an issue.

As for immediate mitigation: I don't know why you would need to deal with the directory the keys are stored in. They should be managed by the addon and the docker virtualization (they are NOT in your typical configuration folder, not sure if that is clear?). Truth be told, I myself have never touched the filesystem in the background.
Could you please uninstall, rebuild, reinstall, start the addon and post your full redacted log here? Thanks

@sboutros
Copy link
Author

sboutros commented Nov 16, 2024 via email

@ThomDietrich
Copy link
Owner

Not the install process. Simply the log from your first execution which you can find on the HA web UI

@sboutros
Copy link
Author 

**Ok - Ddeleted, Restarted, Installed, configured, keygen=true, started, stopped, keygen=false, started: Logged:**

s6-rc: info: service s6rc-oneshot-runner successfully started
s6-rc: info: service fix-attrs: starting
s6-rc: info: service fix-attrs successfully started
s6-rc: info: service legacy-cont-init: starting
s6-rc: info: service legacy-cont-init successfully started
s6-rc: info: service legacy-services: starting
s6-rc: info: service legacy-services successfully started
[21:03:15] INFO: Starting initialization script
The container is connected via the following IP addresses:
1: lo    inet 127.0.0.1/8 scope host lo\       valid_lft forever preferred_lft forever
1: lo    inet6 ::1/128 scope host noprefixroute \       valid_lft forever preferred_lft forever
2: enp1s0    inet 10.0.3.38/24 brd 10.0.3.255 scope global dynamic noprefixroute enp1s0\       valid_lft 2286sec preferred_lft 2286sec
2: enp1s0    inet6 fe80::b77b:f1ea:5ad2:fffe/64 scope link noprefixroute \       valid_lft forever preferred_lft forever
4: docker0    inet 172.30.232.1/23 brd 172.30.233.255 scope global docker0\       valid_lft forever preferred_lft forever
4: docker0    inet6 fe80::42:aeff:fec0:6a25/64 scope link \       valid_lft forever preferred_lft forever
5: hassio    inet 172.30.32.1/23 brd 172.30.33.255 scope global hassio\       valid_lft forever preferred_lft forever
5: hassio    inet6 fe80::42:78ff:fe1f:995a/64 scope link \       valid_lft forever preferred_lft forever
7: veth713478a    inet6 fe80::49:1dff:fe12:84a0/64 scope link \       valid_lft forever preferred_lft forever
15: vethfa1769b    inet6 fe80::7822:f5ff:fe36:f6be/64 scope link \       valid_lft forever preferred_lft forever
17: vethd59dc6e    inet6 fe80::1c8d:e3ff:fe73:9558/64 scope link \       valid_lft forever preferred_lft forever
21: veth88de045    inet6 fe80::7014:9bff:fec0:4c66/64 scope link \       valid_lft forever preferred_lft forever
23: vethbe8613e    inet6 fe80::7483:3fff:fe87:1762/64 scope link \       valid_lft forever preferred_lft forever
49: veth702fa5f    inet6 fe80::400b:d9ff:fe96:8e7a/64 scope link \       valid_lft forever preferred_lft forever
73: veth34224d8    inet6 fe80::6c5b:b8ff:fee6:9335/64 scope link \       valid_lft forever preferred_lft forever
75: veth4383e95    inet6 fe80::b866:d1ff:fe64:ec84/64 scope link \       valid_lft forever preferred_lft forever
[21:03:15] INFO: Deleting existing key pair due to set 'force_keygen'
[21:03:15] WARNING: Do not forget to unset 'force_keygen' in your add-on configuration
[21:03:15] INFO: No previous key pair found
Generating public/private ed25519 key pair.
Your identification has been saved in /data/ssh_keys/autossh_rsa_key
Your public key has been saved in /data/ssh_keys/autossh_rsa_key.pub
The key fingerprint is:
SHA256:ABC hassio-setup-via-autossh
The key's randomart image is:
+--[ED25519 256]--+
| ..+O+. .        |
|...o.E o +       |
|o.o . = +        |
|+o   o   o       |
|o=  . + S        |
|* o o+ B         |
|.o =..* +        |
|o +..=.o         |
| =.   +o.        |
+----[SHA256]-----+
[21:03:15] INFO: The public key is:
ssh-ed25519 ABC123-KEY
 hassio-setup-via-autossh
[21:03:15] WARNING: Add this key to '~/.ssh/authorized_keys' on your remote server now!
[21:03:15] WARNING: Please restart add-on when done. Exiting...
s6-rc: info: service legacy-services: stopping
s6-rc: info: service legacy-services successfully stopped
s6-rc: info: service legacy-cont-init: stopping
s6-rc: info: service legacy-cont-init successfully stopped
s6-rc: info: service fix-attrs: stopping
s6-rc: info: service fix-attrs successfully stopped
s6-rc: info: service s6rc-oneshot-runner: stopping
s6-rc: info: service s6rc-oneshot-runner successfully stopped







7: veth713478a    inet6 fe80::49:1dff:fe12:84a0/64 scope link \       valid_lft forever preferred_lft forever
15: vethfa1769b    inet6 fe80::7822:f5ff:fe36:f6be/64 scope link \       valid_lft forever preferred_lft forever
17: vethd59dc6e    inet6 fe80::1c8d:e3ff:fe73:9558/64 scope link \       valid_lft forever preferred_lft forever
21: veth88de045    inet6 fe80::7014:9bff:fec0:4c66/64 scope link \       valid_lft forever preferred_lft forever
23: vethbe8613e    inet6 fe80::7483:3fff:fe87:1762/64 scope link \       valid_lft forever preferred_lft forever
49: veth702fa5f    inet6 fe80::400b:d9ff:fe96:8e7a/64 scope link \       valid_lft forever preferred_lft forever
73: veth34224d8    inet6 fe80::6c5b:b8ff:fee6:9335/64 scope link \       valid_lft forever preferred_lft forever
75: veth4383e95    inet6 fe80::b866:d1ff:fe64:ec84/64 scope link \       valid_lft forever preferred_lft forever
[21:07:00] INFO: Authentication key pair restored
[21:07:00] INFO: The public key used by this add-on is:
ssh-ed25519 xxx hassio-setup-via-autossh
[21:07:00] INFO: If not done so already, please add the key to '~/.ssh/authorized_keys' on your remote server
[21:07:00] INFO: Testing Home Assistant socket '172.30.32.1:8123' on the local system... Web frontend reachable over HTTP
[21:07:03] INFO: Testing SSH service on 'www.host.com:2222'... SSH service reachable on remote server
[21:07:03] INFO: Remote server host keys:
# www.host.com:2222 SSH-2.0-OpenSSH_8.9p1 Ubuntu-3ubuntu0.10
[www.host.com]:2222 ssh-rsa xxxx
# www.host.com:2222 SSH-2.0-OpenSSH_8.9p1 Ubuntu-3ubuntu0.10
# www.host.com:2222 SSH-2.0-OpenSSH_8.9p1 Ubuntu-3ubuntu0.10
# www.host.com:2222 SSH-2.0-OpenSSH_8.9p1 Ubuntu-3ubuntu0.10
# www.host.com:2222 SSH-2.0-OpenSSH_8.9p1 Ubuntu-3ubuntu0.10
[www.host.com]:2222 ecdsa-sha2-nistp256 xxx
[www.host.com]:2222 ssh-ed25519 xxx
[21:07:11] INFO: Preparations done.
[21:07:11] INFO: Executing command: /usr/bin/autossh -M 0 -o ServerAliveInterval=30 -o ServerAliveCountMax=3 -o StrictHostKeyChecking=no -o ExitOnForwardFailure=yes -p 2222 -t -t -i /data/ssh_keys/autossh_rsa_key [email protected] -R 127.0.0.1:8123:172.30.32.1:8123 -R 127.0.0.1:8124:172.30.32.1:8123 -v -N
autossh 1.4g
OpenSSH_9.7p1, OpenSSL 3.3.2 3 Sep 2024
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 22: include /etc/ssh/ssh_config.d/*.conf matched no files
debug1: Connecting to www.host.com [72.14.176.52] port 2222.
debug1: Connection established.
debug1: identity file /data/ssh_keys/autossh_rsa_key type 3
debug1: identity file /data/ssh_keys/autossh_rsa_key-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_9.7
debug1: Remote protocol version 2.0, remote software version OpenSSH_8.9p1 Ubuntu-3ubuntu0.10
debug1: compat_banner: match: OpenSSH_8.9p1 Ubuntu-3ubuntu0.10 pat OpenSSH* compat 0x04000000
debug1: Authenticating to www.host.com:2222 as 'user'
debug1: load_hostkeys: fopen /root/.ssh/known_hosts2: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: [email protected]
debug1: kex: host key algorithm: ssh-ed25519
debug1: kex: server->client cipher: [email protected] MAC: <implicit> compression: none
debug1: kex: client->server cipher: [email protected] MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: SSH2_MSG_KEX_ECDH_REPLY received
debug1: Server host key: ssh-ed25519 SHA256:dP58FJaJDV4OnmrJnFW41Y/patQ4Heqko6vrC8w1V1g
debug1: load_hostkeys: fopen /root/.ssh/known_hosts2: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
debug1: Host '[www.host.com]:2222' is known and matches the ED25519 host key.
debug1: Found key in /root/.ssh/known_hosts:1
debug1: ssh_packet_send2_wrapped: resetting send seqnr 3
debug1: rekey out after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: ssh_packet_read_poll2: resetting read seqnr 3
debug1: SSH2_MSG_NEWKEYS received
debug1: rekey in after 134217728 blocks
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_ext_info_client_parse: server-sig-algs=<ssh-ed25519,[email protected],ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,[email protected],[email protected]>
debug1: kex_ext_info_check_ver: [email protected]=<0>
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
debug1: Will attempt key: /data/ssh_keys/autossh_rsa_key ED25519 SHA256:xxx explicit
debug1: Offering public key: /data/ssh_keys/autossh_rsa_key ED25519 SHA256:xxx explicit
debug1: Server accepts key: /data/ssh_keys/autossh_rsa_key ED25519 SHA256:xxx explicit
Authenticated to www.host.com ([72.14.176.52]:2222) using "publickey".
debug1: Remote connections from 127.0.0.1:8123 forwarded to local address 172.30.32.1:8123
debug1: Remote connections from 127.0.0.1:8124 forwarded to local address 172.30.32.1:8123
debug1: ssh_init_forwarding: expecting replies for 2 forwards
debug1: Requesting [email protected]
debug1: Entering interactive session.
debug1: pledge: filesystem
debug1: client_input_global_request: rtype [email protected] want_reply 0
debug1: client_input_hostkeys: searching /root/.ssh/known_hosts for [www.host.com]:2222 / (none)
debug1: client_input_hostkeys: searching /root/.ssh/known_hosts2 for [www.host.com]:2222 / (none)
debug1: client_input_hostkeys: hostkeys file /root/.ssh/known_hosts2 does not exist
debug1: Remote: /home/user/.ssh/authorized_keys:576: key options: agent-forwarding port-forwarding pty user-rc x11-forwarding
debug1: Remote: /home/user/.ssh/authorized_keys:576: key options: agent-forwarding port-forwarding pty user-rc x11-forwarding
debug1: remote forward failure for: listen 127.0.0.1:8123, connect 172.30.32.1:8123
Error: remote port forwarding failed for listen port 8123
s6-rc: info: service legacy-services: stopping
s6-rc: info: service legacy-services successfully stopped
s6-rc: info: service legacy-cont-init: stopping
s6-rc: info: service legacy-cont-init successfully stopped
s6-rc: info: service fix-attrs: stopping
s6-rc: info: service fix-attrs successfully stopped
s6-rc: info: service s6rc-oneshot-runner: stopping
s6-rc: info: service s6rc-oneshot-runner successfully stopped

@ThomDietrich
Copy link
Owner

[21:07:00] INFO: Authentication key pair restored

debug1: Server accepts key: /data/ssh_keys/autossh_rsa_key ED25519 SHA256:xxx explicit

All looks good to me. Did this resolve your issue?

@sboutros
Copy link
Author

No connection at the end of that - isn't that what the last 9 or so log lines indicate?
And prior to that - the "Error: remote port forwarding failed for listen port 8123"
I have no ssh connection and the add-on is stopped.

@ThomDietrich
Copy link
Owner

ThomDietrich commented Nov 19, 2024 •
edited
Loading

The connection succeeded but the port forwarding fails.

debug1: Server accepts key: /data/ssh_keys/autossh_rsa_key ED25519 SHA256:xxx explicit
...
Error: remote port forwarding failed for listen port 8123

Because of that the addon fails and stops. That is intended behavior. The problem is that your port forwarding does not work. Is the port blocked on the remote server?

Also I can see the forwarding line two times:

debug1: Remote connections from 127.0.0.1:8123 forwarded to local address 172.30.32.1:8123
debug1: Remote connections from 127.0.0.1:8124 forwarded to local address 172.30.32.1:8123

Are you sure that you have updated your configuration using the new remote_ip_address and remote_port correctly?

@bbkr
Copy link

bbkr commented Nov 30, 2024

I had to set remote_ip_address = *, remote_port to 8123 and remove old *:8123:localhost:8123 forwarding rule.

Mentioning this for future visittors because it is not obvious that you can enter * in this field..

@ThomDietrich
Copy link
Owner

Hey @bbkr you should mention your intent behind using "*". If you do not know what it means, it probably doesn't do what you wanted.

Did you read the DOCS?

https://github.com/ThomDietrich/home-assistant-addons/blob/master/autossh/DOCS.md#option-remote_ip_address

remote_ip_address should ideally be 127.0.0.1, or any public IP you want to open the socket on. Not just "any".

@bbkr
Copy link

bbkr commented Nov 30, 2024 •
edited
Loading

In my setup I have:

Summer house: with HAOS machine accessing internet through LTE modem with no public IP.
Regular house: with Synology DSM + public IP and regularhouse.example domain.

I made summer house accessible under summerhouse.regularhouse.example with following steps:

  1. Installed this addon which gave me SSH keys.
  2. On DSM installed linuxserver/openssh-server + linuxserver/mods:openssh-server-ssh-tunnel Docker images.
  3. Configured 2 to mount some filesystem directory as .ssh and added SSH keys from 1. there to authorized_keys file.
  4. Configured 2 to expose ports 2222 and 8123.
  5. Configured router to forward external port 2222 to DSM port 2222, effectively forwarding to Docker openssh server.
  6. Configured reverse proxy rules on DSM from summerhouse.regularhouse.example port 443 to DSM internal IP port 8123, effectively proxying to Docker SSH server.
  7. Enabled websocket headers on 6.
  8. Generated SSL certificate for summerhouse.regularhouse.example.
  9. Added use_x_forwarded_for: true and trusted proxies with internal IP of HAOS machine to http section in configuration.yaml.
  10. Started this addon with hostname=regularhouse.example, ssh_port=2222 , remote_ip_address=*, remote_port:8123.

So connection to summerhouse.regularhouse.example goes to my router through port 443, then through reverse proxy to Docker SSH server to port 8123,, then through reverse TCP tunnel to port 2222, then through open SSH connection to HAOS.

So in my case * is OK, because I want tunnel to be passing packets both from subdomain (when I'm outside) and directly from DSM on port 8123 on regular house network (when at home).

@ThomDietrich
Copy link
Owner

ThomDietrich commented Dec 1, 2024 •
edited
Loading

Sounds good to me! My setup is actually quite similar (https://github.com/ThomDietrich/home-assistant-addons/blob/master/autossh/DOCS.md#option-3-docker-based-solution), except for the need to access on the regular house network. I always go through the public domain. Why make the difference?

Anyway, in your case the * makes sense.

@ThomDietrich
Copy link
Owner

@sboutros is your issue resolved?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@bbkr @sboutros @ThomDietrich