Stream server component (serial-over-wifi) #660

oxan · 2020-03-30T13:40:41Z

Describe the problem you have/What new integration you would like
A server that allows bidirectional communication with a serial port over WiFi. Now, I know this has been requested and rejected before (#213, #415, #619, #638), but I disagree with the given reasoning (and I have a patch :)):

Re #638: Plenty of HA integrations communicate over a serial port, and can be used with such a server to control remote devices. E.g. acer_projector, alarmdecoder, arduino, aurora_abb_powerone, dsmr, edl21, elv, enocean, etc.
Re #415: I've taken a quick look at some HA integrations using serial ports, and pretty much all of them just send some messages back and forth over the serial port. Nothing more fancy such as control sequences or hardware flow control. So I think 95% of the use cases can be captured with a quite simple implementation, and if it's made extensible the rest can use a custom component.
Re #213: I agree that its frontend and backend should be separated - but luckily ESPHome already has a Stream abstraction implemented by the UART interface. Thus, the backend is already there, and we only need a frontend to expose a Stream over TCP.
One other major benefit of integrating a serial-over-wifi bridge is that it'd allow a single ESP to be used as a serial bridge and for sensors. For example, I have an ESP with OpenTherm GW next to my central heating installation, but I also want to hookup some sensors to monitor power consumption.

Please describe your use case for this integration and alternatives you've tried:
See above. I want to hookup both a serial port and some other sensors (or switches, etc) to a single ESP integrated in Home Assistant.

Alternatives are various different ESP firmwares (ESPEasy, ESPLink); but neither of those allows me to easily integrate a sensor in HA.

External component
For now I've implemented this as a external component, which is available here. It creates a TCP server listening on port 6638, and relays all data between the connected clients and the serial port. It doesn't support any control sequences, telnet options or RFC 2217, just raw data (I don't think the fancy stuff is actually useful, very few devices require it and they aren't actually wired on the common ESP modules).

However, I think it'd be nice if this was available in ESPHome by default. I can create a PR to rework my custom component to integrate it in ESPHome itself (including configuration etc.) if you agree to accept this functionality.

The text was updated successfully, but these errors were encountered:

glmnet · 2020-04-17T12:52:25Z

I like this, if it's useful for you then it should be useful for other people.

The thing being so generic leads to lot of issues, but this might be considered for advanced users, or something like that, that's why Otto recommended this being a custom_component / cookbook thing, so anybody could modify it quickly.

For example your implementation allows for several clients connecting simultaneously to the bridge, this might not be what other people needs, then there will come the "option" to allow one connection and stick to it even if other connection arrives or another "option" to discard the active connection and the last request will win.

Initially I'd like it to be implemented as a cookbook thing, it is really useful because it explains how stuff works and allows you to start from tested code, see e.g. the Arduino Port Expander

Then there is the new custom_components folder method to implement ESPHome components, that's equally good for Cookbooks, and it allows for quickly addition to ESPHome codebase as it will be a matter of copying the files to the repository.

There were a few messages exchanged with Otto, Brandon and other users, Otto is trying to make ESPHome more like HA where many contributors are easily allowed to add their own stuff and "take responsibility" of it, but nothing come out just yet.

Anyway thanks for sharing the code!

Hedda · 2020-04-20T14:11:39Z

This request might be related to #688 = Request for "ESPHome support on Sonoff ZBBridge as a remote Zigbee adapter for Home Assistant's ZHA component via its zigpy and bellows libraries".

As I understand that ##688 would depend on some kind of stream server for Serial or UART over Wi-Fi.

Though while related I would not say that #660 replaced #688 and their ultimate goals are different.

zuidwijk · 2020-05-25T14:41:45Z

I would LOVE (yes capitals) this feature to be added to ESPHome. I'm building many different automation hardware for Home Assistant based on ESPHome, and this is something I really mis. Adding a second ESP only to use esp-link next to ESPHome is such a waste...

schnizki · 2020-08-12T19:41:47Z

I can confirm this custom component works with the new Sonoff ZBBridge and the ZHA intergation.
This is my config in ESPHome:

esphome:
  name: zigbee
  platform: ESP8266
  board: esp01_1m
  
  includes:
    - stream_server.h
    - stream_server.cpp

wifi:
  ssid: "HASSos"
  password: "***"

  ap:
    ssid: "Zigbee Fallback Hotspot"
    password: "***"
captive_portal:

logger:
 baud_rate: 0


api:
  password: "***"

ota:
  password: "***"

uart:
  id: uart_bus
  tx_pin: GPIO1
  rx_pin: GPIO3
  baud_rate: 115200
  
custom_component:
  - lambda: |-
      auto stream_server = new StreamServerComponent(id(uart_bus));
      return {stream_server};

ZHA dosent resolve .local adresses so just use the IP.

It would be awesome if this could be added to ESPHome.

zuidwijk · 2020-08-12T22:53:47Z

I can confirm this custom component works with the new Sonoff ZBBridge and the ZHA intergation.

This is my config in ESPHome:

esphome:

  name: zigbee

  platform: ESP8266

  board: esp01_1m

  

  includes:

    - stream_server.h

    - stream_server.cpp



wifi:

  ssid: "HASSos"

  password: "***"



  ap:

    ssid: "Zigbee Fallback Hotspot"

    password: "***"

captive_portal:



logger:

 baud_rate: 0





api:

  password: "***"



ota:

  password: "***"



uart:

  id: uart_bus

  tx_pin: GPIO1

  rx_pin: GPIO3

  baud_rate: 115200

  

custom_component:

  - lambda: |-

      auto stream_server = new StreamServerComponent(id(uart_bus));

      return {stream_server};

ZHA dosent resolve .local adresses so just use the IP.

It would be awesome if this could be added to ESPHome.

Nice! Is that the same as this one?

https://gist.github.com/phha/651a2eeb30c6d6ca0937b3394e2bd9e6

oxan · 2020-08-13T11:12:28Z

Nice! Is that the same as this one?

https://gist.github.com/phha/651a2eeb30c6d6ca0937b3394e2bd9e6

Yes, that's exactly the same code, someone forked it but made no changes for some reason.

zuidwijk · 2020-08-13T13:30:44Z

Nice! Is that the same as this one?
https://gist.github.com/phha/651a2eeb30c6d6ca0937b3394e2bd9e6

Yes, that's exactly the same code, someone forked it but made no changes for some reason.

Ah ok! I didn't saw the link in your first message, so I Googled it and found this one... found yours now :)

zuidwijk · 2020-08-15T13:56:39Z

Confirmed :)

esphome:
  name: p1test
  platform: ESP8266
  board: d1_mini

  includes:
    - stream_server.h
    - stream_server.cpp
    
wifi:
  ssid: "***"
  password: "***"

  # Enable fallback hotspot (captive portal) in case wifi connection fails
  ap:
    ssid: "P1Test Fallback Hotspot"
    password: "***"

captive_portal:

# Enable logging
logger:

# Enable Home Assistant API
api:

ota:

uart:
  id: uart_bus
  tx_pin: GPIO1
  rx_pin: GPIO3
  baud_rate: 115200
  
custom_component:
  - lambda: |-
      auto stream_server = new StreamServerComponent(id(uart_bus));
      return {stream_server};

Works like a charm!

Harald-P · 2020-08-16T18:13:38Z

I like this too, but i have beginner problems with compiling it:
In file included from src/stream_server.cpp:17:0:
src/stream_server.h:25:25: fatal error: ESPAsyncTCP.h: No such file or directory

So i searched ESPAsyncTCP.h but i found on github only a version for ESP 8266 (https://github.com/me-no-dev/ESPAsyncTCP or i must use an other one?) and for ESP32 i found only AsyncTCP.h (https://github.com/me-no-dev/AsyncTCP/tree/master/src).

Can you give me considerations how can i let compile this custom_component for ESP32?
Thanks in advance...

oxan · 2020-08-16T19:20:38Z

@Harald-P Try adding this:

esphome:
   libraries:
   - [email protected]

(note that this version is for ESPHome 1.14.5, newer/older versions might need a different version).

Usually this library is already included by the api or ota component.

Harald-P · 2020-08-16T19:49:27Z

hmmm, the version is 1.14.5 and i use api and ota ...

i tryed adding your consideration, but i recive now:

Compiling /data/cellar/.pioenvs/cellar/src/stream_server.cpp.o
In file included from src/main.cpp:27:0:
src/stream_server.h:25:25: fatal error: ESPAsyncTCP.h: No such file or directory

*********************************************************************
* Looking for ESPAsyncTCP.h dependency? Check our library registry!
*
* CLI  > platformio lib search "header:ESPAsyncTCP.h"
* Web  > https://platformio.org/lib/search?query=header:ESPAsyncTCP.h
*
*********************************************************************

compilation terminated.
*** [/data/cellar/.pioenvs/cellar/src/main.cpp.o] Error 1
In file included from src/stream_server.cpp:17:0:
src/stream_server.h:25:25: fatal error: ESPAsyncTCP.h: No such file or directory

*********************************************************************
* Looking for ESPAsyncTCP.h dependency? Check our library registry!
*
* CLI  > platformio lib search "header:ESPAsyncTCP.h"
* Web  > https://platformio.org/lib/search?query=header:ESPAsyncTCP.h
*
*********************************************************************

compilation terminated.
*** [/data/cellar/.pioenvs/cellar/src/stream_server.cpp.o] Error 1

i use hass.io and compile it over the esphome-dashboard in home assistant on a Raspberry pi...

oxan · 2020-08-16T20:10:05Z

@Harald-P Oh, I didn't see that you were on ESP-32. I haven't tried it with an ESP-32 personally. Can you try replacing ESPAsyncTCP.h with AsyncTCP.h in both files?

Harald-P · 2020-08-16T20:30:46Z

@oxan
i have done the change in stream_server.h and i have deletet this 2 rows:

   libraries:
   - [email protected]

The compilation concludet than without error, but now my ESP32 its not more working...
It is pingable but not available:
WARNING Initial connection failed. The ESP might not be connected to WiFi yet (Error connecting to 192.168.0.125: [Errno 111] Connection refused). Re-Trying in 11 seconds
INFO Connecting to cellar.local:6053 (192.168.0.125)

So i look tomorrow if i can install the old immage to bring the ESP32 back online...

mjkl-gh · 2020-10-22T19:57:37Z

@oxan
i have done the change in stream_server.h and i have deletet this 2 rows:
   libraries:
   - [email protected]
The compilation concludet than without error, but now my ESP32 its not more working...
It is pingable but not available:
WARNING Initial connection failed. The ESP might not be connected to WiFi yet (Error connecting to 192.168.0.125: [Errno 111] Connection refused). Re-Trying in 11 seconds
INFO Connecting to cellar.local:6053 (192.168.0.125)

So i look tomorrow if i can install the old immage to bring the ESP32 back online...

I'm also on esp32 (m5stickc) did you get any further with this?

edit:

tried it myself still crashes on esp32. Hook it up to serial to see 9 boot attempts:

[22:23:28][I][logger:166]: Log initialized
[22:23:28][C][ota:366]: There have been 7 suspected unsuccessful boot attempts.
[22:23:28][I][app:029]: Running through setup()...
[22:23:28][C][spi:022]: Setting up SPI bus...
[22:23:28][C][uart_esp32:072]: Setting up UART...
[22:23:28][C][streamserver:011]: Setting up stream server...
[22:23:29]/home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/queue.c:1442 (xQueueGenericReceive)- assert failed!
[22:23:29]abort() was called at PC 0x40089505 on core 1
[22:23:29]
[22:23:29]Backtrace: 0x4008d104:0x3ffb0df0 0x4008d335:0x3ffb0e10 0x40089505:0x3ffb0e30 0x4015451a:0x3ffb0e70 0x401547f6:0x3ffb0e90 0x40142e9c:0x3ffb0eb0 0x40142f05:0x3ffb0ed0 0x40144534:0x3ffb0ef0 0x40144620:0x3ffb0f20 0x400e764b:0x3ffb0f40 0x400e6d47:0x3ffb0f90 0x4016ccad:0x3ffb1000 0x4016cd69:0x3ffb1020 0x400dfad5:0x3ffb1040 0x400e6521:0x3ffb1090 0x400f4b8b:0x3ffb1fb0 0x40089819:0x3ffb1fd0
[22:23:29]
[22:23:29]Rebooting...
[22:23:29]ets Jun  8 2016 00:22:57
[22:23:29]
[22:23:29]rst:0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
[22:23:29]configsip: 188777542, SPIWP:0xee
[22:23:29]clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
[22:23:29]mode:DIO, clock div:2
[22:23:29]load:0x3fff0018,len:4
[22:23:29]load:0x3fff001c,len:952
[22:23:29]load:0x40078000,len:6084
[22:23:29]load:0x40080000,len:7944
[22:23:29]entry 0x40080310

after which it proceeds to safe mode

oxan · 2020-10-22T20:49:40Z

@MaartenKleijwegt Can you try inserting this->server_.setNoDelay(false); before line 31 in stream_server.cpp?

EDIT: Or actually, try this first: add before line 36 of stream_server.h:

float get_setup_priority() const override { return esphome::setup_priority::AFTER_WIFI; }

mjkl-gh · 2020-10-22T21:04:13Z

@MaartenKleijwegt Can you try inserting this->server_.setNoDelay(false); before line 31 in stream_server.cpp?

EDIT: Or actually, try this first: add before line 36 of stream_server.h:
float get_setup_priority() const override { return esphome::setup_priority::AFTER_WIFI; }

Thanks for the reply will try. was already looking into how esphome calls the app.setup loop. Figured it was setting up immediately after uart before wifi was setup. this #espressif/arduino-esp32#3467 hinted I was trying to setup some wifi component before wifi was initialized

Could you point to where I can read up on esphome setup priority and such? haven't been able to find much in the last hour

EDIT; tested and doesn't crash anymore. Able to connect through web server telnet (putty) and ESPHome the fix was:

float get_setup_priority() const override { return esphome::setup_priority::AFTER_WIFI; }

didnt't try the other

oxan · 2020-10-23T13:30:31Z

Ok, great! I've updated the gist to incorporate this fix.

Could you point to where I can read up on esphome setup priority and such?

The code, I guess.

mjkl-gh · 2020-10-23T16:36:16Z

Would it be apreciated/an idea for me to open a pull request to update https://esphome.io/custom/custom_component.html with this example?

that's where I would expect it and actually searched for it

oxan · 2020-10-23T16:45:17Z

@MaartenKleijwegt the get_setup_priority() you mean? yeah, that'd make sense I think.

duhow · 2020-12-26T15:18:30Z

Works like a charm with ESP32 and CC2530 (zigbee2mqtt) !
I hope to see this officially integrated with ESPHome, as I'm using it for BLE detection as well :)

yzg011 · 2020-12-26T15:27:57Z

thanks for your project，but when I use this for xiaomi zigbee button
sometimes not working

RoganDawes · 2021-01-22T08:07:21Z

This is great, thank you! I can see that this will work perfectly for my Texecom Premier alarm, as a replacement for their COM-IP or COM-WiFi devices. All I will need to add is a level converter.

It would be great if it was possible to customise the listening port in the configuration, though. Or should I just update the source?

Also, how would I go about limiting this to a single concurrent connection? In theory, more than one concurrent connection should not happen, but in practice, it would probably cause havoc if it did, so it's probably worth while preventing it!

Turns out that is simple enough, just add the following lines into the onClient method:

        if(this->clients_.size() > 0) {
            ESP_LOGW(TAG, "Got a second connection, dropping it!");
            tcpClient->close();
            return;
        }

So, my ultimate objective is to create an "intelligent" device, not purely a simple serial passthrough to allow for the Texecom app to do its own configuration/analytics/etc. Ideally, this device will implement enough of the Texecom SIMPLE protocol to allow it to interact with the panel, activate/deactivate zones, activate/deactivate the alarm itself, and report zone activations/violations both when the alarm is not set, and when it is. All of this would be available via the API, or via MQTT (I still need to figure out how the API works!)

The plan is to have the component interacting with the UART until a new TCP connection comes in, at which point the component will cleanly disconnect from the panel, and allow the external program exclusive access. Once the TCP connection disconnects, the component will resume interaction with the panel, reporting back to HA or any other MQTT subscriber.

Does this sound reasonable? I guess I should move this to a new issue, as it is far beyond a simple TCP-UART server now!

ioanfesteu · 2021-03-11T07:28:13Z

I can confirm too that is working like charm with ESP8266+CC2530 and remote zigbee2mqtt. Thank you!

Hedda · 2021-03-11T07:53:38Z

Tip is also to check out Tube's Zigbee Gateways open-source hardware by @tube0013 which runs ESPHome firmware by default.

Based on WT32-ETH01 (Wireless-Tag) ESP32 board which has wired Ethernet that makes for a much more stable serial connection.

He designed two variants; one has a Silicon Labs EFR32 Series 2 module and one that has a Texas Instruments CC2652P module.

https://github.com/tube0013/tube_gateways

zigpy/zigpy#584

https://www.tubeszb.com/shop/coordinators/2

oxan · 2021-05-19T17:21:53Z

I've made my custom component available as a new external component for ESPHome v1.18+ under my GitHub account:

external_components:
  - source: github://oxan/esphome-stream-server

stream_server:

Jpsy · 2021-11-21T10:46:12Z

Hi all,
I am experimenting with @oxan's custom component and hitting a wall.

Setup is:
CC2530 <-> ESP8266 + ESPHome + oxan component <-> WLAN <-> Zigbee2MQTT on HA 2021.11 on RPi

I have flashed the C2530 with Koenkk´s Z-Stack_Home_1.2/bin/default/CC2530_DEFAULT_20201127.zip.

When Zigbee2MQTT's Herdsman tries to connect, I can see that in the ESPHome log, but it finally fails after 20 seconds:

[11:42:19][D][streamserver:106]: New client connected from 192.168.178.180
[11:42:39][D][streamserver:055]: Client 192.168.178.180 disconnected

The corresponding Z2M log looks like this:

Zigbee2MQTT:info  2021-11-21 11:42:18: Starting zigbee-herdsman (0.13.164)
Zigbee2MQTT:error 2021-11-21 11:42:38: Error while starting zigbee-herdsman
Zigbee2MQTT:error 2021-11-21 11:42:38: Failed to start zigbee
Zigbee2MQTT:error 2021-11-21 11:42:38: Check https://www.zigbee2mqtt.io/information/FAQ.html#help-zigbee2mqtt-fails-to-start for possible solutions
Zigbee2MQTT:error 2021-11-21 11:42:38: Exiting...
Zigbee2MQTT:error 2021-11-21 11:42:39: Error: Failed to connect to the adapter (Error: SRSP - SYS - ping after 6000ms)

May I ask some questions here to help me debug this:

I have verified that I am using the correct IP address and port. Is this the correct notation for the config of Z2M to connect to the stream component?:

serial:
  port: tcp://192.168.178.35:6638

What should the stream component report in the ESPHome logs when there is communication between Z2M and ESPHome? (my logger level is VERBOSE).
Is there some basic test to check whether a communication from HA to the CC2530 is possible? I.e. some shell commands that I could issue from HA?
I get inconsistent information about the wiring between ESP8266 and CC2530. I tried two versions:
- 3v3 -- VCC
  GND -- GND
  TX -- P02
  RX -- P03
- the above plus
  GND -- P20
  GND -- P04
  GND -- P05
Are the 3 additional Ground connections in the second block needed?
Any other ideas where I could start to dig in?

Many thanks! :D

Hedda · 2021-11-22T11:49:10Z

Z2M configuration side at least looks correct so guess is that the issue probably is in wiring between ESP8266 and CC2530:

https://www.zigbee2mqtt.io/guide/adapters/flashing/connecting_cc2530.html

https://www.zigbee2mqtt.io/advanced/zigbee/05_create_a_cc2530_router.html#result

https://www.zigbee2mqtt.io/advanced/remote-adapter/connect_to_a_remote_adapter.html#_2-b-configure-ser2net-4-0

I primarily use ZHA in HA and CC2652P myself (and not Zigbee2MQTT) so can not really help but can suggest posting here:

https://github.com/Koenkk/zigbee2mqtt/discussions

PS: Regardless, strongly recommend that buy CC2652P based module/adapter since CC2530 and CC2531 are really obsolete now (both very old hardware with limited resources and old "Z-Stack Home 1.2" firmware that is end-of-life by Texas Instruments).

https://github.com/Koenkk/Z-Stack-firmware/blob/master/coordinator/Z-Stack_3.x.0/bin/README.md

https://www.zigbee2mqtt.io/guide/adapters/#recommended

Hedda · 2021-11-22T11:51:53Z

Is there some basic test to check whether a communication from HA to the CC2530 is possible? I.e. some shell commands that I could issue from HA?

FYI, could test basic socat connection to CC2530x with zigpy-znp (Python script) which CLI commands for some tools like backup:

https://github.com/zigpy/zigpy-znp/blob/dev/TOOLS.md

https://github.com/zigpy/zigpy-znp

https://pypi.org/project/zigpy-znp/

sergeolkhovik · 2021-11-22T12:05:04Z

Hi all, I am experimenting with @oxan's custom component and hitting a wall.
...
Zigbee2MQTT:error 2021-11-21 11:42:38: Exiting...
Zigbee2MQTT:error 2021-11-21 11:42:39: Error: Failed to connect to the adapter (Error: SRSP - SYS - ping after 6000ms)

That's really strange and interesting, as I'm experiencing the same error but via direct USB connection: Koenkk/zigbee2mqtt#2997
I had to revert down to 1.18.3, and going to try that USB reset tool (not sure if I can run it inside the docker), and reflash latest 3.0 on my CC2530.

RoganDawes · 2021-11-22T12:07:25Z

You could also enable ESPHome serial logging, so that you can see what the dongle receives and responds. Alternatively, logging on the TCP side of the stream-server, to reduce the number of lines of log output.

I modified my stream_server.cpp with the following:

std::string StreamServerComponent::hexencode(const uint8_t *data, uint32_t off, uint32_t len) {
  char buf[20];
  std::string res;
  for (uint32_t i = off; i < off + 16; i++) {
    if (i + 1 <= off + len) {
      sprintf(buf, "%02X ", data[i]);
    } else {
      sprintf(buf, "   ");
    }
    res += buf;
  }
  res += "|";
  for (uint32_t i = off; i < off + 16; i++) {
    if (i + 1 <= off + len) {
      if (data[i]>0x1F && data[i]<0x7F) {
        sprintf(buf, "%c", data[i]);
      } else {
        sprintf(buf, ".");
      }
    } else {
      sprintf(buf, " ");
    }
    res += buf;
  }
  res += "|";
  return res;
}

and

void StreamServerComponent::read() {
    int len;
    while ((len = this->stream_->available()) > 0) {
        char buf[128];
        size_t read = this->stream_->readBytes(buf, min(len, 128));
        for (int i = 0; i < read; i += 0x10) {
          int l = std::min(static_cast<int>(read)-i, 0x10);
          ESP_LOGD(TAG, "(%d) Read : %s", this->port_, this->hexencode((uint8_t *)buf, i, l).c_str());
        }
        for (auto const& client : this->clients_)
            client->tcp_client->write(buf, read);
    }
}

void StreamServerComponent::write() {
    size_t len;
    while ((len = this->recv_buf_.size()) > 0) {
        this->stream_->write(this->recv_buf_.data(), len);
        for (int i = 0; i < len; i += 0x10) {
          int l = std::min(static_cast<int>(len)-i, 0x10);
          ESP_LOGD(TAG, "(%d) Write: %s", this->port_, this->hexencode((uint8_t *)this->recv_buf_.data(), i, l).c_str());
        }
        this->recv_buf_.erase(this->recv_buf_.begin(), this->recv_buf_.begin() + len);
    }
}

Just added logging lines.

Hedda · 2021-11-23T08:38:41Z

I had to revert down to 1.18.3, and going to try that USB reset tool (not sure if I can run it inside the docker), and reflash latest 3.0 on my CC2530.

Great point! While not recommended for "production" for developers it can be a good idea to try Z-Stack 3 on CC2530/CC2531:

https://github.com/Koenkk/Z-Stack-firmware/tree/master/coordinator

Note that it is only this hacked/back-ported Z-Stack_3.0.x (and not Z-Stack_3.x.0) FW which can be used on CC2530 and CC2531:

https://github.com/Koenkk/Z-Stack-firmware/tree/master/coordinator/Z-Stack_3.0.x/bin

Note the comment "Zigbee 3.0 firmwares are not recommended for the CC2530 and CC2531 (since those are not powerful enough)".

Again, not recommended but can be used by developers for lab or test purposes if do not yet have access to CC2652 hardware.

Hedda · 2021-11-23T08:46:00Z

Tip is to place pre-order for a few new inexpensive "SONOFF Zigbee 3.0 USB Dongle Plus" (CC2652P based adapter/stick by ITead):

https://itead.cc/product/sonoff-zigbee-3-0-usb-dongle-plus/

https://sonoff.tech/product/diy-smart-switch/sonoff-dongle-plus/

At least board inside it look to have exposed TX and RX pads for serial so can probably also be used as a Zigbee serial module(?).

Hedda · 2021-11-23T09:04:06Z

Tip is to place pre-order for a few new inexpensive "SONOFF Zigbee 3.0 USB Dongle Plus" (CC2652P based adapter/stick by ITead):

https://itead.cc/product/sonoff-zigbee-3-0-usb-dongle-plus/

https://sonoff.tech/product/diy-smart-switch/sonoff-dongle-plus/

At least board inside it look to have exposed TX and RX pads for serial so can probably also be used as a Zigbee serial module(?).

Alternatively, you can try to order "RFSTAR RF-BM-2652P2" and/or "Ebyte E72-2G4M20S1E" which are CC2652P serial modules:

https://www.szrfstar.com/product/237-en.html

https://www.ebyte.com/en/product-view-news.html?id=1002

Thought due to the current silicon chip shortages it is well known that those modules can be hard to get a hold of at this time.

Another alternative serial module is "RFSTAR RF-TI1352B1" which uses CC1352P which is like CC2652P but 2.4GHz + 1Ghz-sub too:

https://www.szrfstar.com/product/188-en.html

These can be ordered from example Aliexpress or Alibaba and will usually have much longer shipping times. Example:

https://rf-star.aliexpress.com/store/910711003

https://www.aliexpress.com/item/1005001677721370.html

https://www.aliexpress.com/item/1005003527541390.html

https://www.aliexpress.com/item/1005003527468693.html

https://www.aliexpress.com/item/1005001265532720.html

https://www.aliexpress.com/item/1005003157295310.html

https://cdebyte.aliexpress.com/store/2077046

Jpsy · 2021-11-24T08:52:54Z

Ah, thanks for your extensive input. Great!
I will retreat to the cellar lab and report new findings.

Hedda · 2021-11-25T10:35:10Z

FYI, there is also a few ESP32 + CC2652P based open-source hardware gateway/bridge projects you can buy pre-assembled:

ZigStar LAN Gateway (ships from Germany I believe)

https://zig-star.com/projects/zigbee-gw-lan/

https://github.com/mercenaruss/zigstar_gateways

Tube's Zigbee Gateways (ships from Australia or New Zealand I believe)

https://www.tubeszb.com/shop/2

https://github.com/tube0013/tube_gateways

Gio-dot Z-Bee Duo with CC2652P (ships from China I believe)

https://github.com/Gio-dot/Z-Bee-Duo

cyijun OpenZ3Gateway (ESP8266 only and ships from China I believe)

https://github.com/cyijun/OpenZ3Gateway

PS: Note that all those projects rely on the above mentioned TI CC2652P radio modules from RFSTAR and Ebyte/CDEbyte.

Hedda · 2021-11-25T10:46:33Z

By the way, another related feature request for ESPHome is using mDNS DNS record(s) for ZHA automatic network discovery (and the passing of configurations) of Zigbee coordinator bridges/gateways (Ethernet/WiFi network adapters) that support Zeroconf:

https://community.home-assistant.io/t/zha-automatic-network-discovery-of-zigbee-coordinator-bridges-gateways-ethernet-wifi-network-adapters-that-support-zeroconf-or-ssdp/293300/

FYI, ZHA integration Zeroconf network discovery of ESPHome based Tube's Gateways is already partially supported, see:

https://www.home-assistant.io/integrations/zha#discovery-via-usb-or-zeroconf

PS: ZHA automatic network discovery of ESPHome based Zigbee LAN adapters is a simple but awesome feature for new HA users.

oxan · 2021-11-25T10:50:28Z

Can we move the discussion about Zigbee bridges/gateways somewhere else (i.e. a separate issue or a GitHub discussion)? It's fairly off-topic here and the many link-filled posts make this issue hard to read.

Jpsy · 2021-11-25T17:00:11Z

Thanks again for all your valuable input. @oxan is right: This is an issue report / feature request that should not be bloated with off topic discussions. I will digest all your input and create a corresponding topic in the HA forum. Will leave a link here when its done. But please give me some time to work on it all. Many thanks again.

Hedda · 2021-11-26T10:48:09Z

Can we move the discussion about Zigbee bridges/gateways somewhere else (i.e. a separate issue or a GitHub discussion)? It's fairly off-topic here and the many link-filled posts make this issue hard to read.

Thanks again for all your valuable input. oxan is right: This is an issue report / feature request that should not be bloated with off topic discussions. I will digest all your input and create a corresponding topic in the HA forum. Will leave a link here when its done. But please give me some time to work on it all. Many thanks again.

You could maybe also continue relevant Zigbee related discussions in these other feature requests depending on specific topic subject -> https://github.com/esphome/feature-requests/issues?q=zigbee (Zigbee can be used in many different ways and that is why there are so many separate feature requests with slightly different topics/subjects). FYI, the specific idea around using ESPHome as a serial stream server to remotely access a Zigbee UART module from a computer is discussed in #688

tube0013 · 2022-07-20T16:04:51Z

Is there an official alternative now?

https://github.com/oxan/esphome-stream-server is very unreliable with ESPHome > 2021.9, the connection resets every few hours. we are now ~10 version past this and a lot of newer esphome functionality is missed if relying on the stream-server custom component.

Hedda · 2022-08-03T15:53:19Z

Is there an official alternative now?

https://github.com/oxan/esphome-stream-server is very unreliable with ESPHome > 2021.9, the connection resets every few hours. we are now ~10 version past this and a lot of newer esphome functionality is missed if relying on the stream-server custom component.

Is it same issue in https://github.com/syssi/esphome-zb-gw03 as https://github.com/tube0013/tube_gateways ESPHome images?

Alternativly, while not official wondering if it is possible to use ESPHome serial server component by @thegroove if better?

https://github.com/thegroove/esphome-serial-server

It is used as a custom component in his ESPHome for ITead Sonoff Zigbee Bridge (based on ESP8266/ESP8285):

https://github.com/thegroove/esphome-zbbridge

https://github.com/thegroove/esphome-zbbridge/tree/main/custom_components/serial_server

PS: He also started the development of a Zeroconf custom component for ESPHome that can be used with it:

https://github.com/thegroove/esphome-zeroconf

https://github.com/thegroove/esphome-zha-ezsp-zeroconf

https://github.com/thegroove/esphome-zha-ezsp-zeroconf/tree/main/custom_components/zeroconf

tube0013 · 2022-09-19T20:32:31Z

Is there an official alternative now?
https://github.com/oxan/esphome-stream-server is very unreliable with ESPHome > 2021.9, the connection resets every few hours. we are now ~10 version past this and a lot of newer esphome functionality is missed if relying on the stream-server custom component.

Is it same issue in https://github.com/syssi/esphome-zb-gw03 as https://github.com/tube0013/tube_gateways ESPHome images?

Alternativly, while not official wondering if it is possible to use ESPHome serial server component by @thegroove if better?

https://github.com/thegroove/esphome-serial-server

It is used as a custom component in his ESPHome for ITead Sonoff Zigbee Bridge (based on ESP8266/ESP8285):

https://github.com/thegroove/esphome-zbbridge

https://github.com/thegroove/esphome-zbbridge/tree/main/custom_components/serial_server

All of these are derived off of the original code in https://github.com/oxan/esphome-stream-server so they all suffer the same thing.

ESPHome-2022.8.3 seems much more stable and cc2652 connections are stable. EFR32 is still unstable. I've tested the same esp32 with alternate non-esphome fw and it is capable of steady reliant connection, a new component or a re-write of the original custom component is needed for this to be reliable.

nagyrobi · 2022-09-19T20:36:08Z

@tube0013 thanks for testing and clarifying.

Hedda · 2022-11-22T13:53:58Z

FYI, tube0013 has forked oxan's esphome stream server and got it working reliably with ESPHome versions newer than 2021.10:

https://github.com/tube0013/esphome-stream-server-v2

glmnet added the Type: Network label Apr 17, 2020

This was referenced Apr 17, 2020

Serial Wifi bridge to delegate control to other hosts #619

Closed

add ser2net component #638

Closed

glmnet added the Type: UART label Apr 17, 2020

randybb mentioned this issue Apr 20, 2020

Support for non-ESP microcontrollers using RF modules #687

Open

Hedda mentioned this issue Apr 20, 2020

[REQUEST] ESPHome support on Sonoff ZBBridge as a network-attached remote Zigbee Coordinator adapter (or Zigbee Router) for Home Assistant's ZHA integration and Zigbee2MQTT #688

Open

nagyrobi added the alternative-offered label Jul 1, 2022

nagyrobi closed this as completed Jul 1, 2022

nagyrobi reopened this Sep 19, 2022

nagyrobi removed the alternative-offered label Sep 19, 2022

pugmandan mentioned this issue Nov 24, 2023

Failed to read from client 192.168.10.153 with error 128! tube0013/tube_gateways#172

Closed

Stream server component (serial-over-wifi) #660

Stream server component (serial-over-wifi) #660

Comments

oxan commented Mar 30, 2020 • edited Loading

glmnet commented Apr 17, 2020

Hedda commented Apr 20, 2020 • edited Loading

zuidwijk commented May 25, 2020

schnizki commented Aug 12, 2020 • edited Loading

zuidwijk commented Aug 12, 2020

oxan commented Aug 13, 2020

zuidwijk commented Aug 13, 2020

zuidwijk commented Aug 15, 2020

Harald-P commented Aug 16, 2020 • edited Loading

oxan commented Aug 16, 2020 • edited Loading

Harald-P commented Aug 16, 2020 • edited Loading

oxan commented Aug 16, 2020 • edited Loading

Harald-P commented Aug 16, 2020

mjkl-gh commented Oct 22, 2020 • edited Loading

oxan commented Oct 22, 2020 • edited Loading

mjkl-gh commented Oct 22, 2020 • edited Loading

oxan commented Oct 23, 2020

mjkl-gh commented Oct 23, 2020

oxan commented Oct 23, 2020

duhow commented Dec 26, 2020

yzg011 commented Dec 26, 2020

RoganDawes commented Jan 22, 2021 • edited Loading

ioanfesteu commented Mar 11, 2021 • edited Loading

Hedda commented Mar 11, 2021 • edited Loading

oxan commented May 19, 2021

Jpsy commented Nov 21, 2021 • edited Loading

Hedda commented Nov 22, 2021 • edited Loading

Hedda commented Nov 22, 2021 • edited Loading

sergeolkhovik commented Nov 22, 2021 • edited Loading

RoganDawes commented Nov 22, 2021

Hedda commented Nov 23, 2021 • edited Loading

Hedda commented Nov 23, 2021 • edited Loading

Hedda commented Nov 23, 2021 • edited Loading

Jpsy commented Nov 24, 2021

Hedda commented Nov 25, 2021 • edited Loading

Hedda commented Nov 25, 2021 • edited Loading

oxan commented Nov 25, 2021

Jpsy commented Nov 25, 2021

Hedda commented Nov 26, 2021 • edited Loading

tube0013 commented Jul 20, 2022

Hedda commented Aug 3, 2022

tube0013 commented Sep 19, 2022 • edited Loading

nagyrobi commented Sep 19, 2022

Hedda commented Nov 22, 2022

oxan commented Mar 30, 2020 •

edited

Loading

Hedda commented Apr 20, 2020 •

edited

Loading

schnizki commented Aug 12, 2020 •

edited

Loading

Harald-P commented Aug 16, 2020 •

edited

Loading

oxan commented Aug 16, 2020 •

edited

Loading

Harald-P commented Aug 16, 2020 •

edited

Loading

oxan commented Aug 16, 2020 •

edited

Loading

mjkl-gh commented Oct 22, 2020 •

edited

Loading

oxan commented Oct 22, 2020 •

edited

Loading

mjkl-gh commented Oct 22, 2020 •

edited

Loading

RoganDawes commented Jan 22, 2021 •

edited

Loading

ioanfesteu commented Mar 11, 2021 •

edited

Loading

Hedda commented Mar 11, 2021 •

edited

Loading

Jpsy commented Nov 21, 2021 •

edited

Loading

Hedda commented Nov 22, 2021 •

edited

Loading

Hedda commented Nov 22, 2021 •

edited

Loading

sergeolkhovik commented Nov 22, 2021 •

edited

Loading

Hedda commented Nov 23, 2021 •

edited

Loading

Hedda commented Nov 23, 2021 •

edited

Loading

Hedda commented Nov 23, 2021 •

edited

Loading

Hedda commented Nov 25, 2021 •

edited

Loading

Hedda commented Nov 25, 2021 •

edited

Loading

Hedda commented Nov 26, 2021 •

edited

Loading

tube0013 commented Sep 19, 2022 •

edited

Loading