Modbus RTU over Serial on RS485 Unreliable data captured

[EHerzog76]

The problem

Hello to All,

Problemdescription:

If you use Modus RTU over Serial-Interface in combination with RS485 (which is a half-duplex Bussystem),
and you have more devices connected to the bus, then you will get troubles.

Because the used pymodbus-library is an asynchronus implementation and also the Home Assistant core Modbus-Implementation is asynchronus.
So when you have only read-requests, you will see that many request are ok
and some requests will fail with: Unreliable data captured

But when you have also write-requests e.g. write to a coil and verify the status
you will get in troubles => it is also possible that the connected RS485-Device will get in a undefined state and you have to restart it.

=> Summary: As more devices and as more read-/write-request => more troubles !!!

    (e.g. use Sensors, poll this Sensors every 5 seconds and use also write-request to turn on/off switches, coils, ...)

When you use the same RS485-Devices with the same Modbus-Configuration but use a Modbus TCP to RS485 Gateway instead of the Serial-Interface.
The external Modbus TCP to RS485 Gateway will receive the asynchronus Modbus-communication and will send them in a synchrone half-duplex aware way to the RS485-bus and everything works as it should.

best regards
Erwin

What version of Home Assistant Core has the issue?

core-2023.9.2

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant Container

Integration causing the issue

Modbus

Link to integration documentation on our website

https://www.home-assistant.io/integrations/modbus#configuring-serial-connection

Diagnostics information

No response

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

No response

[home-assistant]

Hey there @janiversen, mind taking a look at this issue as it has been labeled with an integration (modbus) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of modbus can trigger bot actions by commenting:

• @home-assistant close Closes the issue.
• @home-assistant rename Awesome new title Renames the issue.
• @home-assistant reopen Reopen the issue.
• @home-assistant unassign modbus Removes the current integration label and assignees on the issue, add the integration domain after the command.

_{^{(message by CodeOwnersMention)}}

---

modbus documentation
modbus source
_{^{(message by IssueLinks)}}

[EHerzog76]

As described above my work-a-round for this problem is to use a Ethernet-to-Serial-Gateway like this one:
*** NO advertising, please ***

best regards
Erwin

[janiversen]

you have a couple of misunderstandings!

HA is async and the modbus integration as well, BUT the modbus integration uses the sync version of pymodbus.

Anyhow async does not have anything to do with your unreliable data, because as you say the modbus protocol serial protocol is half duplex, and when sending a request the library blocks until a response is received, thus guaranteeing that there is data consistency.

If you see unreliable data, then please add a small configuration that demonstrates the problem together with a debug log as pr modbus documentation.

[janiversen]

My own production setup happens to be serial and that works with 7 devices connected to the same bus.

Your problem could very well be configuration, that you e.g. use slave:0.

[janiversen]

Another frequent problem is the terminating resistor on the rs485, your gateway typically have one builtin, whereas an usb does not, and thus you need to add it. A missing terminator would fit the description of your error.

BUT a decent debug log will tell a lot more.

[janiversen]

Closing as not a bug.

[EHerzog76]

First here you have documented the same issue from a other user:
#99784

I know, that you try to serialize the calls to the modbus.

async def async_pb_call(
  ...
  async with self._lock:
            if not self._client:
                return None
            result = await self.hass.async_add_executor_job(
                self.pb_call, unit, address, value, use_call
            )
            ...

and in Homeassistant-core:

 @callback
    def async_add_executor_job(
        self, target: Callable[..., _T], *args: Any
    ) -> asyncio.Future[_T]:
        """Add an executor job from within the event loop."""
        task = self.loop.run_in_executor(None, target, *args)
        self._tasks.add(task)
        task.add_done_callback(self._tasks.remove)

        return task

** And here we have the PROBLEM:
self.loop.run_in_executor(None, target, *args) => this is a call to the asyncio - library => loop.run_in_executor

*** For this method you will find this note:
Changed in version 3.5.3: loop.run_in_executor() no longer configures the max_workers of the thread pool executor it creates, instead leaving it up to the thread pool executor (ThreadPoolExecutor) to set the default.

*** And for the ThreadPoolExecutor you will find this notes:
Changed in version 3.5: If max_workers is None or not given, it will default to the number of processors on the machine, multiplied by 5, assuming that ThreadPoolExecutor is often used to overlap I/O instead of CPU work and the number of workers should be higher than the number of workers for ProcessPoolExecutor.

New in version 3.6: The thread_name_prefix argument was added to allow users to control the threading.Thread names for worker threads created by the pool for easier debugging.

Changed in version 3.7: Added the initializer and initargs arguments.

Changed in version 3.8: Default value of max_workers is changed to min(32, os.cpu_count() + 4). This default value preserves at least 5 workers for I/O bound tasks. It utilizes at most 32 CPU cores for CPU bound tasks which release the GIL. And it avoids using very large resources implicitly on many-core machines.

ThreadPoolExecutor now reuses idle worker threads before starting max_workers worker threads too.

Why it´s working in your environment

Perhaps you have a older python-version  ???

ToDo:

There a 2 ways to solve the problem.

loop.run_in_executor(None, target, *args)

  Change   None   to a self defined Event-Loop with only 1 thread

Define

  max_workers=1  for the ThreadPoolExecutor 

[janiversen]

First of all your explanations point to a problem in core not in the modbus integration.

The modbus integration uses pymodbus synchronous version, and each call is fenced by a lock, which secures that there never are multiple requests outstanding.

Instead of all your long explanations, please supply a modbus debug log as pr modbus documentation so we can see what actually happens.