howto add a driver to Venus

Intro

Every once and a while I get a question similar to 'Hi, I have the idea to read data from sensors such and such (for example tank senders) and I want to show that information on a CCGX, how can I do that?', or 'Hi, how can I make the CCGX read data from my ModbusTCP enabled generator?'.

On this page I'll give points on where to start with that.

Please feel welcome improve this text to make it easier to understand for a newcomer! Writing these types of explanations while head deep inside is not so easy.

For questions, use the mailing list.

Note that this page refers to Venus devices in General, not only the CCGX or Venus GX.

• 1. Whats involved
• 2. Developing a driver
• 3. serial-starter: plug and play for serial devices
• 4. Stability - exceptions handling

1. Whats involved

To show the data on the gui, it first needs to be made available on the D-Bus. D-Bus is the internal databus within Venus. The drivers publish their information on it, and the gui, and also the ModbusTCP bridge for example, take the data from it. Or send messages back on it when you for example change a setting in the device. A schematic overview is given here, the specs of how we use D-Bus are here, and a list of parameters on D-Bus is here: dbus.

So, with that in mind, adding a new sensor (for example a tank sender) or reading a new device (for example a generator) involves the following steps:

1. Choose a protocol: uart (the VE.Direct ports), tcp/ip, canbus, bluetooth, rs485 (requires a usb-rs485 converter) or any other use of usb.
2. Implement a process that interfaces between the comm port and publishes the data on the D-Bus: the driver
3. Modify the GUI, adding pages for the new data
4. Modify vrmlogger & the vrmportal, to make the new data available on the VRM Portal
5. Modify dbus-modbustcp, to make the new data available on ModbusTCP. See github.com/victronenergy/dbus-modbustcp project, especially attributes.csv.

If you use a protocol/message set that is already supported by one of the many existing drivers, then steps 2 and beyond won't be necessary.

Similarly, if you do need to implement a driver, but its for data set that is already supported by the gui, then steps 3 and beyond won't be necessary. And example for this would be a tank sensor that sends its data via the VE.Direct ports (effectively they are galvanically isolated uarts): we already have tank sensor data within Venus. Its data coming from the built-in ports in the Venus GX, see dbus-adc process](https://github.com/victronenergy/dbus-adc) for sources. And also we have tank sensor data coming in via canbus, which is in NMEA2000 format. The sources for that driver are not publicly available.

2. Developing a driver

tbd ..

• different languages can be used (ie c, cpp, python), and we have D-Bus drivers for them available.
• A trick to do this with a minimal software effort is to use the dbus-dummy service Python scripts: make a dbus-dummyservice containing the dbus service name as well as paths you need, make all paths writable from the outside. Then from your external device, which could be a PLC in this case, write to the Venus device with ModbusTCP. For existing data sets, no changes to the pmodbus mapping list](https://github.com/victronenergy/dbus_modbustcp/blob/master/attributes.csv) in Venus would be necessary.
• look at existing bridges. For existing data (ie tanks, solar chargers, generators, etc), make sure that the paths exported to D-Bus by your bridge exactly replicate the existing ones. To go back to the tank sensor example, see the readme of dbus-adc for how that should be.
• for something new, stick to the Venus D-Bus API definition, and also discuss the names of D-Bus paths etcetera with us.
• in case there are settings that cannot be stored on the sensor itself, see localsettings for storing them in Venus.
• for automatically running your new software at startup & not losing it after an update of Venus, see here and here.

3. serial-starter: plug and play for serial devices

The service that's messing up your port is the so-called serial-starter. To see serial starter in action, try these command. You might want to run it a few times.

root@raspberrypi2:~# ps | grep ttyUSB0
 1300 root      1580 S    supervise vedirect-interface.ttyUSB0
 1308 root      1596 S    multilog t s99999 n8 /var/log/vedirect.ttyUSB0
 1390 root      1580 S    supervise gps-dbus.ttyUSB0
 1402 root      1596 S    multilog t s99999 n8 /var/log/gps-dbus.ttyUSB0
 5271 root      3048 S    {start-gps.sh} /bin/bash /opt/victronenergy/gps-dbus/start-gps.sh ttyUSB0
 5284 root      3144 S    /opt/victronenergy/gps-dbus/gps_dbus -v --banner --dbus system --timeout 2 -s /dev/ttyUSB0 -b 38400

In above example you see the gps daemon being started against ttyUSB0, at 38400 bps. All lines showing supervise and multilog can be ignored.

Howto stop serial-starter on a tty port

While developing a driver, just tell serial-starter to skip that port:

/opt/victronenergy/serial-starter/stop-tty.sh ttyUSB0

Howto make serial-starter ignore certain USB types

Alternative to stopping it manually, add a line in /etc/udev/rules/serial-starter.rules to make serial-starter ignore the type every time its plugged in:

Run this code to see the ID for your device:
udevadm info --query=property --name=/dev/ttyUSB0 | sed -n s/^ID_MODEL=//p

Ftdi serial converters, and other brands as well I suppose, can be programmed with a unique ID. See ft_prog for ftdi.

If the cable name is unique and you want serial-starter to ignore it, add the following line to serial-starter.rules

ACTION=="add", ENV{ID_BUS}=="usb", ENV{ID_MODEL}=="MY_UNIQUE_ID", ENV{VE_SERVICE}="ignore"

(replace MY_UNIQUE_ID)

In above example, I've added P1_Converter_Cable. Then restarted the Venus device, and then run the ps | grep command a few times again to make sure the serial port is left alone:

Howto add a driver to serial-starter

Once your driver is completed, and you want plug-and-play to work; you'll need to add your device & driver to the serial-starter.

In brief, the serial-starter & udev do the following for each serial-device detected on the USB:

1. read its ID_MODEL
2. look up the device class in /etc/udev/rules.d/serial-starter.rules
3. look up the available drivers in /etc/venus/serial-starter.conf
4. run all available drivers one by one.

The drivers are implemented such that if they can't detect a device at the other end of the line that they support, they exit again. Serial-starter will then try the next one, and so forth. Restarting from scratch and forever continuing when none of the drivers sticks.

To add your own driver:

1. determine the device Id of your serial cable: udevadm info --query=property --name="/dev/ttyUSB0" | sed -n "s/^ID_MODEL=//p"

2. add/modify a line in /etc/udev/rules.d/serial-starter.rules that maps that ID_MODEL to a device class.

3. add/modify a line in /etc/venus/serial-starter.conf that maps your device class to your driver (name of driver directory in /service). If you want serial starter to ignore that port, use ignore as driver name.

4. install (or at least symlink in case you have your code on the data partition) your software in /opt/victronenergy/ like all the drivers and also other executables are.

5. make sure there is a service directory below that: /opt/victronenergy/your-program/service. Look at another driver for an example. The TTY will need to be passed, see vedirect-interface/service for an example.

6. note that also in the /log/run file there is a TTY placeholder.

7. since USB devices trigger the serial-starter via udev, simply restarting the serial-starter is not enough. So, reboot the device, or replug your USB device to make the magic work.

8. and check logs: tail -F /log/serial-starter/current | tai64nlocal

4. Stability - exceptions handling

Once officially implemented and added to Venus, the process will be managed by daemontools. And in some cases (when it involves a tty) also under the serial-starter.

Which means: don't code too defensively. Don't try to recover from an unrecoverable error situation. Instead, keep the code clean, make sure it exists and let the baby-sitters (daemontools & serial-starter) take care of you.

Do make sure the code does not hang in a weird state.

In more detail:

• when the tty disappears: just exit. serial-start restart the driver in case the tty comes back.
• when whatever strange thing happens: just exit. daemontools will fix that, and provide for a clean start.
• when the dbus disappears: doesn't matter what will happen: Venus will reboot anyway in such case. No need to add code to properly exit in such case; and most probably your code will crash anyway. Disappearing dbus-es is not an use case.

Notes for Python

• without proper precautions, a thread can crash without taking down the full process with it: not good, a complete crash is better. See daemon=True, for Python.
• when you use gobject.timeout_add, then the callback you specify must return True if it wants to be rescheduled. But if there is an exception in that callback that is not handled, the callback is not rescheduled. The main thread continues running though; leaving the program hanging. We have exit_on_error() for that.