CAN bus support for SMA/Goodwe/Pylontech

[Uksa007]

Hi,

I'm wanting to use this BMS with a Goodwe inverter that supports the SMA CAN protocol.
The SMA CAN protocol is used by many eg Pylontech, LG and Goodwe.
It would open up this BMS to be much more useful for home energy storage applications, if it could communicate with the inverter.

The official CAN bus support from JK I believe is only enabled if you order the CAN model, despite the port being present.
Even then their implementation is unknown in origin or what it's based on, it uses 250Kbps, SMA is 500Kbps.
Repeated attempts to ask JK to support the widely adopted SMA CAN protocol are unsuccessful, with them requesting that you make a big order and they might look at it. Not very helpful for individuals, I note that Seplos has support for SMA protocol.
In any case it seems we are on our own.
JK standard (still don't know what it's based on anyone know?)
https://github.com/Uksa007/esphome-jk-bms-can/blob/main/docs/BMS-CAN%20Communication%20protocol.pdf

I have created a fork and started work on implementing changes to support CAN
https://github.com/Uksa007/esphome-jk-bms-can

I have been looking and it seems esphome has the necessary CAN components.
I have started matching the SMA protocol required values to values already present from the RS485 data read.
https://github.com/Uksa007/esphome-jk-bms-can/blob/main/docs/SMA%20CAN%20Protocol%20Mapping.pdf

Any advice or words of wisdom much appreciated.

[syssi]

I would be happy to support CAN too. I've to check my BMS is CAN bus enabled or not and I've to order some CAN bus module first because the BMS speaks native CAN, right?

I assume you have to replace the ModbusDevice implementation with the CAN bus equivalent (in ESPHome).

[Uksa007]

I have been making some headway with testing the CAN comms with the inverter.
It seems to be pretty happy, and the displaying what I send it!
It even send back a 0x305 as a kind of ACK.

[18:01:33][D][canbus:033]: send extended id=0x359 rtr=FALSE size=8
[18:01:33][D][canbus:033]: send extended id=0x351 rtr=FALSE size=8
[18:01:33][D][canbus:033]: send extended id=0x355 rtr=FALSE size=8
[18:01:33][D][canbus:033]: send extended id=0x356 rtr=FALSE size=8
[18:01:33][D][canbus:033]: send extended id=0x354 rtr=FALSE size=8
[18:01:33][D][canbus:070]: received can message (#1) std can_id=0x305 size=8

Working on soldering it all up on PCB.

[syssi]

FYI if you aren't aware of the 0x305 content.

[Uksa007]

Yes I'm aware, but the contents at the moment 0x305 response is empty, all 0x00.

Maybe once inverter has batteries connected and operating correctly it will contain real data.
Understandably it's a little unhappy there is no battery connected.

Good news the BMS arrived and I have it communicating with the ESP32 so testing with real BMS now!
I have it all solder up on pcb and ready for box

[Uksa007]

So the stability issues I have been having are related to No Acks being received on the CAN bus, if the inverter is not responding
I have opened a ticket with esphome here: esphome/issues#3317

Making good progress, I have been emulating a LG Resu 13 battery as it most closely matches my battery capacity and charge/discharge currents etc.
I have done some more work the Alarms making sure each on it bitbanged into the right bit position.
Still formulating a plan to deal with clearing the alarms

[syssi]

Good catch! I'm happy about your found because both solutions (yaml & custom component) have the same WDT reboot issue. I was unable to identify the root cause.

[syssi]

Some ideas what needs to be done:

1. Attach two CAN bus modules to an ESP because of the different baudrates.
2. Implement the CAN bus protocol of JK to talk the BMS via CAN bus.
3. Implement a fake/virtual BMS component which is able to speak the SMA protocol.
4. Feed the fake BMS with the data from the JK-BMS received via CAN bus.

[Uksa007]

So I think maybe keep Modbus and just add CAN for the Inverter
The JK CAN doesn't seem to give anything extra that you can't get from RS485, that I can see, do you agree?

[Uksa007]

So need to do the following:

1. Attach one CAN bus modules to an ESP (need to decide on hardware, different options for ESP32 V ESP8266)
   ESP32 has CAN bus support natively (so could write code without hardware module) see:
   https://esphome.io/components/canbus.html?highlight=can#esp32-can-component

   ESP8266 needs external support via mcp2515 over SPI bus:
   https://esphome.io/components/canbus.html?highlight=can#mcp2515-component

2. Implement the CAN bus for SMA Protocol using existing RS485/TTL sensor/data.
   Seems like this would be bulk of the work

3. Implement option to enable CAN bus for Inverter (disable by default, enable for those that want to use it)

4. Test with fake data initially.
   I have updated the fake data to be more suitable for use eg have added 16 cells for 48v, 280Ah etc

Thoughts?

[syssi]

This is a good plan and safes some effort! I will read the SMA document later the day to get a better understanding what the inverter expects / which values should/must be available.

[Uksa007]

Great, have a read of this document pages 18 to 30 have the SMA protocol details:
https://github.com/Uksa007/esphome-jk-bms-can/blob/main/docs/SMA%20CAN%20protocol.pdf

Should be same as SMA just I have documented them out and I have tried to map them to current JK data from modbus, just got to workout the alarms and warnings.
https://github.com/Uksa007/esphome-jk-bms-can/blob/main/docs/SMA%20CAN%20Protocol%20Mapping.pdf

Started working on a basic code for ESP32 example with CAN:
https://github.com/Uksa007/esphome-jk-bms-can/blob/main/esp32-example-can.yaml

[syssi]

SMA did a decent job in documenting the CAN Bus protocol / the required values. You are right: The ESP32 has a native CAN Bus controller but I assume a logic level shift (from 3.3 to 5V) should be performed. If you connect a ESP to your Sunny Island (properly = using resistors/terminators) you should see periodic status message of the Inverter on the bus. I would recommend to start with this basic setup. The next step is to publish some messages to get rid of the error message complaining about the unavailable BMS.

[syssi]

I've read your mapping document. Looks good to me. The battery_type value of the JK-BMS shouldn't be used because it's always the same value. At static SOH value (100%) is a good idea. The HiResSoC can be skipped at first and probably added later.

[Uksa007]

So having the same issues with the ESP32 crashing constantly I had the with esp8266.

Running your example code https://github.com/syssi/esphome-jk-bms/blob/main/esp32-example.yaml
Just constantly crashes reboots crashes.

Any idea?

[Uksa007]

Tried removing the mqtt section and adding api: still the same.

[syssi]

Please remove most of the sensors and start again. At the ESP32 the stack size can be increased to support all sensors.

[Uksa007]

Yes removing most of the sensors fixes it, where do I increase the stack size?

[syssi]

https://github.com/syssi/esphome-jk-bms#known-issues

[Uksa007]

Tried with the stack increased still crashes

Rebooting...
ets Jun 8 2016 00:22:57

rst:0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0018,len:4
load:0x3fff001c,len:1044
load:0x40078000,len:10124
load:0x40080400,len:5828
entry 0x400806a8
[I][logger:214]: Log initialized
[C][ota:461]: There have been 0 suspected unsuccessful boot attempts.
Guru Meditation Error: Core 1 panic'ed (Unhandled debug exception)
Debug exception reason: Stack canary watchpoint triggered (loopTask)
Core 1 register dump:
PC : 0x4010c181 PS : 0x00060b36 A0 : 0x80111961 A1 : 0x3ffafe40
A2 : 0x00000000 A3 : 0x3ffb0034 A4 : 0x00000000 A5 : 0x00000000
A6 : 0x3ffb0250 A7 : 0x00000008 A8 : 0x3ffb0150 A9 : 0x3ffafe30
A10 : 0x3ffafe50 A11 : 0x3ff9c510 A12 : 0x000000ff A13 : 0x0000ff00
A14 : 0x00ff0000 A15 : 0xff000000 SAR : 0x00000004 EXCCAUSE: 0x00000001
EXCVADDR: 0x00000000 LBEG : 0x400014fd LEND : 0x4000150d LCOUNT : 0xffffffff

ELF file SHA256: 0000000000000000

Backtrace: 0x4010c181:0x3ffafe40 0x4011195e:0x3ffb0150 0x4011199a:0x3ffb01e0 0x400dec6d:0x3ffb0220 0x400deca1:0x3ffb0250 0x400decd5:0x3ffb02a0 0x400dee95:0x3ffb02c0 0x400e5b35:0x3ffb02e0 0x400e5b62:0x3ffb0310 0x400d6262:0x3ffb0370 0x400dfb79:0x3ffb0390 0x400dfc9d:0x3ffb03c0 0x400e6a7e:0x3ffb0400 0x400f034a:0x3ffb1fb0 0x40089b92:0x3ffb1fd0

[Uksa007]

Hi, Tried the example, won't compile on raspberry pi, error:

INFO Reading configuration /config/jk-bms.yaml...
INFO Updating https://github.com/syssi/esphome-jk-bms.git@main
INFO Generating C++ source...
INFO Compiling app...
Processing jk-bms (board: wemos_d1_mini32; framework: espidf; platform: platformio/espressif32 @ 3.5.0)
--------------------------------------------------------------------------------
HARDWARE: ESP32 240MHz, 320KB RAM, 4MB Flash
 - framework-espidf 3.40302.0 (4.3.2) 
 - tool-cmake 3.16.9 
 - tool-ninja 1.10.2 
 - toolchain-riscv32-esp 8.4.0+2021r2-patch2 
 - toolchain-xtensa-esp32 8.4.0+2021r2-patch2 
 - toolchain-xtensa-esp32s2 8.4.0+2021r2-patch2
Reading CMake configuration...
-- The C compiler identification is unknown
-- The CXX compiler identification is unknown
-- The ASM compiler identification is unknown
-- Found assembler: /config/.esphome/platformio/packages/[email protected]+2021r2-patch2/bin/xtensa-esp32-elf-gcc
-- Check for working C compiler: /config/.esphome/platformio/packages/[email protected]+2021r2-patch2/bin/xtensa-esp32-elf-gcc
-- Check for working C compiler: /config/.esphome/platformio/packages/[email protected]+2021r2-patch2/bin/xtensa-esp32-elf-gcc -- broken
-- Configuring incomplete, errors occurred!
See also "/config/.esphome/build/jk-bms/.pioenvs/jk-bms/CMakeFiles/CMakeOutput.log".
See also "/config/.esphome/build/jk-bms/.pioenvs/jk-bms/CMakeFiles/CMakeError.log".

fatal: not a git repository (or any parent up to mount point /)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
CMake Error at /config/.esphome/platformio/packages/tool-cmake/share/cmake-3.16/Modules/CMakeTestCCompiler.cmake:60 (message):
  The C compiler

    "/config/.esphome/platformio/packages/[email protected]+2021r2-patch2/bin/xtensa-esp32-elf-gcc"

  is not able to compile a simple test program.

  It fails with the following output:

    Change Dir: /config/.esphome/build/jk-bms/.pioenvs/jk-bms/CMakeFiles/CMakeTmp
    
    Run Build Command(s):/config/.esphome/platformio/packages/tool-ninja/ninja cmTC_7be51 && [1/2] Building C object CMakeFiles/cmTC_7be51.dir/testCCompiler.c.obj
    FAILED: CMakeFiles/cmTC_7be51.dir/testCCompiler.c.obj 
    /config/.esphome/platformio/packages/[email protected]+2021r2-patch2/bin/xtensa-esp32-elf-gcc   -mlongcalls -Wno-frame-address -o CMakeFiles/cmTC_7be51.dir/testCCompiler.c.obj   -c testCCompiler.c
    /bin/sh: 1: /config/.esphome/platformio/packages/[email protected]+2021r2-patch2/bin/xtensa-esp32-elf-gcc: not found
    ninja: build stopped: subcommand failed.
    
    

  

  CMake will not be able to correctly generate this project.
Call Stack (most recent call first):
  /config/.esphome/platformio/packages/framework-espidf/tools/cmake/project.cmake:311 (__project)
  CMakeLists.txt:3 (project)



========================= [FAILED] Took 36.89 seconds =========================

[syssi]

Could you use another machine with x86 architecture? The build stack isn't armel compatible at the moment.

[Uksa007]

Looks like ESPhome is not very stable or very good for complex devices.
I think it might be best to use Arduino IDE and program in C, it is more stable and you have more control.
Just a simple sensor loses contact with Home assistant API all the time, at this stage it's not looking very stable?
Thoughts?

[syssi]

I've a lot of ESPHome nodes (Yeelights, ESP8266 based bulbs, roller shutter, smartmeter, PV monitoring etc.) in use without any trouble, stability or connection issues. Most of the devices are very complex setups with all hardware UARTs in use + ethernet module to avoid WiFi connections. I don't want to miss ESPHome anymore. ;-)

[syssi]

This screenshot shows the history / logbook of the connection status of the solar powermeter node: 5 reconnects in around 60 days.

[Uksa007]

Maybe it was was more stable, but as you said has become fatter and now doesn't seem so stable, try with jk-bms it disconnects in a few minutes.

[syssi]

Do you use an ESP8266 or ESP32 now? If you provide your config yaml I could help.

[Uksa007]

I have both, trying to use ESP32 as it seem may be better for CAN bus as it has native support.
Just running your ESP32 example, or was before, now it won't compile on raspberry pi and esphome it not really so easy to get working on windows.

[magnetus26]

I has the same situation with my rpi and just followed @syssi instructions to install platformio and esphome in windows (x86) and is not so hard, if I dis it and idk nothing about programming anyone can do it 😁

[Uksa007]

I finally got it going, the instructions on the esphome web page are not so good for windows, eg running under python!

[magnetus26]

my JK-BMS keeps disconnecting not in a few minutes but in a few days. this has been happening since control Bluethooth connection implementation even without adding the switch idk if is a bad esp32 or a bug after the switch implementation
thanks
Miguel

[autox86]

You are such great!
What to disable to have a broken CAN, so I can get the ACK missed?
is this already enough:
#update_interval: 1s

Flashing to any ESP32 is fine, right?
Have some sonoff around which would do the job ;)

[syssi]

I don't understand your question. Could you describe your setup, your goal and what's broken? :-)

[syssi]

@cheisig You are on a wrong track regarding the watchdog issue. The ili9341 fixes doesn't affect the canbus component.

[Uksa007]

I don't understand your question. Could you describe your setup, your goal and what's broken? :-)

Yea, I'm not sure if he is referring to my issue I raised with esp-home for CAN bus retries esphome/issues#3317

[autox86]

App.feed_wdt();
yield();

esphome/issues#2980

I don't understand your question. Could you describe your setup, your goal and what's broken? :-)

Yea, I'm not sure if he is referring to my issue I raised with esp-home for CAN bus retries esphome/issues#3317

Yes indeed, I am refering to that.
Okay understood, Niels did solve his issue differently, while I am not having any clou how :(

But whats about that?

App.feed_wdt();
yield();

esphome/issues#2980

BR
Chris

[Uksa007]

Hi,
Just checking back in.
I have been super busy and have built my battery installed my inverter and have it all cabled up and working, that was a missions that took much longer than I had hoped.

So I have my ESP32 talking to the JKBMS via rs485 and to the goodwe inverter using can bus.
All works as per my esp32-example-can.yaml file.

I have a couple of anomalies that seem to be related to the data from the jkbms not being correct:
total_charging_cycle_capacity: always seem to be 0 yet the BMS app reports cycle capacity correctly.

The issue is that if the BMS turns off charging due to cell overvoltage this is not reflect in any of the charging switches below, so I can't notify the inverter and it gets upset. Is there any way to fix this?
dedicated_charger_switch: always reports OFF no matter what state charing is
charging: always reports ON no matter what state charing is
charging_switch: does report the app switch correctly

Thanks again!