Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Screen displays corrupted images on Mac #7

Open
opless opened this issue Feb 2, 2022 · 31 comments
Open

Screen displays corrupted images on Mac #7

opless opened this issue Feb 2, 2022 · 31 comments
Labels
bug Something isn't working

Comments

@opless
Copy link

opless commented Feb 2, 2022

Someone else has had a similar issue here:

https://youtu.be/4z5OOW1jL94

I’m also using an M1 Mac.

I’m also reverse engineering the vendor .net binaries to attempt to figure out what is going wrong.

Though I’m struggling a little with the obfuscation. 😂

it’s not something as simple as endian issues!

@opless
Copy link
Author

opless commented Feb 3, 2022

Looks like it's something weird with the serial comms or the operating system.

Closing because it's not fixable here.

@opless opless closed this as completed Feb 3, 2022
@mathoudebine
Copy link
Owner

Sorry to hear that, FYI the display you should expect is this one: https://www.youtube.com/watch?v=Y1m4KK881Aw
I had greater success in decompiling the chinese binary than the english one, it was not obfuscated. You will find the link of it in the README

@opless
Copy link
Author

opless commented Feb 3, 2022

I've had a look at all three, and yes, the one you mention is MUCH easier to work with.

It's a shame there's no way to get hold of the creators and have a look at their firmware source.

I'm considering knocking something up in golang, to see if it's just the serial port drivers are broken on the M1 or if it's just pyserial that's not working.

@alanrenouf
Copy link

Im also getting this issue on my Mac, dont get it on windows. the port shows up as "/dev/tty.usbmodemUSB35INCHIPSV21" @opless did you find a workaround ?

@opless
Copy link
Author

opless commented May 13, 2022

It's very much to do with the USB serial implementation, as far as I can tell.

I had some better luck sending data to it via golang, as the serial port initialisation is different.

I pretty much put the project aside after things getting busy with the day job and having less brain in the evening. It's certainly something weird with the M1 serial usb drivers though!

@mathoudebine mathoudebine reopened this May 16, 2022
@mathoudebine mathoudebine changed the title Screen displays images but is corrupted and weird. Screen displays corrupted images on Mac May 16, 2022
@mathoudebine
Copy link
Owner

Hi, i'm reopening this issue for people with Mac that have the same problem. Feel free to discuss here. What you can also try is changing the COM port settings here:

lcd_comm = serial.Serial(COM_PORT, 115200, timeout=1, rtscts=1)

If i remember correctly, the display also works with baudrates slower than 115200

@mathoudebine mathoudebine added bug Something isn't working help wanted Extra attention is needed labels May 16, 2022
@alanrenouf
Copy link

Tried slower baudrates and not getting any luck

@Ebag333
Copy link
Contributor

Ebag333 commented Jun 6, 2022

#15

I think I've fixed it by serializing the updates to the screen (using a queue, so it'll run through each one cleanly before executing the next).

With the default code base I could get it to step on itself, once I started adding a bunch more things to update it just got worse and worse. You could help by increasing the delay after an update was done, but slowing down the updates isn't really a solution as it makes everything slow.

With this PR it runs at a very high rate and has no issues with keeping up, not getting serial errors anymore.

@alanrenouf
Copy link

I still get the corrupt image using the overhaul branch on my mac, I like the other changes to make the code more modular. Nice!

@gerph
Copy link
Contributor

gerph commented Aug 29, 2022

From my own investigation:

In my testing in my own library, I found that the delay between the bitmap data being output and the next command being issued needs to be quite long - about 0.02 seconds. Shorter than this and you get corruption on the display. The current delay in the code on the master branch is about 0.01 seconds, which means that the other slack is being taken up by the code sitting between the time.sleep(0.01) and the next SendReg writing a command. On faster machines that code will be executed faster, and result in the corruption (I believe).

In my library I changed the behaviour so that instead of explicitly sleeping at the end of the update of a bitmap, we remember when the last output happened and if we try to write a command before the required inter-bitmap-delay has expired, we sleep there instead. This means that we get useful work done between the two operations, and if you did enough work you'd never need to sleep at all before the next command was written.

My library changes that did this were here, if they're useful to you:

gerph/turing-smart-screen-python-mit@8ba5901

It looks like the changes in PR #15, mentioned above, move the logic for the delay out of the communications library. I cannot tell how this keeps the bitmap transfers and the subsequent command updates apart enough to avoid the corruption.

@gerph
Copy link
Contributor

gerph commented Sep 1, 2022

I've tried out the feature/system-monitor branch, at f374841, with flagship display, on my macOS system... I tried running the main a few times, and observed corruption on each of them - the first run generally doesn't even display the main image. The main image also appears offset for some reason.

CorruptionInTuringDisplay.mp4

I'm not sure why it's doing that, but that's what I've seen with it so far - I may try and investigate later.

I suspect there's some race in the lcd_comm library - looking at lcd_comm_reb_b.py line 197, I see that we issue a SendCommand DISPLAY_BITMAP without the bypass_queue, and then go into loading the image and try to send it. Because bypass_queue wasn't set, the DISPLAY_BITMAP may not have been sent yet and the data for to send to the device may have been may get sent first. Additionally, the WriteData command isn't protected by the mutex, so it's possible that the queued data gets send DURING the image bitmap data.

@gerph
Copy link
Contributor

gerph commented Sep 1, 2022

Oh, I'm wrong about the data being sent during the bitmap update - SendLine is a queue operation, so it should be queued after the DISPLAY_BITMAP command. However, I think there's still a possibility that the SendCommand could be issued then something else get inbetween it and the data from the scheduler.

I'm really not at all convinced that this queue is the right way to go - for a start the queue code mixes the application-specific clients with the general queue dispatch, which means that if you want to re-use this code you have to copy out the queue handler code. And in any case, I'm not sure that it helps with any corruption problems at all.

@gerph
Copy link
Contributor

gerph commented Sep 1, 2022

I've tried replacing the queuing code with a simpler mutex-around-atomic operations with delay between the bitmap and next command. This has been solid on my flagship device. Unfortunately I cannot speak to the model A device... but I've tried making the necessary changes - it might not work at all on those models, and there might be a completely different processing model causing the problems there.

Here's the video of it working (again, tested 4 times to be sure it's not a fluke):

WorkingTuringDisplay.mp4

PR raised with the way that it worked for me. I've ordered a model A, but it'll be a while until it gets here.

#34

@gerph
Copy link
Contributor

gerph commented Sep 1, 2022

It still might be useful in some cases to have the serial comms on a separate thread, so it's possible the queue could be useful for that. However, that is still a bit complicated, and I didn't attempt it.

@9021007
Copy link

9021007 commented Sep 12, 2022

I have this issue as well, M1 Pro

@mathoudebine mathoudebine removed the help wanted Extra attention is needed label Sep 28, 2022
@SaplingLTD
Copy link

SaplingLTD commented Nov 13, 2022

Im facing same problem with the last version in MacOS. Model / Version A. Trying to figure out why it happened.
Untitled (1)

@gerph
Copy link
Contributor

gerph commented Nov 13, 2022

Oooh, it's built into a keyboard?! Is that a custom build or something that others can get hold of? (sorry I still don't know why there's corruption, though :-( )

@SaplingLTD
Copy link

SaplingLTD commented Nov 14, 2022

So far, I can run the python code normally to connect and give command to the screen.
I also tried to use this software on Windows https://wiki.keebmonkey.com/en/products/kbmscreen. It is recommended by the keyboard maker. And it worked fine. However, I want to use the keyboard with MacOS.

@mathoudebine
Copy link
Owner

mathoudebine commented Nov 14, 2022

Hi @SaplingLTD and welcome to this project

We didn't know the screen had been embedded into a keyboard! Looks really nice
Can you try to use the branch fix/replace-queued-data-send-with-immediate-plus-deferred-delay made by @gerph instead of main branch? It may solve the issue with MacOS

It is actually in PR #60 because of issues when the program stops: sometimes after a relaunch the screen is in an invalid state and the program crashes. When relaunched again it will then work.

@amiltonjr
Copy link
Contributor

@mathoudebine I just tested the branch fix/replace-queued-data-send-with-immediate-plus-deferred-delay in MacOS Ventura 13.1 with a Revision A display and it did not corrupt the display info! I think merging/rebasing it to the main will get it fixed and running with networking / GPU monitoring. @gerph if you do that, please let me know, then I'll test it out as well.

@phuongboss
Copy link

Confirmmed the branch fix/replace-queued-data-send-with-immediate-plus-deferred-delay fix the screen shuttering on macOS.

@mathoudebine mathoudebine pinned this issue Apr 26, 2023
@itsWobble
Copy link

I tried using the branch #60 I'm getting a COMM error "00:06:31 [ERROR] Cannot find COM port automatically, please set it manually in config.yaml".

@nukdokplex
Copy link

Got this on Manjaro Linux
IMG20230528205851

Here is logs:

мая 28 20:42:08 ndp-desktop python3[616]: 28.05.2023 20:42:08 [WARNING] Tray icon is not supported on your platform
мая 28 20:42:08 ndp-desktop python3[616]: 28.05.2023 20:42:08 [INFO] Display reset (COM port may change)...
мая 28 20:42:13 ndp-desktop python3[616]: 28.05.2023 20:42:13 [DEBUG] Auto detected COM port: /dev/ttyACM0
мая 28 20:42:13 ndp-desktop python3[616]: 28.05.2023 20:42:13 [INFO] HW revision A does not support backplate LED color setting
мая 28 20:42:13 ndp-desktop python3[616]: 28.05.2023 20:42:13 [DEBUG] Drawing Image: BACKGROUND
мая 28 20:42:13 ndp-desktop python3[616]: 28.05.2023 20:42:13 [DEBUG] Drawing Text: CPU_MODEL
мая 28 20:42:13 ndp-desktop python3[616]: 28.05.2023 20:42:13 [DEBUG] Drawing Text: GPU_MODEL
мая 28 20:42:13 ndp-desktop python3[616]: 28.05.2023 20:42:13 [DEBUG] Drawing Text: RAM_MODEL
мая 28 20:42:13 ndp-desktop python3[616]: 28.05.2023 20:42:13 [DEBUG] Drawing Text: DOWNLOAD_RATE
мая 28 20:42:13 ndp-desktop python3[616]: 28.05.2023 20:42:13 [DEBUG] Drawing Text: UPLOAD_RATE
мая 28 20:42:13 ndp-desktop python3[616]: 28.05.2023 20:42:13 [DEBUG] Drawing Text: DISK_USAGE
мая 28 20:42:13 ndp-desktop python3[616]: 28.05.2023 20:42:13 [INFO] Detected AMD GPU(s)
мая 28 20:50:44 ndp-desktop python3[616]: Exception in thread Queue_Handler:
мая 28 20:50:44 ndp-desktop python3[616]: Traceback (most recent call last):
мая 28 20:50:44 ndp-desktop python3[616]:   File "/usr/lib/python3.10/site-packages/serial/serialposix.py", line 621, in write
мая 28 20:50:44 ndp-desktop python3[616]:     n = os.write(self.fd, d)
мая 28 20:50:44 ndp-desktop python3[616]: OSError: [Errno 5] Ошибка ввода/вывода
мая 28 20:50:44 ndp-desktop python3[616]: During handling of the above exception, another exception occurred:
мая 28 20:50:44 ndp-desktop python3[616]: Traceback (most recent call last):
мая 28 20:50:44 ndp-desktop python3[616]:   File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
мая 28 20:50:44 ndp-desktop python3[616]:     self.run()
мая 28 20:50:44 ndp-desktop python3[616]:   File "/usr/lib/python3.10/threading.py", line 953, in run
мая 28 20:50:44 ndp-desktop python3[616]:     self._target(*self._args, **self._kwargs)
мая 28 20:50:44 ndp-desktop python3[616]:   File "/opt/turing/library/scheduler.py", line 74, in wrap
мая 28 20:50:44 ndp-desktop python3[616]:     scheduler.run()
мая 28 20:50:44 ndp-desktop python3[616]:   File "/usr/lib/python3.10/sched.py", line 151, in run
мая 28 20:50:44 ndp-desktop python3[616]:     action(*argument, **kwargs)
мая 28 20:50:44 ndp-desktop python3[616]:   File "/opt/turing/library/scheduler.py", line 64, in periodic
мая 28 20:50:44 ndp-desktop python3[616]:     action(*actionargs)
мая 28 20:50:44 ndp-desktop python3[616]:   File "/opt/turing/library/scheduler.py", line 163, in QueueHandler
мая 28 20:50:44 ndp-desktop python3[616]:     f(*args)
мая 28 20:50:44 ndp-desktop python3[616]:   File "/opt/turing/library/lcd/lcd_comm.py", line 99, in WriteData
мая 28 20:50:44 ndp-desktop python3[616]:     self.lcd_serial.write(bytes(byteBuffer))
мая 28 20:50:44 ndp-desktop python3[616]:   File "/usr/lib/python3.10/site-packages/serial/serialposix.py", line 655, in write
мая 28 20:50:44 ndp-desktop python3[616]:     raise SerialException('write failed: {}'.format(e))
мая 28 20:50:44 ndp-desktop python3[616]: serial.serialutil.SerialException: write failed: [Errno 5] Ошибка ввода/вывода

@evilpete
Copy link

evilpete commented Oct 1, 2023

I am seeing the issue, exactly like @SaplingLTD

MacAir (Intel Core i5) Monterey 12.6.8

@katanza12
Copy link

Same issue: MacBook Pro M1 Sonoma 14.0. I tried to remove background image, dynamic text and other graphics, put only a static text (starting with simple.program.py) but without success.

@a3rosol
Copy link

a3rosol commented Feb 23, 2024

I'll pile on. M1 Ultra. I had to specify the com_port manually and it didn't work with the tty device but seems to work most of the time with the cu device. you can find them using ls /dev/{tty,cu}.* mine shows up as /dev/cu.usbmodemUSB35INCHIPSV21 . But no matter which theme I chose, I always get artifacts on the screen when refreshes happen.

@jeffsf
Copy link

jeffsf commented May 8, 2024

Similar defects being seen on macOS 14.4.1 (Sonoma) and Intel in #512

If I've found the right object, changing from RTS/CTS to DSR/DTR did not seem to change the behavior

    # jmk -- try adjusting serial parameters
    lcd_serial: serial.serialposix.Serial = display.lcd.lcd_serial
    from pprint import pprint
    pprint(lcd_serial.get_settings())
    lcd_serial.rtscts = False
    lcd_serial.dsrdtr = True
    pprint(lcd_serial.get_settings())

    # Initialize the display
    display.initialize_display()

@Numbski
Copy link

Numbski commented Jun 12, 2024

Just chiming in - I am on an intel mac and getting this exact same behavior, so it isn't a CPU issue.

@Numbski
Copy link

Numbski commented Jun 12, 2024

Silly question - could it be the baud rate? Per the documentation, none of this is plain text. We're generating bitmaps and then pushing them to the serial port. Could be that the bitmap gets corrupted in the transfer process?

@opless
Copy link
Author

opless commented Jun 12, 2024

It's something to do with the mac drivers, I'm sure.

@oldirtdog
Copy link

Chiming in... I'm getting the same issue on my M3 Max... I'm pretty stuck here... sounds like this might not be for Mac folks :(

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests