Multi-display bugs

[leongc]

I have an external display as the main display connected via USB-C to HDMI adapter to my 2017 Macbook Pro running OS X 12.1 with the built-in retina as an extended display. Launching grid navigation with Scoot 0.8 overlays both display grids on the external display and none on the built-in retina display. Relatedly, launching element navigation only shows targets when the focused window is on the external display.

[mjrusso]

There may be a few different bugs at play here. The next release will have better logging, so it will be easier to see what's going on.

Launching grid navigation with Scoot 0.8 overlays both display grids on the external display and none on the built-in retina display.

Are you able to reproduce this reliably? After exiting and relaunching Scoot?

(Also, if you could repeat the testing, but with laptop as the main display, that would be helpful.)

[mjrusso]

In the next release, you'll be able to filter for subsystem:com.mjrusso.Scoot in Console.app, and easily find all relevant log messages.

Example of what this will look like:

[mjrusso]

Just released v0.9, which has support for the improved logging mentioned in my previous comment: https://github.com/mjrusso/scoot/releases/tag/v0.9

[leongc]

Reproduces reliably on launch: both grids overlaid on one display, none on the other.
Screenshot from Scoot launch when laptop display is main (external display is larger than laptop)

Console log follows for the following activity:

1. launch Scoot with external display as main.
2. show grid (both grids overlaid on external)
   
3. change main display to laptop (using System Preferences > Displays dragging menu bar with mouse, not Scoot)
4. show grid (external has no grid, laptop has one grid)
5. change main display to external
6. show grid (each display has the appropriate grid)
   

default	19:03:22.690193-0600	Scoot	Scoot: applicationDidFinishLaunching.
default	19:03:22.756824-0600	Scoot	For reference: the primary screen is at index 0, contains the menu bar, and has origin (0,0).
default	19:03:22.756870-0600	Scoot	Connected screens: 2
default	19:03:22.757451-0600	Scoot	* Screen 0: <private> <private>
default	19:03:22.757498-0600	Scoot	* Screen 1: <private> <private>
default	19:03:22.772917-0600	Scoot	Using <private> keybindings.
default	19:03:22.776515-0600	Scoot	Scoot invoked with frontmost app: <private>
default	19:03:30.142621-0600	Scoot	Scoot invoked with frontmost app: <private>
default	19:04:00.397681-0600	Scoot	Scoot: received didChangeScreenParametersNotification.
default	19:04:00.397727-0600	Scoot	Connected screens: 2
default	19:04:00.398231-0600	Scoot	* Screen 0: <private> <private>
default	19:04:00.398272-0600	Scoot	* Screen 1: <private> <private>
default	19:04:00.398314-0600	Scoot	> Screen frame has changed: <private> <private>
default	19:04:00.398384-0600	Scoot	> Screen frame has changed: <private> <private>
default	19:04:00.398432-0600	Scoot	Re-initializing relevant data structures...
default	19:04:02.893841-0600	Scoot	Scoot invoked with frontmost app: System Preferences
default	19:04:15.564682-0600	Scoot	Scoot: received didChangeScreenParametersNotification.
default	19:04:15.564719-0600	Scoot	Connected screens: 2
default	19:04:15.565767-0600	Scoot	* Screen 0: <private> <private>
default	19:04:15.565807-0600	Scoot	* Screen 1: <private> <private>
default	19:04:15.565845-0600	Scoot	> Screen frame has changed: <private> <private>
default	19:04:15.565913-0600	Scoot	Re-initializing relevant data structures...
default	19:04:20.246510-0600	Scoot	Scoot invoked with frontmost app: System Preferences

So that does reveal a workaround to flip-flop the main display after Scoot launches.

[mjrusso]

Awesome, thanks for the very detailed testing and write-up!

I just published a new release (v0.10), which adds additional logging for relevant window state (and ensures that the logs don't redact the screen/window/view frame rect details).

There's also a debug menu now, which has an option to "Rebuild Jump Window State". I'm sure that there's a bug with how Scoot handles screen changes (connecting/disconnecting a display, or changing the primary display); invoking this option might fix things after a screen change. Note that there also must be a separate issue with screen/window assignment, otherwise the app wouldn't launch with multiple windows on the same screen.

If you could give this new build a try and post your logs, it will help me track this down. Thanks!

https://github.com/mjrusso/scoot/releases/tag/v0.10

[leongc]

1. launch Scoot 0.10 (1)
2. use grid-based navigation (both grids overlaid on external monitor)
3. log screens and ui state
4. rebuild jump window state (grid appears appropriately on each monitor)

default	17:36:42.750005-0600	Scoot	Scoot: applicationDidFinishLaunching.
default	17:36:42.829829-0600	Scoot	For reference: the primary screen is at index 0, contains the menu bar, and has origin (0,0).
default	17:36:42.829894-0600	Scoot	Number of connected screens: 2
default	17:36:42.830650-0600	Scoot	* Screen #1[/2]: <mask.hash: 'b2HAnEz5SISg8HprszVclg=='> (0.0, 0.0, 2560.0, 1440.0)
default	17:36:42.830698-0600	Scoot	* Screen #2[/2]: <mask.hash: 'jP1AFqjjPkk6OwA+g8/RcA=='> (423.0, -1050.0, 1680.0, 1050.0)
default	17:36:42.851954-0600	Scoot	Number of Jump Window Controllers: 2
default	17:36:42.851979-0600	Scoot	* Jump Window Controller #1[/2]:
default	17:36:42.852024-0600	Scoot	** Assigned Screen: <mask.hash: 'b2HAnEz5SISg8HprszVclg=='>, (0.0, 0.0, 2560.0, 1440.0)
default	17:36:42.852052-0600	Scoot	** Jump Window: (0.0, 0.0, 2560.0, 1415.0)
default	17:36:42.852081-0600	Scoot	** Jump View Controller: (0.0, 0.0, 2560.0, 1415.0)
default	17:36:42.852102-0600	Scoot	* Jump Window Controller #2[/2]:
default	17:36:42.852129-0600	Scoot	** Assigned Screen: <mask.hash: 'jP1AFqjjPkk6OwA+g8/RcA=='>, (423.0, -1050.0, 1680.0, 1050.0)
default	17:36:42.852335-0600	Scoot	** Jump Window: (423.0, 0.0, 1680.0, 1050.0)
default	17:36:42.852366-0600	Scoot	** Jump View Controller: (0.0, 0.0, 1680.0, 1050.0)
default	17:36:42.852467-0600	Scoot	Using vi keybindings.
default	17:36:42.858149-0600	Scoot	Scoot invoked with frontmost app: <mask.hash: 'DeRMAmJ67WAypRB9BUE6NQ=='>
default	17:36:43.003184-0600	Scoot	Using element jump mode
default	17:36:56.101573-0600	Scoot	Debug: logging screen configuration and UI state.
default	17:36:56.101605-0600	Scoot	Number of connected screens: 2
default	17:36:56.101659-0600	Scoot	* Screen #1[/2]: <mask.hash: 'b2HAnEz5SISg8HprszVclg=='> (0.0, 0.0, 2560.0, 1440.0)
default	17:36:56.101712-0600	Scoot	* Screen #2[/2]: <mask.hash: 'jP1AFqjjPkk6OwA+g8/RcA=='> (423.0, -1050.0, 1680.0, 1050.0)
default	17:36:56.101743-0600	Scoot	Number of Jump Window Controllers: 2
default	17:36:56.101785-0600	Scoot	* Jump Window Controller #1[/2]:
default	17:36:56.101882-0600	Scoot	** Assigned Screen: <mask.hash: 'b2HAnEz5SISg8HprszVclg=='>, (0.0, 0.0, 2560.0, 1440.0)
default	17:36:56.101977-0600	Scoot	** Jump Window: (0.0, 0.0, 2560.0, 1415.0)
default	17:36:56.102114-0600	Scoot	** Jump View Controller: (0.0, 0.0, 2560.0, 1415.0)
default	17:36:56.102299-0600	Scoot	* Jump Window Controller #2[/2]:
default	17:36:56.102442-0600	Scoot	** Assigned Screen: <mask.hash: 'jP1AFqjjPkk6OwA+g8/RcA=='>, (423.0, -1050.0, 1680.0, 1050.0)
default	17:36:56.102488-0600	Scoot	** Jump Window: (423.0, 0.0, 1680.0, 1050.0)
default	17:36:56.102554-0600	Scoot	** Jump View Controller: (0.0, 0.0, 1680.0, 1050.0)
default	17:37:00.738641-0600	Scoot	Scoot invoked with frontmost app: <mask.hash: 'DeRMAmJ67WAypRB9BUE6NQ=='>
default	17:37:00.833477-0600	Scoot	Using grid jump mode
default	17:37:11.097917-0600	Scoot	Debug: rebuilding all jump windows.
default	17:37:11.118678-0600	Scoot	Scoot invoked with frontmost app: <mask.hash: 'DeRMAmJ67WAypRB9BUE6NQ=='>
default	17:37:11.228741-0600	Scoot	Using grid jump mode

[mjrusso]

@leongc thank you for repeating the testing and posting the log. I have the info I need now to track this down.

That being said, I really did not expect rebuilding the state to fix the window placement. 🤔

[mjrusso]

Just did a bunch of testing; here's what I've found:

1. Most changes to the screen configuration (in System Preferences), such as changing the position of a display, cause issues (usually, a window will end up misplaced; the bug is likely that the window isn't properly moved to reflect its new location). Rebuilding the windows fixes the problem, which makes sense.
2. If the primary display is not set to the largest display, and you move the cursor with Scoot, it usually won't end up in the right location. I know what the root cause is. (Rebuilding the windows does not help, which is what I would expect, giving the nature of bug.)
3. I have tried every screen configuration option I can think of, but I have never been able to reproduce the issue where Scoot launches and the windows are not already placed correctly. I've set up my monitors exactly as you have (bigger screen as primary on top, with smaller screen contained below), but the windows always start in the right location. (Come to think of it, I haven't tried connecting an external monitor over HDMI. Unlikely to be related, but I'll give it a shot just to be sure.)

Regardless, fixing the first two problems is a good start.

[leongc]

I just remembered a related setting. I configured Mission Control to let windows span screens. Unselect "Displays have separate Spaces".
See https://www.imore.com/how-span-window-between-two-displays-mavericks

[mjrusso]

Ah, that must be related, thanks for flagging. I will confirm later, but it's almost certainly the case that there's a startup issue when windows are allowed to span screens.

(Separately, there may be problems with using Scoot across Spaces; I've been meaning to test this, but haven't got around to it yet. And come to think of it, when an app enters "native" full screen, it gets its own space, and also may not work properly. I don't actually use Spaces or full screen... 😅)

[mjrusso]

(Separately, there may be problems with using Scoot across Spaces; I've been meaning to test this, but haven't got around to it yet. And come to think of it, when an app enters "native" full screen, it gets its own space, and also may not work properly. I don't actually use Spaces or full screen... 😅)

Yup, both scenarios don't work properly: Scoot draws its UI on the first desktop, not the active one. (This is with only a single display connected. Multiple displays is probably an even bigger mess...)

I'll handle this separately. In terms of this ticket, my priority is:

1. Properly handle cursor movement when the primary display is not the largest display.
2. Properly supporting/ responding to display changes.
3. Fix starting window placement, when user has allowed windows to span spaces.

[mjrusso]

Some updates:

Properly handle cursor movement when the primary display is not the largest display.

Fixed in f9a25b2.

Properly supporting/ responding to display changes.

Fixed in 6ab703f.

[re: Spaces] ... both scenarios don't work properly: Scoot draws its UI on the first desktop, not the active one.

Fixed in cf129ce, supporting both "normal" Spaces, and apps in fullscreen. (Although I haven't tested that both of these scenarios work with multiple displays connected, yet.)

Remaining:

Fix starting window placement, when user has allowed windows to span spaces.

I haven't tested this yet. Once I do, I'm hoping I'll be able to reproduce the bug you're seeing.

[mjrusso]

More updates:

• I've tested the new Spaces support, with multiple displays connected, in both "displays have separate Spaces" mode, and "displays do not have separate Spaces" mode, and it seems to work properly.
• I'm still not able to reproduce the issue with the windows starting in the wrong place, even with "displays have separate Spaces" disabled. I've tried with mimicking your screen configuration exactly, with both v0.10 and the latest code on main (which has the fixes described in my previous message applied). Even on v0.10 I'm not able to repro. Not sure what else this could be... if it's some sort of race condition (which is conceivable given that rebuilding the windows after app launch fixes the issue), it's a really odd one, as Scoot does everything on the main thread.

Also, I did find what I thought was a new bug where Scoot is unable to find elements in focused apps on the non-primary display. However, on closer review, this is likely the same bug in your initial report:

Relatedly, launching element navigation only shows targets when the focused window is on the external display.

An important detail I realized, however, is that this bug only manifests if the non-primary display is placed immediately above or below the primary. Other display configurations don't have this problem, which explains why I never personally saw this before.

(I also confirmed that this bug is present regardless of the "displays have separate Spaces" setting.)

[mjrusso]

Alright, we're pretty much there.

Also, I did find what I thought was a new bug where Scoot is unable to find elements in focused apps on the non-primary display. However, on closer review, this is likely the same bug in your initial report:

Relatedly, launching element navigation only shows targets when the focused window is on the external display.

An important detail I realized, however, is that this bug only manifests if the non-primary display is placed immediately above or below the primary. Other display configurations don't have this problem, which explains why I never personally saw this before.

This is fixed by 79f2756.

Re: the bug with multiple windows starting on the same (wrong!) screen, I still haven't been able to reproduce the issue. However, I did add a message to the README (see 895f30d) that provides instructions on what to do if this happens to you.

[mjrusso]

@leongc I just published v0.11, which includes all of the fixes described above: https://github.com/mjrusso/scoot/releases/tag/v0.11

I'm assuming that it still may launch in a broken state (with overlapping windows), but clicking the “Rebuild Jump Windows” debug menu item will still fix this particular issue.

If you could give it a spin and let me know if everything else is working reliably, that would be great.

[leongc]

Confirmed v0.11 launches with overlapping grids and clicking "Rebuild Jump Windows" debug menu item distributes grids to appropriate displays. Everything else seems normal. I do like the grid filter-as-you-type highlighting.

[leongc]

I may have spoken too soon. The tinting doesn't seem to go away after grid navigation. Clicking with scoot or the actual pointer dismisses the grid, but the tint remains. Dismissing grid jump with esc removes both the grid and the tint. Not sure if that is multi-display or a separate tint bug.

[mjrusso]

The tinting doesn't seem to go away after grid navigation.

Are you referring to tint added via the "Toggle Jump Window Tint" debug menu item? If so, that is expected. That (debug) feature adds a tint to every window, independent of the grid or any other Scoot UI. The idea is to make it easier to see where the windows are actually located, for debug purposes.

If enabled, whether or not you see the tinting depends on the z-order of Scoot's jump window, relative to the other windows on your screen, at the given level. (If you hit escape, it will actually hide the window, so you won't see it.)

That being said, one thing that has changed (since v0.10) is that the level of the window is now set to floating (see cf129ce), so it will generally be on top of everything else on your screen.

[mjrusso]

Confirmed v0.11 launches with overlapping grids and clicking "Rebuild Jump Windows" debug menu item distributes grids to appropriate displays.

Great, thanks for confirming. Might take some trial-and-error to find a fix for this.

[xlboy]

Confirmed v0.11 launches with overlapping grids and clicking "Rebuild Jump Windows" debug menu item distributes grids to appropriate displays. Everything else seems normal. I do like the grid filter-as-you-type highlighting.

Me too. But I'm glad that the problem can be solved. In addition, thank you very much @mjrusso for developing such a useful application! 😘