Seemingly random crash when switching from Library to Develop module

[hvilches]

Hello,
I've had this issue for a couple of years, going back several versions of Midi2LR and LR itself, and even macOS. I thought it was my large catalog of almost 500K images, but then came across (#489), which has the exact same error dialog under different circumstances and not random, but similar in the final event, different versions.

Midi2LR 5.2.0.0
LR Classic 11.0
macOS Catalina
Catalog is about 500K

Both the catalog and photos are pulled from SSDs. The catalog has been rebuilt several times, as a course of new LR versions doing so on install, and as an attempt to fix the error.

Seemingly random, since I can't replicate on demand, shortly after opening LR and browsing/editing photos, there comes a time when switching from Library to Develop module causes the error:

This can happen even if you do not quickly switch between images and modules, again seeming randomly, yet always within a few minutes of opening LR. Does not happen on the same image twice.

When I was able to use Midi2LR, it was awesome, and it would be great to find a solution... Thank you!

[rsjaffe]

1. How do you switch from Library to Develop? Is it via MIDI2LR or directly on the computer?
2. Could you post your MIDI2LR log? ~/Library/Logs/MIDI2LR/MIDI2LRinfo.txt

[hvilches]

1. I switch from Library to Develop directly on the computer, with the D shortcut.
2. Here's the last part, and I just saw the Command not found errors. Tons of those, and then this:

2021-12-17T15:25:56.965-05:00: Command not found in CommandTextIndex: GraduatedFilter.
2021-12-17T15:25:56.965-05:00: Command not found in CommandTextIndex: RadialFilter.
2021-12-17T15:25:56.966-05:00: Command not found in CommandTextIndex: GraduatedFilter.
2021-12-17T15:25:56.966-05:00: Command not found in CommandTextIndex: RadialFilter.
2021-12-17T15:25:56.966-05:00: Command not found in CommandTextIndex: AdjustmentBrush.
2021-12-17T15:25:56.966-05:00: Command not found in CommandTextIndex: EnableSplitToning.
2021-12-17T15:25:56.969-05:00: LR_IPC_In Read error: End of file.
2021-12-17T15:25:56.970-05:00: Stopped input device X-TOUCH MINI.
2021-12-17T15:25:56.970-05:00: LR_IPC_In socket shutdown error Socket is not connected.
2021-12-17T15:25:56.971-05:00: Default profile saved to /Users/hvilches 1/Library/Application Support/MIDI2LR/default.xml.
2021-12-17T15:25:56.974-05:00: ControlsModel archive in Main saved to /Users/hvilches 1/Library/Application Support/MIDI2LR/settings.xml.

Thank you!

[hvilches]

If I had to guess, there is a race condition where not all the data related to the image is ready when it's expected to be. Of course, pure speculation.

[hvilches]

I feel bad saying it, but it didn't fix it. However, is it possible for this code to spit out the state of some vars when going from Library to Develop? I guess if this were possible, you'd be doing it on the existing log, but maybe this could be the 'verbose' version that you wouldn't want to run with full-time?

Also, thank you for the help!

[rsjaffe]

Here's what I believe is happening:

1. The plugin detects that you've changed to the Develop module (it checks for that about 3-4 times/second).
2. It switches to the new profile and queries Lightroom for all the develop settings so that it can update the MIDI controller with the new profile mapping values.
3. Sometimes the queries to Lightroom overwhelm Lightroom just as it is changing modules and doing its own internal housekeeping.

My first fix was to yield to Lightroom very briefly before doing the change and queries. Here is a different approach: the attached version doesn't change profiles the first time it detects a change from non-Develop to Develop module, resulting in a very brief (1/3 second) delay before querying for all those values. Could you see if the attached revised file fixes it? As before, unzip and insert into your MIDI2LR.lrplugin directory.
Profiles.zip

[hvilches]

Ok, this is interesting. I placed this new file back in, restarted LR. First time into Develop, it gave me the Internal Error. I restarted LR once more, and it gives Internal Error on the first switch to Develop. Then, instead of restarting LR, I used Plugin Extras > MIDI2LR > Start Application and waited for it to connect. can't crash it!

I then changed the extension to Profiles.lua_HALT, restarted LR and got it to crash. Restarted using Plugin Extras > MIDI2LR > Start Application and waited for it to connect... can't get it to crash!

This has to be a decent clue. Why does this work? Does MIDI2LR not completely initialize when its started alongside LR?

[rsjaffe]

When you restart using Plugin Extras > MIDI2LR > Start Application, you are not restarting the plugin, only the app. To restart the plugin, you have to go to Plug-in Manager... > MIDI2LR > Reload Plug-in (under Plug-in Author Tools in the Plug-in manager dialog).

If you rename Profiles.lua, the plugin will crash. Lightroom doesn't make it obvious all the time, but if you look in Plug-in Manager, you'll see it's not loaded.

MIDI2LR plugin loads when LR starts and stays loaded until LR stops or you reload it using Plug-in Manager. The app can be started and stopped.

Changes to the lua files after the plugin is loaded generally (but not always) will be ignored until the plugin is reloaded or LR is restarted.

Given that, I'm confused about what happened when you renamed Profiles.lua. Also, when you said it didn't crash, did you check to see whether it works? If the plugin isn't working, the MIDI controller won't have any effect on LR, even though the app may show something.

[hvilches]

Ok, I failed reading comprehension. You specifically said to insert the file in the zip, into the plugin directory. What did I do? Of course, put it in the Profiles directory. Sorry for confusing the process. I tried both Profile.lua this morning, and neither fixed it.

With the latest Profiles.lua, I load LR, and as described, the internal error occurs at some point. I note that the app is no longer running, and the MIDI controller is no longer working. Then, not knowing the difference between the app and plugin (I didn't realize there are two things working together, but now that makes sense.), I do Plugin Extras > MIDI2LR > Start Application and waited for it to connect, and then the MIDI controller DOES work, and the internal error appears to have resolved.

Does this suggest the issue is occurring at the app, which terminates when the internal error dialog appears? Since I get it to work merely by restarting the app, then the plugin never crashed.

It may be obvious, but I will note it anyway. After I have used Plugin Extras > MIDI2LR > Start Application, when I exit LR, it does not terminate the app. I need to do this manually, and then I load LR again.

[rsjaffe]

From that description, I'm unsure as to what needs to be done, if anything, to fix a MIDI2LR bug. Please report back when you have more info.

[hvilches]

Understood. I have not had any problems since the last time I posted, and still using the same LR session. The method that got it working (to do Plugin Extras > MIDI2LR > Start Application) did not always work. I remember trying this well over a year or more ago and had not tried it again because it hadn't helped.

I will add more when I notice it and a huge thank you!

[onegrasshopper]

I could repro this putting lots of profiles as @hvilches has and switching back and forth Lib<->Dev. Disabling the constant Profiles.checkProfile() in the main task prevents the issue. Manually assigning a button to change profiles avoids (well, probably just highly reduces probability) errors.

The race condition is probably hard to address since one has to assume LR internals can change. The implementation of the auto profile/tool/panel checking/switching should be addressed but that's out of this thread of discussion?

[rsjaffe]

Thanks. Could you see if this fixes the issue? Replace Profiles.lua with the version from this zip file Profiles.zip It introduces a delay when switching to Develop, which seems to be the one that's causing most of the issues.

[onegrasshopper]

It changes the behavior a little, but it is still possible to time things to mess with it. It looks like the "delay" introduced is simply using isDevelop to push to checkProfile to actually update the next time? This makes it more like a fudge factor that will come back to bite us again.
I suspect in the code's current context, it is better to for doprofilechange not to be chatty so quickly? Since LrDevelopController, etc (such as no photo selected (possible to have no photo selected and switch to develop) might not be ready from the switch to Develop.

[rsjaffe]

I believe it's not the check for develop/photo selected, but the storm of requests for develop settings that occur after that. Could you comment out the develop parameter requests and see if you still get that issue? If so, I'm going to have to switch to something I've been hesitating to do, because it introduces more complications: asynchronously getting the settings with LrApplication.activeCatalog():getTargetPhoto():getDevelopSettings() and reading each from the structure that's returned (and the structure has naming issues with some of the settings that I'll have to work around).

If it is the check for develop/photo selected that is crashing things, then the problem is: I need to delay to check module status until x milliseconds after the module has changed, but I won't know the module has changed without checking module status.

[onegrasshopper]

yes, agree it might be the parameter requests (what i meant by "chatty"). I came to that notion by commenting the LR requests. There is no check for photo selected in the checkprofile->changeprofile->doprofilechange in the version of code I see.
We could try yielding by pulling the Limits.LimitsCanBeSet() block into a separate function to start async? It would clean up the 3 cut and paste duplicates as well for this block.

[rsjaffe]

Could you check the latest release (in beta) to see if it fixes it. I was unable to replicate the issue, even with different profiles in each slot, but did identify two potential problems that I've fixed. See #613 (comment)

[hvilches]

Ok, I closed LR, installed 5.3.0.0, and restarted LR. The MIDI2LR app starts as expected, reports 'Connected to Lightroom', but there is no functionality anymore. Nothing on the MIDI controller is making any changes in LightRoom.

Then, I closed LR, and it did not close MIDI2LR, so I closed it manually.
Start LR, starts the MIDI2LR app, and I wait a bit for the catalog to load, pick a photo, press 'D', but LR isn't responding to controller input. Tried several photos.

One thing I am noticing, is that even if nothing is happening in LR, the controller LEDs are changing in response to encoder rotation, and I think this indicates MIDI2LR is communicating with it, just not with LR. Yes?

Using the Help to Generate Diagnostic report:
MIDI2LR.log

[rsjaffe]

MIDI2LR app and plugin communicate through two ports. The port from the plugin to the app is working but the port from the app to the plugin isn't, for some odd reason. Check your firewall: the app should have permission to access both ports 58763 and 58764.

[hvilches]

I'm on a Mac, so unless I start editing /etc/pf.conf manually, all a user has is the Firewall GUI that lets you pick an app to allow/block connections, and not ports. I added LightRoom to this without effect, and I cannot pick the MIDI2LR app. The plugin is under Library, which cannot ordinarily be opened, and I am not sure I could select it, since it isn't a .app? Also, when an app needs some kind of access, the OS prompts to allow it and this hasn't happened either.

Since LR triggers MIDI2LR to load, does it not inherit this access?

To test, I turned OFF the firewall and it didn't have an effect either. Hard to tell, because I have no way of verifying this.

[rsjaffe]

I've now determined that version 5.3 has a bug for macOS users. I'm not sure where it originated, as I didn't change the code related to the port connections, but I did update a third party library, which is probably where the problem is. I'll have a new version to test soon. Thanks for your help.

[rsjaffe]

Could you try 5.3.0.1? I've rebuilt it with a newer version of the framework. I went through all the changes between 5.2 and 5.3, and nothing should have affected port function. Thinking it's either a permission problem or a third-party library problem. Updated main third-party library and rebuilt.

[rsjaffe]

Could you try 5.3.0.2? I've rebuilt the macOS version again.

[hvilches]

Sorry, life happened...Very well done, I think you did it! I was only able to test with the latest release, and I can't get it to crash. It is handling the switching without any hiccups. {fireworks}

I will continue using and report any issues, related or not. Huge THANK YOU!

[hvilches]

I have a follow up to this. Something still not right, because twice this week, I've had to restart the MIDI2LR application. Plugin extras > Close, then Plugin extras > Open. This restores functionality.

So the original issue may still be there, just failing quietly and I am not noticing when it fails. Normally I am switching and outputting many images, without doing adjustments, so I won't notice when a problem happens.

When it its failing, the midi controller is reacting to input and the leds changing as a result.

MIDI2LR.log

[rsjaffe]

Could you delete the file /Users/hvilches 1/Library/Application Support/MIDI2LR/MenuTrans.xml then start up Lightroom again? It looks like there may be issues with that file.

[hvilches]

Done. Will report what I find in the coming days. TY

[hvilches]

Following up, a month later, this is still failing quietly, so I have been unable to reproduce. The logs only show what you'd expect, even though LR is no longer reacting to controller input. The log attached is with LR no longer reacting to controller input.

MIDI2LR.log