Create v5 schema for settings.json

[jandubois]

We need to apply all pending changes to our settings format before implementing deployment profiles (#3649), as they will share the same internal structure. We don't want IT administrators to have to migrate their profiles to a new schema.

Current schema

Example of v4 `settings.json` file

{
  "version": 4,
  "containerEngine": {
    "imageAllowList": {
      "enabled": false,
      "locked": false,
      "patterns": []
    }
  },
  "kubernetes": {
    "version": "1.25.5",
    "memoryInGB": 6,
    "numberCPUs": 2,
    "port": 6443,
    "containerEngine": "containerd",
    "checkForExistingKimBuilder": false,
    "enabled": true,
    "WSLIntegrations": {},
    "options": {
      "traefik": true,
      "flannel": true
    },
    "suppressSudo": false,
    "hostResolver": true,
    "experimental": {
      "socketVMNet": false
    }
  },
  "portForwarding": {
    "includeKubernetesServices": false
  },
  "images": {
    "showAll": true,
    "namespace": "k8s.io"
  },
  "telemetry": true,
  "updater": false,
  "debug": false,
  "pathManagementStrategy": "rcfiles",
  "diagnostics": {
    "showMuted": false,
    "mutedChecks": {}
  }
}

Considerations

• We should structure the settings to match the organization inside the application or preference pages. We will auto-generate rdctl set options from setting names, so it should be easy for users to connect the names to locations in the app or dialog. For example virtualMachine.memoryInGB becomes --virtual-machine.memory-in-gb.
• We will not include substructures for tabs on each preference page, as we may need to move settings around as additional properties are added.
• Properties should all be inside some container, so we can differentiate between e.g. rdctl start --debug and rdctl start --application.debug=true.
• Property names should resemble the labels used in the UI.

We need to keep these rules in mind as we add additional settings, as they will become user-visible now.

Short names?

Should we use short names for pages (e.g. ce and kube instead of containerEngine and kubernetes), or should we use long names in the settings file (and the source code) and support shorter aliases for the CLI/API?

Suggested abbreviations:

App pages

• general (all settings should be via application)
• portForwarding → ports
• images (no abbreviation)
• troubleshooting (do we even need this one?)
• diagnostics → diag

Properties dialog

• application → app
• virtualMachine → vm (not on Windows)
• WSL (short enough, only on Windows)
• containerEngine → ce
• kubernetes → kube (or k8s?)

Modifications

Remove obsolete fields

• kubernetes.checkForExistingKimBuilder (the setting should already be unused)

Changes

• kubernetes.memoryInGB → virtualMachine.memoryInGB
• kubernetes.numberCPUs → virtualMachine.numberCPUs
• kubernetes.containerEngine → containerEngine.name
• kubernetes.WSLIntegrations → WSL.integrations
• kubernetes.suppressSudo → application.adminAccess (note: logic inverted)
• kubernetes.hostResolver → virtualMachine.hostResolver.enabled
• kubernetes.experimental → virtualMachine.experimental
• telemetry → application.telemetry.enabled (so we can add more settings)
• updater → application.updater.enabled (so we can add more settings)
• debug → application.debug (should we have troubleshooting.debug?)
• pathManagementStrategy → application.pathSetting (name???)`

These main app settings remain unchanged

• portForwarding
• images
• diagnostics

Work breakdown

• Finalize new settings layout
• Update Rancher Desktop to use new settings format
• Implement migration from v4 format during upgrades
• Update rdctl with options for backwards-compatibility

Acceptance criteria

• Verify that settings.yaml corresponds to the agreed layout
• Run full manual tests (on one platform) to verify refactoring has not broken anything
• Upgrade from 1.7.0 with non-default settings and verify that they migrate correctly
• Run rdctl set with old options to make sure they still work

[torchiaf]

It could be a good opportunity (?) to do some code refactoring in httpCommandServer.ts.
Probably we may want to create basic CRUD methods to standardize requests & responses handling; At the moment we have some duplicates doing that.

Some Examples:

rancher-desktop/pkg/rancher-desktop/main/commandServer/httpCommandServer.ts

Lines 200 to 202 in 2b0899f

	console.debug('diagnosticCategories: succeeded 200');
	response.writeHead(200, { 'Content-Type': 'application/json' });
	response.write(jsonStringifyWithWhiteSpace(categories));

rancher-desktop/pkg/rancher-desktop/main/commandServer/httpCommandServer.ts

Lines 227 to 229 in 2b0899f

	console.debug('diagnostic_ids: succeeded 200');
	response.writeHead(200, { 'Content-Type': 'application/json' });
	response.write(jsonStringifyWithWhiteSpace(checkIDs));

rancher-desktop/pkg/rancher-desktop/main/commandServer/httpCommandServer.ts

Lines 480 to 481 in 2b0899f

	response.writeHead(200, { 'Content-Type': 'text/plain' });
	response.write(transientSettings);

[jandubois]

It could be a good opportunity (?) to do some code refactoring

I'm all in favour of simplifying the code, but it should be a separate follow-up task to changing the schema.

The schema changes will be quite big (all references will have to be updated too), so I don't want refactoring mixed into the same PR.

[mook-as]

(Should this be a discussion before making an issue to actually implement the discussed results?)

I believe we should avoid ambiguous abbreviations for talking to other programs (instead of people); in particular, the ce abbreviation for containerEngine is very opaque. kube at least has the advantage of being unambiguous.

We can have separate logic to handle abbreviations in rdctl instead where the user might type them on the command line.

I wonder if it would make sense to group WSL.integrations, application.adminAccess, and application.pathSetting to a new branch under host instead (i.e., host.WSLIntegratinos, host.adminAccess, and host.pathSetting / host.pathManagement).

[jandubois]

(Should this be a discussion before making an issue to actually implement the discussed results?)

Should have been a discussion, but we already had an issue for it, so I just updated it instead.

I believe we should avoid ambiguous abbreviations for talking to other programs (instead of people); in particular, the ce abbreviation for containerEngine is very opaque. kube at least has the advantage of being unambiguous.

Agreed. The actual field names should use the long form.

We kind of agreed on Slack already that we'll hold off for now on adding abbreviations to rdctl as well, but we may add them at a later point.

We can have separate logic to handle abbreviations in rdctl instead where the user might type them on the command line.

I wonder if it would make sense to group WSL.integrations, application.adminAccess, and application.pathSetting to a new branch under host instead (i.e., host.WSLIntegratinos, host.adminAccess, and host.pathSetting / host.pathManagement).

It's an interesting idea. My suggestion come from the desire that the top-level structure name should match the page in the app or preference dialog where you can modify the value.

How would you decide which settings are host settings, and which ones are application settings?

[jandubois]

I wonder if it would make sense to group WSL.integrations, [..] to a new branch under host instead (i.e., host.WSLIntegratinos, [...]).

Alternative question: would it make sense to move WSL from its own page to just a tab under Application?

Related: Would you want to rename the Application page to Host? (Personally, I would not, but let me know any reasons why you might prefer home).

[mook-as]

I was putting things that affect host integration into host (i.e. WSL integrations, the sudo stuff for docker socket integration, shell integration). I left the stuff that doesn't affect other applications on the host (telemetry, logging, updater) under application.

The current application page (on Windows) contains updates & telemetry, so I would definitely not advocate for renaming that; if anything, we might rename "WSL" to "Integration" and move sudo/environment there? But I'm not really advocating for UI changes here though :)

[jandubois]

I was putting things that affect host integration into host

I'll bring this up in the next UI/UX meeting (forgot to put it on the agenda for today). If we want to have separate "Host" and "Application" pages in the Preferences in the future (and move the WSL page into a Home tab), then I would be willing to change the structure layout already, and modify the UI later.

If we are not planning to change the Preference tabs, then I would want to keep the settings under the existing page names.

[jandubois]

I've created a separate discussion #3762 just for the host issue, and asked for feedback in our UI/UX channel. We need to decide this by early next week, as updating the schema is the first work item in the Deployment Profiles epic #3649, which is already on a tight schedule.

[adamkpickering]

I have two points:

• I'd avoid shortening names altogether. Names that are in title case in the GUI can be translated directly into camel case in the settings and in rdctl. That way, there is less for users (and us developers) to remember when working with settings. It is more verbose, but having multiple names for a setting across the application makes it harder to understand what is going on. And as @mook-as pointed out, it can introduce ambiguity.

• We should consider how plugin configuration fits into this. Forgive me if this has been discussed elsewhere. I could see something like plugins.pluginName.specificPluginSetting working. Or perhaps each plugin would have its own settings file.

[jandubois]

• I'd avoid shortening names altogether. Names that are in title case in the GUI can be translated directly into camel case in the settings and in rdctl. That way, there is less for users (and us developers) to remember when working with settings. It is more verbose, but having multiple names for a setting across the application makes it harder to understand what is going on. And as @mook-as pointed out, it can introduce ambiguity.

We already agreed not to do abbreviations at this time.

If we introduce them in the future, they would purely be aliases for the CLI option names, and not part of the API. We can discuss this more once we actually consider adding them.

• We should consider how plugin configuration fits into this. Forgive me if this has been discussed elsewhere. I could see something like plugins.pluginName.specificPluginSetting working. Or perhaps each plugin would have its own settings file.

It has not been discussed yet. I agree that they will have to be in their own structure.

As you can probably tell by now, I will want to align the page names in the app with the names of the top level objects in settings, so that will have to be determined together. I don't think we need to decide this right now.

[ericpromislow]

Just a note that one of the easier things in this work will be to migrate existing settings.json files (version 4) to the next format -- the config system has been designed from the start to handle breaking changes going forward. I'm pretty sure that forward-compatibility is implemented in so far as unrecognized fields in the settings file are ignored but retained.