Support for security-only updates using Dependabot CLI

[rhyskoedijk]

Fixes #1490

This change adds support for "security-only" updates in TaskV2 using Dependabot CLI.

Security-only updates (i.e. open-pull-requests-limit: 0) incur a slight performance overhead due to limitations in Dependabot CLI (#360). To work around this, vulnerable dependencies will first be discovered using an "ignore everything" update job; After which, security advisories for the discovered dependencies will be checked against the GitHub Advisory Database before finally performing the requested security-only update job. It might be possible to optimise this process in the future, but for now this is the best that I can come up with.

High-level sequence diagram of security-only update process

 sequenceDiagram
    participant ext as TaskV2
    participant cli as Dependabot CLI
    participant gha as GitHub Advisory Database

    ext->>ext: Write `list-dependencies-job.yml`
    Note right of ext: The job file contains `ignore: [ 'dependency-name': '*' ]`.<br>This will make Dependabot to discover all dependencies, but not update anything.<br>We can then extract the dependency list from the "depenedency_list" output.
    ext->>+cli: Execute `dependabot update -f list-dependencies-job.yml -o output.yml`
    cli->>cli: Run update job
    cli->>cli: Write `output.yaml`
    cli-->>-ext: Update completed

    ext->>ext: Read and parse `output.yaml`, extract "dependency_list"
    loop for each dependency
      ext->>gha: Check security advisories for dependency
    end
    ext->>ext: Filter dependency list to only ones containing security advisories
    ext->>ext: Write `security-only-update-job.yml`
    Note right of ext: The job file contains the list of `dependency-names` and `security-advisories`.<br>This will make Dependanbot only update the dependencies named in the job file.
    ext->>+cli: Execute `dependabot update -f security-only-update-job-job.yml -o output.yml`
    cli->>cli: Run update job
    cli->>cli: Write `output.yaml`
    cli-->>-ext: Update completed
    ext->>ext: Read and parse `output.yaml`
    Note right of ext: Normal update logic resumes from this point.<br/>Outputs are parsed, pull requests are created/updated/closed based on the outputs

Loading

Example update scenario

Example of security advisories being fetched from GHSA:

...snip...
##[endgroup]
##[section]GHSA dependency vulnerability check
Detected 1 dependencies; Checking for vulnerabilities...
Detected 2 vulnerabilities affecting 1 dependencies [ 'System.Text.Json' ]
##[group]Job 'update-0-nuget-security-only'
...snip...

Example of security advisories being passed in to job.yml:

  "job": {
    "id": "update-0-nuget-security-only",
    "package-manager": "nuget",
    "update-subdependencies": true,
    "updating-a-pull-request": false,
    "dependencies": [
      "System.Text.Json"
    ],
    "allowed-updates": [
      {
        "dependency-type": "all"
      }
    ],
    "security-updates-only": true,
    "security-advisories": [
      {
        "dependency-name": "System.Text.Json",
        "affected-versions": [
          ">= 8.0.0, <= 8.0.4"
        ],
        "patched-versions": [
          "8.0.5"
        ],
        "unaffected-versions": []
      },
      {
        "dependency-name": "System.Text.Json",
        "affected-versions": [
          ">= 7.0.0, < 8.0.4"
        ],
        "patched-versions": [
          "8.0.4"
        ],
        "unaffected-versions": []
      }
    ],
    "source": {
      ...snip...
      ]
    },
    "existing-pull-requests": [],
    "existing-group-pull-requests": [],
    "experiments": {
      "nuget_native_analysis": "true",
      "nuget_dependency_solver": "true"
    },
    "reject-external-code": false,
    "lockfile-only": false,
    "debug": true
  },

Example of Dependabot analysing next safe version:

updater | 2024/12/02 11:41:27 INFO <job_update_0_nuget_security_only> Starting security update job for rhyskoedijk/DevOps%20Extensions/_git/Dependabot-Tests
updater | 2024/12/02 11:41:27 INFO <job_update_0_nuget_security_only> Checking if System.Text.Json 8.0.0 needs updating
updater | 2024/12/02 11:41:27 INFO <job_update_0_nuget_security_only> Writing dependency info: {"Name":"System.Text.Json","Version":"8.0.0","IsVulnerable":true,"IgnoredVersions":[],"Vulnerabilities":[{"DependencyName":"System.Text.Json","PackageManager":"nuget","VulnerableVersions":[">= 8.0.0, <= 8.0.4"],"SafeVersions":["= 8.0.5"]},{"DependencyName":"System.Text.Json","PackageManager":"nuget","VulnerableVersions":[">= 7.0.0, < 8.0.4"],"SafeVersions":["= 8.0.4"]}]}
updater | running NuGet analyze:
updater | /opt/nuget/NuGetUpdater/NuGetUpdater.Cli analyze --repo-root /home/dependabot/dependabot-updater/repo --discovery-file-path /tmp/.dependabot/discovery.1.json --dependency-file-path /tmp/.dependabot/dependency/System.Text.Json.json --analysis-folder-path /tmp/.dependabot/analysis
updater | Starting analysis of System.Text.Json...
updater |   Determining multi-dependency property.
updater |   Finding updated version.
updater |   Finding updated peer dependencies.
updater | Restored /tmp/package-dependency-resolution_kv8HTW/Project.csproj (in 392 ms).
updater | NuGetData::Package=System.Text.Json, Version=8.0.5
updater | 
updater | Build succeeded.
updater | 0 Warning(s)
updater | 0 Error(s)
updater | 
updater | Time Elapsed 00:00:01.23
updater | 
updater | 
updater | Analysis complete.
updater | 2024/12/02 11:41:30 INFO <job_update_0_nuget_security_only> Latest version is 8.0.5
updater | 2024/12/02 11:41:30 INFO <job_update_0_nuget_security_only> Requirements to unlock own
updater | 2024/12/02 11:41:30 INFO <job_update_0_nuget_security_only> Requirements update strategy 
updater | 2024/12/02 11:41:30 INFO <job_update_0_nuget_security_only> Updating System.Text.Json from 8.0.0 to 8.0.5

Example of PR creation:

[rhyskoedijk]

@mburumaxwell this is ready now if you don't have any objections. It's not perfect due to the workarounds I've had to do (mentioned above), but it at least works.