Support for security-only updates using Dependabot CLI (#1394)

* Add more logging * Clean up logging * Use Rest API instead of TypedClient, fixes undefined responses in some environments * Refactor DevOps client to use rest client instead of typed client * Clean up * Revert "Clean up" This reverts commit 9561273. * Revert * Remove double forward slash in DevOps API urls * Add more detailed error info when REST APIs fail * Fix for identity id being mapped as an array instead of a string * Add documentation for configuring assigness and reviewers * Fix typos * Alerts cannot be embeeded inside <details> * Clean up * Tidy up * Tidy up * Fix undefined reference when reading branch name seprarator * Add support for security-only updates * Update warning messages * Fix formatting * Update documentation * Fix merge issues * Fix merge issues * Fix merge issues * Fix merge issues * Make the security-only update warning more concise * Filter out security advisories which do not affected the discovered dependencies during a security-only update * Update documentation * Do not run security-only update when there are no vulnerable dependencies * Fix merge issues * Clean up * Improved GHSA security advisory parsing * Clean up logging * Fix for security-only PRs being closed on next update * Tidy up logging commands * Tidy up logging commands
tinglesoftware · Dec 9, 2024 · 814010c · 814010c
1 parent a0d7902
commit 814010c
Show file tree

Hide file tree

Showing 14 changed files with 518 additions and 150 deletions.
diff --git a/README.md b/README.md
@@ -23,11 +23,11 @@ In this repository you'll find:
 - [Configuring experiments](#configuring-experiments)
 - [Configuring assignees and reviewers](#configuring-assignees-and-reviewers)
 - [Unsupported features and configurations](#unsupported-features-and-configurations)
-   * [Extension Task](#extension-task)
+   * [Dependabot Task](#dependabot-task)
       + [dependabot@V2](#dependabotv2)
       + [dependabot@V1](#dependabotv1)
-   * [Updater Docker image](#updater-docker-image)
-   * [Server](#server)
+   * [Dependabot Updater Docker image](#dependabot-updater-docker-image)
+   * [Dependabot Server](#dependabot-server)
 - [Migration Guide](#migration-guide)
 - [Contributing](#contributing)
    * [Reporting issues and feature requests](#reporting-issues-and-feature-requests)
@@ -194,13 +194,13 @@ Reviewers can be any of the following values:
 - Team name
 
 ## Unsupported features and configurations
-We aim to support all [official configuration options](https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file), but there are some limitations for:
+We aim to support all [official configuration options](https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file), but there are some limitations:
 
-### Extension Task
+### Dependabot Task
 
 #### `dependabot@V2`
 - [`schedule`](https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file#scheduleinterval) is ignored, use [pipeline scheduled triggers](https://learn.microsoft.com/en-us/azure/devops/pipelines/process/scheduled-triggers?view=azure-devops&tabs=yaml#scheduled-triggers) instead.
-- [Security-only updates](https://docs.github.com/en/code-security/dependabot/dependabot-security-updates/configuring-dependabot-security-updates#overriding-the-default-behavior-with-a-configuration-file) (`open-pull-requests-limit: 0`) are not supported. _(coming soon)_
+- [`securityAdvisoriesFile`](#configuring-security-advisories-and-known-vulnerabilities) task input is not yet supported.
 
 #### `dependabot@V1`
 - [`schedule`](https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file#scheduleinterval) is ignored, use [pipeline scheduled triggers](https://learn.microsoft.com/en-us/azure/devops/pipelines/process/scheduled-triggers?view=azure-devops&tabs=yaml#scheduled-triggers) instead.
@@ -210,11 +210,11 @@ We aim to support all [official configuration options](https://docs.github.com/e
 - [`assignees`](https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file#assignees) and [`reviewers`](https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file#reviewers) must be a list of user guids or email addresses; group/team names are not supported.
 - Private feed/registry authentication may not work with all package ecyosystems. Support is _slightly_ improved when task input `useUpdateScriptVNext: true` is set, but not still not fully supported. See [problems with authentication](https://github.com/tinglesoftware/dependabot-azure-devops/discussions/1317) for more.
 
-### Updater Docker image
+### Dependabot Updater Docker Image
 - `DEPENDABOT_ASSIGNEES` and `DEPENDABOT_REVIEWERS` must be a list of user guids; email addresses and group/team names are not supported.
 - Private feed/registry authentication may not work with all package ecyosystems. See [problems with authentication](https://github.com/tinglesoftware/dependabot-azure-devops/discussions/1317) for more.
 
-### Server
+### Dependabot Server
 
 - [`directories`](https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file#directories) are not supported.
 - [`groups`](https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file#groups) are not supported.

diff --git a/docs/migrations/v1-to-v2.md b/docs/migrations/v1-to-v2.md
@@ -20,15 +20,16 @@ The task now uses [Dependabot CLI](https://github.com/dependabot/cli) to perform
 > [!WARNING]
 > **It is strongly recommended that you complete (or abandon) all active Depedabot pull requests created in V1 before migrating to V2.** Due to changes in Dependabot dependency metadata, V2 pull requests are not compatible with V1 (and vice versa). Migrating to V2 before completing existing pull requests will lead to duplication of pull requests.
 
+### Security-only updates
+Security-only updates (i.e. `open-pull-requests-limit: 0`) incur a slight performance overhead due to limitations in Dependabot CLI, detailed in [dependabot/cli#360](https://github.com/dependabot/cli/issues/360). To work around this, vulnerable dependencies will first be discovered using an "ignore everything" update job; After which, security advisories for the discovered dependencies will be checked against the [GitHub Advisory Database](https://github.com/advisories) before finally performing the requested security-only update job.
+
+Currently the [`securityAdvisoriesFile`](../../README.md#configuring-security-advisories-and-known-vulnerabilities) task input is not supported, but is expected to be supported in the near future.
+
 ### New pipeline agent requirements; "Go" must be installed
 Dependabot CLI requires [Go](https://go.dev/doc/install) (1.22+) and [Docker](https://docs.docker.com/engine/install/) (with Linux containers).
 If you use [Microsoft-hosted agents](https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/hosted?view=azure-devops&tabs=yaml#software), we recommend using the [ubuntu-latest](https://github.com/actions/runner-images/blob/main/images/ubuntu/Ubuntu2404-Readme.md) image, which meets all task requirements.
 For self-hosted agents, you will need to install Go 1.22+.
 
-### Security-only updates and "fixed vulnerabilities" are not implemented (yet)
-Using configuration `open-pull-requests-limit: 0` will cause a "not implemented" error. This is [current limitation of V2](../../README.md#unsupported-features-and-configurations). A solution is still under development and is expected to be resolved before general availability.
-See: https://github.com/dependabot/cli/issues/360 for more technical details.
-
 ### Task Input `updaterOptions` has been renamed to `experiments`
 Renamed to match Dependabot Core/CLI terminology. The input value remains unchanged. See [configuring experiments](../../README.md#configuring-experiments) for more details.
 
@@ -70,13 +71,9 @@ The following environment variables have been removed entirely; the feature is n
 
 ## Todo before general availability
 Before removing the preview flag from V2 `task.json`, we need to:
- - [x] Open an issue in Dependabot-CLI, enquire how security-advisories are expected to be provided **before** knowing the list of dependencies. (https://github.com/dependabot/cli/issues/360)
- - [ ] Convert GitHub security advisory client in `vulnerabilities.rb` to TypeScript code
- - [ ] Implement `security-advisories` config once the answer the above is known
- - [x] Review `task.json`, add documentation for new V2 inputs
- - [x] Update `\docs\extension.md` with V2 docs
- - [x] Update `\extension\README.MD` with V2 docs
- - [x] Update `\README.MD` with V2 docs
- - [ ] Do a general code tidy-up pass (check all "TODO" comments)
+ - [ ] Add "superseded by X" close reason when PR is closed during a PR update
+ - [ ] Add documentation for required permissions and PAT scopes
+ - [ ] Add support for 'securityAdvisoriesFile' task input
  - [ ] Add unit tests for V2 utils scripts
- - [ ] Investigate https://zod.dev/
+ - [ ] General code tidy-up (check all "TODO" comments)
+ - [ ] Investigate https://zod.dev/
diff --git a/extension/README.md b/extension/README.md
@@ -64,7 +64,7 @@ Dependabot uses Docker containers, which may take time to install if not already
 |azureDevOpsAccessToken|**_Optional_**. The Personal Access Token for accessing Azure DevOps. Supply a value here to avoid using permissions for the Build Service either because you cannot change its permissions or because you prefer that the Pull Requests be done by a different user. When not provided, the current authentication scope is used. In either case, be use the following permissions are granted: <br/>-&nbsp;Code (Full)<br/>-&nbsp;Pull Requests Threads (Read & Write).<br/>See the [documentation](https://docs.microsoft.com/en-us/azure/devops/organizations/accounts/use-personal-access-tokens-to-authenticate?view=azure-devops&tabs=preview-page#create-a-pat) to know more about creating a Personal Access Token.<br/>Use this in place of `azureDevOpsServiceConnection` such as when it is not possible to create a service connection.|
 |gitHubConnection|**_Optional_**. The GitHub service connection for authenticating requests against GitHub repositories. This is useful to avoid rate limiting errors. The token must include permissions to read public repositories. See the [GitHub docs](https://docs.github.com/en/free-pro-team@latest/github/authenticating-to-github/creating-a-personal-access-token) for more on Personal Access Tokens and [Azure DevOps docs](https://docs.microsoft.com/en-us/azure/devops/pipelines/library/service-endpoints?view=azure-devops&tabs=yaml#sep-github) for the GitHub service connection.|
 |gitHubAccessToken|**_Optional_**. The raw GitHub PAT for authenticating requests against GitHub repositories. Use this in place of `gitHubConnection` such as when it is not possible to create a service connection.|
-|storeDependencyList|**_Optional_**. Determines if the last know dependency list information should be stored in the parent DevOps project properties. If enabled, the authenticated user must have the "Project & Team (Write)" permission for the project. Enabling this option improves performance when doing security-only updates. Defaults to `false`.|
+|storeDependencyList|**_Optional_**. Determines if the last know dependency list information should be stored in the parent DevOps project properties. If enabled, the authenticated user must have the "Project & Team (Write)" permission for the project. Defaults to `false`.|
 |targetRepositoryName|**_Optional_**. The name of the repository to target for processing. If this value is not supplied then the Build Repository Name is used. Supplying this value allows creation of a single pipeline that runs Dependabot against multiple repositories by running a `dependabot` task for each repository to update.|
 |targetUpdateIds|**_Optional_**. A semicolon (`;`) delimited list of update identifiers run. Index are zero-based and in the order written in the configuration file. When not present, all the updates are run. This is meant to be used in scenarios where you want to run updates a different times from the same configuration file given you cannot schedule them independently in the pipeline.|
 |experiments|**_Optional_**. Comma separated list of Dependabot experiments; available options depend on the ecosystem. Example: `tidy=true,vendor=true,goprivate=*`. See: [Configuring experiments](https://github.com/tinglesoftware/dependabot-azure-devops/#configuring-experiments)|

diff --git a/extension/package-lock.json b/extension/package-lock.json
diff --git a/extension/package.json b/extension/package.json
@@ -31,13 +31,15 @@
     "axios": "1.7.8",
     "azure-devops-node-api": "14.1.0",
     "azure-pipelines-task-lib": "4.17.3",
-    "js-yaml": "4.1.0"
+    "js-yaml": "4.1.0",
+    "semver": "7.6.3"
   },
   "devDependencies": {
     "@types/jest": "29.5.14",
     "@types/js-yaml": "4.0.9",
     "@types/node": "22.10.0",
     "@types/q": "1.5.8",
+    "@types/semver": "7.5.8",
     "jest": "29.7.0",
     "ts-jest": "29.2.5",
     "ts-node": "10.9.2",