Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for security-only updates using Dependabot CLI #1394

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
af35bd9
Add more logging
rhyskoedijk Sep 30, 2024
868b20e
Clean up logging
rhyskoedijk Oct 1, 2024
11ae4f8
Merge commit '93eac02825c852ad6c8d4bcd73cf82f2fb0b4ff2' into feature/…
rhyskoedijk Oct 1, 2024
840d5cd
Use Rest API instead of TypedClient, fixes undefined responses in som…
rhyskoedijk Oct 1, 2024
20bdf96
Merge commit '2358a14dfd4c0fe96c8ea18188870b68ab75cc61' into feature/…
rhyskoedijk Oct 4, 2024
587aea9
Refactor DevOps client to use rest client instead of typed client
rhyskoedijk Oct 6, 2024
9561273
Clean up
rhyskoedijk Oct 6, 2024
8012d22
Revert "Clean up"
rhyskoedijk Oct 6, 2024
37917b5
Revert
rhyskoedijk Oct 6, 2024
e9c950b
Remove double forward slash in DevOps API urls
rhyskoedijk Oct 6, 2024
d037409
Add more detailed error info when REST APIs fail
rhyskoedijk Oct 6, 2024
302e36a
Fix for identity id being mapped as an array instead of a string
rhyskoedijk Oct 6, 2024
2758618
Add documentation for configuring assigness and reviewers
rhyskoedijk Oct 6, 2024
a3e87dd
Fix typos
rhyskoedijk Oct 6, 2024
00248ce
Alerts cannot be embeeded inside <details>
rhyskoedijk Oct 6, 2024
d86e16b
Clean up
rhyskoedijk Oct 6, 2024
0d51fb7
Tidy up
rhyskoedijk Oct 7, 2024
29dab82
Tidy up
rhyskoedijk Oct 7, 2024
0ea23db
Fix undefined reference when reading branch name seprarator
rhyskoedijk Oct 7, 2024
fc8fb88
Add support for security-only updates
rhyskoedijk Oct 7, 2024
229c3f1
Update warning messages
rhyskoedijk Oct 8, 2024
fe1ac87
Fix formatting
rhyskoedijk Oct 12, 2024
951c538
Update documentation
rhyskoedijk Oct 12, 2024
58b6169
Merge commit '905ae4d11d6d2ec7002954a91f90af14647747fb' into feature/…
rhyskoedijk Oct 14, 2024
daa1ce5
Fix merge issues
rhyskoedijk Oct 14, 2024
3186ba2
Fix merge issues
rhyskoedijk Oct 14, 2024
3186177
Fix merge issues
rhyskoedijk Oct 14, 2024
08f1c38
Fix merge issues
rhyskoedijk Oct 14, 2024
2dbd735
Merge branch 'main' into feature/security-only-updates
rhyskoedijk Oct 22, 2024
e3987e9
Make the security-only update warning more concise
rhyskoedijk Oct 22, 2024
5108b0d
Filter out security advisories which do not affected the discovered d…
rhyskoedijk Oct 22, 2024
6609b2b
Update documentation
rhyskoedijk Oct 22, 2024
0934cfe
Do not run security-only update when there are no vulnerable dependen…
rhyskoedijk Oct 22, 2024
21f480a
Merge commit 'a55a34798922fdbd6fd9dab3ac4c4fe08782f2ac' into feature/…
rhyskoedijk Oct 22, 2024
1a03790
Fix merge issues
rhyskoedijk Oct 22, 2024
04cdb09
Merge remote-tracking branch 'origin/main' into feature/security-only…
rhyskoedijk Oct 28, 2024
5d738e5
Merge branch 'main' into feature/security-only-updates
rhyskoedijk Nov 18, 2024
6e9cb1b
Merge branch 'main' into feature/security-only-updates
rhyskoedijk Dec 2, 2024
6befe6b
Clean up
rhyskoedijk Dec 2, 2024
e170aca
Improved GHSA security advisory parsing
rhyskoedijk Dec 2, 2024
293684c
Clean up logging
rhyskoedijk Dec 2, 2024
9fd04ab
Fix for security-only PRs being closed on next update
rhyskoedijk Dec 2, 2024
b2e105a
Tidy up logging commands
rhyskoedijk Dec 3, 2024
c516daa
Tidy up logging commands
rhyskoedijk Dec 3, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 8 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,11 +23,11 @@ In this repository you'll find:
- [Configuring experiments](#configuring-experiments)
- [Configuring assignees and reviewers](#configuring-assignees-and-reviewers)
- [Unsupported features and configurations](#unsupported-features-and-configurations)
* [Extension Task](#extension-task)
* [Dependabot Task](#dependabot-task)
+ [dependabot@V2](#dependabotv2)
+ [dependabot@V1](#dependabotv1)
* [Updater Docker image](#updater-docker-image)
* [Server](#server)
* [Dependabot Updater Docker image](#dependabot-updater-docker-image)
* [Dependabot Server](#dependabot-server)
- [Migration Guide](#migration-guide)
- [Contributing](#contributing)
* [Reporting issues and feature requests](#reporting-issues-and-feature-requests)
Expand Down Expand Up @@ -194,13 +194,13 @@ Reviewers can be any of the following values:
- Team name

## Unsupported features and configurations
We aim to support all [official configuration options](https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file), but there are some limitations for:
We aim to support all [official configuration options](https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file), but there are some limitations:

### Extension Task
### Dependabot Task

#### `dependabot@V2`
- [`schedule`](https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file#scheduleinterval) is ignored, use [pipeline scheduled triggers](https://learn.microsoft.com/en-us/azure/devops/pipelines/process/scheduled-triggers?view=azure-devops&tabs=yaml#scheduled-triggers) instead.
- [Security-only updates](https://docs.github.com/en/code-security/dependabot/dependabot-security-updates/configuring-dependabot-security-updates#overriding-the-default-behavior-with-a-configuration-file) (`open-pull-requests-limit: 0`) are not supported. _(coming soon)_
- [`securityAdvisoriesFile`](#configuring-security-advisories-and-known-vulnerabilities) task input is not yet supported.

#### `dependabot@V1`
- [`schedule`](https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file#scheduleinterval) is ignored, use [pipeline scheduled triggers](https://learn.microsoft.com/en-us/azure/devops/pipelines/process/scheduled-triggers?view=azure-devops&tabs=yaml#scheduled-triggers) instead.
Expand All @@ -210,11 +210,11 @@ We aim to support all [official configuration options](https://docs.github.com/e
- [`assignees`](https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file#assignees) and [`reviewers`](https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file#reviewers) must be a list of user guids or email addresses; group/team names are not supported.
- Private feed/registry authentication may not work with all package ecyosystems. Support is _slightly_ improved when task input `useUpdateScriptVNext: true` is set, but not still not fully supported. See [problems with authentication](https://github.com/tinglesoftware/dependabot-azure-devops/discussions/1317) for more.

### Updater Docker image
### Dependabot Updater Docker Image
- `DEPENDABOT_ASSIGNEES` and `DEPENDABOT_REVIEWERS` must be a list of user guids; email addresses and group/team names are not supported.
- Private feed/registry authentication may not work with all package ecyosystems. See [problems with authentication](https://github.com/tinglesoftware/dependabot-azure-devops/discussions/1317) for more.

### Server
### Dependabot Server

- [`directories`](https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file#directories) are not supported.
- [`groups`](https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file#groups) are not supported.
Expand Down
23 changes: 10 additions & 13 deletions docs/migrations/v1-to-v2.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,15 +20,16 @@ The task now uses [Dependabot CLI](https://github.com/dependabot/cli) to perform
> [!WARNING]
> **It is strongly recommended that you complete (or abandon) all active Depedabot pull requests created in V1 before migrating to V2.** Due to changes in Dependabot dependency metadata, V2 pull requests are not compatible with V1 (and vice versa). Migrating to V2 before completing existing pull requests will lead to duplication of pull requests.

### Security-only updates
Security-only updates (i.e. `open-pull-requests-limit: 0`) incur a slight performance overhead due to limitations in Dependabot CLI, detailed in [dependabot/cli#360](https://github.com/dependabot/cli/issues/360). To work around this, vulnerable dependencies will first be discovered using an "ignore everything" update job; After which, security advisories for the discovered dependencies will be checked against the [GitHub Advisory Database](https://github.com/advisories) before finally performing the requested security-only update job.

Currently the [`securityAdvisoriesFile`](../../README.md#configuring-security-advisories-and-known-vulnerabilities) task input is not supported, but is expected to be supported in the near future.

### New pipeline agent requirements; "Go" must be installed
Dependabot CLI requires [Go](https://go.dev/doc/install) (1.22+) and [Docker](https://docs.docker.com/engine/install/) (with Linux containers).
If you use [Microsoft-hosted agents](https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/hosted?view=azure-devops&tabs=yaml#software), we recommend using the [ubuntu-latest](https://github.com/actions/runner-images/blob/main/images/ubuntu/Ubuntu2404-Readme.md) image, which meets all task requirements.
For self-hosted agents, you will need to install Go 1.22+.

### Security-only updates and "fixed vulnerabilities" are not implemented (yet)
Using configuration `open-pull-requests-limit: 0` will cause a "not implemented" error. This is [current limitation of V2](../../README.md#unsupported-features-and-configurations). A solution is still under development and is expected to be resolved before general availability.
See: https://github.com/dependabot/cli/issues/360 for more technical details.

### Task Input `updaterOptions` has been renamed to `experiments`
Renamed to match Dependabot Core/CLI terminology. The input value remains unchanged. See [configuring experiments](../../README.md#configuring-experiments) for more details.

Expand Down Expand Up @@ -70,13 +71,9 @@ The following environment variables have been removed entirely; the feature is n

## Todo before general availability
Before removing the preview flag from V2 `task.json`, we need to:
- [x] Open an issue in Dependabot-CLI, enquire how security-advisories are expected to be provided **before** knowing the list of dependencies. (https://github.com/dependabot/cli/issues/360)
- [ ] Convert GitHub security advisory client in `vulnerabilities.rb` to TypeScript code
- [ ] Implement `security-advisories` config once the answer the above is known
- [x] Review `task.json`, add documentation for new V2 inputs
- [x] Update `\docs\extension.md` with V2 docs
- [x] Update `\extension\README.MD` with V2 docs
- [x] Update `\README.MD` with V2 docs
- [ ] Do a general code tidy-up pass (check all "TODO" comments)
- [ ] Add "superseded by X" close reason when PR is closed during a PR update
- [ ] Add documentation for required permissions and PAT scopes
- [ ] Add support for 'securityAdvisoriesFile' task input
- [ ] Add unit tests for V2 utils scripts
- [ ] Investigate https://zod.dev/
- [ ] General code tidy-up (check all "TODO" comments)
- [ ] Investigate https://zod.dev/
2 changes: 1 addition & 1 deletion extension/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ Dependabot uses Docker containers, which may take time to install if not already
|azureDevOpsAccessToken|**_Optional_**. The Personal Access Token for accessing Azure DevOps. Supply a value here to avoid using permissions for the Build Service either because you cannot change its permissions or because you prefer that the Pull Requests be done by a different user. When not provided, the current authentication scope is used. In either case, be use the following permissions are granted: <br/>-&nbsp;Code (Full)<br/>-&nbsp;Pull Requests Threads (Read & Write).<br/>See the [documentation](https://docs.microsoft.com/en-us/azure/devops/organizations/accounts/use-personal-access-tokens-to-authenticate?view=azure-devops&tabs=preview-page#create-a-pat) to know more about creating a Personal Access Token.<br/>Use this in place of `azureDevOpsServiceConnection` such as when it is not possible to create a service connection.|
|gitHubConnection|**_Optional_**. The GitHub service connection for authenticating requests against GitHub repositories. This is useful to avoid rate limiting errors. The token must include permissions to read public repositories. See the [GitHub docs](https://docs.github.com/en/free-pro-team@latest/github/authenticating-to-github/creating-a-personal-access-token) for more on Personal Access Tokens and [Azure DevOps docs](https://docs.microsoft.com/en-us/azure/devops/pipelines/library/service-endpoints?view=azure-devops&tabs=yaml#sep-github) for the GitHub service connection.|
|gitHubAccessToken|**_Optional_**. The raw GitHub PAT for authenticating requests against GitHub repositories. Use this in place of `gitHubConnection` such as when it is not possible to create a service connection.|
|storeDependencyList|**_Optional_**. Determines if the last know dependency list information should be stored in the parent DevOps project properties. If enabled, the authenticated user must have the "Project & Team (Write)" permission for the project. Enabling this option improves performance when doing security-only updates. Defaults to `false`.|
|storeDependencyList|**_Optional_**. Determines if the last know dependency list information should be stored in the parent DevOps project properties. If enabled, the authenticated user must have the "Project & Team (Write)" permission for the project. Defaults to `false`.|
|targetRepositoryName|**_Optional_**. The name of the repository to target for processing. If this value is not supplied then the Build Repository Name is used. Supplying this value allows creation of a single pipeline that runs Dependabot against multiple repositories by running a `dependabot` task for each repository to update.|
|targetUpdateIds|**_Optional_**. A semicolon (`;`) delimited list of update identifiers run. Index are zero-based and in the order written in the configuration file. When not present, all the updates are run. This is meant to be used in scenarios where you want to run updates a different times from the same configuration file given you cannot schedule them independently in the pipeline.|
|experiments|**_Optional_**. Comma separated list of Dependabot experiments; available options depend on the ecosystem. Example: `tidy=true,vendor=true,goprivate=*`. See: [Configuring experiments](https://github.com/tinglesoftware/dependabot-azure-devops/#configuring-experiments)|
Expand Down
78 changes: 24 additions & 54 deletions extension/package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 3 additions & 1 deletion extension/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -31,13 +31,15 @@
"axios": "1.7.8",
"azure-devops-node-api": "14.1.0",
"azure-pipelines-task-lib": "4.17.3",
"js-yaml": "4.1.0"
"js-yaml": "4.1.0",
"semver": "7.6.3"
},
"devDependencies": {
"@types/jest": "29.5.14",
"@types/js-yaml": "4.0.9",
"@types/node": "22.10.0",
"@types/q": "1.5.8",
"@types/semver": "7.5.8",
"jest": "29.7.0",
"ts-jest": "29.2.5",
"ts-node": "10.9.2",
Expand Down
Loading
Loading