diff --git a/.changelog/README.md b/.changelog/README.md new file mode 100644 index 000000000..deeaa3f5c --- /dev/null +++ b/.changelog/README.md @@ -0,0 +1,224 @@ +# .changelog directory information + +This `.changelog/` directory is primarily used to coordinate unreleased changes and +ultimately to generate the release notes and changelog content for a release. + + + - [Overview](#overview) + - [Adding Changes](#adding-changes) + - [Dependencies](#dependencies) + - [Section Names](#section-names) + - [Unclog](#unclog) + - [Entry Files](#entry-files) + - [Releases](#releases) + - [Finding Changes](#finding-changes) + + + +## Overview + +As changes are made in this repo, entries should be added under the `.changelog/unreleased/` directory. +By storing entries in different files for different changes, we remove a common source of merge conflicts. + +We use [unclog](https://github.com/informalsystems/unclog) to help create and manage these changelog entries. +However, we don't use it to create the entire content of the `CHANGELOG.md` file, just new content. + +The `CHANGELOG.md` is usually only updated as part of the release process. + + + +## Adding Changes + +When a change is being made to this repo, at least one changelog entry should be created about it. +The easiest way to do this is using [unclog](https://github.com/informalsystems/unclog). + +Example for changes that do **not** have a related GitHub issue: + +```console +$ unclog add --pull-request 123 --section bug-fixes --id fix-the-thing --message 'Fix the thing that was broken' +``` + +Example for changes that **do** have a related GitHub issue. +```console +$ unclog add --issue-no 123 --section bug-fixes --id fix-the-thing --message 'Fix the thing that was broken' +``` + +Both of those example commands will create the file `.changelog/unreleased/bug-fixes/123-fix-the-thing.md` +with this content (respectively): + +```md +* Fix the thing that was broken [PR 123](https://github.com/provenance-io/provenance/pull/123). +``` + +or + +```md +* Fix the thing that was broken [#123](https://github.com/provenance-io/provenance/issues/123). +``` + +Each entry file can have as many bullet-points as needed, and bullet-points can span multiple lines, just like regular markdown. + +If the change you are making should have multiple entries in different sections, make an entry file in each applicable section. + +Entry files can also be created manually, but should conform to the following standards: + +* The file should be named `-.md` and placed in a valid `
` directory in `unreleased`. +* The first line of the file should have the format `* .`. +* The file should not have any blank lines. +* The file should use standard markdown. +* Imagine bullet-points both above and below this entry. It all should form a single unbroken unordered list. + + + +## Dependencies + +When making changes to `go.mod`, you should use the `.changelog/get-dep-changes.sh` script to create the dependencies changelog entry. + +```console +$ go mod tidy +$ .changelog/get-dep-changes.sh --pull-request 123 --id bump-the-thing +``` + +That will create the file `.changelog/unreleased/dependencies/123-bump-the-stuff` with content generated by analyzing the changes made to `go.mod`. + +Here too, if there is an issue number for the change, you should use the `--issue-no` (or `-n`) flag instead of `--pull-request` (or `-p) in order to have the correct links. + + + +## Section Names + +The valid `--section` values (for `unclog`) are defined in the comment at the top of the `CHANGELOG.md` file. +The quoted stanza strings are lower-cased and spaces changed to dashes to get the valid section names. +That list also defines the order that the sections will be in for each version. + +The `.changelog/get-valid-sections.sh` script will output the exact list of valid sections. + +```console +$ .changelog/get-valid-sections.sh +features +improvements +bug-fixes +deprecated +client-breaking +api-breaking +state-machine-breaking +dependencies +``` + +You can also get them using the `make get-valid-sections` target. + +All directories in `.changelog/unreleased/` must be one of those or else linting will fail. + + + +## Unclog + +We use [unclog](https://github.com/informalsystems/unclog) primarily to create new entry files that will later be collected into some release notes. +We don't use much else of that system though. E.g. our `CHANGELOG.md` file is **not** simply the output of `unclog build`. + +Some `unclog` commands require that the `EDITOR` environment variable has been set, e.g.: + +```console +$ export EDITOR="$( which vim )" +``` + +To create our release notes (in `.changelog/prep-release.sh`), we start with the output of `unclog build --unreleased` and then clean it up and reorder things. + +You might also want to use the `unclog build [--unreleased|--all]` command to view the content under `.changelog/`. +However, if a `summary.md` file is missing, or is just the default comment, `unclog` will open your editor for it. +If it does, it will end up writing one or more `summary.md` files. +So, if you do this, make sure you don't accidentally check in unwanted changes to the `summary.md` file(s). + +We don't use the `unclog release` command because that will _always_ try to open the editor to make a change to (or create) the `summary.md` as it's moved to the version directory. +But we need to have `.changelog/unreleased/summary.md` written before we're ready to move that content. + + + +## Entry Files + +The `unclog add` command will create files with the path `.changelog/unreleased/
/-.md`. +It's very important to make sure these files are correct in the PR that creates them. +If at all possible, the file should not be touched once the PR has been merged (until it is deleted as being part of a release). +It's easier for git to keep track of it (without complaining) when the content and filename remain unchanged. +Further, we usually backport PRs by using `git cherry-pick` on a squash-and-merge coming from a PR. +If the entry for that change is update by a later PR, there's a very good chance the release branch will not get the followup. + +Standard lifecycle of an entry file that is part of a release candidate: + +1. File is created and included in a PR targeting `main`. +2. The PR is merged into `main`. +3. The file is backported to the `.x` branch (or included in the initial creation of that branch). +4. In the `.x` branch, the file is moved to the rc version dir (from `unreleased/`) as part of the PR to mark the new release candidate version. +5. In `main`, the file is deleted as part of a PR to mark the new release candidate version there too. +6. In the `.x` branch, the file is moved to full version dir (from the rc version dir) as part of the PR to mark the new full version. + +If the entry isn't part of a release candidate, then in step 4 it gets moved to the full version directory, and step 6 doesn't happen. + + + +## Releases + +When preparing to mark a release, you should use the `.changelog/prep-release.sh` script. + +That script will: +1. Create or Update the `RELEASE_NOTES.md` file. +2. Add the new version to the `CHANGELOG.md` file. +3. Create a new version directory in the `.changelog/` folder with content from `unreleased/` and any rcs for this version. + +The results of the `.changelog/prep-release.sh` script are to be applied to the `release/vA.B.x` branch. +When marking the release in `main`, do not include the new version directory, but do delete the applicable entries from `.changelog/unreleased/`. +That is, the `.changelog/` directories should only ever exist on the `.x` branch. +This is primarily to reduce confusion if there is a discrepancy between the `CHANGELOG.md` content and an entry file's content. +It also helps keep things tidy and file counts lower. + +If you need to make tweaks or clarifications to the content, you should make the changes in the `RELEASE_NOTES.md` file first, then copy/paste those into the `CHANGELOG.md` file. +You should NOT update the changelog entry files, though (other than moving them). + +And to reiterate, the `RELEASE_NOTES.md` file and `.changelog/` directories should never exist on `main`, only in the `.x` branch. + +If you can't, or don't want to use the `.changelog/prep-release.sh` script, here's how to do things manually. + +To manually create the new `RELEASE_NOTES.md` file: + +1. If this is a full version and there were release candidates, move all the rc content into unreleased. +2. Run `unclog build --unreleased` to get the preliminary content. +3. Change the version header into a link and include the release date. +4. Change the section header casing to title case (from all upper-case). +5. Reorder the sections to match how they're listed in the top `CHANGELOG.md` comment. +6. Reorder the `Dependencies` section so that entries for the same library are next to each other. +7. Update all the link text to be `[PR ]` for prs, and `[#]` for issues determined by the link path. The `` is extracted from the link too to ensure that the text matches the target. +8. If a link has a period both before (ignoring whitespace) and after it, remove the one before it. +9. Add the "Full Commit Diff" link(s). +10. If this is a release candidate of 2 or more, add the new release notes to the top of the existing one, and put a divider `---` between them. That way all rcs for the given version are in the release notes. +11. If this is a full release, or a release candidate of 1, overwrite any existing release notes with the new content. + +To manually update the `CHANGELOG.md` file: + +1. Generate the new release notes. +2. Copy the new content for only this release from the release notes. I.e. if this is rc2, the release notes will also have rc1, we only want rc2 copied here since rc1 should already be listed in the `CHANGELOG.md` file. +3. In `CHANGELOG.md`, Add a new divider directly above the first one (between the `## Unreleased` section, and the first version section). +4. Past the new content between those two dividers, with an empty line before and after each divider. +5. Remove the blurb between the `##` version header, and the first `###` section header. + +To manually update the `.changelog/` entries: + +1. Move the `.changelog/unreleased` directory to `.changelog/`, e.g. `mv .changelog/unreleased .changelog/v1.13.0`. +2. Delete the old `.gitkeep` file, e.g. `rm .changelog/v1.13.0/.gitkeep`. +2. Create a new `.changelog/unreleased` directory, e.g. `mkdir .changelog/unreleased`. +3. Create the new `.gitkeep` file, e.g. `touch .changelog/unreleased/.gitkeep`. + + + +## Finding Changes + +The content in `CHANGELOG.md` should be trusted over the content in the `.changelog/` directory. +The `CHANGELOG.md` file will contain all the info on all releases (either full or release candidates). +On `main`, the `.changelog/` directory will only contain information about unreleased changes. +On a `.x` branch, the `.changelog/` folder will contain information on all releases for this minor version as well as stuff that is ready to be included in the next patch release. + +As part of the release process, it will be common to update the `CHANGELOG.md` file using the release prep script, but then make tweaks or clarifications to it. +Those tweaks and clarifications probably won't be reflected in the entry files though. +That is why the `CHANGELOG.md` file is the source of truth. + +One way to find information about unreleased changes is to use `unclog build --unreleased`. +That will combine all info in `.changelog/unreleased/` and output an unofficial release-notes to stdout with the unreleased changes. diff --git a/.changelog/change-template.md b/.changelog/change-template.md new file mode 100644 index 000000000..52b2bab75 --- /dev/null +++ b/.changelog/change-template.md @@ -0,0 +1 @@ +{{{ bullet }}} {{{ message }}} [#{{ change_id }}]({{{ change_url }}}). diff --git a/.changelog/config.toml b/.changelog/config.toml new file mode 100644 index 000000000..457e13c71 --- /dev/null +++ b/.changelog/config.toml @@ -0,0 +1,97 @@ +# The GitHub URL for your project. +# +# This is mainly necessary if you need to automatically generate changelog +# entries directly from the CLI. Right now we only support GitHub, but if +# anyone wants GitLab support please let us know and we'll try implement it +# too. +project_url = "https://github.com/provenance-io/provenance" + +# The file to use as a Handlebars template for changes added directly through +# the CLI. +# +# Assumes that relative paths are relative to the `.changelog` folder. If this +# file does not exist, a default template will be used. +change_template = "change-template.md" + +# The number of characters at which to wrap entries automatically added from +# the CLI. +# Provenance Blockchain notes: +# The length of the issue/pr link is included in this length. +# For the most part, we want the link on the same line as the text, though. +# A 4 digit PR or issue link both end up with 64 characters in it. +# It's nice to keep things to 120, but I don't like forcing that. +# So at 200, there's 120 for the main content, 64 for the link, +# and 16 extra chars for wiggle-room (before the wrap). +wrap = 200 + +# The heading right at the beginning of the changelog. +heading = "# CHANGELOG" + +# What style of bullet to use for the instances where unclog has to generate +# bullets for you. Can be "-" or "*". +bullet_style = "*" + +# The message to output when your changelog has no entries yet. +empty_msg = "Nothing to see here! Add some entries to get started." + +# The name of the file (relative to the `.changelog` directory) to use as an +# epilogue for your changelog (will be appended as-is to the end of your +# generated changelog). +epilogue_filename = "epilogue.md" + +# Sort releases by the given property/properties. Possible values include: +# - `version` : Sort releases by semantic version. +# - `date` : Sort releases by release date. +# +# This is an array, as one could potentially first sort by date and then version +# in cases where multiple releases were cut on the same date. +# +# Release dates are currently parsed from release summaries, and are expected to +# be located on the first line of the release summary. +sort_releases_by = ["version"] + +# Release date formats to expect in the release summary, in order of precedence. +# +# See https://docs.rs/chrono/latest/chrono/format/strftime/index.html for +# possible format specifiers. +release_date_formats = [ + # "*December 1, 2023* + "*%B %d, %Y*", + # "*Dec 1, 2023* + "*%b %d, %Y*", + # "2023-12-01" (ISO format) + "%F", +] + + +# Settings relating to unreleased changelog entries. +[unreleased] + +# The name of the folder containing unreleased entries, relative to the +# `.changelog` folder. +folder = "unreleased" + +# The heading to use for the unreleased entries section. +heading = "## Unreleased" + + +# Settings relating to sets (groups) of changes in the changelog. For example, a +# particular version of the software (e.g. "v1.0.0") is typically a change set. +[change_sets] + +# The filename containing a summary of the intended changes. Relative to the +# change set folder (e.g. `.changelog/unreleased/breaking-changes/summary.md`). +summary_filename = "summary.md" + +# The extension of files in a change set. +entry_ext = "md" + + +# Settings relating to all sections within a change set. For example, the +# "BREAKING CHANGES" section for a particular release is a change set section. +[change_set_sections] + +# Sort entries by a particular property. Possible values include: +# - `id` : The issue/PR number (the default value). +# - `entry-text` : The entry text itself. +sort_entries_by = "id" diff --git a/.changelog/dependabot-changelog.sh b/.changelog/dependabot-changelog.sh new file mode 100755 index 000000000..cbfb480c5 --- /dev/null +++ b/.changelog/dependabot-changelog.sh @@ -0,0 +1,219 @@ +#!/bin/bash +# This script will create the changelog entry for a dependabot PR. +# It's designed to be called by a github action kicked off because of a dependabot PR. + +show_usage () { + cat << EOF +dependabot-changelog.sh will create the changelog entries for a dependabot PR. + +Usage: ./dependabot-changelog.sh --pr --title --head-branch <branch> --target-branch <branch> + +--pr <num> + Identifies the PR number for use in the links as well as the changelog filename. +--title <title> + Identifies the title of the PR. It is used when there are no changes to go.mod. + Expected format: "Bump <library> to <new version> from <old version>" +--head-branch <branch> + Identifies the name of the branch with the change that we want merged into the target branch. + For dependabot changes, it will have the format "dependabot/<type>/<library>-<new version>". + The filename containing the new entries is derived from this. +--target-branch <branch> + Identifies the branch that this PR is going into. It will almost always be "main". + +EOF + +} + +if [[ "$#" -eq '0' ]]; then + show_usage + exit 0 +fi + +while [[ "$#" -gt '0' ]]; do + case "$1" in + --help) + show_usage + exit 0 + ;; + -p|--pull-request|--pr) + if [[ -z "$2" ]]; then + printf 'No argument provided after %s\n' "$1" + exit 1 + fi + if [[ "$2" =~ [^[:digit:]] ]]; then + printf 'Invalid %s value: [%s]. Only digits are allowed.\n' "$1" "$2" + exit 1 + fi + pr="$2" + shift + ;; + -t|--title) + if [[ -z "$2" ]]; then + printf 'No argument provided after %s\n' "$1" + exit 1 + fi + title="$2" + shift + ;; + --head-branch) + if [[ -z "$2" ]]; then + printf 'No argument provided after %s\n' "$1" + exit 1 + fi + head_branch="$2" + shift + ;; + --target-branch) + if [[ -z "$2" ]]; then + printf 'No argument provided after %s\n' "$1" + exit 1 + fi + target_branch="$2" + shift + ;; + -v|--verbose) + verbose="$1" + ;; + *) + printf 'Unknown argument: [%s].\n' "$1" + exit 1 + ;; + esac + shift +done + +if [[ -z "$head_branch" ]]; then + printf 'No --head-branch <branch> provided.\n' + stop_early='YES' +fi + +if [[ -z "$target_branch" ]]; then + printf 'No --target-branch <branch> provided.\n' + stop_early='YES' +fi + + +if [[ -z "$pr" ]]; then + printf 'No --pr <num> provided.\n' + stop_early='YES' +fi + +if [[ -z "$title" ]]; then + printf 'No --title <title> provided.\n' + stop_early='YES' +fi + +[[ -n "$stop_early" ]] && exit 1 + +if [[ -n "$verbose" ]]; then + printf ' Head Branch: "%s"\n' "$head_branch" + printf ' Target Branch: "%s"\n' "$target_branch" + printf ' PR: "%s"\n' "$pr" + printf ' Title: "%s"\n' "$title" +fi + +# Dependabot branch names look like this: "dependabot/github_actions/bufbuild/buf-setup-action-1.34.0" +# The "github_actions" part can also be "go_modules" (and probably other things too). +# For the filename, we'll omit the "dependabot/<lib type>/" part and use just what's left. +branch_fn="$( sed -E 's|^[^/]+/[^/]+/||; s|/|-|g;' <<< "$head_branch" )" +[[ -n "$verbose" ]] && printf 'Branch Filename: "%s"\n' "$branch_fn" + +# This script requires another script that must be in the same directory. +# To consistently find them, we'll need to know the absolute path to the dir with this script. +where_i_am="$( cd "$( dirname "${BASH_SOURCE:-$0}" )"; pwd -P )" +[[ -n "$verbose" ]] && printf ' Where I Am: "%s"\n' "$where_i_am" + +[[ -n "$verbose" ]] && printf 'Looking for go.mod dependency changes.\n' +# Run the script to create the entry from the changes in go.mod. +# The $verbose variable is purposely not quoted so that it doesn't count as an arg if it's empty. +"$where_i_am/get-dep-changes.sh" --pr "$pr" --name "$branch_fn" $verbose --force --target-branch "$target_branch" +ec=$? +[[ -n "$verbose" ]] && printf 'Exit code from get-dep-changes.sh: %d\n' "$ec" + +# That script exits with 0 when there are go.mod changes and the new file was created. +# If there were go.mod changes, we're all done here. +# I don't think I've ever seen a dependabot PR that bumps both a go module and something else. +if [[ "$ec" -eq '0' ]]; then + [[ -n "$verbose" ]] && printf 'Changes identified through go.mod. Done.\n' + exit 0 +fi + +# That script exits with 10 to indicate there were no go.mod changes. +# All other (non-zero) exit codes are an error that requires attention. +if [[ "$ec" -ne '10' ]]; then + printf 'An error was encountered.\n' + exit "$ec" +fi + +[[ -n "$verbose" ]] && printf 'Creating changelog entry from PR title.\n' + +# Okay. There weren't any go.mod changes. It's a bump to something else (e.g. a +# github action helper). Create an entry ourselves, based on the title, which +# should look something like this: "Bump <library> from <old version> to <new version>". +# First, though, standardize the spacing so the rest of the regex stuff is cleaner. +title="$( sed -E 's/[[:space:]]+/ /; s/^ //; s/ $//;' <<< "$title" )" +[[ -n "$verbose" ]] && printf 'Clean Title: "%s"\n' "$title" +if ! grep -Eqi '^Bump [^ ]+ from [^ ]+ to [^ ]+$' <<< "$title"; then + printf 'Unknown title format: %s\n' "$title" + exit 1 +fi + +lib="$( sed -E 's/^Bump //; s/ from.*$//;' <<< "$title" )" +[[ -n "$verbose" ]] && printf ' Library: "%s"\n' "$lib" +if [[ -z "$lib" || "$lib" == "$title" || "$lib" =~ ' ' ]]; then + printf 'Could not extract library from title: %s\n' "$title" + exit 1 +fi + +old_ver="$( sed -E 's/^.*from //; s/ to.*$//' <<< "$title" )" +[[ -n "$verbose" ]] && printf ' Old Ver: "%s"\n' "$old_ver" +if [[ -z "$old_ver" || "$old_ver" == "$title" || "$old_ver" =~ ' ' ]]; then + printf 'Could not extract old version from title: %s\n' "$title" + exit 1 +fi + +new_ver="$( sed -E 's/^.*to //' <<< "$title" )" +[[ -n "$verbose" ]] && printf ' New Ver: "%s"\n' "$new_ver" +if [[ -z "$new_ver" || "$new_ver" == "$title" || "$new_ver" =~ ' ' ]]; then + printf 'Could not extract new version from title: %s\n' "$title" + exit 1 +fi + +link="[PR ${pr}](https://github.com/provenance-io/provenance/pull/${pr})" +[[ -n "$verbose" ]] && printf ' Link: "%s"\n' "$link" + +repo_root="$( git rev-parse --show-toplevel 2> /dev/null )" +if [[ -z "$repo_root" ]]; then + if [[ "$where_i_am" =~ /.changelog$ || "$where_i_am" =~ /scripts$ ]]; then + # If this is in the .changelog or scripts directory, assume it's directly in {repo_root}. + repo_root="$( dirname "$where_i_am" )" + else + # Not in a git repo, and who knows where this script is in relation to the root, + # so let's just hope that our current location is the repo root. + repo_root='.' + fi + # Since we're not exactly sure we have the right repo_root here, we want to make sure the .changelog + # dir already exists. If not, we'd probably be trying to create the new file in the wrong place. + if [[ ! -d "${repo_root}/.changelog" ]]; then + printf 'Could not identify target directory.\n' + exit 1 + fi +fi +[[ -n "$verbose" ]] && printf ' Repo Root: "%s"\n' "$repo_root" + +out_dir="${repo_root}/.changelog/unreleased/dependencies" +[[ -n "$verbose" ]] && printf ' Output Dir: "%s"\n' "$out_dir" +if ! mkdir -p "$out_dir"; then + printf 'Could not create directory: %s\n' "$out_dir" + exit 1 +fi + +name="$( sed -E 's/[^[:alnum:]]+/-/g; s/^-//; s/-$//;' <<< "$branch_fn" | tr '[:upper:]' '[:lower:]' )" +[[ -n "$verbose" ]] && printf ' Name: "%s"\n' "$name" +out_fn="${out_dir}/${pr}-${name}.md" +[[ -n "$verbose" ]] && printf 'Output File: "%s"\n' "$out_fn" + +printf '* `%s` bumped to %s (from %s) %s.\n' "$lib" "$new_ver" "$old_ver" "$link" > "$out_fn" || exit 1 + +printf 'Dependabot changelog entry created: %s\n' "$out_fn" +exit 0 diff --git a/.changelog/get-dep-changes.sh b/.changelog/get-dep-changes.sh new file mode 100755 index 000000000..427622fb8 --- /dev/null +++ b/.changelog/get-dep-changes.sh @@ -0,0 +1,381 @@ +#!/bin/bash +# This script will git diff go.mod and identify the changes to it, +# outputting the info in a format ready for the changelog. + +show_usage () { + cat << EOF +get-dep-changes.sh: Analyze changes made to go.mod and generate changelog entries. + +Usage: ./get-dep-changes.sh {-p|--pull-request|--pr <num> | -n|--issue-no|--issue <num>} + [--id <id> [--dir <dir>]] [--target-branch <branch>] + [--force] [--no-clean] [-v|--verbose] [-h|--help] + +You must provide either a PR number or issue number, but you cannot provide both. +If an <id> is provided, the entries are written to a file, otherwise stdout. + +-p|--pull-request|--pr <num> + Append a PR link to the given <num> to the end of each changelog entry. +-n|--issue-no|--issue <num> + Append an issue link to the given <num> to the end of each changelog entry. + +--id <id> + The <id> is cleaned then appended to the <num> to create the filename for this change. + To clean the <id>, it is lowercased, then non-alphanumeric chars are changed to dashes. + If provided, the changelog entries will be written to + <repo root>/.changelog/unreleased/dependencies/<num>-<id>.md + If not provided, the changelog entries will be written to stdout. + If not in a git repo, or to put the file in a different directory, use the --dir <dir> option. + +--dir <dir> + Put the changelog entries in the provided <dir>. + This arg only has meaning if --id is also provided. + The default is '<repo root>/.changelog/unreleased/dependencies'. + +--target-branch <branch> + Providing this option allows you to compare current changes against a branch other than main. + By default, <branch> is "main". + +-v|--verbose + Output extra information. + +--no-clean + Do not delete the temporary directory used for processing. + +--force + If the output file already exists, overwrite it instead of outputting an error. + +Exit codes: + 0 No errors encountered. + 1 An error was encountered. + 10 There are no changes to go.mod. + +EOF + +} + +while [[ "$#" -gt '0' ]]; do + case "$1" in + -h|--help) + show_usage + exit 0 + ;; + --no-clean) + no_clean='YES' + ;; + -v|--verbose) + verbose='YES' + ;; + --target-branch) + if [[ -z "$2" ]]; then + printf 'No argument provided after %s\n' "$1" + exit 1 + fi + branch="$2" + shift + ;; + -p|--pull-request|--pr) + if [[ -z "$2" ]]; then + printf 'No argument provided after %s\n' "$1" + exit 1 + fi + if [[ "$2" =~ [^[:digit:]] ]]; then + printf 'Invalid %s value: [%s]. Only digits are allowed.\n' "$1" "$2" + exit 1 + fi + pr="$2" + shift + ;; + -n|--issue-no|--issue) + # Using -n and --issue-no to match the `unclog add` flags. + if [[ -z "$2" ]]; then + printf 'No argument provided after %s\n' "$1" + exit 1 + fi + if [[ "$2" =~ [^[:digit:]] ]]; then + printf 'Invalid %s value: [%s]. Only digits are allowed.\n' "$1" "$2" + exit 1 + fi + issue="$2" + shift + ;; + -i|--id) + if [[ -z "$2" ]]; then + printf 'No argument provided after %s\n' "$1" + exit 1 + fi + id="$2" + shift + ;; + --dir) + if [[ -z "$2" ]]; then + printf 'No argument provided after %s\n' "$1" + exit 1 + fi + out_dir="$2" + shift + ;; + --force) + force='YES' + ;; + *) + printf 'Unknown argument: %s\n' "$1" + exit 1 + ;; + esac + shift +done + +branch="${branch:-main}" + +if [[ -n "$pr" && -n "$issue" ]]; then + printf 'You cannot provide both a pr (%s) and issue (%s) number.\n' "$issue" "$pr" + exit 1 +elif [[ -n "$pr" ]]; then + link="[PR ${pr}](https://github.com/provenance-io/provenance/pull/${pr})" + num="$pr" +elif [[ -n "$issue" ]]; then + link="[#${issue}](https://github.com/provenance-io/provenance/issues/${issue})" + num="$issue" +else + printf 'You must provide either a --pr <num> or --issue <num>.\n' + exit 1 +fi +[[ -n "$verbose" ]] && printf 'Link: %s\n' "$link" + +if [[ -n "$id" ]]; then + id="$( sed -E 's/[^[:alnum:]]+/-/g; s/^-//; s/-$//;' <<< "$id" | tr '[:upper:]' '[:lower:]' )" + [[ -n "$verbose" ]] && printf 'Cleaned id: %s\n' "'$id'" + if [[ -n "$id" ]]; then + if [[ -z "$out_dir" ]]; then + repo_root="$( git rev-parse --show-toplevel )" || exit 1 + out_dir="${repo_root}/.changelog/unreleased/dependencies" + fi + out_fn="${out_dir}/${num}-${id}.md" + [[ -n "$verbose" ]] && printf 'Output filename: %s\n' "'$out_fn'" + if [[ -z "$force" && -e "$out_fn" ]]; then + printf 'Output file already exists: %s\n' "$out_fn" + exit 1 + fi + fi +fi + +[[ -n "$verbose" ]] && printf 'Creating temp dir.\n' +temp_dir="$( mktemp -d -t dep-updates.XXXX )" || exit 1 +[[ -n "$verbose" ]] && printf 'Created temp dir: %s\n' "$temp_dir" + +# Usage: clean_exit [<code>] +# Default <code> is 0. +# Cleans up the temp dir and exits. +clean_exit () { + local ec + ec="${1:-0}" + if [[ -n "$temp_dir" && -d "$temp_dir" ]]; then + if [[ -z "$no_clean" ]]; then + [[ -n "$verbose" ]] && printf 'Deleting temp dir: %s\n' "$temp_dir" + rm -rf "$temp_dir" + temp_dir='' + else + printf 'NOT deleting temp dir: %s\n' "$temp_dir" + fi + fi + exit "$ec" +} + +# Usage: <stuff> | clean_diff +# This will reformat the lines from a diff and remove lines we don't care about. +# All result lines will have one of these formats: +# "<library> <version>" +# "<library> => <location>" +# "<library> => <other library> <other version>" +# Note that this will strip out the leading + or -, so if that's important, it's up to you to keep track of. +clean_diff () { + # Use sed to: + # Remove the + or - and leading whitespace. + # If it now starts with "require" or "replace", remove that and any whitespace after it. + # Remove any line-ending comment. + # Remove all trailing whitespace. + # Make sure there are spaces around the "=>" in replace lines. + # Change all groups of 1 or more whitespace characters to a single space. + # Then use grep to only keep lines with one of the desired formats. + sed -E 's/^[-+[:space:]][[:space:]]*//; s/^(require|replace)[[:space:]]*//; s|//.*$||; s/[[:space:]]+$//; s/=>/ => /; s/[[:space:]]+/ /g;' \ + | grep -E '^[^ ]+ (v[^ ]+|=> [^ ]+( v[^ ]+)?)$' + return 0 +} + +# Usage: get_replace_str <lib> <filename> +# This will look for a replace line in <filename> for the <lib>. +# If found, it'll print a string describing the replacing version. +# If not found, this won't print anything. +get_replace_str () { + local lib fn repl + lib="$1" + fn="$2" + # Look in the file for a replace line for the library. + # If found, keep only the stuff after the =>. + # Using a variable for the grep expression is a bit clunky, so I'm using fixed string here. + # Technically this will match both "<lib> =>" and "<pre><lib> =>". Matching the second one + # would be problematic, but it's not accounted for in here. The nature of the names of the + # libraries makes it highly unlikely to happen. + repl="$( grep -F "$lib =>" "$fn" | sed -E 's/^.* => //' )" + if [[ -n "$repl" ]]; then + if [[ "$repl" =~ ' ' ]]; then + # $repl has the format "<other library> <other version>" + # We'll want it as "<other version> of `<other library>`". + # It's provided without quotes to printf so that it gets split on that + # space and provided as two separate args to put tics around the <other library>. + printf '%s of `%s`' $( sed -E 's/^([^ ]+) +(.*)$/\2 \1/' <<< "$repl" ) + else + # $repl is a <location>, put tics around it. + printf '`%s`' "$repl" + fi + fi + return 0 +} + +# Define all the temp files that we'll be making. +# The numbers in these roughly reflect the step that they're created in. +full_diff="${temp_dir}/1-full.diff" # The full results of the diff. +minus_lines="${temp_dir}/2-minus-lines.txt" # Just the subtractions we care about. +plus_lines="${temp_dir}/2-plus-lines.txt" # Just the additions we care about. +minus_requires="${temp_dir}/3-minus-requires.txt" # Just the removed requirement lines. +plus_requires="${temp_dir}/3-plus-requires.txt" # Just the added requirement lines. +cur_requires="${temp_dir}/3-cur-requires.txt" # All current requires (even ones not changing). +minus_replaces="${temp_dir}/3-minus-replaces.txt" # Just the removed replace lines. +plus_replaces="${temp_dir}/3-plus-replaces.txt" # Just the added replace lines. +cur_replaces="${temp_dir}/3-cur-replaces.txt" # All current replaces (even ones not changing). +changes="${temp_dir}/4-changes.md" # All the changelog entries (but without the link). +final="${temp_dir}/5-final.md" # The final changelog entry content. + +# Get the go.mod diff. +[[ -n "$verbose" ]] && printf 'Creating full diff: %s\n' "$full_diff" +git diff -U0 "$branch" -- go.mod > "$full_diff" +if ! grep -q '.' "$full_diff"; then + [[ -n "$verbose" ]] && printf 'go.mod does not have any changes.\n' + # Using the exit code of 10 here to indicate no changes. + clean_exit 10 +fi + +# Split it into subtractions and additions. +[[ -n "$verbose" ]] && printf 'Identifying all subtractions: %s\n' "$minus_lines" +grep -E '^-' "$full_diff" | clean_diff > "$minus_lines" +[[ -n "$verbose" ]] && printf 'Identifying all additions: %s\n' "$plus_lines" +grep -E '^\+' "$full_diff" | clean_diff > "$plus_lines" + +# Split it further into require lines and replace lines. +[[ -n "$verbose" ]] && printf 'Identifying subtracted requires: %s\n' "$minus_requires" +grep -Ev '=>' "$minus_lines" > "$minus_requires" +[[ -n "$verbose" ]] && printf 'Identifying subtracted replaces: %s\n' "$minus_replaces" +grep -E '=>' "$minus_lines" > "$minus_replaces" +[[ -n "$verbose" ]] && printf 'Identifying added requires: %s\n' "$plus_requires" +grep -Ev '=>' "$plus_lines" > "$plus_requires" +[[ -n "$verbose" ]] && printf 'Identifying added replaces: %s\n' "$plus_replaces" +grep -E '=>' "$plus_lines" > "$plus_replaces" + +# Identify all libraries that are changing. +[[ -n "$verbose" ]] && printf 'Identifying all changed libraries.\n' +libs=( $( sed -E 's/ .*$//' "$plus_lines" "$minus_lines" | sort -u ) ) + +# Identify all the current replace lines. +# This awk script outputs all lines that are either inside a "replace (" block, +# or start with "replace" (but aren't the beginning of a replace block). +# The clean_diff can be re-used here too to standardize the formatting and get only what we need. +[[ -n "$verbose" ]] && printf 'Identifying all current requires: %s\n' "$cur_requires" +awk '{if (inSec=="1" && /^[[:space:]]*\)[[:space:]]*$/) {inSec="";}; if (inSec=="1" || /^[[:space:]]*require[[:space:]]*[^([:space:]]/) {print $0;}; if (/^[[:space:]]*require[[:space:]]*\(/) {inSec="1";};}' go.mod | clean_diff > "$cur_requires" +[[ -n "$verbose" ]] && printf 'Identifying all current replaces: %s\n' "$cur_replaces" +awk '{if (inSec=="1" && /^[[:space:]]*\)[[:space:]]*$/) {inSec="";}; if (inSec=="1" || /^[[:space:]]*replace[[:space:]]*[^([:space:]]/) {print $0;}; if (/^[[:space:]]*replace[[:space:]]*\(/) {inSec="1";};}' go.mod | clean_diff > "$cur_replaces" + +# Figure out (and output) the changelog entry for each lib being changed. +[[ -n "$verbose" ]] && printf 'Identifying changelog entries for %d libraries: %s\n' "${#libs[@]}" "$changes" +i=0 +for lib in "${libs[@]}"; do + i=$(( i + 1 )) + [[ -n "$verbose" ]] && printf '[%d/%d]: Processing "%s".\n' "$i" "${#libs[@]}" "$lib" + + # These will either be empty or have the format "`<other lib>` <other version" or "`<location>`". + new_repl="$( get_replace_str "$lib" "$plus_replaces" )" + was_repl="$( get_replace_str "$lib" "$minus_replaces" )" + cur_repl="$( get_replace_str "$lib" "$cur_replaces" )" + [[ -n "$verbose" ]] && printf '[%d/%d]: %s="%s" %s="%s" %s="%s"\n' "$i" "${#libs[@]}" 'new_repl' "$new_repl" 'was_repl' "$was_repl" 'cur_repl' "$cur_repl" + + + # These will be either empty or be "<version>". + new_req="$( grep -F "$lib v" "$plus_requires" | sed -E 's/^.* //' )" + was_req="$( grep -F "$lib v" "$minus_requires" | sed -E 's/^.* //' )" + cur_req="$( grep -F "$lib v" "$cur_requires" | sed -E 's/^.* //' )" + [[ -n "$verbose" ]] && printf '[%d/%d]: %s="%s" %s="%s" %s="%s"\n' "$i" "${#libs[@]}" 'new_req' "$new_req" 'was_req' "$was_req" 'cur_req' "$cur_req" + + # If there weren't changes to require lines, but there is a require line + # for the library, we want a warning added to the end of the changelog entry + # since that change would be largely inconsequential. + warning='' + if [[ -z "$new_repl" && -z "$was_repl" && -n "$cur_repl" ]]; then + warning=" but is still replaced by $cur_repl" + fi + + # Pick the strings to use for the old and new versions. + # If there was a change to a replace line, use that over a changed require line. + new="${new_repl:-$new_req}" + was="${was_repl:-$was_req}" + + # Edge case: A replace line was removed, and the main entry didn't change. + if [[ -z "$new" && -n "$was_repl" && -z "$was_req" && -n "$cur_req" ]]; then + # We want to report that we are now on the currently required version. + new="$cur_req" + [[ -n "$verbose" ]] && printf '[%d/%d]: Using currently required version as new.\n' "$i" "${#libs[@]}" + fi + + # Edge case: A replace line was added, and the main entry didn't change. + if [[ -z "$was" && -n "$new_repl" && -z "$new_req" && -n "$cur_req" ]]; then + # We want to report that we were on the the currently required version. + was="$cur_req" + [[ -n "$verbose" ]] && printf '[%d/%d]: Using currently required version as was.\n' "$i" "${#libs[@]}" + fi + + [[ -n "$verbose" ]] && printf '[%d/%d]: %s="%s" %s="%s" %s="%s"\n' "$i" "${#libs[@]}" 'new' "$new" 'was' "$was" 'warning' "$warning" + + # Now generate the changelog line for this library. + if [[ -n "$new" && -n "$was" ]]; then + if [[ "$new" != "$was" ]]; then + [[ -n "$verbose" ]] && printf '[%d/%d]: Creating bump line.\n' "$i" "${#libs[@]}" + printf '* `%s` bumped to %s (from %s)%s\n' "$lib" "$new" "$was" "$warning" >> "$changes" + else + [[ -n "$verbose" ]] && printf '[%d/%d]: No change to report.\n' "$i" "${#libs[@]}" + fi + elif [[ -n "$new" ]]; then + [[ -n "$verbose" ]] && printf '[%d/%d]: Creating add line.\n' "$i" "${#libs[@]}" + printf '* `%s` added at %s%s\n' "$lib" "$new" "$warning" >> "$changes" + elif [[ -n "$was" ]]; then + [[ -n "$verbose" ]] && printf '[%d/%d]: Creating remove line.\n' "$i" "${#libs[@]}" + printf '* `%s` removed at %s%s\n' "$lib" "$was" "$warning" >> "$changes" + else + # It shouldn't be possible to see this, but it's here just in case things go wonky. + [[ -n "$verbose" ]] && printf '[%d/%d]: Creating unknown line.\n' "$i" "${#libs[@]}" + printf '* `%s` TODO: Could not identify dependency change details, fix me.\n' "$lib" >> "$changes" + fi +done + +# Append the link to each line. +[[ -n "$verbose" ]] && printf 'Appending link (%s) to each entry: %s\n' "$link" "$final" +awk -v link="$link" '{print $0 " " link ".";}' "$changes" > "$final" + +# Either put the file in place or output the content. +if [[ -n "$out_fn" ]]; then + out_dir="$( dirname "$out_fn" )" + if [[ -n "$out_dir" && "$out_dir" != '.' ]]; then + [[ -n "$verbose" ]] && printf 'Making dir (if it does not exist yet): %s\n' "$out_dir" + mkdir -p "$out_dir" || failed="YES" + fi + if [[ -z "$failed" ]]; then + [[ -n "$verbose" ]] && printf 'Copying final file from %s to %s\n' "$final" "$out_fn" + if cp "$final" "$out_fn"; then + copied='YES' + printf 'Dependency changelog entry created: %s\n' "$out_fn" + fi + fi +fi + +if [[ -z "$copied" ]]; then + cat "$final" +fi + +clean_exit 0 diff --git a/.changelog/get-valid-sections.sh b/.changelog/get-valid-sections.sh new file mode 100755 index 000000000..85b048efb --- /dev/null +++ b/.changelog/get-valid-sections.sh @@ -0,0 +1,24 @@ +#!/bin/bash +# This script will output all of the valid changelog section options. +# It extracts this list from the comment at the top of the CHANGELOG.md file. +# Any double-quoted string at the start of a line in that comment will be treated as a valid section header. +# Those strings are lower-cased and spaces turned to dashes. +# E.g the line `"Bug Fixes" for any bug fixes.` will result in a valid section of "bug-fixes". +# It's assumed that they're listed in the order that they should appear as sections. + +# Assume that this script is in the {repo_root}/.changelog/ dir and that the CHANGELOG.md file is directly in {repo_root}. +where_i_am="$( cd "$( dirname "${BASH_SOURCE:-$0}" )"; pwd -P )" +cl_file="$( dirname "$where_i_am" )/CHANGELOG.md" +if [[ ! -f "$cl_file" ]]; then + printf 'Changelog file does not exist: %s\n' "$cl_file" >&2 + exit 1 +fi +# This awk script finds the first comment (starts with <!--). +# While in the first comment, it looks for lines that start with a ". +# For each line: +# it strips the leading " and everything after (and including) the next ". +# It then strips an leading and trailing whitespace and turns any remaning whitespace into dashes. +# It exits once it reaches the end of that first comment. +# We then use tr to lower-case the strings. This isn't done in awk because tolower isn't available to all awks. +awk '{ if (in_com) { if (/^".*"/) { sub(/^"/,""); sub(/".*$/,""); sub(/^[[:space:]]+/,""); sub(/[[:space:]]+$/,""); gsub(/[[:space:]]+/,"-"); print $0; } else if (/-->/) { exit 0; }; }; if (/<!--/) { in_com=1; }; }' "$cl_file" | tr '[:upper:]' '[:lower:]' + diff --git a/.changelog/lint-unreleased.sh b/.changelog/lint-unreleased.sh new file mode 100755 index 000000000..42a7ac9df --- /dev/null +++ b/.changelog/lint-unreleased.sh @@ -0,0 +1,50 @@ +#!/bin/bash +# This script will check that the dirs in unreleased are all valid. + +[[ -n "$VERBOSE" ]] && set -x + +where_i_am="$( cd "$( dirname "${BASH_SOURCE:-$0}" )"; pwd -P )" +ur_dir="${where_i_am}/unreleased" +if [[ ! -d "$ur_dir" ]]; then + printf 'Unreleased changes Directory does not exist: %s\n' "$ur_dir" + exit 1 +fi + +valid_sections=( $( "${where_i_am}/get-valid-sections.sh" ) ) +# Usage: is_valid_section <section> +# Returns with exit code 0 if it is valid, or 1 if not. +is_valid_section () { + local s + for s in "${valid_sections[@]}"; do + if [[ "$s" == "$1" ]]; then + return 0 + fi + done + return 1 +} + +ec=0 +bad_sections=() +for section in $( find "$ur_dir" -type d -mindepth 1 -maxdepth 1 | sed 's|^.*/||' ); do + if ! is_valid_section "$section"; then + bad_sections+=( "$section" ) + fi +done + +bad_files="$( find "$ur_dir" -type f -mindepth 2 -name '*[[:space:]]*' | sed -E 's|^.*/\.changelog/|.changelog/|' )" + +[[ -n "$VERBOSE" ]] && set +x + +if [[ "${#bad_sections[@]}" -ne '0' ]]; then + printf 'Invalid unreleased section(s):\n' + printf '.changelog/unreleased/%s\n' "${bad_sections[@]}" + printf 'Valid sections: [%s]\n' "${valid_sections[*]}" + ec=1 +fi + +if [[ -n "$bad_files" ]]; then + printf 'Invalid unreleased filename(s):\n%s\n' "$bad_files" + ec=1 +fi + +exit "$ec" diff --git a/.changelog/prep-release.sh b/.changelog/prep-release.sh new file mode 100755 index 000000000..8006ea4fa --- /dev/null +++ b/.changelog/prep-release.sh @@ -0,0 +1,584 @@ +#!/bin/bash +# This script will update the changelog stuff to mark a release. + +show_usage () { + cat << EOF +prep-release.sh: Prepares the changelog for a new release. + +Usage: prep-release.sh <version> [--date <date> [--force-date]] + [--force-version] [--no-clean] [-v|--verbose] [-h|--help] + +The <version> must have format vA.B.C or vA.B.C-rcX where A, B, C and X are numbers. + +--date <date> is an optional way to define the release date. + Must have the format YYYY-MM-DD. + The default is today's date. +--force-date indicates that the provided date is correct. + By default, this script won't allow dates that are more than 14 days before or after today. + This flag allows you to bypass that check. + +--force-version indicates that the provided version is correct. + By default, this script requires versions to be incremental and not yet exist. + This flag allows you to bypass those requirements. + The version must always have a correct format, though, even with this flag. + +--no-clean causes the temporary directory to remain once the script has exited. + +EOF + +} + +while [[ "$#" -gt '0' ]]; do + case "$1" in + -h|--help) + show_usage + exit 0 + ;; + --date) + if [[ -z "$2" ]]; then + printf 'No argument provided after %s\n' "$1" + exit 1 + fi + date="$2" + shift + ;; + --force-date) + force_date="$1" + ;; + --force-version) + force_version="$1" + ;; + -v|--verbose) + verbose="$1" + ;; + --no-clean) + no_clean="$1" + ;; + *) + if [[ -n "$version" ]]; then + printf 'Unknown argument: %s\n' "$1" + fi + version="$1" + ;; + esac + shift +done + +if [[ -z "$version" ]]; then + show_usage + exit 0 +fi + +if ! command -v unclog > /dev/null 2>&1; then + # Issue standard command-not-found message. + unclog + printf 'See: https://github.com/informalsystems/unclog\n' + exit 1 +fi + +######################################################################################################################## +################################################ Setup and Validation ################################################ +######################################################################################################################## + +printf 'Doing Setup and Validation.\n' + +where_i_am="$( cd "$( dirname "${BASH_SOURCE:-$0}" )"; pwd -P )" +repo_root="$( git rev-parse --show-toplevel 2> /dev/null )" +if [[ -z "$repo_root" ]]; then + if [[ "$where_i_am" =~ /.changelog$ || "$where_i_am" =~ /scripts$ ]]; then + # If this is in the .changelog or scripts directory, assume it's {repo_root}/<dir>. + repo_root="$( dirname "$where_i_am" )" + else + # Not in a git repo, and who knows where this script is in relation to the root, + # so let's just hope that our current location is the repo root. + repo_root='.' + fi +fi +[[ -n "$verbose" ]] && printf ' Repo root dir: [%s].\n' "$repo_root" + +changelog_file="${repo_root}/CHANGELOG.md" +if [[ ! -f "$changelog_file" ]]; then + printf 'Could not find existing CHANGELOG.md file.\n' + exit 1 +fi +[[ -n "$verbose" ]] && printf 'Changelog file: [%s].\n' "$changelog_file" +changelog_dir="${repo_root}/.changelog" +if [[ ! -d "$changelog_dir" ]]; then + printf 'Could not find the .changelog/ dir.\n' + exit 1 +fi +[[ -n "$verbose" ]] && printf ' Changelog dir: [%s].\n' "$changelog_dir" + + +# Do some superficial validation on the provided version. We'll do more later though. +[[ -n "$verbose" ]] && printf 'Doing initial validation on the version: [%s].\n' "$version" +if ! grep -Eq '^v[[:digit:]]+\.[[:digit:]]+\.[[:digit:]]+(-rc[[:digit:]]+)?$' <<< "$version" 2> /dev/null; then + printf 'Invalid version format [%s]. Use vA.B.C or vA.B.C-rcX.\n' "$version" + exit 1 +fi +v_major="$( sed -E 's/^v([^.]+)\..*$/\1/' <<< "$version" )" +v_minor="$( sed -E 's/^[^.]+\.//; s/\..*$//' <<< "$version" )" +v_patch="$( sed -E 's/^[^.]+\.[^.]+\.//; s/-rc.*$//;' <<< "$version" )" +v_rc="$( sed -E 's/^[^.]+\.[^.]+\.[^-]+//; s/^.*-rc//;' <<< "$version" )" +if [[ -n "$v_rc" && "$v_rc" -eq '0' ]]; then + printf 'Invalid version: [%s]. Release candidate numbering starts at 1.\n' "$version" + exit 1 +fi +v_base="v${v_major}.${v_minor}.${v_patch}" +[[ -n "$verbose" ]] && printf 'Version: [%s] = Major: [%s] . Minor: [%s] . Patch: [%s] (%s) - RC: [%s]\n' \ + "$version" "$v_major" "$v_minor" "$v_patch" "$v_base" "$v_rc" + +# Make sure the new version directory does not already exist. +new_ver_dir="${changelog_dir}/${version}" +if [[ -d "$new_ver_dir" ]]; then + # If the new version directory already exists, and you are redoing the changelog for it, you'll need to + # move the things you want into unreleased and then delete the existing version directory. + # Nothing is done about it in here because there's no way to know what all should be included, or even + # if the existing version was just provided by accident. + printf 'The changelog version directory for [%s] already exists: [%s].\n' "$version" "$new_ver_dir" + exit 1 +fi + +# If this is not an rc, there needs to be a summary file. +# If this is an rc and there isn't a summary file, we'll create a default one later (when it's needed). +unreleased_dir="${changelog_dir}/unreleased" +unreleased_sum_file="${unreleased_dir}/summary.md" +[[ -n "$verbose" ]] && printf 'Checking summary file: [%s].\n' "$unreleased_sum_file" +if [[ -f "$unreleased_sum_file" ]] && awk '{ sub(/<!--.*-->/,""); if (in_com) { if (sub(/^.*-->/,"")) { in_com=0; } else { $0=""; }; } else if (sub(/<!--.*$/,"")) { in_com=1; }; if (/./) { all_good=1; exit 0; }; } END { if (!all_good) { exit 1; }; }' "$unreleased_sum_file"; then + have_summary='YES' +fi +if [[ -z "$v_rc" ]]; then + if [[ -z "$have_summary" ]]; then + printf 'A summary is required, but the file either does not exist or does not have any content: [%s].\n' "$unreleased_sum_file" + exit 1 + fi +fi + +# Validate the date (or get it if not provided). +[[ -n "$verbose" ]] && printf 'Getting or validating the date: [%s].\n' "$date" +if [[ -z "$date" ]]; then + date="$( date +'%F' )" +elif [[ ! "$date" =~ ^[[:digit:]]{4}-[[:digit:]]{2}-[[:digit:]]{2}$ ]]; then + printf 'Invalid date format [%s]. Use YYYY-MM-DD.\n' "$date" + exit 1 +else + # The GNU version of `date --help` exits with code 0. + # The BSD/OSX version of `date --help` exits with code 1. + # We use that to identify which version of the date command we have so we can use the correct args. + if date --help > /dev/null 2>&1; then + gnu_date='YES' + # GNU version. If the provided date is not a valid date (e.g. month 13 or day 31 in a 30 day month), + # this command will exit with code 1 and output some stuff to stderr (which we nullify). + # If it's an actual date, it will exit with code 0. + if ! date -d "$date" +'%F' > /dev/null 2>&1; then + printf 'Invalid date: %s\n' "$date" + exit 1 + fi + else + # BSD/OSX version. This one is so good that providing a date of '2024-06-31' is just treated as valid, but `2024-07-01`. + # But something like '2024-13-01' or '2024-07-32' exit with code 1 and print only to stderr. + # So for this one, we have to compare that result back to the date we have. + if [[ "$date" != "$( date -j -f '%F' "$date" +'%F' 2> /dev/null )" ]]; then + printf 'Invalid date: %s\n' "$date" + exit 1 + fi + fi + + # Make sure the date is within the previous or next 14 days. + # This is mostly to make it harder to accidentally use the wrong year or month. + if [[ -n "$gnu_date" ]]; then + date_s="$( date -d "$date" +'%s' 2> /dev/null )" + # GNU date will read the format 'YYYY-MM-DD' as having 00:00:00 for the time, which is what we want. + cur_date_s="$( date -d "$( date +'%F' )" +'%s' )" + else + date_s="$( date -j -f '%F' "$date" +'%s' )" + # BSD/OSX date will read the format 'YYYY-MM-DD' as having the current time, but we want the epoch at + # the start of the day so that we're only paying attention to whole days. + cur_date_s="$( date -j -f '%F %T' "$( date +'%F' ) 00:00:00" +'%s' )" + fi + date_diff_s=$(( date_s - cur_date_s )) + date_diff_s=${date_diff_s#-} # remove the leading minus if there. + # 60s/m * 60m/h * 24h/d * 14d = 1209600s + if [[ "$date_diff_s" -gt '1209600' ]]; then + if [[ -z "$force_date" ]]; then + printf 'The date %s is too far away. If it is correct, rerun this command and include the --force-date flag.\n' "$date" + exit 1 + else + printf 'Warning: [%s] is more than 14 days away from today.\n' "$date" + fi + fi +fi +[[ -n "$verbose" ]] && printf 'Date: [%s].\n' "$date" + +# Create a temp directory to store some processing files. +[[ -n "$verbose" ]] && printf 'Creating temp dir.\n' +temp_dir="$( mktemp -d -t prep-release.XXXX )" || exit 1 +[[ -n "$verbose" ]] && printf 'Created temp dir: %s\n' "$temp_dir" + +# Usage: clean_exit [<code>] +# Default <code> is 0. +# Since we now have a temp dir to clean up, there should not be any exit statements after this. Use this instead. +# Cleans up the temp dir and exits. +clean_exit () { + local ec + ec="${1:-0}" + if [[ -n "$temp_dir" && -d "$temp_dir" ]]; then + if [[ -z "$no_clean" ]]; then + [[ -n "$verbose" ]] && printf 'Deleting temp dir: %s\n' "$temp_dir" + rm -rf "$temp_dir" + temp_dir='' + else + printf 'NOT deleting temp dir: %s\n' "$temp_dir" + fi + fi + exit "$ec" +} + +# Usage: handle_invalid_version <msg> <args> +# Outputs a message about an invalid version. Then, if not forcing the version, it'll exit this script. +handle_invalid_version () { + local msg + msg="$1" + shift + if [[ -z "$force_version" ]]; then + printf 'Invalid version: [%s]. '"$msg"'\n' "$version" "$@" + clean_exit 1 + else + printf 'Warning: Version: [%s]: '"$msg"'\n' "$version" "$@" + fi + return 0 +} + +# Create a file with a list of all the current versions. +versions_file="${temp_dir}/versions.txt" +[[ -n "$verbose" ]] && printf 'Creating versions file: %s.\n' "$versions_file" +grep -oE '^## \[v[^]]+' "$changelog_file" | sed -E 's/^## \[//' > "$versions_file" + +# Do some more validation on the new version. +[[ -n "$verbose" ]] && printf 'Validating new version against existing ones.\n' + +if grep -qFx "$version" "$versions_file" 2> /dev/null; then + handle_invalid_version 'Version already exists.' +fi +if [[ -n "$v_rc" ]] && grep -qFx "$v_base" "$versions_file" 2> /dev/null; then + handle_invalid_version 'Cannot create a release candidate for a version that already exists: [%s].' "$v_base" +fi + +# Get the most recent non-rc version so that we can ensure we're using the right next version. +prev_ver="$( { cat "$versions_file"; printf '%s\n' "$v_base"; } | grep -v -e '-rc' | sort --version-sort --reverse | grep -A 1 -Fx "$v_base" | tail -n 1 )" + +if [[ -n "$v_rc" && "$v_rc" -ge '2' ]]; then + # If the new version is a release candidate of 2 or more, also ensure the previous rc exists. + prev_ver_rc="$( { cat "$versions_file"; printf '%s\n' "$version"; } | sort --version-sort --reverse | grep -A 1 -Fx "$version" | tail -n 1 )" + if [[ "${v_base}-rc$(( v_rc - 1 ))" != "$prev_ver_rc" ]]; then + handle_invalid_version 'Release candidate is not sequential. Previous version: [%s].' "$prev_ver_rc" + fi +fi + +if [[ "$v_patch" -ne '0' ]]; then + if [[ "v${v_major}.${v_minor}.$(( v_patch - 1 ))" != "$prev_ver" ]]; then + handle_invalid_version 'Patch number is not sequential. Previous version: [%s].' "$prev_ver" + fi +elif [[ "$v_minor" -ne '0' ]]; then + if [[ "v${v_major}.$(( v_minor - 1 ))." != "$( sed -E 's/[^.]+$/' <<< "$prev_ver" )" ]]; then + handle_invalid_version 'Minor number is not sequential. Previous version: [%s].' "$prev_ver" + fi +else + if [[ "v${v_major}." != "$( sed -E 's/[^.]+\.[^.]+*$//' <<< "$prev_ver" )" ]]; then + handle_invalid_version 'Major number is not sequential. Previous version: [%s].' "$prev_ver" + fi +fi + +if [[ -n "$verbose" ]]; then + printf ' New Version: [%s].\n' "$version" + printf 'Previous Non-RC Version: [%s].\n' "$prev_ver" + [[ -n "$prev_ver_rc" ]] && printf ' Previous RC Version: [%s].\n' "$prev_ver_rc" +fi + +# Usage: combine_rc_dirs +# This is extracted as a function for easier short-circuit control in this process. +# It will move all the entry files from the rc dirs for this version into unreleased. +combine_rc_dirs () { + local rc_vers v rc_ver v_id rc_ver_dir entries sections s section s_id s_dir e entry e_id v_file u_file + + [[ -n "$verbose" ]] && printf 'Identifying rc version dirs for this version.\n' + rc_vers=( $( find "$changelog_dir" -type d -mindepth 1 -maxdepth 1 -name "${version}-rc*" | sed -E 's|^.*/||' ) ) + [[ -n "$verbose" ]] && printf 'Found %d version dirs: [%s].\n' "${#rc_vers[@]}" "${rc_vers[*]}" + [[ "${#rc_vers[@]}" -eq '0' ]] && return 0 + + printf 'Combining Release Candidate dirs back into unreleased.\n' + v=0 + for rc_ver in "${rc_vers[@]}"; do + v=$(( v + 1 )) + v_id="[${v}/${#rc_vers[@]}=${rc_ver}]" + [[ -n "$verbose" ]] && printf '%s: Identifying entry files.\n' "$v_id" + rc_ver_dir="${changelog_dir}/${rc_ver}" + entries=( $( find "$rc_ver_dir" -type f -mindepth 2 -maxdepth 2 -name '*.md' | grep -Eo '[^/]+/[^/]+$' ) ) + [[ -n "$verbose" ]] && printf '%s: Found %d entry files.\n' "$v_id" "${#entries[@]}" + + if [[ "${#entries[@]}" -gt '0' ]]; then + [[ -n "$verbose" ]] && printf '%s: Identifying sections.\n' "$v_id" + sections=( $( printf '%s\n' "${entries[@]}" | sed -E 's|/.*$||' | sort -u ) ) + + [[ -n "$verbose" ]] && printf '%s: Making sure %d sections exist in unreleased: [%s].\n' "$v_id" "${#sections[@]}" "${sections[*]}" + s=0 + for section in "${sections[@]}"; do + s=$(( s + 1 )) + s_id="${v_id}[${s}/${#sections[@]}]" + [[ -n "$verbose" ]] && printf '%s: Making section [%s] in unreleased (if it does not exist yet).\n' "$s_id" "$section" + s_dir="${unreleased_dir}/${section}" + if [[ ! -d "$s_dir" ]] && ! mkdir "$s_dir"; then + printf '%s: Failed to make section dir: [%s].\n' "$s_id" "$section" + return 1 + fi + done + + [[ -n "$verbose" ]] && printf '%s: Moving %d entry files to unreleased.\n' "$v_id" "${#entries[@]}" + e=0 + for entry in "${entries[@]}"; do + e=$(( e + 1 )) + e_id="${v_id}[${e}/${#entries[@]}]" + + [[ -n "$verbose" ]] && printf '%s: Moving entry to unreleased: [%s].\n' "$e_id" "$entry" + v_file="${rc_ver_dir}/${entry}" + u_file="${unreleased_dir}/${entry}" + if [[ -e "$u_file" ]]; then + if diff -q "$v_file" "$u_file" > /dev/null 2>&1; then + # The unreleased file already exists and is the same as the version one. Just delete the version one. + if ! rm "$v_file"; then + printf "%s: Failed to delete version file: [%s].\n" "$e_id" "$v_file" + return 1 + fi + else + printf '%s: Cannot move entry [%s] into unreleased because it already exists and is different.\n' "$e_id" "$entry" + return 1 + fi + else + if ! mv "$v_file" "$u_file"; then + printf '%s: Failed to move entry file [%s] to [%s].\n' "$e_id" "$v_file" "$u_file" + return 1 + fi + fi + done + fi + + [[ -n "$verbose" ]] && printf '%s: Deleting rc dir.\n' "$v_id" + if [[ -n "$( find "$rc_ver_dir" -type f -not -name '.*' -not -name 'summary.md' )" ]]; then + printf '%s: Cannot delete non-empty rc directory.\n' "$v_id" + return 1 + elif ! rm -rf "$rc_ver_dir"; then + printf '%s: Failed to delete rc directory.\n' "$v_id" + return 1 + fi + done + + return 0 +} + +# If this is not an rc, we want to move all of the "released" rc entries into unreleased so that we can +# get the changelog of all the new stuff in this release. +if [[ -z "$v_rc" ]]; then + combine_rc_dirs || clean_exit 1 +fi + +######################################################################################################################## +############################################## Build New Release Notes ############################################### +######################################################################################################################## + +printf 'Creating the new Release Notes file.\n' + +# If this is an rc and we don't have a summary, create a default one now. +if [[ -n "$v_rc" && -z "$have_summary" ]]; then + printf 'This is Provenance Blockchain release candidate `%s`.\n' "$version" > "$unreleased_sum_file" +fi + +# Use unclog to generate the beginnings of the new release notes. +unclog_build_file="${temp_dir}/1-unclog-build.md" +[[ -n "$verbose" ]] && printf 'Generating initial changelog entry: [%s].\n' "$unclog_build_file" +cd "${repo_root}" || clean_exit 1 +unclog build --unreleased-only > "$unclog_build_file" || clean_exit 1 + +# Make sure all the PR and issue links have the correct link text. +# Also, if there's period right before and after the link, get rid of the one before. +links_fixed_file="${temp_dir}/2-links-fixed.md" +[[ -n "$verbose" ]] && printf 'Fixing the link text: [%s].\n' "$links_fixed_file" +sed -E -e 's|\[[^[:digit:]]*[[:digit:]]+\](\([^)]+/pull/([[:digit:]]+)\))|[PR \2]\1|g' \ + -e 's|\[[^[:digit:]]*[[:digit:]]+\](\([^)]+/issues/([[:digit:]]+)\))|[#\2]\1|g' \ + -e 's|\.([[:space:]]*\[[^]]+\]\([^)]+\))[[:space:]]*\.|\1.|g' \ + "$unclog_build_file" > "$links_fixed_file" || clean_exit 1 +if grep -qE '[^[:space:]]' <<< "$( tail -n 1 "$links_fixed_file" )" > /dev/null 2>&1; then + # The last line is not a blank line, add one now so that the last section is consistent with the rest. + printf '\n' >> "$links_fixed_file" +fi + +# Split it out into individual sections so that we can more easily re-order them. +cur_file="${temp_dir}/3-section-top.md" +[[ -n "$verbose" ]] && printf 'Splitting into section files.\nNow writing to: [%s].\n' "$cur_file" + +# Usage: lower_to_title <words> +# Converts the first letter of each word to upper-case, and standardizes word spacing. +lower_to_title () { + # Unfortunately, there isn't an easy way to do this that is also portable. + # Expansions of ${var,,} (convert everything to lower) and ${var^} (convert first char to upper), + # aren't available everywhere. Also, not all versions of sed allow for the \L and \u directives. + # Even the toupper and tolower functions in awk aren't always available. + # The ${var::} expansion seems to be pretty widely available, though, so we'll use that with tr on each word. + local words + words=() + while [[ "$#" -gt '0' ]]; do + # ${1:0:1} means "in the $1 variable, get a substring starting at char 0 with length 1." + # ${1:1} means "in the $1 variable, get a substring starting at char 1 (and going to the end of the string)." + words+=( "$( tr '[:lower:]' '[:upper:]' <<< "${1:0:1}" )${1:1}" ) + shift + done + # This also standardizes the spacing before, between, and after the words. + printf '%s' "${words[*]}" +} + +actual_sections=() +while IFS="" read -r line || [[ -n "$line" ]]; do + if [[ "$line" =~ ^##[[:space:]]+Unreleased[[:space:]]*$ ]]; then + printf '## [%s](https://github.com/provenance-io/provenance/releases/tag/%s) %s\n' "$version" "$version" "$date" >> "$cur_file" + elif [[ "$line" =~ ^###[[:space:]] ]]; then + [[ -n "$verbose" ]] && printf 'Found new section line: [%s].\n' "$line" + section="$( sed -E 's/^###[[:space:]]+//; s/[[:space:]]+$//; s/[^[:alnum:]]+/-/g;' <<< "$line" | tr '[:upper:]' '[:lower:]' )" + actual_sections+=( "$section" ) + cur_file="${temp_dir}/3-section-${section}.md" + [[ -n "$verbose" ]] && printf 'Now writing to: [%s].\n' "$cur_file" + # I'm providing the section unquoted here so that the shell splits it into words for us. + printf '### %s\n' "$( lower_to_title $( tr '-' ' ' <<< "$section" ) )" >> "$cur_file" + else + printf '%s\n' "$line" >> "$cur_file" + fi +done < "$links_fixed_file" + +# Sort the entries of the dependencies section. +# They have the format "* `<library>` <action> <version> ..." where <action> is one of "added at" "bumped to" or "removed at". +# So, if we just sort them using version sort, it'll end up sorting them by library and version, which a handy way to view them. +dep_file="${temp_dir}/3-section-dependencies.md" +if [[ -f "$dep_file" ]]; then + [[ -n "$verbose" ]] && printf 'Sorting the dependency entries: [%s].\n' "$dep_file" + orig_dep_file="${dep_file}.orig" + mv "$dep_file" "$orig_dep_file" + head -n 2 "$orig_dep_file" > "$dep_file" + grep -E '^[[:space:]]*[-*]' "$orig_dep_file" | sed -E 's/^[[:space:]]+//; s/^(.)[[:space:]]+/\1 /;' | sort --version-sort >> "$dep_file" + printf '\n' >> "$dep_file" +fi + +[[ -n "$verbose" ]] && printf 'Determining desired order for sections.\n' + +section_order=() +add_to_section_order () { + while [[ "$#" -gt '0' ]]; do + for s in "${section_order[@]}"; do + if [[ "$s" == "$1" ]]; then + shift + continue 2 + fi + done + section_order+=( "$1" ) + shift + done +} + +# These sub-commands aren't quoted here because we want it split on spaces (and I know they don't have any glob stuff). +# We want "top" first, it's the version header and blurb. +# Then use the pre-defined section order for the expected sections (but hold off on dependencies). +# Then add all the actual sections so that any unexpected sections are still included. +# And lastly, we have the dependencies section. +add_to_section_order top \ + $( "${where_i_am}/get-valid-sections.sh" | grep -vFx 'dependencies' ) \ + $( printf '%s\n' "${actual_sections[@]}" | grep -vFx 'dependencies' ) \ + dependencies +[[ -n "$verbose" ]] && printf 'Including sections in this order (%d): [%s].\n' "${#section_order[@]}" "${section_order[*]}" + +new_cl_entry_file="${temp_dir}/4-release-notes.md" +[[ -n "$verbose" ]] && printf 'Re-combining sections with proper order: [%s].\n' "$new_cl_entry_file" + +s=0 +for section in "${section_order[@]}"; do + s=$(( s + 1 )) + s_id="[${s}/${#}=${section}]" + s_file="${temp_dir}/3-section-${section}.md" + if [[ ! -f "$s_file" ]]; then + [[ -n "$verbose" ]] && printf '%s: No section file to include: [%s].\n' "$s_id" "$s_file" + continue + fi + [[ -n "$verbose" ]] && printf '%s: Including [%s].\n' "$s_id" "$s_file" + if ! cat "$s_file" >> "$new_cl_entry_file"; then + printf '%s: Could not append [%s] to [%s].\n' "$s_id" "$s_file" "$new_cl_entry_file" + clean_exit 1 + fi +done + +[[ -n "$verbose" ]] && printf 'Appending diff links: [%s].\n' "$new_cl_entry_file" +printf '### Full Commit History\n\n' >> "$new_cl_entry_file" +[[ -n "$prev_ver_rc" ]] && printf '* https://github.com/provenance-io/provenance/compare/%s...%s\n' "$prev_ver_rc" "$version" >> "$new_cl_entry_file" +printf '* https://github.com/provenance-io/provenance/compare/%s...%s\n\n' "$prev_ver" "$version" >> "$new_cl_entry_file" + +# Usage: clean_versions <input file> +# or: <stuff> clean_versions +# This will remove this version and any rcs for this verison from changelog content. +clean_versions () { + awk -v version="$version" '{ if (/^##[[:space:]]/) { if (index($0,version "-rc") || index($0,"[" version "]")) { keep=0; } else { keep=1; }; }; if (keep) { print $0; }; }' "$@" +} + +# If this is an rc and there's an existing release notes, append those to the end, removing any existing section for this version. +# If it's not an rc, or there isn't an existing one, just use what we've already got. +new_rl_file="${temp_dir}/5-release-notes.md" +release_notes_file="${repo_root}/RELEASE_NOTES.md" +cp "$new_cl_entry_file" "$new_rl_file" +if [[ -n "$v_rc" && -f "$release_notes_file" ]]; then + [[ -n "$verbose" ]] && printf 'Including existing release notes: [%s].\n' "$release_notes_file" + printf -- '---\n\n' >> "$new_rl_file" + clean_versions "$release_notes_file" >> "$new_rl_file" +fi + +[[ -n "$verbose" ]] && printf 'Putting release notes into place: cp "%s" "%s".\n' "$new_rl_file" "$release_notes_file" +cp "$new_rl_file" "$release_notes_file" || clean_exit 1 + +######################################################################################################################## +############################################### Building New Changelog ############################################### +######################################################################################################################## + +printf 'Creating the new Changelog file.\n' + +new_cl_file="${temp_dir}/6-new-changelog.md" +[[ -n "$verbose" ]] && printf 'Putting top of changelog in temp file: [%s].\n' "$new_cl_file" +# Get the top of the changelog file up to the first version header that isn't unreleased. +awk '{if (/^##[[:space:]]/ && $0 !~ /[Uu][Nn][Rr][Ee][Ll][Ee][Aa][Ss][Ee][Dd]/) { exit 0; }; print $0; }' "$changelog_file" > "$new_cl_file" + +[[ -n "$verbose" ]] && printf 'Appending the new version entry (without the blurb): [%s].\n' "$new_cl_file" +# Include the new stuff, but remove any blurb. +awk '{ if (/^##[[:space:]]/) { in_top=1; print $0; print ""; } else if (/^###[[:space:]]/) { in_top=0; }; if (!in_top) { print $0; }; }' "$new_cl_entry_file" >> "$new_cl_file" +# Add a divider. +printf -- '---\n\n' >> "$new_cl_file" + +[[ -n "$verbose" ]] && printf 'Appending the rest of the changelog file: [%s].\n' "$new_cl_file" +# Get the rest of the changelog file, but remove any existing entry for this version and also any rcs for +# this version (e.g. if the version is v1.2.3, remove v1.2.3-rc1 etc.). If this version is an rc, it shouldn't +# remove the other rcs, though (e.g. if the version is v1.2.3-rc2, we still want v1.2.3-rc1 in there). +awk '{if (!in_bot && /^##[[:space:]]/ && $0 !~ /[Uu][Nn][Rr][Ee][Ll][Ee][Aa][Ss][Ee][Dd]/) { in_bot=1; }; if (in_bot) { print $0; }; }' "$changelog_file" | clean_versions >> "$new_cl_file" + +[[ -n "$verbose" ]] && printf 'Putting changelog into place: [%s].\n' "$new_cl_file" +cp "$new_cl_file" "$changelog_file" || clean_exit 1 + +######################################################################################################################## +################################################## Finalize Release ################################################## +######################################################################################################################## + +printf 'Finishing the release.\n' + +# Note: I'm not using unlcog release here because they assume you're going to do that before unclog build, +# and will try to open the editor to change the summary. But we don't want that since we've already done +# stuff with the summary and it shouldn't change now. + +[[ -n "$verbose" ]] && printf 'Moving unreleased entries to new version dir: [%s].\n' "$new_ver_dir" +mv "$unreleased_dir" "$new_ver_dir" || clean_exit 1 +rm "$new_ver_dir/.gitkeep" > /dev/null 2>&1 +[[ -n "$verbose" ]] && printf 'Creating new unreleased dir: [%s].\n' "$unreleased_dir" +mkdir -p "$unreleased_dir" || clean_exit 1 +touch "$unreleased_dir/.gitkeep" + +printf 'Done.\n' +clean_exit 0 diff --git a/.changelog/unreleased/.gitkeep b/.changelog/unreleased/.gitkeep new file mode 100644 index 000000000..e69de29bb diff --git a/.changelog/unreleased/improvements/2112-switch-to-unclog.md b/.changelog/unreleased/improvements/2112-switch-to-unclog.md new file mode 100644 index 000000000..4450d7c66 --- /dev/null +++ b/.changelog/unreleased/improvements/2112-switch-to-unclog.md @@ -0,0 +1 @@ +* Switch to `unclog` for unreleased changelog entries [#2112](https://github.com/provenance-io/provenance/pull/2112). diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md index c47b4e537..371c56dae 100644 --- a/.github/PULL_REQUEST_TEMPLATE.md +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -18,11 +18,11 @@ Before we can merge this PR, please make sure that all the following items have checked off. If any of the checklist items are not applicable, please leave them but write a little note why. -- [ ] Targeted PR against correct branch (see [CONTRIBUTING.md](https://github.com/provenance-io/provenance/blob/main/CONTRIBUTING.md#pr-targeting)) +- [ ] Targeted PR against correct branch (see [CONTRIBUTING.md](https://github.com/provenance-io/provenance/blob/main/CONTRIBUTING.md#pr-targeting)). - [ ] Linked to Github issue with discussion and accepted design OR link to spec that describes this work. - [ ] Wrote unit and integration [tests](https://github.com/provenance-io/provenance/blob/main/CONTRIBUTING.md#testing) -- [ ] Updated relevant documentation (`docs/`) or specification (`x/<module>/spec/`) +- [ ] Updated relevant documentation (`docs/`) or specification (`x/<module>/spec/`). - [ ] Added relevant `godoc` [comments](https://blog.golang.org/godoc-documenting-go-code). -- [ ] Added a relevant changelog entry to the `Unreleased` section in `CHANGELOG.md` -- [ ] Re-reviewed `Files changed` in the Github PR explorer -- [ ] Review `Codecov Report` in the comment section below once CI passes +- [ ] Added relevant changelog entries under `.changelog/unreleased` (see [Adding Changes](https://github.com/provenance-io/provenance/blob/main/.changelog/README.md#adding-changes)). +- [ ] Re-reviewed `Files changed` in the Github PR explorer. +- [ ] Review `Codecov Report` in the comment section below once CI passes. diff --git a/.github/workflows/changelog.yml b/.github/workflows/changelog.yml index ae79649f8..2498690d1 100644 --- a/.github/workflows/changelog.yml +++ b/.github/workflows/changelog.yml @@ -15,13 +15,34 @@ permissions: jobs: changelog: runs-on: ubuntu-latest + if: contains(github.event.pull_request.labels.*.name, 'dependencies') + env: + # Getting the title directly in the run script opens an injection vulnerability. + # Using an env var for it (like this) prevents that vulnerability. + PR_TITLE: ${{ github.event.pull_request.title }} + PR_NUM: ${{ github.event.number }} + PR_BASE: ${{ github.event.pull_request.head.ref }} + PR_TARGET: ${{ github.event.pull_request.base.ref }} steps: - - uses: actions/checkout@v4 + - name: Checkout PR Branch + uses: actions/checkout@v4 with: # Depending on your needs, you can use a token that will re-trigger workflows # See https://github.com/stefanzweifel/git-auto-commit-action#commits-made-by-this-action-do-not-trigger-new-workflow-runs token: ${{ secrets.BOT_CPR_PAT }} + - name: Fetch Target Branch + run: git fetch origin "${PR_TARGET}:${PR_TARGET}" --depth 1 + + - name: Generate entry + run: | + ./.changelog/dependabot-changelog.sh \ + --verbose \ + --pr "$PR_NUM" \ + --title "$PR_TITLE" \ + --head-branch "$PR_BASE" \ + --target-branch "$PR_TARGET" + # All commits must be signed, import key and sign commit of updated change log. - name: Import GPG key id: import_gpg @@ -33,12 +54,6 @@ jobs: git_user_signingkey: true git_commit_gpgsign: true - - uses: dangoslen/dependabot-changelog-helper@v3 - with: - version: ${{ needs.setup.outputs.version }} - activationLabels: 'dependencies' - changelogPath: './CHANGELOG.md' - # This step is required for committing the changes to your branch. # See https://github.com/stefanzweifel/git-auto-commit-action#commits-made-by-this-action-do-not-trigger-new-workflow-runs - uses: stefanzweifel/git-auto-commit-action@v5 diff --git a/.github/workflows/lint.yml b/.github/workflows/lint.yml index aad066fb0..3cc12a8e0 100644 --- a/.github/workflows/lint.yml +++ b/.github/workflows/lint.yml @@ -25,6 +25,8 @@ jobs: go.sum .github/workflows/lint.yml scripts/no-now-lint.sh + .changelog/lint-unreleased.sh + .changelog/unreleased/** - uses: actions/setup-go@v5 if: env.GIT_DIFF with: @@ -39,3 +41,6 @@ jobs: - name: No Now Usage if: env.GIT_DIFF run: scripts/no-now-lint.sh + - name: Unreleased Changelog Content + if: env.GIT_DIFF + run: .changelog/lint-unreleased.sh diff --git a/CHANGELOG.md b/CHANGELOG.md index 3d1a801de..56ede08f0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,21 +7,30 @@ The same types of changes should be grouped. Versions and sections should be linkable. The latest version comes first. The release date of each version is displayed. -Mention whether you follow Semantic Versioning. Usage: -Change log entries are to be added to the Unreleased section under the -appropriate stanza (see below). Each entry should ideally include a message and either -an issue number or pull request number using one of these formats: +Change log entries should be added using the unclog application (see: https://github.com/informalsystems/unclog) +to create entry files under the `.changelog/unreleased` directory (in a section/stanza dir). +Example of adding an entry: +$ unclog add --issue-no 123 --section bug-fixes --id fix-the-thing --message 'Fix the thing that was broken' +That will create the file .changelog/unreleased/bug-fixes/123-fix-the-thing.md with this content: +`* Fix the thing that was broken [#123](https://github.com/provenance-io/provenance/issues/123).` +If there is no issue to link to, use the --pull-request flag instead of --issue-no. -* message #<issue-number> +If your changes involve updates to go.mod, you should use the `get-dep-changes.sh` +script to generate the depenencies changelog entry, e.g.: +$ scripts/get-dep-changes.sh --issue-no 123 --id bump-the-stuff +That will create the file .changelog/unreleased/dependencies/123-bump-the-stuff with content +generated by analyzing the go.mod file changes. -If there is no issue number, you can add a reference to a Pull Request like this: -* message PR<pull-request-number> +The content in this CHANGELOG.md file takes precedence over the content of the .changelog directory (in the case of a discrepancy). -The issue numbers and pull request numbers will later be link-ified during the release process -so you do not have to worry about including a link manually, but you can if you wish. +Ultimately, each entry should ideally include a message and a link to either +an issue number or pull request number: +* message #<issue-number> +or +* message PR<pull-request-number> Types of changes (Stanzas): @@ -34,11 +43,15 @@ Types of changes (Stanzas): "State Machine Breaking" for any changes that result in a different AppState given same genesisState and txList. "Dependencies" for changes to library versions. Ref: https://keepachangelog.com/en/1.0.0/ + +Valid --section values are the lower, kebab-case version of the stanza strings, e.g. state-machine-breaking. +You can get an exact list of valid --section values by using the `.changelog/get-valid-sections.sh` script, +or using the `make get-valid-sections` target. --> -## [Unreleased] +## Unreleased -* nothing +See: [.changelog/unreleased](.changelog/unreleased) --- @@ -1659,4 +1672,4 @@ into new 0.40.x base. Minimal unit test coverage and features in place to begin The Provenance Blockchain was started by Figure Technologies in 2018 using a Hyperledger Fabric derived private network. A subsequent migration was made to a new internal private network based on the 0.38-0.39 series of Cosmos SDK and -Tendermint. The Provence-IO/Provenance Cosmos SDK derived public network is the +Tendermint. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index d6cd4774a..eab342fce 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -46,7 +46,7 @@ contributors, the general procedure for contributing has been established: 3. If nobody has been assigned for the issue and you would like to work on it, make a comment on the issue to inform the community of your intentions to begin work - 4. Follow standard Github best practices: fork the repo, branch from the + 4. Follow standard GitHub best practices: fork the repo, branch from the HEAD of `main`, make some commits, and submit a PR to `main` - For core developers working on `provenance-io`, to ensure a clear ownership of branches, branches must be named with the convention @@ -79,7 +79,7 @@ To accommodate review process we suggest that PRs are categorically broken up. Ideally each PR addresses only a single issue. Additionally, as much as possible code refactoring and cleanup should be submitted as a separate PRs from bugfixes/feature-additions. -Draft PRs can be used for preliminary feedback and to see the results of the Github action checks. +Draft PRs can be used for preliminary feedback and to see the results of the GitHub action checks. They can also be used to better indicate that you are working on an issue. ### PR Requirements @@ -88,15 +88,16 @@ Before a PR can be merged: - All commits must be signed. - It must be up-to-date with `main`. - It must be approved by two or more maintainers. -- It must must pass all required Github action checks. +- It must pass all required GitHub action checks. The following are encouraged and may sometimes be required: -- All Github action checks pass (even the non-required ones). +- All GitHub action checks pass (even the non-required ones). - New Unit and/or integration tests have been written. - Documentation has been updated (in `/docs` or `x/<module>/spec`). - Functions and variables have accurate `godoc` comments. - Test code coverage increases. - Running `go mod tidy` should not cause `go.mod` or `go.sum` to change. +- There should be at least one entry in the changelog. See: [.changelog/README.md](.changelog/README.md). ### Process for reviewing PRs @@ -219,7 +220,7 @@ Branch protection might not be set up in all repos, but those branches should al - Using `--force` onto a protected branch is not allowed (except when reverting a broken commit, which should seldom happen). - Protected branches must not fail `make test test-race`. - Protected branches must not fail `make lint`. -- Protected branches should not fail any Github action checks. +- Protected branches should not fail any GitHub action checks. ### Development Branch naming @@ -258,7 +259,7 @@ Definitions: - The Provenance Blockchain network used for testing and integration is "`testnet`". Git tags should only be used for releases. -A release is automatically created by Github when a tag is pushed that has the format `v#.#.#` (where `#` is a whole number of any length). +A release is automatically created by GitHub when a tag is pushed that has the format `v#.#.#` (where `#` is a whole number of any length). A release candidate is created if the tag has the format `v#.#.#-rc#`. As of `v1.13.0`, release tags are created on the `.x` branches. E.g. on `release/v1.13.x`. @@ -301,7 +302,7 @@ If a `.x` branch does not yet exist for the desired minor version, one must be c 1. Start on `main` and make sure you're up-to-date, e.g. `git checkout main && git pull`. 2. Create the new `.x` branch, e.g. `git checkout -b release/v1.13.x`. -3. Push it to Github, e.g. `git push`. +3. Push it to GitHub, e.g. `git push`. #### 2. Update Changelog and Release Notes @@ -309,36 +310,11 @@ You will need to create a new development branch for this and PR it back to the The `CHANGELOG.md` on the `.x` branch must be updated to reflect the new release. -1. Run `make linkify`. -2. Add a horizontal rule and version section heading, e.g. - ```plaintext - --- - - ## [v1.13.0](https://github.com/provenance-io/provenance/releases/tag/v1.13.0) - 2022-10-04 - ``` - This usually goes immediately under the `## Unreleased` heading to indicate that all unreleased things are now released. - There should be an empty line both above the `---` and below the new version header. -3. If going from a release candidate to a full release, the release candidate entries should all be combined into one entry for the full release. -4. Optionally add an extra paragraph or two with general new version information. - This should go below the newly added version heading but above any subheadings (e.g. `### Improvements`). -5. Add a `### Full Commit History` section at the end of the new version section with links to diffs between versions. E.g. - ```plaintext - ### Full Commit History - - * https://github.com/provenance-io/provenance/compare/v1.12.0...v1.13.0 - ``` - Note that the three dot `...` diff is preferred over the two dot `..` one for these links. - For release candidates `2` and above, include links from both the previously released version and the previous release candidate. - This should be the last section before the `---` above the next version entry. - -Now, create/update the `RELEASE_CHANGELOG.md`. -For release candidates above `2`, the existing `RELEASE_CHANGELOG.md` should be updated to include info about the new version at the top. -For full or `-rc1` releases, delete any existing `RELEASE_CHANGELOG.md` and start a new empty one. - -1. Copy the lines from `CHANGELOG.md` starting with the new version header and ending on the blank line before the hr above the next version entry. -2. Paste them into the `RELEASE_CHANGELOG.md`. - -Push up your changes and PR them to the `.x` branch. +1. Run `.changelog/prep-release.sh <version>` to create/update `RELEASE_NOTES.md`, update `CHANGELOG.md`, and move things around in the `.changelog/` folder. +2. Review the changes with extra attention on the new content of `CHANGELOG.md` and `RELEASE_NOTES.md`. +3. Stage and commit the changes. +4. Push up your branch and create a PR for it to the `.x` branch. The PR title should be like `Mark v1.13.0`. +5. Get the PR approved and merged. #### 3. Create the New Version Tag @@ -351,21 +327,24 @@ Do the following locally. 5. Push the branch. E.g. `git push`. 6. Push the tag. E.g. `git push origin v1.13.0`. -You can then monitor the Github actions for the repo and also watch for the new release page to be created. +You can then monitor the GitHub actions for the repo and also watch for the new release page to be created. #### 4. PR the .x Branch Back to Main -This PR should update the `CHANGELOG.md` and contain any changes applied to the `.x` branch but not yet in `main`. +This PR should update the `CHANGELOG.md` and contain any changes applied to the `.x` branch that are not yet in `main`. It should NOT contain the `RELEASE_CHANGELOG.md` file. +It also should NOT have the new version directory under `.changelog/` (e.g. `.changelog/v1.13.0`), but it SHOULD remove applicable entries from `.changelog/unreleased`. Do the following locally. 1. Navigate to your locally cloned repo. -2. Check out the `main` branch and make sure it's up-to-date. E.g. `git checkout main && git pull`. -3. Check out the `.x` branch and make sure it's up-to-date. E.g. `git checkout release/v1.13.x && git pull`. +2. Check out the `.x` branch and make sure it's up-to-date. E.g. `git checkout release/v1.13.x && git pull`. +3. Check out the `main` branch and make sure it's up-to-date. E.g. `git checkout main && git pull`. 4. Create a new development branch. E.g. `git checkout -b myuser/v1.13.0-back-to-main`. -5. Remove the `RELEASE_CHANGELOG.md` file. -6. Update your branch with `main`. E.g. `git merge main`. -7. Make sure the `CHANGELOG.md` correctly indicates the contents of the new release and still contains any unreleased entries. -8. Address any other conflicts that might exist. -9. Create a PR from your branch targeting `main`. +5. Identify the commit that updated the changelog on the .x branch. E.g. `git log -5 --oneline release/v1.13.x`, then copy the commit hash. +6. Cherry-pick (without committing) that commit to your development branch. E.g. `git cherry-pick 5f57e13 --no-commit`. +7. Remove the `RELEASE_CHANGELOG.md` file. +8. Remove the new version directory under `.changelog`. +9. Stage the removals and finish the cherry-pick to commit the changes. +10. Identify any other changes made to the `.x` branch that were not made to `main` (and should be); cherry-pick and commit them. +11. Create a PR from your branch targeting `main`. The PR title should be like `Mark v1.13.0`. diff --git a/Makefile b/Makefile index 18999e7f0..5ac17ddc0 100644 --- a/Makefile +++ b/Makefile @@ -277,6 +277,7 @@ lint: find . -name '*.go' -type f -not -path "./vendor*" -not -path "./client/*" -not -path "*.git*" -not -path "*.pb.go" | xargs gofmt -d -s scripts/no-now-lint.sh $(GO) mod verify + .changelog/lint-unreleased.sh lint-fix: $(GOLANGCI_LINT) run --fix @@ -301,9 +302,12 @@ linkify: python ./scripts/linkify.py CHANGELOG.md update-tocs: - scripts/update-toc.sh x docs CONTRIBUTING.md + scripts/update-toc.sh x docs CONTRIBUTING.md .changelog/README.md -.PHONY: go-mod-cache go.sum lint clean format check-built linkify update-tocs +get-valid-sections: + .changelog/get-valid-sections.sh + +.PHONY: go-mod-cache go.sum lint clean format check-built linkify update-tocs get-valid-sections validate-go-version: ## Validates the installed version of go against Provenance's minimum requirement. diff --git a/scripts/update-toc.sh b/scripts/update-toc.sh index c195b9a11..d93a3dda3 100755 --- a/scripts/update-toc.sh +++ b/scripts/update-toc.sh @@ -137,7 +137,7 @@ update_toc () { has_toc_loc="$( grep -q "$TOC_LOC_REGEX" "$filename" && printf 'YES' )" has_heading_two="$( grep -q '^##' "$filename" && printf 'YES' )" - tempfile="$( mktemp -t "$( sed 's/\//-/g' <<< 'x/metadata/spec/03_messages.md' )" )" + tempfile="$( mktemp -t "$( sed 's/\//-/g' <<< "$filename" )" )" # if there's no pre-defined TOC location, and no level 2 heading, put the TOC at the top. if [[ -z "$has_toc_loc" && -z "$has_heading_two" ]]; then