diff --git a/RELEASING.md b/RELEASING.md index 3f29819a3..5d83259bc 100644 --- a/RELEASING.md +++ b/RELEASING.md @@ -45,3 +45,62 @@ git push 4. Run the release script `./release.sh` from the repo root. This will use `tox` and your pypi credentials and will create a signed release tag. At the end of this step, new packages will be published to pypi and the tag to GitHub. + + +### DEB/RPM Packaging Workflow + +#### Pre-requisites + +Before building the packages, ensure that the release itself, either the alpha +or prod versions, is published on PyPI. + +Additionally, VPN needs to be enabled for the build page. + +#### Build Process + +In the future, building the DEB/RPM packages will be a simple one-step button +click of the green **Build** button on the Globus Compute Agent +[build page here](https://builds.globus.org/jenkins/job/BuildGlobusComputeAgentPackages/build?delay=0sec). + +As a temporary workaround, we need to add a few lines to manually set some +env variables in our [JenkinsFile](https://github.com/globus/globus-compute/blob/743fa1e398fd40a00efb5880c55e3fa6e47392fc/compute_endpoint/packaging/JenkinsFile#L24) before triggering the build, as detailed below. + +1. Git checkout both the current release branch that was recently pushed to + PyPI, ie. ``v2.23.0`` or ``v2.25.0a0`` and the ``build_for_stable`` branch +2. Rebase ``build_for_stable`` on the release branch which should result in + adding the following ~6 lines: + + ... + env.BRANCH_NAME = scmVars.GIT_BRANCH.replaceFirst(/^.*origin\//, "") + + env.TAG_NAME = sh(returnStdout: true, script: "git tag --contains | head -1").trim() + env.SOURCE_STASH_NAME = "${UUID.randomUUID()}" + echo "env.BRANCH_NAME = ${env.BRANCH_NAME}" + sh "git clean -fdx" + + + // temporary hack to build for stable + + sh "git checkout build_for_stable" + + env.TAG_NAME = "v2.23.0" + + env.DEFAULT_BRANCH = "build_for_stable" + + + dir("compute_endpoint/packaging/") { + ... + +3. Change the ``env.TAG_NAME`` above to the current production release version + * Note that ``env.TAG_NAME`` determines whether the build is sent to + the ``unstable`` repo or also to the ``testing`` and ``stable`` ones. + * Example of unstable repo: + * https://downloads.globus.org/globus-connect-server/unstable/rpm/el/9/x86_64/ + * Example of stable repo: + * https://downloads.globus.org/globus-connect-server/stable/rpm/el/9/x86_64/ + * The logic of whether a release is stable is determined by whether the + package version of Globus Compute Endpoint set in ``version.py`` or + ``setup.py`` matches ``env.TAG_NAME`` above. If they are unequal, then + [publishResults.groovy line 85](https://github.com/globusonline/gcs-build-scripts/blob/168617a0ccbb0aee7b3bee04ee67940bbe2a80f6/vars/publishResults.groovy#L85) + will be (``tag`` : v2.23.0) != (``stable_tag`` : v2.23.0a0), where + stable_tag is constructed from the package version of an alpha release. +4. Commit and push your ``build_for_stable`` branch +5. (Access on VPN) Click the [build button here](https://builds.globus.org/jenkins/job/BuildGlobusComputeAgentPackages/build?delay=0sec) +6. Wait 20-30 minutes and confirm that the [build is green](https://builds.globus.org/jenkins/job/BuildGlobusComputeAgentPackages/) +7. For production release, we will have finished the build before the GCS + team, and can notify them that our build is complete. They then will + publish all packages when they finish their builds, which includes ours. diff --git a/changelog.d/20240410_130138_LeiGlobus_delete_remote_endpoint_sc_31206.rst b/changelog.d/20240410_130138_LeiGlobus_delete_remote_endpoint_sc_31206.rst deleted file mode 100644 index 465373e9b..000000000 --- a/changelog.d/20240410_130138_LeiGlobus_delete_remote_endpoint_sc_31206.rst +++ /dev/null @@ -1,7 +0,0 @@ -.. New Functionality -.. ^^^^^^^^^^^^^^^^^ -.. -- The ``delete`` command can now delete endpoints by name or UUID from the - Compute service remotely when local config files are not available. Note - that without the ``--force`` option the command may exit early if the - endpoint is currently running or local config files are corrupted. diff --git a/changelog.d/20240617_133114_30907815+rjmello_fix_gcengine_channel_check.rst b/changelog.d/20240617_133114_30907815+rjmello_fix_gcengine_channel_check.rst deleted file mode 100644 index 57689e5cc..000000000 --- a/changelog.d/20240617_133114_30907815+rjmello_fix_gcengine_channel_check.rst +++ /dev/null @@ -1,5 +0,0 @@ -Bug Fixes -^^^^^^^^^ - -- We no longer raise an exception when using the ``GlobusComputeEngine`` with Parsl - providers that do not utilize ``Channel`` objects (e.g., ``KubernetesProvider``). \ No newline at end of file diff --git a/changelog.d/20240622_114437_30907815+rjmello_add_bin_paths_to_self_diag.rst b/changelog.d/20240622_114437_30907815+rjmello_add_bin_paths_to_self_diag.rst deleted file mode 100644 index 892f340c7..000000000 --- a/changelog.d/20240622_114437_30907815+rjmello_add_bin_paths_to_self_diag.rst +++ /dev/null @@ -1,5 +0,0 @@ -New Functionality -^^^^^^^^^^^^^^^^^ - -- Included the paths to the ``globus-compute-endpoint`` and ``process_worker_pool.py`` - executables in the ``self-diagnostic`` command output. diff --git a/changelog.d/20240628_090506_30907815+rjmello_bump_parsl_2024_06_10.rst b/changelog.d/20240628_090506_30907815+rjmello_bump_parsl_2024_06_10.rst deleted file mode 100644 index 6beb6713e..000000000 --- a/changelog.d/20240628_090506_30907815+rjmello_bump_parsl_2024_06_10.rst +++ /dev/null @@ -1,4 +0,0 @@ -Changed -^^^^^^^ - -- Bumped ``parsl`` dependency version to 2024.6.10. diff --git a/changelog.d/20240702_104011_yadudoc1729_gce_run_tasks_in_working_dir.rst b/changelog.d/20240702_104011_yadudoc1729_gce_run_tasks_in_working_dir.rst deleted file mode 100644 index 3c72ff079..000000000 --- a/changelog.d/20240702_104011_yadudoc1729_gce_run_tasks_in_working_dir.rst +++ /dev/null @@ -1,7 +0,0 @@ -Changed -^^^^^^^ - -- ``GlobusComputeEngine.working_dir`` now defaults to ``tasks_working_dir`` - * When ``working_dir=relative_path``, tasks run in a path relative to the endpoint.run_dir. - The default is ``tasks_working_dir`` set relative to endpoint.run_dir. - * When ``working_dir=absolute_path``, tasks run in the specified absolute path \ No newline at end of file diff --git a/compute_endpoint/globus_compute_endpoint/version.py b/compute_endpoint/globus_compute_endpoint/version.py index e9da77694..84d854ec3 100644 --- a/compute_endpoint/globus_compute_endpoint/version.py +++ b/compute_endpoint/globus_compute_endpoint/version.py @@ -1,6 +1,6 @@ # single source of truth for package version, # see https://packaging.python.org/en/latest/single_source_version/ -__version__ = "2.22.0" +__version__ = "2.23.0" # TODO: remove after a `globus-compute-sdk` release # this is needed because it's imported by `globus-compute-sdk` to do the version check diff --git a/compute_endpoint/packaging/JenkinsFile b/compute_endpoint/packaging/JenkinsFile index d46f598d4..8b828856b 100644 --- a/compute_endpoint/packaging/JenkinsFile +++ b/compute_endpoint/packaging/JenkinsFile @@ -22,10 +22,16 @@ pipeline { script { def scmVars = checkout scm env.BRANCH_NAME = scmVars.GIT_BRANCH.replaceFirst(/^.*origin\//, "") + env.TAG_NAME = sh(returnStdout: true, script: "git tag --contains | head -1").trim() env.SOURCE_STASH_NAME = "${UUID.randomUUID()}" echo "env.BRANCH_NAME = ${env.BRANCH_NAME}" sh "git clean -fdx" + // temporary hack to build for stable + sh "git checkout build_for_stable" + env.TAG_NAME = "v2.23.0" + env.DEFAULT_BRANCH = "build_for_stable" + dir("compute_endpoint/packaging/") { sh(script: "make show_vars") sh(script: "make setup_dist_for_deb setup_dist_for_rpm") diff --git a/compute_endpoint/setup.py b/compute_endpoint/setup.py index 4df605289..9a3dccb65 100644 --- a/compute_endpoint/setup.py +++ b/compute_endpoint/setup.py @@ -6,7 +6,7 @@ REQUIRES = [ "requests>=2.31.0,<3", "globus-sdk", # version will be bounded by `globus-compute-sdk` - "globus-compute-sdk==2.22.0", + "globus-compute-sdk==2.23.0", "globus-compute-common==0.4.1", "globus-identity-mapping==0.3.0", # table printing used in list-endpoints diff --git a/compute_sdk/globus_compute_sdk/version.py b/compute_sdk/globus_compute_sdk/version.py index 3cd0e6d4e..ca2c727f5 100644 --- a/compute_sdk/globus_compute_sdk/version.py +++ b/compute_sdk/globus_compute_sdk/version.py @@ -3,7 +3,7 @@ # single source of truth for package version, # see https://packaging.python.org/en/latest/single_source_version/ -__version__ = "2.22.0" +__version__ = "2.23.0" def compare_versions( diff --git a/docs/changelog.rst b/docs/changelog.rst index 9fa157b11..2fabfd596 100644 --- a/docs/changelog.rst +++ b/docs/changelog.rst @@ -3,6 +3,38 @@ Changelog .. scriv-insert-here +.. _changelog-2.23.0: + +globus-compute-sdk & globus-compute-endpoint v2.23.0 +------------------------------------------------------ + +- The ``delete`` command can now delete endpoints by name or UUID from the + Compute service remotely when local config files are not available. Note + that without the ``--force`` option the command may exit early if the + endpoint is currently running or local config files are corrupted. + +New Functionality +^^^^^^^^^^^^^^^^^ + +- Included the paths to the ``globus-compute-endpoint`` and ``process_worker_pool.py`` + executables in the ``self-diagnostic`` command output. + +Bug Fixes +^^^^^^^^^ + +- We no longer raise an exception when using the ``GlobusComputeEngine`` with Parsl + providers that do not utilize ``Channel`` objects (e.g., ``KubernetesProvider``). + +Changed +^^^^^^^ + +- Bumped ``parsl`` dependency version to 2024.6.10. + +- ``GlobusComputeEngine.working_dir`` now defaults to ``tasks_working_dir`` + * When ``working_dir=relative_path``, tasks run in a path relative to the endpoint.run_dir. + The default is ``tasks_working_dir`` set relative to endpoint.run_dir. + * When ``working_dir=absolute_path``, tasks run in the specified absolute path + .. _changelog-2.22.0: globus-compute-sdk & globus-compute-endpoint v2.22.0