Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: enable hermetic library generation #2234

Merged
merged 35 commits into from
Aug 19, 2024

Conversation

diegomarquezp
Copy link
Contributor

@diegomarquezp diegomarquezp commented May 15, 2024

This is part of the effort to enable multi-version support in hand-written libraries. More context and details in this doc.

This PR adds two workflows and their scripts:

  • update_generation_config which updates generation_config.yaml (added in this PR) to contain the latest googleapis_committish (source of truth of library generation) as well as other dependencies such as gapic_generator_version. This job runs nightly and creates a PR that modifies the config yaml.
  • hermetic_library_generation which reads generation_config.yaml and pushes a new commit to any PR whose latest commit affects generation_config.yaml (e.g. the nightly PR created by update_generation_config). It essentially translates changes in the config yaml to changes in the source code (of the generated part). This workflow will be a required check.

As an example of how they work in practice:

  1. Once per night, update_generation_config creates a PR that updates generation_config.yaml with the latest dependencies and googleapis committish
  2. hermetic_library_generation is triggered by this PR and generates the corresponding code changes. A new commit with the code changes is added to the PR. This step may occur for any PR affecting generation_config.yaml.

Follow ups

  • Add CLOUD_JAVA_BOT_TOKEN to repository for the hermetic generation workflow to work

@product-auto-label product-auto-label bot added size: l Pull request size is large. api: bigtable Issues related to the googleapis/java-bigtable API. labels May 15, 2024
@diegomarquezp diegomarquezp marked this pull request as ready for review May 23, 2024 20:44
@diegomarquezp diegomarquezp requested review from a team as code owners May 23, 2024 20:44
@blakeli0 blakeli0 added the do not merge Indicates a pull request not ready for merge, due to either quality or timing. label May 24, 2024
@@ -0,0 +1,37 @@
protoc_version: '25.3'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you add a comment here and in the existing owl bot config that these are mirrors of each other?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually protoc version is now baked into our docker image, @diegomarquezp can you please remove it to reflect the recent changes?

The protoc version used to come from the googleapis/WORKSPACE, which is shared by all languages. This is now configurable by Java.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

- name: Generate changed libraries
shell: bash
run: |
set -x
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this need a set -e?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added -e

- name: Update params in generation config to latest
shell: bash
run: |
set -x
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this need a set -e?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added -e

# 1. git
# 2. gh
# 3. jq

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you either add logging which commitish this processing? or at least set -x?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@diegomarquezp diegomarquezp added owlbot:run Add this label to trigger the Owlbot post processor. and removed do not merge Indicates a pull request not ready for merge, due to either quality or timing. labels Aug 19, 2024
@gcf-owl-bot gcf-owl-bot bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label Aug 19, 2024
@diegomarquezp diegomarquezp added the automerge Merge the pull request once unit tests and other checks pass. label Aug 19, 2024
@diegomarquezp diegomarquezp merged commit 169aea5 into googleapis:main Aug 19, 2024
22 checks passed
@gcf-merge-on-green gcf-merge-on-green bot removed the automerge Merge the pull request once unit tests and other checks pass. label Aug 19, 2024
gcf-merge-on-green bot pushed a commit that referenced this pull request Aug 23, 2024
🤖 I have created a release *beep* *boop*
---


## [2.43.0](https://togithub.com/googleapis/java-bigtable/compare/v2.42.0...v2.43.0) (2024-08-22)


### Features

* Add fields and the BackupType proto for Hot Backups ([#2300](https://togithub.com/googleapis/java-bigtable/issues/2300)) ([acaa3ff](https://togithub.com/googleapis/java-bigtable/commit/acaa3ff26ab0d317362e2be65ac5edcf803b13a1))
* Allow non default service account in DirectPath ([#2312](https://togithub.com/googleapis/java-bigtable/issues/2312)) ([09d0f23](https://togithub.com/googleapis/java-bigtable/commit/09d0f23032488dfa55c7a4c1c571a4f36bd94728))
* **bigtable:** Remove deprecated Bytes from BigEndianBytesEncoding ([#2309](https://togithub.com/googleapis/java-bigtable/issues/2309)) ([32f244f](https://togithub.com/googleapis/java-bigtable/commit/32f244f13d0c8571654d314310a4756fe275d609))
* Enable hermetic library generation ([#2234](https://togithub.com/googleapis/java-bigtable/issues/2234)) ([169aea5](https://togithub.com/googleapis/java-bigtable/commit/169aea5c43485a8d13ed53f57495609c142944df))


### Bug Fixes

* Add missing call to EqualsTester#testEquals ([#2307](https://togithub.com/googleapis/java-bigtable/issues/2307)) ([8b49f9c](https://togithub.com/googleapis/java-bigtable/commit/8b49f9ce84871f0f423f5837785604c3119ccd88))


### Dependencies

* Update shared dependencies ([#2314](https://togithub.com/googleapis/java-bigtable/issues/2314)) ([ab392ee](https://togithub.com/googleapis/java-bigtable/commit/ab392ee8d0c4535b5d3f31b3e111cbc41b399dd9))

---
This PR was generated with [Release Please](https://togithub.com/googleapis/release-please). See [documentation](https://togithub.com/googleapis/release-please#release-please).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigtable Issues related to the googleapis/java-bigtable API. size: l Pull request size is large.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants