-
Notifications
You must be signed in to change notification settings - Fork 394
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Allow using empty strings as job parameters #3158
Conversation
31497c7
to
fd0a121
Compare
fd0a121
to
54a87b7
Compare
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #3158 +/- ##
==========================================
- Coverage 84.29% 83.14% -1.15%
==========================================
Files 162 168 +6
Lines 14405 14811 +406
==========================================
+ Hits 12143 12315 +172
- Misses 1565 1771 +206
- Partials 697 725 +28
|
@kanterov @gaborratky-db Isn't it valid to define a job where some job parameters are required? I.e. you have to specify the parameters when running it? If so, then this approach does not allow that use case. I believe this fix needs to use the |
@pietern, we don't really have 'required' job parameters in this case. But, the 'default' field in the API is required. If it's set to an empty string, it gets skipped in the API request because of the 'omitempty' thing, and that's causing a 400 error. So, we've got to make sure we include this 'default' field every time, even if it's just an empty string. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@kanterov Please add a unit test to confirm that empty strings are now specified. There are two cases to consider; a user explicitly passes an empty string (good), and a user doesn't specify the default at all, which would default to an empty string. In the latter case, we could see configuration drift on the TF side.
cfdcef2
to
01fbcb8
Compare
@@ -90,8 +90,22 @@ func TestAccJobTasks(t *testing.T) { | |||
|
|||
notebook_task { | |||
notebook_path = databricks_notebook.this.path | |||
base_parameters = { | |||
"param_0" = "{{job.parameters.empty_default}}" | |||
"param_0" = "{{job.parameters.non_empty_default}}" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@kanterov This is a map, so only one of these will end up on the backend.
Does that impact the validity of the integration test?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh... my bad, we need to fix it. It shouldn't matter because we don't run it, and I've tried empty value manually. But let's fix the test.
### New Features and Improvements * Exporter: timestamps are now added to log entries ([#3146](#3146)). * Validate metastore id for databricks_grant and databricks_grants resources ([#3159](#3159)). * Exporter: Skip emitting of clusters that come from more cluster sources ([#3161](#3161)). * Fix typo in docs ([#3166](#3166)). * Migrate cluster schema to use the go-sdk struct ([#3076](#3076)). * Introduce Generic Settings Resource ([#2997](#2997)). * Update actions/setup-go to v5 ([#3154](#3154)). * Change default branch from `master` to `main` ([#3174](#3174)). * Add .codegen.json configuration ([#3180](#3180)). * Exporter: performance improvements for big workspaces ([#3167](#3167)). * update ([#3192](#3192)). * Exporter: fix generation of cluster policy resources ([#3185](#3185)). * Fix unit test ([#3201](#3201)). * Suppress diff should apply to new fields added in the same chained call to CustomizableSchema ([#3200](#3200)). * Various documentation updates ([#3198](#3198)). * Use common.Resource consistently throughout the provider ([#3193](#3193)). * Extending customizable schema with `AtLeastOneOf`, `ExactlyOneOf`, `RequiredWith` ([#3182](#3182)). * Fix `databricks_connection` regression when creating without owner ([#3186](#3186)). * add test code for job task order ([#3183](#3183)). * Allow using empty strings as job parameters ([#3158](#3158)). * Fix notebook parameters in acceptance test ([#3205](#3205)). * Exporter: Add retries for `Search`, `ReadContext` and `Import` operations when importing the resource ([#3202](#3202)). * Fixed updating owners for UC resources ([#3189](#3189)). * Adds `databricks_volumes` as data source ([#3150](#3150)). ### Documentation Changes ### Exporter ### Internal Changes
* Release v1.35.1 ### New Features and Improvements * Exporter: timestamps are now added to log entries ([#3146](#3146)). * Validate metastore id for databricks_grant and databricks_grants resources ([#3159](#3159)). * Exporter: Skip emitting of clusters that come from more cluster sources ([#3161](#3161)). * Fix typo in docs ([#3166](#3166)). * Migrate cluster schema to use the go-sdk struct ([#3076](#3076)). * Introduce Generic Settings Resource ([#2997](#2997)). * Update actions/setup-go to v5 ([#3154](#3154)). * Change default branch from `master` to `main` ([#3174](#3174)). * Add .codegen.json configuration ([#3180](#3180)). * Exporter: performance improvements for big workspaces ([#3167](#3167)). * update ([#3192](#3192)). * Exporter: fix generation of cluster policy resources ([#3185](#3185)). * Fix unit test ([#3201](#3201)). * Suppress diff should apply to new fields added in the same chained call to CustomizableSchema ([#3200](#3200)). * Various documentation updates ([#3198](#3198)). * Use common.Resource consistently throughout the provider ([#3193](#3193)). * Extending customizable schema with `AtLeastOneOf`, `ExactlyOneOf`, `RequiredWith` ([#3182](#3182)). * Fix `databricks_connection` regression when creating without owner ([#3186](#3186)). * add test code for job task order ([#3183](#3183)). * Allow using empty strings as job parameters ([#3158](#3158)). * Fix notebook parameters in acceptance test ([#3205](#3205)). * Exporter: Add retries for `Search`, `ReadContext` and `Import` operations when importing the resource ([#3202](#3202)). * Fixed updating owners for UC resources ([#3189](#3189)). * Adds `databricks_volumes` as data source ([#3150](#3150)). ### Documentation Changes ### Exporter ### Internal Changes * upd * readable * upd * upd
Changes
Allow using an empty string as the "default" value for job parameters, otherwise, such jobs are rejected because "default" field is required.
Remove "omitempty" modifier because both fields are required.
Tests
Verified manually with Terraform
make test
run locallydocs/
folderinternal/acceptance