Skip to content

Commit

Permalink
Merge branch 'main' into feature/#52-Added_timeout_options_for_SLC_de…
Browse files Browse the repository at this point in the history
…ployer_to_CLI
  • Loading branch information
ckunki committed Oct 7, 2024
2 parents f05df2e + 886bb11 commit 6bcc7c5
Show file tree
Hide file tree
Showing 24 changed files with 1,409 additions and 137 deletions.
4 changes: 4 additions & 0 deletions doc/changes/changelog.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
# 📝 Changes

* [unreleased](unreleased.md)
* [0.7.0](changes_0.7.0.md)
* [0.6.0](changes_0.6.0.md)
* [0.5.0](changes_0.5.0.md)
* [0.4.0](changes_0.4.0.md)
* [0.3.1](changes_0.3.1.md)
Expand All @@ -13,6 +15,8 @@
hidden:
---
unreleased
changes_0.7.0
changes_0.6.0
changes_0.5.0
changes_0.4.0
changes_0.3.1
Expand Down
7 changes: 7 additions & 0 deletions doc/changes/changes_0.6.0.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# 0.6.0 - 2024-09-23

This release allows using the `extract_validator` for SLCs, even when lacking the permissions to create a database schema.

## Features

* #41: Make `temp_schema optional` for `wait_for_completion`.
13 changes: 13 additions & 0 deletions doc/changes/changes_0.7.0.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# 0.7.0 - 2024-10-07

## Features

* #66: Implemented a standard CLI options builder.
* #70: Implemented a CLI callback function for the language container deployment.
* #69: Added create_bucketfs_location function.
* #75: Added create_bucketfs_location_from_conn_object function.
* #74: Implemented a CLI callback function for creating a bucket-fs connection object.

# Refactoring

* #68: Made open_pyexasol_connection(...) using kwargs derived from the cli options.
67 changes: 34 additions & 33 deletions doc/user_guide/user-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,10 @@ similar to the one below.
python -m <exasol_extension>.deploy language-container <options>
```

The name of the script (```<exasol_extension>.deploy``` in the above command) can vary from one extension to another.
Please check the user guide of a particular extension. The rest of the command line will have a common format. It
will include the command - ```language-container``` - and selected options. The choice of options is primarily
determined by the storage backend being used - On-Prem or SaaS.
The name of the script (```<exasol_extension>.deploy``` in the above command) and the command name
(e.g. ```language-container```) can vary from one extension to another. Please check the user guide of a particular
extension. The rest of the command line will have a common format. It will include some of the options defined below.
The choice of options is primarily determined by the storage backend being used - On-Prem or SaaS.

### List of options

Expand All @@ -32,35 +32,36 @@ an option in the command line, without providing its value. In this case, the co
interactively. For long values, such as the SaaS account id, it is more practical to copy/paste the value from
another source.

| Option name | On-Prem | SaaS | Comment |
|:-----------------------------|:-------:|:----:|:--------------------------------------------------|
| dsn | [x] | | i.e. <db_host:db_port> |
| db-user | [x] | | |
| db-pass | [x] | | Env. [DB_PASSWORD] |
| bucketfs-name | [x] | | |
| bucketfs-host | [x] | | |
| bucketfs-port | [x] | | |
| bucketfs-user | [x] | | |
| bucketfs-password | [x] | | Env. [BUCKETFS_PASSWORD] |
| bucketfs-use-https | [x] | | Optional boolean, defaults to False |
| bucket | [x] | | |
| saas-url | | [x] | Env. [SAAS_HOST] |
| saas-account-id | | [x] | Env. [SAAS_ACCOUNT_ID] |
| saas-database-id | | [x] | Optional, Env. [SAAS_DATABASE_ID] |
| saas-database-name | | [x] | Optional, provide if the database_id is unknown |
| saas-token | | [x] | Env. [SAAS_TOKEN] |
| path-in-bucket | [x] | [x] | |
| language-alias | [x] | [x] | |
| version | [x] | [x] | Optional, provide for downloading SLC from GitHub |
| container-file | [x] | [x] | Optional, provide for uploading SLC file |
| ssl-cert-path | [x] | [x] | Optional |
| [no_]use-ssl-cert-validation | [x] | [x] | Optional boolean, defaults to True |
| ssl-client-cert-path | [x] | | Optional |
| ssl-client-private-key | [x] | | Optional |
| [no_]upload-container | [x] | [x] | Optional boolean, defaults to True |
| [no_]alter-system | [x] | [x] | Optional boolean, defaults to True |
| [dis]allow-override | [x] | [x] | Optional boolean, defaults to False |
| [no_]wait_for_completion | [x] | [x] | Optional boolean, defaults to True |
| Option name | On-Prem | SaaS | Comment |
|:-----------------------------|:-------:|:----:|:--------------------------------------------------------|
| dsn | [x] | | i.e. <db_host:db_port> |
| db-user | [x] | | |
| db-pass | [x] | | Env. [DB_PASSWORD] |
| bucketfs-name | [x] | | |
| bucketfs-host | [x] | | |
| bucketfs-port | [x] | | |
| bucketfs-user | [x] | | |
| bucketfs-password | [x] | | Env. [BUCKETFS_PASSWORD] |
| bucketfs-use-https | [x] | | Optional boolean, defaults to False |
| bucket | [x] | | |
| saas-url | | [x] | Env. [SAAS_HOST] |
| saas-account-id | | [x] | Env. [SAAS_ACCOUNT_ID] |
| saas-database-id | | [x] | Optional, Env. [SAAS_DATABASE_ID] |
| saas-database-name | | [x] | Optional, provide if the database_id is unknown |
| saas-token | | [x] | Env. [SAAS_TOKEN] |
| path-in-bucket | [x] | [x] | |
| language-alias | [x] | [x] | |
| schema | [x] | [x] | Required if the user has no permission to create a database schema |
| version | [x] | [x] | Optional, provide for downloading SLC from GitHub |
| container-file | [x] | [x] | Optional, provide for uploading SLC file |
| ssl-cert-path | [x] | [x] | Optional |
| [no-]use-ssl-cert-validation | [x] | [x] | Optional boolean, defaults to True |
| ssl-client-cert-path | [x] | | Optional |
| ssl-client-private-key | [x] | | Optional |
| [no-]upload-container | [x] | [x] | Optional boolean, defaults to True |
| [no-]alter-system | [x] | [x] | Optional boolean, defaults to True |
| [no-]allow-override | [x] | [x] | Optional boolean, defaults to False |
| [no-]wait_for_completion | [x] | [x] | Optional boolean, defaults to True |

### Container selection

Expand Down
10 changes: 10 additions & 0 deletions exasol/python_extension_common/cli/bucketfs_conn_object_cli.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
import exasol.python_extension_common.connections.bucketfs_location as bl


class BucketfsConnObjectCli:
def __init__(self, conn_name_arg: str):
self._conn_name_arg = conn_name_arg

def __call__(self, **kwargs):
conn_name = kwargs.pop(self._conn_name_arg)
bl.create_bucketfs_conn_object(conn_name=conn_name, **kwargs)
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
from pathlib import Path

from exasol.python_extension_common.deployment.language_container_deployer import LanguageContainerDeployer
from exasol.python_extension_common.connections.pyexasol_connection import open_pyexasol_connection
from exasol.python_extension_common.connections.bucketfs_location import create_bucketfs_location
from exasol.python_extension_common.cli.std_options import StdParams


class LanguageContainerDeployerCli:
"""
The class provides a CLI callback function that creates and runs the
LangaugeContainerDeployer.
At first glance, it may look a bit over-designed. The reason for wrapping a
callback function in a class is to let the user define the option names for the
container URL and container name. These options are not defined in the StdParams
but rather generated by formatters. The user can give them arbitrary names.
Hence, we don't want to assume any particular names in the callback function.
"""

def __init__(self,
container_url_arg: str | None = None,
container_name_arg: str | None = None) -> None:
self._container_url_arg = container_url_arg
self._container_name_arg = container_name_arg

def __call__(self, **kwargs):

pyexasol_connection = open_pyexasol_connection(**kwargs)
bucketfs_location = create_bucketfs_location(**kwargs)

language_alias = kwargs[StdParams.language_alias.name]
container_file = kwargs[StdParams.container_file.name]
upload_container = kwargs[StdParams.upload_container.name]
alter_system = kwargs[StdParams.alter_system.name]
allow_override = kwargs[StdParams.allow_override.name]
wait_for_completion = kwargs[StdParams.wait_for_completion.name]

deployer = LanguageContainerDeployer(pyexasol_connection,
language_alias,
bucketfs_location)
if not upload_container:
deployer.run(alter_system=alter_system,
allow_override=allow_override,
wait_for_completion=wait_for_completion)
elif container_file:
deployer.run(container_file=Path(container_file),
alter_system=alter_system,
allow_override=allow_override,
wait_for_completion=wait_for_completion)
elif kwargs.get(self._container_url_arg) and kwargs.get(self._container_name_arg):
deployer.download_and_run(kwargs[self._container_url_arg],
kwargs[self._container_name_arg],
alter_system=alter_system,
allow_override=allow_override,
wait_for_completion=wait_for_completion)
else:
raise ValueError("To upload a language container either its release version "
f"(--{StdParams.version.name}) or a path of the already "
f"downloaded container file (--{StdParams.container_file.name}) "
"must be provided.")
Loading

0 comments on commit 6bcc7c5

Please sign in to comment.