From e3a91c16fe4b79ac568aa1110e1171b730b3142d Mon Sep 17 00:00:00 2001 From: smvgarcia <111767892+smvgarcia@users.noreply.github.com> Date: Tue, 5 Nov 2024 12:59:49 -0600 Subject: [PATCH 1/3] Update fence_usersync_job.md Fix broken links, make some hyperlinked text more visible --- docs/fence_usersync_job.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/fence_usersync_job.md b/docs/fence_usersync_job.md index 26b50492..b01f170f 100644 --- a/docs/fence_usersync_job.md +++ b/docs/fence_usersync_job.md @@ -13,14 +13,14 @@ User lists can be synced from three sources: # S3 user.yaml Setup {#s3-setup} -Please see [this](https://github.com/uc-cdis/fence/blob/master/docs/user.yaml_guide.md) documentation that details user.yaml formatting. +Please see [this](https://github.com/uc-cdis/fence/blob/master/docs/additional_documentation/user.yaml_guide.md) documentation that details user.yaml formatting. You can pull this file from an S3 bucket that is set in the `.Values.usersync.userYamlS3Path` field. Then input the IAM credentials for a user that has read access to the specified S3 bucket in the `.Values.secrets.awsAccessKeyId` and `.Values.secrets.awsSecretAccessKey` fields. You can utilize a local secret to avoid pasting credentials in the values.yaml file. Just set `.global.aws.useLocalSecret.enabled` to true and supply your secret name. ***Notice: -The Gen3 Helm chart has various jobs and uses for an IAM user. To enhance code reusability, we've implemented the option for jobs and services to share the same AWS IAM global user. If you would like to use the same IAM user for Fence Usersync, External Secrets, etc.- you can follow [THIS](global_iam_helm_user.md) guide that details how to setup a Helm global user.*** +The Gen3 Helm chart has various jobs and uses for an IAM user. To enhance code reusability, we've implemented the option for jobs and services to share the same AWS IAM global user. If you would like to use the same IAM user for Fence Usersync, External Secrets, etc.- you can follow [THIS guide that details how to setup a Helm global user](global_iam_helm_user.md).*** As previously mentioned, if the `.Values.usersync.userYamlS3Path` string is set to "none", the user.yaml file from Fence values.yaml will be used. @@ -28,7 +28,7 @@ As previously mentioned, if the `.Values.usersync.userYamlS3Path` string is set # Dbgap ## Sftp Setup {#sftp-setup} -You can configure one or more dbGaP SFTP servers to sync telemetry files from. To configure one single dbGaP server, add credentials and information to the fence-config.yaml under dbGaP, this is outlined [here](https://github.com/uc-cdis/gen3-helm/blob/c7b8959cdf5f7756b29c33ff330923e95981827c/helm/fence/values.yaml#L1796). +You can configure one or more dbGaP SFTP servers to sync telemetry files from. To configure one single dbGaP server, add credentials and information to the fence-config.yaml under dbGaP, [this is outlined here](https://github.com/uc-cdis/gen3-helm/blob/c7b8959cdf5f7756b29c33ff330923e95981827c/helm/fence/values.yaml#L1796). To configure additional dbGaP servers, include in the config.yaml a list of dbGaP servers under dbGaP, like so: @@ -48,9 +48,9 @@ dbGaP: ... ```` -You can find more detailed information on the setup with examples [here](https://github.com/uc-cdis/fence/blob/master/docs/usersync.md). +You can find more detailed information on the setup with examples [here](https://github.com/uc-cdis/fence/blob/master/docs/additional_documentation/usersync.md). -For an example of a dbGap auth file (csv), please see [this](https://github.com/uc-cdis/fence/blob/master/docs/usersync.md#example-of-dbgap-authorization-file-csv-format) example for formatting. +For an example of a dbGap auth file (csv), please see [this](https://github.com/uc-cdis/fence/blob/master/docs/additional_documentation/usersync.md#example-of-dbgap-authorization-file-csv-format) example for formatting. ## Dbgap Options Set `.Values.usersync.addDbgap` to "true" to attempt a dbgap sync and fall back on user.yaml. @@ -67,4 +67,4 @@ For an example of a dbGap auth file (csv), please see [this](https://github.com/ ## Other Customizations The `.Values.usersync.schedule` option can be set to customize the cron schedule expression. The default setting is to have the job run once every 30 minutes. - The `.Values.usersync.custom_image` can be set to override the default "awshelper" image for the init container of the userync cronjob. \ No newline at end of file + The `.Values.usersync.custom_image` can be set to override the default "awshelper" image for the init container of the userync cronjob. From 83333948a64a5028f85e94f04353f75777cd3c3b Mon Sep 17 00:00:00 2001 From: smvgarcia <111767892+smvgarcia@users.noreply.github.com> Date: Tue, 5 Nov 2024 13:21:31 -0600 Subject: [PATCH 2/3] Update CONFIGURATION.md update broken links, make some hyperlinked phrases more visible --- docs/CONFIGURATION.md | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/docs/CONFIGURATION.md b/docs/CONFIGURATION.md index 6ba633f8..5a8fe5a7 100644 --- a/docs/CONFIGURATION.md +++ b/docs/CONFIGURATION.md @@ -104,7 +104,7 @@ arborist: ## Extra Information -Common arborist database queries can be found [here](https://github.com/uc-cdis/cdis-wiki/blob/master/dev/gen3-sql-queries.md#arborist-database). +[Find common arborist database queries here](https://github.com/uc-cdis/cdis-wiki/blob/master/dev/gen3-sql-queries.md#arborist-database). --- @@ -160,12 +160,12 @@ You need to ensure a proper working fence-config file. Fence is highly configura 8. CIRRUS_CFG * If google buckets are used you need to configure this block. It is used to setup the google bucket workflow, which essentially creates google users and google bucket access groups, which get filled with users and added to bucket policies to allow implicit access to users. -For more infomation see [this](https://github.com/uc-cdis/fence/blob/master/fence/config-default.yaml) +For more infomation, [see this](https://github.com/uc-cdis/fence/blob/master/fence/config-default.yaml) -For user.yaml see this how to construct one properly. This will control access to your data commons: +A user.yaml will control access to your data commons. To see how to construct a user.yaml properly: -https://github.com/uc-cdis/fence/blob/master/docs/user.yaml_guide.md +https://github.com/uc-cdis/fence/blob/master/docs/additional_documentation/user.yaml_guide.md ## Extra Information @@ -175,7 +175,7 @@ Fence is split into 2 deployments. There is the regular fence deployment which h ### Troubleshooting Fence -There are some commons sql queries that can be found [here](https://github.com/uc-cdis/cdis-wiki/blob/master/dev/gen3-sql-queries.md#fence-database). +There are [some commons sql queries that can be found here](https://github.com/uc-cdis/cdis-wiki/blob/master/dev/gen3-sql-queries.md#fence-database). ### Setting up OIDC clients @@ -196,7 +196,7 @@ Guppy is used to render the explorer page. It uses elastic search indices to ren For a full set of configuration see the [helm README.md for guppy](../helm/guppy/README.md) or read the [values.yaml](../helm/guppy/values.yaml) directly -There is also config that needs to be set within the global block around the tier access level, defining how the explorer page should handle displaying unauthorized files, and the limit to how far unauthroized user can filter down files. Last there is a guppy block that needs to be configured with the elastic search indices guppy will use to render the explorer page. +There is also config that needs to be set within the global block around the tier access level, defining how the explorer page should handle displaying unauthorized files, and the limit to how far unauthorized user can filter down files. Last, there is a guppy block that needs to be configured with the elastic search indices guppy will use to render the explorer page. ``` global: @@ -230,7 +230,7 @@ guppy: ``` -You will also need a mapping file to map the fields you want to pull from postgres into the elasticsearch indices. There are too many fields to describe here, but an example mapping file can be found [here](https://github.com/uc-cdis/cdis-manifest/blob/master/gen3.biodatacatalyst.nhlbi.nih.gov/etlMapping.yaml). +You will also need a mapping file to map the fields you want to pull from postgres into the elasticsearch indices. There are too many fields to describe here, but [an example mapping file can be found here](https://github.com/uc-cdis/cdis-manifest/blob/master/gen3.biodatacatalyst.nhlbi.nih.gov/etlMapping.yaml). Last, guppy works closely with portal to render the explorer page. You will need to ensure a proper [dataExplorer block](https://github.com/uc-cdis/cdis-manifest/blob/master/gen3.biodatacatalyst.nhlbi.nih.gov/portal/gitops.json#L212) is setup within the gitops.json file, referencing fields that have been pulled from postgres into the elasticsearch indices. @@ -454,9 +454,9 @@ portal: ``` -To do this you can follow the example [here](https://github.com/uc-cdis/data-portal/blob/master/docs/portal_config.md). +To do this you can follow [the example here](https://github.com/uc-cdis/data-portal/blob/master/docs/portal_config.md). -Portal can also be configured with different images and icons by updating the values, similar to [this](https://github.com/uc-cdis/cdis-manifest/tree/master/gen3.biodatacatalyst.nhlbi.nih.gov/portal). +Portal can also be configured with different images and icons by updating the values, [similar to this](https://github.com/uc-cdis/cdis-manifest/tree/master/gen3.biodatacatalyst.nhlbi.nih.gov/portal). ## Extra Information @@ -605,4 +605,4 @@ sower: restart_policy: Never ``` -## Extra Information --> \ No newline at end of file +## Extra Information --> From b4c3920c3d477e95b174ca6d1730c505e07a0889 Mon Sep 17 00:00:00 2001 From: smvgarcia <111767892+smvgarcia@users.noreply.github.com> Date: Tue, 5 Nov 2024 14:00:30 -0600 Subject: [PATCH 3/3] Update CONFIGURATION.md update language in paragraph and fix incorrect link. --- docs/CONFIGURATION.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/CONFIGURATION.md b/docs/CONFIGURATION.md index 5a8fe5a7..3323c0e1 100644 --- a/docs/CONFIGURATION.md +++ b/docs/CONFIGURATION.md @@ -179,7 +179,9 @@ There are [some commons sql queries that can be found here](https://github.com/u ### Setting up OIDC clients -OIDC clients are used by applications to authenticate to fence. Many times this is external users to setup apps which leverage gen3 and an OIDC will have to be client will need to be setup for them. After creation, the client_id/secret will need to be shared with the application owner. To create these clients you will need to exec into a fence container and run the [following commands](https://github.com/uc-cdis/fence#register-oauth-client). +OIDC (OpenID Connect) clients allow applications to authenticate with Fence. This setup is often necessary for external users who want to integrate their applications with Gen3. For each application, you'll need to create a unique OIDC client, which will provide a client_id and client_secret for the application to use. + +Once the client is created, share the client_id and client_secret with the application owner so they can configure their application to authenticate with Fence. To create these clients, you will need to exec into a fence container and run the [following commands](https://github.com/uc-cdis/fence/blob/master/docs/additional_documentation/setup.md#register-oauth-client). ---