Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OCM-3227: Add support for regionalized urls #876

Closed
wants to merge 5 commits into from
Closed
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 23 additions & 5 deletions connection.go
Original file line number Diff line number Diff line change
Expand Up @@ -51,11 +51,12 @@ import (
// Default values:
const (
// #nosec G101
DefaultTokenURL = authentication.DefaultTokenURL
DefaultClientID = authentication.DefaultClientID
DefaultClientSecret = authentication.DefaultClientSecret
DefaultURL = "https://api.openshift.com"
DefaultAgent = "OCM-SDK/" + Version
DefaultTokenURL = authentication.DefaultTokenURL
DefaultClientID = authentication.DefaultClientID
DefaultClientSecret = authentication.DefaultClientSecret
DefaultURL = "https://api.openshift.com"
FormattedDefaultRegionURL = "https://api.%s.openshift.com"
DefaultAgent = "OCM-SDK/" + Version
)

// DefaultScopes is the ser of scopes used by default:
Expand Down Expand Up @@ -232,6 +233,17 @@ func (b *ConnectionBuilder) URL(url string) *ConnectionBuilder {
return b.AlternativeURL("", url)
}

// Region builds the base URL of an API gateway based on an OCM region name. The default is
// `https://api.%s.openshift.com`. Where `%s` is replaced by the region name.
//
// This method should be used in-lieu of URL when connecting to a regionalized API gateway.
func (b *ConnectionBuilder) Region(region string) *ConnectionBuilder {
if b.err != nil {
return b
}
return b.AlternativeURL("", fmt.Sprintf(FormattedDefaultRegionURL, region))
tylercreller marked this conversation as resolved.
Show resolved Hide resolved
}

// AlternativeURL sets an alternative base URL for the given path prefix. For example, to configure
// the connection so that it sends the requests for the clusters management service to
// `https://my.server.com`:
Expand Down Expand Up @@ -566,6 +578,7 @@ func (b *ConnectionBuilder) Load(source interface{}) *ConnectionBuilder {
URL *string `yaml:"url"`
AlternativeURLs map[string]string `yaml:"alternative_urls"`

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

any idea what use cases these alternativeURLs are used for? I can guess at it from their context, just want to make sure we've done some due diligence to make sure they dont also need to be region aware (I dont think they do since they seems like explicit overrides)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alternative URLs are used to set different URLs for different services. For example if you wanted to run the SDK against integration accounts mgmt and staging clusters service. URL uses Alternative URLs under the hood with an empty identifier (the default URL). Region utilizes this same precedent albeit with an explicit pattern to uphold.

TokenURL *string `yaml:"token_url"`
Region *string `yaml:"region"`

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should region and url be mutually exclusive? they effectively are, with region winning since its set last. just wondering if would be better with an explicit check with a warning/error. or maybe that should be in the end client not the sdk.

bit nit picky, not too worried about it, use your best judgement.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking the same thing. I don't believe the SDK enforces anything like this today and leaves these decisions up to the consumers.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's a valid point. because region is now meaningful, then --region=integration works exactly as you think. no region is commercial prod.

User *string `yaml:"user"`
Password *string `yaml:"password"`
ClientID *string `yaml:"client_id"`
Expand Down Expand Up @@ -660,6 +673,11 @@ func (b *ConnectionBuilder) Load(source interface{}) *ConnectionBuilder {
b.MetricsSubsystem(*view.MetricsSubsystem)
}

// Regions:
if view.Region != nil {
b.Region(*view.Region)
}

return b
}

Expand Down
171 changes: 171 additions & 0 deletions examples/regionalized_list_clusters.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,171 @@
/*
Copyright (c) 2023 Red Hat, Inc.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

// This example shows how to retrieve the collection of clusters from a specific OCM region

package main

import (
"context"
"encoding/json"
"fmt"
"os"

sdk "github.com/openshift-online/ocm-sdk-go"
cmv1 "github.com/openshift-online/ocm-sdk-go/clustersmgmt/v1"
"github.com/openshift-online/ocm-sdk-go/logging"
)

func main() {
// Create a context:
ctx := context.Background()

// Create a logger that has the debug level enabled:
logger, err := logging.NewGoLoggerBuilder().
Debug(true).
Build()
if err != nil {
fmt.Fprintf(os.Stderr, "Can't build logger: %v\n", err)
os.Exit(1)
}

// USE CASE 1:
// You do not have the desired region ID and need to query the global region to find URL

// Create the global connection
token := os.Getenv("OCM_TOKEN")
globalConnection, err := sdk.NewConnectionBuilder().
Logger(logger).
Tokens(token).
Region("integration"). // Define the integration global region
BuildContext(ctx)
if err != nil {
fmt.Fprintf(os.Stderr, "Can't build connection: %v\n", err)
os.Exit(1)
}
defer globalConnection.Close()

// Fetch the shards from the global region
response, err := globalConnection.Get().Path("/static/ocm-shards.json").SendContext(ctx)
if err != nil {
fmt.Fprintf(os.Stderr, "Can't retrieve shards: %s\n", err)
os.Exit(1)
}

// Turn response into an interface with regions
data := response.Bytes()

// turn region bytes into a map
var regions map[string]interface{}
err = json.Unmarshal(data, &regions)
if err != nil {
fmt.Fprintf(os.Stderr, "Can't unmarshal shards: %s\n", err)
os.Exit(1)
}

// Grab the singapore region URL
regionURL := regions["rh-singapore"].(map[string]interface{})["url"].(string)

// Build a regionalized connection based on the desired shard
connection, err := sdk.NewConnectionBuilder().
Logger(logger).
Tokens(token).
URL(fmt.Sprintf("https://%s", regionURL)). // Apply the region URL

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so you connect to "integration" as a Region, get ocm-shards.json, to extract the singapore region url and set it with URL rather than Region...i understand why its done like this as written, but it seems a bit...muddy. Like what real value is ConnectionBuilder.Region providing here besides a slightly different way of setting the url?

My initial though with adding region support to ocm-sdk was to reduce duplicate work in downstream consumers. We are basically implementing a service discovery mechanism. Point a client at a global OCM instance, ask it what regions is knows about (ocm-shards.json, which admittedly is a short term WIP concept which will probably change as we work with it in UI and cli tools), and select the right url based on a short human usable string (i.e. "rh-singapore"). We need to implement that same basic thing in both rosa installer and ocm cli.

Does this MR facilitate that workflow?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, we've already chatted about this a bit but to bring it over here...

This is a byproduct given the current setup of ocm shards. Ideally during the discovery phase we would only care about the region ID and that region ID would directly correlate to the region URL. Such as api.rh-singapore.openshift.com. This would reduce some of the "mud" in this example.

This MR can facilitate the workflow you are talking about once we settle into our design of the region source-of-truth. Ultimately we are just overwriting the URL based on the desired region. IMO I don't think the SDK should be attempting to auto-discover regions when the Region flag is set, it should just attempt to connect and consumers should dictate how that region is discovered.

BuildContext(ctx)
if err != nil {
fmt.Fprintf(os.Stderr, "Can't build connection: %v\n", err)
os.Exit(1)
}
defer connection.Close()

// Get the client for the resource that manages the collection of clusters:
collection := connection.ClustersMgmt().V1().Clusters()

// Retrieve the list of clusters using pages of ten items, till we get a page that has less
// items than requests, as that marks the end of the collection:
size := 10
page := 1
for {
// Retrieve the page:
response, err := collection.List().
Search("name like 'my%'").
Size(size).
Page(page).
SendContext(ctx)
if err != nil {
fmt.Fprintf(os.Stderr, "Can't retrieve page %d: %s\n", page, err)
os.Exit(1)
}

// Display the page:
response.Items().Each(func(cluster *cmv1.Cluster) bool {
fmt.Printf("%s - %s - %s\n", cluster.ID(), cluster.Name(), cluster.State())
return true
})

// Break the loop if the size of the page is less than requested, otherwise go to
// the next page:
if response.Size() < size {
break
}
page++
}

// USE CASE 2:
// You have a specific region ID and want to query that region directly

// Build the regionalized connection based on the desired region ID
connection, err = sdk.NewConnectionBuilder().
Logger(logger).
Tokens(token).
Region("aws.xcm.integration"). // Apply the region URL
BuildContext(ctx)
if err != nil {
fmt.Fprintf(os.Stderr, "Can't build connection: %v\n", err)
os.Exit(1)
}
defer connection.Close()

collection = connection.ClustersMgmt().V1().Clusters()

// Retrieve the list of clusters using pages of ten items, till we get a page that has less
// items than requests, as that marks the end of the collection:
for {
// Retrieve the page:
response, err := collection.List().
Search("name like 'my%'").
Size(size).
Page(page).
SendContext(ctx)
if err != nil {
fmt.Fprintf(os.Stderr, "Can't retrieve page %d: %s\n", page, err)
os.Exit(1)
}

// Display the page:
response.Items().Each(func(cluster *cmv1.Cluster) bool {
fmt.Printf("%s - %s - %s\n", cluster.ID(), cluster.Name(), cluster.State())
return true
})

// Break the loop if the size of the page is less than requested, otherwise go to
// the next page:
if response.Size() < size {
break
}
page++
}
}
Loading