refactor: Make gsctl to be more like a database design (#3648)

### gsctl
```bash
graphscope@host:/work$ gsctl

    ______                 __   _____
   / ____/________ _____  / /_ / ___/_________  ____  ___
  / / __/ ___/ __ `/ __ \/ __ \\__ \/ ___/ __ \/ __ \/ _ \
 / /_/ / /  / /_/ / /_/ / / / /__/ / /__/ /_/ / /_/ /  __/
 \____/_/   \__,_/ .___/_/ /_/____/\___/\____/ .___/\___/
                /_/                         /_/

Currently, gsctl hasn't connect to any service.

you can use gsctl as an utility script.
Or you can connect to a launched GraphScopoe service by `gsctl connect --coordinator-endpoint <address>`.
See more detailed information at https://graphscope.io/docs/utilities/gs.

Usage: gsctl [OPTIONS] COMMAND [ARGS]...

Options:
  --help  Show this message and exit.

Commands:
  close         Disconnect from coordinator
  connect       Connect to a launched coordinator
  flexbuild     Build docker image for Interactive, Insight product.
  install-deps  Install dependencies for building GraphScope.
  make          Build executive binaries of COMPONENT.
  make-image    Make docker images from source code for deployment.
  test          Trigger tests on built artifacts.
```

### connect to a coordinator(global scope)
```
graphscope@host:/work$ gsctl connect --coordinator-endpoint xxxxx && gsctl
Using GLOBAL.
Run `gsctl use GRAPH <graph_identifier>` to switch to a specific graph context.

Usage: gsctl [OPTIONS] COMMAND [ARGS]...

Options:
  --help  Show this message and exit.

Commands:
  close    Disconnect from coordinator
  connect  Connect to a launched coordinator
  create   Create a new graph in database
  delete   Delete a graph by identifier
  ls       Display graph resources in database
  service  Start, stop, and restart the database service
  use      Switch to GRAPH context, see identifier with `ls` command
```

### change to a specific graph scope (graph scope)
```bash
graphscope@host:/work$ gsctl use GRAPH dongze_test_graph
Using GRAPH dongze_test_graph

graphscope@host:/work$ gsctl ls
Using GRAPH dongze_test_graph.
Run `gsctl use GLOBAL` to switch back to GLOBAL context.

schema
├── vertex types
│   ├── person
│   │   ├── Property(name: id, type: DT_SIGNED_INT64, is_primary_key: True)
│   │   ├── Property(name: name, type: DT_STRING, is_primary_key: False)
│   │   └── Property(name: age, type: DT_SIGNED_INT32, is_primary_key: False)
│   └── software
│       ├── Property(name: id, type: DT_SIGNED_INT64, is_primary_key: True)
│       ├── Property(name: name, type: DT_STRING, is_primary_key: False)
│       └── Property(name: lang, type: DT_STRING, is_primary_key: False)
└── edge types
    ├── (person) -[knows]-> (person)
    │   └── Property(name: weight, type: DT_DOUBLE, is_primary_key: False)
    └── (person) -[created]-> (software)
        └── Property(name: weight, type: DT_DOUBLE, is_primary_key: False)

data sources
├── vertex mappings
└── edge mappings

stored procedure

job
├── Job(identifier: job_dongze_test_graph_1709023420190_51285, type: bulk_loading, status: SUCCESS, start time: 2024-02-27 16:43:40, end time: 2024-02-27 16:44:10)
```

### create a procedure
```bash
graphscope@host:/work$  gsctl create procedure -f /path/to/your/procedure/file
[SUCCESS] Create stored procedure xxx successfully.

```

### switch back to the global context (global scope)
```bash
graphscope@host:/work$ gsctl use GLOBAL
Using GLOBAL
```

docs/utilities/gs.md

@@ -6,27 +6,26 @@
 
 Since it is shipped with python package `graphscope-client`, the `gsctl` command will be available in your terminal after installing GraphScope:
 ```bash
-pip install graphscope-client
+$ pip3 install graphscope-client
 ```
 
-In some cases, such as development on `gsctl`, you may want to build it from source. 
+In some cases, such as development on `gsctl`, you may want to build it from source.
 To do this, navigate to the directory where the source code is located and run the following command:
 
 ```bash
-cd REPO_HOME/python
-# If you want to develop gsctl, 
-# please note the entry point is located on: 
+$ cd REPO_HOME/python
+# If you want to develop gsctl,
+# please note the entry point is located on:
 # /python/graphscope/gsctl/gsctl.py
-pip install --editable .
+$ pip3 install --editable .
 ```
 This will install `gsctl` in an editable mode, which means that any changes you make to the source code will be reflected in the installed version of `gsctl`.
 
 ## Commands
 
 With `gsctl`, you can do the following things. Always remember to use `--help` on a command to get more information.
 
-The `gsctl` command-line utility supports two modes of operation: utility scripts and client/server mode. You can switch between these modes using the
-`gsctl connect` and `gsctl close` commands.
+The `gsctl` command-line utility supports two modes of operation: utility scripts and client/server mode. You can switch between these modes using the `gsctl connect` and `gsctl close` commands.
 
 ### Utility Scripts
 
@@ -39,12 +38,31 @@ Default, the `gsctl` provide helper functions and utilities that can be run usin
 - `gsctl test`, trigger test suites.
 - `gsctl connect`, connect to the launched coordinator by ~/.gs/config.
 - `gsctl close`, Close the connection from the coordinator.
-
+- `gsctl flexbuild`, Build docker image for Interactive, Insight product.
 
 ### Client/Server Mode
 
-To switch to the client/server mode, use the `gsctl connect` command. This command connects gsctl to a launched coordinator using the configuration file located at ~/.gsconfig.
-Once connected, you can use `gsctl` to communicate with the coordinator and send commands that will be executed on the coordinator side.
+To switch to the client/server mode, use the `gsctl connect` command. By default, this command connects gsctl to a launched coordinator using the configuration file located at `${HOME}/.gsctl`;  If `--coordinator-endpoint` param is specified, it will treat it as the current context and override the configuration file.
+
+Once connected, you can use `gsctl` to communicate with the coordinator which serves the specific Flex product behind it.
+
+#### Change scope
+
+In `gsctl`, you can run commands on a global scope or a local scope. When you connect to a coordinator, you are in the global scope. To change to local scope of a graph, run the `gsctl use GRAPH <graph_identifier>` command. You can find the graph identifier with `gsctl ls` command.
+
+```bash
+$ gsctl use GRAPH modern_graph
+Using GRAPH modern_graph
+```
+
+To switch back to the global scope. run `gsctl use GLOBAL` command.
+
+```bash
+$ gsctl use GLOBAL
+Using GLOBAL
+```
+Different scopes have different commands. Always remember to use `--help` on a command to get more information.
+
+#### Close the connection
 
-To disconnect from the coordinator and switch back to the utility scripts mode, you can use the `gsctl close` command. This command closes the connection from the coordinator
-and allows you to use `gsctl` as a standalone utility again.
+To disconnect from the coordinator and switch back to the utility scripts mode, you can use the `gsctl close` command. This command closes the connection from the coordinator and allows you to use `gsctl` as a standalone utility again.

python/graphscope/gsctl/commands/__init__.py

@@ -16,14 +16,21 @@
 # limitations under the License.
 #
 
+import sys
+
 import click
 
 from graphscope.gsctl.commands.common import cli as common
 from graphscope.gsctl.commands.dev import cli as dev
-from graphscope.gsctl.commands.insight import cli as insight
-from graphscope.gsctl.commands.interactive import cli as interactive
+from graphscope.gsctl.commands.insight.glob import cli as insight
+from graphscope.gsctl.commands.interactive.glob import cli as interactive
+from graphscope.gsctl.commands.interactive.graph import cli as interactive_graph
 from graphscope.gsctl.config import Context
+from graphscope.gsctl.config import load_gs_config
+from graphscope.gsctl.config import logo
 from graphscope.gsctl.impl import connect_coordinator
+from graphscope.gsctl.utils import err
+from graphscope.gsctl.utils import info
 
 
 def get_command_collection(context: Context):
@@ -34,27 +41,57 @@ def get_command_collection(context: Context):
     # e.g. initialize and manage cluster, install the dependencies required to
     # build graphscope locally.
     if context is None:
+        if len(sys.argv) == 1:
+            info(logo, fg="green", bold=True)
+            click.secho("Currently, gsctl hasn't connect to any service.", fg="yellow")
+            message = """
+you can use gsctl as an utility script.
+Or you can connect to a launched GraphScopoe service by `gsctl connect --coordinator-endpoint <address>`.
+See more detailed information at https://graphscope.io/docs/utilities/gs.
+            """
+            info(message)
         return commands
 
-    # connect to coordinator and parse the commands with solution
-    try:
-        response = connect_coordinator(context.coordinator_endpoint)
-        solution = response.solution
-        if solution == "INTERACTIVE":
+    if context.is_expired():
+        try:
+            # connect to coordinator and reset the timestamp
+            response = connect_coordinator(context.coordinator_endpoint)
+            solution = response.solution
+        except Exception as e:
+            err(
+                "Failed to connect to coordinator at {0}: {1}".format(
+                    context.coordinator_endpoint, str(e)
+                )
+            )
+            info(
+                "Please check the availability of the service, fall back to the default commands."
+            )
+            return commands
+        else:
+            # check consistency
+            if solution != context.flex:
+                raise RuntimeError(
+                    f"Instance changed: {context.flex} -> {solution}, please close and reconnect to the coordinator"
+                )
+            context.reset_timestamp()
+            config = load_gs_config()
+            config.update_and_write(context)
+
+    if context.flex == "INTERACTIVE":
+        if context.context == "global":
+            if len(sys.argv) < 2 or sys.argv[1] != "use":
+                info("Using GLOBAL.", fg="green", bold=True)
+                info(
+                    "Run `gsctl use GRAPH <graph_identifier>` to switch to a specific graph context.\n"
+                )
             commands = click.CommandCollection(sources=[common, interactive])
-        elif solution == "GRAPHSCOPE_INSIGHT":
+        else:
+            if len(sys.argv) < 2 or sys.argv[1] != "use":
+                info(f"Using GRAPH {context.context}.", fg="green", bold=True)
+                info("Run `gsctl use GLOBAL` to switch back to GLOBAL context.\n")
+            commands = click.CommandCollection(sources=[common, interactive_graph])
+    elif context.flex == "GRAPHSCOPE_INSIGHT":
+        if context.context == "global":
             commands = click.CommandCollection(sources=[common, insight])
-    except Exception as e:
-        click.secho(
-            "Failed to connect to coordinator at {0}: {1}".format(
-                context.coordinator_endpoint, str(e)
-            ),
-            fg="red",
-        )
-        click.secho(
-            "Please check the availability of the service, or close/reconnect the service.",
-            fg="blue",
-        )
-        click.secho("Fall back to the default commands.", fg="blue")
-    finally:
-        return commands
+
+    return commands

python/graphscope/gsctl/commands/common.py

@@ -25,6 +25,9 @@
 from graphscope.gsctl.config import load_gs_config
 from graphscope.gsctl.impl import connect_coordinator
 from graphscope.gsctl.impl import disconnect_coordinator
+from graphscope.gsctl.utils import err
+from graphscope.gsctl.utils import info
+from graphscope.gsctl.utils import succ
 
 
 @click.group()
@@ -33,49 +36,50 @@ def cli():
     pass
 
 
-@click.command()
+@cli.command()
 @click.option(
+    "-c",
     "--coordinator-endpoint",
-    help="Coordinator endpoint which gsctl connect to, e.g. http://127.0.0.1:9527",
+    help="Coordinator endpoint, e.g. http://127.0.0.1:9527",
 )
 def connect(coordinator_endpoint):
-    """Connect to the launched coordinator
+    """Connect to a launched coordinator
 
-    By default, it will read context from  ~/.graphscope/config. If '--coordinator-endpoint'
-    is specified, use it as the current context and override the config file.
+    By default, it will read context from  ~/.gsctl. If '--coordinator-endpoint'
+    is specified, use it as the current context and override the configuration file.
     """
     if coordinator_endpoint is None:
         context = get_current_context()
         if context is None:
-            click.secho(
-                "No available context found, try to connect to coordinator with --coordinator-endpoint",
-                fg="blue",
+            err(
+                "No available context found, try to connect by `gsctl conenct --coordinator-endpoint <addr>`."
             )
             return
         coordinator_endpoint = context.coordinator_endpoint
     # connect
     try:
-        connect_coordinator(coordinator_endpoint)
+        resp = connect_coordinator(coordinator_endpoint)
     except Exception as e:
-        click.secho(f"Unable to connect to server: {str(e)}", fg="red")
+        err(f"Unable to connect to server: {str(e)}")
     else:
-        click.secho(f"Coordinator at {coordinator_endpoint} connected.", fg="green")
+        succ(
+            f"Connected to {coordinator_endpoint}, coordinator is serving with {resp.solution} mode.\n"
+        )
+        info("Try 'gsctl --help' for help.")
 
 
-@click.command()
+@cli.command()
 def close():
-    """Close the connection from the coordinator"""
+    """Disconnect from coordinator"""
     try:
         context = disconnect_coordinator()
     except Exception as e:
-        click.secho(f"Disconnect to server failed: {str(e)}", fg="red")
+        err(f"Disconnect from coordinator failed: {str(e)}")
     else:
         if context is not None:
-            click.secho(f"Coordinator disconnected: {context.to_dict()}.", fg="green")
-
-
-cli.add_command(connect)
-cli.add_command(close)
+            info(f"Disconnecting from the context: {context.to_dict()}")
+            succ("Coordinator service disconnected.\n")
+            info("Try 'gsctl --help' for help.")
 
 
 if __name__ == "__main__":

python/graphscope/gsctl/commands/dev.py

@@ -56,7 +56,7 @@ def cli():
 
 @cli.group()
 def flexbuild():
-    """Build a customized stack using specific components."""
+    """Build docker image for Interactive, Insight product."""
     pass