Skip to content

Commit

Permalink
Update README.md with instructions to run the builder server with sup…
Browse files Browse the repository at this point in the history
…… (#11678)

Co-authored-by: Lake Mossman <[email protected]>
  • Loading branch information
girarda and lmossman committed Mar 18, 2024
1 parent 6259173 commit d520cab
Showing 1 changed file with 42 additions and 0 deletions.
42 changes: 42 additions & 0 deletions airbyte-connector-builder-server/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,12 @@ export CDK_PYTHON=<path_to_CDK_virtual_environment>
export CDK_ENTRYPOINT=<path_to_CDK_connector_builder_main.py>
```

Example commands:
```
export CDK_PYTHON=~/code/airbyte/airbyte-cdk/python/.venv/bin/python
export CDK_ENTRYPOINT=~/code/airbyte/airbyte-cdk/python/airbyte_cdk/connector_builder/main.py
```

Then run the server (You can also do this w/o build)
```bash
./gradlew -p oss airbyte-connector-builder-server:run
Expand All @@ -41,6 +47,42 @@ CONNECTOR_BUILDER_SERVER_API_HOST=http://host.docker.internal:80

Note: there are two different, but very similarly-named, environment variables; you must edit `CONNECTOR_BUILDER_SERVER_API_HOST`, not `CONNECTOR_BUILDER_API_HOST`.

### Running the platform with support for custom components (docker-compose only)

1. Run the OSS platform locally with builder docker-compose extension
1. Example command: PATH_TO_CONNECTORS=/Users/alex/code/airbyte/airbyte-integrations/connectors docker compose -f docker-compose.yaml -f docker-compose.builder.yaml up
2. Where PATH_TO_CONNECTORS points to the airbyte-integrations/connectors subdirectory in the opensource airbyte repository
2. Open the connector builder and develop your connector
3. When needing a custom componentt:
1. Switch to the YAML view
2. Define the custom component
4. Write the custom components and its unit tests
5. Run test read

Note that connector modules are added to the path at startup time. The platform instance must be restarted if you add a new connector module.

Follow these additional instructions if the connector requires 3rd party libraries that are not available in the CDK:

Developing connectors that require 3rd party libraries can be done by running the connector-builder-server locally and pointing to a custom virtual environment.

1. Create a virtual environment and install the CDK + any 3rd party library required
2. export CDK_PYTHON=<path_to_virtual_environment>
- `CDK_PYTHON` should point to the virtual environment's python executable (example: `export CDK_PYTHON=~/code/airbyte/airbyte-cdk/python/.venv/bin/python`)
3. export CDK_ENTRYPOINT=<path_to_CDK_connector_builder_main.py>
4. ./gradlew -p oss airbyte-connector-builder-server:run
1. The server is now reachable on localhost:80
5. Update the server to point to port 80 by editing .env and replacing

```
CONNECTOR_BUILDER_SERVER_API_HOST=http://airbyte-connector-builder-server:80
```
with
```
CONNECTOR_BUILDER_SERVER_API_HOST=http://host.docker.internal:80
```
6. Follow the standard instructions
## OpenAPI generation
Run it via Gradle by running this from the Airbyte project root:
Expand Down

0 comments on commit d520cab

Please sign in to comment.