Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: attach all logs on chart failure #150

Merged
merged 2 commits into from
Dec 2, 2024
Merged

Conversation

abuchanan-airbyte
Copy link
Collaborator

The log attachments in Sentry only contain the logs from the failed pod, not all pods. Very often it's the bootloader that's failed, but the bootloader logs don't say anything interesting – they just say the bootloader can't connect to the DB. So this PR would attach all logs from platform pods (only on airbyte chart failure) which hopefully gives us something more interesting to look at.

Java log parse code gets removed

The code I wrote to try to parse java logs from the platform in diagnoseAirbyteChartFailure is mostly a failure. Most of the time it just returns

unable to install airbyte chart: pod airbyte-abctl-airbyte-bootloader: unknown

There are a few variants in the telemetry, such as:

unable to install airbyte chart: pod airbyte-abctl-airbyte-bootloader: Caused by: io.airbyte.db.check.DatabaseCheckException: Unable to connect to the database.

unable to install airbyte chart: pod airbyte-abctl-airbyte-bootloader: Caused by: io.micronaut.context.exceptions.ConfigurationException: Could not resolve placeholder ${DATABASE_PASSWORD}

unable to install airbyte chart: pod airbyte-abctl-airbyte-bootloader: Caused by: org.postgresql.util.PSQLException: ERROR: type "refresh_type" already exists

unable to install airbyte chart: pod airbyte-abctl-airbyte-bootloader: Caused by: org.postgresql.util.PSQLException: ERROR: must be owner of table airbyte_configs

These don't seem very clear for the user. The code to parse java logs lines is clunky and broken and not clearly valuable, so I've removed it in this PR. When the bootloader is the only failed pod, I return an error with a help message that says to run again with --verbose to see the full bootloader logs.

@abuchanan-airbyte abuchanan-airbyte requested a review from a team as a code owner November 13, 2024 21:01
@abuchanan-airbyte abuchanan-airbyte merged commit 2cf1af0 into main Dec 2, 2024
2 checks passed
@abuchanan-airbyte abuchanan-airbyte deleted the abuch/attach-all-logs branch December 2, 2024 16:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants