Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle schema discovery of tables that are short-lived #21

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

drdee
Copy link

@drdee drdee commented Mar 4, 2020

Description of change

Some tables are short lived and they don't exist any more during the discovery phase.

This is the relevant stacktrace:

2020-03-04 16:50:05,457Z    tap - INFO fetching tables: 
2020-03-04 16:50:05,457Z    tap -    SELECT owner, table_name
2020-03-04 16:50:05,457Z    tap -    FROM all_tables
2020-03-04 16:50:05,457Z    tap -    WHERE owner != 'SYS' []
2020-03-04 16:50:12,206Z    tap - CRITICAL 'GL_POSTING_INTERIM_6872031'
2020-03-04 16:50:12,207Z    tap - Traceback (most recent call last):
2020-03-04 16:50:12,207Z    tap -   File "tap-env/bin/tap-oracle", line 8, in <module>
2020-03-04 16:50:12,207Z    tap -     sys.exit(main())
2020-03-04 16:50:12,207Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_oracle/__init__.py", line 572, in main
2020-03-04 16:50:12,207Z    tap -     raise exc
2020-03-04 16:50:12,208Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_oracle/__init__.py", line 569, in main
2020-03-04 16:50:12,208Z    tap -     main_impl()
2020-03-04 16:50:12,208Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_oracle/__init__.py", line 559, in main_impl
2020-03-04 16:50:12,208Z    tap -     do_discovery(conn_config, filter_schemas)
2020-03-04 16:50:12,208Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_oracle/__init__.py", line 331, in do_discovery
2020-03-04 16:50:12,208Z    tap -     'row_count': row_counts[table],

by using get and with a default value of 0 we can make the discovery phase more resilient.

Rollback steps

  • revert this branch

@drdee
Copy link
Author

drdee commented Mar 4, 2020

@nick-mccoy @KAllan357 -- please have a look.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant