Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

TypeError: '>=' not supported between instances of 'NoneType' and 'str' - When saving state #203

Closed
visch opened this issue Aug 14, 2023 · 0 comments · Fixed by #233
Closed

TypeError: '>=' not supported between instances of 'NoneType' and 'str' - When saving state #203

visch opened this issue Aug 14, 2023 · 0 comments · Fixed by #233
Assignees
@visch

Comments

@visch
Copy link
Member

visch commented Aug 14, 2023 

2023-08-14T18:15:27.592513Z [info     ] Traceback (most recent call last): cmd_type=elb consumer=False name=tap-postgres--integrationsdb producer=True stdio=stderr string_id=tap-postgres--integrationsdb
2023-08-14T18:15:27.592836Z [info     ]   File "/project/.meltano/extractors/tap-postgres/venv/lib/python3.9/site-packages/singer_sdk/streams/core.py", line 1166, in sync cmd_type=elb consumer=False name=tap-postgres--integrationsdb producer=True stdio=stderr string_id=tap-postgres--integrationsdb
2023-08-14T18:15:27.593151Z [info     ]     for _ in self._sync_records(context=context): cmd_type=elb consumer=False name=tap-postgres--integrationsdb producer=True stdio=stderr string_id=tap-postgres--integrationsdb
2023-08-14T18:15:27.593466Z [info     ]   File "/project/.meltano/extractors/tap-postgres/venv/lib/python3.9/site-packages/singer_sdk/streams/core.py", line 1091, in _sync_records cmd_type=elb consumer=False name=tap-postgres--integrationsdb producer=True stdio=stderr string_id=tap-postgres--integrationsdb
2023-08-14T18:15:27.593783Z [info     ]     self._increment_stream_state(record, context=current_context) cmd_type=elb consumer=False name=tap-postgres--integrationsdb producer=True stdio=stderr string_id=tap-postgres--integrationsdb
2023-08-14T18:15:27.594100Z [info     ]   File "/project/.meltano/extractors/tap-postgres/venv/lib/python3.9/site-packages/singer_sdk/streams/core.py", line 745, in _increment_stream_state cmd_type=elb consumer=False name=tap-postgres--integrationsdb producer=True stdio=stderr string_id=tap-postgres--integrationsdb
2023-08-14T18:15:27.594402Z [info     ]     increment_state(           cmd_type=elb consumer=False name=tap-postgres--integrationsdb producer=True stdio=stderr string_id=tap-postgres--integrationsdb
2023-08-14T18:15:27.594711Z [info     ]   File "/project/.meltano/extractors/tap-postgres/venv/lib/python3.9/site-packages/singer_sdk/helpers/_state.py", line 218, in increment_state cmd_type=elb consumer=False name=tap-postgres--integrationsdb producer=True stdio=stderr string_id=tap-postgres--integrationsdb
2023-08-14T18:15:27.595022Z [info     ]     if old_rk_value is None or not check_sorted or new_rk_value >= old_rk_value: cmd_type=elb consumer=False name=tap-postgres--integrationsdb producer=True stdio=stderr string_id=tap-postgres--integrationsdb
2023-08-14T18:15:27.595328Z [info     ] TypeError: '>=' not supported between instances of 'NoneType' and 'str' cmd_type=elb consumer=False name=tap-postgres--integrationsdb producer=True stdio=stderr string_id=tap-postgres--integrationsdb
2023-08-14T18:15:27.595652Z [info     ] Traceback (most recent call last): cmd_type=elb consumer=False name=tap-postgres--integrationsdb producer=True stdio=stderr string_id=tap-postgres--integrationsdb
2023-08-14T18:15:27.595961Z [info     ]   File "/project/.meltano/extractors/tap-postgres/venv/bin/tap-postgres", line 8, in <module> cmd_type=elb consumer=False name=tap-postgres--integrationsdb producer=True stdio=stderr string_id=tap-postgres--integrationsdb
2023-08-14T18:15:27.596272Z [info     ]     sys.exit(TapPostgres.cli()) cmd_type=elb consumer=False name=tap-postgres--integrationsdb producer=True stdio=stderr string_id=tap-postgres--integrationsdb
2023-08-14T18:15:27.596528Z [info     ]   File "/project/.meltano/extractors/tap-postgres/venv/lib/python3.9/site-packages/click/core.py", line 1157, in __call__ cmd_type=elb consumer=False name=tap-postgres--integrationsdb producer=True stdio=stderr string_id=tap-postgres--integrationsdb
2023-08-14T18:15:27.596792Z [info     ]     return self.main(*args, **kwargs) cmd_type=elb consumer=False name=tap-postgres--integrationsdb producer=True stdio=stderr string_id=tap-postgres--integrationsdb
2023-08-14T18:15:27.596980Z [info     ]   File "/project/.meltano/extractors/tap-postgres/venv/lib/python3.9/site-packages/click/core.py", line 1078, in main cmd_type=elb consumer=False name=tap-postgres--integrationsdb producer=True stdio=stderr string_id=tap-postgres--integrationsdb
2023-08-14T18:15:27.597154Z [info     ]     rv = self.invoke(ctx)      cmd_type=elb consumer=False name=tap-postgres--integrationsdb producer=True stdio=stderr string_id=tap-postgres--integrationsdb
2023-08-14T18:15:27.597325Z [info     ]   File "/project/.meltano/extractors/tap-postgres/venv/lib/python3.9/site-packages/click/core.py", line 1434, in invoke cmd_type=elb consumer=False name=tap-postgres--integrationsdb producer=True stdio=stderr string_id=tap-postgres--integrationsdb
2023-08-14T18:15:27.597497Z [info     ]     return ctx.invoke(self.callback, **ctx.params) cmd_type=elb consumer=False name=tap-postgres--integrationsdb producer=True stdio=stderr string_id=tap-postgres--integrationsdb
2023-08-14T18:15:27.597665Z [info     ]   File "/project/.meltano/extractors/tap-postgres/venv/lib/python3.9/site-packages/click/core.py", line 783, in invoke cmd_type=elb consumer=False name=tap-postgres--integrationsdb producer=True stdio=stderr string_id=tap-postgres--integrationsdb
2023-08-14T18:15:27.597832Z [info     ]     return __callback(*args, **kwargs) cmd_type=elb consumer=False name=tap-postgres--integrationsdb producer=True stdio=stderr string_id=tap-postgres--integrationsdb
2023-08-14T18:15:27.597998Z [info     ]   File "/project/.meltano/extractors/tap-postgres/venv/lib/python3.9/site-packages/singer_sdk/tap_base.py", line 492, in invoke cmd_type=elb consumer=False name=tap-postgres--integrationsdb producer=True stdio=stderr string_id=tap-postgres--integrationsdb
2023-08-14T18:15:27.598165Z [info     ]     tap.sync_all()             cmd_type=elb consumer=False name=tap-postgres--integrationsdb producer=True stdio=stderr string_id=tap-postgres--integrationsdb
2023-08-14T18:15:27.598332Z [info     ]   File "/project/.meltano/extractors/tap-postgres/venv/lib/python3.9/site-packages/singer_sdk/tap_base.py", line 450, in sync_all cmd_type=elb consumer=False name=tap-postgres--integrationsdb producer=True stdio=stderr string_id=tap-postgres--integrationsdb
2023-08-14T18:15:27.598500Z [info     ]     stream.sync()              cmd_type=elb consumer=False name=tap-postgres--integrationsdb producer=True stdio=stderr string_id=tap-postgres--integrationsdb
2023-08-14T18:15:27.598667Z [info     ]   File "/project/.meltano/extractors/tap-postgres/venv/lib/python3.9/site-packages/singer_sdk/streams/core.py", line 1173, in sync cmd_type=elb consumer=False name=tap-postgres--integrationsdb producer=True stdio=stderr string_id=tap-postgres--integrationsdb
2023-08-14T18:15:27.598835Z [info     ]     raise ex                   cmd_type=elb consumer=False name=tap-postgres--integrationsdb producer=True stdio=stderr string_id=tap-postgres--integrationsdb
2023-08-14T18:15:27.599002Z [info     ]   File "/project/.meltano/extractors/tap-postgres/venv/lib/python3.9/site-packages/singer_sdk/streams/core.py", line 1166, in sync cmd_type=elb consumer=False name=tap-postgres--integrationsdb producer=True stdio=stderr string_id=tap-postgres--integrationsdb
2023-08-14T18:15:27.599177Z [info     ]     for _ in self._sync_records(context=context): cmd_type=elb consumer=False name=tap-postgres--integrationsdb producer=True stdio=stderr string_id=tap-postgres--integrationsdb
2023-08-14T18:15:27.599344Z [info     ]   File "/project/.meltano/extractors/tap-postgres/venv/lib/python3.9/site-packages/singer_sdk/streams/core.py", line 1091, in _sync_records cmd_type=elb consumer=False name=tap-postgres--integrationsdb producer=True stdio=stderr string_id=tap-postgres--integrationsdb
2023-08-14T18:15:27.599512Z [info     ]     self._increment_stream_state(record, context=current_context) cmd_type=elb consumer=False name=tap-postgres--integrationsdb producer=True stdio=stderr string_id=tap-postgres--integrationsdb
2023-08-14T18:15:27.599681Z [info     ]   File "/project/.meltano/extractors/tap-postgres/venv/lib/python3.9/site-packages/singer_sdk/streams/core.py", line 745, in _increment_stream_state cmd_type=elb consumer=False name=tap-postgres--integrationsdb producer=True stdio=stderr string_id=tap-postgres--integrationsdb
2023-08-14T18:15:27.599849Z [info     ]     increment_state(           cmd_type=elb consumer=False name=tap-postgres--integrationsdb producer=True stdio=stderr string_id=tap-postgres--integrationsdb
2023-08-14T18:15:27.600015Z [info     ]   File "/project/.meltano/extractors/tap-postgres/venv/lib/python3.9/site-packages/singer_sdk/helpers/_state.py", line 218, in increment_state cmd_type=elb consumer=False name=tap-postgres--integrationsdb producer=True stdio=stderr string_id=tap-postgres--integrationsdb
2023-08-14T18:15:27.600181Z [info     ]     if old_rk_value is None or not check_sorted or new_rk_value >= old_rk_value: cmd_type=elb consumer=False name=tap-postgres--integrationsdb producer=True stdio=stderr string_id=tap-postgres--integrationsdb
2023-08-14T18:15:27.600346Z [info     ] TypeError: '>=' not supported between instances of 'NoneType' and 'str' cmd_type=elb consumer=False name=tap-postgres--integrationsdb producer=True stdio=stderr string_id=tap-postgres--integrationsdb

Not sure how to replicate attempting to now
@visch visch self-assigned this Aug 14, 2023
@sebastianswms sebastianswms mentioned this issue Sep 11, 2023
Merged
@visch visch mentioned this issue Sep 12, 2023
Closed
1 task
visch pushed a commit that referenced this issue Sep 12, 2023
@sebastianswms
fix: Fixes null state attempting to be written (#233)
bcc0db1 
Having a replication key column that contains null values was causing
null state to be written under some circumstances. This PR sorts null
values as the "lowest" result for replication key purposes, preventing
nulls from taking precedence over real replication values.

Closes #203
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@visch visch

Labels
None yet
Projects
MeltanoLabs Overview
Archived in project
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix: Fixes null state attempting to be written sebastianswms/tap-postgres
1 participant
@visch