Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CBS loading process - improvements #1918

Closed
5 tasks
atalyaalon opened this issue Sep 21, 2021 · 0 comments
Closed
5 tasks

CBS loading process - improvements #1918

atalyaalon opened this issue Sep 21, 2021 · 0 comments
Assignees
Milestone

Comments

@atalyaalon
Copy link
Collaborator

atalyaalon commented Sep 21, 2021

  • Get to know current loading process - relevant commandpython main.py process cbs
  • Note - CBS loading process is now both in anyway-etl repo and in anyway repo.
    Ideally it we'll use it only in anyway-etl -
    However - in anyway-etl implementation is not ready - there is a logical bug there - discussed here
  • CBS loading - right now it's not transactional (!)- meaning if a news flash is inserted during CBS loading process - we might add bad data to infographics - since we first delete data and then insert it in a non-transactional way - perhaps this can be fixed using temporary tables?
  • Optional/Future - CBS DB structure- should we modify schema? Should we remove current "Hebrew" "views"?
  • Additional thoughts and improvements on CBS loading process and loading outputs are welcome
@atalyaalon atalyaalon added this to the v0.9.0 milestone Sep 21, 2021
@atalyaalon atalyaalon self-assigned this Sep 21, 2021
@atalyaalon atalyaalon changed the title Remove CBS loading code from anyway repo CBS loading process - improvements Dec 18, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant