-
Notifications
You must be signed in to change notification settings - Fork 49
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Refactor: clean up codebase, simplify where possible #189
Conversation
I'm sorry if there is a conflict between the changes made to #190 and what you are trying to do with this pull request 🙏🙏🙏 . |
2695252
to
ba430de
Compare
febe2df
to
06890b3
Compare
655152d
to
87157a3
Compare
@syou6162 there is some more to do, but at least you can see the significant changes here which will simplify maintainability and even extensibility for LLM integrations, user defined plugins / column fuzzing logic, etc... My approach in such a refactor as this is to collapse code into one file as much as possible such that we can centrally reason on the abstractions and code paths. From there, I can re-break out the functions into different namespaces/modules (if really needed) once the codebase refactor is done. The tests will be updated as appropriate soon too and hopefully things reveal themselves as easier to test as well which would evidence we move in the right direction. |
70fb173
to
7e501d4
Compare
7e501d4
to
e9c040a
Compare
…updated and enabled
…h are more correct in current form
5c632e4
to
77c99af
Compare
… to support config block syntax
…raph to node, add example prefix in config block
a780930
to
5b31abf
Compare
6ffb027
to
30acd2e
Compare
chore: only use get node yaml if needed otherwise stick to node itself
30acd2e
to
a2b7afc
Compare
…func, clean empty resource keys
471d200
to
1d265f0
Compare
refactor!: clean up codebase, simplify where possible chore: fix workbench req txt fix: get release working
refactor!: clean up codebase, simplify where possible chore: fix workbench req txt fix: get release working
refactor!: clean up codebase, simplify where possible chore: fix workbench req txt fix: get release working
This will be a WIP for a bit. dbt has evolved enough since this tool was created that we can probably simplify some things. Furthermore, certain experimental things can be removed. I still like the workbench. Suffice to say there are interesting streamlit applications we have not explored like an interactive documentation input flow for progenitors, etc.
But otherwise a lot we can kill. We can also remove core interface folding it into osmosis.py and leverage whatever dbt core has available now in its pythonic interface -- and be more strict on what dbt version(s) are supported with a focus on newer versions.
Also includes cleanups with py 3.9 being min version.