Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: set config in environment vars/config files #10523

Open
NickCrews opened this issue Nov 22, 2024 · 2 comments
Open

feat: set config in environment vars/config files #10523

NickCrews opened this issue Nov 22, 2024 · 2 comments

Comments

@NickCrews
Copy link
Contributor

Originally posted by @lboller-pwbm in #10231 (comment)

It would be sometimes nice to be able to avoid boilerplate of configuring ibis for every notebook/script. I for instance have this in my machine-wide settings for vscode to run on startup of every jupyter notebook:

    "jupyter.runStartupCommands": [
        "%load_ext autoreload",
        "%autoreload 2",
        "import contextlib",
        "with contextlib.suppress(ImportError): import ibis; ibis.options.interactive=True; ibis.Expr.to_sql = lambda self: ibis.to_sql(self); ibis.Expr.psql = lambda self: print(ibis.to_sql(self))",
        "with contextlib.suppress(ImportError): from ibis_widget import install; install()",
    ],

I see a few options on how to do this:

  1. environment variables
  2. local config files, eg searching up the dir tree eg ./ibis.toml, ../ibis.toml, ../../ibis.toml, etc
  3. user/machine-wide config, eg ~/.config/ibis/config.toml

I think if it's not quite easy to do this, then it's not worth it. But I wanted to have an issue/discussion for this. Feel free to convert to discussion or close this.

@gforsyth
Copy link
Member

gforsyth commented Nov 22, 2024

I definitely agree that this is a good feature to have -- one of the reasons that from ibis.interactive import * is a thing is to scratch this same itch, but with a project that works with so many backends, I think we need to make the knobs customizable to all users.

I've spent way more time than I care to think about on problems like this -- here is my general outline of things that we could/should do and the order to do them in (roughly). It's ok to bail out at any point along here, including at step 0.

  1. Support user-level configuration in XDG style directories (I have a reference somewhere for where Windows likes to store these files, so we can be OS-specific).
  2. Add in a option in Ibis to disable all config files, because once we introduce hidden state, bug reports get harder to reproduce. So we'll need an easy-to-follow setup for "without any customization, this thing broke in this way"
  3. Per-project/per directory config files. These should supersede any options set in the user-level config file (via a dictionary merge or similar)
  4. Environment variables. I would strongly suggest avoiding these because env-variables are effectively entries in a single global configuration file that ALL installed software shares. If we do go down this path, all environment variables should be prepended with IBIS_ because there will be name collisions and unwinding them sucks.

@NickCrews
Copy link
Contributor Author

NickCrews commented Nov 22, 2024

I think uv probably has things to teach us about how to do configuration. I like how they do it. I think they do everything at the rust level though, so I don't think we can take advantage of it :(

spaCy uses confection, which looks simple and interesting.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: backlog
Development

No branches or pull requests

2 participants