Environment variables support #37
Comments
|
Hey, @misuzu! Thanks for your suggestion, sounds like an useful addition to the library 🙂 Do you have an idea of how this feature should be used? Two possibilities come quickly to mind: either use a special class or an annotation. The special class version could look like this: ... and the annotation version like this: Perhaps you got other ideas? |
Maybe there's a way to automatically generate environment variables keys and if that keys have values use that values instead of the values in the config file? |
|
Automatically resolving the names of auto-generated environment variables would currently (i.e. without some major rewrite) only be possible for the simple case in which configurations are not nested. For example, if you have a configuration like this package a.b.c;
record MyConfig(int k) {}then an environment variable For the following example, however, no auto-generated environment variables can be used. package a.b.c;
record RootConfig(SubConfig s) {}
package d.e.f;
record SubConfig(int k) {}The reason for that is that when the library is deserializing an object it doesn't know about all the parents of this object. In the example above, when Another problem I see is that the name implicitly comes from the structure of your code, most likely without any documentation: What happens when the plugin author moves Also, from a security perspective, I don't like the idea of having some magic way of updating configuration values and no indication that this is wanted. And as a last point: Without special class/annotation support you cannot access the values of other environment variables like, for example, When going for annotation based support for environment variables, I can see a solution where support for setting values via annotation can be enabled on a type level like in this example: package a.b.c;
@Env(prefix = "MY_PLUGIN")
record MyConfig(int a, int b){}This would allow setting the values for |
The environment variables keys should be generated from the structure of config files, not code. Using the code structure to do this wouldn't make sense.
The LuckPerms just prints used environment variables to stdout on startup, it makes it pretty obvious where the configuration came from:
Why would a plugin author do this? The The user interface should look something like this: If the user needs to override the value of So a A more real life example, if I need to configure database backend and enable/disable some features I just include the following environment variables to all servers that needs such configuration:
If there's a need for plugins authors to make such changes there wouldn't be much value adding this feature to the ConfigLib, plugins authors could already just use |
|
Alright, thanks for the clarifying examples, now I better understand how you would like to use this feature. The fact that during deserialization objects don't know about their parents (as mentioned in my previous post) still remains. That means that the replacement of environment variables would have to happen earlier in the process. This could possibly happen during a post-processing step of the Since the core parts of this library are plugin agnostic and the YAML content does not necessarily need to be loaded from a file (reading from YamlConfigurationProperties.newBuilder()
.resolveEnvironmentVariables("HUSKCHAT_FILTERS_")
// ...or alternatively
.resolveEnvironmentVariables((String envVar) -> envVar.startsWith("HUSKCHAT_FILTERS_"))
.build();The library would then iterate over all environment variables, select the ones with the correct prefix, split each of them at some character (by default Because alpha:
beta: 1
ALPHA:
BETA: 2 What do you think? If you want a solution that requires absolutely no change from plugin authors, then what you want is not possible, because the information which plugin called |
|
Here's how I'd do it:
The most difficult part is step 2, here's a POC in python (I don't know java, sorry): def get_env_by_config(environ: dict, config: dict, prefix: str) -> dict:
result = {}
for key, value in config.items():
env_key = prefix + '_' + key.upper()
if isinstance(value, dict):
envs = get_env_by_config(environ, value, env_key)
if envs:
result[key] = envs
elif isinstance(value, list):
# could be handled by something more sophisticated
# but at this point it's not that useful
continue
else:
env_value = environ.get(env_key)
print(env_key, '=', env_value)
if env_value is not None:
if isinstance(value, str):
result[key] = env_value
elif isinstance(value, bool):
result[key] = env_value == 'true'
elif isinstance(value, int):
result[key] = int(env_value)
elif isinstance(value, float):
result[key] = float(env_value)
return result
result = get_env_by_config(
{
'MYPLUGIN_TEST_DATABASE_TYPE': 'POSTGRES',
'MYPLUGIN_TEST_DATABASE_CREDENTIALS_HOST': '127.0.0.1',
'MYPLUGIN_TEST_DATABASE_CREDENTIALS_PORT': '5432',
'MYPLUGIN_TEST_SYNCHRONIZATION_FEATURES_GAME_MODE': 'false',
'MYPLUGIN_TEST_SYNCHRONIZATION_FEATURES_FLIGHT_STATUS': 'true',
},
{
'database': {
'type': 'MYSQL',
'credentials': {
'host': 'localhost',
'port': 3306,
'username': 'root',
'password': '',
},
},
'synchronization': {
'features': {
'inventory': True,
'game_mode': True,
'flight_status': False,
},
},
},
'MYPLUGIN_TEST',
)
print(result)The first argument is basically a mock of Here's an output of this script: % python envs.py
MYPLUGIN_TEST_DATABASE_TYPE = POSTGRES
MYPLUGIN_TEST_DATABASE_CREDENTIALS_HOST = 127.0.0.1
MYPLUGIN_TEST_DATABASE_CREDENTIALS_PORT = 5432
MYPLUGIN_TEST_DATABASE_CREDENTIALS_USERNAME = None
MYPLUGIN_TEST_DATABASE_CREDENTIALS_PASSWORD = None
MYPLUGIN_TEST_SYNCHRONIZATION_FEATURES_INVENTORY = None
MYPLUGIN_TEST_SYNCHRONIZATION_FEATURES_GAME_MODE = false
MYPLUGIN_TEST_SYNCHRONIZATION_FEATURES_FLIGHT_STATUS = true
{'database': {'type': 'POSTGRES', 'credentials': {'host': '127.0.0.1', 'port': 5432}}, 'synchronization': {'features': {'game_mode': False, 'flight_status': True}}} |
|
Again, if you want a solution that requires absolutely no change from plugin authors, then what you want is not possible. Plugins authors need to change their code and manually pass a prefix ( If you are okay with that, I can implement this feature as described in my previous post. If you think that it's pointless if it doesn't work without changes from plugin authors (because they can already use |
If they have to declare every single environment variable then yes, it's pointless. If they only need to update the lib and enable the feature by adding a prefix for config file or whatever, then it will be useful.
This should work the other way around: by recursively iterating over map and generating environment variables keys on the way, like in my post. Parsing environment variable key by splitting at
The convention for environment variables keys is to be all uppercase with Either way, this is how I see this feature based on my experience of using and writing different software configured using environment variables. You aren't obliged to do anything (obviously) and I still have the option of asking for this feature in every single plugin (by manually calling |
|
Alright, I'll try to implement it over the weekend and will keep you updated.
Yes, I agree that it's weird, but since it's possible, I'd like to be cover it somehow so that it does not cause some subtle bug - will see. |
|
Quick update: The feature itself is ready; only the documentation is missing. However, I found another quite annoying bug in the library that I need to fix first which might take a little bit more time. |
Is your feature request related to a problem? Please describe.
When dealing with a lot of servers it's impossible to use automatic deployment tools (e.g. docker-compose) because it's required to edit configuration files manually.
Describe the solution you'd like
The ability to override values in config using environment variables.
Something like this: https://luckperms.net/wiki/Configuration#environment-variables
Describe alternatives you've considered
Copying the entire configuration files between servers, but it quickly becomes unmaintainable if I need to use different values for different servers. The ability to only set options that needs changing is much easier and convenient.
I've tried generating config file with the only options I've changed but e.g. HuskHomes/HuskChat/Velocitab just overwrites some of the settings (the nested ones).
#26 should probably fix this, but it would be better to not deal with files at all.
Additional context
The text was updated successfully, but these errors were encountered: