Docker production environment

[almereyda]

The Docker environment introduced with https://github.com/moritzfl/docker-fiduswriter that runs on SQLite allows local testing and has configuration facilities.

Considering to run an instance of Fidus Writer on our own, we would like to describe a production deployment in code and seek which design patterns can be employed to work towards the https://12factors.net. moritzfl/docker-fiduswriter#9

This would include to prefer reading the application configuration from environment variables, instead of configuration files, and show exemplary how a third-party database can be employed.

[almereyda]

An example about how I imagine a Fidus Writer production environment is given with moritzfl/docker-fiduswriter#10

• Further iterations of configuration.py in Fidus Writer itself might wish to adapt the 12 factor compatible example given in there.
• If the Dockerfile and startup script are adopted by the main repository here, the Compose environment could focus on the environment and containers to orchestrate, and run from an officially published image instead.

Later one could also integrate https://github.com/migonzalvar/dj-email-url

[johanneswilm]

Hey,
sorry for the late reply. I assume you mean https://12factor.net/ ?

Besides Docker, Fidus Writer is mainly distributed using Ubuntu Snap and we are maintaining startup scripts, etc. for that. There isn't really a need to make major changes to anything in configuration.py for that to work.

I am not entirely sure what you are asking for, but it sounds like you are talking about packaging Fidus Writer in some other way and that is fine and can be done in another repository (also in this org, for example), but I don't see any need for changing this repository fundamentally at this stage because it works just fine as it is for Ubuntu Snap.

Does that make sense? Or maybe I just didn't understand your proposal @almereyda ?

[almereyda]

Indeed, the example given is a Docker Compose setup in a third-party repository that exemplifies how one can implement the 12factors.net design into a complete environment. It could very well live as compose-fiduswriter in here.

There are two changes in the remote, that I would like to propose and put for discussion here. These are their rationales:

1. The Dockerfile could live here in upstream, in so one gets updated binary assets for free (through setting up an automated build on Docker Hub) whenever there is a merge into master. Having the Dockerfile tied closely to the application allows for changes to propagate quickly through various build and release chains.
   Note 1: Currently it is distributed alongside a startup script that runs specific tasks to initialise the state of the container, if not present already. These checks and conventions could also be implemented in the application, in so it assumes authority over its dedicated, containerised environment.
   Note 2: UIDs are currently hardcoded to meet a certain expectation, but one could investigate ways how to omit the additional user in the container image, and run directly as root.
2. The configuration.py file is functionally equivalent with your upstream distribution, but only adds some syntactic sugar on top of it to define the application state through the environment, as suggested by the 3rd of the 12 factors. If applied to the soruce tree, it greatly increases compatibility with PaaS environments, such as Heroko or Dokku, and with Cloud-native environments like Docker or Kubernetes, where we like to separate mutable (configuration) state from immutable computation assets. Additionally, are there notable experiences with using a separate local_settings.py file in Fidus Writer, from which one could draw further conclusions?

• The (docker-)compose.yaml and associated .env templates can very well live outside of the main repository. Their learnings would ideally be found to be represented in the documentation of the application.

Under these assumptions, is it possible for you and your team to consider supporting a production-ready Docker release directly in the codebase?

[johanneswilm]

Hey,
we can make a secondary repository for the docker stuff that can live here in the fiduswriter organization, the same way we have it for Ubuntu Snap if someone is willing to maintain that repository (I am not volunteering for that unless someone pays me to do that).

I'd like to not put it into the main repository as I really do not want to make this Docker dependent which will make the repository more complicated than what it needs to be. I understand Docker is useful for some people in some situations, but not everyone all the time. For example, for me with the Ubuntu Snap packaging approach it seems a lot easier to maintain 300+ servers in sync with the newest Fidus Writer version than anything I have ever experienced doing with Docker. I have to use Docker for other projects, and for me it just always eats up more time trying to figure out what broke last than what Docker possible can have helped me gain.

[johanneswilm]

As for your comments 1. and 2. @almereyda :

1. Yeah not here but in a secondary repository. Sorry, I've experienced too many projects where everything stopped working and it became a nightmare to maintain when Docker was introduced. The minor versions of Fidus Writer (3.5.x, 3.6.x, 3.7.x) will usually not change in structure, so it should be possible to tag the fiduswriter-docker repository once to then work with all FW 3.6.x releases, another time to work with FW 3.7.x, and so on. There are roughly two minor releases a year, so there should not be too many problems of things getting out of sync.

2. I'm not sure I understand what configuration.py file you are referring to. As you probably have discovered, in Fidus Writer we internally override some of the base django settings and so the configuration.py file the user provides is filling the role of a local_settings.py file in many other django-based projects. I have never played around with Heroko or Dokku, but under all circumstances, but all that should be possible to keep all that in a separate repository so that it doesn't bother people who just quickly want to set up a sevrer on an ubuntu machine with a few commands, right? Then everything can live in that repository and the person can maintain it all together (and the main FW developers don't need to be bothered with PRs for things we don't know anything about), right?

[almereyda]

Thank you for the details about your Snap packaging. I wasn't aware of it.

To run the container sufficiently, instead of relying on the hardcoded fiduswriter startproject in the Docker repository/-ies, we provide a configuration.py.example and then adapt the generated configuration.py file to the desired local configuration mostly via setting environmental variables. A special trick is played with using the dj-database-url PIP package, that allows to connect to Postgres via passing a convenient DATABASE_URL variable.

The last iteration of that setup is available through https://github.com/moritzfl/docker-fiduswriter/pull/10/files and works sufficently in a production environment scheduled with Docker Compose through libre.sh. It could possibly also be orchestrated with Docker Swarm. The image file should as well run in Kubernetes with minimum effort, which is not an issue here anymore.

[almereyda]

Maybe I was not reading this correctly. I never understood completely that you were offering to have an official docker repository in this organisation, which you brought up twice. Sorry for the debate caused.

This is now https://github.com/fiduswriter/fiduswriter-docker

[johanneswilm]

Yes, it would be greatly appreciated if you could help with that package!

[almereyda]

It's on my radar now, and I will update our instance to it, providing comments and rectifications where needed.

I'm still sorry for having misunderstood you back then.