Interest in an official docker?

[Bockiii]

Hi,

I can see that there is an inofficial docker with ~100k pulls on dockerhub.

Are you interested in an official docker container provided by you? I already started a PR on this but since I wanted to integrate it to into your release workflow, it's taking a bit longer (it could be done quicker, but then it's not integrated). But before I spend more time on it, I wanted to know if you are even interested in that.

So it would require you create an account on dockerhub and in addition to that, it would also create the image on the github repo.

I've set this up for another application with a similar release workflow than yours and would configure it to be "latest" (actually last commit), stable (last released version) and versionized tags (e.g. jamulus:3.8.1).

[rdica]

Isn't dockerhub now limiting pulls and integrations for non-commercial use?

[Bockiii]

Correct, thats why you can offer both with a single build process. Offering it on dockerhub doesn't bring you any downsides, but some heavier users might run into the limits.

Also, autobuilding isnt supported anymore, but thats done via github action anyways.

[Bockiii]

The other thing I've setup: https://github.com/RSS-Bridge/rss-bridge/blob/master/.github/workflows/dockerbuild.yml

[Bockiii]

This builds for x64, armv7 and armv8 btw which would also be directly available to you. so people can run their server on a pi with very little effort.

[gilgongo]

We're not a very big team on Jamlus, so I'd be a bit wary of having to maintain a Docker build alongside the other ones, particularly if it involves a 3rd party account to keep hold of. Not sure what others think.

[pljones]

I wouldn't object to someone contributing a Dockerfile. I think, once working, it should be fairly stable.

I would probably not be comforable about having the build process push a Docker image to the hub.

[Bockiii]

It doesnt necessarily need a 3rd party account, as you could skip dockerhub all together. It's just "the" repo to search for things, so if someone would look for a docker image of jamulus, they would search for it on dockerhub and not go to the repo to see if there is an image in the container repo on github. So you could keep the image on github if you want to, it just means less visibility.

As for the build process pushing: That's very common practice today. Since dockerhub limited the use of their build system (because of bad actors who did crypto mining using the build system), most docker containers moved their build systems to github/gitlab and then push to dockerhub.

I also agree: If you dont switch up something dramatic in the background, the docker image should be pretty stable.

[gilgongo]

BTW There is this ticket if anyone wants to maintain a page summarising the many and various methods of Linux installation for Jamulus. Over 15 years it's built up quite a few (although I don't think Linux use in general has been more than 5% of downloads).

[ann0see]

But that’s up to @jamulussoftware/maindevelopers

[Bockiii]

We would need to do changes to the current dockerfile because of the different build process but in general, sure that would work. It wouldn't help with the detached situation though :)

[Bockiii]

I have used most of grundics dockerfile as the base for the "new" image.

https://github.com/Bockiii/jamulus/pkgs/container/jamulus

Right now it's basically the same functionality (the only thing I really changed was upping the alpine version and using the lsio alpine for the runner since it provides things like s6 etc) except that it is already available in multi-arch.

The next thing I want to do is provide a different option to do the settings than entrypoint extensions, but that will require some scripting and some thoughts as I understand that there is no config-file I could just parse.

[melcon]

Right now it's basically the same functionality (the only thing I really changed was upping the alpine version and using the lsio alpine for the runner since it provides things like s6 etc) except that it is already available in multi-arch.

Do you know by any chance if that lsio alpine has support for aws-efs-utils? Or have you tried to add support for it yourself?

[Bockiii]

Is there a reason why you're not mounting the efs drive on the host and then just mount the volume? I assume you want to share it between containers, so... why not mount on the host?

A container is usually a do-one-thing kind of deal. So if you need extra things to mount something, it would make more sense to use a container thats specific to do that mounting and share a volume between that container and the application container.

To give you an example: You dont add a VPN capability to a torrent-client container. You create a vpn container and route the torrent-client container through that vpn container.

[gene96817]

I don't have any hands-on docker experience. I would be interested in running 4+ server instances in a single (hw) server. It would be awesome to know how to size each docker instance for a max number of participants (where the max is not all the same). Consider four containers, each for 8 participants, and one container for 40 participants.

[gene96817]

Thanks for the long response. Too bad a container does not enforce a resource constraint. I don't expect containers to create more capacity. I was hoping that a number of containers can be created so that groups with low utilization can have their own container without worrying about crashing the server when all the resources are used. Of course, scheduling and coordination is a different function.

Using Kubernetes to spin up new instances doesn't preserve the behavior of a public server. I would like to see public servers available and listed on the directory server. Since I expect the public server has a low utilization rate, the (hw) server can keep a number of instances running with low resource utilization because utilization is low. (I know this creates new challenges. I consider them good side effects.)

[Bockiii]

Too bad a container does not enforce a resource constraint.

Thats not what I said. You can set container max-usages for CPU, RAM, disk space and so on, but that's not really what you would need to solve your initial problem.

From your theroretical setup, I assume you know you have a limit of people that your hardware (cpu+ram) supports. Lets say that's 25. what you could do is configure 5 containers (configured to host maximum of 5 people) to use a maximum of 0.2 of your CPU (so 20%) and ram. That way, no container on it's own can use more than 20% of the hardware and all containers can run at full capacity (5*5, 25 people). The problem with this setup is: The max-cpu setting will not really do anything, as you have already limited the people to 5 per instance. If you set the max-people value higher (lets say 10 people per instance), you will run into problems with your 20% setting per instance (as your hardware is only capable of hosting 25 all together ... see where I'm going?)

As for crashing the host because of docker container utilization: I've honestly never heard of that, but of course it could happen. I've been running containerization on metal for a few years now (from single docker runs to docker-compose to k3s) and never had a host crash when a container crashed (I'm running plex so yes... containers have crashed :D ).

As for the public hosting: that should 100% be doable with a reverse proxy and a service like cloudflare dns. I have a domain, dns managed by cloudflare, traefik reverseproxy configured and a cloudflare-helper-container that immediately creates subdomains for containers I spin up. If I start a jamulus instance, its instantly available under jamulus.mydomain.com . You can do jamulus1.mydomain, jamulus2.mydomain etc.

[gene96817]

@Bockiii Thank you for this dialog.

You understand my scenario. I would add one twist. There are more containers and it is assumed that not all containers can have users, thus not exceeding the hardware resource limits.

Clearly, I don't have any experience with docker. This topic is my next Jamulus frontier. Is anyone interested in working with me on testing this concept?

[Bockiii]

If you want a good guide to start, follow this:

https://www.smarthomebeginner.com/traefik-2-docker-tutorial/

If you read it carefully (and follow his links to topics that dont tell you anything) you can get a lot of this running relatively quickly (minus the kubernetes orchestration part. but thats for V2 I would say ;) ).

[gilgongo]

@gene96817 In your case it perhaps might be better to run the servers just using systemd with nice and ionice settings. Resource allocations would be calculated from a combination of the different processes' niceness values, plus other things like the I/O of each, not just a simple percentage. Like @Bockiii says, the main advantage of using docker is so that you don't have to worry about installing libraries and things on systems that don't have them - it's not about process tuning.

[gene96817]

@gilgongo What do I know about each instance? What files can be shared between instance and which files have to be unique? Thanks.

[gilgongo]

I've not tried it (and this doesn't involved Docker) but if you create a bash script for each of your servers containing the desired startup and nice value options, eg: jamulusA.sh, jamulusB.sh etc. each containing something like:

 nice -n [niceValue] /usr/local/bin/jamulus-headless  \
        --directoryserver genreServer:port \
        --serverinfo "yourServerName;yourCity;[country ID]"
	... [etc]

then have a systemd "template" unit file with a %i variable in it:

ExecStart=/bin/sh -c 'exec /usr/local/bin/jamulus%i.sh'

You can then control them with systemctl [start/stop] [email protected] systemctl [start/stop] [email protected] etc. The bit directly after the @ is used by the %i variable in the unit file. A more detailed illustration is here.

[pljones]

I do something much more complex in https://github.com/pljones/jamulus-systemd