Separating dev tools from deploy tools

[camertron]

This discussion pertains to the article on the Evil Martians blog entitled "Kubing Rails: stressless Kubernetes deployments with Kuby". It is part of a series of discussions pertaining to the article that will hopefully help improve Kuby and its ecosystem.

The Problem

In the article, Vladimir lays out the case for separating dev and deployment dependencies by creating several additional gemfiles for Kuby-specific deps. From what I understand, this will help avoid loading the entire bundle when running kuby commands, since we don't really need any of the Rails gems, etc when deploying and building Docker images.

Unfortunately this also means kuby-core isn't part of the Rails app's Gemfile, so two things happen:

1. Kuby isn't able to auto-detect the presence of a webserver. Under normal circumstances, Kuby looks for a webserver gem in the list of loaded gemspecs. Currently the only supported webserver is Puma, which I believe is also the Rails default.
2. The health check middleware Kuby automatically injects into the Rails app isn't available, so deployments fail.
3. The kuby:rails_app:assets:copy and kuby:rails_app:db:create_unless_exists tasks live in kuby-core and therefore aren't available during the Docker build.

Potential Solutions

Originally, Kuby was designed to tightly integrate with Rails. Since Kuby isn't a dev tool, I thought developers would already have the entire bundle installed on their computers and could execute simple bundle exec kuby commands. In practice however, it appears most people deploy their apps from a CI system like GitHub Actions. In CI scenarios, only a few gems are necessary: kuby-* and activesupport. It feels wasteful of both time and compute resources to set up all an app's dependencies in CI when only a few will actually get used.

I really like the idea of separating app/dev and deployment dependencies by using separate gemfiles, so I think that's the direction Kuby should go. It should be pretty easy to modify Kuby's Rails generator to spit out a few extra gemfiles and the bin/kuby executable.

Detecting the Webserver

Kuby should be able to parse the app's lockfile using Bundler::LockfileParser and check for the presence of Puma (and other webserver gems). Unfortunately there's not an easy way to know which group the webserver gem is in, but hopefully that doesn't matter much in practice.

Health Check Middleware

In the article, Vladimir describes using the rack-health gem instead of relying on Kuby's built-in health check middleware. That could be a good solution, but I'm wondering if it makes sense to create a small runtime gem (with only a dependency on railties) that would be added to the app's gemfile (maybe named kuby-runtime or kuby-rails-runtime?) The runtime gem would contain the two rake tasks mentioned above as well as the health check middleware, and could configure them automatically. I like this approach better than bundle adding rack-health because 1) it's automatic, and 2) a runtime gem provides a nice vector through which we can add other behavior if/when necessary.

/cc @palkan

[palkan]

using the rack-health gem instead of relying on Kuby's built-in health check middleware. That could be a good solution, but I'm wondering if it makes sense to create a small runtime gem

IMO, it makes sense. I wish adding all necessary app dependencies to be as easy as dropping kuby-rails-runtime gem to the Gemfile (which adds Rake tasks, middlewares, etc.).

[camertron]

Ok cool, I'll work on that.

I do have another question for you, @palkan. Re-reading the article, you mention including eval_gemfile "gemfiles/kuby.gemfile" in your main Gemfile. Correct me if I'm wrong, but doesn't that mean Kuby and all its dependencies are available to the Rails app? If so, then what's the reason for separating the Kuby deps out into their own .gemfile? Moreover, the health check middleware etc will be available to the app when it runs. So what is the separation actually doing aside from loading fewer gems when running bin/kuby commands?

[palkan]

you mention including eval_gemfile "gemfiles/kuby.gemfile" in your main Gemfile

That was an intermediate step. The final Gemfile doesn't include kuby.gemfile: https://github.com/anycable/anycable_rails_demo/pull/25/files#diff-d09ea66f8227784ff4393d88a19836f321c915ae10031d16c93d67e6283ab55f