[10.x] Add support for custom $PATH's (like DBngin) for the Schema Dump/import command

[ralphjsmit]

In short: this PR allows developers to set a custom path for the database binaries using a bin option in the database connection config, in situations where developers have the mysql, mysqldump pg_dump, pg_restore or sqlite3 commands not automatically in their $PATH, like when using DBngin.

Background

I'm using DBngin to create and manage databases like MySQL and Postgres. Since DBngin manages its installation completely by itself and separate from the "normal" mysql bin in the system (if you have it).

I don't have mysql installed on my system in a general bin directory, since I've been always using DBngin and never had a reason for it.
However, this poses a problem when running artisan schema:dump or when importing the dump:

The command "mysql  --user="${:LARAVEL_LOAD_USER}" --password="${:LARAVEL_LOAD_PASSWORD}" --host="${:LARAVEL_LOAD_HOST}" --port="${:LARAVEL_LOAD_PORT}" --database="${:LARAVEL_LOAD_DATABASE}" < "${:LARAVEL_LOAD_PATH}"" failed.

Exit Code: 127(Command not found)

Working directory: /Users/ralphjsmit/Documents/Code/Sites/cj

Output:
================


Error Output:
================
sh: mysql: command not found

In order to solve this problem, DBngin offers a special button to export the environment variables to the terminal. This will open a terminal and paste in the following code:

export PATH=/Users/Shared/DBngin/mysql/8.0.19/bin:$PATH

If you then run the artisan schema:dump command directly after the export, it will work, because we now gave an explicit pointer to the location of the mysql command.

However, if you have tests that are running on MySQL as well, this won't work. Because when running tests (also e.g. via PhpStorm) it won't have the export ... command before the test command.

Summary

This PR aims to solve this problem by providing developers with a way to register a custom location of the mysql, mysqldump pg_dump, pg_restore or sqlite3 command using a 'bin' option in the config/database.php.
I have provided a PR laravel/laravel#6268 to keep the skeleton up-to-date as well.

The PR also adds a full new test for the Postgres schema state, which wasn't tested before.

Thanks!

[rodrigopedra]

Well, you are supposed to add this export command to your ~/.profile file, so this path is available globally on your bash session.

Mind that, depending on the shell you are using, this file might be ~/.bashrc, ~/.zshrc, or some other.

And if you want it available globally you can add this command to your /etc/environment file (using sudo).

In Windows you can add custom binary parhs through Control Panel (at least it used to be there, there are some years I don't use Windows for any development task).

In my opinion, having custom binary paths is not something that should be handled by the framework.

People could have custom paths for many things, like Redis, Memcached, and others.

How multiple paths should be handled?

This is an environment detail, that should be handled by the environment, not by the framework.

In this particular case, if you are running a Linux/Unix/macOS box, you can also use a unix socket, which Laravel already supports to have custom paths. There is some performance gains by using a socket, as there is no TCP negotiation involved.

[taylorotwell]

See above.

[ralphjsmit]

@rodrigopedra Thanks!

I know about setting up the .zshrc globally and I also get your part about "how many paths should be handled". However, for DBngin's case this path is supposed to be dynamic and not hard-coded anywhere, so I'm a bit weary to add it to my global .zshrc. In order to solve this then, is there a way to like e.g. create a .zshrc per project/folder, which I can then put in the root of my project?

[rodrigopedra]

However, for DBngin's case this path is supposed to be dynamic

Well, from their GitHub issue tracker, this would be the case, if you are using it to run multiple database versions.

Anyway, it seems they are considering having the export added to the user's $PATH

TablePlus/DBngin#117 (comment)

I didn't know DBngin from before, but from the example in that issue, it seems the path only changes when you use a different database version:

export PATH="$PATH:/Users/Shared/DBngin/postgresql/15.1/bin"

And, in that case, you would need to update your project's settings anyway, by your proposal.

This is similar to about running a database server from docker. Unless one maps the binary to a known location, one will need to add a custom directory to their $PATH

Mind that adding to your .zshrc is only "globally" regarding the current user session. If you share your computer with someone else, or use a different user account for personal tasks, the other user's $PATH won't be affected.

Also, if you are running, for example, multiple MySQL versions, you could add multiple directories to the path, so your OS will try to find the binary on each of them.

Something like this (untested, just as an example, not even sure there is a postgresql 16.0)

export PATH="$PATH:/Users/Shared/DBngin/postgresql/15.1/bin:/Users/Shared/DBngin/postgresql/16.0/bin"

From what I could understand, on how DBngin works, unless you have multiple versions of a DB running at the same time, only one of these paths will have the binary when a particular version is running, right?

Is it a must to have multiple DB versions, running at the same time? In a manner, that having multiple directories added to your path, would cause a conflict?

As a last resource, you can add an event listener to the Illuminate\Console\Events\ArtisanStarting, or the Illuminate\Console\Events\CommandStarting events, that run that export command for the current artisan run.

Your listener can read those binaries from a config file if you wish.

It would look like this, from a Service Provider's boot method:

Event::listen(ArtisanStarting::class, function () {
    if ($path = config('app.my_cutom_db_path')) {
        exec(sprintf('export PATH = "$PATH:%s"', $path));
    }
});

I hope this helps =)

[ralphjsmit]

Thanks! No, in case it's not possible to e.g. have a folder-specific .zshrc then I guess I need to add it globally.
The thing with DBngin is also that each database server takes the MySQL version that was available at the moment the server was created.

So by now I am running 3 different versions 😄

You can't upgrade these individual servers, so the only way would be to create new services, export db and import a dump again. That's quite time-consuming though, so unless I have a reason to upgrade, I won't.

But anyway, I understand that it's not the intention of the framework framework's to take responsibility for all different paths, so I'll just work with it.

Thanks!

[rodrigopedra]

Ah, ok, I didn't know it worked like that... and does the event listener solution work?

If it does, you could make it into a package

[dennisprudlo]

@ralphjsmit Quick off topic question: Is there any reason you have multiple server instances for the same version? What distinguishes these instances?

[ralphjsmit]

I now added the latest DBngin MySQL version in my .zshrc, but I don't like it and it still doesn't work with PhpStorm testing feature. Using the exec()-things feels kind of dirty tbh.

[ralphjsmit]

@dennisprudlo When creating a new database, DBngin takes the latest MySQL version at the moment of creation of the service. There is no way to upgrade a version, unless you manually migrate by creating a new service to replace the old one and export/import all databases etc. So that's why you see these different versions. No functional reason for it therefore.

[dennisprudlo]

@ralphjsmit then why not simply creating the databases for an existing server instance? I use a single MySQL 8 DBngin server which has all databases for different projects I am developing. This could reduce some running services on your device – not that this is the issue at hand here 😅

[ralphjsmit]

@dennisprudlo Thanks, yeah, that is indeed an option. I don't have all services activated at the same time though :) But I am a freelancer and therefore have multiple projects that I work. For my main project I automatically start the service on login and for the other projects I activate the service if I want to work on that project. I like to keep things organized, so therefore I have a different MySQL service for each different client, plus some general-purpose DBs for myself (and some client DBs I still need to clean up ;)).

[dennisprudlo]

@ralphjsmit alright, thanks for the clarification. I thought that maybe I am missing something and theres is a functional difference when using one service per project. But I see the organizational point 👍🏼