The brand new Magento2Docker v3

- Introduced a brand new way of using Magento2Docker (single docker-compose.yml file approach);
- Added CLI Tool to manage settings and containers (./bin/m2d);
- Added support for 'bind' type volumes (for Linux users);
- A single version of the project will now be sufficient for both Intel and Apple M1/M2 CPUs;
- Services and tools configured via the .env file;
- Full support for native docker-compose commands;
- Reorganized project structure - no more env and src folders;
- Added friendly setup - now single entry point to set up everything that is mandatory;
- The source folder can be placed anywhere, also outside of the project directory;
- Added all supported versions of all core services used by Magento;
- Fixed documentation;
- Improved and fixed help for m2d command;
- Fixed issue with a configuration file path when it is not yet created;
- Exclude the '.env.back' file from the repository;
- Added option for project suffix;
- Fixed the Mailcatcher configuration;
- Project clean-up;
- Fixed issue about the incorrect extension for docker-compose when the sync method is set to bind;
- Fixed incorrect container name when invoking 'bin/m2d go web';
- Fixed issue about error thrown on Linux when any of multistage image name ends with an underscore symbol;
- Fixed spelling and clarity in prompts, example files, and help;
- Fixed issue with COMPOSER_AUTH variable interfering with auth.json files;
- Fine-tune Mutagen sync options + added new action to fix stuck sync session -> 'bin/m2d sync fix';
- Reorganize Dockerfile for Apache server for better maintainability;
- Fixed issue with fixed configuration for Mailcatcher even if it is not enabled;
- Change hostnames for Mailcatcher and Search Engine from external to internal ones for simplicity;
- Fixed issue about PHP warning about setting memory limit to 0 during the build phase;
- Added crontab, so now Magento cron is one command away: 'bin/magento cron:install';
- Fixed issue with Adobe internal tool setup;
- Fixed error on container build step about missing -y option for apt-get install command;

.env.example

@@ -0,0 +1,136 @@
+##########################################
+##            MAIN SETTINGS             ##
+##########################################
+
+# Set the path on the host that will be mapped to /var/www/html folder in the container
+# e.g. /var/www/html or ~/src/html
+M2D_SOURCE_DIRECTORY='~/src/html'
+# Set the project suffix. It will be used to distinguish different instances of Magento2Docker
+# If you only work with one instance, you can skip it.
+M2D_PROJECT_SUFFIX=''
+# Define a synchronisation type used to sync source directory between host and container
+# - 'mutagen' container will use named volume and Mutagen [https://mutagen.io/] will be used to sync files (default)
+# - 'sshfs' container will use named volume and SSHFS will be used to mount it to the host
+# - 'bind' will binds source directory on host to container (performance on Mac and Windows may be not sufficient)
+M2D_SOURCE_DIRECTORY_SYNC_TYPE='mutagen'
+# Set your CPU type:
+# - 'm1' for Apple M1 CPUs
+# - 'intel' for Intel or AMD based CPUs
+M2D_CPU_TYPE='m1'
+# Set your IDE key for xDebug:
+# - 'PHPSTORM' if you are PHP Storm user
+# - 'VSCODE' if you are Microsoft Visual Studio Code user
+# - '***' you can set any value your IDE requires
+M2D_XDEBUG_IDE_KEY='PHPSTORM'
+# Provide Magento CLI authentication token:
+MAGENTO_CLOUD_CLI_TOKEN=''
+
+##########################################
+##     TOOLS CONFIGURATION SECTION      ##
+##########################################
+
+## MAILCATCHER
+# Set to 'yes' to enable Mailcatcher or to 'no' to disable it:
+M2D_ENABLE_MAILCATCHER='no'
+# Set the port number exposed to the host by the Mailcatcher web interface:
+M2D_PORT_FOR_MAILCATCHER=1080
+
+## BLACKFIRE
+# Set to 'yes' to enable Blackfire or to 'no' to disable it:
+M2D_ENABLE_BLACKFIRE='no'
+
+## SELENIUM
+# Set to 'yes' to enable Selenium or to 'no' to disable it:
+M2D_ENABLE_SELENIUM='no'
+# Set the Selenium version you want to use:
+M2D_SELENIUM_VERSION='3.14.0'
+# Set port number exposed to the host by Selenium HUB:
+M2D_PORT_FOR_SELENIUM_HUB='4444'
+# Set the port number exposed to the host by VNC in the Selenium container:
+M2D_PORT_FOR_SELENIUM_VNC='5900'
+
+## M2INSTALL
+# Provide link to the Magento 2 CE repository:
+M2INSTALL_GIT_CE_REPO='[email protected]:magento/magento2.git'
+# Provide link to the Magento 2 EE repository:
+M2INSTALL_GIT_EE_REPO=''
+
+##########################################
+##    SERVICES CONFIGURATION SECTION    ##
+##########################################
+
+## WEB SERVER
+# Set which Web server engine you want to use:
+# - 'apache' for Apache [https://www.apache.org/]
+M2D_WEB_SERVER_VENDOR='apache'
+# Set which PHP version you want to use:
+# - PHP versions: 7.3, 7.4, 8.1, 8.2
+M2D_WEB_SERVER_PHP_VERSION='8.1'
+# Sets the amount of memory available for PHP (php.ini: memory_limit)
+M2D_WEB_SERVER_PHP_MEMORY_LIMIT='2G'
+# Set the SSH port number exposed to the host:
+M2D_PORT_FOR_WEB_SERVER_SSH='2222'
+# Set the HTTP port number exposed to the host:
+M2D_PORT_FOR_WEB_SERVER_HTTP='80'
+# Set the HTTPS port number exposed to the host:
+M2D_PORT_FOR_WEB_SERVER_HTTPS='443'
+
+## DB ENGINE
+# Set which DB engine you want to use:
+# - 'mariadb' for MariaDB [https://www.elastic.co/]
+# - 'mysql' for MySQL [https://www.mysql.com/]
+M2D_DB_ENGINE_VENDOR='mariadb'
+# Set the expected version of the DB engine:
+# - MariaDB versions: 10, 10.2, 10.3, 10.4, 10.6
+# - MySQL versions: 5, 5.7, 8, 8.0, 8.0-oracle
+M2D_DB_ENGINE_VERSION='10'
+# Set the DB port number exposed to the host:
+M2D_PORT_FOR_DB_ENGINE='3306'
+
+## SEARCH ENGINE
+# Set to 'yes' to enable the search engine or to 'no' to disable it:
+M2D_ENABLE_SEARCH_ENGINE='yes'
+# Set which search engine you want to use:
+# - 'elasticsearch' for ElasticSearch [https://www.elastic.co/]
+# - 'opensearch' for Opensearch [https://opensearch.org/]
+M2D_SEARCH_ENGINE_VENDOR='elasticsearch'
+# Set expected version of the search engine:
+# - ElasticSearch versions: 6, 7, 7.6, 7.7, 7.9, 7.10, 7.16, 7.17, 8, 8.4
+# - OpenSearch versions: 1, 1.2, 2, 2.5
+M2D_SEARCH_ENGINE_VERSION='7'
+# Set the search engine port number exposed to the host:
+M2D_PORT_FOR_SEARCH_ENGINE='9200'
+
+## MESSAGE-BROKER
+# Set to 'yes' to enable message broker or to 'no' to disable it:
+M2D_ENABLE_MESSAGE_BROKER='no'
+# Set which message broker you want to use:
+# - 'rabbitmq' for RabbitMQ [https://www.rabbitmq.com/]
+M2D_MESSAGE_BROKER_VENDOR='rabbitmq'
+# Set the expected version of the message broker:
+# - RabbitMQ versions: 3.8, 3.9
+M2D_MESSAGE_BROKER_VERSION='3.8'
+# Set the broker port number exposed to the host:
+M2D_PORT_FOR_MESSAGE_BROKER='15672'
+
+## WEB CACHE
+# Set to 'yes' to enable web cache or to 'no' to disable it
+M2D_ENABLE_WEB_CACHE='no'
+# Set which web cache solution to use:
+# - 'varnish' for Varnish HTTP Cache [https://varnish-cache.org/]
+M2D_WEB_CACHE_VENDOR='varnish'
+# Set the expected version of the web cache:
+# - Varnish versions: 6, 6.0, 6.2, 6.4, 6.5, 7, 7.0, 7.1
+M2D_WEB_CACHE_VERSION='6'
+# Set the web cache port number exposed to the host:
+M2D_PORT_FOR_WEB_CACHE='8080'
+
+## DB CACHE
+# Set to 'yes' to enable DB cache or to 'no' to disable it
+M2D_ENABLE_DB_CACHE='no'
+# Set which db cache solution to use:
+# - 'redis' for Redis [https://redis.io/]
+M2D_DB_CACHE_VENDOR='redis'
+# Set the expected version of the db cache:
+# - Redis versions: 5, 6, 6.2, 7
+M2D_DB_CACHE_VERSION='6.2'

.gitignore

@@ -1,24 +1,2 @@
-src/*
-!src/index.php
-
-env/.env
-env/etc/composer/auth.json
-!env/etc/git/*
-!env/etc/git/gitconfig.example
-env/etc/git/gitconfig
-
-env/etc/m2install/.m2install.conf.override
-env/docker-compose.override.yml
-
-.idea
-env/etc/apache/*
-!env/etc/vhost.conf.example
-!env/etc/apache/envvars
-!env/etc/apache/000-default.conf
-!env/etc/apache/second-level.nip.io.conf
-
-env/misc/*
-!env/misc/xdebug-php.sh
-!env/misc/prepare-mtf.sh
-
-env/etc/host/config.sh.override
+.env
+.env.back

README.md

@@ -1,19 +1,24 @@
-# Magento2 Docker Environment
-An Ideal Magento2 Development Environment OSX Centric.
+# Magento2Docker Environment
+A near to perfect Magento2 Development Environment OS agnostic, OSX focused.
 Key features of the project:
  - Simple Apache PHP container based on original images.
- - Ideal to work with multiple projects same time
+ - Ideal to work with multiple projects same time.
  - Multi-project setup with clean host names. Based on external nip.io wildcard DNS server.
- - Provides real-time file synchronization by Mutagen
- - Includes PHPStorm container which can be rendered by X.ORG port for OSX
+ - Provides real-time file synchronization by Mutagen.
+ - Includes PHPStorm container which can be rendered by X.ORG port for OSX.
  - Includes great set of tools with zero configuration like Blackfire, XDebug.
- - Includes external services: ElasticSearch 2.x - 6.x, Redis, MailCatcher, RabbitMQ.
- - Provides Make tool as a wrapper. Simplify managing containers and support bash completion to hightlight commands.
-
+ - Includes all external services needed by Magento: ElasticSearch, Opensearch, Redis, Varnish, MySQL, MariaDB, MailCatcher, RabbitMQ, other.
+ - Provides bash CLI tool a wrapper. Simplify managing containers.
+ - Fully compatible with standard docker-compose commands.
+ - Intel and Apple M1 CPU support.
+ - Easily extensible.
+ - Single docker-compose.yaml file approach with .env file to configure everything.
 ## Contents
-
 - [Pre-requirements](#pre-requirements)
+ - [How to install it on Mac](#how-to-install-it-on-mac)
+ - [Directory requirements](#directory-requirements)
 - [Installation](#installation)
+- [Supported services and tools](#supported-services-and-tools)
 - [Usage](#usage)
  - [Quick Start](#quick-start)
  - [How to install a magento](#how-install-magento)
@@ -29,72 +34,120 @@ Key features of the project:
 ## Pre-requirements
  - [Install Docker](https://docs.docker.com/engine/installation/mac/)
  - [Install Mutagen](https://mutagen.io/documentation/introduction/installation/)
- - [Install bash completion (optional)](https://github.com/bobthecow/git-flow-completion/wiki/Install-Bash-git-completion)
-
+ - Bash > 4.0
+ - Realpath
+
+ ### How to install it on Mac
+```bash
+# Mutagen:
+brew install mutagen-io/mutagen/mutagen
+# Bash:
+brew install bash
+# Realpath:
+brew install coreutils
+```
+
+### Directory requirements
+1. Magento2Docker as a tool can be located in any directory, e.g. ~/tools/m2d
+2. Web container will use /var/www/html as its root folder for web content.
+3. The M2D_SOURCE_DIRECTORY must point to a local path where you want to sync /var/www/html from the web container.
+4. The M2D_SOURCE_DIRECTORY must not point to symlink or it will trigger errors when Mutagen is responsible for sync process.
+
 ## Installation
 You can download archive of this project on [Release Page](https://github.com/yvoronoy/magento2docker/releases).
 
  - Clone or Download the repository ```git clone [email protected]:yvoronoy/magento2docker.git```
- - Copy or create `env/etc/composer/auth.json` and put your [Access Keys](http://devdocs.magento.com/guides/v2.0/install-gde/prereq/dev_install.html)
-   - `cp env/etc/composer/auth.json.example env/etc/composer/auth.json`
-   - Edit env/etc/composer/auth.json and put your credentials [Access Keys](http://devdocs.magento.com/guides/v2.0/install-gde/prereq/dev_install.html)
- - Update your gitconfig if needed
-   - `cp env/etc/git/gitconfig.example env/etc/git/gitconfig`
- - Update and edit Commerce Cloud CLI Token
-   - `cp env/.env.example env/.env`
- - (Optional) Copy your private ssh keys, configs to have access to resources from inside container
-    - `cp ~/.ssh/id_rsa env/etc/ssh/`
-    - `cp ~/.ssh/config env/etc/ssh/`
+ - Go to project folder
+ - Execute setup creator `./bin/m2d setup init`
+ - Optional: fine tune all setings by editing `.env` file
+
+## Supported services and tools
+### Web Servers
+- Apache with PHP: 7.3, 7.4, 8.0, 8.1, 8.2
+
+### DB Engines
+- MySQL: 5.7, 8.0, 8.0-oracle, and 5.7 as 5, 8.0.28 as 8
+- MariaDB: 10.2, 10.3, 10.4, 10.6, and 10.4 as 10
+
+### Search Engines
+- Opensearch: 1.2.4 as 1.2, 1.2.4 as 1, 2.5.0 as 2.5, 2.5.0 as 2
+- Elasticsearch: 6.8.23 as 6, 7.16.3 as 7, 7.6.2 as 7.6, 7.7.1 as 7.7, 7.9.3 as 7.9, 7.10.1 as 7.10, 7.16.3 as 7.16, 7.17.9 as 7.17, 8.4.3 as 8, 8.4.3 as 8.4
+
+### DB Cache Engines
+- Redis: 6.2, 5.0 as 5, 6.0 as 6, 7.0 as 7
+
+### Web Cache Engines
+- Varnish: 6.0, 6.2, 6.4, 6.5, 7.0, 7.1, and 7.0 as 7, 6.5 as 6
+
+### Tools
+- Blackfire: latest
+- Mailcatcher: latest
+- Selenium: 3.14.0
+
 
 ## Usage
+To work with Magento2Docker you can use `m2d` CLI command located in `bin` of Magento2Docker project.
 ### Quick Start
-Commands should be executed from _env_ directory.
-Run make command to run environment.
+With Magento2Docker v3 it is super easy to start or stop containers:
+```bash
+# Display help
+./bin/m2d --help
 
-```
-# Build and mount containers (default: php-7.4)
-bin/up
+# To start containers:
+./bin/m2d up
+
+# To stop containers:
+./bin/m2d stop
+
+# To stop and remove containers and networks:
+./bin/m2d down
 
-# Login on web server container
-bin/shell
+# To enable service e.g. mailcatcher
+./bin/m2d enable mailcatcher
 
-# Change php version
-  - For php-7.3: make up73
-  - For php-7.4: make up74
-  - For php-8.0: make up80
-  - For php-8.1: make up81
+# To disable service e.g. elasticsearch:
+./bin/m2d disable search-engine
+
+# To edit any .env parameter (e.g. PHP version):
+./bin/m2d set M2D_WEB_SERVER_PHP_VERSION 8.1
+
+# To display any .env parameter (e.g. search engine type):
+./bin.m2d show M2D_SEARCH_ENGINE_VENDOR
+
+# To start containers after enabling or disabling services or editing parameters:
+./bin/m2d up --buld
 ```
 
+
 ### How to install a magento inside container
-   - Login to container `make web`
+   - Login to container `./bin/m2d go web`
    - Create a directory e.g: magento2
-   - Inside the magento2 directory run `m2install.sh -s composer -v 2.3.3`
+   - Inside the magento2 directory run `m2install.sh -s composer -v 2.4.6`
    - Open browser and go to http://magento2.127.0.0.1.nip.io/
 
 ### How to deploy dumps (backups) inside container
  - Put dumps to src folder on your host machine
-   - Login to container `make web` 
+   - Login to container `./bin/m2d go web`
    - Run `m2install.sh`
 
 ## How to link Composer versions
 
-Containers for PHP 7.4, PHP 8.0, and PHP 8.1 has Composer 2 because of Magento supports Composer 2 since 2.4.2
-version.
+Containers with PHP 7.x have Composer 1 set as default, containers with PHP 8.x have Composer 2 set as default version.
 
 ### Usage:
-- Login to your container `bin/shell-root`
+- Login to container `./bin/m2d go web`
 - To use composer as default you have two commands:
-  - Run command `composer-link.sh 1` to use Composer 1 
-  - Run command `composer-link.sh 2` to use Composer 2 
+  - Run command `sudo composer-link.sh 1` to use Composer 1
+  - Run command `sudo composer-link.sh 2` to use Composer 2
 
 ## How to Enable xDebug
 
 The container already includes PHP xDebug extension. The xDebug extension is disabled by default because
 it is dramatically decrease performance.
 
 ### Usage
- - Login to your container `make web`
- - Run command `xdebug-php.sh 1`
+ - Login to your container `./bin/m2d go web`
+ - Run command `sudo xdebug-php.sh 1`
  - Run IDE (PHPStorm) and press button _Start Listening for PHPDebug Connection_
 
 
@@ -110,13 +163,14 @@ it is dramatically decrease performance.
 Profiling with Blackfire is on-demand. This means that Blackfire adds no overhead for your end users, which makes it safe to use in production.
 
 ### Get your Blackfire credentials
-Blackfire provides you a free account "Hack" which allows you to run profiles on your development environment. 
+Blackfire provides you a free account "Hack" which allows you to run profiles on your development environment.
  - Create account and login here: https://blackfire.io/login
  - Install Browser Extension https://blackfire.io/docs/integrations/chrome
  - Go to the page https://blackfire.io/docs/integrations/docker
    - Define these environment variables from this page on the host system (OSX)
    - You can save them permanently by putting them into ~/.bash_profile file
- - Recreate containers by using command `make up`
+ - Enable blackfire container `./bin/m2d enable blackfire`
+ - Recreate containers by using command `./bin/m2d up --build`
 
 ## How to run PHPStorm inside container
 
@@ -130,18 +184,17 @@ Blackfire provides you a free account "Hack" which allows you to run profiles on
 Run the following command inside env directory
 
 ```
-make phpstorm
+docker-compose -f docker-compose.phpstorm.yml up -d --build
 ```
 
 ## Todo List
 
 ## Contributing
 1. Fork this repository.
-2. Create your feature branch: `git checkout -b my-new-feature`.
+2. Create your feature branch: `git checkout -b feature/my-cool-feature`.
 3. Commit your changes: `git commit -am 'Add some feature'`.
-4. Push to the branch: `git push origin my-new-feature`.
+4. Push to the branch: `git push origin feature/my-cool-feature`.
 5. Submit a pull request.
 
 ## Credits
 Special thanks to @snosov and @tshabatyn who share their ideas and inspired to build this project.
-