From b2f1bdcc273332b27500790135428313a60d031d Mon Sep 17 00:00:00 2001 From: "Chinmay D. Pai" Date: Mon, 2 Mar 2020 10:59:25 +0530 Subject: [PATCH] chore: clean up and update readme Signed-off-by: Chinmay D. Pai --- README.md | 247 +++++++++++++++++++++++++++++++++-------- installation/README.md | 193 -------------------------------- 2 files changed, 203 insertions(+), 237 deletions(-) delete mode 100644 installation/README.md diff --git a/README.md b/README.md index b51b0885..611cde45 100644 --- a/README.md +++ b/README.md @@ -1,73 +1,232 @@ -# Frappe on Docker +### Getting Started -[![Build Status](https://travis-ci.com/frappe/frappe_docker.svg)](https://travis-ci.com/frappe/frappe_docker) +The templates in this repository will help deploy Frappe/ERPNext docker in a production environment. -This is a repo designed to aide setting up frappe/ERPNext on docker. +This docker installation takes care of the following: -## Getting Started +* Setting up the desired version of Frappe/ERPNext. +* Setting up all the system requirements: eg. MariaDB, Node, Redis. +* [OPTIONAL] Configuring networking for remote access and setting up LetsEncrypt -These instructions will get you a copy of the project up and running on your local machine for development and testing purposes. +### Installation Process -Unfortunately, this container is not currently suited for a production environment (but we're working towards that goal!). +#### Setting up Pre-requisites -### Build the container and initialize the bench +This repository requires Docker and Git to be setup on the instance to be used. -**Note:** These instructions assume you have both [Docker](https://docs.docker.com/engine/installation) and [Docker Compose](https://docs.docker.com/compose/install/) installed on your system. +#### Setup Letsencrypt Nginx Proxy Companion -1. Clone this repo and change your working directory to it: +This is an optional first step. This step is only required if you want to have SSL setup on your docker instance. - ```bash - git clone https://github.com/frappe/frappe_docker.git - cd frappe_docker/ - ``` +This step also assumes that the DNS is preconfigured since it automatically handles setup and renewal of SSL certificates. -2. Build and start the container, and initialize the bench: +For more details, see: https://github.com/evertramos/docker-compose-letsencrypt-nginx-proxy-companion - ```bash - ./dbench setup docker - ./dbench init - ``` +To setup the proxy companion, run the following steps: - **Note:** This will take a while, as docker will now build the container. +```sh +cd $HOME +git clone https://github.com/evertramos/docker-compose-letsencrypt-nginx-proxy-companion.git +cd docker-compose-letsencrypt-nginx-proxy-companion +cp .env.sample .env +./start.sh +``` -3. Add a new site and start Frappe: +#### Setting up Frappe/ERPNext Docker - ```bash - ./dbench new-site site1.local - ./dbench setup hosts - ./dbench start - ``` +Clone this repository somewhere in your system: -4. Use Frappe: - Open your browser to `localhost:8000/login`. Then log in using the username `Administrator` and the password `admin`. +```sh +git clone https://github.com/frappe/frappe_docker.git +cd frappe_docker +``` -### Basic Usage of `./dbench` +Copy the example docker environment file to `.env`: -**IMPORTANT: Always make sure that your current directory is the root directory of the repo (i.e. `frappe_docker/`)** +``` +cp installation/env-example installation/.env +``` -- `./dbench`: Launches you into an interactive shell in the container as the user `frappe`. +Make a directory for sites: -- `./dbench setup docker [ stop | down ]`: Starts and builds the docker containers using `docker-compose up -d`. - - `stop`: Stops the containers with `docker-compose stop`. - - `down`: Deletes the containers and the corresponding volumes with `docker-compose down`. +```sh +mkdir installation/sites +``` -- `./dbench setup hosts`: Adds all sites to the containers hosts file. - **Note:** Run this after you've added a new site to avoid errors. +#### Setup Environment Variables -- `./dbench -c frappe | root `: Runs a command in the container, as the selected user. +Docker allows passing an environment file to aide in setting up containers, which is used by this repository to pass secret and variable data. -- `./dbench -h`: Shows this help message. +To get started, copy the existing `env-example` file to `.env` inside the `installation` directory. By default, the file will contain the following variables: -- `./dbench `: Runs a command in bench (i.e. Running `./dbench new-site site1.local`, will run `bench new-site site1.local` in the container). +- `VERSION=edge` + - In this case, `edge` corresponds to `develop`. To setup any other version, you may use the branch name or version specific tags. (eg. version-12, v11.1.15, v11) +- `MYSQL_ROOT_PASSWORD=admin` + - Bootstraps a MariaDB container with this value set as the root password. If a managed MariaDB instance is to be used, there is no need to set the password here. +- `MARIADB_HOST=mariadb` + - Sets the hostname to `mariadb`. This is required if the database is managed with the containerized MariaDB instance. + - In case of a separately managed database setup, set the value to the database's hostname/IP/domain. +- `SITES=site1.domain.com,site2.domain.com` + - List of sites that are part of the deployment "bench". Each site is separated by a comma(,). + - If LetsEncrypt is being setup, make sure that the DNS for all the site domains are pointing to the current instance. +- `LETSENCRYPT_EMAIL=your.email@your.domain.com` + - Email for LetsEncrypt expiry notification. This is only required if you are setting up the nginx proxy companion. -## For More Info +#### Start Frappe/ERPNext Services -For more info on building this docker container refer to this [Wiki](https://github.com/frappe/frappe_docker/wiki/Hitchhiker's-guide-to-building-this-frappe_docker-image) +To start the Frappe/ERPNext services, run the following command: -## Contributing +```sh +docker-compose \ + --project-name \ + -f installation/docker-compose-common.yml \ + -f installation/docker-compose-erpnext.yml \ + -f installation/docker-compose-networks.yml \ + --project-directory installation up -d +``` -Feel free to contribute to this project and make it better. +Make sure to replace `` with any desired name you wish to set for the project. -## License +Note: use `docker-compose-frappe.yml` in case you need bench with just frappe installed. -This project is licensed under the MIT License - see the [LICENSE.md](LICENSE.md) file for details +#### Setup New Sites + +Note: + +- Wait for the mariadb service to start before trying to create a new site. + - If new site creation fails, retry after the mariadb container is up and running. + - If you're using a managed database instance, make sure that the database is running before setting up a new site. +- Use `.env` file or environment variables instead of passing secrets as command arguments. + +```sh +# Create ERPNext site +docker exec -it \ + -e "SITE_NAME=$SITE_NAME" \ + -e "DB_ROOT_USER=$DB_ROOT_USER" \ + -e "MYSQL_ROOT_PASSWORD=$MYSQL_ROOT_PASSWORD" \ + -e "ADMIN_PASSWORD=$ADMIN_PASSWORD" \ + -e "INSTALL_ERPNEXT=1" \ + _erpnext-python_1 docker-entrypoint.sh new +``` + +Environment Variables needed: + +- `SITE_NAME`: name of the new site to create. +- `DB_ROOT_USER`: MariaDB Root user. The user that can create databases. +- `MYSQL_ROOT_PASSWORD`: In case of mariadb docker container use the one set in `MYSQL_ROOT_PASSWORD` in previous steps. In case of managed database use appropriate password. +- `ADMIN_PASSWORD`: set the administrator password for new site. +- `INSTALL_ERPNEXT=1`: available only in erpnext-worker and erpnext containers. Installs ERPNext on this new site. +- `FORCE=1`: is optional variable which force installs the same site. + +#### Backup Sites + +Environment Variables + +- `SITES` is list of sites separated by (:) colon to migrate. e.g. `SITES=site1.domain.com` or `SITES=site1.domain.com:site2.domain.com` By default all sites in bench will be backed up. +- `WITH_FILES` if set to 1, it will backup user uploaded files for the sites. + +```sh +docker exec -it \ + -e "SITES=site1.domain.com:site2.domain.com" \ + -e "WITH_FILES=1" \ + _erpnext-python_1 docker-entrypoint.sh backup +``` + +Backup will be available in the `sites` mounted volume. + +#### Updating and Migrating Sites + +Switch to the root of the `frappe_docker` directory before running the following commands: + +```sh +# Update environment variable VERSION +nano .env + +# Pull new images +docker-compose \ + -f installation/docker-compose-common.yml \ + -f installation/docker-compose-erpnext.yml \ + pull + +# Restart containers +docker-compose \ + --project-name \ + -f installation/docker-compose-common.yml \ + -f installation/docker-compose-erpnext.yml \ + -f installation/docker-compose-networks.yml \ + --project-directory installation up -d + +docker exec -it \ + -e "MAINTENANCE_MODE=1" \ + _erpnext-python_1 docker-entrypoint.sh migrate +``` + +### Troubleshoot + +1. Remove containers and volumes, and clear redis cache: + +This can be used when existing images are upgraded and migration fails. + +``` +# change to repo root +cd $HOME/frappe_docker + +# Stop all bench containers +docker-compose \ + --project-name \ + -f installation/docker-compose-common.yml \ + -f installation/docker-compose-erpnext.yml \ + -f installation/docker-compose-networks.yml \ + --project-directory installation stop + +# Remove redis containers +docker-compose \ + --project-name \ + -f installation/docker-compose-common.yml \ + -f installation/docker-compose-erpnext.yml \ + -f installation/docker-compose-networks.yml \ + --project-directory installation rm redis-cache redis-queue redis-socketio + +# Clean redis volumes +docker volume rm \ + _redis-cache-vol \ + _redis-queue-vol \ + _redis-socketio-vol + +# Restart project +docker-compose \ + --project-name \ + -f installation/docker-compose-common.yml \ + -f installation/docker-compose-erpnext.yml \ + -f installation/docker-compose-networks.yml \ + --project-directory installation up -d +``` + +2. Clear redis cache using `docker exec` command: + +In case of following error during container restarts: + +``` +frappe-worker-short_1 | Traceback (most recent call last): +frappe-worker-short_1 | File "/home/frappe/frappe-bench/commands/worker.py", line 5, in +frappe-worker-short_1 | start_worker(queue, False) +frappe-worker-short_1 | File "/home/frappe/frappe-bench/apps/frappe/frappe/utils/background_jobs.py", line 147, in start_worker +frappe-worker-short_1 | Worker(queues, name=get_worker_name(queue)).work(logging_level = logging_level) +frappe-worker-short_1 | File "/home/frappe/frappe-bench/env/lib/python3.7/site-packages/rq/worker.py", line 474, in work +frappe-worker-short_1 | self.register_birth() +frappe-worker-short_1 | File "/home/frappe/frappe-bench/env/lib/python3.7/site-packages/rq/worker.py", line 261, in register_birth +frappe-worker-short_1 | raise ValueError(msg.format(self.name)) +frappe-worker-short_1 | ValueError: There exists an active worker named '8dfe5c234085.10.short' already +``` + +Use commands : + +```sh +# Clear the cache which is causing problem. + +docker exec -it _redis-cache_1 redis-cli FLUSHALL +docker exec -it _redis-queue_1 redis-cli FLUSHALL +docker exec -it _redis-socketio_1 redis-cli FLUSHALL +``` + +Note: Environment variables from `.env` file located at current working directory will be used. diff --git a/installation/README.md b/installation/README.md deleted file mode 100644 index 831bd00a..00000000 --- a/installation/README.md +++ /dev/null @@ -1,193 +0,0 @@ -# Production deployment using docker - -### Setup Letsencrypt Nginx Proxy Companion - -DNS needs to be configured for following to work. - -Read more: https://github.com/evertramos/docker-compose-letsencrypt-nginx-proxy-companion - -```sh -cd $HOME -git clone https://github.com/evertramos/docker-compose-letsencrypt-nginx-proxy-companion.git -cd docker-compose-letsencrypt-nginx-proxy-companion -cp .env.sample .env -./start.sh -``` - -### Clone frappe_docker repository - -```sh -cd $HOME -git clone https://github.com/frappe/frappe_docker.git -cd frappe_docker -cp installation/env-example .env - -# make directory for sites -mkdir installation/sites -``` - -### Setup Environment Variables - -Environment variables are set to pass secret and variable data. - -If `env-example` is copied to `.env` following values are set. - -- `VERSION=edge` set version tag or latest for major version e.g. v12.3.0, v12. -- `MYSQL_ROOT_PASSWORD=admin`, set mariadb root password (bootstraps a mariadb container with this root password). If managed database mariadb is used NO need to set the password here. -- `MARIADB_HOST=mariadb` set hostname to `mariadb` in case of docker container for mariadb is used. In case managed db is used set the hostname/IP/domain name here. -- `SITES=site1.domain.com,site2.domain.com` these are list of sites that are part of the deployment "bench". Each site is separated by (,) comma. -- `LETSENCRYPT_EMAIL=your.email@your.domain.com` email for letsencrypt expiry notification. - -### Start frappe-bench services - -```sh -docker-compose \ - --project-name frappebench00 \ - -f installation/docker-compose-common.yml \ - -f installation/docker-compose-erpnext.yml \ - -f installation/docker-compose-networks.yml \ - --project-directory installation up -d -``` - -Note: use `docker-compose-frappe.yml` in case you need bench with just frappe installed. - -### Create new sites - -Note: - -- Wait for mariadb to start. If new site creation fails re-try again after mariadb container is up and running. -- Use `.env` file or environment variables instead of passing secrets as command arguments. - -```sh -# Create ERPNext site -docker exec -it \ - -e "SITE_NAME=$SITE_NAME" \ - -e "DB_ROOT_USER=$DB_ROOT_USER" \ - -e "MYSQL_ROOT_PASSWORD=$MYSQL_ROOT_PASSWORD" \ - -e "ADMIN_PASSWORD=$ADMIN_PASSWORD" \ - -e "INSTALL_ERPNEXT=1" \ - frappebench00_erpnext-python_1 docker-entrypoint.sh new -``` - -Environment Variables needed: - -- `SITE_NAME`, name of the new site to create. -- `DB_ROOT_USER`, MariaDB Root user. The user that can create databases. -- `MYSQL_ROOT_PASSWORD`, In case of mariadb docker container use the one set in `MYSQL_ROOT_PASSWORD` in previous steps. In case of managed database use appropriate password. -- `ADMIN_PASSWORD` set the administrator password for new site. -- `INSTALL_ERPNEXT=1` available only in erpnext-worker and erpnext containers. Installs ERPNext on this new site. -- `FORCE=1` is optional variable which force installs the same site. - -### Backup sites - -Environment Variables - -- `SITES` is list of sites separated by (:) colon to migrate. e.g. `SITES=site1.domain.com` or `SITES=site1.domain.com:site2.domain.com` By default all sites in bench will be backed up. -- `WITH_FILES` if set to 1, it will back up user uploaded files for the sites. - -```sh -docker exec -it \ - -e "SITES=site1.domain.com:site2.domain.com" \ - -e "WITH_FILES=1" \ - frappebench00_erpnext-python_1 docker-entrypoint.sh backup -``` - -Backup will be available in `sites` mounted volume. - -### Update and migrate site - -```sh -# Change to repo root -cd $HOME/frappe_docker - -# Update environment variable VERSION -nano .env - -# Pull new images -docker-compose \ - -f installation/docker-compose-common.yml \ - -f installation/docker-compose-erpnext.yml \ - pull - -# Restart containers -docker-compose \ - --project-name frappebench00 \ - -f installation/docker-compose-common.yml \ - -f installation/docker-compose-erpnext.yml \ - -f installation/docker-compose-networks.yml \ - --project-directory installation up -d - -docker exec -it \ - -e "MAINTENANCE_MODE=1" \ - frappebench00_erpnext-python_1 docker-entrypoint.sh migrate -``` - -### Troubleshoot - -1. Remove containers and volumes clear redis cache: - -This can be used when images are upgraded and if migration fails. - -``` -# change to repo root -cd $HOME/frappe_docker - -# Stop all bench containers -docker-compose \ - --project-name frappebench00 \ - -f installation/docker-compose-common.yml \ - -f installation/docker-compose-erpnext.yml \ - -f installation/docker-compose-networks.yml \ - --project-directory installation stop - -# Remove redis containers -docker-compose \ - --project-name frappebench00 \ - -f installation/docker-compose-common.yml \ - -f installation/docker-compose-erpnext.yml \ - -f installation/docker-compose-networks.yml \ - --project-directory installation rm redis-cache redis-queue redis-socketio - -# Clean redis volumes -docker volume rm \ - frappebench00_redis-cache-vol \ - frappebench00_redis-queue-vol \ - frappebench00_redis-socketio-vol - -# Restart project -docker-compose \ - --project-name frappebench00 \ - -f installation/docker-compose-common.yml \ - -f installation/docker-compose-erpnext.yml \ - -f installation/docker-compose-networks.yml \ - --project-directory installation up -d -``` - -2. Clear redis cache by exec command: - -In case of following error during container restarts, - -``` -frappe-worker-short_1 | Traceback (most recent call last): -frappe-worker-short_1 | File "/home/frappe/frappe-bench/commands/worker.py", line 5, in -frappe-worker-short_1 | start_worker(queue, False) -frappe-worker-short_1 | File "/home/frappe/frappe-bench/apps/frappe/frappe/utils/background_jobs.py", line 147, in start_worker -frappe-worker-short_1 | Worker(queues, name=get_worker_name(queue)).work(logging_level = logging_level) -frappe-worker-short_1 | File "/home/frappe/frappe-bench/env/lib/python3.7/site-packages/rq/worker.py", line 474, in work -frappe-worker-short_1 | self.register_birth() -frappe-worker-short_1 | File "/home/frappe/frappe-bench/env/lib/python3.7/site-packages/rq/worker.py", line 261, in register_birth -frappe-worker-short_1 | raise ValueError(msg.format(self.name)) -frappe-worker-short_1 | ValueError: There exists an active worker named '8dfe5c234085.10.short' already -``` - -Use commands : - -```sh -# Clear the cache which is causing problem. - -docker exec -it frappebench00_redis-cache_1 redis-cli FLUSHALL -docker exec -it frappebench00_redis-queue_1 redis-cli FLUSHALL -docker exec -it frappebench00_redis-socketio_1 redis-cli FLUSHALL -``` - -Note: Environment variables from `.env` file located at current working directory will be used.