Compare commits

..

1 Commits

Author SHA1 Message Date
dependabot[bot]
20edc1b2c4
chore(deps): bump pytest from 8.1.0 to 8.1.1
Bumps [pytest](https://github.com/pytest-dev/pytest) from 8.1.0 to 8.1.1.
- [Release notes](https://github.com/pytest-dev/pytest/releases)
- [Changelog](https://github.com/pytest-dev/pytest/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pytest-dev/pytest/compare/8.1.0...8.1.1)

---
updated-dependencies:
- dependency-name: pytest
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-03-11 08:20:37 +00:00
9 changed files with 8 additions and 656 deletions

View File

@ -49,104 +49,6 @@ Wait for 5 minutes for ERPNext site to be created or check `create-site` contain
### [Troubleshoot](docs/troubleshoot.md)
### Shiloh Setup
based on: https://raw.githubusercontent.com/frappe/bench/develop/easy-install.py
`git clone ssh://git@githaven.org:2222/Shiloh/frappe_docker.git`
sites = [erp.sprinklersnorthwest.com]
email = support@lasthourhosting.org
`cd frappe_docker`
`cp example.env .env`
Write inside of env:
<!-- Commented items because the script doesn't read them. -->
<!-- f"ERPNEXT_VERSION={erpnext_version}\n", -->
f"DB_PASSWORD={db_pass}\n",
<!-- "DB_HOST=db\n",
"DB_PORT=3306\n",
"REDIS_CACHE=redis-cache:6379\n",
"REDIS_QUEUE=redis-queue:6379\n",
"REDIS_SOCKETIO=redis-socketio:6379\n",
f"LETSENCRYPT_EMAIL={email}\n", -->
f"SITE_ADMIN_PASS={admin_pass}\n",
<!-- f"SITES={quoted_sites}\n", -->
<!-- from inside the frappe-docker repo: -->
<!-- Setup reference to our own erpnext repo -->
`APPS_JSON_BASE64=$(base64 -w 0 ./apps.json)`
<!-- build the docker image -->
`sudo docker build \
--build-arg=FRAPPE_PATH=https://github.com/frappe/frappe \
--build-arg=FRAPPE_BRANCH=version-15 \
--build-arg=PYTHON_VERSION=3.11.6 \
--build-arg=NODE_VERSION=18.18.2 \
--build-arg=APPS_JSON_BASE64=$APPS_JSON_BASE64 \
--tag=githaven.org/shiloh/frappe_docker:production \
--file=images/custom/Containerfile \
--rm \
--no-cache \
.`
<!-- Push to githaven -->
<!-- sudo docker image tag NAME:TAG githaven.org/shiloh/frappe_docker:production -->
`sudo docker login githaven.org`
setup credential service to hide password
`sudo docker push githaven.org/shiloh/frappe_docker:production`
https://docs.gitea.com/packages/packages/container
<!-- Install erpnext for first time -->
`sudo python3 easy-installer.py --prod --email support@lasthourhosting.org --site erp.sprinklersnorthwest.com`
`bench use` should already be done by the installer.
<!-- `sudo docker compose -p frappe exec backend bench use erp.sprinklersnorthwest.com` -->
Create a project through the dashboard with the sidebar, then login with the admin account and view the projects page through the other dashboard, you should see edits.
<!-- to enter the container: -->
`sudo docker compose -p frappe exec -it backend bash`
To update erpnext, add any commits you want to use to the "production" branch of the erpnext repo on githaven. Then ensure the "backend" container has git repositories in the frappe and erpnext apps:
`cd ~/frappe-bench/apps/erpnext`
`git init`
`git remote add upstream <repo>`
`git fetch upstream production`
`git checkout production --force`
`git clean -fd`
Then pull the latest erpnext, and it will be updated.
<!-- Bench update hangs on node esbuild, so we're just manually pulling the latest changes rather than doing it through this command. -->
<!-- Then from ./frappe-bench run:
`bench update` -->
<!-- `sudo docker compose --project-name brotherton -f compose.yaml -f overrides/compose.mariadb.yaml -f overrides/compose.redis.yaml -f overrides/compose.https.yaml --env-file .env config`
Make images/production/Containerfile available as a package on githaven? need to replace the erpnext repo argument inside it.
inside of compose.yaml, replace "image: frappe/erpnext" with the above package.
`. .env`
`sudo docker compose -p brotherton -f compose.yaml up -d`
The "erpnext" in this command is probably fine, I think the bench get-app inside the docker compose gets the erpnext repository and puts it into an erpnext "app", regardless of what the erpnext repo env variable is. This command only works if SITES contains a single site.
`sudo docker compose -p brotherton exec backend bench new-site "$SITES" --no-mariadb-socket --db-root-password "$DB_PASSWORD" --admin-password "$SITE_ADMIN_PASS" --install-app erpnext --set-default` -->
# Contributing
If you want to contribute to this repo refer to [CONTRIBUTING.md](CONTRIBUTING.md)

View File

@ -1,147 +0,0 @@
ARG PYTHON_VERSION=3.11.6
ARG DEBIAN_BASE=bookworm
FROM python:${PYTHON_VERSION}-slim-${DEBIAN_BASE} AS base
ARG WKHTMLTOPDF_VERSION=0.12.6.1-3
ARG WKHTMLTOPDF_DISTRO=bookworm
ARG NODE_VERSION=18.18.2
ENV NVM_DIR=/home/frappe/.nvm
ENV PATH ${NVM_DIR}/versions/node/v${NODE_VERSION}/bin/:${PATH}
RUN useradd -ms /bin/bash frappe \
&& apt-get update \
&& apt-get install --no-install-recommends -y \
curl \
git \
vim \
nginx \
gettext-base \
# weasyprint dependencies
libpango-1.0-0 \
libharfbuzz0b \
libpangoft2-1.0-0 \
libpangocairo-1.0-0 \
# For backups
restic \
gpg \
# MariaDB
mariadb-client \
less \
# Postgres
libpq-dev \
postgresql-client \
# For healthcheck
wait-for-it \
jq \
# NodeJS
&& mkdir -p ${NVM_DIR} \
&& curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.5/install.sh | bash \
&& . ${NVM_DIR}/nvm.sh \
&& nvm install ${NODE_VERSION} \
&& nvm use v${NODE_VERSION} \
&& npm install -g yarn \
&& nvm alias default v${NODE_VERSION} \
&& rm -rf ${NVM_DIR}/.cache \
&& echo 'export NVM_DIR="/home/frappe/.nvm"' >>/home/frappe/.bashrc \
&& echo '[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh" # This loads nvm' >>/home/frappe/.bashrc \
&& echo '[ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion" # This loads nvm bash_completion' >>/home/frappe/.bashrc \
# Install wkhtmltopdf with patched qt
&& if [ "$(uname -m)" = "aarch64" ]; then export ARCH=arm64; fi \
&& if [ "$(uname -m)" = "x86_64" ]; then export ARCH=amd64; fi \
&& downloaded_file=wkhtmltox_${WKHTMLTOPDF_VERSION}.${WKHTMLTOPDF_DISTRO}_${ARCH}.deb \
&& curl -sLO https://github.com/wkhtmltopdf/packaging/releases/download/$WKHTMLTOPDF_VERSION/$downloaded_file \
&& apt-get install -y ./$downloaded_file \
&& rm $downloaded_file \
# Clean up
&& rm -rf /var/lib/apt/lists/* \
&& rm -fr /etc/nginx/sites-enabled/default \
&& pip3 install frappe-bench \
# Fixes for non-root nginx and logs to stdout
&& sed -i '/user www-data/d' /etc/nginx/nginx.conf \
&& ln -sf /dev/stdout /var/log/nginx/access.log && ln -sf /dev/stderr /var/log/nginx/error.log \
&& touch /run/nginx.pid \
&& chown -R frappe:frappe /etc/nginx/conf.d \
&& chown -R frappe:frappe /etc/nginx/nginx.conf \
&& chown -R frappe:frappe /var/log/nginx \
&& chown -R frappe:frappe /var/lib/nginx \
&& chown -R frappe:frappe /run/nginx.pid
COPY resources/nginx-template.conf /templates/nginx/frappe.conf.template
COPY resources/nginx-entrypoint.sh /usr/local/bin/nginx-entrypoint.sh
FROM base AS builder
RUN apt-get update \
&& DEBIAN_FRONTEND=noninteractive apt-get install --no-install-recommends -y \
# For frappe framework
wget \
# For psycopg2
libpq-dev \
# Other
libffi-dev \
liblcms2-dev \
libldap2-dev \
libmariadb-dev \
libsasl2-dev \
libtiff5-dev \
libwebp-dev \
redis-tools \
rlwrap \
tk8.6-dev \
cron \
# For pandas
gcc \
build-essential \
libbz2-dev \
# for erpnext repo
openssh-client \
&& rm -rf /var/lib/apt/lists/*
USER frappe
RUN mkdir -p /home/frappe/.ssh
RUN echo -e "Host *\n\tStrictHostKeyChecking no\n" >> /home/frappe/.ssh/config
ARG FRAPPE_BRANCH=version-15
ARG FRAPPE_PATH=https://github.com/frappe/frappe
ARG ERPNEXT_REPO=https://githaven.org/Shiloh/brotherton-erpnext.git
ARG ERPNEXT_BRANCH=production
# RUN ssh -T ${ERPNEXT_REPO}
RUN bench init \
--frappe-branch=${FRAPPE_BRANCH} \
--frappe-path=${FRAPPE_PATH} \
--no-procfile \
--no-backups \
--skip-redis-config-generation \
--verbose \
/home/frappe/frappe-bench && \
cd /home/frappe/frappe-bench && \
bench get-app --branch=${ERPNEXT_BRANCH} --resolve-deps erpnext ${ERPNEXT_REPO} && \
echo "{}" > sites/common_site_config.json
FROM base as erpnext
USER frappe
COPY --from=builder --chown=frappe:frappe /home/frappe/frappe-bench /home/frappe/frappe-bench
WORKDIR /home/frappe/frappe-bench
VOLUME [ \
"/home/frappe/frappe-bench/sites", \
"/home/frappe/frappe-bench/sites/assets", \
"/home/frappe/frappe-bench/logs" \
]
CMD [ \
"/home/frappe/frappe-bench/env/bin/gunicorn", \
"--chdir=/home/frappe/frappe-bench/sites", \
"--bind=0.0.0.0:8000", \
"--threads=4", \
"--workers=2", \
"--worker-class=gthread", \
"--worker-tmp-dir=/dev/shm", \
"--timeout=120", \
"--preload", \
"frappe.app:application" \
]

View File

@ -1,6 +0,0 @@
[
{
"url": "https://githaven.org/Shiloh/brotherton-erpnext.git",
"branch": "production"
}
]

View File

@ -1,23 +0,0 @@
#!/bin/bash
export APPS_JSON='[
{
"url": "https://githaven.org/Shiloh/brotherton-erpnext",
"branch": "production"
},
{
"url": "https://github.com/frappe/hrms",
"branch": "v15.15.0"
}
]'
export APPS_JSON_BASE64=$(echo ${APPS_JSON} | base64 -w 0)
export TAG=1.0.1 # Change this
docker build --platform=linux/amd64 \
--build-arg=FRAPPE_PATH=https://github.com/frappe/frappe \
--build-arg=FRAPPE_BRANCH=v15.15.0 \
--build-arg=PYTHON_VERSION=3.11.6 \
--build-arg=NODE_VERSION=18.18.2 \
--build-arg=APPS_JSON_BASE64=$APPS_JSON_BASE64 \
--tag=githaven.org/shiloh/frappe_docker:$TAG \
--file=images/custom/Containerfile .
docker push githaven.org/shiloh/frappe_docker:$TAG

View File

@ -2,10 +2,7 @@ x-customizable-image: &customizable_image
# By default the image used only contains the `frappe` and `erpnext` apps.
# See https://github.com/frappe/frappe_docker/blob/main/docs/custom-apps.md
# about using custom images.
image: githaven.org/shiloh/frappe_docker:production
# build:
# context: .
# dockerfile: ./Shilohimage
image: frappe/erpnext:${ERPNEXT_VERSION:?No ERPNext version set}
x-depends-on-configurator: &depends_on_configurator
depends_on:
@ -27,8 +24,6 @@ services:
command:
- >
ls -1 apps > sites/apps.txt;
rm -rf sites/assets || true;
ln -s /home/frappe/frappe-bench/assets sites/assets || true;
bench set-config -g db_host $$DB_HOST;
bench set-config -gp db_port $$DB_PORT;
bench set-config -g redis_cache "redis://$$REDIS_CACHE";

View File

@ -1,363 +0,0 @@
#!/usr/bin/env python3
import argparse
import fileinput
import logging
import os
import platform
import subprocess
import sys
import time
import urllib.request
from shutil import move, unpack_archive, which
from typing import Dict
logging.basicConfig(
filename="easy-install.log",
filemode="w",
format="%(asctime)s - %(levelname)s - %(message)s",
level=logging.INFO,
)
def cprint(*args, level: int = 1):
"""
logs colorful messages
level = 1 : RED
level = 2 : GREEN
level = 3 : YELLOW
default level = 1
"""
CRED = "\033[31m"
CGRN = "\33[92m"
CYLW = "\33[93m"
reset = "\033[0m"
message = " ".join(map(str, args))
if level == 1:
print(CRED, message, reset)
if level == 2:
print(CGRN, message, reset)
if level == 3:
print(CYLW, message, reset)
def clone_frappe_docker_repo() -> None:
try:
urllib.request.urlretrieve(
"https://github.com/frappe/frappe_docker/archive/refs/heads/main.zip",
"frappe_docker.zip",
)
logging.info("Downloaded frappe_docker zip file from GitHub")
unpack_archive(
"frappe_docker.zip", "."
) # Unzipping the frappe_docker.zip creates a folder "frappe_docker-main"
move("frappe_docker-main", "frappe_docker")
logging.info("Unzipped and Renamed frappe_docker")
os.remove("frappe_docker.zip")
logging.info("Removed the downloaded zip file")
except Exception as e:
logging.error("Download and unzip failed", exc_info=True)
cprint("\nCloning frappe_docker Failed\n\n", "[ERROR]: ", e, level=1)
def get_from_env(dir, file) -> Dict:
env_vars = {}
with open(os.path.join(dir, file)) as f:
for line in f:
if line.startswith("#") or not line.strip():
continue
key, value = line.strip().split("=", 1)
env_vars[key] = value
return env_vars
def write_to_env(
wd: str,
sites,
db_pass: str,
admin_pass: str,
email: str,
erpnext_version: str = None,
) -> None:
quoted_sites = ",".join([f"`{site}`" for site in sites]).strip(",")
example_env = get_from_env(wd, "example.env")
erpnext_version = erpnext_version or example_env["ERPNEXT_VERSION"]
with open(os.path.join(wd, ".env"), "w") as f:
f.writelines(
[
f"ERPNEXT_VERSION={erpnext_version}\n", # defaults to latest version of ERPNext
f"DB_PASSWORD={db_pass}\n",
"DB_HOST=db\n",
"DB_PORT=3306\n",
"REDIS_CACHE=redis-cache:6379\n",
"REDIS_QUEUE=redis-queue:6379\n",
"REDIS_SOCKETIO=redis-socketio:6379\n",
f"LETSENCRYPT_EMAIL={email}\n",
f"SITE_ADMIN_PASS={admin_pass}\n",
f"SITES={quoted_sites}\n",
]
)
def generate_pass(length: int = 12) -> str:
"""Generate random hash using best available randomness source."""
import math
import secrets
if not length:
length = 56
return secrets.token_hex(math.ceil(length / 2))[:length]
def check_repo_exists() -> bool:
return os.path.exists(os.path.join(os.getcwd(), "../frappe_docker"))
def setup_prod(
project: str, sites, email: str, version: str = None, image=None
) -> None:
if len(sites) == 0:
sites = ["site1.localhost"]
if check_repo_exists():
compose_file_name = os.path.join(
os.path.expanduser("~"), f"{project}-compose.yml"
)
docker_repo_path = os.path.join(os.getcwd(), "../frappe_docker")
cprint(
"\nPlease refer to .example.env file in the frappe_docker folder to know which keys to set\n\n",
level=3,
)
admin_pass = ""
db_pass = ""
with open(compose_file_name, "w") as f:
# Writing to compose file
if not os.path.exists(os.path.join(docker_repo_path, ".env")):
admin_pass = generate_pass()
db_pass = generate_pass(9)
write_to_env(
docker_repo_path, sites, db_pass, admin_pass, email, version
)
cprint(
"\nA .env file is generated with basic configs. Please edit it to fit to your needs \n",
level=3,
)
with open(
os.path.join(os.path.expanduser("~"), "passwords.txt"), "w"
) as en:
en.writelines(f"ADMINISTRATOR_PASSWORD={admin_pass}\n")
en.writelines(f"MARIADB_ROOT_PASSWORD={db_pass}\n")
else:
env = get_from_env(docker_repo_path, ".env")
admin_pass = env["SITE_ADMIN_PASS"]
db_pass = env["DB_PASSWORD"]
try:
# TODO: Include flags for non-https and non-erpnext installation
subprocess.run(
[
which("docker"),
"compose",
"--project-name",
project,
"-f",
"compose.yaml",
"-f",
"overrides/compose.mariadb.yaml",
"-f",
"overrides/compose.redis.yaml",
# "-f", "overrides/compose.noproxy.yaml", TODO: Add support for local proxying without HTTPs
"-f",
"overrides/compose.https.yaml",
"--env-file",
".env",
"config",
],
cwd=docker_repo_path,
stdout=f,
check=True,
)
except Exception:
logging.error("Docker Compose generation failed", exc_info=True)
cprint("\nGenerating Compose File failed\n")
sys.exit(1)
# Use custom image
if image:
for line in fileinput.input(compose_file_name, inplace=True):
if "image: frappe/erpnext" in line:
line = line.replace("image: frappe/erpnext", f"image: {image}")
sys.stdout.write(line)
try:
subprocess.run(
[
which("docker"),
"compose",
"-p",
project,
"-f",
compose_file_name,
"up",
"-d",
],
check=True,
)
logging.info(f"Docker Compose file generated at ~/{project}-compose.yml")
except Exception as e:
logging.error("Prod docker-compose failed", exc_info=True)
cprint(" Docker Compose failed, please check the container logs\n", e)
sys.exit(1)
for sitename in sites:
create_site(sitename, project, db_pass, admin_pass)
else:
install_docker()
clone_frappe_docker_repo()
setup_prod(project, sites, email, version, image) # Recursive
def setup_dev_instance(project: str):
if check_repo_exists():
try:
subprocess.run(
[
"docker",
"compose",
"-f",
"devcontainer-example/docker-compose.yml",
"--project-name",
project,
"up",
"-d",
],
cwd=os.path.join(os.getcwd(), "../frappe_docker"),
check=True,
)
cprint(
"Please go through the Development Documentation: https://github.com/frappe/frappe_docker/tree/main/development to fully complete the setup.",
level=2,
)
logging.info("Development Setup completed")
except Exception as e:
logging.error("Dev Environment setup failed", exc_info=True)
cprint("Setting Up Development Environment Failed\n", e)
else:
install_docker()
clone_frappe_docker_repo()
setup_dev_instance(project) # Recursion on goes brrrr
def install_docker():
if which("docker") is not None:
return
cprint("Docker is not installed, Installing Docker...", level=3)
logging.info("Docker not found, installing Docker")
if platform.system() == "Darwin" or platform.system() == "Windows":
print(
f"""
This script doesn't install Docker on {"Mac" if platform.system()=="Darwin" else "Windows"}.
Please go through the Docker Installation docs for your system and run this script again"""
)
logging.debug("Docker setup failed due to platform is not Linux")
sys.exit(1)
try:
ps = subprocess.run(
["curl", "-fsSL", "https://get.docker.com"],
capture_output=True,
check=True,
)
subprocess.run(["/bin/bash"], input=ps.stdout, capture_output=True)
subprocess.run(
["sudo", "usermod", "-aG", "docker", str(os.getenv("USER"))], check=True
)
cprint("Waiting Docker to start", level=3)
time.sleep(10)
subprocess.run(["sudo", "systemctl", "restart", "docker.service"], check=True)
except Exception as e:
logging.error("Installing Docker failed", exc_info=True)
cprint("Failed to Install Docker\n", e)
cprint("\n Try Installing Docker Manually and re-run this script again\n")
sys.exit(1)
def create_site(
sitename: str,
project: str,
db_pass: str,
admin_pass: str,
):
cprint(f"\nCreating site: {sitename} \n", level=3)
try:
subprocess.run(
[
which("docker"),
"compose",
"-p",
project,
"exec",
"backend",
"bench",
"new-site",
sitename,
"--no-mariadb-socket",
"--db-root-password",
db_pass,
"--admin-password",
admin_pass,
"--install-app",
"erpnext",
"--set-default",
],
check=True,
)
logging.info("New site creation completed")
except Exception as e:
logging.error(f"Bench site creation failed for {sitename}", exc_info=True)
cprint(f"Bench Site creation failed for {sitename}\n", e)
if __name__ == "__main__":
parser = argparse.ArgumentParser(description="Install Frappe with Docker")
parser.add_argument(
"-p", "--prod", help="Setup Production System", action="store_true"
)
parser.add_argument(
"-d", "--dev", help="Setup Development System", action="store_true"
)
parser.add_argument(
"-s",
"--sitename",
help="Site Name(s) for your production bench",
default=[],
action="append",
dest="sites",
)
parser.add_argument("-n", "--project", help="Project Name", default="frappe")
parser.add_argument("-i", "--image", help="Full Image Name")
parser.add_argument(
"--email", help="Add email for the SSL.", required="--prod" in sys.argv
)
parser.add_argument(
"-v", "--version", help="ERPNext version to install, defaults to latest stable"
)
args = parser.parse_args()
if args.dev:
cprint("\nSetting Up Development Instance\n", level=2)
logging.info("Running Development Setup")
setup_dev_instance(args.project)
elif args.prod:
cprint("\nSetting Up Production Instance\n", level=2)
logging.info("Running Production Setup")
if "example.com" in args.email:
cprint("Emails with example.com not acceptable", level=1)
sys.exit(1)
setup_prod(args.project, args.sites, args.email, args.version, args.image)
else:
parser.print_help()

View File

@ -131,9 +131,11 @@ COPY --from=builder --chown=frappe:frappe /home/frappe/frappe-bench /home/frappe
WORKDIR /home/frappe/frappe-bench
# Move the generated assets folder out of the "sites" folder, which will be attached to a volume and thus persisted
# So that it can be referenced by symlink even after "sites" is replaced with the volume
RUN mv /home/frappe/frappe-bench/sites/assets /home/frappe/frappe-bench/assets
VOLUME [ \
"/home/frappe/frappe-bench/sites", \
"/home/frappe/frappe-bench/sites/assets", \
"/home/frappe/frappe-bench/logs" \
]
CMD [ \
"/home/frappe/frappe-bench/env/bin/gunicorn", \

View File

@ -93,19 +93,13 @@ RUN apt-get update \
gcc \
build-essential \
libbz2-dev \
# for erpnext repo
openssh-ssh \
&& rm -rf /var/lib/apt/lists/*
USER frappe
ARG SSH_PRIVATE_KEY
RUN mkdir ~/.ssh/
RUN echo "${SSH_PRIVATE_KEY}" > ~/.ssh/id_ed25519
ARG FRAPPE_BRANCH=version-15
ARG FRAPPE_PATH=https://github.com/frappe/frappe
ARG ERPNEXT_REPO=ssh://git@githaven.org:2222/Shiloh/brotherton-erpnext.git
ARG ERPNEXT_REPO=https://github.com/frappe/erpnext
ARG ERPNEXT_BRANCH=version-15
RUN bench init \
--frappe-branch=${FRAPPE_BRANCH} \
@ -124,8 +118,6 @@ FROM base as erpnext
USER frappe
RUN mkdir testFolder
COPY --from=builder --chown=frappe:frappe /home/frappe/frappe-bench /home/frappe/frappe-bench
WORKDIR /home/frappe/frappe-bench

View File

@ -1 +1 @@
pytest==8.1.0
pytest==8.1.1