Installing and Enabling pyDE1
Overview
To enhance security, the pyDE1
executables run as a dedicated user,
one with limited privilege. When possible, only read access is granted
to the files. None of the executables or configuration files should be
writable by pyde1. In this guide, they are set to root ownership.
Warning
Accessing the database as any user other than pyde1 may cause files to be written by that user, preventing access by pyde1 [1]
There are sh
scripts provided that should make the process relatively
straightforward. As they are often run as root with sudo
, read them
and convince yourself that they are going to do what you expect, before
running them blindly.
Unless specifically noted, all scripts are to be run with root
privilege using sudo
.
Walk-Through
The scripts have been broken down into relatively small operations. This allows them to be easily re-run should an error occur.
The scripts are not distributed in the pip
package. They are present
in the pyDE1 git repo, in the install/
directory.
The repo can be cloned, or individual files downloaded. Make sure that
the _config
file is in the same directory as the shell scripts.
Generally no changes need to be made to the _config
file.
# Copyright © 2021 Jeff Kletsky. All Rights Reserved.
#
# License for this software, part of the pyDE1 package, is granted under
# GNU General Public License v3.0 only
# SPDX-License-Identifier: GPL-3.0-only
# sourced by install scripts
PYDE1_USER=pyde1
PYDE1_GROUP="${PYDE1_USER}"
VENV_PATH="/home/${PYDE1_USER}/venv/pyde1"
10-create-user.sh
This script will create the pyde1 user if it does not exist and give it access to the bluetooth group.
#!/usr/bin/sh -e
# Copyright © 2021 Jeff Kletsky. All Rights Reserved.
#
# License for this software, part of the pyDE1 package, is granted under
# GNU General Public License v3.0 only
# SPDX-License-Identifier: GPL-3.0-only
. "$(dirname $0)"/_config
if [ -z "$SUDO_USER" ] ; then
>&2 echo "Script must be run with sudo"
elif [ "$SUDO_UID" = 0 ] ; then
>&2 echo "Script must be run with sudo by a normal user"
fi
# Create the pyde1 user if they do not yet exist.
if getent passwd "$PYDE1_USER" ; then
echo "User $PYDE1_USER already exists"
else
echo "Creating user $PYDE1_USER"
adduser --system --group "$PYDE1_USER"
fi
usermod -a -G bluetooth "$PYDE1_USER"
id "$PYDE1_USER"
20-create-dirs.sh
This script will create the following directories and set their ownership and permissions:
/var/log/pyde1
/var/lib/pyde1
#!/usr/bin/sh -e
# Copyright © 2021, 2022 Jeff Kletsky. All Rights Reserved.
#
# License for this software, part of the pyDE1 package, is granted under
# GNU General Public License v3.0 only
# SPDX-License-Identifier: GPL-3.0-only
. "$(dirname $0)"/_config
echo "Creating target directories"
mkdir -p /var/log/pyde1
chown $PYDE1_USER /var/log/pyde1
mkdir -p /var/lib/pyde1
chown $PYDE1_USER /var/lib/pyde1
ls -ld /var/log/pyde1
ls -ld /var/lib/pyde1
30-populate-venv
This creates a root-owned, Python virtual environment (“venv”).
It then updates pip
and setuptools
and adds the pyDE1
package
and its dependencies to the venv.
Note
If you installed a non-default version of Python, such as 3.9 or 3.10 on an install of Raspberry Pi OS based on “Buster”, you will need to explicitly reference that version when creating the venv.
python -m venv $VENV_PATH
would need to be edited to explicitly to
refer to your chosen version. References after . $VENV_PATH/bin/activate
should not need modification, as that sets python
to refer to the one
in the venv.
#!/usr/bin/sh -e
# Copyright © 2021 Jeff Kletsky. All Rights Reserved.
#
# License for this software, part of the pyDE1 package, is granted under
# GNU General Public License v3.0 only
# SPDX-License-Identifier: GPL-3.0-only
. "$(dirname $0)"/_config
echo "Creating Python venv at $VENV_PATH"
if ! dpkg --get-selections | egrep '^python3-venv\s+install$' ; then
apt install python3-venv
fi
mkdir -p $VENV_PATH
python -m venv $VENV_PATH
. $VENV_PATH/bin/activate
pip install -U pip
pip install -U setuptools
pip install pyDE1
pip list
# "Where is pyDE1?"
python -c \
'import importlib.resources ; print(importlib.resources.files("pyDE1"))'
40-config-files.sh
This copies the config files from the location where pip
installed them
in the venv and into /usr/local/etc/pyde1
. It will make a timestamped
backup of any file that would be overwritten.
Note
Some of the configuration files may contain sensitive credentials, such as MQTT and Visualizer usernames and passwords. These files are set to root:pyde1 ownership with no other read access.
It also copies the pyde1.service
and pyde1-visualizer.service
files,
similarly making backups. These files are edited in place to adjust for
the specifics of the local install from the previous steps. The editor
(sed
) backs up the original version with a .bak
suffix.
Rather than run disconnect-btid.sh
directly from the install, it is
copied to /usr/local/bin/pyde1-disconnect-btid.sh
. This script is run
by pyde1.service
to help clean up any “stale” Bluetooth connections
related to a prior run that may have terminated ungracefully.
#!/usr/bin/sh -e
# Copyright © 2021, 2023 Jeff Kletsky. All Rights Reserved.
#
# License for this software, part of the pyDE1 package, is granted under
# GNU General Public License v3.0 only
# SPDX-License-Identifier: GPL-3.0-only
. "$(dirname $0)"/_config
mkdir -p /usr/local/bin/pyde1
mkdir -p /usr/local/etc/pyde1
CP_BACKUP="cp -v --backup --suffix=$(date +'.%Y%m%d_%H%M')"
$CP_BACKUP ${PYDE1_ROOT}/services/config/* /usr/local/etc/pyde1/
# As the config files may contain credentials,
# make them unreadable to anyone but root and pyde1
chown root:${PYDE1_GROUP} /usr/local/etc/pyde1/*
chmod 640 /usr/local/etc/pyde1/*
$CP_BACKUP ${PYDE1_ROOT}/services/unit-files/* /usr/local/etc/pyde1/
chown root:root /usr/local/etc/pyde1/*.service
chmod 644 /usr/local/etc/pyde1/*.service
for f in /usr/local/etc/pyde1/*.service ; do
# This is rather fragile. TODO: Consider a template
sed -i'.bak' \
-e "s|^User=.*|User=${PYDE1_USER}|" \
-e "s|^Group=.*|Group=${PYDE1_GROUP}|" \
-e "s|/home/pyde1/venv/pyde1|${VENV_PATH}"
$f
done
ls -l /usr/local/etc/*
Adjust Config Files to Suit
Examine the various config files /usr/local/etc/pyde1/*.conf
and edit
as needed.
Changes that are commonly needed include:
In pyde1.conf
mqtt:
USERNAME
PASSWORD
de1:
LINE_FREQUENCY
In pyde1-visualizer.conf
mqtt:
USERNAME
PASSWORD
visualizer:
USERNAME
PASSWORD
After completing the edits, ensure that they are readable by the pyde1 group and not by anyone else, other than root.
ls -l *.conf
-rw-r----- 1 root pyde1 3555 Nov 3 18:18 pyde1.conf
-rw-r----- 1 root pyde1 1789 Nov 3 18:18 pyde1-replay.conf
-rw-r----- 1 root pyde1 2308 Nov 3 18:18 pyde1-visualizer.conf
If needed, the ownership and permissions can be corrected with
sudo chown root:pyde1 *.conf
sudo chmod 640 *.conf
50-enable-services.sh
This links the service definitions in /usr/local/etc/pyde1
for the
pyde1.service
and the pyde1-visualizer.service
to where systemd
(the “startup manager” for Debian) knows about them, enables the services,
and restarts them. Unless they are explicitly disabled, they will start
on every boot.
Further information on service management can be found with
man systemctl
man journalctl
#!/usr/bin/sh -e
# Copyright © 2021 Jeff Kletsky. All Rights Reserved.
#
# License for this software, part of the pyDE1 package, is granted under
# GNU General Public License v3.0 only
# SPDX-License-Identifier: GPL-3.0-only
for service in /usr/local/etc/pyde1/pyde1.service \
/usr/local/etc/pyde1/pyde1-visualizer.service ; do
systemctl link -f $service
systemctl daemon-reload
service_name=$(basename $service)
systemctl enable $service_name
systemctl restart $service_name
done
Log Rotation
The standard log-rotation utility on Debian is logrotate
with
configuration in /etc/logrotate.d/
One configuration that rotates daily, compresses, and retains 60 days’ of logs is
/var/log/pyde1/pyde1.log {
daily
missingok
rotate 60
compress
delaycompress
notifempty
create
}
/var/log/pyde1/visualizer.log {
daily
missingok
rotate 60
compress
delaycompress
notifempty
create
}
Both the mosquitto
and nginx
packages install self-named config into
/etc/logrotate.d/