# Django - Celery, RabbitMQ, Supervisor
###### tags: `Django` `Celery` `RabbitMQ` `Supervisor`
# Reference
* [Celery Tutorial: A Must-Learn Technology for Python Developer](https://medium.com/swlh/python-developers-celery-is-a-must-learn-technology-heres-how-to-get-started-578f5d63fab3)
---
# Supervisor
關於 Supervisor, 有 Linux 版, 也有 Python 版, 經測試後，
發現要使用 Linux 版, 而非使用 ``pip`` 去安裝
## 安裝 Supervisor
```
sudo apt install supervisor
```
```
正在讀取套件清單... 完成
正在重建相依關係
正在讀取狀態資料... 完成
下列的額外套件將被安裝：
python-meld3 python-pkg-resources
建議套件：
python-setuptools supervisor-doc
下列【新】套件將會被安裝：
python-meld3 python-pkg-resources supervisor
```
安裝完成後, ``/etc/supervisor/conf.d`` 資料夾
與 ``/etc/supervisor/supervisord.conf`` 會被建立
---
# [Implementing Celery using Django for Background Task Processing](https://www.botreetechnologies.com/blog/implementing-celery-using-django-for-background-task-processing/)
建立 worker 與 beat 會用到的 supervisor ``*.conf``檔案
* ``sso_worker_supervisor.conf``
```
; ==================================
; celery worker supervisor
; ==================================
; the name of your supervisord program
[program:sso_celery_worker]
; Set full path to celery program if using virtualenv
command=/home/elite_lin/AWS/venv/bin/celery -A test_project2.celery worker --loglevel=debug
; The directory to your Django project
directory=/home/elite_lin/AWS/sso_project/src
; If supervisord is run as the root user, switch users to this UNIX user account
; before doing any processing.
user=elite_lin
; Supervisor will start as many instances of this program as named by numprocs
numprocs=1
; Put process stdout output in this file
stdout_logfile=/var/log/celery/sso_worker.log
; Put process stderr output in this file
stderr_logfile=/var/log/celery/sso_worker.log
; If true, this program will start automatically when supervisord is started
autostart=true
; May be one of false, unexpected, or true. If false, the process will never
; be autorestarted. If unexpected, the process will be restart when the program
; exits with an exit code that is not one of the exit codes associated with this
; process’ configuration (see exitcodes). If true, the process will be
; unconditionally restarted when it exits, without regard to its exit code.
autorestart=true
; The total number of seconds which the program needs to stay running after
; a startup to consider the start successful.
startsecs=10
; Need to wait for currently executing tasks to finish at shutdown.
; Increase this if you have very long running tasks.
stopwaitsecs = 600
; When resorting to send SIGKILL to the program to terminate it
; send SIGKILL to its whole process group instead,
; taking care of its children as well.
killasgroup=true
; if your broker is supervised, set its priority higher
; so it starts first
priority=998
```
* ``sso_scheduler_supervisor.conf``:
```
; ================================
; celery beat supervisor
; ================================
; the name of your supervisord program
[program:sso_celery_beat]
; Set full path to celery program if using virtualenv
command=/home/elite_lin/AWS/venv/bin/celery -A test_project2 beat --loglevel=debug
; The directory to your Django project
directory=/home/elite_lin/AWS/sso_project/src
; If supervisord is run as the root user, switch users to this UNIX user account
; before doing any processing.
user=elite_lin
; Supervisor will start as many instances of this program as named by numprocs
numprocs=1
; Put process stdout output in this file
stdout_logfile=/var/log/celery/sso_beat.log
; Put process stderr output in this file
stderr_logfile=/var/log/celery/sso_beat.log
; If true, this program will start automatically when supervisord is started
autostart=true
; May be one of false, unexpected, or true. If false, the process will never
; be autorestarted. If unexpected, the process will be restart when the program
; exits with an exit code that is not one of the exit codes associated with this
; process’ configuration (see exitcodes). If true, the process will be
; unconditionally restarted when it exits, without regard to its exit code.
autorestart=true
; The total number of seconds which the program needs to stay running after
; a startup to consider the start successful.
startsecs=10
; if your broker is supervised, set its priority higher
; so it starts first
priority=999
```
將 ``sso_worker_supervisor.conf`` 與 ``sso_scheduler_supervisor.conf``
``cp`` 到 ``/etc/supervisor/supervisord.conf``
然後 建立 ``/var/log/celery`` 資料夾, 再建立2個空 log 檔:
```
cd /var/log/celery
sudo touch sso_worker.log
sudo touch sso_beat.log
```
理論上，在執行下列3個指令後，
```
sudo supervisorctl reread
sudo supervisorctl update
sudo supervisorctl status
```
會看見
```
sso_celery_beat RUNNING pid 10058, uptime 0:17:32
sso_celery_worker RUNNING pid 10057, uptime 0:17:32
```
若有 Error, 要去看指定的 log 檔中有什麼異常的資訊來除錯
```
elite_lin@elite-lin-VirtualBox:/etc/supervisor$ ls
conf.d supervisord.conf
elite_lin@elite-lin-VirtualBox:/etc/supervisor$
elite_lin@elite-lin-VirtualBox:/etc/supervisor$
elite_lin@elite-lin-VirtualBox:/etc/supervisor$ sudo supervisorctl reread
sso_celery_beat: available
sso_celery_worker: available
elite_lin@elite-lin-VirtualBox:/etc/supervisor$
elite_lin@elite-lin-VirtualBox:/etc/supervisor$
elite_lin@elite-lin-VirtualBox:/etc/supervisor$ sudo supervisorctl update
sso_celery_worker: added process group
sso_celery_beat: added process group
elite_lin@elite-lin-VirtualBox:/etc/supervisor$
elite_lin@elite-lin-VirtualBox:/etc/supervisor$
elite_lin@elite-lin-VirtualBox:/etc/supervisor$ sudo supervisorctl status
sso_celery_beat FATAL Exited too quickly (process log may have details)
sso_celery_worker FATAL Exited too quickly (process log may have details)
elite_lin@elite-lin-VirtualBox:/etc/supervisor$
elite_lin@elite-lin-VirtualBox:/etc/supervisor$
elite_lin@elite-lin-VirtualBox:/etc/supervisor$ sudo supervisorctl stop
Error: stop requires a process name
stop <name> Stop a process
stop <gname>:* Stop all processes in a group
stop <name> <name> Stop multiple processes or groups
stop all Stop all processes
elite_lin@elite-lin-VirtualBox:/etc/supervisor$
elite_lin@elite-lin-VirtualBox:/etc/supervisor$
elite_lin@elite-lin-VirtualBox:/etc/supervisor$ sudo supervisorctl start sso_celery_worker
sso_celery_worker: ERROR (spawn error)
```
---
``tasks.py``:
```
# created in 2021/01/15
from celery import task
from django.core.mail import send_mail
@task(name="send_request_access_mail")
def send_request_access_mail(subject, message, sender, receivers):
send_mail(subject, message, sender, receivers)
```
``signal.py``:
```
from django.db.models.signals import post_save
from django.dispatch import receiver
from django.utils.encoding import force_bytes
from django.utils.http import urlsafe_base64_encode
from django.template.loader import render_to_string
# from .tokens import account_activation_signup_token # the app does not have that
from django.conf import settings
from .tasks import send_request_access_mail
@receiver(post_save, sender=RequestAccessModel)
def approval(sender, instance, created, **kwargs):
if instance.accepted == True:
subject = 'Activate Your MySite Account'
message = render_to_string('request_access/request_approved.html', {
'user': instance,
'domain': 'localhost:8000',
'uid': urlsafe_base64_encode(force_bytes(instance.id)),
'token': 'aA1234567890',
})
send_request_access_mail.delay(subject, message, settings.EMAIL_HOST_USER, [instance.email])
```
---
# [Django Celery + RabbitMQ with Supervisor](https://riptutorial.com/django/example/32120/celery-plus-rabbitmq-with-supervisor)
Celery requires a __broker__ to handle message-passing.
We use __RabbitMQ__ because it’s easy to setup and it is well supported.
Install rabbitmq using the following command
```
sudo apt-get install rabbitmq-server
```
Once the installation is complete, create user, add a virtual host and set permissions.
```
sudo rabbitmqctl add_user myuser mypassword
sudo rabbitmqctl add_vhost myvhost
sudo rabbitmqctl set_user_tags myuser mytag
sudo rabbitmqctl set_permissions -p myvhost myuser ".*" ".*" ".*"
```
To start the server:
```
sudo rabbitmq-server
```
We can install ``celery`` with ``pip``:
```
pip install celery
```
In your Django ``settings.py`` file,
your __broker URL__ would then look something like
```
BROKER_URL = 'amqp://myuser:mypassword@localhost:5672/myvhost'
```
Now start the celery worker
```
celery -A your_app worker -l info
```
This command start a Celery worker to run any tasks defined in your django app.
__Supervisor__ is a Python program that
allows you to control and keep running any unix processes.
It can also restart crashed processes.
We use it to make sure Celery workers are always running.
First, Install supervisor
```
sudo apt-get install supervisor
```
Create ``your_proj.conf`` file in your supervisor ``conf.d``
(``/etc/supervisor/conf.d/your_proj.conf``):
```
[program:your_proj_celery]
command=/home/your_user/your_proj/.venv/bin/celery --app=your_proj.celery:app worker -l info
directory=/home/your_user/your_proj
numprocs=1
stdout_logfile=/home/your_user/your_proj/logs/celery-worker.log
stderr_logfile=/home/your_user/your_proj/logs/low-worker.log
autostart=true
autorestart=true
startsecs=10
```
Once our configuration file is created and saved,
we can inform Supervisor of our new program
through the ``supervisorctl`` command.
First we tell Supervisor to look for any new or changed program configurations
in the ``/etc/supervisor/conf.d`` directory with:
```
sudo supervisorctl reread
```
Followed by telling it to enact any changes with:
```
sudo supervisorctl update
```
Once our programs are running,
there will undoubtedly be a time
when we want to stop, restart, or see their status.
```
sudo supervisorctl status
```
For restart your celery instance:
```
sudo supervisorctl restart your_proj_celery
```
---
# [Celery + Supervisor With Django](https://gist.github.com/mau21mau/9371a95b7c14ddf7000c1827b7693801)
---
# [django-celery - Celery Integration for Django](https://docs.celeryproject.org/projects/django-celery/en/stable/introduction.html)
Version: 2.5.3 - 2021/01/15
django-celery 套件/ 模組
但這應不是我想要的結果
----
# [Supervisor: A Process Control System](http://supervisord.org/)
## Introduction
### Overview
Supervisor is a client/server system that allows its users
to control a number of processes on UNIX-like operating systems.
It was inspired by the following:
* __Convenience__
It is often inconvenient to need to write ``rc.d`` scripts
for every single process instance.
``rc.d`` scripts are a great lowest-common-denominator form
of process initialization/autostart/management,
but they can be painful to write and maintain. Additionally,
``rc.d`` scripts cannot automatically restart a crashed process
and many programs do not restart themselves properly on a crash.
Supervisord starts processes as its subprocesses,
and can be configured to automatically restart them on a crash.
It can also automatically be configured to start processes on its own invocation.
* __Accuracy__
It’s often difficult to get accurate up/down status on processes on UNIX.
Pidfiles often lie.
Supervisord starts processes as subprocesses,
so it always knows the true up/down status of its children and
can be queried conveniently for this data.
* __Delegation__
Users who need to control process state often need only to do that.
They don’t want or need full-blown shell access to the machine on which the processes are running. Processes which listen on “low” TCP ports often need to be started and restarted as the root user (a UNIX misfeature). It’s usually the case that it’s perfectly fine to allow “normal” people to stop or restart such a process, but providing them with shell access is often impractical, and providing them with root access or sudo access is often impossible. It’s also (rightly) difficult to explain to them why this problem exists. If supervisord is started as root, it is possible to allow “normal” users to control such processes without needing to explain the intricacies of the problem to them. Supervisorctl allows a very limited form of access to the machine, essentially allowing users to see process status and control supervisord-controlled subprocesses by emitting “stop”, “start”, and “restart” commands from a simple shell or web UI.
* __Process Groups__
Processes often need to be started and stopped in groups, sometimes even in a “priority order”. It’s often difficult to explain to people how to do this. Supervisor allows you to assign priorities to processes, and allows user to emit commands via the supervisorctl client like “start all”, and “restart all”, which starts them in the preassigned priority order. Additionally, processes can be grouped into “process groups” and a set of logically related processes can be stopped and started as a unit.
### Supervisor Components
* __supervisord__
__The server piece of supervisor is named ``supervisord``__.
It is responsible for starting child programs at its own invocation,
responding to commands from clients,
restarting crashed or exited subprocesseses,
logging its subprocess stdout and stderr output, and
generating and handling “events” corresponding to points in subprocess lifetimes.
The server process uses a __configuration file__.
This is typically located in ``/etc/supervisord.conf``.
This configuration file is a “Windows-INI” style config file.
__It is important to keep this file secure
via proper filesystem permissions
because it may contain unencrypted usernames and passwords__.
* __supervisorctl__
The __command-line client piece of the supervisor is named ``supervisorctl``__.
It provides a shell-like interface to the features provided by supervisord.
From ``supervisorctl``, a user can connect to
different ``supervisord`` processes (one at a time),
get status on the subprocesses controlled by,
stop and start subprocesses of,
and get lists of running processes of a ``supervisord``.
The command-line client talks to the server
across a UNIX domain socket or an internet (TCP) socket.
The server can assert that the user of a client
should present authentication credentials
before it allows him to perform commands.
The client process typically uses the same configuration file
as the server but any configuration file with a [supervisorctl] section in it will work.
* __Web Server__
A (sparse) web user interface with functionality comparable to supervisorctl may be accessed via a browser if you start supervisord against an internet socket. Visit the server URL (e.g. http://localhost:9001/) to view and control process status through the web interface after activating the configuration file’s [inet_http_server] section.
* __XML-RPC Interface__
The same HTTP server which serves the web UI serves up an XML-RPC interface that can be used to interrogate and control supervisor and the programs it runs. See XML-RPC API Documentation.
## [Installing to A System With Internet Access](http://supervisord.org/installing.html#installing-to-a-system-with-internet-access)
### Internet-Installing With Pip
Supervisor can be installed with ``pip`` install:
```
pip install supervisor
```
Depending on the permissions of your system’s Python,
you might need to be the root user to install Supervisor successfully using ``pip``.
You can also install ``supervisor`` in a ``virtualenv`` via ``pip``.
---
Sign in with Wallet
Connect another wallet