# Setting Up Celery for Your Django Project on Ubuntu Server: A Comprehensive Guide

In this tutorial, we'll walk you through the process of configuring Celery, a distributed task queue, with your Django project on an Ubuntu server. Celery enables the asynchronous execution of tasks, enhancing the performance and scalability of your application. We'll break down the configuration of the `celeryd` and `celery.service` files step by step, helping even novice developers get started.

![Setting Up Celery for Your Django Project on Ubuntu Server](https://cdn.hashnode.com/res/hashnode/image/upload/v1697359470540/b47622c1-a411-4192-9891-c669a14ea3da.jpeg align="center")

## Setting Up Celery

In this section, we will dive into the crucial process of configuring *Celery* for your Django project. Celery is a powerful tool that enables you to run asynchronous tasks in the background, enhancing the performance and scalability of your web application.

### **Prerequisites**

Before we proceed, make sure you have the following tools and prerequisites in place:

* A working **Django** project.
    
* A *Linux* server (in this case, we'll focus on configuring Celery on an **Ubuntu** server).
    
* *Python 3.x* installed.
    
* A virtual environment is set up for your Django project.
    

If you haven't set up your Django project or installed Python 3.x, please ensure you do so before continuing.

## Creating /etc/default/celeryd

**Section 1:** `celeryd` Configuration

1. **Create the Configuration File:** To begin, navigate to the `/etc/default` directory and create a new file named `celeryd` to hold your Celery configuration.
    
2. **Defining Node Configuration:** Specify the number of nodes for Celery using the `CELERYD_NODES` variable. For most setups, a single node is sufficient:
    
    ```plaintext
    CELERYD_NODES="celery"
    ```
    
3. **Setting Celery Binary Path:** Define the absolute or relative path to the Celery command using the `CELERY_BIN` variable. For example:
    
    ```plaintext
    CELERY_BIN="/home/akshaykaushik.eu.org/AI/.venv/bin/celery"
    ```
    
4. **Specifying Django App Instance:** Set the Django app instance to be used with Celery using the `CELERY_APP` variable, It is the Django project name:
    
    ```plaintext
    CELERY_APP="Project"
    ```
    
5. **Setting Working Directory:** Specify the directory where Celery should operate from using the `CELERYD_CHDIR` variable(path of directory in which Django manage.py exists):
    
    ```plaintext
    CELERYD_CHDIR="/home/akshaykaushik.eu.org/AI/"
    ```
    
6. **Configuring Logging:** Define the logging level and paths for Celery logs using the following variables:
    
    ```plaintext
    CELERYD_LOG_LEVEL="INFO"
    CELERYD_LOG_FILE="/home/akshaykaushik.eu.org/AI/log/celery/%n%I.log"
    CELERYD_PID_FILE="/home/akshaykaushik.eu.org/AI/run/celery/%n.pid"
    ```
    
7. **Specifying User and Group:** Set the unprivileged user and group for Celery workers using the following variables(user and group of the django folder):
    
    ```plaintext
    CELERYD_USER="aksha1706"
    CELERYD_GROUP="aksha1706"
    ```
    
8. **Directory Creation and Permissions:** Enable directory creation for missing PID and log directories and configure them to be owned by the specified user/group:
    
    ```plaintext
    CELERY_CREATE_DIRS=1
    ```
    
9. **Activating Virtual Environment:** Specify the virtual environment activation command:
    
    ```plaintext
    CELERYD_ENV="+/home/akshaykaushik.eu.org/AI/.venv/bin/activate"
    ```
    
10. **Setting Permissions:** To ensure the Celery worker has the necessary permissions, follow these steps:
    
    * Execute the command `chmod +x /etc/init.d/celeryd`.
        
    
    * Create the log and PID folders specified in the configuration file and grant them write permissions.
        
    
    * With these permissions in place, your Celery worker will operate smoothly.
        

```powershell
# most people will only start one node:
CELERYD_NODES="celery"

# Absolute or relative path to the 'celery' command:
CELERY_BIN="/home/akshaykaushik.eu.org/AI/.venv/bin/celery"

# App instance to use
CELERY_APP="Project"

# Where to chdir at start.
CELERYD_CHDIR="/home/akshaykaushik.eu.org/AI/"

# Set logging level to INFO
CELERYD_LOG_LEVEL="INFO"

# %n will be replaced with the first part of the node name.
CELERYD_LOG_FILE="/home/akshaykaushik.eu.org/AI/log/celery/%n%I.log"
CELERYD_PID_FILE="/home/akshaykaushik.eu.org/AI/run/celery/%n.pid"

# Workers should run as an unprivileged user.
CELERYD_USER="aksha1706"
CELERYD_GROUP="aksha1706"

# If enabled, PID and log directories will be created if missing,
# and owned by the userid/group configured.
CELERY_CREATE_DIRS=1

# Activate the virtual environment
CELERYD_ENV="+/home/akshaykaushik.eu.org/AI/.venv/bin/activate"
```

## Creating a systemd Service for Celery

**Section 2:** `celery.service` Configuration

1. **Create the Service File:** Navigate to the `/etc/systemd/system` directory and create a new file named `celery.service`.
    
2. **Defining the Unit:** In the `celery.service` file, describe the service and set its dependencies:
    
    ```plaintext
    [Unit]
    Description=Celery Service
    After=network.target
    ```
    
3. **Configure the Service:** Define the service type, user, group, and reference the `celeryd` configuration file:
    
    ```plaintext
    [Service]
    Type=forking
    User=aksha1706
    Group=aksha1706
    EnvironmentFile=/etc/default/celeryd
    ```
    
4. **Set Working Directory and ExecStart:** Specify the working directory and the command to start the Celery service:
    
    ```plaintext
    WorkingDirectory=/home/akshaykaushik.eu.org/AI
    ExecStart=/bin/bash -c "source /home/akshaykaushik.eu.org/AI/.venv/bin/activate && \
      /home/akshaykaushik.eu.org/AI/.venv/bin/celery multi start ${CELERYD_NODES} \
      -A Project --pidfile=${CELERYD_PID_FILE} \
      --logfile=${CELERYD_LOG_FILE} --loglevel=${CELERYD_LOG_LEVEL}"
    ```
    
5. **Define ExecStop and ExecReload:** Specify the commands to stop and reload the Celery service:
    
    ```plaintext
    ExecStop=/bin/bash -c "source /home/akshaykaushik.eu.org/AI/.venv/bin/activate && \
      /home/akshaykaushik.eu.org/AI/.venv/bin/celery multi stopwait ${CELERYD_NODES} \
      --pidfile=${CELERYD_PID_FILE}"
    ExecReload=/bin/bash -c "source /home/akshaykaushik.eu.org/AI/.venv/bin/activate && \
      /home/akshaykaushik.eu.org/AI/.venv/bin/celery multi restart ${CELERYD_NODES} \
      -A Project --pidfile=${CELERYD_PID_FILE} \
      --logfile=${CELERYD_LOG_FILE} --loglevel=${CELERYD_LOG_LEVEL}"
    ```
    
6. **Define Installation:** Specify the target for installation:
    
    ```plaintext
    [Install]
    WantedBy=multi-user.target
    ```
    
    ```powershell
    [Unit]
    Description=Celery Service
    After=network.target
    
    [Service]
    Type=forking
    User=aksha1706
    Group=aksha1706
    
    EnvironmentFile=/etc/default/celeryd
    WorkingDirectory=/home/akshaykaushik.eu.org/AI
    ExecStart=/bin/bash -c "source /home/akshaykaushik.eu.org/AI/.venv/bin/activate && \
      /home/akshaykaushik.eu.org/AI/.venv/bin/celery multi start ${CELERYD_NODES} \
      -A Project --pidfile=${CELERYD_PID_FILE} \
      --logfile=${CELERYD_LOG_FILE} --loglevel=${CELERYD_LOG_LEVEL}"
    ExecStop=/bin/bash -c "source /home/akshaykaushik.eu.org/AI/.venv/bin/activate && \
      /home/akshaykaushik.eu.org/AI/.venv/bin/celery multi stopwait ${CELERYD_NODES} \
      --pidfile=${CELERYD_PID_FILE}"
    ExecReload=/bin/bash -c "source /home/akshaykaushik.eu.org/AI/.venv/bin/activate && \
      /home/akshaykaushik.eu.org/AI/.venv/bin/celery multi restart ${CELERYD_NODES} \
      -A Project --pidfile=${CELERYD_PID_FILE} \
      --logfile=${CELERYD_LOG_FILE} --loglevel=${CELERYD_LOG_LEVEL}"
    
    [Install]
    WantedBy=multi-user.target
    ```
    
    ## Enabling and Starting Celery
    
    To activate and initiate Celery as a systemd service, follow these steps:
    
    * Run `sudo systemctl daemon-reload` to reload systemd manager configuration.
        
    * Enable the Celery service: `sudo systemctl enable celery`.
        
    * Start the Celery service: `sudo systemctl restart celery`.
        
    
    With these commands, you'll have Celery up and running as a background service.
    
    ## Running Celery
    
    To use Celery with your Django project, you need to run it. After activating the virtual environment and navigating to your Django project folder, execute the following command:
    
    ```powershell
    celery -A CELERY_APP_NAME worker -l INFO
    ```
    
    Make sure to replace `CELERY_APP_NAME` with your actual project name. This command initiates Celery as a worker, ready to process tasks.
    
    In case you need to make changes or updates, rerun the three commands mentioned above. This ensures that your Celery service remains active and responsive.
    

**Conclusion:** Congratulations! You've successfully configured Celery for your Django project on an Ubuntu server. The `celeryd` and `celery.service` files are now optimized to handle asynchronous tasks efficiently. By following these detailed steps, even newcomers to development can set up Celery with ease. Don't forget to run the specified commands to finalize the configuration and enjoy enhanced performance and scalability for your Django application. Happy coding!