๐ Need help with your Django project? HackSoft is here for you. Reach out at
[email protected]
Table of contents:
- How to ask a question or propose something?
- What is this?
- Structure
- General API Stuff
- Authentication - JWT
- Authentication - Sessions
- Example List API
- File uploads
- Helpful commands for local development without
docker compose
- Helpful commands for local development with
docker compose
- Deployment
- Linters and Code Formatters
Few points to navigate yourself:
- If you have an issue with something related to the Django Styleguide Example - just open an issue. We will respond.
- If you have a general question or suggestion - just open na issue. We will respond.
- Even if you have a question that you are not sure if it's related to the Django Styleguide - just open an issue anyway. We will respond.
That's about it โจ
Hello ๐
This projects serves as the following:
- As an example of our Django Styleguide, where people can explore actual code & not just snippets.
- As a Django project, where we can test various things & concepts. A lot of the things you see here are being used as a foundation of our internal projects at HackSoft.
- Usually, this is how something ends up as a section in the Django Styleguide
- As a place for all code examples from our blog.
- Code snippets tend to decay & we want most of our blog articles to be up to date. That's why we place the code here, write tests for it & guarantee a longer shelf life of the examples.
If you want to learn more about the Django Styleguide, you can watch the videos below:
Radoslav Georgiev's Django structure for scale and longevity for the philosophy behind the styleguide:
Radoslav Georgiev & Ivaylo Bachvarov's discussion on HackCast, around the Django Styleguide:
The initial structure was inspired by cookiecutter-django.
The structure now is modified based on our work & production experience with Django.
Few important things:
- Linux / Ubuntu is our primary OS and things are tested for that.
- It's dockerized for local development with
docker compose
. - It uses Postgres as the primary database.
- It comes with
whitenoise
setup, even for local development. - It comes with
mypy
configured, using both https://github.com/typeddjango/django-stubs and https://github.com/typeddjango/djangorestframework-stubs/- Basic
mypy
configuration is located insetup.cfg
mypy
is ran as a build step in.github/workflows/django.yml
โ ๏ธ The provided configuration is quite minimal. You should figure out your team needs & configure accordingly - https://mypy.readthedocs.io/en/stable/config_file.html
- Basic
- It comes with GitHub Actions support, based on that article
- It can be easily deployed to Heroku, Render or AWS ECS.
- It comes with an example list API, that uses
django-filter
for filtering & pagination from DRF. - It comes with setup for Django Debug Toolbar
- It comes with examples for writing tests with fakes & factories, based on the following articles - https://www.hacksoft.io/blog/improve-your-tests-django-fakes-and-factories, https://www.hacksoft.io/blog/improve-your-tests-django-fakes-and-factories-advanced-usage
- It comes with examples for how to add Google login, based on the following article - https://www.hacksoft.io/blog/adding-google-login-to-your-existing-django-and-django-rest-framework-applications
The project is running django-cors-headers
with the following general configuration:
CORS_ALLOW_CREDENTIALS = True
CORS_ALLOW_ALL_ORIGINS = True
For production.py
, we have the following:
CORS_ALLOW_ALL_ORIGINS = False
CORS_ORIGIN_WHITELIST = env.list('CORS_ORIGIN_WHITELIST', default=[])
The project is using https://github.com/Styria-Digital/django-rest-framework-jwt for having authentication via JWT capabilities.
All JWT related settings are located in config/settings/jwt.py
.
โ ๏ธ We highly recommend reading the entire settings page from the project documentation - https://styria-digital.github.io/django-rest-framework-jwt/#additional-settings - to figure out your needs & the proper defaults for you!
The default settings also include the JWT token as a cookie.
The specific details about how the cookie is set, can be found here - https://github.com/Styria-Digital/django-rest-framework-jwt/blob/master/src/rest_framework_jwt/compat.py#L43
The JWT related APIs are:
/api/auth/jwt/login/
/api/auth/jwt/logout/
The current implementation of the login API returns just the token:
{
"token": "eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ1c2VybmFtZSI6InJhZG9yYWRvQGhhY2tzb2Z0LmlvIiwiaWF0IjoxNjQxMjIxMDMxLCJleHAiOjE2NDE4MjU4MzEsImp0aSI6ImIyNTEyNmY4LTM3ZDctNGI5NS04Y2M0LTkzZjI3MjE4ZGZkOSIsInVzZXJfaWQiOjJ9.TUoQQPSijO2O_3LN-Pny4wpQp-0rl4lpTs_ulkbxzO4"
}
This can be changed from auth_jwt_response_payload_handler
.
We follow this concept:
- All APIs are public by default (no default authentication classes)
- If you want a certain API to require authentication, you add the
ApiAuthMixin
to it.
This project is using the already existing cookie-based session authentication in Django:
- On successful authentication, Django returns the
sessionid
cookie:
sessionid=5yic8rov868prmfoin2vhtg4vx35h71p; expires=Tue, 13 Apr 2021 11:17:58 GMT; HttpOnly; Max-Age=1209600; Path=/; SameSite=Lax
- When making calls from the frontend, don't forget to include credentials. For example, when using
axios
:
axios.get(url, { withCredentials: true });
axios.post(url, data, { withCredentials: true });
-
For convenience,
CSRF_USE_SESSIONS
is set toTrue
-
Check
config/settings/sessions.py
for all configuration that's related to sessions.
Since the default implementation of SessionAuthentication
enforces CSRF check, which is not the desired behavior for our APIs, we've done the following:
from rest_framework.authentication import SessionAuthentication
class CsrfExemptedSessionAuthentication(SessionAuthentication):
"""
DRF SessionAuthentication is enforcing CSRF, which may be problematic.
That's why we want to make sure we are exempting any kind of CSRF checks for APIs.
"""
def enforce_csrf(self, request):
return
Which is then used to construct an ApiAuthMixin
, which marks an API that requires authentication:
from rest_framework.permissions import IsAuthenticated
class ApiAuthMixin:
authentication_classes = (CsrfExemptedSessionAuthentication, )
permission_classes = (IsAuthenticated, )
By default, all APIs are public, unless you add the ApiAuthMixin
We have the following general cases:
- The current configuration works out of the box for
localhost
development. - If the backend is located on
*.domain.com
and the frontend is located on*.domain.com
, the configuration is going to work out of the box. - If the backend is located on
somedomain.com
and the frontend is located onanotherdomain.com
, then you'll need to setSESSION_COOKIE_SAMESITE = 'None'
andSESSION_COOKIE_SECURE = True
POST
to/api/auth/session/login/
requires JSON body withemail
andpassword
.GET
to/api/auth/me/
returns the current user information, if the request is authenticated (has the correspondingsessionid
cookie)GET
orPOST
to/api/auth/logout/
will remove thesessionid
cookie, effectively logging you out.
The current implementation of /api/auth/session/login
does 2 things:
- Sets a
HTTP Only
cookie with the session id. - Returns the actual session id from the JSON payload.
The second thing is required, because Safari is not respecting the SameSite = None
option for cookies.
More on the issue here - https://www.chromium.org/updates/same-site/incompatible-clients
Since cookies can be somewhat elusive, check the following urls:
- https://docs.djangoproject.com/en/3.1/ref/settings/#sessions - It's a good idea to just read every description for
SESSION_*
- https://developer.mozilla.org/en-US/docs/Web/HTTP/Cookies - It's a good idea to read everything, several times.
You can find the UserListApi
in styleguide_example/users/apis.py
List API is located at:
http://localhost:8000/api/users/
The API can be filtered:
- http://localhost:8000/api/users/?is_admin=True
- http://localhost:8000/api/users/?id=1
- http://localhost:8000/api/users/[email protected]
Example data structure:
{
"limit": 1,
"offset": 0,
"count": 4,
"next": "http://localhost:8000/api/users/?limit=1&offset=1",
"previous": null,
"results": [
{
"id": 1,
"email": "[email protected]",
"is_admin": false
}
]
}
Following this article - https://www.hacksoft.io/blog/direct-to-s3-file-upload-with-django - there's a rich file-upload implementation in the Django Styleguide Example.
Everything is located in the files
app.
Configuration wise, everything is located in config/settings/files_and_storages.py
Additionally, you can check the available options in .env.example
Currently, the following is supported:
- Standard local file upload.
- Standard S3 file upload.
- Using CloudFront as CDN.
- The so-called "direct" upload that can work both locally and with S3 (for more context, check the article)
Feel free to use this as the basis of your file upload needs.
To create Postgres database:
sudo -u postgres createdb -O your_postgres_user_here database_name_here
If you want to recreate your database, you can use the bootstrap script:
./scripts/bootstrap.sh your_postgres_user_here
To start Celery:
celery -A styleguide_example.tasks worker -l info --without-gossip --without-mingle --without-heartbeat
To start Celery Beat:
celery -A styleguide_example.tasks beat -l info --scheduler django_celery_beat.schedulers:DatabaseScheduler
To build and run everything
docker compose up
To run migrations
docker compose run django python manage.py migrate
To shell
docker compose run django python manage.py shell
This project is ready to be deployed either on Heroku Render or AWS ECS.
Deploying a Python / Django application on Heroku is quite straighforward & this project is ready to be deployed.
To get an overview of how Heroku deployment works, we recommend reading this first - https://devcenter.heroku.com/articles/deploying-python
Files related to Heroku deployment:
Procfile
- Comes with default
web
,worker
andbeat
processes. - Additionally, there's a
release
phase to run migrations safely, before releasing the new build.
- Comes with default
runtime.txt
- Simply specifies the Python version to be used.
requirements.txt
- Heroku requires a root-level
requirements.txt
, so we've added that.
- Heroku requires a root-level
Additionally, you need to specify at least the following settings:
DJANGO_SETTINGS_MODULE
, usually toconfig.django.production
SECRET_KEY
to something secret. Check here for ideas.ALLOWED_HOSTS
, usually to the default heroku domain (for example -hacksoft-styleguide-example.herokuapp.com
)
On top of that, we've added gunicorn.conf.py
with some example settings.
We recommend the following materials, to figure out gunicorn
defaults and configuration:
- https://devcenter.heroku.com/articles/python-gunicorn
- https://adamj.eu/tech/2019/09/19/working-around-memory-leaks-in-your-django-app/
- https://adamj.eu/tech/2021/12/29/set-up-a-gunicorn-configuration-file-and-test-it/
- Worker settings - https://docs.gunicorn.org/(en/latest/settings.html#worker-processes
- A brief description of the architecture of Gunicorn - https://docs.gunicorn.org/en/latest/design.html
To get an overview of how Render deployment works, we recommend reading this first - https://render.com/docs/deploy-django
There's a current deployment that can be found here - https://django-styleguide.hacksoft.io/
Files related to Heroku deployment:
render.yaml
- Describes the setup. Also known as Render Blueprint
docker/*_entrypoint.sh
- Entrypoint for every different process type.
docker/production.Dockerfile
- Dockerfile for production build.
requirements.txt
- Heroku requires a root-level
requirements.txt
, so we've added that.
- Heroku requires a root-level
Coming soon
In all our Django projects we use:
- flake8 - a linter that ensures we follow the PEP8 conventions.
- black - a code formatter that ensures we have the same code style everywhere.
- isort - a code formatter that ensures we have the same import style everywhere.
- pre-commit - a tool that triggers the linters before each commit.
To make sure all of the above tools work in symbiosis, you'd need to add some configuration:
- Add
.pre-commit-config.yaml
file to the root of your project. There you can add the instructions forpre-commit
- Add
pyproject.toml
file to the root of your project. There you can add theblack
config. NOTE:black
does not respect any other config files. - Add the following to
setup.cfg
for theisort
config:
[isort]
profile = black
This will tell isort
to follow the black
guidelines.
[isort]
filter_files = true
skip_glob = */migrations/*
This will tell pre-commit
to respect the isort
config.
- You can add a custom
flake8
configuration tosetup.cfg
as well. We usually have the following config in all our projects:
[flake8]
max-line-length = 120
extend-ignore = E203
exclude =
.git,
__pycache__,
*/migrations/*
- Make sure the linters are run against each PR on your CI. This is the config you need if you use GH actions:
build:
runs-on: ubuntu-latest
steps:
- name: Run isort
uses: isort/isort-action@master
- name: Run black
uses: psf/black@stable
- name: Run flake8
run: flake8
- Last but not least, we highly recommend you to setup you editor to run
black
andisort
every time you save a new Python file.
In order to test if your local setup is up to date, you can either:
- Try making a commit, to see if
pre-commit
is going to be triggered. - Or run
black --check .
andisort --check .
in the project root directory.