pantsbuild / example-django Goto Github PK

View Code? Open in Web Editor NEW

32.0 32.0 24.0 161 KB

An example repo to demonstrate Django support in Pants

License: Apache License 2.0

Starlark 13.75% Shell 1.94% Python 83.58% HTML 0.65% CSS 0.09%

example-django's People

Contributors

Stargazers

Watchers

example-django's Issues

Dependency of one service on another while deploying

Hi,

I am implementing a DRF based SAAS application based on the concept and structure mentioned in this repo. My application has apps which handles the necessary models, serializers and views. It also has services which depend on one or more apps. By this i mean that an app can be used in one or more services which in turn deduces the fact that a database can be shared by multiple services. I have encountered 2 edge case which i need to solve before moving it into production.

Case 1: How do we handle the deployment of services dependent on a common app when the database schema related to the app changes? Let us say, If serviceA and serviceB uses the appC which connects to databaseZ. Both the services are deployed separately using pex files. If i make some model changes in appC and initiate the deployment of both services A and B. If the deployment of serviceA happens fast, it makes the necessary migrations in the databaseZ which might effect the business logic in serviceB and results in the failure of requests to B.

Case 2: How do we handle race conditions in accessing databaseZ by both services? Let us say, serviceA is writing a transaction to databaseZ and serviceB also wants to write to the databaseZ. Won't this result in a lock and make serviceB wait.?

Please help me understand how to handle the above 2 cases and let me know if i have missed some detail while implementing.

Thanks,

Migrate is not creating any tables

Haven't figured out why exactly but ./pants run helloworld/service/admin/manage.py -- migrate --database=users --database=greetings is not creating any django-related tables for me, making the example not usable.

Migrations are running and obviously applying, but no django related tables are getting created.

❯ ./pants run helloworld/service/admin/manage.py -- migrate --database=users --database=greetings

Operations to perform:
  Apply all migrations: admin, auth, contenttypes, greet, person, sessions, translate
Running migrations:
  Applying contenttypes.0001_initial... OK
  Applying auth.0001_initial... OK
  Applying admin.0001_initial... OK
  Applying admin.0002_logentry_remove_auto_add... OK
  Applying admin.0003_logentry_add_action_flag_choices... OK
  Applying contenttypes.0002_remove_content_type_name... OK
  Applying auth.0002_alter_permission_name_max_length... OK
  Applying auth.0003_alter_user_email_max_length... OK
  Applying auth.0004_alter_user_username_opts... OK
  Applying auth.0005_alter_user_last_login_null... OK
  Applying auth.0006_require_contenttypes_0002... OK
  Applying auth.0007_alter_validators_add_error_messages... OK
  Applying auth.0008_alter_user_username_max_length... OK
  Applying auth.0009_alter_user_last_name_max_length... OK
  Applying auth.0010_alter_group_name_max_length... OK
  Applying auth.0011_update_proxy_permissions... OK
  Applying auth.0012_alter_user_first_name_max_length... OK
  Applying greet.0001_initial... OK
  Applying greet.0002_data... OK
  Applying person.0001_initial... OK
  Applying person.0002_data... OK
  Applying sessions.0001_initial... OK
  Applying translate.0001_initial... OK
  Applying translate.0002_data... OK

Here's the resulting users.sqlite3 inspected using DataGrip:

Here's the resulting greetings.sqlite3 inspected using DataGrip:

As you can see, greet_greetings is created as well as translate_translations but the other django related tables are not being created.

Full example

Is there an ETA when this Example is done?

pytest-django could not find a Django project

Thanks to this example repo, I started experimenting with Django. I have added a simple django project following this tutorial while using the same dependency versions as in this repo. My pants.toml looks like this:

[GLOBAL]
pants_version = "2.11.0"

backend_packages = [
  "pants.backend.python",
]

[anonymous-telemetry]
enabled = false


[python]
# We use a narrow interpreter constraint to ensure that Python PEX'es will be executable by our
# selected Docker base image, `python:3.10`.
interpreter_constraints = ["==3.9.*"]
enable_resolves = true
resolves = { python-default = "lockfiles/python-default.txt" }
lockfile_generator = "pex"

[source]
root_patterns = [
  "/django"
]

[pytest]
lockfile = "lockfiles/pytest.txt"
version = "pytest>=6.2.4,<6.3"
extra_requirements.add = [
  "pytest-django>=4,<5",
]

[python-infer]
# Infer dependencies from strings that look like module/class names, such as are often
# found in settings.py, where dependencies are enumerated as strings and not directly imported.
string_imports = true
string_imports_min_dots = 1

My project strucuture looks like this

pants.toml
pytest.ini
  /django
    manage.py
    /mysite
      settings.py
    /polls
      tests.py

My pytest.ini looks as follows:

[pytest]
DJANGO_SETTINGS_MODULE = mysite.settings
pythonpath = .

I have been experimenting with the pythonpath option, but I am not able to run the test with ./pants test django/polls/tests.py as it throws the following error:

15:17:29.99 [INFO] Initializing scheduler...
15:17:30.13 [INFO] Scheduler initialized.
15:17:31.63 [WARN] Failed to generate JUnit XML data for django/polls/tests.py:tests.
15:17:31.63 [ERROR] Completed: Run Pytest - django/polls/tests.py:tests failed (exit code 1).
Traceback (most recent call last):
  File "/home/vscode/.cache/pants/named_caches/pex_root/venvs/s/64a59a10/venv/lib/python3.9/site-packages/pytest_django/plugin.py", line 179, in _handle_import_error
    yield
  File "/home/vscode/.cache/pants/named_caches/pex_root/venvs/s/64a59a10/venv/lib/python3.9/site-packages/pytest_django/plugin.py", line 351, in pytest_load_initial_conftests
    dj_settings.DATABASES
  File "/home/vscode/.cache/pants/named_caches/pex_root/venvs/s/64a59a10/venv/lib/python3.9/site-packages/django/conf/__init__.py", line 82, in __getattr__
    self._setup(name)
  File "/home/vscode/.cache/pants/named_caches/pex_root/venvs/s/64a59a10/venv/lib/python3.9/site-packages/django/conf/__init__.py", line 69, in _setup
    self._wrapped = Settings(settings_module)
  File "/home/vscode/.cache/pants/named_caches/pex_root/venvs/s/64a59a10/venv/lib/python3.9/site-packages/django/conf/__init__.py", line 170, in __init__
    mod = importlib.import_module(self.SETTINGS_MODULE)
  File "/usr/local/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 972, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 984, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'mysite'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/tmp/process-execution6KqLW2/.cache/pex_root/venvs/0b6a9b078e047f3c7341f09b2e9163fb0d6a7996/83912a035d4805bd1b6618bfd9e98f78cda4d615/pex", line 182, in <module>
    sys.exit(func())
  File "/home/vscode/.cache/pants/named_caches/pex_root/venvs/s/64a59a10/venv/lib/python3.9/site-packages/_pytest/config/__init__.py", line 185, in console_main
    code = main()
  File "/home/vscode/.cache/pants/named_caches/pex_root/venvs/s/64a59a10/venv/lib/python3.9/site-packages/_pytest/config/__init__.py", line 143, in main
    config = _prepareconfig(args, plugins)
  File "/home/vscode/.cache/pants/named_caches/pex_root/venvs/s/64a59a10/venv/lib/python3.9/site-packages/_pytest/config/__init__.py", line 318, in _prepareconfig
    config = pluginmanager.hook.pytest_cmdline_parse(
  File "/home/vscode/.cache/pants/named_caches/pex_root/venvs/s/64a59a10/venv/lib/python3.9/site-packages/pluggy/_hooks.py", line 265, in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
  File "/home/vscode/.cache/pants/named_caches/pex_root/venvs/s/64a59a10/venv/lib/python3.9/site-packages/pluggy/_manager.py", line 80, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/home/vscode/.cache/pants/named_caches/pex_root/venvs/s/64a59a10/venv/lib/python3.9/site-packages/pluggy/_callers.py", line 55, in _multicall
    gen.send(outcome)
  File "/home/vscode/.cache/pants/named_caches/pex_root/venvs/s/64a59a10/venv/lib/python3.9/site-packages/_pytest/helpconfig.py", line 100, in pytest_cmdline_parse
    config: Config = outcome.get_result()
  File "/home/vscode/.cache/pants/named_caches/pex_root/venvs/s/64a59a10/venv/lib/python3.9/site-packages/pluggy/_result.py", line 60, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/home/vscode/.cache/pants/named_caches/pex_root/venvs/s/64a59a10/venv/lib/python3.9/site-packages/pluggy/_callers.py", line 39, in _multicall
    res = hook_impl.function(*args)
  File "/home/vscode/.cache/pants/named_caches/pex_root/venvs/s/64a59a10/venv/lib/python3.9/site-packages/_pytest/config/__init__.py", line 1003, in pytest_cmdline_parse
    self.parse(args)
  File "/home/vscode/.cache/pants/named_caches/pex_root/venvs/s/64a59a10/venv/lib/python3.9/site-packages/_pytest/config/__init__.py", line 1283, in parse
    self._preparse(args, addopts=addopts)
  File "/home/vscode/.cache/pants/named_caches/pex_root/venvs/s/64a59a10/venv/lib/python3.9/site-packages/_pytest/config/__init__.py", line 1191, in _preparse
    self.hook.pytest_load_initial_conftests(
  File "/home/vscode/.cache/pants/named_caches/pex_root/venvs/s/64a59a10/venv/lib/python3.9/site-packages/pluggy/_hooks.py", line 265, in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
  File "/home/vscode/.cache/pants/named_caches/pex_root/venvs/s/64a59a10/venv/lib/python3.9/site-packages/pluggy/_manager.py", line 80, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/home/vscode/.cache/pants/named_caches/pex_root/venvs/s/64a59a10/venv/lib/python3.9/site-packages/pluggy/_callers.py", line 60, in _multicall
    return outcome.get_result()
  File "/home/vscode/.cache/pants/named_caches/pex_root/venvs/s/64a59a10/venv/lib/python3.9/site-packages/pluggy/_result.py", line 60, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/home/vscode/.cache/pants/named_caches/pex_root/venvs/s/64a59a10/venv/lib/python3.9/site-packages/pluggy/_callers.py", line 39, in _multicall
    res = hook_impl.function(*args)
  File "/home/vscode/.cache/pants/named_caches/pex_root/venvs/s/64a59a10/venv/lib/python3.9/site-packages/pytest_django/plugin.py", line 351, in pytest_load_initial_conftests
    dj_settings.DATABASES
  File "/usr/local/lib/python3.9/contextlib.py", line 137, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/home/vscode/.cache/pants/named_caches/pex_root/venvs/s/64a59a10/venv/lib/python3.9/site-packages/pytest_django/plugin.py", line 183, in _handle_import_error
    raise ImportError(msg)
ImportError: No module named 'mysite'

Any idea? Is the pytest PYTHONPATH messing with the pants PYTHONPATH? I also thought, that I would not need to specify the path as it is a standard Django installation with a default location for manage.py.

You can find the source code here.

pants tailor command generating BUILD files without dependencies

First of all, Thanks for the repo and code structure. I intend to use this structure along with pants and docker for an ongoing project but facing issue while generating BUILD files.

I did a checkout of this repo and deleted BUILD files in helloworld/person and helloworld/service/admin/. When i try to run the pants tailor :: command now, the generated BUILD files are missing the dependencies.

The initial checkout BUILD file in helloworld/person looked like:

python_sources(
    dependencies=[
        "helloworld/person/migrations",
    ]
)

python_tests(
    name="tests",
    dependencies=[
        "helloworld",  # For settings.py.
    ],
)

python_test_utils(
    name="test_utils",
)

Whereas after deleting and running pants tailor :: command, the BUILD file changed to:

python_sources()

python_test_utils(
    name="test_utils",
)

python_tests(
    name="tests",
)

my pants.toml looks like:

[GLOBAL]
pants_version = "2.15.0"

backend_packages.add = [
  'pants.backend.python',
  'pants.backend.python.lint.docformatter',
  'pants.backend.python.lint.black',
  'pants.backend.python.lint.flake8',
  'pants.backend.python.lint.isort',
  'pants.backend.python.typecheck.mypy',
  'pants.backend.docker'
]

[source]
# The Python source root is the repo root. See https://www.pantsbuild.org/docs/source-roots.
root_patterns = ["/"]

[python]
# The default interpreter compatibility for code in this repo. Individual targets can override
#  this with the `compatibility` field. See
#  https://www.pantsbuild.org/docs/python-interpreter-compatibility.
interpreter_constraints = [">=3.11"]
# Use a lockfile. See https://www.pantsbuild.org/docs/python-third-party-dependencies.
enable_resolves = true
resolves = { python-default = "lockfiles/python-default.lock" }

[python-bootstrap]
# We search for interpreters on both on the $PATH and in the `$(pyenv root)/versions` folder.
# Note the gotcha that the order of entries does not matter in this option, as Pants will consider
# all interpreters it finds and select a compatible one.
# So, if you're using macOS, you may want to leave off the <PATH> entry to avoid using the
# problematic system Pythons. See
# https://www.pantsbuild.org/docs/python-interpreter-compatibility#changing-the-interpreter-search-path.
search_path = ["<PATH>", "<PYENV>"]

[python-infer]
# Infer dependencies from strings that look like module/class names, such as are often
# found in settings.py, where dependencies are enumerated as strings and not directly imported.
string_imports = true

[pytest]
lockfile = "lockfiles/pytest.lock"
version = "pytest>=7.3.1"
extra_requirements.add = [
  "pytest-django>=4,<5",
]

execution_slot_var = "PANTS_EXECUTION_SLOT"

[mypy]
lockfile = "lockfiles/mypy.lock"
version = "mypy==1.3.0"
interpreter_constraints = [">=3.11"]
extra_requirements.add = [
  # This is a mypy plugin, as well as a type stubs repository. So it must be mentioned
  # here as well as in requirements.txt.
  "django-stubs==4.2.0"
]

[flake8]
args=[
  "--max-line-length=120"
]

Do we have to write the dependencies ourselves if we want to maintain this monorepo structure for Django project? or am i missing something that supports autogeneration of BUILD files with dependencies?

Templates are not covered by any target or inference.

It does not appear that the templates are owned by any target.

/tmp/tmp.wRy8z5idAO/example-django  ‹main› 
╰─➤  ./pants list :: | grep ".html"

/tmp/tmp.wRy8z5idAO/example-django  ‹main› 
╰─➤  ./pants dependees helloworld/ui/templates/helloworld/ui/index.html 
15:30:21.73 [ERROR] Exception caught: (pants.engine.internals.scheduler.ExecutionError)
  File "/home/xlevus/.cache/pants/setup/bootstrap-Linux-x86_64/2.8.0rc0_py39/lib/python3.9/site-packages/pants/bin/local_pants_runner.py", line 235, in _run_inner
    return self._perform_run(goals)
  File "/home/xlevus/.cache/pants/setup/bootstrap-Linux-x86_64/2.8.0rc0_py39/lib/python3.9/site-packages/pants/bin/local_pants_runner.py", line 174, in _perform_run
    return self._perform_run_body(goals, poll=False)
  File "/home/xlevus/.cache/pants/setup/bootstrap-Linux-x86_64/2.8.0rc0_py39/lib/python3.9/site-packages/pants/bin/local_pants_runner.py", line 191, in _perform_run_body
    return self.graph_session.run_goal_rules(
  File "/home/xlevus/.cache/pants/setup/bootstrap-Linux-x86_64/2.8.0rc0_py39/lib/python3.9/site-packages/pants/init/engine_initializer.py", line 133, in run_goal_rules
    exit_code = self.scheduler_session.run_goal_rule(
  File "/home/xlevus/.cache/pants/setup/bootstrap-Linux-x86_64/2.8.0rc0_py39/lib/python3.9/site-packages/pants/engine/internals/scheduler.py", line 548, in run_goal_rule
    self._raise_on_error([t for _, t in throws])
  File "/home/xlevus/.cache/pants/setup/bootstrap-Linux-x86_64/2.8.0rc0_py39/lib/python3.9/site-packages/pants/engine/internals/scheduler.py", line 516, in _raise_on_error
    raise ExecutionError(

Exception message: 1 Exception encountered:

  ResolveError: No owning targets could be found for the file `helloworld/ui/templates/helloworld/ui/index.html`.

Please check that there is a BUILD file in the parent directory helloworld/ui/templates/helloworld/ui with a target whose `sources` field includes the file. See https://www.pantsbuild.org/v2.8/docs/targets for more information on target definitions.

You may want to run `./pants tailor` to autogenerate your BUILD files. See https://www.pantsbuild.org/v2.8/docs/create-initial-build-files.

If you would like to ignore un-owned files, please pass `--owners-not-found-behavior=ignore`.



Use --print-stacktrace for more error details and/or -ldebug for more logs. 
See https://www.pantsbuild.org/v2.8/docs/troubleshooting for common issues.
Consider reaching out for help: https://www.pantsbuild.org/v2.8/docs/getting-help

Broken migrations example

When running the example at the bottom of the README, the database migration code is run...

(note: the README is missing the run goal)

$ ./pants run helloworld/service/admin/manage.py -- migrate --database=users --database=greetings
Operations to perform:
  Apply all migrations: admin, auth, contenttypes, greet, person, sessions, translate
Running migrations:
  Applying contenttypes.0001_initial... OK
  Applying auth.0001_initial... OK
  Applying admin.0001_initial... OK
  Applying admin.0002_logentry_remove_auto_add... OK
  Applying admin.0003_logentry_add_action_flag_choices... OK
  Applying contenttypes.0002_remove_content_type_name... OK
  Applying auth.0002_alter_permission_name_max_length... OK
  Applying auth.0003_alter_user_email_max_length... OK
  Applying auth.0004_alter_user_username_opts... OK
  Applying auth.0005_alter_user_last_login_null... OK
  Applying auth.0006_require_contenttypes_0002... OK
  Applying auth.0007_alter_validators_add_error_messages... OK
  Applying auth.0008_alter_user_username_max_length... OK
  Applying auth.0009_alter_user_last_name_max_length... OK
  Applying auth.0010_alter_group_name_max_length... OK
  Applying auth.0011_update_proxy_permissions... OK
  Applying auth.0012_alter_user_first_name_max_length... OK
  Applying greet.0001_initial... OK
  Applying greet.0002_data... OK
  Applying person.0001_initial... OK
  Applying person.0002_data... OK
  Applying sessions.0001_initial... OK
  Applying translate.0001_initial... OK
  Applying translate.0002_data... OK

But the .sqlite3 database files are not created.

$ find -name "*.sqlite3"   
... <nothing> ...
$

A similar issue exists when running other filesystem mutating commands such as makemigrations.

pantsbuild / example-django Goto Github PK

example-django's People

Contributors

Stargazers

Watchers

Forkers

example-django's Issues

Dependency of one service on another while deploying

Migrate is not creating any tables

Full example

pytest-django could not find a Django project

pants tailor command generating BUILD files without dependencies

Templates are not covered by any target or inference.

Broken migrations example

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent