Linux Installation Issues about promgen HOT 15 CLOSED

At a glance that looks correct but I will try to double check when I'm back in the office on Tuesday :)
For nginx, I typically use a fairly standard reverse proxy back to uwsgi but I think gunicorn would work similar. I'll try to prepare an example of that as well.

from promgen.

Hi There,

Thank you for replying , is it possible to have a installation guide or maybe check the above steps and see what is missing or how to complete the installation and configuration?

I can help in testing and recheck the guide and come out a install guide for everyone.

Thank You

from promgen.

Looks like you've written a lot already!
One minor concern I have is that some of the instructions seem to be very CentOS7 specific, for example: "Create MariaDB Repository" or "easy_install-3.6 pip" which might be very different for someone wanting to install on Ubuntu or another distribution.

I think for the main README, I should leave it as a simple overview (only the basic steps) but perhaps I can add a more detailed guide to https://promgen.readthedocs.io/en/latest/ and consider a section for each distro regarding the packages to install via yum or apt or other distributions.

from promgen.

Hi Paul,

Well i think that will be great idea, will not mind helping as much as i can but the problem now is fixing the issues such as showing HTTP error, integrate with nginx and such.

Hope to see some resolution soon and to have promgen up and running in my CentOS 7 environment

from promgen.

If you run touch ~/.config/promgen/DEBUG and re-run Promgen, it should use the Django debug page and print out information.

from promgen.

Environment:

Request Method: GET
Request URL: http://10.10.10.13:8000/login/?next=/

Django Version: 1.11
Python Version: 3.6.4
Installed Applications:
['django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.sites',
'django.contrib.staticfiles',
'social_django',
'promgen']
Installed Middleware:
['django.middleware.security.SecurityMiddleware',
'django.middleware.locale.LocaleMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
'promgen.middleware.PromgenMiddleware']

Template loader postmortem
Django tried loading these templates, in this order:

Using engine django:
* django.template.loaders.app_directories.Loader: /usr/lib64/python3.6/site-packages/django/contrib/admin/templates/registration/login.html (Source does not exist)
* django.template.loaders.app_directories.Loader: /usr/lib64/python3.6/site-packages/django/contrib/auth/templates/registration/login.html (Source does not exist)

Traceback:

File "/usr/lib64/python3.6/site-packages/django/core/handlers/exception.py" in inner
41. response = get_response(request)

File "/usr/lib64/python3.6/site-packages/django/core/handlers/base.py" in _get_response
217. response = self.process_exception_by_middleware(e, request)

File "/usr/lib64/python3.6/site-packages/django/core/handlers/base.py" in _get_response
215. response = response.render()

File "/usr/lib64/python3.6/site-packages/django/template/response.py" in render
107. self.content = self.rendered_content

File "/usr/lib64/python3.6/site-packages/django/template/response.py" in rendered_content
82. template = self.resolve_template(self.template_name)

File "/usr/lib64/python3.6/site-packages/django/template/response.py" in resolve_template
64. return select_template(template, using=self.using)

File "/usr/lib64/python3.6/site-packages/django/template/loader.py" in select_template
53. raise TemplateDoesNotExist(', '.join(template_name_list), chain=chain)

Exception Type: TemplateDoesNotExist at /login/
Exception Value: registration/login.html

from promgen.

Interesting. I suppose if you check that path then the templates do not exist under site-packages. I'll try to test it on my own end as well but I suspect when you run python3 setup.py install it only copies the Python files and not the html files. Usually when I develop I install it as python3 setup.py develop (actually I really run pip install -e . or pip install .) which does a setuptools style symlink. I need to test with python setup.py install and make sure I add a MANIFEST.in to ensure the files get copied in properly.

Thank you for the detailed bug report :)

from promgen.

Hi Taylor,

I think after the setup.py is fixed, is it possible to recheck on the required python version and it's dependencies like the pip package to be installed?

Also, how to setup nginx to serve the web page

Lastly, any sample on how to use promgen?

from promgen.

Thank you for your patience. I think I have a fix for it in 5321ac7 if you would like to try it out.

As for Nginx, I use a fairly standard looking configuration similar to this

server {
  listen 80;
  server_name {{promgen_host}};

  access_log /var/log/nginx/promgen.access_log main;
  error_log  /var/log/nginx/promgen.error_log;

  location /static/ { alias /home/promgen/.cache/promgen/; }

  location / {
    proxy_pass http://localhost:{{promgen_port}};
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
  }
}

from promgen.

Hi Taylor,

Good news is template is copied correctly but facing a weird issue as below:

Seeing This If I Run promgen runserver 0.0.0.0:8000 without touch ~/.config/promgen/DEBUG

[09/Jan/2018 09:47:14] "GET /static/admin/css/forms.css HTTP/1.1" 404 7441
[09/Jan/2018 09:47:14] "GET /static/admin/js/core.js HTTP/1.1" 404 7435
[09/Jan/2018 09:47:14] "GET /static/admin/js/vendor/jquery/jquery.min.js HTTP/1.1" 404 7475
[09/Jan/2018 09:47:14] "GET /static/admin/js/urlify.js HTTP/1.1" 404 7439
[09/Jan/2018 09:47:14] "GET /static/admin/js/jquery.init.js HTTP/1.1" 404 7449
[09/Jan/2018 09:47:14] "GET /static/admin/js/admin/RelatedObjectLookups.js HTTP/1.1" 404 7479
[09/Jan/2018 09:47:14] "GET /static/admin/js/actions.min.js HTTP/1.1" 404 7449
[09/Jan/2018 09:47:14] "GET /static/admin/js/prepopulate.min.js HTTP/1.1" 404 7457
[09/Jan/2018 09:47:14] "GET /static/admin/js/vendor/xregexp/xregexp.min.js HTTP/1.1" 404 7479
[09/Jan/2018 09:47:14] "GET /static/admin/js/prepopulate_init.js HTTP/1.1" 404 7459
[09/Jan/2018 09:47:14] "GET /static/admin/js/change_form.js HTTP/1.1" 404 7449
[09/Jan/2018 09:47:14] "GET /static/admin/js/urlify.js HTTP/1.1" 404 7439
[09/Jan/2018 09:47:15] "GET /static/admin/js/prepopulate.min.js HTTP/1.1" 404 7457
[09/Jan/2018 09:47:15] "GET /static/admin/js/vendor/xregexp/xregexp.min.js HTTP/1.1" 404 7479
[09/Jan/2018 09:47:15] "GET /static/admin/js/change_form.js HTTP/1.1" 404 7449
[09/Jan/2018 09:47:15] "GET /static/admin/js/prepopulate_init.js HTTP/1.1" 404 7459
[09/Jan/2018 09:49:34] "GET /login/?next=/ HTTP/1.1" 200 5349
[09/Jan/2018 09:49:34] "GET /static/css/bootstrap-theme.min.css HTTP/1.1" 404 4539
[09/Jan/2018 09:49:34] "GET /static/css/promgen.css HTTP/1.1" 404 4527
[09/Jan/2018 09:49:34] "GET /static/css/bootstrap-datetimepicker.min.css HTTP/1.1" 404 4548
[09/Jan/2018 09:49:34] "GET /static/js/jquery.selection.js HTTP/1.1" 404 4534
[09/Jan/2018 09:49:34] "GET /static/css/bootstrap.min.css HTTP/1.1" 404 4533
[09/Jan/2018 09:49:34] "GET /static/js/jquery.min.js HTTP/1.1" 404 4528
[09/Jan/2018 09:49:34] "GET /static/js/bootstrap.min.js HTTP/1.1" 404 4531
[09/Jan/2018 09:49:34] "GET /static/css/bootstrap-switch.min.css HTTP/1.1" 404 4540
[09/Jan/2018 09:49:34] "GET /static/js/bootstrap3-typeahead.min.js HTTP/1.1" 404 4542
[09/Jan/2018 09:49:34] "GET /static/js/moment.min.js HTTP/1.1" 404 4528
[09/Jan/2018 09:49:34] "GET /static/js/bootstrap-datetimepicker.min.js HTTP/1.1" 404 4546
[09/Jan/2018 09:49:34] "GET /static/js/bootstrap-switch.min.js HTTP/1.1" 404 4538
[09/Jan/2018 09:49:34] "GET /static/js/promgen.js HTTP/1.1" 404 4525
[09/Jan/2018 09:49:34] "GET /static/js/jquery.selection.js HTTP/1.1" 404 4534
[09/Jan/2018 09:49:34] "GET /static/js/bootstrap.min.js HTTP/1.1" 404 4531
[09/Jan/2018 09:49:34] "GET /static/js/bootstrap3-typeahead.min.js HTTP/1.1" 404 4542
[09/Jan/2018 09:49:34] "GET /static/js/moment.min.js HTTP/1.1" 404 4528
[09/Jan/2018 09:49:34] "GET /static/js/bootstrap-datetimepicker.min.js HTTP/1.1" 404 4546
[09/Jan/2018 09:49:34] "GET /static/js/bootstrap-switch.min.js HTTP/1.1" 404 4538
[09/Jan/2018 09:49:34] "GET /static/js/promgen.js HTTP/1.1" 404 4525

If I Run promgen runserver 0.0.0.0:8000 with touch ~/.config/promgen/DEBUG

[09/Jan/2018 09:51:55] "GET /login/?next=/ HTTP/1.1" 200 5349
[09/Jan/2018 09:51:55] "GET /static/css/bootstrap-theme.min.css HTTP/1.1" 200 23409
[09/Jan/2018 09:51:55] "GET /static/css/bootstrap-datetimepicker.min.css HTTP/1.1" 200 7785
[09/Jan/2018 09:51:55] "GET /static/css/promgen.css HTTP/1.1" 200 494
[09/Jan/2018 09:51:55] "GET /static/css/bootstrap-switch.min.css HTTP/1.1" 200 5612
[09/Jan/2018 09:51:55] "GET /static/js/jquery.min.js HTTP/1.1" 200 86709
[09/Jan/2018 09:51:55] "GET /static/js/jquery.selection.js HTTP/1.1" 200 12885
[09/Jan/2018 09:51:55] "GET /static/css/bootstrap.min.css HTTP/1.1" 200 121200
[09/Jan/2018 09:51:55] "GET /static/js/bootstrap3-typeahead.min.js HTTP/1.1" 200 8681
[09/Jan/2018 09:51:55] "GET /static/js/moment.min.js HTTP/1.1" 200 61317
[09/Jan/2018 09:51:55] "GET /static/js/bootstrap.min.js HTTP/1.1" 200 37045
[09/Jan/2018 09:51:55] "GET /static/js/bootstrap-switch.min.js HTTP/1.1" 200 14920
[09/Jan/2018 09:51:55] "GET /static/js/bootstrap-datetimepicker.min.js HTTP/1.1" 200 38514
[09/Jan/2018 09:51:55] "GET /static/js/promgen.js HTTP/1.1" 200 6402
[09/Jan/2018 09:51:55] "GET /ajax/alert HTTP/1.1" 302 0
[09/Jan/2018 09:51:55] "POST /ajax/silence HTTP/1.1" 302 0
Not Found: /favicon.ico
[09/Jan/2018 09:51:55] "GET /favicon.ico HTTP/1.1" 404 14241
[09/Jan/2018 09:51:55] "GET /login/?next=/ajax/alert HTTP/1.1" 200 5369
[09/Jan/2018 09:51:55] "GET /login/?next=/ajax/silence HTTP/1.1" 200 5373
[09/Jan/2018 09:52:01] "POST /login/ HTTP/1.1" 302 0
[09/Jan/2018 09:52:01] "GET / HTTP/1.1" 200 8035
Error connecting to http://alertmanager:9093/api/v1/alerts
[09/Jan/2018 09:52:01] "GET /ajax/alert HTTP/1.1" 200 2
Error connecting to http://alertmanager:9093/api/v1/silences
[09/Jan/2018 09:52:01] "POST /ajax/silence HTTP/1.1" 200 2
[09/Jan/2018 09:52:03] "GET /admin/ HTTP/1.1" 200 9249
[09/Jan/2018 09:52:03] "GET /static/admin/css/base.css HTTP/1.1" 304 0
[09/Jan/2018 09:52:03] "GET /static/admin/css/dashboard.css HTTP/1.1" 304 0
[09/Jan/2018 09:52:04] "GET /static/admin/css/fonts.css HTTP/1.1" 304 0
[09/Jan/2018 09:52:04] "GET /static/admin/fonts/Roboto-Bold-webfont.woff HTTP/1.1" 304 0
[09/Jan/2018 09:52:04] "GET /static/admin/fonts/Roboto-Regular-webfont.woff HTTP/1.1" 304 0
[09/Jan/2018 09:52:04] "GET /static/admin/img/icon-changelink.svg HTTP/1.1" 304 0
[09/Jan/2018 09:52:04] "GET /static/admin/img/icon-addlink.svg HTTP/1.1" 304 0
[09/Jan/2018 09:52:04] "GET /static/admin/fonts/Roboto-Light-webfont.woff HTTP/1.1" 304 0

I am planning to use Nginx, gunicorn and WSGI to serve the promgen, but cant seem to get it working as showing 502 Bad Gateway

/promgen/gunicorn_config.py
command = '/usr/bin/gunicorn'
pythonpath = '/promgen/promgen'
bind = '127.0.0.1:8000'
workers = 3
user = 'nginx'

/etc/supervisord.conf
[program:promgen]
command = gunicorn -c /promgen/gunicorn_config.py promgen.wsgi
directory = /promgen/promgen
user = nginx

Running gunicorn -c /promgen/gunicorn_config.py promgen.wsgi

Shows the following error

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/bin/gunicorn", line 11, in
sys.exit(run())
File "/usr/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 74, in run
WSGIApplication("%(prog)s [OPTIONS] [APP_MODULE]").run()
File "/usr/lib/python3.6/site-packages/gunicorn/app/base.py", line 203, in run
super(Application, self).run()
File "/usr/lib/python3.6/site-packages/gunicorn/app/base.py", line 72, in run
Arbiter(self).run()
File "/usr/lib/python3.6/site-packages/gunicorn/arbiter.py", line 231, in run
self.halt(reason=inst.reason, exit_status=inst.exit_status)
File "/usr/lib/python3.6/site-packages/gunicorn/arbiter.py", line 344, in halt
self.stop()
File "/usr/lib/python3.6/site-packages/gunicorn/arbiter.py", line 393, in stop
time.sleep(0.1)
File "/usr/lib/python3.6/site-packages/gunicorn/arbiter.py", line 244, in handle_chld
self.reap_workers()
File "/usr/lib/python3.6/site-packages/gunicorn/arbiter.py", line 524, in reap_workers
raise HaltServer(reason, self.WORKER_BOOT_ERROR)
gunicorn.errors.HaltServer: <HaltServer 'Worker failed to boot.' 3>
[root@lab04 conf.d]# [2018-01-09 13:25:14 +0000] [4724] [ERROR] Exception in worker process
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/gunicorn/arbiter.py", line 578, in spawn_worker
worker.init_process()
File "/usr/lib/python3.6/site-packages/gunicorn/workers/base.py", line 126, in init_process
self.load_wsgi()
File "/usr/lib/python3.6/site-packages/gunicorn/workers/base.py", line 135, in load_wsgi
self.wsgi = self.app.wsgi()
File "/usr/lib/python3.6/site-packages/gunicorn/app/base.py", line 67, in wsgi
self.callable = self.load()
File "/usr/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 65, in load
return self.load_wsgiapp()
File "/usr/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 52, in load_wsgiapp
return util.import_app(self.app_uri)
File "/usr/lib/python3.6/site-packages/gunicorn/util.py", line 352, in import_app
import(module)
File "/usr/lib/python3.6/site-packages/Promgen-0.25.dev0-py3.6.egg/promgen/wsgi.py", line 12, in
application = get_wsgi_application()
File "/usr/lib/python3.6/site-packages/Django-1.11.9-py3.6.egg/django/core/wsgi.py", line 13, in get_wsgi_application
django.setup(set_prefix=False)
File "/usr/lib/python3.6/site-packages/Django-1.11.9-py3.6.egg/django/init.py", line 27, in setup
apps.populate(settings.INSTALLED_APPS)
File "/usr/lib/python3.6/site-packages/Django-1.11.9-py3.6.egg/django/apps/registry.py", line 116, in populate
app_config.ready()
File "/usr/lib/python3.6/site-packages/Promgen-0.25.dev0-py3.6.egg/promgen/apps.py", line 15, in ready
from promgen import signals # NOQA
File "/usr/lib/python3.6/site-packages/Promgen-0.25.dev0-py3.6.egg/promgen/signals.py", line 13, in
from promgen import models, prometheus
File "/usr/lib/python3.6/site-packages/Promgen-0.25.dev0-py3.6.egg/promgen/prometheus.py", line 24, in
from promgen.celery import app as celery
File "/usr/lib/python3.6/site-packages/Promgen-0.25.dev0-py3.6.egg/promgen/celery.py", line 10, in
import celery
File "/promgen/promgen/celery.py", line 12, in
from celery.signals import celeryd_after_setup
ModuleNotFoundError: No module named 'celery.signals'; 'celery' is not a package
[2018-01-09 13:25:14 +0000] [4724] [INFO] Worker exiting (pid: 4724)

Is it possible to disable celery or not use redis?

from promgen.

The static files (css, js, etc) is working as expected I believe. When you run with DEBUG, Django serves the files directly but when you deploy it, you'll need to run 'promgen collectstatic' and ensure STATIC_ROOT is set so that Nginx can serve things properly

https://docs.djangoproject.com/en/2.0/howto/static-files/#deployment

As for celery, currently we use Celery to push configuration files to Prometheus servers by default, but it should be possible to run without installing Promgen on the Prometheus nodes and just use a cron job to query http://promgen/api/v1/targets and write there where Prometheus can read it.

As for gunicorn, this is the line I have in the Dockerfile to run things

gunicorn "promgen.wsgi:application"

Since the error says it can't find the celery module, I wonder if the module path is not quite correct

from promgen.

Dear Taylor,

I will retry and revert later.

Hopefully this round all will be happy and the above installation guide can be finalized for everyone to benefit

Since the error says it can't find the celery module, I wonder if the module path is not quite correct
: how to resolve this?

from promgen.

Typically I use a virtual environment (virtualenv) to install everything to, since it provides a more self contained environment and plays better with the system installed stuff by not modifying it. So typically I would install python3 + virtualenv through rpm, then create a virtualenv and install everything into that. Some of this is mentioned in https://github.com/line/promgen#setting-up-promgen-for-development

I see it shows /usr/lib/python3.6 which makes me think that gunicorn and promgen are properly sharing the correct Python path, but I'm not certain why it would be unable to find celery. Do you see celery listed if you run pip list ?

from promgen.

@kfdm Can provide a simple tutorial on how to use promgen? As Prometheus is on a different server from promgen

from promgen.

Issue Resolved

from promgen.