Koku's goal is to provide an open source solution for cost management of cloud and hybrid cloud environments. This solution is offered via a web interface that exposes resource consumption and cost data in easily digestible and filterable views. The project also aims to provide insight into this data and ultimately provide suggested optimizations for reducing cost and eliminating unnecessary resource usage.
Full documentation is available through readthedocs.
This project is developed using Python 3.6. Make sure you have at least this version installed.
- Docker
- PostgreSQL
Install PostgreSQL:
brew install postgresql
To get started developing against Koku first clone a local copy of the git repository.
git clone https://github.com/project-koku/koku
Developing inside a virtual environment is recommended. A Pipfile is provided. Pipenv is recommended for combining virtual environment (virtualenv) and dependency management (pip). To install pipenv, use pip
pip3 install pipenv
Then project dependencies and a virtual environment can be created using
pipenv install --dev
Note for Mac OSX users
psycopg2 is a dependency of Django and installing the psycopg2 wheel will likely fail. The following steps should be taken to allow installation to succeed:
brew install openssl brew unlink openssl && brew link openssl --force `/usr/local/opt/openssl/bin` should be appended to the PATH environment variable The following environment variables can be set in the koku repo's .env file LDFLAGS="-L/usr/local/opt/openssl/lib" CPPFLAGS="-I/usr/local/opt/openssl/include" These environment variables will then be available next time you activate your virtualenv. For immediate use running `source .env` will load the environment variables into your existing terminal environment. Alternatively, run the following commands: `export LDFLAGS="-L/usr/local/opt/openssl/lib"` `export CPPFLAGS="-I/usr/local/opt/openssl/include"`
If dependency installation still fails, try using
pipenv install --dev --sequential
To activate the virtual environment run
pipenv shell
Please refer to Working with Openshift.
If deploying with Openshift seems overly complex you can try an alternate local environment where you will need to install and setup some of the dependencies and configuration.
This project is developed using the Django web framework. Many configuration settings can be read in from a .env file. An example file .env.example is provided in the repository. To use the defaults simply
cp .env.example .env
Modify as you see fit.
PostgreSQL is used as the database backend for Koku. A docker-compose file is provided for creating a local database container. If modifications were made to the .env file the docker-compose file will need to be modified to ensure matching database credentials. Several commands are available for interacting with the database.
# Initialize the docker network for koku services if it doesn't already exist docker network create koku-network # This will launch a Postgres container make docker-up-db # This will run Django's migrations against the database make run-migrations # This will stop and remove a currently running database and run the above commands make docker-reinitdb
Assuming the default .env file values are used, to access the database directly using psql run
PGPASSWORD=postgres psql postgres -U postgres -h localhost -p 15432
There is a known limitation with docker-compose and Linux environments with SELinux enabled. You may see the following error during the postgres container deployment:
"mkdir: cannot create directory '/var/lib/pgsql/data/userdata': Permission denied" can be resolved by granting ./pg_data ownership permissions to uid:26 (postgres user in centos/postgresql-96-centos7)
If a docker container running Postgres is not feasible, it is possible to run Postgres locally as documented in the Postgres tutorial. The default port for local Postgres installations is 5432. Make sure to modify the .env file accordingly. To initialize the database run
make run-migrations
Koku uses tox to standardize the environment used when running tests. Essentially, tox manages its own virtual environment and a copy of required dependencies to run tests. To ensure a clean tox environment run
tox -r
This will rebuild the tox virtual env and then run all tests.
To run unit tests specifically:
tox -e py36
To lint the code base
tox -e lint
To run IQE Smoke, Vortex or API tests, while on the Red Hat network and koku deployed via docker-compose run:
make docker-iqe-smokes-tests make docker-iqe-vortex-tests make docker-iqe-api-tests
Individual IQE tests can be ran with run_test.sh:
<koku_topdir>/testing/run_test.sh iqe tests plugin hccm -k test_api_cost_model_markup_calculation_ocp
If you want to interact with the Postgres database from a GUI:
- Copy the pgadmin_servers.json.example into a pgadmin_servers.json file and if necessary, change any variables to match your database.
- docker-compose up causes pgAdmin to run on http://localhost:8432
- In the login screen, the default login email is postgres
Side note: The pgadmin_servers.json file uses [pgadmin servers.json syntax](https://www.pgadmin.org/docs/pgadmin4/development/import_export_servers.html#json-format)
Please refer to Contributing.