pensando / pensando-elk Goto Github PK

Integrations with 3rd parties usually requires some sort of meta data about each DSC that is not readily available via the logs - i.e. labels, names, etc. This information is, however, stored in PSM.
Utilizing a combination of object webhooks in PSM's APIs and an intermediary bit of code, we can update the metadata for all DSCs under mgmt of PSM in the elasticsearch DB. This can then be used for log enrichment at ingest time.

The setup seems to work as it installs the correct index-pattern, but the dashboards are not installed automatically.

TASK [Allow user to run docker commands] ***********************************************************************************************************************************************************************
fatal: [localhost]: FAILED! => {“changed”: false, “cmd”: “setfacl --modify user:labadmin:rw /var/run/docker.sock”, “msg”: “[Errno 2] No such file or directory: b’setfacl’: b’setfacl’“, “rc”: 2}
PLAY RECAP *****************************************************************************************************************************************************************************************************
localhost : ok=6 changed=5 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0
labadmin@ubuntu:/pensando-elk$ setfacl --modify user:labadmin:rw /var/run/docker.sock
Command ‘setfacl’ not found, but can be installed with:
apt install acl
Please ask your administrator.
labadmin@ubuntu:/pensando-elk$

Getting to be too much at the top level of the pensando-elk directory, especially while running. Move ansible and respective configs to their own folder

executing the ansible playbook (start_elk.yml) to start the stack after a successful installation will execute the setup part as well (no needed).

See #24
This issue is for tracking.

Currently installing the pensando-elk/files/elastiflow.ndjson is a manual process. Need to see if we can do this automatically. Two options for this:
1.) When running the start up playbook.
- Pros: no extra steps needed.
- Cons: Will require a check to see if things are loaded already - may be a PITA

2.) Via a "setup" script/playbook that can add all the info after it's up and running
- Pros: This should only be run once and doesn't require checking to see if things are already loaded
- Cons: Another step, albeit small and only occuring once.

Currently we have to run the ansible-playbook start_elk.yml command whenever the system is rebooted. Need to change the docker start up options to have the containers start at runtime if they have already run the playbook at least once.

@david-pensando has this already in his setup. Use that as the basis for what we have to add.

We have the filebeat module for dfw but it lacks the ability to enrich the logs as they are pushed to elastic. Need to add filters for both filebeat and raw based logs for our dfw logs.

Currently we have to do this manually for Kibana connections to work. Add to the setup playbook or to the post-install config playbook.

PSM can send alert logs to endpoints. Need to add those into our ELK implementation for use by customers.