cci-moc / ore Goto Github PK

either:

• no clean.sh
• turn online after clean.sh

Trying to ./stack.sh in the devstack directory using the current single_node_devstack.yml file on s-large flavor Centos created Dec. 18 produces the attached error (the text file includes the last part of output, not all the output).
ORE_error.txt

The solutions mentioned here (i.e. upgrading pip or issuing virtualenv ../requirements/.venv/) do not work:
https://askubuntu.com/questions/1040238/openstack-devstack-installation-ubuntu-16-04

This part of the ORE readme looks relevant:

after cloning requirements repo, change python-libvirt to 3.9 in upper-constraints.txt (this is because CentOS 7 ships with python-libvirt 3.9 but devstack pike sets upper limit of python-libvirt 3.5)

However, manually changing this also did not work. upper-constraints.txt appears to be 3 different files:

./requirements/openstack_requirements/tests/files/upper-constraints.txt
./requirements/.venv/lib/python2.7/site-packages/openstack_requirements/tests/files/upper-constraints.txt
./requirements/upper-constraints.txt

These files do not have a python-libvirt field, only a libvirt-python field. I did update that field to libvirt-python===3.9.0 and ran ./stack.sh in /devstack again. But I got the same error as previously.

But is neutron even instrumented?

Can we list the specific changes made to make DevStack work with CentOS in the top-level file? Perhaps the readme.

I specified stable/queens for osprofiler in local.conf, but after stack.sh completed I had the stable/pike version of osprofiler

A timeout occurred when ./stack.sh was on command sudo systemctl start [email protected].

Error message: Job for [email protected] failed because a timeout was exceeded. See "systemctl status [email protected]" and "journalctl -xe" for details.
Then it gave exit 1.

The output of journalctl -xe is attached.

journalctl.txt

Running sed -i 's/\s\+$//g' moc-devstack.patch should fix it.

It's possible this may have just been my system because Emre did not have this problem, but in my newest ORE, /etc/neutron/neutron.conf had neutron.db tracepoints enabled with the line trace_sqlalchemy = True, which gave me really wonky results on the traces until we figured it out.

I edited it to say trace_sqlalchemy = False and restarted with sudo systemctl restart devstack@* and then it worked. But I didn't edit that file to begin with, so I'm not sure why this happened. I brought up the ORE the same way as before, following the instructions in the README.

An issue about binding something to port 6640 comes up, which can be prevented by setting selinux to permissive: sudo setenforce Permissive.

I don't know if this is a new issue or not, but there's also this mail, which has the same issue I was facing. http://lists.openstack.org/pipermail/openstack-dev/2016-November/107499.html

We can immediately close the issue, I just wanted to write here since it happened to me twice.

Pull request #16 doesn't fix the issue, after a fresh VM creation, the OSprofiler version installed is still 1.11, not stable/queens, even though /opt/stack/osprofiler is queens.

Doing sudo pip install in /opt/stack/osprofiler (without anything else), and then restarting services fixes this issue.

In order to run ./stack.sh

Because we are using a git patch, we must manually rebuild the patch each time we want to modify devstack. Instead, we should just download the files directly from this repo. This also centralizes everything into one repo, rather than using a different modified devstack to build the patch.

This is mostly up to me to fix, but I'm writing some progress here. Any help is most welcome.

Steps to reproduce

1. Install devstack according to the instructions
2. Source openrc with parameters admin admin
3. Run an openstack command - I used openstack --os-profile Devstack1 server create --flavor m1.tiny --image cirros-0.3.5-x86_64-disk test_server_1
4. Retrieve the trace using osprofiler trace show --json --connection-string redis://localhost:6379 $TRACE_ID

Result

The trace only contains traces from glance after these operations. I've also checked redis using redis-cli and then KEYS osprofiler* and the traces match - there are only 8 entries for the VM create command, all from glance.

Additional Info

After devstack is running, every service can be restarted using sudo systemctl restart devstack@*, which fixes something in keystone. After this, I can get keystone traces as well, but still nothing from nova.

I noticed that in order to be able to use openstack I have to run source /opt/stack/devstack/openrc admin admin every time I ssh to the instance I created with this devstack, and I wanted this to run automatically instead.

So I altered /opt/stack/devstack/openrc to have admin as default for both OS_USERNAME and OS_PROJECT_NAME and stuck a single line script into /etc/profile.d/ that runs source /opt/stack/devstack/openrc admin admin &> /dev/null on start-up of the shell.

Is there a way to make these changes part of the default configuration for the ORE, or do these types of changes reside in the image we launch from (rather than the config .yml files or patches) and therefore we can't really alter them?

Obviously this is pretty low priority.

osprofiler can support jaeger in the master branch, so I'm trying to create an ORE with the master version of osprofiler.

So far, I tried to change local.conf to include OSPROFILER_BRANCH and OSPROFILER_REPO, with no luck (the old version of osprofiler is fetched). I'll try it more systematically tomorrow and update any progress over here.

Why do we need to insert our ssh keys info config.yml now, instead of copying /home/centos/.ssh?

In /home/centos openstack automatically places the selected security key, and if we copy it to /opt/stack, we can ssh directly to the stack user. Is this version more secure? Did the other version break for some reason?

Should we move this repo into docc-lab? (Raja's organization)

Even though the commits are not commented out in local.conf, the pulled repos have different commits. I think this might also be the reason for #24.