Giter VIP home page Giter VIP logo

bosh-acceptance-tests's People

Contributors

aaronshurley avatar ankurcha avatar cdutra avatar charleshansen avatar codeword avatar cppforlife avatar cunnie avatar d avatar dennisdenuto avatar dpb587-pivotal avatar frodenas avatar h4xnoodle avatar jpalermo avatar ljfranklin avatar lnguyen avatar lwoydziak avatar mariash avatar maximilien avatar miguelverissimo avatar mkocher avatar mmb avatar oppegard avatar pmenglund avatar selzoc avatar st3v avatar tjvman avatar tylerschultz avatar xingzhou avatar ytolsk avatar zaksoup avatar

Stargazers

 avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

bosh-acceptance-tests's Issues

Stemcell deleted during tests

Hi

We are running the tests from branch gocli-bats. We are using bosch cli 2 and we have a bosh running in aws.

After running some of then successfully, all the others starts failing with message:

  • " Error: Stemcell 'bosh-aws-xen-ubuntu-trusty-go_agent/3421.11' doesn't exist"

Could you help us on that? What are we doing wrong?
This is my configuration:

            export BAT_STEMCELL_URL="https://bosh.io/d/stemcells/bosh-aws-xen-hvm-ubuntu-trusty-go_agent?v=3421.11"
            export BAT_STEMCELL_VERSION=$(echo $BAT_STEMCELL_URL | sed ''s/.*v=//)
            export BAT_STEMCELL=bosh-aws-xen-hvm-ubuntu-trusty-go_agent
            export BAT_DEPLOYMENT_SPEC=/bats/bats.yml
            export BAT_BOSH_CLI=bosh
            export BAT_DNS_HOST=8.8.8.8
            export BAT_INFRASTRUCTURE=aws
            export BAT_NETWORKING=manual
            export BAT_PRIVATE_KEY=bosh/bosh.pem
            export BOSH_OS_BATS=true

            export BOSH_ENVIRONMENT=${BOSH_IP}
            export BOSH_CLIENT=admin
            export BOSH_CLIENT_SECRET=`bosh int creds.yml --path /admin_password`
            export BOSH_CA_CERT=`bosh int creds.yml --path /director_ssl/ca`

            echo "Importing stemcell ---------------------------"
            curl -O -sL $BAT_STEMCELL_URL
            mv bosh-aws-xen-hvm-ubuntu-* $BAT_STEMCELL

            echo "Create config file ---------------------------"
            cat <<< '---
            cpi: aws
            properties:
              stemcell:
                name: bosh-aws-xen-ubuntu-trusty-go_agent
                version: 3421.11 
              pool_size: 1
              instances: 1
              vip: 54.54.54.54 
              second_static_ip: 10.***.101
              networks:
              - name: default 
                static_ip: 10.***.100 
                cidr: 10.***.0/24
                reserved: ["10.***.2 - 10.***.25"]
                static: ["10.***.100 - 10.***.110"]
                gateway: 10.***.1
                subnet: subnet-***
                security_groups: sg-***
              key_name: ***
            ' > bats.yml

            echo "Running tests --------------------------------"
            bundle exec rspec spec

The error logs:

BATs example started 'auditd, sshd, cron, rsyslogd should be running' ./spec/system/auditd_spec.rb:15
BATs example failed  'auditd, sshd, cron, rsyslogd should be running' ./spec/system/auditd_spec.rb:15 

  1) auditd, sshd, cron, rsyslogd should be running
     Failure/Error: @requirements.requirement(deployment, @spec)
       expected command to exit with 0 but was 1. output was
       {
           "Tables": null,
           "Blocks": [
               "\n13:38:43 | ",
               "Deprecation: Ignoring cloud config. Manifest contains 'networks' section.",
               "\n13:38:43 | ",
               "Preparing deployment: Preparing deployment",
               " (00:00:00)",
               "\n            L Error: Stemcell 'bosh-aws-xen-ubuntu-trusty-go_agent/3421.11' doesn't exist",
               "\n",
               "\n13:38:43 | ",
               "Error: Stemcell 'bosh-aws-xen-ubuntu-trusty-go_agent/3421.11' doesn't exist"
           ],

When I check from my console the stemcell is there at the beginning. After some tests the stemcell is not any more there.

bosh -e 10.*** stemcells 

Permission denied due to inconsistent users when run test case command

I found the test case of persistent disk migration, we create the file by root but read it by bosh_ssh_user.

https://github.com/cloudfoundry/bosh-acceptance-tests/blob/master/spec/system/with_release_stemcell_spec.rb#L88
https://github.com/cloudfoundry/bosh-acceptance-tests/blob/master/spec/system/with_release_stemcell_spec.rb#L105

Why did we think the process by two different users?
When we built and tested stemcell-3422.7(for umask case), the case would be failed due to permission.

cat: /var/vcap/store/batarang/save: Permission denied

So I dug into the command execution process and found the file is 600 because of umask '0077'.

So how about to run write/read in the same user, because this case it to test persistent disk migration.

BATS should deploy with --no-redact

Otherwise the deployment manifest shown by the tests is of very limited use. Consider e.g.

+ properties:
+   openstack:
+     connection_options:
+       idempotent: "<redacted>"
+   batlight:
+     fail: "<redacted>"
+     missing: "<redacted>"

Will it fail or not? Wdyt? :D

There is no easy way to determine the commands and workflow that BATs covers

While BATs are a good test bucket that has served us well to help verify and allow developers and users of BOSH to know that any change(s) to stemcell, agent, and/or director are working together well, there is a need to understand at a high-level the various commands and workflows that BATs is covering.

The current tests do not seem to have a test plan associated with it, and the only way to determine the actual plan for BATs is to run the tests with --dry-run which simply shows the sequence of tests' descriptions.

Ideally some type of test plan (automatically generated with code or in a document) would allow others who are interested in adding or improving BATs a means to see what is missing and where to improve the coverage and or workflow that BATs is doing.

ubuntu bionic running bats got an error: no implicit conversion of nil into String

When I running bats based on ubuntu bionic release and got the following error:

Failures:

  1) network configuration when using manual networking deploys multiple manual networks
     Failure/Error: expect(bosh_ssh('batlight', 0, 'PATH=/sbin:/usr/sbin:$PATH; ifconfig', deployment: deployment.name).output).to include(static_ips[1])
     TypeError:
       no implicit conversion of nil into String
     # ./spec/system/network_configuration_spec.rb:79:in `block (3 levels) in <top (required)>'

Finished in 35 minutes 39 seconds (files took 0.41484 seconds to load)
72 examples, 1 failure, 12 pending

Failed examples:

rspec ./spec/system/network_configuration_spec.rb:73 # network configuration when using manual networking deploys multiple manual networks

This error did not happen in the ubuntu xenial release.

Flaky test 'should survive agent dying'

The test should survive agent dying [1] is flaky in our build pipeline. It fails with:

Failure/Error: expect(bosh_safe("logs batlight 0 --agent --dir #{tmpdir}")).to succeed
       expected command to exit with 0 but was 1. output was
       RSA 1024 bit CA certificates are loaded due to old openssl compatibility
       Acting as user 'admin' on deployment 'bat' on 'micro'

       Director task 11
         Started fetching logs for batlight/1b0362eb-6deb-41a1-a837-99d0511e8c8b (0) > Finding and packing log files. Failed: Timed out sending `fetch_logs' to 39f5c7ff-89ce-42b3-9e6f-30e66cd7849a after 45 seconds (00:00:45)

       Error 450002: Timed out sending `fetch_logs' to 39f5c7ff-89ce-42b3-9e6f-30e66cd7849a after 45 seconds

To us it seems that the test scenario is not correct. First the agent is killed, after that wait_for_vm [2] is called which doesn't check the vm state just searches for this vm existence. At the end the test tries to fetch the agent logs but agent could be still in unresponsive state. Is the scenario correct or should the wait_for_vm wait for state running before returning?

We can provide a fix if need.

[1]

it 'should survive agent dying', ssh: true do

[2]
def wait_for_vm(name)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.