Closed Model Error about orca5 HOT 17 CLOSED

I think I found a way to reliably reproduce this error.

1. Start with 4 vms (A,B,C,D) with a links between A-B, B-C, C-D.
2. Remove link B-C
3. Add a link A-D
   4 Add a link B-C

It should be noted that step 4 creates a loop. This may or may not be part of the problem.

from orca5.

Note: steps 3 and 4 are separate. I'm not sure why they ended up on the same line.

from orca5.

Also note that they 4 nodes are on different racks.

from orca5.

I think this is critical - that all nodes were in different domains. We've tested this for single domain.

from orca5.

@YufengXin please take a look at this when you have a chance. We will not reopen today, because I'd like to have this addressed.

from orca5.

Paul,

I'm trying to reproduce the case to fix it. How did you make the modify, in Flukes or NDLLab?

Step 2: there would three domains on the B-C inter-rack link, did you delete all the 3 domains all together?

Step 3 & 4, I could not add new links in Flukes in the new manifest view. I used the latest Flukes:
ORCA FLUKES v.0.7-SNAPSHOT.build-git24de7 built on 01/15/2016 16:22

Thanks

-Yufeng

from orca5.

Sorry, false alarm. I can add new links, I right click "pan" in Flukes accidentally.

from orca5.

I'm pretty sure Paul was doing it interactively via flukes. Ndllib has no modify support.

-ilya

Sent from some sort of a mobile device equipped with an autocorrect function that has a mind of its own.

-------- Original message --------
From: YufengXin [email protected]
Date: 01/16/2016 10:51 PM (GMT-05:00)
To: RENCI-NRIG/orca5 [email protected]
Cc: Ilya Baldin [email protected]
Subject: Re: [orca5] Closed Model Error (#42)

Paul,

I'm trying to reproduce the case to fix it. How did you make the modify, in Flukes or NDLLab?

Step 2: there would three domains on the B-C inter-rack link, did you delete all the 3 domains all together?

Step 3 & 4, I could not add new links in Flukes in the new manifest view. I used the latest Flukes:
ORCA FLUKES v.0.7-SNAPSHOT.build-git24de7 built on 01/15/2016 16:22

Thanks

-Yufeng

Reply to this email directly or view it on GitHubhttps://github.com//issues/42#issuecomment-172288021.

from orca5.

I refactored the code a little bit and checked in a fix, which didn’t show the exception in the emulator, please give it a try.

-Yufeng

On Jan 17, 2016, at 9:47 AM, Ilya Baldin <[email protected]mailto:[email protected]> wrote:

I'm pretty sure Paul was doing it interactively via flukes. Ndllib has no modify support.

-ilya

Sent from some sort of a mobile device equipped with an autocorrect function that has a mind of its own.

-------- Original message --------
From: YufengXin <[email protected]mailto:[email protected]>
Date: 01/16/2016 10:51 PM (GMT-05:00)
To: RENCI-NRIG/orca5 <[email protected]mailto:[email protected]>
Cc: Ilya Baldin <[email protected]mailto:[email protected]>
Subject: Re: [orca5] Closed Model Error (#42)

Paul,

I'm trying to reproduce the case to fix it. How did you make the modify, in Flukes or NDLLab?

Step 2: there would three domains on the B-C inter-rack link, did you delete all the 3 domains all together?

Step 3 & 4, I could not add new links in Flukes in the new manifest view. I used the latest Flukes:
ORCA FLUKES v.0.7-SNAPSHOT.build-git24de7 built on 01/15/2016 16:22

Thanks

-Yufeng

Reply to this email directly or view it on GitHubhttps://github.com//issues/42#issuecomment-172288021.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/42#issuecomment-172331335.

from orca5.

I can confirm that after testing these and other scenarios both in single site and multi-site scenarios I can no longer see this problem. I did find another issue (I think) that I'm putting here.

The scenario was single site (BBN) just like Paul's steps 1-4 above, then add storage to nodes A and D in step 5 (100G and 30G, I think). That succeeded. Then I attempted to add storage to node B (10G) and got the error "Not enough resources" in bbnvmsite for LUNs which was perhaps legitimate. Then I tried to delete the existing two storage LUNs and I kept getting the same (!) error about insufficient resources for LUNs, even though I was deleting, not adding. I tried restarting flukes to make sure this wasn't an artifact there, but it didn't help.

from orca5.

I should say I was working in emulation.

from orca5.

This appears to be a bug.
I will work on it later tonight.

Yufeng

Sent from my Verizon Wireless 4G LTE smartphone

-------- Original message --------
From: Ilya Baldin [email protected]
Date: 01/17/2016 4:42 PM (GMT-05:00)
To: RENCI-NRIG/orca5 [email protected]
Cc: Yufeng Xin [email protected]
Subject: Re: [orca5] Closed Model Error (#42)

I can confirm that after testing these and other scenarios both in single site and multi-site scenarios I can no longer see this problem. I did find another issue (I think) that I'm putting here.

The scenario was single site (BBN) just like Paul's steps 1-4 above, then add storage to nodes A and D in step 5 (100G and 30G, I think). That succeeded. Then I attempted to add storage to node B (10G) and got the error "Not enough resources" in bbnvmsite for LUNs which was perhaps legitimate. Then I tried to delete the existing two storage LUNs and I kept getting the same (!) error about insufficient resources for LUNs, even though I was deleting, not adding. I tried restarting flukes to make sure this wasn't an artifact there, but it didn't help.

Reply to this email directly or view it on GitHubhttps://github.com//issues/42#issuecomment-172385986.

from orca5.

I checked in the fix. The problem was that the error msg was not properly reset.

-Yufeng

On Jan 17, 2016, at 5:29 PM, Yufeng Xin <[email protected]mailto:[email protected]> wrote:

This appears to be a bug.
I will work on it later tonight.

Yufeng

Sent from my Verizon Wireless 4G LTE smartphone

-------- Original message --------
From: Ilya Baldin <[email protected]mailto:[email protected]>
Date: 01/17/2016 4:42 PM (GMT-05:00)
To: RENCI-NRIG/orca5 <[email protected]mailto:[email protected]>
Cc: Yufeng Xin <[email protected]mailto:[email protected]>
Subject: Re: [orca5] Closed Model Error (#42)

I can confirm that after testing these and other scenarios both in single site and multi-site scenarios I can no longer see this problem. I did find another issue (I think) that I'm putting here.

The scenario was single site (BBN) just like Paul's steps 1-4 above, then add storage to nodes A and D in step 5 (100G and 30G, I think). That succeeded. Then I attempted to add storage to node B (10G) and got the error "Not enough resources" in bbnvmsite for LUNs which was perhaps legitimate. Then I tried to delete the existing two storage LUNs and I kept getting the same (!) error about insufficient resources for LUNs, even though I was deleting, not adding. I tried restarting flukes to make sure this wasn't an artifact there, but it didn't help.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/42#issuecomment-172385986.

from orca5.

The submitted fix corrects the problem, I tested it. I will deploy the code on ExoSM today.

from orca5.

Latest orca build based on tag 9c0e14b is deployed on ExoSM with recovery preserving existing slice from @anriban

Please test some more.

from orca5.

Mentioning Victor @vjorlikowski just so he knows - model closing bug in its latest incarnation has been extinguished. Latest code is deployed on ExoSM only for now. If no new issues are identified before tomorrow afternoon, let's redeploy across the testbed.

from orca5.

Paul confirmed it works.

from orca5.