[Question] What to do about frozen testcase

[olw-medcom]

Hey there,

I have a test case that has been submitted for admin interaction by a vendor. I then submitted my admin interaction, but then... Nothing happened? And now it looks like this in the UI:

It doesn't seem to have registered the admin interaction, but there is no button of viewing pending admin interaction either? I tried to search for the session id in the gitb-srv and gitb-ui logs, but couldn't find anything...

What would you do in this case? I need to finish this test case so that I can download the report and give the vendor a stamp of approval 🤔 What would termination do?

[costas80]

Hi @olw-medcom. I assume that this happening is unusual and that you normally complete test sessions with admin interactions without issues (if not please should out to investigate deeper).

The only case I am aware of that would result in what you are seeing here is that the test session appears as active in the frontend (gitb-ui), but does not exist in the test engine (gitb-srv). A scenario where this could come up is that the test engine was restarted while test sessions were still pending. In such cases the UI still keeps a record of them but there is nothing in the backend to continue execution. The fact that you completed the interaction and that this option no longer appears is "understandable" in this case, as this part is managed in the frontend.

Does this ring a bell in any way? Regarding what I would do, if it's not too much trouble I'd terminate this session and ask them to execute the test again.

P.S: Ideally the frontend should periodically sync with the backend to not display such dead sessions as still active. Also, we have had in mind to add a toggle for the Test Bed admin to "prepare for shutdown", meaning that the test engine will refuse new test sessions until the next restart (solutions like Jenkins also follow this approach).

[olw-medcom]

Ohhh.. Yes that does sound true. The test case was completed thursday at 09 and I restarted the ui and srv at 12:30 to get your new update...

Well that is quite annoying and awkward as they have completed quite a few test case that all needs to be repeated 😅

I'm not sure a prepare for shudown toggle would matter much for us, I just wasn't aware that something could break when I restart. The ability to actually save the pending test sessions in the srv would be a much better feature, but since you aren't suggesting this, I imagine it's not that easy for you to make it so.

Thank you for the explanation @costas80, closing here

[costas80]

The ability to actually save the pending test sessions in the srv would be a much better feature.

Yes, that would indeed be the obvious improvement but one that would require a hardcore change to persist and read the state of runtime sessions. Currently tests are in-memory which gives us big efficiency and concurrency gains while avoiding performance issues (serialising/deserialising state at each step and/or transactionally recording state updates).

To be honest this only comes up upon a service restart which offers the loophole of just making sure that no tests are running during that period (also keeping in mind that a restart takes seconds for the test engine). I would still opt with a Test Bed admin feature to just make sure you can safely restart.

[olw-medcom]

Thinking a little bit more about it, I suppose yes it's a good enough workaround.

I guess the workflow would then be to 1. Finish all pending admin interactions, 2. set the prepare for shutdown toggle (possibly with a notification and explanation to the end user?) 3. restart 4. remove the toggle (and possible notification)

[costas80]

I guess the workflow would then be to 1. Finish all pending admin interactions, 2. set the prepare for shutdown toggle (possibly with a notification and explanation to the end user?) 3. restart 4. remove the toggle (and possible notification)

Yes, something like that. Basically setting the toggle would:

• Prevent new test sessions from starting.
• Show a fixed banner on the UI informing users that the ITB is preparing for shutdown and disable test execution controls. Existing test sessions would of course allow to complete.
• Raise a relevant error when attempting to launch tests via the REST API.

Upon restart, the toggle would be deactivated by default, so you wouldn't need to switch it off (the restart would effectively do this automatically). Like that you don't need to remember to switch it off.

[olw-medcom]

That sounds good. My initial scepticism has succumbed, sorry about that.
This would indeed be a good feature. Is this something that is already in the works?

[costas80]

Is this something that is already in the works?

Well, it was on our backlog already and well thought out but not implemented. It's pretty simple to do though so I would like to include it in the upcoming release (I would skip it normally but as this starts coming up I'd like to include it).

I'll reopen the issue to keep track of this so that you know when its available (I'll ping you as usual).

[olw-medcom]

Sounds good, thank you @costas80!