Keep the state of an Imported Library about robotkernel HOT 15 OPEN

Ouch. That's a design compromise to which I have no easy solution.

It comes from that design choice that RobotKernel executes each cell separately, as separate Robot Framework run.

That said, because notebook itself has global state, RobotKernel does transmit some data between cells. It does preserve "suite level variables". Also some libraries have special support. E.g. SeleniumLibrary driver connections are preserved to avoid need to start new browser for each cell, and also allow fast iteration on Selenium interaction. This is done by injecting some RobotKernel specific listeners into Robot Framework executions.

So, right now, the only option is to save state into "suite level variable". It might even possible for the library to initialize its value from "suite level variable" (by using methods from BuiltIn library instance). Yet, because "suite level variables" are only restored on "start suite" event, it might be possible that library is initialized before "suite level variables" are restored. Of course, the ugly part in this is that this is would be something only required for RobotKernel and not for regular Robot execution :(

Let's keep this issue open in case that we figure out better options. It might be technically possible to save library instances at the end of a cell run, and inject them into new execution, but there might be unexpected side-effects (or unexpected lose of expected side-effects some libraries could do on their import). Similarly just storing some state variables for library is technically possible, but hard to make generic without unexpected side-effects...

from robotkernel.

Oh, and thank you for feedback.

There has been very little development for some time, because we've been watching, where Robocorp develop its for of the kernel or https://github.com/jupyter-xeus/xeus-robot

Now it seems that there is again oxygen to think about the future of Robot Framework and Jupyter, because Robocorp abandoned JupyterLab to focus on VSCode and their upcoming Automation Studio.

from robotkernel.

Thank you very much for your reply! I find this approach of using robot within jupyter notebooks very promising. I really enjoy this to use it for prototyping robot test cases.

But in the scenarios, where I want to use it, I have libraries, which are in general instantiated and setup during the suite setup process, and latest in the suite teardown they get cleaned up. They keep connections to remote hosts on different protocols and they keep aliases for initialized connections and other stuff. So unfortunately, there is no simple "state" to be pushed into some variable.

So there is no kind of "execution context" of the running robot, which can be retrieved from and injected to each cell? So every set variable can be given from one cell to the next? I just remember, that it is possible in robot to list all set variables, maybe we could find some mechanism, to get all those... Just guessing...

(BTW: if it would be easy, every could do it ;) ;) )

from robotkernel.

@Thonuck That all sound exactly like what RobotKernel is already doing especially for SeleniumLibrary

https://github.com/robots-from-jupyter/robotkernel/blob/master/src/robotkernel/listeners.py#L221

But I am not sure, how to make that generic. Is there generic approach for connection used in all of those libraries that we could use?

from robotkernel.

@Thonuck I just realized that there is a common convention in Jupyter kernels to affect the kernel itself: %%magics.

How would it feel like to have something like

%%sticky LibraryName._attributeName

And it would "push" those named suite level library instance attributes into "context" at the end of the cell execution (suite end) and pop them back in the beginning of the next cell (suite start).

Or would you prefer other name or syntax?

from robotkernel.

I need to check this. The problem might be, that those libraries are depending on other libraries and their state and so on and forth. So they are not kind of "closed"/"simple". But as mentioned, I need to check this.

Hmm... if I look at this listeners, they use this builtin-function from robot "get_library_instance".
If I check the sources of robot, there is something written like:

    If the optional argument ``all`` is given a true value, then a
    dictionary mapping all library names to instances will be returned.
    This feature is new in Robot Framework 2.9.2.

with the function call

def get_library_instance(self, name=None, all=False):

Maybe I'm a bit too naive now, but wouldn't it maybe be possible, to have here a listener for set libraries, with "all=True"? Kind of iterating over what we get by this call to set all of them?

But I assume, you checked this already, and that this can lead to problems...

from robotkernel.

I'm afraid to preserve all state of all libraries automatically, because libraries are free to do anything in their init and therefore that may lead to hard-to-debug issues later on some libraries.

But keeping state of a single library is something we should try. So that

%%sticky LibraryName

would save all attributes on matching library instance and restore them for the subsequent cell executions. That might still lead to weird issues, if connections etc are created already on library instance init, but at least it would easier to debug, because those explicit %%sticky configurations.

from robotkernel.

That would be already very helping, if this is possible.

And needs to be checked, if this would solve also keeping open connections and such. Or lead to weird situations...
If it is not too difficult to achieve, it would be really great, if you could supply such a sticky-keyword. It would be also possible to name a few sticky Libraries like this, I assume.

Thank you very much!

from robotkernel.

I will look into this at the end of the week and comment back early next week. Your use case sounds like exactly the use case for Robot Framework support in Jupyter, so I'll try to find a solution. 🤞

from robotkernel.

I tried that naive approach and it seems to work. I'll try to make a beta for you during the next days. Still need to test, which RF versions I am able to support with the feature.

Sticky keyword library will need either ROBOT_LIBRARY_SCOPE = 'SUITE' or ROBOT_LIBRARY_SCOPE = 'SUITE' scope, because the default scope is test, where keeping state between runs would be wrong.

from robotkernel.

https://mybinder.org/v2/gh/robots-from-jupyter/robotkernel/sticky-magic?urlpath=tree/tests/StickyLibraryMagic.ipynb

from robotkernel.

https://pypi.org/project/robotkernel/1.6a1/

from robotkernel.

Thank you very much! Sorry for the late reply. I will test this the next days.

from robotkernel.

I'm still in the process of understanding, why this is not working in my case.
The challenge, that I have, is, that the library is depending on other libraries as well.

My first try was to get all loaded python modules, and try to inject all python modules I need.
But this does not fit for now.
I'm still investigating. I'll keep you informed, if I have some updates.

Can you tell me, how I can debug robotkernel? I first tried it's installation from source with "pip -e path_to_robotkernel_clone", but then the jupyter did not recognize the robotkernel anymore. So I'm now changing code within my virtualenv/lib/... which is not that nice...

from robotkernel.

@Thonuck I assume you confirmed that your initial example did work?

And the library has scope declaration ROBOT_LIBRARY_SCOPE = 'SUITE'?

You can use Display keyword from Library IPython.display to log variables onto notebook itself while executing the cells.
You should be able to check that Builtins.Get library instance returns the same library instance (as in same Python object).

from robotkernel.