Add support for readthedocs about sage_sample HOT 21 OPEN

With some cajoling, and with proper support from our end, RTD might be willing to make some Sage version available on their build machines. This is still non-trivial as a project's documentation might build correctly against one Sage version but not another and Sage has no reliable API (a serious problem, but one beyond the scope of this).

In the past I believe they've been willing to add non-trivial binary dependencies on their build system if there was enough demand. And if we are willing to help support that effort (as opposed to just saying "you must install Sage on your machines" and expecting them to just do it), they might be amenable. I could ask...

from sage_sample.

Alternatively, if that proves infeasible, all of RTD's server / build system software is available and documented. A few years ago I set it up locally in order to try to fix a couple issues I had. I don't remember what went into it, and the setup has probably changed since then. But worse comes to worse we can run our own RTD instance for Sage.

from sage_sample.

In fact, the more I think about it, the more convinced I am that we should just run our own instance. Configuring and installing Sage is not trivial, and ideally we'll want to provide any and all Sage versions a project might want to build their documentation against.

This will undoubtedly spiral beyond what RTD's already overstretched support "staff" (volunteers, really) can handle. We should try setting up and maintaining our own instance for Sage, and only bother them with specific issues with running our own RTD server as they occur. Perhaps William would be willing to help provide hosting for such a site....

from sage_sample.

from sage_sample.

Perhaps I'll go ahead and try setting up a RTD server on our OpenStack infrastructure and see how it goes.

(Unrelatedly, I finally figured out how to get to their helpdesk, and have asked them to open the remote management ports needed for Windows; we'll see what they say...)

from sage_sample.

from sage_sample.

It would be easy to extend the Travis CI scripts so that they deploy the built documentation to github pages (see #8)

from sage_sample.

@embray I think making all of sage available on the RTD server just so we can build the documentation of sage packages is extreme overkill. The part of sage that is needed in order to build the documentation is actually really small. If we would want this then we should put the relevant documentation building code of sage in a separate python package on which but sage and sage packages can depend for documentation building.

from sage_sample.

from sage_sample.

Yes this faking of dependencies is certainly possible and actually quite a common problem on RTD. They also kindly provide a solution to this in their FAQ

from sage_sample.

from sage_sample.

You are right that mocking all sage modules would be a huge pain. But a package would only need to mock the sage modules it is actually importing from. That being said, I tried what I could do by hooking the import mechanism of python. And it seems you can solve it this way:

from mock import MagicMock

class MockedModule(MagicMock):
    __all__=[]
    @classmethod
    def __getattr__(cls, name):
        return MagicMock()

mocked_module = MockedModule()

class MockImporter(object):
    def __init__(self, mocked_modules = []):
        self.mocked_modules = mocked_modules

    def find_module(self, name, path=None):
        if any(name == m or name.startswith(m+'.') for m in self.mocked_modules):
            return self
        return None

    def load_module(self, name):
        return mocked_module

sys.meta_path=[MockImporter(["a"])]
import a.v.g.d.s.f
from a.hs.sdfhd.gs import *
from a.adgagds.gad.agd import l

import random
import non_existing_agsgasdga

As it shows, you can still import existing modules, and raises an error on non existing modules. But only the submodules of "a" are imported.
This handles all forms of imports. But like the original workaround on RTD it doesn't work if some code is doing:

from a import *
some_function_imported_from_a()

since there is no way to know at the point of the from a import * statement which names are actually expected to be imported. And returning a list of all possible names is also not possible.

from sage_sample.

So we should encourage people to not use * imports, which is always a good idea.

from sage_sample.

from sage_sample.

There's one minor issues with this, namely Sphinx won't generate documentation for classes that inherit from a MagicMock. I am not sure why exactly but something needs to be disabled on the mocks to make this fully work.

from sage_sample.

To quote from my e-mail just now:

RTD has changed a little bit lately. I haven't configured a new
project since those changes so I am not very familiar. But they now
use Docker containers as the build environments (this did not used to
be the case, IIRC, so it was more restrictive). You can configure
which version of the RTD container image to build from. I don't know
if it's possible to supply an arbitrary custom image. If it is, it
would have to at least be compatible in some way with their official
images (sort of like how Binder images have certain interface
requirements). I couldn't find any documentation on how to set that
up though; I don't know that it's a supported use case (yet).

from sage_sample.

My point is, in principle, mocking might not be necessary at all if you can create docker image based on RTD's official images but that includes Sage. I don't know for sure yet how feasible that is.

from sage_sample.

Ok. Let's move this discussion here rather than into these private emails ;)
If I understand the documentation correctly, you don't get to specify arbitrary docker images but have to pick from a small list of rtd images.
Installing sagelib from conda-forge might be an option. RTD supports environment.yml files, I haven't tried but it could work.

from sage_sample.

See MCLF/mclf#107 for an attempt to build with conda-forge on RTD.

from sage_sample.

I did not manage to get this to work. Running conda install sagelib exceeds the RAM limits it seems. Probably a newer version of conda could help here. I also see segfaults when I try to reproduce this locally with the readthedocs docker images (which might be the reason for errors without any further information on the readthedocs interface.)

from sage_sample.

See readthedocs/readthedocs-docker-images#83.

from sage_sample.