Giter VIP home page Giter VIP logo

autodoc-traits's Introduction

Technical Overview | Installation | Configuration | Docker | Contributing | License | Help and Resources


Latest PyPI version Latest conda-forge version Documentation build status GitHub Workflow Status - Test Test coverage of code GitHub Discourse Gitter

With JupyterHub you can create a multi-user Hub that spawns, manages, and proxies multiple instances of the single-user Jupyter notebook server.

Project Jupyter created JupyterHub to support many users. The Hub can offer notebook servers to a class of students, a corporate data science workgroup, a scientific research project, or a high-performance computing group.

Technical overview

Three main actors make up JupyterHub:

  • multi-user Hub (tornado process)
  • configurable http proxy (node-http-proxy)
  • multiple single-user Jupyter notebook servers (Python/Jupyter/tornado)

Basic principles for operation are:

  • Hub launches a proxy.
  • The Proxy forwards all requests to Hub by default.
  • Hub handles login and spawns single-user servers on demand.
  • Hub configures proxy to forward URL prefixes to the single-user notebook servers.

JupyterHub also provides a REST API for administration of the Hub and its users.

Installation

Check prerequisites

  • A Linux/Unix based system

  • Python 3.8 or greater

  • nodejs/npm

    • If you are using conda, the nodejs and npm dependencies will be installed for you by conda.

    • If you are using pip, install a recent version (at least 12.0) of nodejs/npm.

  • If using the default PAM Authenticator, a pluggable authentication module (PAM).

  • TLS certificate and key for HTTPS communication

  • Domain name

Install packages

Using conda

To install JupyterHub along with its dependencies including nodejs/npm:

conda install -c conda-forge jupyterhub

If you plan to run notebook servers locally, install JupyterLab or Jupyter notebook:

conda install jupyterlab
conda install notebook

Using pip

JupyterHub can be installed with pip, and the proxy with npm:

npm install -g configurable-http-proxy
python3 -m pip install jupyterhub

If you plan to run notebook servers locally, you will need to install JupyterLab or Jupyter notebook:

python3 -m pip install --upgrade jupyterlab
python3 -m pip install --upgrade notebook

Run the Hub server

To start the Hub server, run the command:

jupyterhub

Visit http://localhost:8000 in your browser, and sign in with your system username and password.

Note: To allow multiple users to sign in to the server, you will need to run the jupyterhub command as a privileged user, such as root. The wiki describes how to run the server as a less privileged user, which requires more configuration of the system.

Configuration

The Getting Started section of the documentation explains the common steps in setting up JupyterHub.

The JupyterHub tutorial provides an in-depth video and sample configurations of JupyterHub.

Create a configuration file

To generate a default config file with settings and descriptions:

jupyterhub --generate-config

Start the Hub

To start the Hub on a specific url and port 10.0.1.2:443 with https:

jupyterhub --ip 10.0.1.2 --port 443 --ssl-key my_ssl.key --ssl-cert my_ssl.cert

Authenticators

Authenticator Description
PAMAuthenticator Default, built-in authenticator
OAuthenticator OAuth + JupyterHub Authenticator = OAuthenticator
ldapauthenticator Simple LDAP Authenticator Plugin for JupyterHub
kerberosauthenticator Kerberos Authenticator Plugin for JupyterHub

Spawners

Spawner Description
LocalProcessSpawner Default, built-in spawner starts single-user servers as local processes
dockerspawner Spawn single-user servers in Docker containers
kubespawner Kubernetes spawner for JupyterHub
sudospawner Spawn single-user servers without being root
systemdspawner Spawn single-user notebook servers using systemd
batchspawner Designed for clusters using batch scheduling software
yarnspawner Spawn single-user notebook servers distributed on a Hadoop cluster
wrapspawner WrapSpawner and ProfilesSpawner enabling runtime configuration of spawners

Docker

A starter docker image for JupyterHub gives a baseline deployment of JupyterHub using Docker.

Important: This quay.io/jupyterhub/jupyterhub image contains only the Hub itself, with no configuration. In general, one needs to make a derivative image, with at least a jupyterhub_config.py setting up an Authenticator and/or a Spawner. To run the single-user servers, which may be on the same system as the Hub or not, Jupyter Notebook version 4 or greater must be installed.

The JupyterHub docker image can be started with the following command:

docker run -p 8000:8000 -d --name jupyterhub quay.io/jupyterhub/jupyterhub jupyterhub

This command will create a container named jupyterhub that you can stop and resume with docker stop/start.

The Hub service will be listening on all interfaces at port 8000, which makes this a good choice for testing JupyterHub on your desktop or laptop.

If you want to run docker on a computer that has a public IP then you should (as in MUST) secure it with ssl by adding ssl options to your docker configuration or by using an ssl enabled proxy.

Mounting volumes will allow you to store data outside the docker image (host system) so it will be persistent, even when you start a new image.

The command docker exec -it jupyterhub bash will spawn a root shell in your docker container. You can use the root shell to create system users in the container. These accounts will be used for authentication in JupyterHub's default configuration.

Contributing

If you would like to contribute to the project, please read our contributor documentation and the CONTRIBUTING.md. The CONTRIBUTING.md file explains how to set up a development installation, how to run the test suite, and how to contribute to documentation.

For a high-level view of the vision and next directions of the project, see the JupyterHub community roadmap.

A note about platform support

JupyterHub is supported on Linux/Unix based systems.

JupyterHub officially does not support Windows. You may be able to use JupyterHub on Windows if you use a Spawner and Authenticator that work on Windows, but the JupyterHub defaults will not. Bugs reported on Windows will not be accepted, and the test suite will not run on Windows. Small patches that fix minor Windows compatibility issues (such as basic installation) may be accepted, however. For Windows-based systems, we would recommend running JupyterHub in a docker container or Linux VM.

Additional Reference: Tornado's documentation on Windows platform support

License

We use a shared copyright model that enables all contributors to maintain the copyright on their contributions.

All code is licensed under the terms of the revised BSD license.

Help and resources

We encourage you to ask questions and share ideas on the Jupyter community forum. You can also talk with us on our JupyterHub Gitter channel.

JupyterHub follows the Jupyter Community Guides.


Technical Overview | Installation | Configuration | Docker | Contributing | License | Help and Resources

autodoc-traits's People

Contributors

betatim avatar blink1073 avatar consideratio avatar dependabot[bot] avatar manics avatar minrk avatar pre-commit-ci[bot] avatar rkdarst avatar willingc avatar yuvipanda avatar

Stargazers

 avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

autodoc-traits's Issues

`autoclass` is not the same as `autoconfigurable` - but should be?

I've realized that the autoclass directive is not the same as autoconfigurable.

  1. We should update the README.md for now to not mention autoclass, but autoconfigurable.
  2. We should consider providing a can_document_member function also for the TraitsletsDocumenter class, and perhaps increase the priority or similar. That is whats done for the other class for Attributes.

Unsure, but more investigations are needed.

Setup tests

It would be nice to have some primitive tests for this project.

Release automation

@willingc I noted that we need to cut a release of this project for jupyterhub/oauthenticator to be able to use Sphinx 4 without installing it via a git reference. That made me motivated to setup some common JupyterHub org automation with regards to releases etc in this repo. What do you think?

Goals

Action points

  • @willingc could you review the goals and action points?
  • @willingc could you add jupyterhubbot and perhaps another human or two (consideratio) to the associated PyPI project (https://pypi.org/project/autodoc-traits/) ?
  • I'll go ahead and work towards all the goals if found to be acceptable

Update

I missed that @minrk has opened #4 already! There are some steps to take with documentation to follow that up with.

New pip release request

Bug description

Hi, I hate to pester (and apologies if this isn't an appropriate place to put this request!), but please could you put a new release on PyPi, or create a new tag here on github? The version on PyPi doesn't work with sphinx v4, due to the removal of PyClassmember. There have been additional autodoc-traits fixes too (eg. regarding duplicate documentation of configurable properties), since the last PyPi release. I ask because I'd rather not have to either pin to a sphinx version < 4, and/or put the git+git:// ref in a docs/requirements.txt file.

Thanks, A

Document that sphinx.ext.autodoc lacks MyST support, so autodoc_traits doesn't have it either

I understand it as autodoc_traits will influence sphinx.ext.autodoc, which in turn will emit rST, which makes the autodoc directives need to be parsed as rST and not MyST.

I think this is relevant to document clearly in autodoc_traits README file, and if possible, link to an issue upstream where we can track effort to resolve this in a sensible way.

So practically:

# this will work in a MyST parsed .md file
```{eval-rst}
.. automodule:: kubespawner.utils
```

# this won't work in a MyST parsed .md file
# it will emit rST directives like `.. py:module:: kubespawner.utils`
# and docstrings in raw format, which then also should be in rST
```{automodule} kubespawner.utils

```

So this example MyST markdown file:

# Utilities

```{automodule} kubespawner.utils

```

Will end up looking like this as rendered HTML:

image

Related

Help strings incorrectly considered missing for some but not all trait types

It seems that the help strings for traits of type Dict, List, Instance are missing since ipython/traitlets#818 released with traitlets 5.10.0 in september, but they aren't missing for Bool, Integer, Unicode.

I've added a test to capture this in #54, but I don't have an understanding on what happened in ipython/traitlets#818 that caused this.

Related code

config_trait_members = self.object.class_traits(config=True).items()
for trait_tuple in config_trait_members:
name, trait = trait_tuple
if not trait.__doc__:
warnings.warn(
f"""
Documenting {self.object.__name__}.{trait.name} without a help string because it has config=True.
Including undocumented config=True traits is deprecated in autodoc-traits 1.1.
Add a help string:
{trait.name} = {trait.__class__.__name__}(
help="...",
)
to keep this trait in your docs,
or include it explicitly via :members:
""",
FutureWarning,
)
# FIXME: in the unlikely event that the patched trait
# is documented multiple times in the same build,
# this patch will cause it to have a truthy help string
# elsewhere, not just in this autoconfigurable instance.
trait.__doc__ = trait.help = "No help string is provided."
if name not in existing_member_names:
existing_member_names.add(name)
members.append(ObjectMember(name, trait))
return check, members

The `inherited-members` Sphinx directive option is respected backwards

The sphinx.ext.autodoc provided directives support a inherited-members option, and acts as an opt-in option - defaulting to not including inherited members.

But I think we have implemented it backwards, so that we by default include inherited members, and when inherited-members is passed we don't include them.

A first step to address this issue could be to setup basic tests (#17), and/or investigate it further to verify how things work with and without the inherited-members directive option.

Autoregister sphinx.ext.autodoc when autodoc_traits is registered as a sphinx extension?

In this sphinx example on how to write an extension for sphinx.ext.autodoc, they register the sphinx.ext.autodoc extension in the extension itself like this:

def setup(app: Sphinx) -> None:
    app.setup_extension('sphinx.ext.autodoc')  # Require autodoc extension
    app.add_autodocumenter(IntEnumDocumenter)

Should we do the same in this extension? That would make someone able to just add autodoc_traits as a sphinx extensions in their sphinx conf.py file.

def setup(app):
app.add_autodocumenter(ConfigurableDocumenter)
app.add_autodocumenter(TraitDocumenter)

Add a summary of how it works in the readme

I think having a "How it works" section (example from jupyterhub-idle-culler) in the readme would be great. I'm currently debugging an issue where I look to determine if its an issue with autodoc_traits, sphinx.ext.autodoc, myst_parser, or something else. Not understanding how all the pieces work makes me unable to figure out what my problem is.

Current understanding of autodoc_traits

  • autodoc_traits registeres two new "autodocumenter" classes that sphinx.ext.autodoc will honor using add_autodocumenter
    def setup(app):
    app.add_autodocumenter(ConfigurableDocumenter)
    app.add_autodocumenter(TraitDocumenter)
  • autodoc_trait's autodocumenter classes include details like below, which helps the sphinx.ext.autodoc directives decide if and when the autodocumenter is relevent. The best documentation of these I've found are in this example
    class TraitDocumenter(AttributeDocumenter):
    objtype = "trait"
    directivetype = "attribute"
    member_order = 1
    priority = 100
  • From here on, autodoc_traits work is done, and sphinx.ext.autodoc takes over using the added autodocumenter classes when suitable
  • sphinx.ext.autodoc will generate .. py: directives with provided docstrings etc.

To summarize, autodoc_traits will register autodocumenter classes with sphinx.ext.autodoc, which in turn will scan source code and emit rST declarations of .. py: directives.

Request for review

I can document this in the README, but I'd love some input/review of this understanding. Review questions could be:

  • is it accurate?
  • does it capture how things work?
  • is there other things to include or link out to?

autoclass finds traits, but doesn't find their docstrings

Bug description

When using autoclass (not autoconfigurable), traits with help strings are treated as undocumented. That means they are excluded from :members:, but included with :undoc-members:. When they are included with :undoc-members:, they are all represented with their class docstring instead of their help string, like:

Expected behaviour

members/undoc-members should recognize the help string of traitlets as being documented, and be included/excluded accordingly.

Actual behaviour

all traits are treated as undocumented members, help strings are not extracted, and all traits are represented as configurable, even when they are not.

How to reproduce

..  autoclass:: test_module.TestConfigurable
   :noindex:
   :members:
   # :undoc-members:

produces output like

    trait_noconfig = Bool(
        help="""trait_noconfig help text""",
    )

into

trait_noconfig c.SampleConfigurable.trait_noconfig = Bool(False)
    A boolean (True, False) trait.

Note:

  • help string is wrong
  • formatting is for a config trait, which is also wrong

More info

Issue prompted by #28 (comment)

This is a low priority issue, since most things should use autoconfigurable, which does this right. But good general autoclass support for HasTraits would be nice, if possible.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.