It looks like jupinx is using 100% cpu while it waits for users to close the server using Ctrl-C.

https://files.slack.com/files-pri/T6A2RJC1F-FM69P2T2S/htop.png

We should review how Popen opens the servers and closes them at exit.

Let's add this and change the tutorial / quickstart to reflect it.

• now that we launch an instance of notebookapp we can also specify a file to open. Perhaps we should add this functionality into the notebook preview and html preview to open a specific page.

I am unsure how we would handle a list of files though (probably assume open the first in the list?).

It would be nice to have -c, --clean to force a clean run through jupinx

We should support preview option to open a notebook server in _build/jupyter

We should update jupinx cmd backend to call sphinx-build directly as it will make the tool more flexible without any reliance on Makefile and less reliance on setting up a repository in a jupinx friendly way.

Frameworks to look at:

• Fabric (http://www.fabfile.org/) [remote task execution and invoke local tasks]

CLI Libraries

• Click (https://click.palletsprojects.com/en/7.x/)
• CMD2 https://github.com/python-cmd2/cmd2
• Fire (https://github.com/google/python-fire)
• Cement (https://builtoncement.com)

This version jupinx 0.2.1, running in a conda environment list included conda-env.txt.

 $ mkdir jupinx-project
(jupinx) [[email protected] ~/Desktop/Inbox]
 $ cd jupinx-project/
(jupinx) [[email protected] ~/Desktop/Inbox/jupinx-project]
 $ jupinx-quickstart 
Welcome to the Jupinx 0.2.1 quickstart utility.

Please enter values for the following settings (just press Enter to
accept a default value, if one is given in brackets).

Enter the root path for documentation.
> Root path for the documentation [.]: 

The project name will occur in several places in the built documentation.
> Project name: blah
> Author name(s): blah

Jupinx has the notion of a "version" and a "release" for the
software. Each version can have multiple releases. For example, for
Python the version is something like 2.5 or 3.0, while the release is
something like 2.5.1 or 3.0a1.  If you don't need this dual structure,
just set both to the same value.
> Project version []: 2019
> Project release [2019]: 

If the documents are to be written in a language other than English,
you can select a language here by its language code. Sphinx will then
translate text that it generates into that language.

For a list of supported codes, see
https://www.sphinx-doc.org/en/master/usage/configuration.html#confval-language.
> Project language [en]: 

Select the kernels which you want for your jupyter notebook conversion
> Do you want to have python3 in your kernel list? (y/n) [y]: 
> Do you want to have python2 in your kernel list? (y/n) [y]: n
> Do you want to have julia-1.1 in your kernel list? (y/n) [y]: n
> Do you want to have julia-1.2 in your kernel list? (y/n) [y]: n
Indicate which of the following Sphinx extensions should be installed:
> sphinxcontrib-jupyter package has been found in your system. Would you like to upgrade it? (y/n) [y]: 
> sphinxcontrib-bibtex package has been found in your system. Would you like to upgrade it? (y/n) [y]: 

Traceback (most recent call last):
  File "/Users/gavin/miniconda3/envs/jupinx/bin/jupinx-quickstart", line 10, in <module>
    sys.exit(main())
  File "/Users/gavin/miniconda3/envs/jupinx/lib/python3.7/site-packages/jupinx/cmd/quickstart.py", line 527, in main
    generate(d, overwrite=False)
  File "/Users/gavin/miniconda3/envs/jupinx/lib/python3.7/site-packages/jupinx/cmd/quickstart.py", line 379, in generate
    copy_tree(source_theme_path + "/", target_theme_path , preserve_symlinks=1)
  File "/Users/gavin/miniconda3/envs/jupinx/lib/python3.7/distutils/dir_util.py", line 124, in copy_tree
    "cannot copy tree '%s': not a directory" % src)
distutils.errors.DistutilsFileError: cannot copy tree '/Users/gavin/miniconda3/envs/jupinx/lib/python3.7/site-packages/jupinx/theme/minimal/
[conda-env.txt](https://github.com/QuantEcon/jupinx/files/3737302/conda-env.txt)
': not a directory

Target address = https://quantecon.github.io/jupinx/

Hi @jstac and @mmcky , I find we still use pip install --upgrade python in the tutorial of Jupinx. Since Python 2.7 reached its end of life on January 1st, 2020, this command line would cause some warnings, like the one attached below.

Shall we update it by using pip3? I am afraid that this command line would cause warnings elsewhere, such as our lectures.

Not sure the issue belongs here. At the end of this notebook https://notes.quantecon.org/submission/5ddb3c926bad3800109084bf there is a markdown code area. It should be rendered with fixed fonts, but apparently it isn't. More precisely, it seems like spaces are ignored. Apparently, they are stripped from the output.

It appears os x doesn't have Liberation Mono as a default font

WARNING: xelatex exited with returncode 1 , encounterd in index with error -- b"\nkpathsea: Running mktextfm LiberationMono\n/usr/local/texlive/2019/texmf-dist/web2c/mktexnam: Could not map source abbreviation L for LiberationMono.\n/usr/local/texlive/2019/texmf-dist/web2c/mktexnam: Need to update /usr/local/texlive/2019/texmf-dist/fonts/map/fontname/special.map?\nmktextfm: Running mf-nowin -progname=mf \\mode:=ljfour; mag:=1; nonstopmode; input LiberationMono\nThis is METAFONT, Version 2.7182818 (TeX Live 2019) (preloaded base=mf)\n\nkpathsea: Running mktexmf LiberationMono\n\n! I can't find file `LiberationMono'.\n<*> ...; mag:=1; nonstopmode; input LiberationMono\n                                                  \nPlease type another input file name\n! Emergency stop.\n<*> ...; mag:=1; nonstopmode; input LiberationMono\n 

may need to consider changing to a different font for cross platform suport

The folder structure needed for source and build directories while creating a project with jupinx.
It can be either hard-coded or configurable, and can have different folders or only one base folder etc.

Should just exit and issue this once

Error: an existing conf.py has been found in the selected root path.
jupinx-quickstart will not overwrite existing Jupinx projects.


Error: an existing conf.py has been found in the selected root path.
jupinx-quickstart will not overwrite existing Jupinx projects.

https://pypi.org/project/Jupinx/

@AakashGfude do you mind if we remove python=2 support.

sphinxcontrib-jupyter has it to support the Sage project but I don't think we will need python2 support in this cli application.

This is because they should be uppercase. e.g. index.html links http://penguin.linux.test:8901/demo_notebook.html#math for the math section, but it should be #Math (same for #tables vs. #Tables).

It would be easier to edit if the page site was available in markdown documents.

@DrDrij can you please investigate:
https://github.blog/2016-12-09-publishing-with-github-pages-now-as-easy-as-1-2-3/

It looks like Github pages uses Jekyll in the background.

Using a new anaconda environment we should be able to:

1. pip install jupinx
2. ``jupinx -ninlecture-source-py` repo

@AakashGfude can we ensure:

sphinxcontrib-bibtex and sphinxcontrib-jupyter are installed when jupinx is installed.

@AakashGfude let's do some environment testing for jupinx:

1. base anaconda environment

Check:

1. all dependencies are met to go straight ahead and git clone lecture-source-py and jupinx -n

@DrDrij can we make the Jupinx logo on the site clickable to return to the index.md page. Tried to get back from the tutorial that way.

It might be nice to add a --fetch to clone a repository and run the equivalent of make setup. This could be added if we bring configuration of sphinx-build closer to the jupinx command line tool.

We should add the ability to specify filenames to sphinx-build to run targeted builds of individual files as a final input. sphinx-build will support this - but we will need to construct the full sphinx-build command.

sphinx-build [options] <sourcedir> <outputdir> [filenames …]

• use as a test case for using fabric as a task runner

We could consider adding sync tools for updating the live site with s3 support

So -v, --view can be used independently we should add directory checking to make sure the results are available for viewing.

It would be nice to have argument auto-completion.

• https://argcomplete.readthedocs.io/en/latest/
• https://github.com/python-cmd2/cmd2

The cmd2 project looks like it provides some nice features like highlighting etc.

I first ran the quickstart as normal, and then wanted to try it with the notebooks executing via adding jupyter_execute_notebooks = True to conf.py. But re-running jupinx -n . didn't do anything, since no files had changed:

looking for now-outdated files... none found
no targets are out of date.

Is there a way to force jupinx -n to re-build?

We should add some automatic tests for this tool.

@DrDrij we would like to add templates to support html conversion using nbconvert but it should be a simple minimal example for jupinx-quickstart. It would be great if you could put one together and we will remove the lectures-nbconvert.tpl:

https://github.com/QuantEcon/jupinx/tree/master/jupinx/templates/html

The minimal template for html should include:

1. simple looking layout
2. lots of documentation in the template file for users to make adjustments
3. minimal css file (if required).

• update jupinx-quickstart and add question if user wants to build websites from sphinxcontrib-jupyter to populate with make website and templates
• populate theme/templates with html template and add necessary options to conf.py file.

For example::

Present situation:

RST - :math:\$ 25
ipynb - $ \\$ 25 $ ##in this case the $ after // is considered as closing dollar for math

It should be:

ipynb - $\$ 25$

I guess some update to the conversion script (rst to ipynb) could make it work.

The current setup requires a directory to have a trailing /

jupinx -n lectures_source_py/

this is added by typical shell auto-completions but if a user just types

jupinx -n lectures_source_py

it will fail.

This should be handled better

Would help people get started with projects that include citations.

We should add a default / basic template for html conversion or use the default template provided by nbconvert to populate a minimum repo.

• add question if user wants to build websites from sphinxcontrib-jupyter
• populate theme/templates with html template and add necessary options to conf.py file.

I've used Jupinx to create this notebook, which includes a Bokeh data visualization that renders correctly in the notebook. However, in the webpage (preview) it doesn't show up. Is there a way to get these interaction visualizations to render in jupinx-built webpages?

Screenshots:
Jupyter

Webpage (locally)

Webpage (using htmlpreview.github.io)

Windows uses .bat files instead of Makefiles.

We are not currently testing or supporting Windows at this stage - but it will be a big user group to cater for.

After running jupinx-quickstart and replying n to Julia, the conf.py file contains the lines

"julia-1.1": false,
"julia-1.2": false,

Here false should be replaced by False.

If I run something like this:

jupinx -w --files source/rst/more_julia/data_statistical_packages.rst

it works, but if I do:

jupinx -w -f source/rst/more_julia/data_statistical_packages.rst

or:

jupinx -wf source/rst/more_julia/data_statistical_packages.rst

then it tells me that -f is not recognized.

This is also in line with jupinx -h, which doesn't have a -f flag under additional options (just --files).

At the moment, we have the following two variables with there default values in conf.py, which needs to be adjusted according to our requirements (if necessary)

templates_path = ['templates']
html_static_path = ['static']

I'm using jupinx to build an rst that pulls data from the web, The executed notebook builds fine, but the webpage gives this error:

NewConnectionError: <urllib3.connection.VerifiedHTTPSConnection object at 0x7e2a5d9b4690>: Failed to establish a new connection: [Errno -2] Name or service not known
...
MaxRetryError: HTTPSConnectionPool(host='raw.githubusercontent.com', port=443): Max retries exceeded with url: /PSLmodels/Tax-Calculator/master/taxcalc/reforms/2017_law.json (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7e2a5d9b4690>: Failed to establish a new connection: [Errno -2] Name or service not known'))

I've tried rebuilding a couple times, but the error is consistent.

• update https://github.com/QuantEcon/sphinxcontrib-jupyter.minimal

jupinx-quickstart gives the option of including/not julia 1.1 etc. I selected n. Then running jupinx -w fails with:

 $ jupinx -w
Running: make website
Running Sphinx v2.2.0

Configuration error:
There is a programmable error in your configuration file:

Traceback (most recent call last):
  File "/Users/gavin/miniconda3/envs/jupinx/lib/python3.7/site-packages/sphinx/config.py", line 361, in eval_config_file
    execfile_(filename, namespace)
  File "/Users/gavin/miniconda3/envs/jupinx/lib/python3.7/site-packages/sphinx/util/pycompat.py", line 81, in execfile_
    exec(code, _globals)
  File "/Users/gavin/Desktop/Inbox/jupinx-project/conf.py", line 73, in <module>
    "julia-1.1": false,
NameError: name 'false' is not defined

make: *** [website] Error 2

Checking conf.py reveals the following json statements:

    "julia-1.1": false,
    "julia-1.2": false,
    "python2": false,

Changing the false to False allows successful generation of the website.

When we have the initial Quickstart setup we should release via PyPI.

jupinx requires sphinx=1.8.5 and we should fail on install if this condition is not met.

The docs should be setup in docs and use sphinx to build

• a section on the defaults used and structure of the repository
• a section on how to customise using conf.py
• links to upstream sphinx docs

The default question list which sphinx has at the moment are mentioned below as task lists, to indicate which one should be kept.

NOTE: The fields/questions which require an input are written in bold

• 1) Selected root path: .
  You have two options for placing the build directory for Sphinx output.
  Either, you use a directory "_build" within the root path, or you separate
  "source" and "build" directories within the root path.
  Separate source and build directories (y/n)

@mmcky: I would vote for a default directory structure not customisable in Quickstart but rather via conf.py and detailed in docs of the structure and how to change it.

• 2) Inside the root directory, two more directories will be created; "_templates"
  for custom HTML templates and "_static" for custom stylesheets and other static
  files. You can enter another prefix (such as ".") to replace the underscore.
  Name prefix for templates and static dir []:

@mmcky: I would vote for a default templates and _static location

• 3) The project name will occur in several places in the built documentation.
  Project name:
  Author name(s):
  Project release[]:

@mmcky I think this is needed. Project release is probably the only confusing one for scientific publishing use case.

• 4) If the documents are to be written in a language other than English, you can select a language here by its language code. Sphinx will then
  translate text that it generates into that language.
  For a list of supported codes, see
  http://sphinx-doc.org/config.html#confval-language.
  Project language [en]:

@mmcky: I think we should remove this for now (as sphinxcontrib-jupyter doesn't support language sets)

• 5) The file name suffix for source files. Commonly, this is either ".txt"
  or ".rst". Only files with this suffix are considered documents.
  Source file suffix [.rst]:

• 6) One document is special in that it is considered the top node of the
  "contents tree", that is, it is the root of the hierarchical structure
  of the documents. Normally, this is "index", but if your "index"
  document is a custom template, you can also set this to another filename.
  Name of your master document (without suffix) [index]:

@mmcky: I think this is a useful question but perhaps not necessary if we force index as the top level document.

• 7) Indicate which of the following Sphinx extensions should be enabled:
  **autodoc: automatically insert docstrings from modules (y/n) [n]: **
  doctest: automatically test code snippets in doctest blocks (y/n) [n]:
  intersphinx: link between Sphinx documentation of different projects (y/n) [n]:
  todo: write "todo" entries that can be shown or hidden on build (y/n) [n]:
  coverage: checks for documentation coverage (y/n) [n]:
  imgmath: include math, rendered as PNG or SVG images (y/n) [n]:
  mathjax: include math, rendered in the browser by MathJax (y/n) [n]:
  ifconfig: conditional inclusion of content based on config values (y/n) [n]:
  viewcode: include links to the source code of documented Python objects (y/n) [n]:
  githubpages: create .nojekyll file to publish the document on GitHub pages (y/n) [n]:

@mmcky: I think this should be culled to the set of packages we have tested with sphinxcontrib-jupyter such as sphinxcontrib-bibtex. And then we can add more to the list as the needs or use grows.

• 8) A Makefile and a Windows command file can be generated for you so that you
  only have to run e.g. `make html' instead of invoking sphinx-build
  directly.
  Create Makefile? (y/n) [y]:
  Create Windows command file? (y/n) [y]:

@mmcky: Let's put windows support on hold (as discussed in Slack) but writing the Makefile could be a default.

We should setup jupinx command line tool to assist with triggering builds using the following commands:

-j, --jupyter : make jupyter
-n, --notebook : make jupyter
-c, --coverage : make coverage
-w, --website : make website
-h, --help : help file
-p, --preview={website} : make preview (default to notebooks, with option to target website)

the jupinx tool should also provide a minimal help guide if entered without options.

Currently the python dict is written as a single line

# -- jupyter build configuration ---------------------------------------------------
jupyter_kernels = {'python3': {'kernelspec': {'display_name': 'Python', 'language': 'python3', 'name': 'python3'}, 'file_extension': '.py'}, 'python2': {'kernelspec': {'display_name': 'Python', 'language': 'python2', 'name': 'python2'}, 'file_extension': '.py'}, 'julia-1.1': {'kernelspec': {'display_name': 'Julia 1.1', 'language': 'julia', 'name': 'julia-1.1'}, 'file_extension': '.jl'}}

it would be nice to write this more neatly as

    # -- jupyter build configuration ---------------------------------------------------
    jupyter_kernels = {
        'python3': {
            'kernelspec': {
                'display_name': 'Python', 
                'language': 'python3', 
                'name': 'python3'
            }, 
            'file_extension': '.py'
        }, 
        'python2': {
            'kernelspec': {
                'display_name': 'Python', 
                'language': 'python2', 
                'name': 'python2'
            }, 
            'file_extension': '.py'
        }, 
        'julia-1.1': {
            'kernelspec': {
                'display_name': 'Julia 1.1', 
                'language': 'julia', 
                'name': 'julia-1.1'
            }, 
            'file_extension': '.jl'
        }
    }

given conf.py is a human readable document

Hi!
I downloaded jupinx via pip (it downloaded version 0.2.2) and after following the steps detailed in the tutorial I ran the command to build the website but it failed with the following error:

$ jupinx -w .                                                                                                          (16m 20s 268ms)  
Running: make website
Running Sphinx v2.3.1
making output directory... done
WARNING: html_static_path entry 'static' does not exist
building [mo]: targets for 0 po files that are out of date
building [jupyter]: targets for 3 source files that are out of date
updating environment: [new config] 3 added, 0 changed, 0 removed
reading sources... [100%] index                                                                                                          
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
writing output... [100%] index                                                                                                           
copying static files... copying static files to executed folder... 
done
Starting notebook execution for website and html conversion(if set in config)...

Exception occurred:
  File "/home/<user>/.pyenv/versions/3.7.0/lib/python3.7/site-packages/sphinxcontrib/jupyter/writers/execute_nb.py", line 125, in task_execution_time
    computing_time = time_tuple[2] - time_tuple[1]
KeyError: 2
The full traceback has been saved in /tmp/sphinx-err-wdeugawu.log, if you want to report the issue to the developers.
Please also report this if it was a user error, so that a better error message can be provided next time.
A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!
make: *** [Makefile:44: website] Error 2

Do i need to do something before to get the website working?