d2l-ai / d2l-book Goto Github PK

Say we use tab.selected:

Newline following such tab_select may disappear in certain tabs:

When deploying the generated html to github, it will be treated as a jekyll website thus the content won't be rendering correctly. Namely, in this case, the html pages won't be able to load static files like css and images which leads to having a broken page.

A quick fix is to add an empty file named .nojekyll to the root of the html folder.

Since this is essential to have correct display, we can:

• Automatically add this file .nojekyll when d2lbook deploy html is run
• Or instruct the user to add in the documentation.

Here's an example deployment I did on github with the above tip

• https://github.com/mohamed-ali/d2l-ar-demo
• https://mohamed-ali.github.io/d2l-ar-demo/

Thanks to 06ce665, now D2L replaces fluent alias (e.g., d2l.reshape(a, (1,2)) -> a.reshape((1,2))) correctly!

Due to the use of yapf in the code prettification step, the following are new minor style issues:

• d2l.plt.scatter(d2l.numpy(features[:, 1]), d2l.numpy(labels), 1); will become d2l.plt.scatter(features[:, (1)].asnumpy(), labels.asnumpy(), 1), where the trailing semicolon (useful for showing figures only in notebooks) is trimmed and 1 is surrounded with unnecessary parentheses.

• yapf may unnecessarily produce two empty lines instead of one empty line (space inefficient for physical books):

• yapf may unnecessarily produce empty lines within our import blocks (space inefficient for physical books):

nbconvert is pinned to version 5.6.1 in d2lbook. This older version doesn't include this fix for the earlier deprecated (and recently completely remove functions in jinja2).

Hence recently when the latest version (v3.1.0) of jinja2 was released on Mar 24, 2022, we started to get the following import error because of the removed functions like contextfilter.

ImportError: cannot import name 'contextfilter' from 'jinja2'

To fix this issue either we can pin jinja2 to 3.0 (which may not be the best idea) or we can unpin nbconvert and use the latest version which already handles the fix for removed functions.

Note that d2lbook has two dependencies: nbconvert and mu-notedown. For nbconvert>=6.2.0 (nbconvert 6.0.0 and 6.1.0 have other issues when running jupyter notebook), when using Jupyter Notebook to edit a markdown file (enabled by mu-notedown), saving a file throws the following error message:

In my locale, editing

~/miniconda3/envs/np3/lib/python3.9/site-packages/notedown/templates/markdown.tpl
or
~/miniconda3/envs/np3/lib/python3.8/site-packages/mu_notedown-2.0.3-py3.8.egg/notedown/templates/markdown.tpl

as https://github.com/aaren/notedown/pull/96/files resolves the issue.

Thus, the following changes are proposed:

• Edit https://github.com/mli/notedown/blob/master/setup.py#L19 as 'nbconvert>=6.2.0',
• Edit https://github.com/mli/notedown/blob/master/notedown/templates/markdown.tpl#L1 as {% extends 'base/display_priority.j2' %}
• Upload an upgraded version of mu-notedown after bumping a version at https://github.com/mli/notedown/blob/master/setup.py#L10
• Remove https://github.com/d2l-ai/d2l-book/blob/master/setup.py#L11-L12 since both are required in mu-notedown

Related discussions:

https://github.com/aaren/notedown/pull/96/files
aaren/notedown#95
jupyter/nbconvert#1369

I was following CONTRIBUTING docs to work on an issue raised earlier today in d2l-en

When i run pip install git+https://github.com/d2l-ai/d2l-book in my anaconda environment, pip is stuck in a continuous loop of fetching the awscli package, decrementing the package version by a minor version every iteration. See pic below.

Am i doing something wrong or is it a version constraint issue in setup.py?

Thanks for the tool. I found some render problems in d2l ipynb and pdf that might be related to this repo.

Screenshot of d2l-en/xxx/chapter_multilayer-perceptrons/mlp.ipynb:

I'm not sure whether the rst strings like :numref: is an error because there are raw rst strings in the top index.ipynb as well. Should some extensions be installed for jupyter notebook to render this, or is it a compilation error?

Screenshot of d2l-en.pdf, page170:

\$ should be $

When installing d2lbook in a completely new environment with sphinx not cached by previous pip installations, the following error is raised:

Collecting mxtheme>=0.3.16
  Downloading mxtheme-0.3.16.tar.gz (8.3 MB)
    ERROR: Command errored out with exit status 1:
     command: /home/d2l-worker/miniconda3/envs/d2l-en-master-0/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-g2gp4y2_/mxtheme_a6ceecb6302d4ebaa879512c5d3e27b6/setup.py'"'"'; __file__='"'"'/tmp/pip-install-g2gp4y2_/mxtheme_a6ceecb6302d4ebaa879512c5d3e27b6/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-sjca2a5v
         cwd: /tmp/pip-install-g2gp4y2_/mxtheme_a6ceecb6302d4ebaa879512c5d3e27b6/
    Complete output (9 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-g2gp4y2_/mxtheme_a6ceecb6302d4ebaa879512c5d3e27b6/setup.py", line 2, in <module>
        from mxtheme import __version__
      File "/tmp/pip-install-g2gp4y2_/mxtheme_a6ceecb6302d4ebaa879512c5d3e27b6/mxtheme/__init__.py", line 2, in <module>
        from .card import CardDirective
      File "/tmp/pip-install-g2gp4y2_/mxtheme_a6ceecb6302d4ebaa879512c5d3e27b6/mxtheme/card.py", line 1, in <module>
        from sphinx.locale import _
    ModuleNotFoundError: No module named 'sphinx'
    ----------------------------------------
WARNING: Discarding https://files.pythonhosted.org/packages/ed/74/9bdb93269b054b9aefdc16b91e8ec8603afa16ba2036cdd5a10c5b0c9c71/mxtheme-0.3.16.tar.gz#sha256=e3524a6ea78f56686b14c866539aa49c8507b01ab414937454a7fbe6b55aaeac (from https://pypi.org/simple/mxtheme/). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
ERROR: Could not find a version that satisfies the requirement mxtheme>=0.3.16 (from d2lbook) (from versions: 0.3.2, 0.3.3, 0.3.5, 0.3.6, 0.3.7, 0.3.8, 0.3.9, 0.3.10, 0.3.11, 0.3.12, 0.3.13, 0.3.14, 0.3.15, 0.3.16)
ERROR: No matching distribution found for mxtheme>=0.3.16
script returned exit code 1

Attaching an example failed CI Log here.
This is because d2lbook depends on sphinx and mxtheme is also one of the d2lbook dependencies which depends on sphinx.

Hi! I just found this project and I saw on its homepage that it mentions Jupyter Book.

Jupyter book: very similar to d2lbook for
publishing Jupyter notebooks. Two main design differences are 1) d2lbook
encourage you to use markdown file format and remove all code outputs before
saving notebooks. 2) d2lbook uses Sphinx instead of Jekyll adopted by
Jupyter book to build the document.
Jekyll is easier to customize the theme, while Sphinx has better supports for PDF and analyzing Python codes.

(from https://github.com/d2l-ai/d2l-book/blob/master/docs/index.md)

However, I think it's not 100% accurate since Jupyter Book uses Sphinx behind the scenes and also encourage people to use Markdown by using the myst-parser (https://myst-parser.readthedocs.io/en/latest/) created by the same organization, Executable Books.

cc @choldgraf

Currently, CI errors out with exit code 255 in case of Latex syntax errors and doesn't show the exact errors on the CI log.

Latexmk: Summary of warnings:
  Latex failed to resolve 92 reference(s)
  Latex failed to resolve 38 citation(s)
Collected error summary (may duplicate other messages):
  pdflatex: Command for 'pdflatex' gave return code 256
Latexmk: Use the -f option to force complete processing,
 unless error was exceeding maximum runs of latex/pdflatex.
Latexmk: Errors, so I did not complete making targets
Makefile:29: recipe for target 'd2l-pt.pdf' failed
make: *** [d2l-pt.pdf] Error 12
script returned exit code 255

One example where this could have been useful is attached here.

This is then handled only by running d2lbook build pdf locally to reproduce the issue and fix the latex errors, which is frustrating and quite inconvenient.

The proposed feature request is for improved logging of these latex errors to facilitate easier and transparent debugging using CI Logs.

cc @astonzhang

In the tutorial, we can typeset code in markdown as such:

1+2+3

however, I'd like the original RTD syntax with additional code blocks such as linenos, which helps to point out the line number.

.. code-block:: python
    :linenos:

    a = 2
    b = 3

Are there any hacks to make this appear?

At the moment, it seems the build system creates empty notebooks for those branches where there is no tab yet (for example some of the pytorch parts in some notebooks in the d2l book). It would be useful to create the notebooks (with all texts and images) and include there, where the code should be, a message which says 'not implemented yet'.

In this section:
http://preview.d2l.ai.s3-website-us-west-2.amazonaws.com/d2l-en/master/chapter_computational-performance/multiple-gpus-concise.html

we have several code cells where only MXNet implementations are available but PyTorch implementations are not applicable:

However, when clicking the PyTorch tabs, those MXNet tabs do not disappear properly:

After running d2lbook translate once, the source git repo (eg. d2l-en) needs to be removed manually before running again.
I'll send a PR to fix this.

Eg:

d2lbook translate --commit 8c24d68 chapter_recommender-systems/deepfm.md
rm -rf _build/origin_repo/

I am using python 3.7 with d2lbook 0.1.9.

$ python --version 
Python 3.7.2
$ pip freeze |grep d2lbook 
d2lbook==0.1.9

When I try to initialize a new project with the two example commands in the documentation, I get the following error message:

$ d2lbook create mybook

Error:

--- Logging error ---
Traceback (most recent call last):
  File "/usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/logging/__init__.py", line 1034, in emit
    msg = self.format(record)
  File "/usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/logging/__init__.py", line 880, in format
    return fmt.format(record)
  File "/usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/logging/__init__.py", line 619, in format
    record.message = record.getMessage()
  File "/usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/logging/__init__.py", line 380, in getMessage
    msg = msg % self.args
TypeError: not all arguments converted during string formatting
Call stack:
  File "/Users/SomeUser/Desktop/workspace/books/bin/d2lbook", line 10, in <module>
    sys.exit(main())
  File "/Users/SomeUser/Desktop/workspace/books/lib/python3.7/site-packages/d2lbook/main.py", line 17, in main
    config = Config('config.ini')
  File "/Users/SomeUser/Desktop/workspace/books/lib/python3.7/site-packages/d2lbook/config.py", line 8, in __init__
    logging.fatal('Failed to find the config', config_fname)
Message: 'Failed to find the config'
Arguments: ('config.ini',)
[d2lbook:config.py:L9] CRITICAL You can use "d2lbook create ." to create a default config

$ d2lbook create mybook --demo

--- Logging error ---
Traceback (most recent call last):
  File "/usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/logging/__init__.py", line 1034, in emit
    msg = self.format(record)
  File "/usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/logging/__init__.py", line 880, in format
    return fmt.format(record)
  File "/usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/logging/__init__.py", line 619, in format
    record.message = record.getMessage()
  File "/usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/logging/__init__.py", line 380, in getMessage
    msg = msg % self.args
TypeError: not all arguments converted during string formatting
Call stack:
  File "/Users/SomeUser/Desktop/workspace/books/bin/d2lbook", line 10, in <module>
    sys.exit(main())
  File "/Users/SomeUser/Desktop/workspace/books/lib/python3.7/site-packages/d2lbook/main.py", line 17, in main
    config = Config('config.ini')
  File "/Users/SomeUser/Desktop/workspace/books/lib/python3.7/site-packages/d2lbook/config.py", line 8, in __init__
    logging.fatal('Failed to find the config', config_fname)
Message: 'Failed to find the config'
Arguments: ('config.ini',)
[d2lbook:config.py:L9] CRITICAL You can use "d2lbook create ." to create a default config

My assumption is that if the config.ini doesn't exist it will be created automatically.

@mli Do you know how to fix this? Thanks.

How to manage the localization-translation strings ? I put lang = fr in config.ini but it has no effect. I would like to translate the « Provious » and « Next » in the bottom of the html pages.

Thanks.

d2l.reshape(X, (1,1)) and d2l.size(y) need to be rendered back to pytorch/tf/mx version like X.reshape(1,1) and y.size in d2l.ai.

You can find these being rendered with the d2l alias which is not expected. You can find these rendering issues here and in many other sections of the book.

Attaching a few screenshots below.

d2l.reshape

d2l.size

在**内地无法访问

There is a package revision mismatch during my installation. The setup.py only specific jupyter-client version while there is no jupyter-console version.

$ python setup.py install
......
Installed /home/**/miniconda3/lib/python3.9/site-packages/jupyter_client-6.1.11-py3.9.egg
Searching for six
Reading https://pypi.org/simple/six/
Downloading https://files.pythonhosted.org/packages/d9/5a/e7c31adbe875f2abbb91bd84cf2dc52d792b5a01506781dbcf25c91daf11/six-1.16.0-py2.py3-none-any.whl#sha256=8abb2f1d86890a2dfb989f9a77cfcfd3e47c2a354b01111771326f8aa26e0254
Best match: six 1.16.0
Processing six-1.16.0-py2.py3-none-any.whl
Installing six-1.16.0-py2.py3-none-any.whl to /home/**/miniconda3/lib/python3.9/site-packages
Adding six 1.16.0 to easy-install.pth file

Installed /home/**/miniconda3/lib/python3.9/site-packages/six-1.16.0-py3.9.egg
Searching for jupyter-console
Reading https://pypi.org/simple/jupyter-console/
Downloading https://files.pythonhosted.org/packages/fc/e9/5d4e1e616f7d7a8a9d7f313ac14bf43d1ea33cae6859eeb761b8cac364c2/jupyter_console-6.4.3-py3-none-any.whl#sha256=e630bcb682c0088dda45688ad7c2424d4a825c8acf494cb036ced03ed0424841
Best match: jupyter-console 6.4.3
Processing jupyter_console-6.4.3-py3-none-any.whl
Installing jupyter_console-6.4.3-py3-none-any.whl to /home/**/miniconda3/lib/python3.9/site-packages
Adding jupyter-console 6.4.3 to easy-install.pth file
Installing jupyter-console script to /home/**/miniconda3/bin

Installed /home/**/miniconda3/lib/python3.9/site-packages/jupyter_console-6.4.3-py3.9.egg
error: jupyter-client 6.1.11 is installed but jupyter-client>=7.0.0 is required by {'jupyter-console'}

After I disabled jupyter-client version requirement, there is another similar issue occured. I also disable nbformat version requirement, this installation succeded.

Installed /home/**/miniconda3/lib/python3.9/site-packages/d2lbook-0.1.17-py3.9.egg
Processing dependencies for d2lbook==0.1.17
error: nbformat 5.0.7 is installed but nbformat>=5.2.0 is required by {'jupyter-server'}

It seems this project could not work normally now. Is there anyone know which version of jupyter-console and jupyter-server is needed?

Multiple modules including sphinx.py, config.py, and build.py do not explicitly set the encoding='utf-8' option for Python open() function. As a result, the default encoding is gbk on Windows, but the file encoding is utf-8, thus resulting in

UnicodeDecodeError: 'gbk' codec can't decode byte 0x80 in position 884

In addition, the build script uses Linux shell commands such as, rm, cp, etc.
As there are no equivalent commands in Windows shell, the script cannot be run on Windows.

Future updates might address the aforementioned issues in order to support d2lbook to run on Windows.

• APA style bib with natbib enabled
• Support runtime
  • colab,
  • mybinder
  • ThebeLab
  • sagemaker
• Use AWS Batch to launch jobs
• Click to copy for code blocks
• multi-tabs
  • multitab code
  • multitab text block
  • make the whole doc a tab only
  • activate a particular tab for editing
  • support @tab all
  • save code into different module
  • switch to the corresponding resource when click a particular tab:
    • pdf
    • zip (all tabs in a single zip)
    • forum
    • colab
  • track the clicks of a particular tab
  • embed forum threads
  • fix the tab position after clicking
• refer to apis in other project by :class:xx and :func:yy
• --s3 arg in d2lbook deploy
• use multi-processes for parallel evaluation, needs to mark the gpu usage in a notebook
• unify release and version in config.ini
• use multi-workers to execute, be smart on request GPUs
• a new task translate
• automatically correct a {.python .input .toc} block into a toc block.

Cost me 5 days to figure out it.
Share with anyone who need.

pip install git+https://github.com/aieye-top/d2l-book2

The part changed:

You can check here to get the result: https://aieye-top.github.io/d2cl/

If you can give a option to choose /root or /docs/ , it will be great.

加个功能，可以让代码存到指定的文件夹。使得代码结构可以更加复杂，同时可以提前解释。
Add a feature that allows code to be saved in a specified folder.Enables more complex code structure, and you can explain it in advance.

Hey D2L team,

Thanks for providing us with this great tool!

One small question or feature request: Is it possible to support the functionalities for theorems/proof blocks, e.g., something like this: https://sphinx-proof.readthedocs.io/en/latest/syntax.html?
Maybe this requires defining new syntax in Markdown (e.g., custom tokens for start/end of environments) and translate that into rst.

Thanks!