Create recent book version from scratch about scientific-visualization-book HOT 24 CLOSED

Ok, I will check tomorrow and tell you about result, thanks a lot!

from scientific-visualization-book.

Ok, thank you, then I will share my thoughts in PR and you will decide if it is correct or we need to change it in some way.

from scientific-visualization-book.

Hi @rougier,

I added information in the article about further builds (after initial configuration and the first execution), and PDF compression (I found that you have pdf-compressed option in Makefile and decided to include it for completeness).

Thank you.

from scientific-visualization-book.

Yes, you can compile the book from the repository but you'll need to install some supplementary packages. I'm using TeXLive that install pretty much everything. The ean13 error is package for generating barcodes.

from scientific-visualization-book.

Great, thank you!

from scientific-visualization-book.

Hi @rougier,

Sorry but I can't build the book even after the following steps (I am using Ubuntu 20.04):

• Clone repository
• Install TeX Live supported version (now it is version 2022) using the following links:
  • https://www.tug.org/texlive/acquire-netinstall.html
  • https://www.tug.org/texlive/quickinstall.html
• Install required system libraries for cartopy (with proj 9.0.0)
• Create and activate Python 3.7.4 environment
• Install the following Python libraries:

pip install numpy scipy matplotlib docutils scikit-image tqdm shapely cartopy python-dateutil GitPython glm imageio mpmath noise

Version of the libraries:

Cartopy==0.20.2
certifi==2022.5.18.1
cycler==0.11.0
docutils==0.18.1
fonttools==4.33.3
gitdb==4.0.9
GitPython==3.1.27
glm==0.4.3
imageio==2.19.3
kiwisolver==1.4.2
matplotlib==3.5.2
mpmath==1.2.1
networkx==2.6.3
noise==1.2.2
numpy==1.21.6
packaging==21.3
Pillow==9.1.1
pyparsing==3.0.9
pyproj==3.2.1
pyshp==2.3.0
python-dateutil==2.8.2
PyWavelets==1.3.0
scikit-image==0.19.2
scipy==1.7.3
Shapely==1.8.2
six==1.16.0
smmap==5.0.0
tifffile==2021.11.2
tqdm==4.64.0
typing_extensions==4.2.0

• Run the following commands from the project root:

make clean
make all

However I have the following error in the terminal:

mkdir -p pdf
Building tex/main.tex
Building tex/00-dedication.tex
Building tex/00-preface.tex
Building tex/00-introduction.tex
Building tex/00-acknowledgments.tex
Rc files read:
  NONE
Latexmk: Run number 1 of rule 'xelatex'
This is XeTeX, Version 3.141592653-2.6-0.999994 (TeX Live 2022) (preloaded format=xelatex)
 \write18 enabled.
entering extended mode
Latexmk: Getting log file 'book.log'
Latexmk: 'xelatex': source file 'book.bbl' doesn't exist. I'll try making it...
make[1]: Entering directory '/home/dlabazkin/education/scientific-visualization-book/tex'
make[1]: Leaving directory '/home/dlabazkin/education/scientific-visualization-book/tex'
Latexmk: Summary of warnings from last run of *latex:
  Latex failed to resolve 122 reference(s)
Collected error summary (may duplicate other messages):
  xelatex: Command for 'xelatex' gave return code 1
      Refer to 'book.log' for details

And tex/book.log file has the following lines with Error word:

cat tex/book.log | grep Error

! LaTeX Error: \begin{mdframed} on input line 367 ended by \end{DUadmonition}.
! LaTeX Error: \begin{mdframed} on input line 367 ended by \end{DUclass}.
! LaTeX Error: \begin{mdframed} on input line 1150 ended by \end{DUadmonition}.
! LaTeX Error: \begin{mdframed} on input line 1150 ended by \end{DUclass}.
! LaTeX Error: \begin{mdframed} on input line 1741 ended by \end{DUadmonition}.
! LaTeX Error: \begin{mdframed} on input line 1741 ended by \end{DUclass}.
! LaTeX Error: \begin{mdframed} on input line 1741 ended by \end{document}.

Also I attached full text of book.log - book.log

Could you please help with this problem?

If it is possible to solve and build successfully then I can create PR and prepare instructions how to create it for Ubuntu if you want.

Thank you.

from scientific-visualization-book.

Sorry, I'm very late in answer. Did you fix the problem?

from scientific-visualization-book.

Hi, no problem :)
Unfortunately no because I didn’t have any idea how to resolve it.
Could you please check which additional step do I need?
Thank you.

from scientific-visualization-book.

Yes, I'll look into it.

from scientific-visualization-book.

Which version of docututils do you have by the way ? I think I remember we had the problem on the private repo and I had to stick to a precise version. I'll search for the version number.

from scientific-visualization-book.

I can check the version and any other ideas tomorrow and will definitely answer to you.

from scientific-visualization-book.

It seems docutils 0.18 fails while docutils 0.17 is ok. Tell me if it is the problem.

from scientific-visualization-book.

With GitHub action, we have:

     run: sudo apt-get install --yes python && sudo pip install docutils==0.17 # docutils 0.18 breaks the build

from scientific-visualization-book.

Hi Nicolas,

I reproduced all the steps from scratch and using version 0.17 of docutils and few other steps I can generate the book successfully, thank you!

Below I am describing all the steps for Ubuntu 20.04 which I did to make a successful build.

1. Install TeX Live
   This guide was helpful to install it - https://www.tug.org/texlive/quickinstall.html
   Here I used the latest version of TeX Live (20220621)

cd ~/Downloads/
wget https://mirror.ctan.org/systems/texlive/tlnet/install-tl-unx.tar.gz
tar -xzvf install-tl-unx.tar.gz
cd install-tl-20220621/
sudo perl install-tl --no-interaction

# Add to ~/.bashrc
export PATH="$PATH:/usr/local/texlive/2022/bin/x86_64-linux"

source ~/.bashrc

# Check installation
which latex

2. Install Proj 9.0.0 for cartopy
   I used several steps from this answer on Stack Overflow - https://stackoverflow.com/a/72442088

sudo apt-get update
sudo apt-get install libgeos-dev cmake sqlite3 curl libcurl4-openssl-dev libtiff5-dev

cd ~/Downloads
wget https://download.osgeo.org/proj/proj-9.0.0.tar.gz
tar -xzvf proj-9.0.0.tar.gz
cd proj-9.0.0/
mkdir build && cd build
cmake ..
cmake --build .
sudo cmake --build . --target install 

3. Configure project & build the book
   For cartopy I used the latest version from GitHub because for some reason I had the errors with metadata for cartopy package from PyPI repository.

# Clone the repository
mkdir -p ~/education/sci-viz
cd ~/education/sci-viz
git clone https://github.com/rougier/scientific-visualization-book
cd scientific-visualization-book

# Create virtual environment
python -m venv ~/venvs/scientific-visualization-book
source ~/venvs/scientific-visualization-book/bin/activate
python -m pip install --upgrade pip setuptools wheel 
pip install numpy scipy matplotlib docutils==0.17 scikit-image tqdm shapely python-dateutil GitPython glm imageio mpmath noise
pip install git+https://github.com/SciTools/cartopy.git

# Fix the version of Python packages
pip freeze > requirements.txt

# Build the book
make clean
make all

Also I attached my requirements.txt file:
requirements.txt

Maybe all these Python packages are not required for building the book itself but can help resolve #33 which I used for creating the list with the packages.

Probably these instructions and this file can help other people to quickly build the latest version of the book.

from scientific-visualization-book.

Thank you very much for the detailed build instructions. Actually, you could put them in a how-to-build.txt and make a PR (if you want).

from scientific-visualization-book.

I think I can create PR this week, could you please tell which format of this file should be?
Maybe we can use Markdown file?
The versions of TeX Live and cartopy in the description above can change over time, should I fix it somehow?
I can also replace exact folder names for downloading and for the project with template names.

from scientific-visualization-book.

Perfect, thanks! Markdown would be fine. As for versions, maybe it's safer to freeze versions even though TeX Live might be more stable. Not sur what you mean with exact folder name but go ahead and we can discuss it on your PR.

from scientific-visualization-book.

Ok, I will create PR this week to discuss the details and propose possible description.
Maybe it will be useful to implement it along with issue #33?
Then it would be complete guide how to create the book and prepare to use all the code used in the book.
But then your help will be required to check if all required libraries are specified for reproducibility.

from scientific-visualization-book.

It would be great if you could answer about the scope of PR (only building the book or with code reproducibility but with your help) described above, so I can start prepare PR this Friday.
I updated previous message so maybe there were no additional notifications about it, therefore I decided to repeat the question.

from scientific-visualization-book.

Basically, what you wrote in the issue would be a very good start. For the requirements file, I wrote in the book which package I used we can use your file provided the book compile, including the various python scripts. Else, we would need to freeze versions to a lower version. Also, maybe we could use a dedicated virtual environment just for the book such as not to pollute user's environment.

from scientific-visualization-book.

Hi Nicolas,
I spent a lot of time to check all the steps and prepare Pull Request but it was failed.

A few comments about it:

• Previously I reproduced the steps using my working Ubuntu OS which is used for Python software development and some libraries were installed for it. Therefore now I used almost completely clean OS and slightly changed the list of system libraries.
• I used full TeX Live distribution which is pretty large (~7.6 GB) but another installation schemes (basic/small/medium) do not have all necessary packages so I decided to use full scheme to automate this step.
• I didn't find official method how to fix TeX Live version so I am using the latest version of the installer, but all the commands for installation can be applied to any version.

After all these steps I have the following error when building the book:

Biber error: [345] Utils.pm:409> ERROR - Error: Found biblatex control file version 3.9, expected version 3.8.

I don't know why this new error has happened but I didn't find how to resolve it.
I saw that biblatex was updated recently (22.06.2022) but I am not sure if this is the reason.

Because I didn't expect to spend so much time to prepare PR, I will just show you updated steps.

Install TeX Live
https://www.tug.org/texlive/quickinstall.html

cd ~/Downloads/
wget https://mirror.ctan.org/systems/texlive/tlnet/install-tl-unx.tar.gz
tar -xzvf install-tl-unx.tar.gz
cd install-tl-*/

export TEXLIVE_INSTALL_PREFIX=$HOME/texlive
export TEXLIVE_INSTALL_TEXDIR=$HOME/texlive/2022
perl install-tl --no-interaction

# Add to ~/.bashrc
echo -e '\nexport PATH="$PATH:$HOME/texlive/2022/bin/x86_64-linux"\n' >> ~/.bashrc
source ~/.bashrc

# Check installation
which latex

Create virtual environment

sudo apt-get -y install python3-pip python3-venv
python3 -m venv ~/venvs/scientific-visualization-book
source ~/venvs/scientific-visualization-book/bin/activate
python3 -m pip install --upgrade pip setuptools wheel 
pip install docutils==0.17

Clone the repository

sudo apt-get -y install git 
mkdir -p ~/education/sci-viz
cd ~/education/sci-viz
git clone https://github.com/rougier/scientific-visualization-book
cd scientific-visualization-book

Build the book

make clean
make all

from scientific-visualization-book.

I think that the problem is in repository which is used for installing packages.
As I understand by default the repository is chosen automatically based on internal logic and can be different for subsequent installation executions, so it is not deterministic and you need to find the repository without this problem and define it while installing packages.
I think that it was useful and I will stop here :)
Thank you!

from scientific-visualization-book.

Hi @rougier,
I wrote the article about the improved solution for the building of your book - https://labdmitriy.github.io/blog/building-scientific-visualization-book/

from scientific-visualization-book.

Many many thanks for all your effort in making the build straightforward and sorry for late answer. I also had issue with biber at some point and had to remove everything to compile it from scratch. I'll merge your PR. Again, many thanks.

from scientific-visualization-book.