inveniosoftware / training Goto Github PK

Invenio v3 Training Material

Home Page: https://training.readthedocs.io

License: MIT License

Python 83.37% Shell 2.77% HTML 11.21% SCSS 2.65%

training's Introduction

Invenio v3 Training Material

Warning This repository is unmaintained and code and dependencies are outdated and not working. However, tutorials are useful to learn how to develop on Invenio. Please refer to the InvenioRDM documentation for Invenio documentation.

This training material takes you through a series of tutorials that will help you get started with developing repositories with Invenio.

If training fails, there are many chances it is not your fault, take it as a challange to learn, contribute to the team (by fixing it) and ask for help to your new and very nice colleagues.

Prerequisites

Before starting on the tutorials, make sure that you have properly setup your environment.

Prerequisites

Tutorials

Invenio at a glance

Starting development with Invenio

Learning about data models

Invenio in production

Advanced topics

Tutorial 15 - Application architecture

training's People

Contributors

Stargazers

Watchers

training's Issues

Tutorial 5: Add information on how to add a menu item

WIth Invenio-Theme and Flask-Menu (might need to go to a more advanced section)

Data model: create fields and data model that are real world use cases

Author is too similar to contributors. We should come up with a new data model the it actually make sense in the real world.
Change all examples around from "fake" to real use cases.

Invalid virtualenv situation when changing python executable

I'm reporting something that break my envirornment while following the first training sessions.

Basically, in session 3 when you first run the commands to bring up the full docker-compose set up, it asks you to generate a pipfile. If you change your Python executable after having generated this pipfile lock (in my case, 3.8 -> 3.6 happened while trying to fix some other issues that popped out later) this generates a broken situation.

Maybe looking for the pipfile lock and running pipenv lock at the start of build-images.sh can avoid this kind of situations

./start-from.sh doesn't apply chosen state

xxx@xxx:src $ ls -l
total 8
drwxr-xr-x 31 xxx staff 992 5 Sep 11:40 my-site
drwxr-xr-x 8 xxx staff 256 5 Sep 10:32 my_site
-rw-r--r-- 1 xxx staff 2737 5 Sep 11:17 setup.py
drwxr-xr-x 26 xxx staff 832 4 Sep 14:23 training

After applying ./start-from.sh 09-deposit-form I have both my-site and my_site in src folder and the application doesn't work properly.

After studying ./start-from.sh I noticed an error at line 38:
replace

cp -R "$invenio_training_folder/$exercise_tutorial_folder/solution/my-site/" "$invenio_src_folder"

cp -R "$invenio_training_folder/$exercise_tutorial_folder/solution/my-site/" "$invenio_instance_folder"

and it'll work fine

tutorial-07: search using "q=owner:1" to demonstrate mappings/search

v3.2 tutorials walk through

tutorial-07: bootstrap fix

Bootstrap exercise should be an optional step (only if you messed up the previous step.
Commands should not assume a starting directory: cd 03-add-new-field-> cd ~/src/training/03-add-new-field/ (otherwise users cannot just copy it in the terminal)

tutorial:04/05: include info about configuration

The configuration flow needs to be explained somewhere somehow :-)

I would think perhaps at the end of 04? or beginning of customize look and feel

tutorial 1: "incompatible versions in resolved dependencies" running bootstrap script

Hello,

In both the Vagrant setup and the OVA box, I'm unable to lock my dependencies when running
$ ./scripts/bootstrap. There seems to be an issue with invenio-base ...?

The full response is reproduced below:

bootcamp@invenio:~/src/my-site$ ./scripts/bootstrap
'Pipfile.lock' not found. Generating via 'pipenv lock --dev'...
Creating a virtualenv for this project…
Pipfile: /home/bootcamp/src/my-site/Pipfile
Using /usr/bin/python3.6m (3.6.7) to create virtualenv…
⠹ Creating virtual environment...Using base prefix '/usr'
New python executable in /home/bootcamp/.local/share/virtualenvs/my-site-7Oi5HgLM/bin/python3.6m
Also creating executable in /home/bootcamp/.local/share/virtualenvs/my-site-7Oi5HgLM/bin/python
Installing setuptools, pip, wheel...
done.
Running virtualenv with interpreter /usr/bin/python3.6m

✔ Successfully created virtual environment! 
Virtualenv location: /home/bootcamp/.local/share/virtualenvs/my-site-7Oi5HgLM
Locking [dev-packages] dependencies…
✔ Success! 
Locking [packages] dependencies…
✘ Locking Failed! 
[pipenv.exceptions.ResolutionFailure]:       req_dir=requirements_dir
[pipenv.exceptions.ResolutionFailure]:   File "/home/bootcamp/.local/lib/python3.6/site-packages/pipenv/utils.py", line 726, in resolve_deps
[pipenv.exceptions.ResolutionFailure]:       req_dir=req_dir,
[pipenv.exceptions.ResolutionFailure]:   File "/home/bootcamp/.local/lib/python3.6/site-packages/pipenv/utils.py", line 480, in actually_resolve_deps
[pipenv.exceptions.ResolutionFailure]:       resolved_tree = resolver.resolve()
[pipenv.exceptions.ResolutionFailure]:   File "/home/bootcamp/.local/lib/python3.6/site-packages/pipenv/utils.py", line 395, in resolve
[pipenv.exceptions.ResolutionFailure]:       raise ResolutionFailure(message=str(e))
[pipenv.exceptions.ResolutionFailure]:       pipenv.exceptions.ResolutionFailure: ERROR: ERROR: Could not find a version that matches invenio-base<1.1.0,>=1.0.0,>=1.0.2,>=1.2.2
[pipenv.exceptions.ResolutionFailure]:       Tried: 0.1.0, 0.1.0, 0.2.0, 0.2.0, 0.2.1, 0.2.1, 0.3.0, 0.3.0, 0.3.1, 0.3.1, 1.0.0, 1.0.0, 1.0.1, 1.0.1, 1.0.2, 1.0.2, 1.1.0, 1.1.0, 1.2.0, 1.2.0, 1.2.1, 1.2.1, 1.2.2, 1.2.2, 1.2.3, 1.2.3
[pipenv.exceptions.ResolutionFailure]:       Skipped pre-versions: 1.0.0a1, 1.0.0a1, 1.0.0a2, 1.0.0a2, 1.0.0a3, 1.0.0a3, 1.0.0a4, 1.0.0a4, 1.0.0a5, 1.0.0a5, 1.0.0a6, 1.0.0a7, 1.0.0a7, 1.0.0a8, 1.0.0a8, 1.0.0a9, 1.0.0a9, 1.0.0a10, 1.0.0a10, 1.0.0a11, 1.0.0a12, 1.0.0a12, 1.0.0a13, 1.0.0a13, 1.0.0a14, 1.0.0a14, 1.0.0a15, 1.0.0a15, 1.0.0a16, 1.0.0a16, 1.0.0b1, 1.0.0b1
[pipenv.exceptions.ResolutionFailure]: Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
  First try clearing your dependency cache with $ pipenv lock --clear, then try the original command again.
 Alternatively, you can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
  Hint: try $ pipenv lock --pre if it is a pre-release dependency.
ERROR: ERROR: Could not find a version that matches invenio-base<1.1.0,>=1.0.0,>=1.0.2,>=1.2.2
Tried: 0.1.0, 0.1.0, 0.2.0, 0.2.0, 0.2.1, 0.2.1, 0.3.0, 0.3.0, 0.3.1, 0.3.1, 1.0.0, 1.0.0, 1.0.1, 1.0.1, 1.0.2, 1.0.2, 1.1.0, 1.1.0, 1.2.0, 1.2.0, 1.2.1, 1.2.1, 1.2.2, 1.2.2, 1.2.3, 1.2.3
Skipped pre-versions: 1.0.0a1, 1.0.0a1, 1.0.0a2, 1.0.0a2, 1.0.0a3, 1.0.0a3, 1.0.0a4, 1.0.0a4, 1.0.0a5, 1.0.0a5, 1.0.0a6, 1.0.0a7, 1.0.0a7, 1.0.0a8, 1.0.0a8, 1.0.0a9, 1.0.0a9, 1.0.0a10, 1.0.0a10, 1.0.0a11, 1.0.0a12, 1.0.0a12, 1.0.0a13, 1.0.0a13, 1.0.0a14, 1.0.0a14, 1.0.0a15, 1.0.0a15, 1.0.0a16, 1.0.0a16, 1.0.0b1, 1.0.0b1
There are incompatible versions in the resolved dependencies.
[pipenv.exceptions.ResolutionFailure]:   File "/home/bootcamp/.local/lib/python3.6/site-packages/pipenv/utils.py", line 726, in resolve_deps
[pipenv.exceptions.ResolutionFailure]:       req_dir=req_dir,
[pipenv.exceptions.ResolutionFailure]:   File "/home/bootcamp/.local/lib/python3.6/site-packages/pipenv/utils.py", line 480, in actually_resolve_deps
[pipenv.exceptions.ResolutionFailure]:       resolved_tree = resolver.resolve()
[pipenv.exceptions.ResolutionFailure]:   File "/home/bootcamp/.local/lib/python3.6/site-packages/pipenv/utils.py", line 395, in resolve
[pipenv.exceptions.ResolutionFailure]:       raise ResolutionFailure(message=str(e))
[pipenv.exceptions.ResolutionFailure]:       pipenv.exceptions.ResolutionFailure: ERROR: ERROR: Could not find a version that matches invenio-base<1.1.0,>=1.0.0,>=1.0.2,>=1.2.2
[pipenv.exceptions.ResolutionFailure]:       Tried: 0.1.0, 0.1.0, 0.2.0, 0.2.0, 0.2.1, 0.2.1, 0.3.0, 0.3.0, 0.3.1, 0.3.1, 1.0.0, 1.0.0, 1.0.1, 1.0.1, 1.0.2, 1.0.2, 1.1.0, 1.1.0, 1.2.0, 1.2.0, 1.2.1, 1.2.1, 1.2.2, 1.2.2, 1.2.3, 1.2.3
[pipenv.exceptions.ResolutionFailure]:       Skipped pre-versions: 1.0.0a1, 1.0.0a1, 1.0.0a2, 1.0.0a2, 1.0.0a3, 1.0.0a3, 1.0.0a4, 1.0.0a4, 1.0.0a5, 1.0.0a5, 1.0.0a6, 1.0.0a7, 1.0.0a7, 1.0.0a8, 1.0.0a8, 1.0.0a9, 1.0.0a9, 1.0.0a10, 1.0.0a10, 1.0.0a11, 1.0.0a12, 1.0.0a12, 1.0.0a13, 1.0.0a13, 1.0.0a14, 1.0.0a14, 1.0.0a15, 1.0.0a15, 1.0.0a16, 1.0.0a16, 1.0.0b1, 1.0.0b1
[pipenv.exceptions.ResolutionFailure]: Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
  First try clearing your dependency cache with $ pipenv lock --clear, then try the original command again.
 Alternatively, you can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
  Hint: try $ pipenv lock --pre if it is a pre-release dependency.
ERROR: ERROR: Could not find a version that matches invenio-base<1.1.0,>=1.0.0,>=1.0.2,>=1.2.2
Tried: 0.1.0, 0.1.0, 0.2.0, 0.2.0, 0.2.1, 0.2.1, 0.3.0, 0.3.0, 0.3.1, 0.3.1, 1.0.0, 1.0.0, 1.0.1, 1.0.1, 1.0.2, 1.0.2, 1.1.0, 1.1.0, 1.2.0, 1.2.0, 1.2.1, 1.2.1, 1.2.2, 1.2.2, 1.2.3, 1.2.3
Skipped pre-versions: 1.0.0a1, 1.0.0a1, 1.0.0a2, 1.0.0a2, 1.0.0a3, 1.0.0a3, 1.0.0a4, 1.0.0a4, 1.0.0a5, 1.0.0a5, 1.0.0a6, 1.0.0a7, 1.0.0a7, 1.0.0a8, 1.0.0a8, 1.0.0a9, 1.0.0a9, 1.0.0a10, 1.0.0a10, 1.0.0a11, 1.0.0a12, 1.0.0a12, 1.0.0a13, 1.0.0a13, 1.0.0a14, 1.0.0a14, 1.0.0a15, 1.0.0a15, 1.0.0a16, 1.0.0a16, 1.0.0b1, 1.0.0b1
There are incompatible versions in the resolved dependencies.

Any advice for how to proceed? Happy to share more info if useful.

tutorial-03: infrastructure overview

I would include the image from documentation, to give a quick overview of the infrastructure:

tutorial-07: should "owner" be a required field?

tutorial-06: show how to run an individual test

pytest tests/tests_file.py::test_name

Training 9

https://github.com/inveniosoftware/training/tree/v3.4/09-deposit-form
On step 6 after when trying to create a record, the layout looks unformatted:

update pre-requisites

e.g. node 14 (currently saying 8 or 10 https://github.com/inveniosoftware/training/tree/master/00-prerequisites)

Tell a story!

We have already a good base. What about if now we chain all tutorial as if we have to develop a new product add we add use case by use case? Like this, it will be like a story.

We now have a new use case:...
Our users would like to have a new feature: they would like to search by author.

tutorial-08: bootstrap script not working

$ ./bootstrap.sh 
cp: failed to access '../my-site/my_site': Not a directory

copy should go to ~/src/my-site/

tutorial 11: no such command 'records'.

Problem
Tutorial 11 relies on creating records with references using the invenio records CLI. However, Invenio-Records CLI has been remove in version 1.3.0.

Solution
Modify tutorial to enable the creation of linked records through the REST API.

tutorial-02: show query string syntax

how to query a specific field

tutorial-07/08: use "Step N" format for sections

Having links to reference documentation

If manageable on the long term, it would be nice to have some pointers to the reference documentation in the trainings material.
Exemple: tutorial-05 about Data model could have some pointers to the pages under Understanding data models

tutorial-07: ES mapping for owner: keyword -> integer

tutorial-02: include oai-pmh server

tutorial-10: before_record_index signature changed

As of version 1.0.2 version of invenio-indexer the signature of the before_record_index has changed and adds an arguments attribute.

Errors spotted by @carantunes. Will provide a PR soon :)

00-Prerequisites Local Setup: Node/NPM version needs to be <16.10.0-1?

I was unable to install the required npm modules via ./scripts/bootstrap (or invenio webpack clean buildall) using nodejs 16.10.0-1, but succeeded using nodejs 14.18.0 and 12.22.6.

Installing with nodejs 16.10.0-1 results in:

npm ERR! code ERESOLVE
npm ERR! ERESOLVE unable to resolve dependency tree
npm ERR!
npm ERR! While resolving: [email protected]
npm ERR! Found: [email protected]
npm ERR! node_modules/react-overridable
npm ERR!   react-overridable@"^0.0.2" from the root project
npm ERR!
npm ERR! Could not resolve dependency:
npm ERR! peer react-overridable@"^0.0.3" from [email protected]
npm ERR! node_modules/react-searchkit
npm ERR!   react-searchkit@"^1.0.0-alpha.12" from the root project
npm ERR!
npm ERR! Fix the upstream dependency conflict, or retry
npm ERR! this command with --force, or --legacy-peer-deps
npm ERR! to accept an incorrect (and potentially broken) dependency resolution.
npm ERR!
npm ERR! See /home/USER/.npm/eresolve-report.txt for a full report.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/USER/.npm/_logs/2021-09-28T14_48_57_934Z-debug.log
Traceback (most recent call last):
  File "/path/to/my/venv/bin/invenio", line 8, in <module>
    sys.exit(cli())
  File "/path/to/my/venv/lib/python3.9/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/path/to/my/venv/lib/python3.9/site-packages/flask/cli.py", line 586, in main
    return super(FlaskGroup, self).main(*args, **kwargs)
  File "/path/to/my/venv/lib/python3.9/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/path/to/my/venv/lib/python3.9/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/path/to/my/venv/lib/python3.9/site-packages/click/core.py", line 1289, in invoke
    rv.append(sub_ctx.command.invoke(sub_ctx))
  File "/path/to/my/venv/lib/python3.9/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/path/to/my/venv/lib/python3.9/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/path/to/my/venv/lib/python3.9/site-packages/click/decorators.py", line 21, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/path/to/my/venv/lib/python3.9/site-packages/flask/cli.py", line 426, in decorator
    return __ctx.invoke(f, *args, **kwargs)
  File "/path/to/my/venv/lib/python3.9/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/path/to/my/venv/lib/python3.9/site-packages/flask_webpackext/cli.py", line 72, in buildall
    current_webpack.project.buildall()
  File "/path/to/my/venv/lib/python3.9/site-packages/pywebpack/project.py", line 141, in buildall
    super(WebpackTemplateProject, self).buildall()
  File "/path/to/my/venv/lib/python3.9/site-packages/pywebpack/project.py", line 69, in buildall
    self.install()
  File "/path/to/my/venv/lib/python3.9/site-packages/pywebpack/helpers.py", line 48, in inner
    raise RuntimeError("Process exited with code {}".format(exit_code))
RuntimeError: Process exited with code 1

tutorial-07: include the commands to run more prominently

e.g. ./script/server /setup are very hidden

add warning not to use the sqlite on prod or in docker-compose.full.yml

For starters here:
https://github.com/inveniosoftware/training/tree/v3.1/14-deployment-monitoring/
review if there is need for adding somewhere in the main invenio docs
should be highlighted that it is not recomended

build-images.sh fails at the Invenio Previewer step

Environment

I'm trying with a clean Vagrant VM, untouched situation after vagrant up and vagrant ssh. No errors are reported during the VM setting up phase.

Steps to Reproduce

While trying to follow the third tutorial, I run ./docker/build-images.sh and then it stops

Expected behavior

Logs

Context

vagrant@ubuntu-bionic:~/src/my-site$ ./docker/build-images.sh                                                                                                                                                                                
Sending build context to Docker daemon  317.4kB                                                                                                                                                                                              
Step 1/3 : FROM inveniosoftware/centos7-python:3.6                                                                                                                                                                                           
 ---> d207351414dd                                                                                                                                                                                                                           
Step 2/3 : COPY Pipfile Pipfile.lock ./                                                                                                                                                                                                      
 ---> Using cache                                                                                                                                                                                                                            
 ---> ab621130c632                                                                                                                                                                                                                           
Step 3/3 : RUN pipenv install --deploy --system                                                                                                                                                                                              
 ---> Using cache                                                                                                                                                                                                                            
 ---> 6afee51bf358                                                                                                                                                                                                                           
Successfully built 6afee51bf358                                                                                                                                                                                                              
Successfully tagged my-site-base:ba3d35704f4387fbdbe6d0db684a81e57a265493308d80d2caba4152e44e7f3e                                                                                                                                            
Sending build context to Docker daemon    319kB                                                                                                                                                                                              
Step 1/6 : ARG DEPENDENCIES_VERSION=latest                                                                                                                                                                                                   
Step 2/6 : FROM my-site-base:${DEPENDENCIES_VERSION}                                                                                                                                                                                         
 ---> 6afee51bf358                                                                                                                                                                                                                           
Step 3/6 : COPY ./ .                                                                                                                                                                                                                         
 ---> Using cache                                                                                                                                                                                                                            
 ---> 0b52f10dd51f                                                                                                                                                                                                                           
Step 4/6 : COPY ./docker/uwsgi/ ${INVENIO_INSTANCE_PATH}                                                                                                                                                                                     
 ---> Using cache                                                                                                                                                                                                                            
 ---> 199838a1de89                                                                                                                                                                                                                           
Step 5/6 : RUN pip install . &&     invenio collect -v  &&     invenio webpack create &&     invenio webpack install --unsafe &&     invenio webpack build                                                                                   
 ---> Running in f882cd7567e6                                                                                                                                                                                                                
Processing /opt/invenio/src                                                                                                                                                                                                                  
Building wheels for collected packages: my-site                                                                                                                                                                                              
  Building wheel for my-site (setup.py): started                                                                                                                                                                                             
  Building wheel for my-site (setup.py): finished with status 'done'                                                                                                                                                                         
  Created wheel for my-site: filename=my_site-1.0.0-py2.py3-none-any.whl size=24312 sha256=a1e9ba8099b254715d151645822816bb6e39fde62249c9961aefe615d5b5a136                                                                                  
  Stored in directory: /tmp/pip-ephem-wheel-cache-ujejvfd4/wheels/cd/64/dc/8d45ea69b4d5772d61f1bbee2a57f91b275778dcf302a6f83a                                                                                                                
Successfully built my-site                                                                                                                                                                                                                   
Installing collected packages: my-site                                                                                                                                                                                                       
Successfully installed my-site-1.0.0

Error

The above exception was the direct cause of the following exception:
Successfully installed my-site-1.0.0
WARNING: You are using pip version 20.1.1; however, version 20.2.3 is available.                                                                                                                                 
You should consider upgrading via the '/opt/rh/rh-python36/root/usr/bin/python3 -m pip install --upgrade pip' command.
[2020-09-15 09:00:58,354] ERROR in app: Failed to initialize entry point: invenio_previewer = invenio_previewer:InvenioPreviewer
Traceback (most recent call last):
  File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2751, in requires
    deps.extend(dm[safe_extra(ext)])
KeyError: 'execute'
Traceback (most recent call last):
  File "/opt/rh/rh-python36/root/usr/bin/invenio", line 8, in <module>
    sys.exit(cli())
  File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/flask/cli.py", line 586, in main
    return super(FlaskGroup, self).main(*args, **kwargs)
  File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/click/decorators.py", line 21, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/flask/cli.py", line 425, in decorator
    with __ctx.ensure_object(ScriptInfo).load_app().app_context():
  File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/flask/cli.py", line 381, in load_app
    app = call_factory(self, self.create_app)
  File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/flask/cli.py", line 117, in call_factory
    return app_factory(script_info)
  File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/invenio_base/app.py", line 154, in create_cli_app
    app = create_app(debug=get_debug_flag())
  File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/invenio_base/app.py", line 113, in _create_app
    modules=extensions,
  File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/invenio_base/app.py", line 174, in app_loader
    modules=modules)
  File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/invenio_base/app.py", line 233, in _loader
    init_func(ep.load())
  File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2461, in load
    self.require(*args, **kwargs)
  File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2484, in require
    items = working_set.resolve(reqs, env, installer, extras=self.extras)
  File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 794, in resolve
    new_requirements = dist.requires(req.extras)[::-1]
  File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2755, in requires
    ) from e
pkg_resources.UnknownExtra: nbconvert 6.0.2 has no such extra feature 'execute'

Tutorials review

Review all tutorials to ensure they work as intended with Invenio v3.2

tutorial-06: reorganize tutorial with steps

The README doesn't follow the same structure as the previous tutorials. I would expect a couple of steps with a bit of text and images. I think most of it can just be taken from the presentation.

Step 1: Inspect project structure
Step 2: Initialize the git repository
Step 3: Build the documentation
Step 4: Running tests
Step 5: End-to-end testing
Step 6: Fix the failing test
Step 7: Installing a new Python dependency

tutorial-06: include instructions for how to install the Gecko driver

https://github.com/inveniosoftware/training/blob/master/00-prerequisites/bootstrap.sh#L46-L55

start-from.sh: creating my-site subfolder inside my-site

The start-from.sh copies the folder my-site itself and the content of it. As a result, you have ./my-site/my-site/my_site/....

The problem is in the following line:

cp -R "$invenio_training_folder/$exercise_tutorial_folder/solution/my-site/" "$invenio_instance_folder"

A possible solution is to change line with:

cp -R "$invenio_training_folder/$exercise_tutorial_folder/solution/my-site/" "$invenio_src_folder"

Link in the "About" section of the repository is broken

The link mentioned in the About section in the main repository page is leading to training.readthedocs.io, a broken link

Paths to file before each code snippet

Some examples are providing path like my-site/my_site/ while others are starting directly with my_site/. It would be convenient to have everything aligned (maybe assuming people are in my-site so that all commands like pipenv can be run without changing path.

Default vagrant box size of 10GB is not enough

Describe the bug

The Vagrant configuration described in the provided Vagrantfile does not specify the disk size, thus leaving the choice to the virtualisation provider. In case of VirtualBox, the default disk size, when not specified, is 10 GB.

When trying to run docker-compose with the full infrastructure setup, it stops at the kibana steps because it runs out of disk space.

vagrant@ubuntu-bionic:~/src/my-site$ docker-compose -f docker-compose.full.yml up -d    
Pulling kibana (docker.elastic.co/kibana/kibana-oss:7.2.0)...
7.2.0: Pulling from kibana/kibana-oss        
48914619bcd3: Already exists                                                                            
f2bc79fc77ab: Pull complete                                                                             
1ebe2db78ce5: Extracting [==================================================>]  97.65MB/97.65MB
f1e2b4f43bdd: Download complete
b0ef6d560815: Download complete                                                                         
7d6d6c5a5bfe: Download complete          
2492572a3fe4: Download complete                 
5ee5d10f355d: Download complete            
ERROR: failed to register layer: Error processing tar file(exit status 1): write /usr/share/kibana/src/legacy/ui/public/chrome/api/angular.js: no space left on device

Note that resizing the partition / Guest OS disk after the creation is not trivial

Steps to Reproduce

Follow the instructions for Vagrant until part 03 of the tutorial.

New tutorial how to add webassets (css/js) webpack and bundles

tutorial-07: use full path from root of instance

many paths are missing: my_site/records/

tutorial-04: show ``invenio instance entrypoints``

The command invenio instance entrypoints is useful for new developers to understand which entry points have been registered.

tutorial-07: adjust title to current numbering

Training 14 and docker

https://github.com/inveniosoftware/training/tree/v3.4/14-deployment-monitoring

Tutorial 11: cmd line create record is deprecated

Since it is deprecated, we need to change the tutorial and have another way to create records with $ref. Maybe implement the example at the end of the tutorial overriding the create method?

New Tutorial: Managing files

tutorial-04: execute a background task (e.g register a user - send email)

tutorial-08: simplify PID example

Simplify the author data mode example with uuid as PIDs.

Fix style consistency

The training contain many consistency issues in term of styling. Many of them have been address by @topless in the PRs mentioned here #81. Note that the issue has been closed since the changes of those PRs have been partially merged as part of #82. Nonetheless, the PRs are left open due to the extra changes.

Some of the issues:

$ at the begginng of the commands. This might cause errors when copying.
The type of codeblock used along the trainings varies betweenbash, commandline (e.g. tutorial 6) and console (e.g. tutorial 13). However, it is not clear when to use which.
Python 2 __future__ imports.
The records mappings' folder present in the solutions is not a python module. It has missing __init__.py files.
Using * vs - for enumerations. Even though at display time there is no difference.

reference markdown lint rules

tutorial-07: typos/corrections

-it will be an integer that represents
+it will be an integer (a User ID) that represents
---
# ...in a couple of places
-.../record-v10.0.json
+.../record-v1.0.0.json
---
-Save somewhere the id value of this value
+Save somewhere the id value of this response

tutorial-03: uwsgitop not installed

Step 7 includes the use of uwsgitop but it doesn't seem to be installed in the docker image.

Bring up the full docker-compose setup

When running ./docker/build-images.sh it shows #8 220.3 ERROR in ./js/invenio_theme/templates.js #8 220.3 Module not found: Error: Can't resolve '@templates' in '/opt/invenio/var/instance/assets/js/invenio_theme'

Tutorial 07 requires jsonschema 3.2.0

Package version (if known):

Describe the bug

Step 5 in tutorial 07 fails if the latest version of jsonschema is installed (see stacktrace below). This can be fixed by downgrading to jsonschema 3.2.0 (by running pip install --upgrade jsonschema==3.2.0 in the virtual environment), avoiding the API change introduced in version 4 of jsonschema.

Steps to Reproduce

Follow tutorial 07
Observe TypeError

Expected behavior

Successfully complete step 5 of the tutorial.

Screenshots (if applicable)

Additional context

Stacktrace:

127.0.0.1 - - [30/Sep/2021 15:00:38] "POST /api/records/?prettyprint=1 HTTP/1.1" 500 -
Traceback (most recent call last):
  File "/path/to/my/venv/lib/python3.9/site-packages/flask/app.py", line 2464, in __call__
    return self.wsgi_app(environ, start_response)
  File "/path/to/my/venv/lib/python3.9/site-packages/werkzeug/middleware/dispatcher.py", line 66, in __call__
    return app(environ, start_response)
  File "/path/to/my/venv/lib/python3.9/site-packages/flask/app.py", line 2464, in __call__
    return self.wsgi_app(environ, start_response)
  File "/path/to/my/venv/lib/python3.9/site-packages/flask/app.py", line 2450, in wsgi_app
    response = self.handle_exception(e)
  File "/path/to/my/venv/lib/python3.9/site-packages/flask_restful/__init__.py", line 271, in error_router
    return original_handler(e)
  File "/path/to/my/venv/lib/python3.9/site-packages/flask/app.py", line 1867, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/path/to/my/venv/lib/python3.9/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/path/to/my/venv/lib/python3.9/site-packages/flask/app.py", line 2447, in wsgi_app
    response = self.full_dispatch_request()
  File "/path/to/my/venv/lib/python3.9/site-packages/flask/app.py", line 1952, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/path/to/my/venv/lib/python3.9/site-packages/flask_restful/__init__.py", line 271, in error_router
    return original_handler(e)
  File "/path/to/my/venv/lib/python3.9/site-packages/flask/app.py", line 1821, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/path/to/my/venv/lib/python3.9/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/path/to/my/venv/lib/python3.9/site-packages/flask/app.py", line 1950, in full_dispatch_request
    rv = self.dispatch_request()
  File "/path/to/my/venv/lib/python3.9/site-packages/flask/app.py", line 1936, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/path/to/my/venv/lib/python3.9/site-packages/flask/views.py", line 89, in view
    return self.dispatch_request(*args, **kwargs)
  File "/path/to/my/venv/lib/python3.9/site-packages/invenio_rest/views.py", line 239, in dispatch_request
    result = super(ContentNegotiatedMethodView, self).dispatch_request(
  File "/path/to/my/venv/lib/python3.9/site-packages/flask/views.py", line 163, in dispatch_request
    return meth(*args, **kwargs)
  File "/path/to/my/venv/lib/python3.9/site-packages/invenio_records_rest/views.py", line 428, in need_record_permission_decorator
    return f(self, record=record, *args, **kwargs)
  File "/path/to/my/venv/lib/python3.9/site-packages/invenio_records_rest/views.py", line 695, in post
    record = self.record_class.create(data, id_=record_uuid)
  File "/path/to/my/venv/lib/python3.9/site-packages/invenio_records_files/api.py", line 322, in create
    record = super(Record, cls).create(data, id_=id_, **kwargs)
  File "/path/to/my/venv/lib/python3.9/site-packages/invenio_records/api.py", line 303, in create
    record._validate(
  File "/path/to/my/venv/lib/python3.9/site-packages/invenio_records/api.py", line 183, in _validate
    _records_state.validate(
  File "/path/to/my/venv/lib/python3.9/site-packages/invenio_records/ext.py", line 35, in validate
    return validate(
  File "/path/to/my/venv/lib/python3.9/site-packages/jsonschema/validators.py", line 964, in validate
    validator = cls(schema, *args, **kwargs)
TypeError: __init__() got an unexpected keyword argument 'types'

JSON schema validation

While playing around with @hjhsalo, we noticed that the default records data model coming out of the invenio-cookiecutter-instance actually does not use the provided JSON schema for validation of incoming records.

We also noticed that the new Authors data model was using the JSON schema for validation. Thanks to @zzacharo for pointing out the hack responsible for this
https://github.com/inveniosoftware/training/blob/master/08-data-models-from-scratch/author_module/my_site/authors/api.py#L26

Since the introduction to data models in invenio starts with the JSON schema, I believe the natural expectation of a new user is that this JSON schema will be used for validation (otherwise why are we writing it?). I would suggest to enable validation based on the JSON schema for the records data model in invenio-cookiecutter-instance

Levarage usage of git throughout all the examples

I understand the goal is not to teach about git, but as from step 06 git was introduced it could be nice to have tutorials ask to commit after at least each parts, it could help getting back to a known state and also help understanding what was changed during a specific part of a tutorial or by a script/copy/whatever.

inveniosoftware / training Goto Github PK

training's Introduction

Invenio v3 Training Material

Prerequisites

Tutorials

Invenio at a glance

Starting development with Invenio

Learning about data models

Invenio in production

Advanced topics

training's People

Contributors

Stargazers

Watchers

Forkers

training's Issues

Environment

Steps to Reproduce

Expected behavior

Logs

Describe the bug

Steps to Reproduce

Describe the bug

Steps to Reproduce

Expected behavior

Screenshots (if applicable)

Additional context

Recommend Projects

Recommend Topics

Recommend Org