Not working about isisdl HOT 43 CLOSED

Sorry to hear that. Could you provide me with the ~/isisdl/.errors/{timestamp}.txt file? The error you have posted does not reveal where the actual error happened.

Also, what filesystem are you running and do you have the lsblk binary installed? The first error indicates that the filesystem detection went wrong but this shouldn't lead to crashing.

from isisdl.

Hi!
Yes, here's the file.
And yes, I have lsblk. It was ran on a life system because the installed system is currently broken. So maybe it detected that. I ran the program from my home directory and it successfully created the directory isisdl and subdirectories, including the .errors subdirectory.
1708454970.txt

from isisdl.

Okay, that makes sense. I'll fix it for live environments in the future, as it shouldn't break things.

As for the crash you are getting: This should now be solved in the latest git commit on main. Install it with

pip install git+https://github.com/Emily3403/isisdl

and check if it still crashes.

from isisdl.

Thank you very much. I'll try as soon as I get back home.
By the way, an option to specify the path where to save it would also be nice. But it's not urgent , just for convenience, or does it download always to the current working directory? If that's the case, one night just run the program from within the desired path

from isisdl.

Yes there is an option to change the working directory for isisdl on Linux. To do so, modify the ~/.config/isisdl/config.yaml and change working_dir_location: /your/desired/path to the path your prefer.

Note that the ~ will only be expanded in the newest version (1.3.19) of isisdl.

from isisdl.

Sorry, am new to all this, so I'm still failing to install:

zorin@zorin:~$ pip install git+https://github.com/Emily3403/isisdl
Collecting git+https://github.com/Emily3403/isisdl
Cloning https://github.com/Emily3403/isisdl to /tmp/pip-req-build-1f9bjplt
Running command git clone -q https://github.com/Emily3403/isisdl /tmp/pip-req-build-1f9bjplt
Running command git submodule update --init --recursive -q
[email protected]: Permission denied (publickey).
fatal: Konnte nicht vom Remote-Repository lesen.

Bitte stellen Sie sicher, dass die korrekten Zugriffsberechtigungen bestehen
und das Repository existiert.
fatal: klonen von »[email protected]:Emily3403/isisdl.wiki.git« in Submodul-Pfad »/tmp/pip-req-build-1f9bjplt/isisdl.wiki« fehlgeschlagen
Fehler beim Klonen von 'isisdl.wiki'. Weiterer Versuch geplant
[email protected]: Permission denied (publickey).
fatal: Konnte nicht vom Remote-Repository lesen.

Bitte stellen Sie sicher, dass die korrekten Zugriffsberechtigungen bestehen
und das Repository existiert.
fatal: klonen von »[email protected]:Emily3403/tldr.git« in Submodul-Pfad »/tmp/pip-req-build-1f9bjplt/tldr« fehlgeschlagen
Fehler beim Klonen von 'tldr'. Weiterer Versuch geplant
[email protected]: Permission denied (publickey).
fatal: Konnte nicht vom Remote-Repository lesen.

Bitte stellen Sie sicher, dass die korrekten Zugriffsberechtigungen bestehen
und das Repository existiert.
fatal: klonen von »[email protected]:Emily3403/isisdl.wiki.git« in Submodul-Pfad »/tmp/pip-req-build-1f9bjplt/isisdl.wiki« fehlgeschlagen
Zweiter Versuch 'isisdl.wiki' zu klonen fehlgeschlagen, breche ab.
ERROR: Command errored out with exit status 1: git submodule update --init --recursive -q Check the logs for full command output.
zorin@zorin:~$

from isisdl.

Should I wait until it's available via pip install isisdl ?

from isisdl.

Seams like it wants to also clone the submodules. I should remove them. Can you instead only clone the isisdl repo and install it?

git clone https://github.com/Emily3403/isisdl
cd isisdl
pip install .

And I want to wait until the issue is fixed before releasing it on pip since I can't re-release the same version. So, if the issue is not yet fixed, I'll have to release 1.3.20, .21 and so on if the issue persists

from isisdl.

Thanks, yes, I could install it via git clone and pip install .
Ran it again, and the error is still there :(
1708602509.txt
errrr.txt

from isisdl.

Right, you'll have to throw away the database as there are entries that won't work. Simply delete the directory ~/isisdl and re-execute isisdl

from isisdl.

Ok, done that. Deleted an re-ran

errrr2.txt

1708603515.txt

from isisdl.

Okay, could you try pulling and running it again?

cd isisdl
git pull --rebase

from isisdl.

Had to clone again since I removed the isisdl directory .
It seems to be working now. It's currently building a request cache for files

from isisdl.

Perfect. Let it download all files and check if anything goes wrong ^^

from isisdl.

Great, it started downloading, so it is working now. Thank you very much!
I had to abort it, because for testing I was just running quickly without changing config , and it would have downloaded 81 GB (without videos) to my live systems which would have crashed it.
So I'm now changing config.yaml which by the way was empty except for the line that I would want to copy the example first. I would have thought that after initial run where I was asked the many config questions, those would have been saved to . config/isisdl/config.yaml but they weren't

from isisdl.

Ok, have set the new path in that config file. Can I just run isisdl again or will it be confused because of the previous aborted run that had already downloaded a tiny bit?
Also , after new stuff is added to Isis, i just run isisdl again and it'll just pick up what's new without having to redownload everything?

from isisdl.

And while we're at it: will it only download PDFs, doc, Excel, etc , and videos or also forum posts that sometime contain useful info too?

from isisdl.

Great, it started downloading, so it is working now. Thank you very much! I had to abort it, because for testing I was just running quickly without changing config , and it would have downloaded 81 GB (without videos) to my live systems which would have crashed it. So I'm now changing config.yaml which by the way was empty except for the line that I would want to copy the example first. I would have thought that after initial run where I was asked the many config questions, those would have been saved to . config/isisdl/config.yaml but they weren't

No, the configuration is saved in the database, the config file is only there for overwriting the config in the database (or setting in the settings.py).

Ok, have set the new path in that config file. Can I just run isisdl again or will it be confused because of the previous aborted run that had already downloaded a tiny bit?
Also , after new stuff is added to Isis, i just run isisdl again and it'll just pick up what's new without having to redownload everything?

In this current release, paths of the files are saved in the database, meaning you'll have to delete the database before re-downloading it. It should be possible to isisdl --sync the database to adapt to the new paths, but I'm not sure if that will work. Best delete it, if you haven't downloaded stuff.

And while we're at it: will it only download PDFs, doc, Excel, etc , and videos or also forum posts that sometime contain useful info too?

Forum attachments are currently not supported, but that is planned for the 2.0 release. Posts will not be downloaded though – only the files. For now, only files and urls on the ISIS page is supported

from isisdl.

Ok .where is the database that's to be deleted? I simply delete ~/isisdl which it created on it's own and where stuff was downloaded to? Yes, I guess so. Will do that now. Done. Error

from isisdl.

The database is currently located at ~/isisdl/.state.db, but deleting the entire directory should be fine. What error are you facing?

from isisdl.

Error the config file is malformed.
Reason : unrecognized key: 'video_size_discover_num_threads'
I did not touch that setting at all, so it might just be from an older version and is no longer supported? I guess I can just uncomment that line and be fine ? Or will it break things ?

from isisdl.

Yeah, apparently I haven't updated the example file for the new config. But it's best to only set the key working_dir_location and not the others (unless you want to force a specific setting), as reasonable defaults are used

from isisdl.

Ok, since I had simply copied the example file, I now uncommented everything but the working dir (for downloads) and the database dir (so that it persists after reboot of lice system) and the file system set to btrfs since it was not recognized.
Deleting everything else and restarting it now

from isisdl.

It aborted after 14 of 87 GB with sqlite3.OperationalError: database is locked.
Traceback lines 1286, -05, 452, all in request_helper.py, and line 108 in database_helper.py in add_pre_confainer.
Log:

1708610417.txt

from isisdl.

I guess I simply restart isisdl and it'll fix it itself ?
Maybe it was too much data coming in too fast for the external HD

from isisdl.

Yeah, I've only tested isisdl with SSDs and if there is much activity but slow writes, the database could get locked. This issue will probably also be fixed in the 2.0 update.

Simply restart it and let it run

from isisdl.

Restarted and it's running. Is it normal that the overall total size goes down? It now says 23 of 69 GB downloaded , now 68, and it used to be 8/eighty something

from isisdl.

39.9 out of 52 now, do it's gone down from 87 to 52 of total download size so far . Says "Done in: 3 days, [...]"
43882/43883 files
So it seems only one file is missing, and that file should then have 52-40=12 GB, but the only file currently downloading says 49.6 MB | 69.74 MB.

So something is odd with the size info

from isisdl.

Downloaded 40.00GB of 52.55 GB, "Done! Have a nice day"

from isisdl.

crontab -l and sudo crontab -l both say no entries, Eve though isisdl said would run hourly

from isisdl.

Restarted and it's running. Is it normal that the overall total size goes down? It now says 23 of 69 GB downloaded , now 68, and it used to be 8/eighty something

During the download, it removes the size from the total if the file is not downloadable. The Size is not too reliable though as files on ISIS are deleted after 2 years from moodle itself. But it should not be that big of a discrepancy. Is the course you are downloading older than 2 years?

crontab -l and sudo crontab -l both say no entries, Eve though isisdl said would run hourly

Yes, isisdl is not deployed via crontab but as a systemd user service. Check its status by executing systemctl --user status isisdl.service or systemctl --user status isisdl.timer.

You can list the pending timers by executing systemctl --user list-timers

from isisdl.

Ah , perfect .
Yes it's there. Thx.
And yes, also had some old courses there . Apparently 35GB got deleted (87 down to 52). But then out of those 52 it also said it downloaded only 40. But I take it the total is just an estimate.
Would be good to have a list of those that couldn't be downloaded, so that one could manually check for them. Sometimes it's deleted but the name is still there and if it's something that can be searched for on the Internet it might not be lost . Personal scripts of the profs are lost though.

Anyway, thank you for the fix and for your help. I guess my error might have be triggered by old courses whose content was no longer downloadable

from isisdl.

Got another problem. Now no longer on live system, so I wanted to get this on my newly installed system to run too. Probably something stupid on my side, so I'm not opening a new ticket:
.local/bin/isisdl --init
Traceback (most recent call last):
File "/home/user/.local/bin/isisdl", line 5, in
from isisdl.main import main
File "/home/user/.local/lib/python3.10/site-packages/isisdl/main.py", line 4, in
import isisdl.compress as compress
File "/home/user/.local/lib/python3.10/site-packages/isisdl/compress.py", line 18, in
from isisdl.backend.crypt import get_credentials
File "/home/user/.local/lib/python3.10/site-packages/isisdl/backend/crypt.py", line 11, in
from isisdl.utils import User, config, logger
File "/home/user/.local/lib/python3.10/site-packages/isisdl/utils.py", line 1513, in
startup()
File "/home/user/.local/lib/python3.10/site-packages/isisdl/utils.py", line 449, in startup
shutil.copy(source_code_location.joinpath("resources", "completions", "zsh", "_isisdl"), final_path)
File "/usr/lib/python3.10/shutil.py", line 417, in copy
copyfile(src, dst, follow_symlinks=follow_symlinks)
File "/usr/lib/python3.10/shutil.py", line 254, in copyfile
with open(src, 'rb') as fsrc:
FileNotFoundError: [Errno 2] No such file or directory: '/home/user/.local/lib/python3.10/site-packages/isisdl/resources/completions/zsh/_isisdl'
Exception ignored in atexit callback: <function OnKill.exit at 0x7fd9c01bcf70>
Traceback (most recent call last):
File "/home/user/.local/lib/python3.10/site-packages/isisdl/utils.py", line 910, in exit
OnKill.do_funcs()
File "/home/user/.local/lib/python3.10/site-packages/isisdl/utils.py", line 935, in do_funcs
OnKill._funcs.get_nowait()1
File "/home/user/.local/lib/python3.10/site-packages/isisdl/utils.py", line 947, in _impl
return function()
File "/home/user/.local/lib/python3.10/site-packages/isisdl/utils.py", line 994, in remove_lock_file
if not created_lock_file:
NameError: name 'created_lock_file' is not defined

I guess it has to do with this:
pip install .
Defaulting to user installation because normal site-packages is not writeable
.....

Should I sudo pip install or is that bad practice and I need to change a path in a file somewhere?

from isisdl.

The relevant line is:

FileNotFoundError: [Errno 2] No such file or directory: '/home/user/.local/lib/python3.10/site-packages/isisdl/resources/completions/zsh/_isisdl'

The resources should have been installed alongside the code. Could you check if something is in the /home/user/.local/lib/python3.10/site-packages/isisdl/resources/completions/zsh/ directory? If not, what is inside the resources/ directory?

As a sidenote: The last few lines are due to the variable created_lock_file being defined after the startup routine. I have shifted that definition up in the last commit.

from isisdl.

Thanks for the quick answer.
I have checked, those paths don't exixt here, but I did pip install.

ls /home/user/.local/lib/python3.10/site-packages/isisdl/resources/completions/zsh/
ls: Zugriff auf '/home/user/.local/lib/python3.10/site-packages/isisdl/resources/completions/zsh/' nicht möglich: Datei oder Verzeichnis nicht gefunden

ls /home/user/.local/lib/python3.10/site-packages/isisdl/resources/
ls: Zugriff auf '/home/user/.local/lib/python3.10/site-packages/isisdl/resources/' nicht möglich: Datei oder Verzeichnis nicht gefunden

ls /home/user/.local/lib/python3.10/site-packages/isisdl/
autorun.py  backend  compress.py  __init__.py  __main__.py  __pycache__  settings.py  utils.py  version.py

pip install .
Defaulting to user installation because normal site-packages is not writeable
Processing /home/user/repos/isisdl_git/isisdl
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: requests~=2.28.1 in /home/user/.local/lib/python3.10/site-packages (from isisdl==1.3.19) (2.28.2)
Requirement already satisfied: pyinotify~=0.9.6 in /usr/lib/python3/dist-packages (from isisdl==1.3.19) (0.9.6)
Requirement already satisfied: distro~=1.8.0 in /home/user/.local/lib/python3.10/site-packages (from isisdl==1.3.19) (1.8.0)
Requirement already satisfied: PyYAML~=6.0 in /home/user/.local/lib/python3.10/site-packages (from isisdl==1.3.19) (6.0.1)
Requirement already satisfied: psutil~=5.9.4 in /home/user/.local/lib/python3.10/site-packages (from isisdl==1.3.19) (5.9.8)
Requirement already satisfied: packaging~=22.0 in /home/user/.local/lib/python3.10/site-packages (from isisdl==1.3.19) (22.0)
Requirement already satisfied: colorama~=0.4.6 in /home/user/.local/lib/python3.10/site-packages (from isisdl==1.3.19) (0.4.6)
Requirement already satisfied: cryptography~=38.0.4 in /home/user/.local/lib/python3.10/site-packages (from isisdl==1.3.19) (38.0.4)
Requirement already satisfied: cffi>=1.12 in /home/user/.local/lib/python3.10/site-packages (from cryptography~=38.0.4->isisdl==1.3.19) (1.16.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/lib/python3/dist-packages (from requests~=2.28.1->isisdl==1.3.19) (2020.6.20)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests~=2.28.1->isisdl==1.3.19) (1.26.18)
Requirement already satisfied: charset-normalizer<4,>=2 in /home/user/.local/lib/python3.10/site-packages (from requests~=2.28.1->isisdl==1.3.19) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in /usr/lib/python3/dist-packages (from requests~=2.28.1->isisdl==1.3.19) (3.3)
Requirement already satisfied: pycparser in /home/user/.local/lib/python3.10/site-packages (from cffi>=1.12->cryptography~=38.0.4->isisdl==1.3.19) (2.22)
Building wheels for collected packages: isisdl
  Building wheel for isisdl (pyproject.toml) ... done
  Created wheel for isisdl: filename=isisdl-1.3.19-py3-none-any.whl size=74090 sha256=2c913dc9827e88460e09aa3d0c8daacc33e3918f8158f149ad4a7e9cf1f42720
  Stored in directory: /tmp/pip-ephem-wheel-cache-4uxhuyjh/wheels/13/ad/1e/57de3c12d34c82ecd1374f7f3c4d82cf0ca0542fa1884a3b3b
Successfully built isisdl
Installing collected packages: isisdl
  Attempting uninstall: isisdl
    Found existing installation: isisdl 1.3.19
    Uninstalling isisdl-1.3.19:
      Successfully uninstalled isisdl-1.3.19
  WARNING: The script isisdl is installed in '/home/user/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed isisdl-1.3.19

So I guess I need to change that path to /home/user/repos/isisdl_git/isisdl or symlink it?
No, there it's also not present:

ls /home/user/repos/isisdl_git/isisdl/ 
aur    checkCode.sh  diff.txt                  isisdl.wiki  pyproject.toml  release_notes.md      requirements_server.txt  setup.cfg  src    tldr
build  compile       fasterRequestsChanges.md  LICENSE      README.md       requirements_dev.txt  requirements.txt         setup.py   tests  uploadPyPI.sh

from isisdl.

Could you check if the directory /home/user/repos/isisdl_git/isisdl/src/isisdl/resources exists? I'm pretty sure it does exist since it is committed to the repository.

The resources should then be installed automatically (from setup.cfg):

[options.package_data]
isisdl = resources/**

I have no clue why your python interpreter would not install the resources alongside the code. Do they get installed when you download the package from pypi instead of git with pip install isisdl?

Of course, this should not be a fatal error. If resources are not found, then they simply shouldn't be installed. I'll fix this.

from isisdl.

Anyway, I've fixed this in the latest commit, when you update the repository the bug should be fixed. If you want, you can help me debug this further but it's also fine if not

from isisdl.

'''
ls /home/user/repos/isisdl_git/isisdl/src/isisdl/resources/
changelog completions manpages
'''

'''
pip install .
Defaulting to user installation because normal site-packages is not writeable
Processing /home/user/repos/isisdl_git/isisdl
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: psutil~=5.9.4 in /home/user/.local/lib/python3.10/site-packages (from isisdl==1.3.19) (5.9.8)
Requirement already satisfied: distro~=1.8.0 in /home/user/.local/lib/python3.10/site-packages (from isisdl==1.3.19) (1.8.0)
Requirement already satisfied: packaging~=22.0 in /home/user/.local/lib/python3.10/site-packages (from isisdl==1.3.19) (22.0)
Collecting requests~=2.28.1
Using cached requests-2.28.2-py3-none-any.whl (62 kB)
Requirement already satisfied: pyinotify~=0.9.6 in /usr/lib/python3/dist-packages (from isisdl==1.3.19) (0.9.6)
Requirement already satisfied: colorama~=0.4.6 in /home/user/.local/lib/python3.10/site-packages (from isisdl==1.3.19) (0.4.6)
Requirement already satisfied: cryptography~=38.0.4 in /home/user/.local/lib/python3.10/site-packages (from isisdl==1.3.19) (38.0.4)
Requirement already satisfied: PyYAML~=6.0 in /home/user/.local/lib/python3.10/site-packages (from isisdl==1.3.19) (6.0.1)
Requirement already satisfied: cffi>=1.12 in /home/user/.local/lib/python3.10/site-packages (from cryptography~=38.0.4->isisdl==1.3.19) (1.16.0)
Requirement already satisfied: charset-normalizer<4,>=2 in /home/user/.local/lib/python3.10/site-packages (from requests~=2.28.1->isisdl==1.3.19) (3.3.2)
Requirement already satisfied: certifi>=2017.4.17 in /usr/lib/python3/dist-packages (from requests~=2.28.1->isisdl==1.3.19) (2020.6.20)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests~=2.28.1->isisdl==1.3.19) (1.26.18)
Requirement already satisfied: idna<4,>=2.5 in /usr/lib/python3/dist-packages (from requests~=2.28.1->isisdl==1.3.19) (3.3)
Requirement already satisfied: pycparser in /home/user/.local/lib/python3.10/site-packages (from cffi>=1.12->cryptography~=38.0.4->isisdl==1.3.19) (2.22)
Building wheels for collected packages: isisdl
Building wheel for isisdl (pyproject.toml) ... done
Created wheel for isisdl: filename=isisdl-1.3.19-py3-none-any.whl size=74090 sha256=ace8d058960f9e553e40e263cf4f426ca172429049c2fe7f0e9cc92599a4e493
Stored in directory: /tmp/pip-ephem-wheel-cache-idaback1/wheels/13/ad/1e/57de3c12d34c82ecd1374f7f3c4d82cf0ca0542fa1884a3b3b
Successfully built isisdl
Installing collected packages: requests, isisdl
Attempting uninstall: requests
Found existing installation: requests 2.31.0
Uninstalling requests-2.31.0:
Successfully uninstalled requests-2.31.0
Attempting uninstall: isisdl
Found existing installation: isisdl 1.3.19
Uninstalling isisdl-1.3.19:
Successfully uninstalled isisdl-1.3.19
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
yt-dlp 2024.4.9 requires requests<3,>=2.31.0, but you have requests 2.28.2 which is incompatible.
Successfully installed isisdl-1.3.19 requests-2.28.2
'''
isisdl --init
Traceback (most recent call last):
File "/home/user/.local/bin/isisdl", line 5, in
from isisdl.main import main
File "/home/user/.local/lib/python3.10/site-packages/isisdl/main.py", line 4, in
import isisdl.compress as compress
File "/home/user/.local/lib/python3.10/site-packages/isisdl/compress.py", line 18, in
from isisdl.backend.crypt import get_credentials
File "/home/user/.local/lib/python3.10/site-packages/isisdl/backend/crypt.py", line 11, in
from isisdl.utils import User, config, logger
File "/home/user/.local/lib/python3.10/site-packages/isisdl/utils.py", line 1513, in
startup()
File "/home/user/.local/lib/python3.10/site-packages/isisdl/utils.py", line 449, in startup
shutil.copy(source_code_location.joinpath("resources", "completions", "zsh", "_isisdl"), final_path)
File "/usr/lib/python3.10/shutil.py", line 417, in copy
copyfile(src, dst, follow_symlinks=follow_symlinks)
File "/usr/lib/python3.10/shutil.py", line 254, in copyfile
with open(src, 'rb') as fsrc:
FileNotFoundError: [Errno 2] No such file or directory: '/home/user/.local/lib/python3.10/site-packages/isisdl/resources/completions/zsh/_isisdl'
Exception ignored in atexit callback: <function OnKill.exit at 0x7f93d68631c0>
Traceback (most recent call last):
File "/home/user/.local/lib/python3.10/site-packages/isisdl/utils.py", line 910, in exit
OnKill.do_funcs()
File "/home/user/.local/lib/python3.10/site-packages/isisdl/utils.py", line 935, in do_funcs
OnKill._funcs.get_nowait()1
File "/home/user/.local/lib/python3.10/site-packages/isisdl/utils.py", line 947, in _impl
return function()
File "/home/user/.local/lib/python3.10/site-packages/isisdl/utils.py", line 994, in remove_lock_file
if not created_lock_file:
NameError: name 'created_lock_file' is not defined
'''

'''
git pull --rebase
remote: Enumerating objects: 9, done.
remote: Counting objects: 100% (9/9), done.
remote: Compressing objects: 100% (1/1), done.
remote: Total 5 (delta 3), reused 5 (delta 3), pack-reused 0
Entpacke Objekte: 100% (5/5), 1.20 KiB | 154.00 KiB/s, fertig.
Von https://github.com/Emily3403/isisdl
a4ce9e7..2b58637 main -> origin/main
Aktualisiere a4ce9e7..2b58637
Fast-forward
src/isisdl/utils.py | 25 +++++++++++++------------
1 file changed, 13 insertions(+), 12 deletions(-)
isisdl --init
Traceback (most recent call last):
File "/home/user/.local/bin/isisdl", line 5, in
from isisdl.main import main
File "/home/user/.local/lib/python3.10/site-packages/isisdl/main.py", line 4, in
import isisdl.compress as compress
File "/home/user/.local/lib/python3.10/site-packages/isisdl/compress.py", line 18, in
from isisdl.backend.crypt import get_credentials
File "/home/user/.local/lib/python3.10/site-packages/isisdl/backend/crypt.py", line 11, in
from isisdl.utils import User, config, logger
File "/home/user/.local/lib/python3.10/site-packages/isisdl/utils.py", line 1513, in
startup()
File "/home/user/.local/lib/python3.10/site-packages/isisdl/utils.py", line 449, in startup
shutil.copy(source_code_location.joinpath("resources", "completions", "zsh", "_isisdl"), final_path)
File "/usr/lib/python3.10/shutil.py", line 417, in copy
copyfile(src, dst, follow_symlinks=follow_symlinks)
File "/usr/lib/python3.10/shutil.py", line 254, in copyfile
with open(src, 'rb') as fsrc:
FileNotFoundError: [Errno 2] No such file or directory: '/home/user/.local/lib/python3.10/site-packages/isisdl/resources/completions/zsh/_isisdl'
Exception ignored in atexit callback: <function OnKill.exit at 0x7fcbcd7331c0>
Traceback (most recent call last):
File "/home/user/.local/lib/python3.10/site-packages/isisdl/utils.py", line 910, in exit
OnKill.do_funcs()
File "/home/user/.local/lib/python3.10/site-packages/isisdl/utils.py", line 935, in do_funcs
OnKill._funcs.get_nowait()1
File "/home/user/.local/lib/python3.10/site-packages/isisdl/utils.py", line 947, in _impl
return function()
File "/home/user/.local/lib/python3.10/site-packages/isisdl/utils.py", line 994, in remove_lock_file
if not created_lock_file:
NameError: name 'created_lock_file' is not defined

'''

'''
pip uninstall isisdl
Found existing installation: isisdl 1.3.19
Uninstalling isisdl-1.3.19:
Would remove:
/home/user/.local/bin/isisdl
/home/user/.local/lib/python3.10/site-packages/isisdl-1.3.19.dist-info/*
/home/user/.local/lib/python3.10/site-packages/isisdl/*
Proceed (Y/n)?
Successfully uninstalled isisdl-1.3.19
'''

'''
pip install isisdl
Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: isisdl in /usr/local/lib/python3.10/dist-packages (1.3.19)
Requirement already satisfied: packaging~=22.0 in /home/user/.local/lib/python3.10/site-packages (from isisdl) (22.0)
Requirement already satisfied: distro~=1.8.0 in /home/user/.local/lib/python3.10/site-packages (from isisdl) (1.8.0)
Requirement already satisfied: requests~=2.28.1 in /home/user/.local/lib/python3.10/site-packages (from isisdl) (2.28.2)
Requirement already satisfied: cryptography~=38.0.4 in /home/user/.local/lib/python3.10/site-packages (from isisdl) (38.0.4)
Requirement already satisfied: psutil~=5.9.4 in /home/user/.local/lib/python3.10/site-packages (from isisdl) (5.9.8)
Requirement already satisfied: pyinotify~=0.9.6 in /usr/lib/python3/dist-packages (from isisdl) (0.9.6)
Requirement already satisfied: colorama~=0.4.6 in /home/user/.local/lib/python3.10/site-packages (from isisdl) (0.4.6)
Requirement already satisfied: PyYAML~=6.0 in /home/user/.local/lib/python3.10/site-packages (from isisdl) (6.0.1)
Requirement already satisfied: cffi>=1.12 in /home/user/.local/lib/python3.10/site-packages (from cryptography~=38.0.4->isisdl) (1.16.0)
Requirement already satisfied: idna<4,>=2.5 in /usr/lib/python3/dist-packages (from requests~=2.28.1->isisdl) (3.3)
Requirement already satisfied: certifi>=2017.4.17 in /usr/lib/python3/dist-packages (from requests~=2.28.1->isisdl) (2020.6.20)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests~=2.28.1->isisdl) (1.26.18)
Requirement already satisfied: charset-normalizer<4,>=2 in /home/user/.local/lib/python3.10/site-packages (from requests~=2.28.1->isisdl) (3.3.2)
Requirement already satisfied: pycparser in /home/user/.local/lib/python3.10/site-packages (from cffi>=1.12->cryptography~=38.0.4->isisdl) (2.22)
'''
'''
isisdl --init
bash: /home/user/.local/bin/isisdl: Datei oder Verzeichnis nicht gefunden
'''
:(
I'm clueless

from isisdl.

First of all, with three backticks (`) you can highlight code, not with three quotation marks (").

Next, you've updated the repo with git pull --rebase, however not installed it afterwards. Pip copies the files to the ~/.local/lib/python3.10/site-packages directory. So, you'll have to re-install it again with pip install .

As for the error afterwards, it seams as if isisdl is already installed in /usr/local/lib/python3.10/site-packages, probably when you installed it with sudo pip install . or sudo pip install isisdl. Maybe that is also the reason resources is not found as it was not installed with that run but the isisdl from the sudo install. Could you uninstall that with sudo pip uninstall isisdl and then try again?

from isisdl.

Oh, sorry, I must have missed that step. Sorry. It now seems to work. Thank you again very much.

Building request cache for files . . . 
Done in: 0:03:48

19983 / 40737 documents
  4869 / 5409 external links
        0 /       0 videos

(will be cached)

But I know recently some videos have been put in some isis courses long after my last sync (for some pureley online lectures and some tutorials that didn't take place in person...

.config/isisdl/config.yaml:

# Should videos be downloaded on this device?
# Possible values {"true", "false"}
download_videos: true

/home/user/.local/bin/isisdl --sync
                                                                                                                                                                         
Building request cache for files .                                                                                                                                       
Done in: 0:00:00                                                                                                                                                         
                                                                                                                                                                         
40737 / 40737 documents                                                                                                                                                  
 5409 /  5409 external links                                                                                                                                             
    0 /     0 videos                                                                                                                                                     
                                                                                                                                                                         
(will be cached)                                                                                                                                                         
                                                                                                                                                                         
[██████████████████████████████████████████████████]                                                                                                                     
                                                                                                                                                                         
                                                                                                                                                                         
Discovering files                                                                                                                                                        
Done in: 0:00:00                                                                                                                                                         
                                                                                                                                                                         
                                                                                                                                                                         
[██████████████████████████████████████████████████]                                                                                                                     
                                                                                                                                                                         
I have achieved the following:

Recovered files:     0,   0.00   B
Unchanged files: 33311,  31.06 GiB
Corrupted files:     0,   0.00   B

Dropped 306 entries from the database to be re-downloaded.

Why dropped? :(
Well, I'll try again.
And am also using moodle-dl and whenever I install isisdl, it uninstalls the version of requests that moodle-dl uses , and vice versa:

Installing collected packages: requests
  Attempting uninstall: requests
    Found existing installation: requests 2.28.2
    Uninstalling requests-2.28.2:
      Successfully uninstalled requests-2.28.2
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
isisdl 1.3.19 requires requests~=2.28.1, but you have requests 2.31.0 which is incompatible.
Successfully installed requests-2.31.0

Is there a way to have both?

from isisdl.

ran it again and this time it found corrupted files.

/home/user/.local/bin/isisdl --sync
                                                                                                                                                                        
Building request cache for files ...                                                                                                                                     
Done in: 0:00:00                                                                                                                                                         
                                                                                                                                                                        
40737 / 40737 documents                                                                                                                                                  
5409 /  5409 external links                                                                                                                                             
   0 /     0 videos                                                                                                                                                     
                                                                                                                                                                        
(will be cached)                                                                                                                                                         
                                                                                                                                                                        
[██████████████████████████████████████████████████]                                                                                                                     
                                                                                                                                                                        
                                                                                                                                                                        
Discovering files ..                                                                                                                                                     
Done in: 0:00:00                                                                                                                                                         
                                                                                                                                                                        
                                                                                                                                                                        
[██████████████████████████████████████████████████]                                                                                                                     
                                                                                                                                                                        
I have achieved the following:

Recovered files:    22,  13.77 MiB
Unchanged files: 33000,  30.71 GiB
Corrupted files:   289, 345.80 MiB
Do you want me to bulk delete all 289 corrupted files? [y/n]

assuming they are really corrupted, I guess I should select y here.
No idea why they are corrupted or what corrupted them, though

from isisdl.

But it never recoeverred those files that were corrupt, either:

/home/user/.local/bin/isisdl --sync
                                                                                                                                                                         
Building request cache for files .                                                                                                                                       
Done in: 0:00:00                                                                                                                                                         
                                                                                                                                                                         
40737 / 40737 documents                                                                                                                                                  
 5409 /  5409 external links                                                                                                                                             
    0 /     0 videos                                                                                                                                                     
                                                                                                                                                                         
(will be cached)                                                                                                                                                         
                                                                                                                                                                         
[██████████████████████████████████████████████████]                                                                                                                     
                                                                                                                                                                         
                                                                                                                                                                         
Discovering files ..                                                                                                                                                     
Done in: 0:00:00                                                                                                                                                         
                                                                                                                                                                         
                                                                                                                                                                         
[██████████████████████████████████████████████████]                                                                                                                     
                                                                                                                                                                         
I have achieved the following:

Recovered files:     0,   0.00   B
Unchanged files: 33022,  30.72 GiB
Corrupted files:     0,   0.00   B

Dropped 0 entries from the database to be re-downloaded.

So if I interpret the output correctly, there are 40737 files it should get (plus the videos that it hasn't found) but it only got 33022 files previously and didn't donwload any of the remainning 7715 files.

from isisdl.

It seams as if you have disabled video download in the --config settings. I'm not quite sure why the config file would not overwrite this, but this is my only guess. Try setting this option to enabled.

As for the corrupted files, they are files which do not match the expected hash in the database. Entries that are dropped are entries where the corresponding file (by hash) can not be found on the filesystem. It checks both ways, meaning that existing_files ⊆ database_files and existing_files ⊇ database_files implying existing_files = database_files.

My guess is that, because the entries were dropped it can't find the files and then they are recognized as corrupted because of that. The other 7715 files are probably unavailable due to legal reasons as every isis course has a max lifespan of 2 years, meaning that all files will be unavailable after that. If the file is old enough it should probably just not be counted towards the total. As with most issues, this will be fixed in 2.0™.

And no, there is no way of getting the requests dependency to be satisfied with both moodle-dl and isisdl installed on the same interpreter, as moodle-dl specifies requests>=2.24.0 while isisdl specifies requests~=2.28.1. Now, because moodle-dl allows the newest version of requests to be installed, pip goes ahead and does so.

isisdl uses the Semantic Versioning spec python provides. This is useful because requests isn't allowed by the spec to add any backward-incompatible changes in the 2.28 version. So isisdl won't suddenly break when they decide to change something major. This, however, also has the disadvantage of requiring a specific requests version and breaking when used in combination with other packages. Originally, I used the >= spec until #18 as packaging introduced a breaking change that lead to isisdl breaking when the dependency got updated.

from isisdl.