Comments (5)
julianharty
commented on June 26, 2024
x-ref #32 which has an additional trace of what appears to be the same problem.
from kiwix-build.
mgautierfr
commented on June 26, 2024
I don't know what happen here.
The download is skipped because the archive exists and has the same sha256sum than expected.
So the "extract step" should find the archive_path correctly.
What is the content of the ARCHIVE directory ?
from kiwix-build.
julianharty
commented on June 26, 2024
(kiwix)pi@rachel ~/kiwix-dev/kiwix-build $ ls -lart ARCHIVE/
total 5272
-rw-r--r-- 1 pi pi 363189 Apr 16 13:28 pugixml-1.2.tar.gz
-rw-r--r-- 1 pi pi 1048328 Apr 16 13:28 xz-5.0.4.tar.bz2
-rw-r--r-- 1 pi pi 2799492 Apr 16 13:28 xapian-core-1.4.2.tar.xz
-rw-r--r-- 1 pi pi 571091 Apr 16 16:03 zlib-1.2.8.tar.gz
drwxr-xr-x 2 pi pi 4096 Apr 16 16:03 .
-rw-r--r-- 1 pi pi 601699 Apr 16 16:03 e2fsprogs-libs-1.43.4.tar.gz
drwxr-xr-x 13 pi pi 4096 Apr 19 08:24 ..
I then ran kiwix-build.py as follows:
python -v kiwix-build.py which provided some clues. It seems utils.py wants to use gzip
prepare sources xapian-core :
download xapian-core : SKIP
extract xapian-core : # /home/pi/.pyenv/versions/3.4.3/lib/python3.4/__pycache__/lzma.cpython-34.pyc matches /home/pi/.pyenv/versions/3.4.3/lib/python3.4/lzma.py
# code object from '/home/pi/.pyenv/versions/3.4.3/lib/python3.4/__pycache__/lzma.cpython-34.pyc'
# destroy lzma
# /home/pi/.pyenv/versions/3.4.3/lib/python3.4/__pycache__/gzip.cpython-34.pyc matches /home/pi/.pyenv/versions/3.4.3/lib/python3.4/gzip.py
# code object from '/home/pi/.pyenv/versions/3.4.3/lib/python3.4/__pycache__/gzip.cpython-34.pyc'
import 'gzip' # <_frozen_importlib.SourceFileLoader object at 0x76507f90>
ERROR
Traceback (most recent call last):
File "/home/pi/kiwix-dev/kiwix-build/utils.py", line 75, in extract_archive
archive = tarfile.open(archive_path)
File "/home/pi/.pyenv/versions/3.4.3/lib/python3.4/tarfile.py", line 1553, in open
raise ReadError("file could not be opened successfully")
tarfile.ReadError: file could not be opened successfully
BTW https://docs.python.org/3/library/tarfile.html does not mention the ability to directly decompress .tar.xz format although it covers various similar compression formats.
I managed to decompress the file on the command line
history
-- cut --
1291 cp ARCHIVE/xapian-core-1.4.2.tar.xz ~
1292 cd ~
1293 ls -l
1294 tar xf xapian-core-1.4.2.tar.xz
1295 ls -l
1296 ls -la xapian-core-1.4.2/
I handcoded python interactively to see if I could isolate the problem
(kiwix)pi@rachel ~/kiwix-dev/kiwix-build $ python
Python 3.4.3 (default, Apr 16 2017, 09:52:16)
[GCC 4.9.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import tarfile
>>> tarfile.open('ARCHIVE/xapian-core-1.4.2.tar.xz')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/pi/.pyenv/versions/3.4.3/lib/python3.4/tarfile.py", line 1553, in open
raise ReadError("file could not be opened successfully")
tarfile.ReadError: file could not be opened successfully
>>> quit()
This fails for the particular file, but not for a small test.tar.xz file I created by hand (with a single small file).
(kiwix)pi@rachel ~ $ tar -cJf test.tar.xz minicom.log
(kiwix)pi@rachel ~ $ ls -lart
-- cut --
-rw-r--r-- 1 pi pi 204 Apr 19 09:07 test.tar.xz
(kiwix)pi@rachel ~ $ python
Python 3.4.2 (default, Oct 19 2014, 13:31:11)
[GCC 4.9.1] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import tarfile
>>> tarfile.open('test.tar.xz')
<tarfile.TarFile object at 0x76a4bf50>
>>> quit()
I tried seeing what Linux thinks both files contain
(kiwix)pi@rachel ~ $ file test.tar.xz
test.tar.xz: XZ compressed data
(kiwix)pi@rachel ~ $ file xapian-core-1.4.2.tar.xz
xapian-core-1.4.2.tar.xz: XZ compressed data
(kiwix)pi@rachel ~ $ tar -xJf test.tar.xz
Both files have the same permissions and owner
ls -la *.xz
-rw-r--r-- 1 pi pi 204 Apr 19 09:07 test.tar.xz
-rw-r--r-- 1 pi pi 2799492 Apr 19 08:45 xapian-core-1.4.2.tar.xz
Sooooo, it seems python 3 (v3.4.3) isn't able to decompress this specific file for some reason. Any more ideas? I'll also keep looking / exploring e.g. what about providing the file in another format? and are there ways we can convince the python tarfile.open to reveal why it doesn't like this file?
PS: I've tried opening the file in the local folder in case the problem was related to the path.
from kiwix-build.
kelson42
commented on June 26, 2024
We now use xapian 1.4.5, @julianharty Do you still have problems?
from kiwix-build.
kelson42
commented on June 26, 2024
I close that ticket because of lack of feedback.
from kiwix-build.
Related Issues (20)
- Last publication of kiwix-tools seems to not work on raspberryPi HOT 6
- Instruction to use kiwix-build seems wrong in the `README.md` HOT 1
- Build libzim & libkiwix for Android with debug symbol HOT 1
- Failed to build zim-tools on ubuntu22.04 (wsl) HOT 3
- Use libxapian 1.4.23 HOT 7
- Fix double publications of release archives
- Incorrect docker trigger on macOS HOT 2
- aarch64 bionic should not require GLIBCXX 3.4.26 and 3.4.29 HOT 8
- Update ICU to a more recent version. HOT 2
- Why compiling libkiwix for iOS/macOS does embeds the libzim? HOT 7
- Use latest/future version of aria2c HOT 2
- Fix cross_compilation environment setting.
- Last version of aria2 1.37.0 doesn't compile on bionic. HOT 2
- CoreKiwix.xcframework not usable with iOS Simulator HOT 5
- Wrap XCFramework into a swift package, for Apple (iOS, macOS) distribution HOT 3
- Explicit dependence names for macOS HOT 2
- post checksums for AppImage HOT 2
- Recent versions of XZ (since 2022) may be compromised and should be checked HOT 2
- No Kiwix Desktop for Windows anymore
- Move out of bionic. HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from kiwix-build.