Giter VIP home page Giter VIP logo

Comments (7)

albert597 avatar albert597 commented on July 28, 2024 1

Thank you for finding this issue! I was not aware that every user must have git-lfs installed to download the file. I will update the README.

Please let me know if you have anymore concerns

from trailmap.

albert597 avatar albert597 commented on July 28, 2024

Thank you for bringing up this issue. While testing at first, I found that the trailmap_model.hdf5 downloaded from the git repository does work on my computer. However, I also installed HDFView to inspect the file, and noticed that after opening the hdf5 file on HDFView, I received the same error as you did with the file being unable to be open when running segment_brain_batch.py.

It appears that HDFView is altering the trailmap_model.hdf5 file. This can be evidenced from the date modified updating when I open it. I also ran ubuntu's cmp command on the hdf5 file in comparison to a freshly downloaded hdf5 model, and it indeed does say the files are different.

Could you try downloading a fresh copy of trailmap_model.hdf5 again without opening it in HDFView? Please let me know if you continue to have issues.

from trailmap.

VolkerH avatar VolkerH commented on July 28, 2024

Hi Albert,

thanks for your reply. I did notice the problem first when running

python segment_brain_batch.py data/testing/example-chunk

on a freshly cloned repo. Only after running into errors there did I try to dig into the hdf5 file with HDFView and other tools.
I will try once more with a freshly cloned repo and report back.

from trailmap.

VolkerH avatar VolkerH commented on July 28, 2024 
(base) vhil0002@MU00060589:~/Github$ mkdir tmp
(base) vhil0002@MU00060589:~/Github$ conda activate trailmap
(trailmap) vhil0002@MU00060589:~/Github$ cd tmp
(trailmap) vhil0002@MU00060589:~/Github/tmp$ git clone https://github.com/AlbertPun/TRAILMAP
Cloning into 'TRAILMAP'...
remote: Enumerating objects: 76, done.
remote: Counting objects: 100% (76/76), done.
remote: Compressing objects: 100% (50/50), done.
remote: Total 455 (delta 21), reused 66 (delta 21), pack-reused 379
Receiving objects: 100% (455/455), 53.30 MiB | 12.06 MiB/s, done.
Resolving deltas: 100% (30/30), done.
(trailmap) vhil0002@MU00060589:~/Github/tmp$ cd TRAILMAP/
(trailmap) vhil0002@MU00060589:~/Github/tmp/TRAILMAP$ ls -l data/
model-weights/           registration-paramaters/ testing/                 tf-logs/                 training/                validation/              
(trailmap) vhil0002@MU00060589:~/Github/tmp/TRAILMAP$ ls -l data/
model-weights/           registration-paramaters/ testing/                 tf-logs/                 training/                validation/              
(trailmap) vhil0002@MU00060589:~/Github/tmp/TRAILMAP$ ls -l data/model-weights/trailmap_model.hdf5 
-rw-r--r-- 1 vhil0002 vhil0002 134 Feb 11 14:11 data/model-weights/trailmap_model.hdf5
(trailmap) vhil0002@MU00060589:~/Github/tmp/TRAILMAP$ python segment_brain_batch.py data/testing/example-chunk/
/home/vhil0002/anaconda3/envs/trailmap/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:523: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/home/vhil0002/anaconda3/envs/trailmap/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:524: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/home/vhil0002/anaconda3/envs/trailmap/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
/home/vhil0002/anaconda3/envs/trailmap/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:526: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/home/vhil0002/anaconda3/envs/trailmap/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:527: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
/home/vhil0002/anaconda3/envs/trailmap/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:532: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])
2020-02-11 14:12:54.135352: W tensorflow/core/util/tensor_slice_reader.cc:95] Could not open /home/vhil0002/Github/tmp/TRAILMAP/data/model-weights/trailmap_model.hdf5: Data loss: not an sstable (bad magic number): perhaps your file is in a different file format and you need to use a different restore operator?
Traceback (most recent call last):
  File "segment_brain_batch.py", line 22, in <module>
    model.load_weights(weights_path)
  File "/home/vhil0002/anaconda3/envs/trailmap/lib/python3.6/site-packages/tensorflow/python/keras/engine/network.py", line 1385, in load_weights
    with h5py.File(filepath, 'r') as f:
  File "/home/vhil0002/anaconda3/envs/trailmap/lib/python3.6/site-packages/h5py/_hl/files.py", line 408, in __init__
    swmr=swmr)
  File "/home/vhil0002/anaconda3/envs/trailmap/lib/python3.6/site-packages/h5py/_hl/files.py", line 173, in make_fid
    fid = h5f.open(name, flags, fapl=fapl)
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "h5py/h5f.pyx", line 88, in h5py.h5f.open
OSError: Unable to open file (file signature not found)

from trailmap.

VolkerH avatar VolkerH commented on July 28, 2024

note though that in contrast to your instructions I do not have h5py version 2.1

pip3 install h5py==2.1

as this version is no longer available on PyPI for installation with pip nor is it available with conda-forge. I tried to build version 2.1 from sources but haven't had luck so far. I also doubt that 2.1 would have produced incompatible hdf5 files.

Edited to add:

I have tested h5py version 2.6 and version 2.10 on Linux. HDFView was an older install on Windows.

from trailmap.

VolkerH avatar VolkerH commented on July 28, 2024

Ok, I think figured out the problem.

You seem to be using git-lfs (large file store) for storing the model weights. If one (like me) just has regular git installed, there will only be a placeholder file cloned.

After I installed git-lfs according to the instructions here the correct file was cloned. This might be worth a mention in the Readme.

from trailmap.

jesusdpa1 avatar jesusdpa1 commented on July 28, 2024

Hi,

I am getting the following error when trying to download the weights

git lfs pull
batch response: This repository is over its data quota. Account responsible for LFS bandwidth should purchase more data packs to restore access.
error: failed to fetch some objects from 'https://github.com/AlbertPun/TRAILMAP.git/info/lfs'

from trailmap.

Related Issues (14)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.