Comments (4)
schlegelp
commented on July 19, 2024
Hi. This is not an issue on your end. Instead the issue appears to be that the HEAD request to fetch the file size fails. Here is the curl equivalent for the JRC2018F_FAFB.h5 transform:
$ $ curl -L --head https://ndownloader.figshare.com/files/14362754?private_link=3a8b1d84c5e197edc97c
HTTP/1.1 302 Found
Server: nginx
Date: Sun, 01 Aug 2021 11:48:52 GMT
Content-Type: application/octet-stream
Content-Length: 0
Connection: keep-alive
X-Filename: JRC2018F_FAFB.h5
X-Storage-Protocol: https
Location: https://s3-eu-west-1.amazonaws.com/pfigshare-u-files/14362754/JRC2018F_FAFB.h5?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Expires=10&X-Amz-SignedHeaders=host&X-Amz-Signature=b3849f006c169fdb24a3fbd45457b1e32c08b268b7252fee2bbb445f7d1b9ab9&X-Amz-Date=20210801T114852Z&X-Amz-Credential=AKIAIYCQYOYV5JSSROOA/20210801/eu-west-1/s3/aws4_request
X-Storage-Host: s3-eu-west-1.amazonaws.com
X-Storage-File: 14362754/JRC2018F_FAFB.h5
X-Storage-Bucket: pfigshare-u-files
Content-Disposition: attachment;filename=JRC2018F_FAFB.h5
Cache-Control: no-cache, no-store
Set-Cookie: fig_tracker_client=d3d71476-932a-4788-a9d7-45744ef63973; Max-Age=31536000; Path=/; expires=Mon, 01-Aug-2022 11:48:52 GMT; secure; HttpOnly
X-Robots-Tag: noindex
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
Strict-Transport-Security: max-age=31536000; includeSubDomains;
Cache-Control: public, must-revalidate, proxy-revalidate
Access-Control-Allow-Credentials: true
Access-Control-Allow-Methods: GET, OPTIONS
Access-Control-Allow-Headers: Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Authorization,Range
Access-Control-Expose-Headers: Location,Accept-Ranges,Content-Encoding,Content-Length,Content-Range
HTTP/1.1 403 Forbidden
x-amz-request-id: 0FS8G6DPVJD9H447
x-amz-id-2: c63ugheKNdLMZtDvy1Pwt+dSZ9t5RhKKGje+/68VPNa8/DLV8f0gxhTZLPpFuY6zRegLZU6zogE=
Content-Type: application/xml
Date: Sun, 01 Aug 2021 11:48:52 GMT
Server: AmazonS3figshare (https://ndownloader.figshare.com/files/14362754?private_link=3a8b1d84c5e197edc97c) redirects us to AWS (https://s3-eu-west-1.amazonaws.com/pfigshare-u-files/14362754/JRC2018F_FAFB.h5?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Expires=10&X-Amz-SignedHeaders=host&X-Amz-Signature=17be44bf7ab705f3f8c19b714781f0e63277b9a10059d780a3cd7c046a489576&X-Amz-Date=20210801T113223Z&X-Amz-Credential=AKIAIYCQYOYV5JSSROOA/20210801/eu-west-1/s3/aws4_request) which is where the HEAD requests fails with a 403 Forbidden. Strangely, an equivalent GET request works just fine.
This used to work just fine so I'm assuming something changed on the AWS side. @jefferis any ideas?
@tedwu-neuro In the meantime, you can download these files manually, put them in ~/flybrain-data/ and start a fresh Python session before importing flybrains.
https://ndownloader.figshare.com/files/14362754?private_link=3a8b1d84c5e197edc97c
https://ndownloader.figshare.com/files/14368703?private_link=2a684586d5014e31076c
https://ndownloader.figshare.com/files/14369093?private_link=d5965dad295e46241ae1
https://ndownloader.figshare.com/files/21749535?private_link=ca603876efb33fdf3028
https://ndownloader.figshare.com/files/14371574?private_link=b7120207f38b35f1e372
https://ndownloader.figshare.com/files/14368358?private_link=b29e25b6e47ccf9187a8
from navis-flybrains.
tedwu-neuro
commented on July 19, 2024
Hi Philipp,
Thanks for your prompt reply!
I did it manually, and it works.
However, after I tried to install the jefferislab_transforms with the code (flybrains.download_jefferislab_transforms()), I got some other error messages when I started a new Python session and then import flybrains.
--------
C:\Users\u0144374\Anaconda3\lib\site-packages\flybrains\core.py:238: UserWarning:
Error registering C:\Users\u0144374\flybrain-data\BridgingRegistrations\Cell07_IS2.list\original.list as transform: list index out of range
C:\Users\u0144374\Anaconda3\lib\site-packages\flybrains\core.py:238: UserWarning:
Error registering C:\Users\u0144374\flybrain-data\BridgingRegistrations\FCWB_JFRC2.list\original.list as transform: list index out of range
C:\Users\u0144374\Anaconda3\lib\site-packages\flybrains\core.py:238: UserWarning:
Error registering C:\Users\u0144374\flybrain-data\BridgingRegistrations\IS2_Cell07.list\original.list as transform: list index out of range
C:\Users\u0144374\Anaconda3\lib\site-packages\flybrains\core.py:238: UserWarning:
Error registering C:\Users\u0144374\flybrain-data\BridgingRegistrations\JFRC2_FCWB.list\original.list as transform: list index out of range
C:\Users\u0144374\Anaconda3\lib\site-packages\flybrains\core.py:238: UserWarning:
Error registering C:\Users\u0144374\flybrain-data\BridgingRegistrations\JFRC2_IS2.list\original.list as transform: list index out of range
C:\Users\u0144374\Anaconda3\lib\site-packages\flybrains\core.py:238: UserWarning:
Error registering C:\Users\u0144374\flybrain-data\BridgingRegistrations\JFRC2_JFRC2013.list\original.list as transform: list index out of range
C:\Users\u0144374\Anaconda3\lib\site-packages\flybrains\core.py:238: UserWarning:
Error registering C:\Users\u0144374\flybrain-data\BridgingRegistrations\T1_IS2.list\original.list as transform: list index out of range
C:\Users\u0144374\Anaconda3\lib\site-packages\flybrains\core.py:238: UserWarning:
Error registering C:\Users\u0144374\flybrain-data\DrosophilidBridgingRegistrations\Dmel_DsecI.list\original.list as transform: list index out of range
C:\Users\u0144374\Anaconda3\lib\site-packages\flybrains\core.py:238: UserWarning:
Error registering C:\Users\u0144374\flybrain-data\DrosophilidBridgingRegistrations\Dmel_Dsim.list\original.list as transform: list index out of range
C:\Users\u0144374\Anaconda3\lib\site-packages\flybrains\core.py:238: UserWarning:
Error registering C:\Users\u0144374\flybrain-data\DrosophilidBridgingRegistrations\Dmel_Dvir.list\original.list as transform: list index out of range
C:\Users\u0144374\Anaconda3\lib\site-packages\flybrains\core.py:238: UserWarning:
Error registering C:\Users\u0144374\flybrain-data\DrosophilidBridgingRegistrations\Dmel_Dyak.list\original.list as transform: list index out of range
C:\Users\u0144374\Anaconda3\lib\site-packages\flybrains\core.py:238: UserWarning:
Error registering C:\Users\u0144374\flybrain-data\DrosophilidBridgingRegistrations\Dmel_IS2.list\original.list as transform: list index out of range
C:\Users\u0144374\Anaconda3\lib\site-packages\flybrains\core.py:238: UserWarning:
Error registering C:\Users\u0144374\flybrain-data\DrosophilidBridgingRegistrations\Dmel_JFRC2.list\original.list as transform: list index out of range
C:\Users\u0144374\Anaconda3\lib\site-packages\flybrains\core.py:238: UserWarning:
Error registering C:\Users\u0144374\flybrain-data\DrosophilidBridgingRegistrations\DsecI_DsecF.list\original.list as transform: list index out of range
C:\Users\u0144374\Anaconda3\lib\site-packages\flybrains\core.py:238: UserWarning:
Error registering C:\Users\u0144374\flybrain-data\DrosophilidBridgingRegistrations\DsecI_DsecM.list\original.list as transform: list index out of range
C:\Users\u0144374\Anaconda3\lib\site-packages\flybrains\core.py:238: UserWarning:
Error registering C:\Users\u0144374\flybrain-data\DrosophilidBridgingRegistrations\DsecI_IS2.list\original.list as transform: list index out of range
C:\Users\u0144374\Anaconda3\lib\site-packages\flybrains\core.py:238: UserWarning:
Error registering C:\Users\u0144374\flybrain-data\DrosophilidBridgingRegistrations\DsecI_JFRC2.list\original.list as transform: list index out of range
C:\Users\u0144374\Anaconda3\lib\site-packages\flybrains\core.py:238: UserWarning:
Error registering C:\Users\u0144374\flybrain-data\DrosophilidBridgingRegistrations\Dsim_DsecI.list\original.list as transform: list index out of range
C:\Users\u0144374\Anaconda3\lib\site-packages\flybrains\core.py:238: UserWarning:
Error registering C:\Users\u0144374\flybrain-data\DrosophilidBridgingRegistrations\IS2_DsecI.list\original.list as transform: list index out of range
C:\Users\u0144374\Anaconda3\lib\site-packages\flybrains\core.py:238: UserWarning:
Error registering C:\Users\u0144374\flybrain-data\DrosophilidBridgingRegistrations\JFRC2_DsecI.list\original.list as transform: list index out of range
C:\Users\u0144374\Anaconda3\lib\site-packages\flybrains\core.py:238: UserWarning:
Error registering C:\Users\u0144374\flybrain-data\MirrorRegistrations\ATAG_mirror.list\orig.list as transform: list index out of range
C:\Users\u0144374\Anaconda3\lib\site-packages\flybrains\core.py:238: UserWarning:
Error registering C:\Users\u0144374\flybrain-data\MirrorRegistrations\JFRC2_mirror.list\orig.list as transform: list index out of range
----------
and when I remove those three folders (BridgingRegistrations,DrosophilidBridgingRegistrations, MirrorRegistrations) within flybrain-date folder, I can import flybrains normally.
Do you have any idea about why?
Thanks a lot!
Best,
Yifan
from navis-flybrains.
schlegelp
commented on July 19, 2024
My disclaimer is that I have not tested any of the packages on Windows and you might run into issues further down the line but we can try debugging as best as possible.
Re those warnings:
CMTK transforms are essentially folders like Cell07_IS2.list where Cell07 is the target and IS2 is the source. flybrains looks for folders that end with .list (for CMTK) and expects the name to be {target}_{source}.list for parsing.
As you can see from the warnings it also tries to register the orig.list or original.list. These appear to show as symlinks in Linux/OSX which is why they are skipped entirely. In Windows, they apparently to show up as file/folder which is why flybrains tries to register them as transforms and then fails because they don't follow the {target}_{source}.list convention.
When you delete the transform folders that warning goes away because there is nothing for flybrains to register.
Now to the fix:
- You can actually just ignore those warnings since the actual transforms should be correctly registered.
- I also just released a new version (
0.1.10) that should fix the issue.
from navis-flybrains.
tedwu-neuro
commented on July 19, 2024
I just upgrade the flybrains to the new version (0.1.10), and the issue is fixed!
Thanks a lot!
Best,
Yifan
from navis-flybrains.
Related Issues (8)
- improve FAFB mirror registration HOT 1
- Import flybrains creates flybrain-data folder im home directory HOT 3
- Importing flybrains - UnicodeDecodeError: 'charmap' codec can't decode byte 0x9d in position 19631: character maps to <undefined> HOT 1
- The registration from hemibrain to FAFB is squished HOT 1
- Issue with saalfeldlab_transforms download HOT 2
- Better document alternative data_home
- Re-organize transform download
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from navis-flybrains.