Comments (8)
Are you sure? The X-Bz-Content-Sha1
header on the download should contain the SHA1 passed in during the upload: https://www.backblaze.com/b2/docs/b2_download_file_by_id.html and https://www.backblaze.com/b2/docs/b2_download_file_by_name.html
from blazer.
From what I can tell, that doesn't provide anything beyond allowing you to verify the checksum on download.
I think this probably fits into a larger discussion about validating files on download in general. Right now blazer does no verification at all, and I have some ideas about how to change that, but specifically for large files I definitely want to avoid reading the file twice.
from blazer.
It definitely would be useful during Download verification, and I already do this outside your library using a MultiWriter to compute checksums as the data is downloaded.
It would be useful to have the hash available on large files to begin with though.
from blazer.
I'm considering adding a method Verify(ctx) (error, bool)
to the Reader, which would perform the round-trip and verify the file if it can. It would only work when (a) the SHA1 hash is available on the remote end, and (b) when the entire file has been downloaded (so no range readers).
But I'm still not sure how to allow users to set the sha1 hash for large files on upload. Doing it manually (by e.g. exposing a field on Writer) feels like punting. An alternative could be a field that simply says "performing multiple passes when uploading this is okay", and then I can collect all the hashes that I need in two passes in-place.
(And then the file type is starting to get complicated and I need to look at passing options to that... blep.)
from blazer.
I think it may make more sense to just do the check behind the scenes and trigger a retry or return an error on the download if there is a checksum mismatch, why make it an additional call when you'll probably compute the checksum as you download it and know the file to be bad already?
In my use case, I already have the checksum computed so I don't need this package to do it for me. you can do something like: If provided by user, utilize this value, else, if option is set to compute checksum, compute checksum before beginning upload (obviously, introduce an option as you mentioned)
from blazer.
In this case the additional call is to determine what value I should check against, since I don't get that information during the download.
from blazer.
Oh, look at that, that's super convenient. Yeah that'll make it a lot easier.
from blazer.
Confirmed to be working, thanks!
from blazer.
Related Issues (20)
- Add an option to turn off error handling
- Unable to create an empty file HOT 7
- EOF issue with zfsbackup-go HOT 24
- There is a race condition when a writer is cancelled. HOT 1
- No docs on creating Bucket object
- Slightly extend `base` interface for stateless programs HOT 4
- How to set custom info headers
- Get a file at a specific version (ID) HOT 3
- How to perform Delete operation on sequence of .JPEG files? HOT 2
- Error while trying to create *b2.Bucket object - bucket not found HOT 3
- Generating Auth Token and re-using it for list of image files stored on B2 bucket
- Question about the state of this library. HOT 1
- Operations on 0-byte-files fail with 416: out of content HOT 6
- Is bonfire able to be used as a local emulator? HOT 3
- Possible crashing bug HOT 2
- Infinite loop in NewClient() if system certificates are not installed HOT 1
- Supply optional bucketName parameter to listBuckets HOT 1
- Deleting a file downloads it first HOT 2
- go-fuzz is deprecated with Go 1.18
- Upload file with custom content type? HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from blazer.