Comments (5)
Hi, it would probably be nice to have storage information in the Upload
model, which is passed along in hooks. Here is what tusd has in it:
// Storage contains information about where the upload is stored. The exact values
// depend on the storage that is used and are not available in the pre-create hook.
// This example belongs to the file store.
"Storage": {
// For example, the filestore supplies the absolute file path:
"Type": "filestore",
"Path": "/my/upload/directory/14b1c4c77771671a8479bc0444bbc5ce",
// The S3Store and GCSStore supply the bucket name and object key:
"Type": "s3store",
"Bucket": "my-upload-bucket",
"Key": "my-prefix/14b1c4c77771671a8479bc0444bbc5ce"
}
Then you can use onUploadFinish
to store this information somewhere in order to serve it without tus.
2. When I access the files in s3 directly then they are served using the contentType:
application/offset+octet-stream
and hence the file is downloaded instead of being rendered as image which I think I can override using the contentType metadata in tus client. But is there a way to do this in tus node server automatically for all files based on original file mimetype or something?
No, content-type MUST be set to application/offset+octet-stream
for PATCH requests to the tus server, as per the protocol specification. You should have the original file name in the metadata though.
3. Is there any way I can force all the files to be stored using a custom name along with unique id and extension
Yes with namingFunction
, as you already have in your example.
4. Since I want to serve files directly from respective datastore so can i somehow return a custom url (url of cdn or s3) once the file is uploaded
Not immediately as a response from the tus server to a tus client. Although you could alter the metadata server-side to include the your CDN URL to make your client aware of it.
5. Can I simply disable the generation of metadata files along with the actual file because I want to serve directly from s3 without the head request?
They are required unfortunately.
from tus-node-server.
@Murderlon Thanks for your inputs. I took some time to try out your suggestions:
1.
Then you can use
onUploadFinish
to store this information somewhere in order to serve it without tus.
This makes sense. But the second upload param I received (with name upload
) doesn't contain information regarding storage
. Is that something I have to enable myself somewhere?
2.
- When I access the files in s3 directly then they are served using the contentType: application/offset+octet-stream and hence the file is downloaded instead of being rendered as image which I think I can override using the contentType metadata in tus client. But is there a way to do this in tus node server automatically for all files based on original file mimetype or something?
No, content-type MUST be set to application/offset+octet-stream for PATCH requests to the tus server, as per the protocol specification. You should have the original file name in the metadata though.
Without a proper content type in s3, all files are downloaded automatically when accessed instead of rendering as images or something else. This is happening obviously because the content type in s3 is binary/octet-stream
instead of something like image/png
. Is there any possible solution to fix this. Because this particular content type restriction is kind of forcing the use of tus for serving files. I can understand that tus requires content type to be binary/octet-stream
for resumable uploads but once the upload is finished it would be great to fix the content type in s3 and for the files stored in local files store.
3.
- Is there any way I can force all the files to be stored using a custom name along with unique id and extension
Yes with namingFunction, as you already have in your example.
This works and makes sense as well.
4.
- Since I want to serve files directly from respective datastore so can i somehow return a custom url (url of cdn or s3) once the file is uploaded
Not immediately as a response from the tus server to a tus client. Although you could alter the metadata server-side to include the your CDN URL to make your client aware of it.
I am not sure if I fully understand you here. Do you mean to say that while returning the response from server when the upload finishes, I just add the cdn url in the response metadata?
5.
- Can I simply disable the generation of metadata files along with the actual file because I want to serve directly from s3 without the head request?
They are required unfortunately.
Okay. Can I instead force the info files to be stored into a separate folder or is it necessary for both the files (main file and info file) to be stored side by side?
Further Question:
- As per typescript typings, the second param of
namingFunction
calledmetadata
is optional. In which scenario can that be undefined? If I want to use the combination of filename and unique id as the name with which the file will be stored then I have to read metadata to get the original filename but if the metadata is undefined in some scenario then that will be a problem.
from tus-node-server.
This makes sense. But the second upload param I received (with name
upload
) doesn't contain information regardingstorage
. Is that something I have to enable myself somewhere?
No I meant to say that tusd implements this into the upload model and tus Node.js does not (yet). So that would require a PR first before you have access. I can take a look at that.
Without a proper content type in s3, all files are downloaded automatically when accessed instead of rendering as images or something else. This is happening obviously because the content type in s3 is
binary/octet-stream
instead of something likeimage/png
. Is there any possible solution to fix this. Because this particular content type restriction is kind of forcing the use of tus for serving files. I can understand that tus requires content type to bebinary/octet-stream
for resumable uploads but once the upload is finished it would be great to fix the content type in s3 and for the files stored in local files store.
When we create an upload in S3, we set the ContentType
to whatever the content type is inside the metadata. Can you check whether your client sets the content type in meta data?
tus-node-server/packages/s3-store/index.ts
Lines 522 to 524 in a0f9da1
I am not sure if I fully understand you here. Do you mean to say that while returning the response from server when the upload finishes, I just add the cdn url in the response metadata?
You can change the status code, body, and headers in onUploadFinish
. This could be used to return your new URL in the body, for which you have to set the status_code
to 200
(theoretically not allowed in the protocol, but practically fine). The default response for PATCH is 204
which according to HTTP itself is not allowed to have a body.
const server = new Server({
// ..
async onUploadFinish(req, res, upload) {
const url = await getURLForCDN(req, res, upload)
// any headers you may want too
const headers = {}
return { res, status_code: 200, headers, body: JSON.stringify({ url }) }
},
})
Okay. Can I instead force the info files to be stored into a separate folder or is it necessary for both the files (main file and info file) to be stored side by side?
For now they are always stored together. Vast majority of use cases don't need it separate.
6. As per typescript typings, the second param of
namingFunction
calledmetadata
is optional. In which scenario can that be undefined? If I want to use the combination of filename and unique id as the name with which the file will be stored then I have to read metadata to get the original filename but if the metadata is undefined in some scenario then that will be a problem.
If the client doesn't send any metadata, you don't have any metadata on the server either. Make sure your client always sends it.
from tus-node-server.
@Murderlon Thanks again for your thorough response.
No I meant to say that tusd implements this into the upload model and tus Node.js does not (yet). So that would require a PR first before you have access. I can take a look at that.
Yeah, that would be really helpful. Thanks.
When we create an upload in S3, we set the ContentType to whatever the content type is inside the metadata. Can you check whether your client sets the content type in meta data?
Yes, this is what I meant originally:
Can we set this ContentType
inside metadata server side based on some logic instead of appending/controlling it client side? Probably in a function called onUploadCreate
or maybe some other function where we can modify this metadata before uploading the file to S3?
from tus-node-server.
Can we set this
ContentType
inside metadata server side based on some logic instead of appending/controlling it client side? Probably in a function calledonUploadCreate
or maybe some other function where we can modify this metadata before uploading the file to S3?
Yes you can:
const server = new Server({
// ..
async onUploadCreate(req, res, upload) {
const contentType = await extractContentType(req, res, upload)
const metadata = { ...upload.metadata, contentType }
return { res, metadata }
},
})
from tus-node-server.
Related Issues (20)
- CRC errors using @google-cloud/storage 7.7.0 HOT 2
- Mixed content error - Page load over https buyt requested url is insecure HOT 1
- @tus/s3-store: Doesn't work with R2 HOT 1
- S3-store doen't work with Cloudflare R2 storage HOT 2
- Respect X-Forwarded-Port and X-Forwarded-Prefix as well HOT 2
- serverless integration HOT 1
- Upload fail after 10min request HOT 3
- Error installing 1.4.0 from npm HOT 2
- tus-node-server and ietf HOT 11
- Simultaneous file upload leads to corrupted files HOT 2
- Is there a way to pass variables between the `onUploadCreate` and the `onUploadFinish` functions? HOT 12
- Allow hooks to override metadata HOT 1
- Recommended nginx configuration HOT 2
- How to get final name back in client? HOT 4
- [S3Store] 0 bytes files are not uploaded HOT 1
- onUploadFinish should be able to alter response body HOT 5
- "Store files in custom nested directories" example fails with can't patch error HOT 10
- @tus/[email protected] breaking upload. 404 error HOT 8
- S3store creating extra .info file HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from tus-node-server.