Giter VIP home page Giter VIP logo

wt-js-sdk's Introduction

WeTransfer JavaScript SDK

npm version Maintainability Test Coverage CircleCI

The JavaScript SDK provides convenient access to WeTransfer's Public API.

User guide

Our user guide includes information on different topics, such as:

Installation

Install the SDK with:

npm i @wetransfer/js-sdk --save

Getting started

In order to be able to use the SDK and access our public APIs, you must provide an API key, which is available in our Developers Portal.

This is the bare minimum code needed to create a transfer. Copy and paste into a file, place your API Key there, and run with node path/to/file.js. Voilà, you just created your very first transfer!

const createWTClient = require('@wetransfer/js-sdk');

(async function() {
  // An authorization call is made when you create the client.
  // Keep that in mind to perform this operation
  // in the most suitable part of your code
  const wtClient = await createWTClient('/* YOUR PRIVATE API KEY GOES HERE*/');

  const content = Buffer.from('Look ma, a transfer!');
  const transfer = await wtClient.transfer.create({
    message: 'My very first transfer!',
    files: [
      {
        name: 'hello.txt',
        size: content.length,
        content: content
      }
    ]
  });

  console.log(transfer.url)
})();

Transfers

Built to get files from one place to the other, this is the classic WeTransfer experience. Send it up to 2GB of files per transfer and this thing will handle it with ease, with a built-in 7 day expiry.

Create a transfer

Transfers must be created with files. Once the transfer has been created and finalized, new files cannot be added to it. When creating a transfer you must provide a list of files you want to transfer. Files must include name (unique) and size (in bytes) properties.

const transfer = await wtClient.transfer.create({
  // Message is optional
  message: 'My very first transfer!',
  // Files are mandatory. Must include file names (unique!) and file sizes, in bytes.
  files: [
    {
      name: 'hello.txt',
      size: 1024
    },
    {
      name: 'big-bobis.jpg',
      size: 13370099
    }
  ]
});

As you can see, the content of the files has not been included as a part of the payload. The contend is not a required property because depending on your use case, you will have access to the file content when the transfer is created, but in other cases, this can be separate process.

These are some possible options:

  1. You are writing a CLI tool, where you know where the files are located, our you just expect paths to files that you need to read before you create the transfer. If that's the case, the content will be a Buffer, returned by fs.readFile method. The SDK will take care of the file upload process, we got you covered. Your code will be something like this, provided you know the path of the file:
// Create a promise-based function to read files.
function readFile(path) {
  return new Promise((resolve, reject) => {
    fs.readFile(path, (error, data) => {
      if (error) {
        return reject(error);
      }

      resolve(data);
    });
  });
}

// This is variable, and will depend on your application.
const filePaths = ['path/to/file.txt', 'path/to/image.png'];

// Read the content of the files, in parallel
const fileContents = await Promise.all(filePaths.map(readFile));

// Create the files array with names, sizes and content.
const files = filePaths.map((file, index) => {
  const content = fileContents[index];
  return {
    name: file.split('/').pop(),
    size: content.length,
    content: content
  };
});

const transfer = await wtClient.transfer.create({
  message: 'My very first transfer!',
  files: files
});

console.log(transfer.url); // https://we.tl/t-Sa7dYYlOdF
  1. If you are trying to create a transfer from the browser, please make use of the File Web API, which gives you access to file information and content. Be aware that using the SDK directly in the browser, will expose your API Key to the wild, and this is not desired. Considering that you have an input to upload multiple files:
<input type="file" id="files-input" multiple />
const filesElement = document.getElementById('files-input');

// Create a transfer everytime files are selected.
filesElement.addEventListener('change', async (event) => {
  const fileList = event.target.files;

  const files = fileList.map((file) => {
    return {
      name: file.name,
      size: file.size,
      content: file
    };
  });

  const transfer = await wtClient.transfer.create({
    message: 'My very first transfer!',
    files: files
  });

  console.log(transfer.url); // https://we.tl/t-Sa7dYYlOdF
});
  1. A proper solution for the previous example would be to have a client/server application where your API Key is not exposed in the browser. This is so you can control which clients can create transfers based to CORS settings, for example. That requires a more complicated setup, but it's the best solution both in terms of security and performance. The process is as follows:

    1. Create a transfer, specifing only file names and size.
    2. Request upload URLs for each part of the file.
    3. Complete the file upload.
    4. Repeat 2 and 3 for each file.
    5. Complete the transfer. The final WeTransfer URL will be returned here.

    Please check this example using vanilla JS which shows how to upload chunks.

Find a transfer

If you have a transfer id saved from previous steps, you can retrieve the transfer object and files information:

const transfer = await wtClient.transfer.find('/* your transfer_id */');
console.log(transfer.url); // https://we.tl/t-Sa7dYYlOdF

Boards

Our Board API is in line with our new mobile app. It can store traditional files as well as links, and is flexible in how it displays it all.

Create an empty board

Boards are the latest addition to our Public API. It was built with our iOS and Android apps in mind, but it's also suitable for web/desktop users. It is designed for collecting content rather than transmitting content from A to B (though it can do that, too) and it supports both files and links. Boards are created emtpy, without files or links. Imagine a board like an empty canvas where you can add items at any time.

const board = await wtClient.board.create({
  name: 'My very first board!',
  // Description is optional.
  description: 'Something about cats, most probably.'
});

console.log(board.url) // https://we.tl/b-oR7ufV43ZS

As you can see, boards are created without items, but you can already access the public URL. It's time to add some items!

Add links to a board

Once a board has been created you can then add links to it. Please provide a complete URL and the title od the page.

const links = await apiClient.board.addLinks(board, [{
  url: 'https://en.wikipedia.org/wiki/Japan',
  title: 'Japan - Wikipedia'
}, {
  url: 'https://en.wikipedia.org/wiki/Netherlands',
  title: 'Netherlands - Wikipedia'
}]);

console.log(links);
console.log(board.links);
// [{
//   id: 'sj63ugt996w8b4c1v20180913113842',
//   url: 'https://en.wikipedia.org/wiki/Japan',
//   meta: {
//     title: 'Japan - Wikipedia'
//   },
//   type: 'link'
// }, {
//   id: 'sj63ugt996w8b4c1v20180913113843',
//   url: 'https://en.wikipedia.org/wiki/Netherlands',
//   meta: {
//     title: 'Netherlands - Wikipedia'
//   },
//   type: 'link'
// }]

links contains the list of links added to the board, with some extra information like an unique id or extra meta information, if available.

Add files to a board

Once a board has been created you can then add files to it. Unlike transfers, files can be added at any time, one by one, in batches, it will depend on your use case.

const files = await apiClient.board.addFiles(board, [{
  filename: 'kittie.gif',
  filesize: 1024
}]);

From here, the process to upload files is the same as for transfer. If you already have the content of the file, you can pass it as an extra property named content, we will take care of it.

Please check this example using vanilla JS which shows how to add items to a board.

Retrieve a board

If you have a board id saved from previous steps, you can retrieve the board object and files information:

const transfer = await wtClient.board.find('/* your board_id */');
console.log(transfer.url); // https://we.tl/b-Sa7dYYlOdF

Note that unlike transfers, boards are can always be modified and only expire if not interacted with in 90 days, so you can use this method to retrieve and update a board.

Retry network requests

We intercept failed requests and retry them whenever possible. But default, we retry each request 15 times using exponential backoff. Both parameters can be configured when creating the client:

const apiClient = await createWTClient('/* YOUR PRIVATE API KEY GOES HERE*/', {
  retries: 5,
  retryDelay: (retryCount) => retryCount * 1000,
});

Logging levels

Logging levels in this SDK conform to the severity ordering specified by [RFC5424]: severity of all levels is assumed to be numerically ascending from most important to least important.

Each level is given a specific integer priority. The higher the priority the more important the message is considered to be, and the lower the corresponding integer priority.

{
  error: 0,
  warn: 1,
  info: 2,
  verbose: 3,
  debug: 4,
  silly: 5
}

We make use of the npm levels showed above.

Setting the level for your logging message can be done providing the value when creating WeTransfer. For example, using the warn level you could log warn messages and below to the console, which in this case will only be warn and error.

const apiClient = await createWTClient('/* YOUR PRIVATE API KEY GOES HERE*/', {
  logger: {
    level: 'debug'
  }
});

If no value is provided, by default we will use info.

Documentation

Visit https://developers.wetransfer.com/documentation to access the latest API related documentation.

Development

After checking out the repo, run yarn to install all dependencies. To run all tests:

$ npm test
$ npm run test:watch

Release process

First, make sure that you have a NPM account at https://www.npmjs.com/, and you are part of the WeTransfer developer's team. Use npm login to store the credentials on the client aka, your computer. Check that your authentication token for registry.npmjs.org is part of your ~/.npmrc file.

We use semantic-release to manage release process. Please run npm run release:dry to check relevant changes and possible new versions. If you are happy with it, run npm run release, it should do the following:

  • Verify authentication for registry
  • Verify GitHub authentication
  • Find latest release and associated git tag
  • Find last commits since last release
  • Generate release notes
  • Create Git tag
  • Prepare the package and release it 📦

Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/wetransfer/wt-js-sdk. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the Contributor Covenant code of conduct.

License

The package is available as open source under the terms of the MIT License.

Code of Conduct

Everyone interacting in the WetransferJSSdk project’s codebases, issue trackers, chat rooms and mailing lists is expected to follow the code of conduct.

wt-js-sdk's People

Contributors

arkaitzgarro avatar dependabot[bot] avatar olokobayusuf avatar renovate[bot] avatar semantic-release-bot avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

wt-js-sdk's Issues

Add an upload progress callback

When uploading files, it's nice to have a progress callback every time a chunk has been uploaded.

A callback can be registered when the transfer is created, and it's called every time a chunk is uploaded on any of the files. Something like that:

function callback({transfer, progress}) {
  // Where transfer contains the information about the transfer, like id, etc
  // And progress contains the upload progress for each file and for the total
  progress = {
    uploaded: 1234, // bytes uploaded,
    total: 12345, // total bytes for transfer
    files: [
      { ...fileInfo, uploaded: 123, total: 1234},
      { ...fileInfo, uploaded: 123, total: 1234}
    ]
  }
}

const transfer = await apiClient.transfer.create({
  name: 'My very first transfer!',
  // Description is optional.
  description: 'Something about cats, most probably.'
}, callback);

Or pass the callback as part of an options object:

const transfer = await apiClient.transfer.create({
  name: 'My very first transfer!',
  // Description is optional.
  description: 'Something about cats, most probably.',
  options: {
    progress: callback
  }
});

Implement deleting of files and links of a board

Is your feature request related to a problem? Please describe.
In the near future, the API allows deleting files and links of a board. The Javascript SDK should be able to handle that as well

Describe the solution you'd like

Describe alternatives you've considered

Additional context

See related issue: WeTransfer/wt-api-docs#96

Handle duplicated file names

Describe the bug
When providing two files with the same filename, the create Transfer process fail, with the following error:

RuntimeError: Response error: "NuBackend::Utils::NotFound: No such key or download error - '2aeb2197128a0e92a61f371813fb2eb820181016174309/wt-multipart-meta` in bucket 'wetransfer-eu-prod-outgoing'", code 500 talking to http://wetransfer:************@production.backend.service.eu-west-1.wt:9292/api/v1.rpc

The SDK should not allow duplicated filenames.

Input Code

const client = await createWTClient('******', {
  logger: {
    level: 'debug'
  }
})

const transfer = await client.transfer.create({
  message: 'This is my transfer',
  files: [
    { name: `file.txt` },
    { name: `file.txt` },
  ]
})

Expected behavior
Ignore one of the duplicated files or throw an error so the user can react accordingly.

Environment

  • SDK version(s): v1.0.0
  • Node/npm version: v8.12.0
  • OS: OSX 10.12.6

Possible Solution
Lodash could help here:

_.uniqBy(transfer.files, 'name');

Normalize error handling

Normalize errors from requests, data validation, etc.

const error = {
  message: 'Descriptive error',
  date: new Date(),
  response: {} // For network errors
};

Deal with rate limits when sending requests to Public API

Is your feature request related to a problem? Please describe.
The Public API throttles the requests, throwing a 429 error if the limit is reached at some point. Right now, we just pass the error to the client, but we should take care of it and retry the request again.

Since we use axios as a request library, we could write an interceptor or use an already existent library like axios-retry. Either way, we should give the client the possibility to configure the number of retries and the delay between retries.

Describe the solution you'd like
The SDK should retry at least 429 errors returned by the API, and most probably failures for network timeouts.

Find a better solution for logging

console.log it's fine for now, but we should find a better solution for logging the output messages.

Requirements:

  • We must be able to disable the output, and allow de user to decode if want some output or not.
  • Levels of verbosity
  • Tags and timestamps, maybe.

Retry chunk upload if something goes wrong

Is your feature request related to a problem? Please describe.
If the upload of a file chunk fails (the network request fails for example), we don't try to upload again, even it is one of the advantages of splitting the file in small pieces.

Describe the solution you'd like
If a chunk upload fails, we should retry X times.

Related to #107

Not enought multipart_parts

Hello, I'm new in node. I tried your code and each time my file exceeds 30MB, there are not enought multipart_parts to reach the file size. For example, if I want to upload a file of 104 857 600 Bytes, I only get 17 multipart_parts of 5242880 Bytes each (MIN_CHUNK_SIZE) and get the following error

"Expected 89128195 to be 104857600"

Expected behavior
I expected to get as much as multipart_part as I needed. Is that a restriction on the file size?

Environment

  • SDK version: v0.4.0
  • Node/npm version: Node 10.3.0/npm 6.1.0
  • OS: Windows 10

Thank you.

Add function to download a file from a transfer URL

Is your feature request related to a problem? Please describe.
It's not a problem, but a worthwhile feature request. This API is one-way, only offering ways to upload files. It'll be great to also have a way to download a file to a buffer. People have done it, but it'll be great to have in the official SDK like Dropbox.

Describe the solution you'd like
Something along the lines of:

const downloadMetadata = await WeTransfer.transfer.downloadTransfer({ url: ... });
// Do stuff with `downloadMetadata.buffer/contents`

Describe alternatives you've considered
Using a hacky solution, like the link I referenced earlier.

Additional context
Add any other context or screenshots about the feature request here.

Typescript Typings

Is your feature request related to a problem? Please describe.
Not a problem. Typescript is pretty common nowadays, so it'll be great to have typings for it.

Describe the solution you'd like
typings.d.ts

Describe alternatives you've considered
Using the SDK without typings in TypeScript.

Additional context
None.

Transfer API uploading files in wrong order

Describe the bug
After upgrading to the new v1 SDK I wanted to use the Transfer API because I prefer the non-boards download page. But I noticed all my transfers were processing for a long time and then suddenly deleted. After some research I determined that the files are uploaded incorrectly, resulting in a broken transfer. The order of the remoteTransfer.files and transfer.files arrays are not the same, causing a mismatch during the file upload. In my testing so far (only having 1 part chunks) there is no error, it simply returns a shortened url that looks like it is processing for a really long time until finally the transfer is deleted.

Input Code

// create wetransfer client
const client = await createWTClient('******', {
  logger: {
    level: 'debug'
  }
})

// create files array
const files = images.map(image => ({
  name: image.filename,
  size: image.filesize,
  content: image.buffer
}))

// create transfer
const transfer = await client.transfer.create({
  message: 'This is my transfer',
  files
})

Expected behavior
I expect the transfer to result in a downloadable transfer.

Environment

  • SDK version(s): v1.0.0
  • Node/npm version: Node v8.9.0 / NPM v5.5.1 / Yarn v1.7.0
  • OS: MacOS 10.14

Possible Solution
This is the line where things go awry, because it is iterating the remoteTransfer.files and using that index to access the file contents with transfer.files[index].content. However since they are in a different order the wrong file is uploaded.

One quick fix could look like this, however this does assume filenames are unique and the remoteTransfer.files contains identical filenames (which it might not, considering input sanitation for example).

transfer.files.find(f => f.name === file.name).content

Additional context/Screenshots
I discovered this while trying to figure out why my transfers kept failing, when I noticed the size suddenly changing when uploading parts.

[WeTransfer SDK] [IMG_090821.jpg] Starting file upload.
[WeTransfer SDK] [IMG_090821.jpg] Splitting file into 1 parts. Total size to upload: 48885 bytes.
[WeTransfer SDK] [IMG_090821.jpg] Part #1 of 1. Bytes from 0 to 127314.

File hashes

Is your feature request related to a problem? Please describe.

When uploading files, to ensure data integrity, it would be great to be able to provide a hash (either for each chunk, or the file as a whole), and for that hash to be checked at the receiving end. If the hash doesn't match, an error should be thrown, so can retry sending.

Additional context

I would be happy to implement this, if it were supported by the WeTransfer API.

I assume that files are hashed in storage anyway, so this would be a matter of exposing that info through the API.

Connection not working

I added this code and I get this error message: wetransfer.min.js:126 Uncaught (in promise) WTError: The authorization call failed and no usable JWT could be found there. Please check your API Key.

Input Code

const createWTClient = require('@wetransfer/js-sdk');

(async function() {
    const apiClient = await createWTClient('vKDFBOB7LQ1s8O64Rri9I7UgbHWyHXZP5YCUa1GP', {
        logger: {
            level: 'silly'
        }
    });

    const transfer = await apiClient.transfer.create({
        name: 'My very first transfer!'
    });
    console.log(transfer);
})();

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.