A Complete File Upload API for AWS S3
Evaporate is a JS library for uploading files from a browser to AWS S3, using parallel S3's multipart uploads with MD5 checksum support and control over pausing / resuming the upload.
Table of Contents
We're in the final stages of migrating the library to Typescript and Webpack, and we're doing it to increase the maintainability of the project, but we also had reports of increased performance and lower memory usage!
The new version will foster an increase in the ease of contributing and onboarding of new maintainers.
But don't worry, as there were no contract changes, if you're using our v2 it should work out of the box.
To test it, it's very simple, you just have to install the library like this:
npm install evaporate@TTLabs/EvaporateJS#pull/448/headAnd that's it! It should immediately work. If you have some feedback about it, please post it here.
-
Configurable number of parallel uploads for each part (
maxConcurrentParts) -
Configurable MD5 Checksum calculations and handling for each uploaded part (
computeContentMd5) -
Pluggable signing methods with
customAuthMethodto support AWS Lambda, async functions and more.
-
S3 Transfer Acceleration (
s3Acceleration) -
Robust recovery when uploading huge files. Only parts that have not been fully uploaded again. (
s3FileCacheHoursAgo,allowS3ExistenceOptimization) -
Ability to pause and resume downloads at will
-
Signing methods can respond to 401 and 403 response statuses and not trigger the automatic retry feature.
-
AWS Signature Version 2 and 4 (
awsSignatureVersion)
-
Reduced memory footprint when calculating MD5 digests.
-
Parallel file uploads while respecting
maxConcurrentParts. -
If Evaporate reuses an interrupted upload or avoids uploading a file that is already available on S3, the new callback
nameChangedwill be invoked with the previous object name at the earliest moment. This indicates that requested object name was not used.
-
The
progress()andcomplete()callbacks provide upload stats like transfer rate and time remaining. -
Pause, Resume, Cancel can act on all in-progress file uploads
- Support for Node.js FileSystem (fs) ReadbleStreams. This means you can use Electron to upload a file directly from the file system's native File picker and avoid the usual browser restrictions.
$ npm install evaporateThe documentation for the usage of the whole API is available here.
This is a simple example of how you can configure it:
const Evaporate = require('EvaporateJS');
const Crypto = require('crypto');
const config = {
signerUrl: SIGNER_URL,
aws_key: AWS_KEY,
bucket: AWS_BUCKET,
cloudfront: true,
computeContentMd5: true,
cryptoMd5Method: data => Crypto
.createHash('md5')
.update(data)
.digest('base64');
};
const uploadFile = evaporate => {
const file = new File([""], "file_object_to_upload");
const addConfig = {
name: file.name,
file: file,
progress: progressValue => console.log('Progress', progressValue),
complete: (_xhr, awsKey) => console.log('Complete!'),
}
/*
The bucket and some other properties
can be changed per upload
*/
const overrides = {
bucket: AWS_BUCKET_2
};
evaporate.add(addConfig, overrides)
.then(
awsObjectKey =>
console.log('File successfully uploaded to:', awsObjectKey),
reason =>
console.log('File did not upload sucessfully:', reason);
)
}
return Evaporate.create(config).then(uploadFile);More examples are available here.
Don't forget to check out the Browser Compatibility and Important Usage Notes pages for usage details.
- Bobby Wallace - @bikeath1337
- Tom Saffell - @tomsaffell
- Jakub Zitny - @jakubzitny
- Matheus Moreira - @mattmoreira
Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.
Please make sure to update tests as appropriate.
This package is licensed under the BSD 3-Clause license