s3-media-sync's Introduction

S3 Media Sync

This plugin syncs the uploads directory of a VIP Platform WordPress environment to an AWS S3 instance.

Props to S3-Uploads and Human Made for creating much of the functionality: https://github.com/humanmade/S3-Uploads

Setup

Build the plugin

This plugin uses composer as a package manager. After downloading the plugin (as a ZIP file or via git pull) run one of the following commands:

For production: composer install --no-dev --optimize-autoloader
For development: composer install

Running one of the above commands will create a vendor directory which is required for the plugin to function correctly. Applications that are using CI/CD already run one of these commands automatically and can skip this step.

Activate the plugin

Commit the plugin to your application's plugins directory.
Activate the plugin through code or within the WordPress Admin dashboard.
Create an IAM user with Programmatic Access.
Enter the provided AWS S3 API keys on the plugins's Settings page.
Backfill the uploads directory on AWS by running the following command:

wp s3-media upload-all --url=example-site.com

FAQ

How can I upload media to a subdirectory in S3?

As an example, you already have a bucket named my-awesome-site but you want all of your media to go into a preprod subdirectory of that bucket. To configure media to upload to that subdirectory, go to the S3 Media Sync settings page and enter the following for the S3 Bucket Name field:

my-awesome-site/preprod

Then, all media will automatically be kept in-sync within my-awesome-site/preprod/wp-content/uploads.

How can I confirm if all of the attachments were uploaded?

You can check which attachments were skipped by running the following command:

wp vip migration validate-attachments invalid-attachments.csv --url=example-site.com

The generated log file will be available at invalid-attachments.csv. The full command can be found here:

https://github.com/Automattic/vip-go-mu-plugins/blob/master/wp-cli/vip-migrations.php#L165-L187

Changelog

1.2.0

Update Composer dependencies for PHP 8.0 compatibility.

1.1.0

Fix: Upload images edited within WordPress to the bucket.

s3-media-sync's People

Contributors

Stargazers

Watchers

s3-media-sync's Issues

Add WP_CLI subcommand to upload single attachment

It would be great if we can add another subcommand to upload a single attachment only, like

wp s3-media upload <attachment_id>

The benefit would be that, based on a known list of attachment IDs that we need to upload, we could combine WP_CLI with xargs to upload each attachment using parallel threads giving us much more refined control on the execution of the upload process.

Images edited with the WordPress admin aren't uploaded to the S3 bucket.

After editing an image in the WordPress admin, the image is not copied to the S3 bucket.

Allow custom endpoints.

As an alternative to S3 it would be great if we could allow additional endpoints. Human Made does this with the s3_uploads_s3_client_params filter](https://github.com/humanmade/S3-Uploads/blob/master/inc/class-s3-uploads.php#L272) which I think we can add here:
https://github.com/Automattic/s3-media-sync/blob/master/inc/class-s3-media-sync.php#L302

Let me know if this makes sense, then all one has to do to change the endpoint is:

// Filter S3 Uploads params.
add_filter( 's3_uploads_s3_client_params', function ( $params ) {
	$params['endpoint'] = 'https://your.endpoint.com';
	$params['use_path_style_endpoint'] = true;
	$params['debug'] = false; // Set to true if uploads are failing.
	return $params;
} );

Dependency conflict with other plugins

We noticed that UpdraftPlus is unable to connect to AWS when this plugin is active. It turns out UpdrafPlus also bundles the AWS PHP SDK, but requires a much older version of the package. In order to prevent conflicts like this it would be helpful to namespace the dependencies.

Recommend Projects

automattic / s3-media-sync Goto Github PK