Giter VIP home page Giter VIP logo

360-image-uploader's Introduction

360-image-uploader

A collection of lambdas to upload, unzip and store in DynamoDB The Science Museum's 360 image files (.ggpkg format).

Deployment

We're using serverless to handle the deployment, so everything is handled in our serverless.yml file.

See here to install serverless to your machine.

To deploy, simply run:

sls deploy --bucket {bucket-name} --region {aws-region} --stage {stage-name}

All three of the flags are optional, they will default to the following if not included:

  • bucket - smgco-360
  • region - eu-west-1
  • stage - dev

To take down the services, run the exact same command, but replace deploy with remove.

sls remove --bucket {bucket-name} --region {aws-region} --stage {stage-name}

If you have anything in the {bucket-name} s3 bucket, the removal may fail, and you may have to delete the remaining resources manually.

To do this, go to s3, empty the bucket and delete it. Then go to CloudFormation and delete the CloudFormation stack. It will be called smgco-360-uploader-{stage-name}

Accessible Endpoints

/ - Returns an html form to be used to upload .ggpkg files to s3.

/list-objects - Returns an html table of all the 360 files in s3.

/list-objects?format=json - Returns a json list of all the 360 files in s3.

360-image-uploader's People

Contributors

danwhy avatar jamieu avatar nelsonic avatar

Stargazers

 avatar Sophie Levens avatar Ines Teles Correia avatar

Watchers

 avatar James Cloos avatar John Stack avatar Ines Teles Correia avatar Simon avatar Angela Wolff avatar  avatar  avatar  avatar

360-image-uploader's Issues

Extra fields to parse (if we have time)

Focus on the existing issues as a priority. But if you get time / it's easy would be good to also parse out the backgroundcolor and copyright fields (both are optional fields)

We're not going to use the fields immediately, it's more covering off edge cases, so don't jeopardise getting the other more important issues resolved first. But if you have item and it's just a case of cut/pasting some extra fields in the serveless .yaml file it would be a nice to have.

Both fields are optional and most of the time will be empty.

  • backgroundcolor="0x000000"
  • copyright="© The Board of Trustees of the Science Museum"
  <view>
    <start row="0" column="0" state="0"/>
    <zoom default="1.000" centerx="0.00000" centery="0.00000" min="1" max="2.000"/>
    <viewer imagescaling="1" backgroundcolor="0x000000" background="1"/>
  </view>
  <autorotate speed="1.000" delay="5.00" start="0"/>
  <userdata title="Model of the ‘Optimus’ water closet," datetime="" description="" copyright="© The Board of Trustees of the Science Museum" author="" source="" comment="" info="1961-32"/>

Lambda function to receive POST request

We'll need a Lambda function to receive the POST request from our form. It then needs to:

  • unzip the .ggpkg
  • verify the xml file and its contents
    • title
    • info - which is the object ID
  • upload unzipped file to S3 using title and Id to create name

High level requirements

The Science museum have a number of 360 images that they need to keep track of in order to use on the Collections Online site. To facilitate this, we will need to provide a way for the images to be uploaded to AWS S3, keep track of those that have, and provide a record of the images to reference from Collections Online.

  • Host a html form using AWS S3/Lambda for uploading 360 images to S3
  • Lambda function to store 360 image details in DynamoDB
  • Lambda functions to list/delete 360 images
  • Lambda function to retrieve json representation of 360 images

Serverless configuration tasks

  • prefix all resources with 'smgco-''

  • tag all resources with 'System=CollectionsOnline' (if possible)

  • make uploaded S3 resources public

  • make forms/handlers private (@jamieu can setup an IP restricted IAM)

  • show error if no id or title (ideally on upload form, but if not just go to list and show error)

  • increase Lambda timeout from 6 secs

  • JSON feed of what's in bucket (id : path)

  • add modification_date to JSON feed

  • set individual IAM permissions per Resource

Serverless spike

The deployment of the numerous different AWS services for this project are likely to be very time consuming. To hopefully ease this somewhat we're going to spend some time looking into the Serverless framework

Post review feedback and amends

  • [JU] Check that new updates to codebase don't overwrite/clear existing data.
  • Make smg-360 bucket publicly accessible (with open CORS headers)
  • Add a link to list view on upload page (or create a new 'index' page that links to both)
  • Add a refresh button to the 'list' view
  • Show modified date in list view
  • Swap order of name and id header column names in the list view (currently the wrong way around)
  • Nicer error messages if no Object ID or Title
  • Allow use to see failed uploads on the 'lists' view and to delete/clear them from the system at that point (basically so we don't end up with half broken folders/uploads in S3 and stale data kicking around).
  • Handle duplicates gracefully ie.user uploads a 360, makes it inactive, user uploads a new 360 with same ID weeks later that they do want live. Each object ID can only have one 360. Maybe force the user to delete--or make inactive--the old upload first?
  • Show a small thumbnail of the preview.jpg image at the root of each folder https://s3-eu-west-1.amazonaws.com/smgco-360/a5f9721b-5233-4e43-a696-00c55ade9d51/preview.jpg
  • Add access to JSON feed to AWS resources owned by this account (or I could just add in another IP address?)
  • Hide/toggle display of 'inactive' 360 records more than a week old and have a toggle to 'show all' (could load them all and just hide the inactive using JS/CSS if easier?)

Create list of uploaded images

Create a Lambda function that will:

  • create a list of all uploaded images
  • make it possible to expire/disable/archive images (not delete them)

Optimise uploaded images

Not essential, but nice to have:
The object2vr software doesn't do a great job of minimising file size on it's generated images.
Savings of 25% on it's generated jpg's can easily be made by running them through imageoptim tools without affecting quality.
Could be done as Lambda function

Authorisation

Currently we have the static html form publicly accessible.
When we have the image upload functionality, we will have to make sure this has some kind of authorisation to ensure that anybody cannot upload files.
A solution may be to make the file accessible only to the Science Museum internal network.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.