See here for more info

Maybe store the lastReindexTime on the second index.

Show status of all nodes on radiator, alert on non-green status.

Dropping an image on the trail doesn't get you any text to edit down as alt text if the title is missing. It should probably use the description instead in that case.

Description is usually mapped to caption, but there's no caption for thumbnail so the copy is not passed through.

Likely to be required by pan-domain auth.

Why not?

Let's make sure there are no secrets or pseudo-secrets first.

Doesn't even seem to startup correctly in DEV anymore?

Given how much we care about image metadata and successful ingestion, they would give us confidence and a way to regression test the behaviour on a growing sample of images.

Will also make it easier to figure out metadata extraction for other formats in the future.

Should fail if not a 5:3, and perhaps also if doesn't contain a 140-wide crop? There should be similar logic already, maybe we're just mistakenly bypassing it.

Theoretically a Composer issue but put it here for visibility.

Currently Content-Type: application/octet-stream which makes them hard to open in your browser, and is wrong.

When ready.

This will expose all services under a local.dev-gutools.co.uk domain, simplify CORS with Composer (once Composer also adopts it), etc.

Need to ensure that we retain the ability to auth by API key for apps.

On the image and crop screen, we load the original image file, which can be large. This is slow unless you're on a fast connection.

We should explore dropping in a dynamic resizer to dramatically reduce the file size.

The 3-tries policy doesn't seem to be respected, which makes the loader error logs a lot noisier than they should be.

No reason to keep them if we cull the original image.

Having our config come from Play is not a great dependency.

There is a good discussion going on about it with a few solutions / hacks.

Test if ingestion of CMYK is supported and if not, do what is necessary to support them (maybe passing -colorspace RGB to ImageMagick, suggests @blishen?).

For interest / graphing's sake

Refused to display 'https://accounts.google.com/AccountChooser?service=lso&continue=https%3A%2F….com%26hl%3Den%26from_login%3D1%26as%3D759d38844d7b4d3&btmpl=authsub&hl=en' in a frame because it set 'X-Frame-Options' to 'DENY'.

For instance we sometimes end up with 599px when we would expect 600px, or 2000x1202 when it should really be 2000x1200.

Let's review the way dimensions rounding works and see if this can be tidied up.

Also affects frontend that are particularly strict on dimensions.

Currently only shows the count up to 50 and loads the last 50 images. Should load all the images since the initial search.

Cheap optimisation

Should probably also include a cleanup of the produced crop.

This will make deployments more stateless and resilient (e.g. our deploys were failing due to a broken transitive python dependency).

e.g. see how membership or workflow have been doing it.

Join the hipster bandwagon

e.g. currently, the search query is lost when coming back from an image page. We could investigate angular routers to maintain such state in the URL.

Alarms didn't work today because the email wasn't verified. Haven't found a way to verify it, and generally it seems that there's a new better way to do it using an HTTPS endpoint:

http://www.pagerduty.com/docs/guides/aws-cloudwatch-integration-guide/

We just need to replace the SNS email subscription with the correct HTTPS endpoint, somehow passing in that to the UpdateStack script as a parameter.

We should also:

• Ensure the alarms are correctly configured for each environment (all seem to point to PROD atm)
• Test that it actually works.
• Maybe fix the UpdateScript to allow not passing in the panda and alarm parameters every time

Currently reloading thumbnails and original assets every time, which can be expensive on a slow connection

It's inefficient and much slower to load all the sources and transpile them in the browser as we do currently.

Let's keep this as a DEV workflow, but compile, bundle and minimise all assets into a bundle file that can be efficiently loaded and cached.

Metrics are here

On occasion we will need to add / modify mappings.

To add a field we can just PUT it into the mappings, but we need to make sure we do this before data is added.

To modify mappings we can follow [ES's advice on reindexing / aliasing].
CAPI have a good, albeit incomplete, tools for this. We should look into generifying this to be able to use it.

The CSS is becoming WET so we should look at DRYing it.

• decide on pre-processor

No reason to send it, and it bloats the response.

We currently store unknown tags in fileMetadata, even though we do nothing with them and don't know what they are.

We should review if we can make sense of any of it, but generally should be ignored unless standard.

i.e.
https://media-service-dev-jgorrie-imageoriginbucket-y7ur5ovu2chv.s3-website-eu-west-1.amazonaws.com/f4d1e4727822cae658df13af99089626772ce36c/213_1977_1094_657/140.jpg

Not

http://media-service-dev-jgorrie-imageoriginbucket-y7ur5ovu2chv.s3-website-eu-west-1.amazonaws.com/f4d1e4727822cae658df13af99089626772ce36c/213_1977_1094_657/140.jpg

It should be provided by the client, e.g. "5:3" and passed through in the response so that the client can record it if necessary.

Currently, there's no way to see more results

That way it gets correctly mounted again if the box is restarted for some reason.

Applying this fix should do it

There seems to be a problem with Drew's image parsing of JPGs.

We are then throwing these images away, and this has proved, in regards to Australia, to be a problem as the images they're looking for are bust, thus not showing up.

A solution might be to move to another JPG reader, not really sure what else we can do.

Since our last image upload server went up until now (November 13, 2014 2:44:28 PM UTC - November 24, 2014 17:00:00 PM UTC - 266 Hours) we've had 8251 of these errors - which seems significant.

Requested by [email protected] (and likely many others):

Is there - or could there be - a way to search pic desk/library only for pictures with a green tick? I appreciate pic editors need to see the full spread, but when liveblogging, for example, I want to be able to search only for pictures I can actually use, and not have to click through pages of red/orange-flagged ones first, which really slows me down. Would be great if possible!

This would help making it possible to sync all files onto a backup disk without hitting annoying fs limits

See snapshotter

For instance we're missing byline title, date taken, etc. as well as some location-related metadata (province, etc). We should go over all the fileMetadata fields and ensure they are mapped to a canonical entry in the metadata so it can be indexed, searched, displayed, etc.

If there is only one result, or if there are less than the maximum number of results to fill the last row, thumbnails expand to fill the width. This breaks the nice grid, and also in the case of the single result makes the cost info float oddly at the bottom.

Can we make the flex items always fit to the grid regardless? I haven't look at the use of flexbox in details so let's not waste too much time if it's a pain.

PS: I've had a few chats yesterday and the solution would really be CSS3 grids, but sadly they're only supported in IE for now, so we have to wait a little longer...

The crop button is now hidden for invalid images, but it's still possible to do a crop as it's not enforced by the server.

The cropper should refuse to crop or export invalid images.

To maintain data integrity for #305

This is for when we want to add to the ES mapping - for that we don't have to reindex.
For the time being we're using the ES UI.

This could be automatically run on build perhaps?

http://www.vips.ecs.soton.ac.uk/index.php?title=Libvips

May be able to take in image stream?

It looks like the cropper isn't correctly computing the target dimensions when crop+resizing in portrait aspect ratio, e.g.

http://media-origin.test.dev-guim.co.uk/n5ljadkfwkqc46lowugw6qatce/0_0_3372_5058/1000.jpg
(note the 1000, but it's neither 1000 tall or wide)

Add an overlay to make it clear that dropping files onto the search results uploads them.