jncc / deli Goto Github PK

• Update to boto3
• Use .aws credentials instead of password in script
• Reference the geojson grids on the fileshare instead of locally
• Change .las files to .laz
• Update the metadata links

We really should add an attribution statement to the leaflet map, for the baselayer and for the data itself

Comes from the CSS, currently from a CDN https://api.mapbox.com/mapbox.js/plugins/leaflet-fullscreen/v1.0.1/fullscreen.png

We need to allow people to opt out of cookies from the deli sites

Looks easy http://leaflet.github.io/Leaflet.fullscreen/

Particularly useful for time series like S2-ARD, for example sorting through scenes over time looking for different cloud-coverings. Use Leaflet's layerGroup.bringToFront() to bring the GeoJSON layer to the front (and simultaneously select the product).

To open this up to a wider user base that does not have the satellite orbits imprinted on their brains how about (something like):

scene: Hebrides    orbit: 23      capturedate: 9th May 2016

The first step is to make sure it is possible to name the scenes in a meaningful / non-confusing way. The scene footprints are here
https://github.com/jncc/dnode/tree/master/s2-ard/scenes , for example
https://github.com/jncc/dnode/blob/master/s2-ard/scenes/123.geojson.

Just need a list / spreadsheet of (orbit, row, name) initially.

Explanatory diagram

We could also have somewhere a look up diagram with the orbits, scenes and names eg: linked into the discovery level meta data for the whole Sentinel 2 ARD resource

We could have a bit of fun choosing the names for each scene position within each orbit and involve implementation group

And it would demystify the resource of new users

It would also help uses get familiar with the orbit and scene numbers

Filtering

We could potentially then offer a filter by the scene name with a nice list of these names to pick from. We haven't thought about how to implement filtering from managed / restricted lists yet. Could do this as a one-off hack, or a more general feature of the Deli.

Both CDNs for sematic-ui and leaflet stylesheets set cookies, which we can't explain very well to users in our cookies page.

We could serve our own CSS by updating the webpack build process.

Need to change the URL in settings.ts

Suddenly as if by magic :(

This error occurs on build in eocoe and master following a branch switch and clean npm install in attempt to resolve issue.:

1] Project is running at http://localhost:8080/
[1] webpack output is served from /
[1] ts-loader: Using [email protected] and /home/felix/Development/deli/tsconfig.json
[0] app.server/server.ts(29,52): error TS2345: Argument of type 'string | undefined' is not assignable to parameter of type 'string'.
[0] Type 'undefined' is not assignable to type 'string'.
[0] npm
[0]
[0] ERR!
[0]
[0] code
[0] ELIFECYCLE
[0] npm
[0]

Hacky fix for the moment is to change settings.getRealWmsUrl from:
export function getRealWmsUrl(env, hostHeader: string, protocol: string)
to:
export function getRealWmsUrl(env, hostHeader: any, protocol: string)

If you modify the elasticbeanstalk environment in AWS that triggers an environment rebuild (i.e. modify environment variables) the service wont start. This is down to an issue with the nginx proxy.conf in the .ebextension folder, looks like it duplicates the config rather than overwrites it (error being 'duplicate nodejs upstream').

Fixed by going to the applications versions page for the environment and redeploying the current version of the code to the environment.

Add a hyperlink statement to the Scottish Lidar portal to link to a privacy statement. See statisticians site for example text and hyperlink. Home for link: http://statistics.gov.scot/home (bottom right grouped with cookies statement), and actual statement on this page: http://statistics.gov.scot/policy#privacy

The Deli / Lidar with the new catalogue API can be run locally, but it isn't obvious how.

• update the Deli readme.md
• integrate the feature/docker-container in jncc/catalog

Matt's idea to make lidar data available more easily for programmatic download.

(This is hard to test as there's a possible confounding issue that the EB environment doesn't restart after a manual change to its environment variables - which appears to be related to the custom nginx config. This is possibly due to a recent change in EB https://stackoverflow.com/a/38420816/40759 ...?)

• edit the CATALOG_API host environment var to something nonexistent (to simulate transitory network failure)
• the EB environment will not start successfully, even on a redeploy

502 Bad Gateway

from Nginx (the Node / Express server won't respond)

I suspect that the network requests in the nested promises at the start of in server.ts means that the server cannot respond to any requests, even a for something like a favicon, until two network database requests have successfully completed.

Needs further investigation @mattdebont could you take a look?

The code in /deli/app.server/handlers/products/spike.ts

Breaks everything:

[1] Project is running at http://localhost:8080/
[1] webpack output is served from /
[1] ts-loader: Using [email protected] and /home/felix/Development/deli/tsconfig.json
[0] app.server/handlers/products/spike.ts(2,5): error TS2451: Cannot redeclare block-scoped variable 'turf'.
[0] node_modules/@types/turf/index.d.ts(90,15): error TS2451: Cannot redeclare block-scoped variable 'turf'.
[0] node_modules/@types/turf/index.d.ts(90,21): error TS2503: Cannot find namespace 'turf'.
[0] node_modules/@types/turf/index.d.ts(99,16): error TS2451: Cannot redeclare block-scoped variable 'turf'.
[0] npm

Selecting a search with less than 50 results doesn't resize the page properly leaving you able to scroll past the new footer.

Don't request layers that are completely hidden. An easier less generic way to do this for S2-ARD would be to rely on the fact that all named scenes (orbit+row) will be exactly the same footprint .

Google analytics reporting for the lidar site appears to have terminated at the end of March 2018. This needs looking into and reinstating.

Create a working standard docker container to add create configured variants of.

Move open CORS config to / to simplify the nginx config.

We need a better cookie message / banner than links to our cookies page. See www.gov.uk.

Instead of date pickers. So you can see visually what products are actually available,

If set both dates to 2017-01-05, should show products, but doesn't

Instead of e.g. 10+. Catalogue should support it.

The app.server Express setup may need to be production-hardened.

This http://expressjs.com/en/starter/generator.html may have examples.

Last successful build was 940aaee
Guessing the next commit 9925f18 with lots of npm updates has broken it.

The network requests in the nested promises at the start of in server.ts means that the server cannot respond to any requests, even a for something like a favicon, until two network database requests have successfully completed.

• The web server shouldn't perform network requests before becoming responsive.
• Network database requests can and do go down temporarily

There's no need at all to make network requests from web server until they are actually needed - i.e. until they are initiated by the web client.

The responsibility of the server.ts file is basically to define the web server, i.e., define the sorts of requests that it can handle. But currently it needs to have intimate knowledge of the new Catalog class, in that it needs to know to call the side-effecting getOGCServiceList() before it can call the other methods.

Suggest redesigning the Catalog class so that it alone is responsible for knowing about the database requests that it needs to make (and not relying on its caller knowing to call things in a certain order). The server.ts code should just make the two calls it needs to make to the Catalog (essentially GetCollections and GetProducts). The Catalog class should be responsible for the actual network requests. The networking code should be removed from the server.ts file.

Relatedly, it might help to remove the custom ngnix code which was needed from the custom WMS service due to possibly due to a recent change in how EB supports it (or not) https://stackoverflow.com/a/38420816/40759

The Deli really needs to become a "real" website, and have cookies & privacy policy pages.

Alter the build process:

• Build separate HTML pages
• Create the help/cookies HTML pages

The new pages can be at:

/help/cookies
/help/privacy

It's fine to have separate markdown content for different tenants. There's no point trying to share the content; it's too different.

Need to stop the map zooming in past a particular point as it just stops working when zooming in too far right now

The current v2.8.28 of Uglify has a bug, workaround in shippable.yml. Follow mishoo/UglifyJS#2054

The UI is noticeable sluggish when moving the mouse over the list items. Removing the onMouseOver prop fixes. Why is this happening? Try a different implementation.

Users will F5, and right now it looks broken.

Almost ready Semantic-Org/Semantic-UI-React#198