The gh-pages branch contains content for https://earthcube.github.io/earthcube/, and is the default branch for this repository.
geocodes_documentation's Introduction
geocodes_documentation's People
geocodes_documentation's Issues
ESIP SOS.. fork, branch to setup relative urls
Figues using /assets which means no figures on mkdocs import
branch copy of guidance, fork fix, use fork in mkdocs. submit patch
Someone Uses Documentation to deploy Geocodes
Build docs build once a week
Note: schedule only works for default branch (main)
Dagster Deploy and use documentation
update docs https://earthcube.github.io/ landing page, where geocodes is running
This needs to point at the
https://earthcube.github.io/geocodes_documentation/
earhtcube.org
Geocodes stack information in the geocodes docs needs to be update
Data load validation for data loading
- What do we need to validate a data load
- (Gleaner/nabu) logs, and scripts to look for errors
- Notebook to Validate Dataloading for ECO Staff
- Duplicate checking
- Loaded counts
Documentation
mkdocs: Refactor to use nav_repos:
https://github.com/jdoiro3/mkdocs-multirepo-plugin#nav-repos-config
This will allow for linking to pages in a repository
aka link to data loading in geocodes
- Load Data: data_loading/indexing_with_gleanerio_for_testing.md
or installing:
- Install Glcon: data_loading/install_glcon.md
Dagster Deploy Documentation
Instructions on how to deploy dagster for a second user to work through
Production Truth for Each Tennants
Have a page with listing examples
Master List of
- source
- code for each instance
-
- contact
Production Instructions
Specify Logic of Data Loading and validation
There is a validation notebook prototypes
There are site map, sparql, s3 and other counts shown
Put the existing and future logic into a document. This will be the basis for discussing the what parts of the validation are implemented, and how we test them.
It it's a living document (aka notebook), please add full descriptions of the steps, then implement.
Aka test description first, code second.
Document Machine and Stack Deployment
Outline of User Tutorial
This should be based on the onbording a source document.
Ideally people attending would have a repository/organization.
Document Data Loading Validation Logic
ADD TO THIS DOCUMENT: https://github.com/earthcube/geocodes_documentation/wiki/DataLoadingValidationStory#what-do-we-need-do-to-setup-testing
Document the testing that will be needed to validate the data loading.
Repeat, document the steps we will need to implement to validate the data loading.
This is not asking for the implementation of the tests. It is asking for the testing plan with the steps that will will need to implement.
Places to possibly look:
Examples
Spec: Summon working
- count of records from a sitemap (matches) count of records that made it into the bucket
- possible tool: sitemap assay
Spec: expected JSON-LD
possible tooling: https://github.com/gleanerio/notebooks/tree/master/notebooks/validation
- validate jsonld using ?
- does it have xxxx
Spec: JSON-LD data load to triple store
- count of files in s3 bucket (matches) count of graphs for an organization/record
- For (% of data load or all if < 100 records)
- retrieve JSONLD
- urn
- name
- retrieve a graph by a urn from the service api
- did we get a record
- does information (name, urn, [other properties]) match
- retrieve from triplestore using user interface query with name as
- retrieve JSONLD
Spec: JSON LD Renders in UI
- for a random number of jsonld in org.
- create a set of curl url
- open in a web test suite (selenium, etc) and see that they at least partially render to expectations
Spec Tool Linkage:
- for a set of known files, do we match the look linkages.
- This probably needs to be an approval test.
spec: org provenance information
- where is the org provenance, does it look useful.
- pssoble tool: https://github.com/gleanerio/notebooks/blob/master/notebooks/exploring/orgAndProv.ipynb
Community Onboarding Documentation by April 20
Create a document that can be used for onboarding communities.
Let's start with an outline of what we think needs to be incorporated, then fill is out.
https://docs.google.com/document/d/1SSeh6deu5WYhTo526ZeSyZ4Mg_XpJyrvgLG7phO9YFU/edit
We have a meeting with Deep Oceans on April 20th. So let's have a draft by then
Add Test Data to new stack
Document Test Dataset and Test Configuration
https://github.com/earthcube/GeoCODES-Metadata
Sitemaps are generated by action.
Setup configurations for pushing to (developement) staging and production
System Overview Documentation
Over the next few weeks onboard Luigi and Ya-Lan
Create systems overview documentation.
Let's put in the geocodes_documentation repository.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. ๐๐๐
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google โค๏ธ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.