podaac / hitide Goto Github PK

Release Backfill Tool v0.4.1
Includes new:
forge 0.9.0
tig 0.8.0

Acceptance Criteria: Work with Dpub team to come up with a procedure and deploy software for DPub to run and be able to (re)generate forge-tig-configuration files and generate thumbnail images using TIG, and generate footprints using FORGE. Plan to start simple and then improve the process and "ease of use" over time.

Try to make a runnable Jar.

Then the user can run it with the JRE, just using java forge.jar etc.

Move repos from JPL Github to Github.com:

• hitide-backfill-sqs-to-step
• hitide-backfill-post-step
• cumulus-postworkflow-normalizer

Reduce cost of tig running in AWS and add tags to services to track hitide costs more precisely

Run Phase 8:
https://wiki.jpl.nasa.gov/display/PD/Backfilling+Progress+Report

Associate to l2ss-py service:
SWOT_L2_LR_SSH_BASIC_2.0
SWOT_L2_LR_SSH_EXPERT_2.0
SWOT_L2_LR_SSH_WINDWAVE_2.0
SWOT_L2_NALT_GDR_GDR_2.0
SWOT_L2_NALT_GDR_SGDR_2.0
SWOT_L2_NALT_IGDR_GDR_2.0
SWOT_L2_NALT_IGDR_SGDR_2.0
SWOT_L2_NALT_OGDR_GDR_2.0
SWOT_L2_NALT_GDR_SSHA_2.0
SWOT_L2_NALT_IGDR_SSHA_2.0
SWOT_L2_NALT_OGDR_SSHA_2.0

What are the differences exactly? Comparing to CMR?

Testrails runs here:
https://cae-testrail.jpl.nasa.gov/testrail/index.php?/runs/view/33983&group_by=cases:section_id&group_order=asc

UI changes in v4.16.0 seemed to have broken the tests. Please update tests to pass again.

OPS team want to shut down all hitide on-prem services on the week of September 11th.

Use the backfill tool to update some SWOT_L2_LR_SSH_EXPERT 1.0 and 1.1 granules and produce the new thumbnail image for ssha_karin_2. This image should be a high resolution image and should be verified with Jinbo that it is correct.

Add collections to HiTIDE:

SWOT_L2_LR_SSH_BASIC_1.1
SWOT_L2_LR_SSH_EXPERT_1.1
SWOT_L2_LR_SSH_WINDWAVE_1.1

This includes adding functionality to run subsetting (L2ss-py), merging (l2ss-py with concise), and include footprints and thumbnail images, and Collection Info in HiTIDE fully complete and correct.

Acceptance Criteria: After the reassociation tool runs the Backfill tool, the "system" (either tig itself or a new step in the AWS Step function) updates and corrects the ssha_karin_2 thumbnail image for all processed granules.

Talk with Curtis

Ed Armstrong too.

Joe Roberts

Reach out to company (Unidata)

Haley Johnson (Unidata)

Wikio?

Here's the official SWOT L2 Collections list to add to HiTIDE:

SWOT_L2_NALT_IGDR_SSHA_1.0
SWOT_L2_NALT_OGDR_SSHA_1.0
SWOT_L2_LR_SSH_BASIC_1.0
SWOT_L2_LR_SSH_EXPERT_1.0
SWOT_L2_LR_SSH_WINDWAVE_1.0

Can NOT do:

SWOT_L2_NALT_GDR_SSHA_1.0 (No data yet)
SWOT_L2_LR_SSH_UNSMOOTHED_1.0 (Footprint issues, will revisit in a future PI)

HiTIDE POCLOUD shouldn't contain any collections that end in -PODAAC.

Verify here:
https://cmr.earthdata.nasa.gov/search/tools.umm_json?name=hitide&pretty=true&provider=pocloud

and in the HiTIDE UI

Details defined here:

https://jira.jpl.nasa.gov/browse/PODAAC-5927

Being done by I&A, Stepheny.

Collection List details here:

https://jira.jpl.nasa.gov/browse/PODAAC-6117

Work done by Jack, but Jim may try to help to get this done sooner.

Add automated tests for these collections.
Test in both OPS and UAT environments.

SWOT_L2_LR_SSH_BASIC_1.0
SWOT_L2_LR_SSH_EXPERT_1.0
SWOT_L2_LR_SSH_WINDWAVE_1.0
SWOT_L2_NALT_GDR_SSHA_1.0 *** SKIP FOR NOW ***
SWOT_L2_NALT_IGDR_SSHA_1.0
SWOT_L2_NALT_OGDR_SSHA_1.0

Skip SWOT_L2_NALT_GDR_SSHA_1.0 for now though since it has no data.

Details here:

https://docs.google.com/spreadsheets/d/1crT6KK19rPZSxS8r45Wupd1ikTp02Pn-MnnTtcnaYMo/edit?usp=sharing

The thumbnail should be inside the footprint.

Collections include:

SWOT_L2_LR_SSH_BASIC_2.0
SWOT_L2_LR_SSH_EXPERT_2.0
SWOT_L2_LR_SSH_WINDWAVE_2.0

Should work according to this JIRA ticket:

https://jira.jpl.nasa.gov/browse/PODAAC-6013

Add configs to forge-tig-configuration for image only:
SWOT_L2_LR_SSH_BASIC_2.0
SWOT_L2_LR_SSH_EXPERT_2.0
SWOT_L2_LR_SSH_WINDWAVE_2.0
SWOT_L2_NALT_GDR_SSHA_2.0
SWOT_L2_NALT_IGDR_SSHA_2.0
SWOT_L2_NALT_OGDR_SSHA_2.0

Add to forge-tig-configuration for footprint only:
SWOT_L2_NALT_GDR_2.0
SWOT_L2_NALT_IGDR_2.0
SWOT_L2_NALT_OGDR_2.0

Associate to l2ss-py service:
SWOT_L2_LR_SSH_BASIC_2.0
SWOT_L2_LR_SSH_EXPERT_2.0
SWOT_L2_LR_SSH_WINDWAVE_2.0
SWOT_L2_NALT_GDR_GDR_2.0
SWOT_L2_NALT_GDR_SGDR_2.0
SWOT_L2_NALT_IGDR_GDR_2.0
SWOT_L2_NALT_IGDR_SGDR_2.0
SWOT_L2_NALT_OGDR_GDR_2.0
SWOT_L2_NALT_GDR_SSHA_2.0
SWOT_L2_NALT_IGDR_SSHA_2.0
SWOT_L2_NALT_OGDR_SSHA_2.0

Associate to HiTIDE Tool:
SWOT_L2_LR_SSH_BASIC_2.0
SWOT_L2_LR_SSH_EXPERT_2.0
SWOT_L2_LR_SSH_WINDWAVE_2.0

Tickets here:

• podaac/forge-tig-configuration#41
• podaac/forge-tig-configuration#42
• #47
• #48
• #49
• #50
• #51
• #52
• #53
• #36

Release Backfill Tool v0.4.0 to also deploy to new forge and tig to Cumulus AWS

Acceptance Criteria:

Added the following collections to HiTIDE, and have the ability to subset (l2ss-py) and merge (concise) inside HiTIDE.

Collections include:

• SWOT_L2_NALT_IGDR_GDR_1.0
• SWOT_L2_NALT_IGDR_SGDR_1.0
• SWOT_L2_NALT_OGDR_GDR_1.0
• SWOT_L2_NALT_IGDR_1.0 (?)
• SWOT_L2_NALT_OGDR_1.0 (?)
• SWOT_L2_RAD_GDR_1.0 (?)
• SWOT_L2_LR_SSH_1.0 (?)
• SWOT_L2_HR_Raster_1.0 (?)
• SWOT_L2_LR_SSH_1.1 (?)
• SWOT_L2_LR_SSH_BASIC_1.1
• SWOT_L2_LR_SSH_EXPERT_1.1
• SWOT_L2_LR_SSH_WINDWAVE_1.1
• SWOT_L2_LR_SSH_UNSMOOTHED_1.1
• SWOT_L2_HR_RiverSP_1.1 (No forge & no tig) (?)
• SWOT_L2_HR_RiverSP_node_1.1 (No forge & no tig) (?)
• SWOT_L2_HR_RiverSP_reach_1.1 (No forge & no tig) (?)
• SWOT_L2_HR_LakeSP_1.1 (No forge & no tig) (?)
• SWOT_L2_HR_LakeSP_obs_1.1 (No forge & no tig) (?)
• SWOT_L2_HR_LakeSP_prior_1.1 (No forge & no tig) (?)
• SWOT_L2_HR_LakeSP_unassigned_1.1 (No forge & no tig) (?)
• SWOT_L2_HR_Raster_1.1 (No forge) (?)
• SWOT_L2_HR_Raster_100m_1.1
• SWOT_L2_HR_Raster_250m_1.1

(?) Should add to HiTIDE?

Update this:
https://github.jpl.nasa.gov/podaac/hitide-backfill-tool/blob/develop/terraform-deploy/post_to_cmr.zip

and this:
https://github.jpl.nasa.gov/podaac/hitide-backfill-tool/blob/develop/terraform-deploy/post_to_cmr.tf

to support Cumulus 16.

Need to check with @skorper to get info on new version.

List of collections to backfill TBD

Tracking this ticket for @skorper

https://jira.jpl.nasa.gov/browse/PODAAC-6094

Doing these repos:
podaac/l2ss-py#224
podaac/concise#97
podaac/forge#28
podaac/tig#38

Setup README for DPub to use tig.

Make super easy to run

https://wiki.earthdata.nasa.gov/pages/viewpage.action?spaceKey=ENS&title=PI+24.1+Tenant+Action+Items

Needed for repos:

hitide-ui
hitide-profile

Next repos to migrate are:

• forge
• tig