Table of contents

• Introduction
  • About This Document
• Folder structure
• Input data
  • Provenance
• Metadata
  • Landsat 8
  • MOD09GA
• Test data creation
  • Landsat 8
  • MOD09GA
• BIPified data
  • Generate SCAG BIP data
  • SCAG BIP md5sum hashes
  • SCAG BIP contents
• SCAG outputs
  • Run SCAG
  • SCAG output md5sum hashes

Introduction

This folder describes the SCAG test data.

Motivation:

1. Provide the full pipeline of test data so that changes in outputs can be traced upstream. We record metadata of the raw upstream inputs (e.g., Landsat tarball) so that if upstream USGS processing changes occur, we can determine that and adjust the regression test expectations accordingly.
2. Provide small subsets for rapid regression testing.

Organization:

• The original full files are archived at https://osf.io/4cp58/.
• This document and repository contains the code, metadata, and subsetted and processed data.

About This Document

This document is an Emacs Org Mode plain-text file with code and text embedded. If you are viewing:

• A DOC, PDF, or other non-Org file, then it was generated by exporting from Org. Not all of the Org parts (code, results, comments, etc.) were exported. The Org source file is available upon request, and may be embedded as an attachment inside the PDF.
• An Org file on GitHub or other web viewer, then see comment above for viewing a non-Org file. GitHub renders Org files well, but for example you may see source code but not the results of running the code, or vice versa. Comments are not rendered.
• A file with an org extension in something other than Emacs, then you are seeing the canonical version and the full source, but without any syntax highlighting, document structure, or the ability to execute the code blocks.
• An Org file within Emacs, then this is the canonical version. You should be able to fully interact and reproduce the contents of this document, although it may require 3rd-party applications and setup (e.g, Python environments) and a similar Emacs configuration.

Folder structure

.
├── 0_upstream
├── 1_subset
├── 2_scag_bip
└── 3_output

4 directories

Description of folders and contents

0_upstream

Upstream products can be stored here if desired. md5sum recorded to help determine if upstream processing changes

1_subset

Subsets of 0_upstream small enough that the full processing pipeline takes O(10s)

2_scag_bip

Pre-processed BIPified data. Temporary. Reduces external dependencies for CI. Will be removed when BIPifier is moved from external dependency to inside this project.

3_output

Outputs of SCAG algorithm for regression testing comparison purposes

Input data

• [ ] Landsat 7
• [X] Landsat 8
• [ ] Landsat 9
• [X] MODIS MOD09GA
• [ ] VIIRS

Provenance

Files downloaded manually from respective repositories and archived at https://osf.io/4cp58/ to protect from upstream processing changes.

wget -nc -O 0_upstream.zip https://files.osf.io/v1/resources/4cp58/providers/osfstorage/?zip=
unzip  0_upstream.zip -d 0_upstream
# rm 0_upstream.zip

Metadata

Here we define the specific scenes and files, and record the metadata and md5sum hash of the files in case upstream products change.

Landsat 8

md5sum

md5sum LC08_L2SP_046027_20200316_20200822_02_T1.tar
tar --to-command=md5sum -xvf LC08_L2SP_046027_20200316_20200822_02_T1.tar | paste - -

Metadata

tar -axf LC08_L2SP_046027_20200316_20200822_02_T1.tar LC08_L2SP_046027_20200316_20200822_02_T1_MTL.txt -O | cat

GROUP = LANDSAT_METADATA_FILE
  GROUP = PRODUCT_CONTENTS
    ORIGIN = "Image courtesy of the U.S. Geological Survey"
    DIGITAL_OBJECT_IDENTIFIER = "https://doi.org/10.5066/P9OGBGM6"
    LANDSAT_PRODUCT_ID = "LC08_L2SP_046027_20200316_20200822_02_T1"
    PROCESSING_LEVEL = "L2SP"
    COLLECTION_NUMBER = 02
    COLLECTION_CATEGORY = "T1"
    OUTPUT_FORMAT = "GEOTIFF"
    FILE_NAME_BAND_1 = "LC08_L2SP_046027_20200316_20200822_02_T1_SR_B1.TIF"
    FILE_NAME_BAND_2 = "LC08_L2SP_046027_20200316_20200822_02_T1_SR_B2.TIF"
    FILE_NAME_BAND_3 = "LC08_L2SP_046027_20200316_20200822_02_T1_SR_B3.TIF"
    FILE_NAME_BAND_4 = "LC08_L2SP_046027_20200316_20200822_02_T1_SR_B4.TIF"
    FILE_NAME_BAND_5 = "LC08_L2SP_046027_20200316_20200822_02_T1_SR_B5.TIF"
    FILE_NAME_BAND_6 = "LC08_L2SP_046027_20200316_20200822_02_T1_SR_B6.TIF"
    FILE_NAME_BAND_7 = "LC08_L2SP_046027_20200316_20200822_02_T1_SR_B7.TIF"
    FILE_NAME_BAND_ST_B10 = "LC08_L2SP_046027_20200316_20200822_02_T1_ST_B10.TIF"
    FILE_NAME_THERMAL_RADIANCE = "LC08_L2SP_046027_20200316_20200822_02_T1_ST_TRAD.TIF"
    FILE_NAME_UPWELL_RADIANCE = "LC08_L2SP_046027_20200316_20200822_02_T1_ST_URAD.TIF"
    FILE_NAME_DOWNWELL_RADIANCE = "LC08_L2SP_046027_20200316_20200822_02_T1_ST_DRAD.TIF"
    FILE_NAME_ATMOSPHERIC_TRANSMITTANCE = "LC08_L2SP_046027_20200316_20200822_02_T1_ST_ATRAN.TIF"
    FILE_NAME_EMISSIVITY = "LC08_L2SP_046027_20200316_20200822_02_T1_ST_EMIS.TIF"
    FILE_NAME_EMISSIVITY_STDEV = "LC08_L2SP_046027_20200316_20200822_02_T1_ST_EMSD.TIF"
    FILE_NAME_CLOUD_DISTANCE = "LC08_L2SP_046027_20200316_20200822_02_T1_ST_CDIST.TIF"
    FILE_NAME_QUALITY_L2_AEROSOL = "LC08_L2SP_046027_20200316_20200822_02_T1_SR_QA_AEROSOL.TIF"
    FILE_NAME_QUALITY_L2_SURFACE_TEMPERATURE = "LC08_L2SP_046027_20200316_20200822_02_T1_ST_QA.TIF"
    FILE_NAME_QUALITY_L1_PIXEL = "LC08_L2SP_046027_20200316_20200822_02_T1_QA_PIXEL.TIF"
    FILE_NAME_QUALITY_L1_RADIOMETRIC_SATURATION = "LC08_L2SP_046027_20200316_20200822_02_T1_QA_RADSAT.TIF"
    FILE_NAME_ANGLE_COEFFICIENT = "LC08_L2SP_046027_20200316_20200822_02_T1_ANG.txt"
    FILE_NAME_METADATA_ODL = "LC08_L2SP_046027_20200316_20200822_02_T1_MTL.txt"
    FILE_NAME_METADATA_XML = "LC08_L2SP_046027_20200316_20200822_02_T1_MTL.xml"
    DATA_TYPE_BAND_1 = "UINT16"
    DATA_TYPE_BAND_2 = "UINT16"
    DATA_TYPE_BAND_3 = "UINT16"
    DATA_TYPE_BAND_4 = "UINT16"
    DATA_TYPE_BAND_5 = "UINT16"
    DATA_TYPE_BAND_6 = "UINT16"
    DATA_TYPE_BAND_7 = "UINT16"
    DATA_TYPE_BAND_ST_B10 = "UINT16"
    DATA_TYPE_THERMAL_RADIANCE = "INT16"
    DATA_TYPE_UPWELL_RADIANCE = "INT16"
    DATA_TYPE_DOWNWELL_RADIANCE = "INT16"
    DATA_TYPE_ATMOSPHERIC_TRANSMITTANCE = "INT16"
    DATA_TYPE_EMISSIVITY = "INT16"
    DATA_TYPE_EMISSIVITY_STDEV = "INT16"
    DATA_TYPE_CLOUD_DISTANCE = "INT16"
    DATA_TYPE_QUALITY_L2_AEROSOL = "UINT8"
    DATA_TYPE_QUALITY_L2_SURFACE_TEMPERATURE = "INT16"
    DATA_TYPE_QUALITY_L1_PIXEL = "UINT16"
    DATA_TYPE_QUALITY_L1_RADIOMETRIC_SATURATION = "UINT16"
  END_GROUP = PRODUCT_CONTENTS
  GROUP = IMAGE_ATTRIBUTES
    SPACECRAFT_ID = "LANDSAT_8"
    SENSOR_ID = "OLI_TIRS"
    WRS_TYPE = 2
    WRS_PATH = 46
    WRS_ROW = 27
    NADIR_OFFNADIR = "NADIR"
    TARGET_WRS_PATH = 46
    TARGET_WRS_ROW = 27
    DATE_ACQUIRED = 2020-03-16
    SCENE_CENTER_TIME = "18:55:29.9626189Z"
    STATION_ID = "LGN"
    CLOUD_COVER = 0.38
    CLOUD_COVER_LAND = 0.41
    IMAGE_QUALITY_OLI = 9
    IMAGE_QUALITY_TIRS = 9
    SATURATION_BAND_1 = "N"
    SATURATION_BAND_2 = "Y"
    SATURATION_BAND_3 = "Y"
    SATURATION_BAND_4 = "Y"
    SATURATION_BAND_5 = "Y"
    SATURATION_BAND_6 = "Y"
    SATURATION_BAND_7 = "Y"
    SATURATION_BAND_8 = "N"
    SATURATION_BAND_9 = "N"
    ROLL_ANGLE = -0.001
    SUN_AZIMUTH = 154.12711692
    SUN_ELEVATION = 38.15764074
    EARTH_SUN_DISTANCE = 0.9949596
    TRUNCATION_OLI = "UPPER"
    TIRS_SSM_MODEL = "FINAL"
    TIRS_SSM_POSITION_STATUS = "ESTIMATED"
  END_GROUP = IMAGE_ATTRIBUTES
  GROUP = PROJECTION_ATTRIBUTES
    MAP_PROJECTION = "UTM"
    DATUM = "WGS84"
    ELLIPSOID = "WGS84"
    UTM_ZONE = 10
    GRID_CELL_SIZE_REFLECTIVE = 30.00
    GRID_CELL_SIZE_THERMAL = 30.00
    REFLECTIVE_LINES = 7881
    REFLECTIVE_SAMPLES = 7771
    THERMAL_LINES = 7881
    THERMAL_SAMPLES = 7771
    ORIENTATION = "NORTH_UP"
    CORNER_UL_LAT_PRODUCT = 48.51213
    CORNER_UL_LON_PRODUCT = -123.39673
    CORNER_UR_LAT_PRODUCT = 48.47976
    CORNER_UR_LON_PRODUCT = -120.24214
    CORNER_LL_LAT_PRODUCT = 46.38504
    CORNER_LL_LON_PRODUCT = -123.38105
    CORNER_LR_LAT_PRODUCT = 46.35498
    CORNER_LR_LON_PRODUCT = -120.35099
    CORNER_UL_PROJECTION_X_PRODUCT = 470700.000
    CORNER_UL_PROJECTION_Y_PRODUCT = 5373300.000
    CORNER_UR_PROJECTION_X_PRODUCT = 703800.000
    CORNER_UR_PROJECTION_Y_PRODUCT = 5373300.000
    CORNER_LL_PROJECTION_X_PRODUCT = 470700.000
    CORNER_LL_PROJECTION_Y_PRODUCT = 5136900.000
    CORNER_LR_PROJECTION_X_PRODUCT = 703800.000
    CORNER_LR_PROJECTION_Y_PRODUCT = 5136900.000
  END_GROUP = PROJECTION_ATTRIBUTES
  GROUP = LEVEL2_PROCESSING_RECORD
    ORIGIN = "Image courtesy of the U.S. Geological Survey"
    DIGITAL_OBJECT_IDENTIFIER = "https://doi.org/10.5066/P9OGBGM6"
    REQUEST_ID = "L2"
    LANDSAT_PRODUCT_ID = "LC08_L2SP_046027_20200316_20200822_02_T1"
    PROCESSING_LEVEL = "L2SP"
    OUTPUT_FORMAT = "GEOTIFF"
    DATE_PRODUCT_GENERATED = 2020-08-22T20:39:36Z
    PROCESSING_SOFTWARE_VERSION = "LPGS_15.3.1c"
    ALGORITHM_SOURCE_SURFACE_REFLECTANCE = "LaSRC_1.5.0"
    DATA_SOURCE_OZONE = "MODIS"
    DATA_SOURCE_PRESSURE = "Calculated"
    DATA_SOURCE_WATER_VAPOR = "MODIS"
    DATA_SOURCE_AIR_TEMPERATURE = "MODIS"
    ALGORITHM_SOURCE_SURFACE_TEMPERATURE = "st_1.3.0"
    DATA_SOURCE_REANALYSIS = "GEOS-5 FP-IT"
  END_GROUP = LEVEL2_PROCESSING_RECORD
  GROUP = LEVEL2_SURFACE_REFLECTANCE_PARAMETERS
    REFLECTANCE_MAXIMUM_BAND_1 = 1.602213
    REFLECTANCE_MINIMUM_BAND_1 = -0.199972
    REFLECTANCE_MAXIMUM_BAND_2 = 1.602213
    REFLECTANCE_MINIMUM_BAND_2 = -0.199972
    REFLECTANCE_MAXIMUM_BAND_3 = 1.602213
    REFLECTANCE_MINIMUM_BAND_3 = -0.199972
    REFLECTANCE_MAXIMUM_BAND_4 = 1.602213
    REFLECTANCE_MINIMUM_BAND_4 = -0.199972
    REFLECTANCE_MAXIMUM_BAND_5 = 1.602213
    REFLECTANCE_MINIMUM_BAND_5 = -0.199972
    REFLECTANCE_MAXIMUM_BAND_6 = 1.602213
    REFLECTANCE_MINIMUM_BAND_6 = -0.199972
    REFLECTANCE_MAXIMUM_BAND_7 = 1.602213
    REFLECTANCE_MINIMUM_BAND_7 = -0.199972
    QUANTIZE_CAL_MAX_BAND_1 = 65535
    QUANTIZE_CAL_MIN_BAND_1 = 1
    QUANTIZE_CAL_MAX_BAND_2 = 65535
    QUANTIZE_CAL_MIN_BAND_2 = 1
    QUANTIZE_CAL_MAX_BAND_3 = 65535
    QUANTIZE_CAL_MIN_BAND_3 = 1
    QUANTIZE_CAL_MAX_BAND_4 = 65535
    QUANTIZE_CAL_MIN_BAND_4 = 1
    QUANTIZE_CAL_MAX_BAND_5 = 65535
    QUANTIZE_CAL_MIN_BAND_5 = 1
    QUANTIZE_CAL_MAX_BAND_6 = 65535
    QUANTIZE_CAL_MIN_BAND_6 = 1
    QUANTIZE_CAL_MAX_BAND_7 = 65535
    QUANTIZE_CAL_MIN_BAND_7 = 1
    REFLECTANCE_MULT_BAND_1 = 2.75e-05
    REFLECTANCE_MULT_BAND_2 = 2.75e-05
    REFLECTANCE_MULT_BAND_3 = 2.75e-05
    REFLECTANCE_MULT_BAND_4 = 2.75e-05
    REFLECTANCE_MULT_BAND_5 = 2.75e-05
    REFLECTANCE_MULT_BAND_6 = 2.75e-05
    REFLECTANCE_MULT_BAND_7 = 2.75e-05
    REFLECTANCE_ADD_BAND_1 = -0.2
    REFLECTANCE_ADD_BAND_2 = -0.2
    REFLECTANCE_ADD_BAND_3 = -0.2
    REFLECTANCE_ADD_BAND_4 = -0.2
    REFLECTANCE_ADD_BAND_5 = -0.2
    REFLECTANCE_ADD_BAND_6 = -0.2
    REFLECTANCE_ADD_BAND_7 = -0.2
  END_GROUP = LEVEL2_SURFACE_REFLECTANCE_PARAMETERS
  GROUP = LEVEL2_SURFACE_TEMPERATURE_PARAMETERS
    TEMPERATURE_MAXIMUM_BAND_ST_B10 = 372.999941
    TEMPERATURE_MINIMUM_BAND_ST_B10 = 149.003418
    QUANTIZE_CAL_MAXIMUM_BAND_ST_B10 = 65535
    QUANTIZE_CAL_MINIMUM_BAND_ST_B10 = 1
    TEMPERATURE_MULT_BAND_ST_B10 = 0.00341802
    TEMPERATURE_ADD_BAND_ST_B10 = 149.0
  END_GROUP = LEVEL2_SURFACE_TEMPERATURE_PARAMETERS
  GROUP = LEVEL1_PROCESSING_RECORD
    ORIGIN = "Image courtesy of the U.S. Geological Survey"
    DIGITAL_OBJECT_IDENTIFIER = "https://doi.org/10.5066/P975CC9B"
    REQUEST_ID = "L2"
    LANDSAT_SCENE_ID = "LC80460272020076LGN00"
    LANDSAT_PRODUCT_ID = "LC08_L1TP_046027_20200316_20200822_02_T1"
    PROCESSING_LEVEL = "L1TP"
    COLLECTION_CATEGORY = "T1"
    OUTPUT_FORMAT = "GEOTIFF"
    DATE_PRODUCT_GENERATED = 2020-08-22T20:28:36Z
    PROCESSING_SOFTWARE_VERSION = "LPGS_15.3.1c"
    FILE_NAME_BAND_1 = "LC08_L1TP_046027_20200316_20200822_02_T1_B1.TIF"
    FILE_NAME_BAND_2 = "LC08_L1TP_046027_20200316_20200822_02_T1_B2.TIF"
    FILE_NAME_BAND_3 = "LC08_L1TP_046027_20200316_20200822_02_T1_B3.TIF"
    FILE_NAME_BAND_4 = "LC08_L1TP_046027_20200316_20200822_02_T1_B4.TIF"
    FILE_NAME_BAND_5 = "LC08_L1TP_046027_20200316_20200822_02_T1_B5.TIF"
    FILE_NAME_BAND_6 = "LC08_L1TP_046027_20200316_20200822_02_T1_B6.TIF"
    FILE_NAME_BAND_7 = "LC08_L1TP_046027_20200316_20200822_02_T1_B7.TIF"
    FILE_NAME_BAND_8 = "LC08_L1TP_046027_20200316_20200822_02_T1_B8.TIF"
    FILE_NAME_BAND_9 = "LC08_L1TP_046027_20200316_20200822_02_T1_B9.TIF"
    FILE_NAME_BAND_10 = "LC08_L1TP_046027_20200316_20200822_02_T1_B10.TIF"
    FILE_NAME_BAND_11 = "LC08_L1TP_046027_20200316_20200822_02_T1_B11.TIF"
    FILE_NAME_QUALITY_L1_PIXEL = "LC08_L1TP_046027_20200316_20200822_02_T1_QA_PIXEL.TIF"
    FILE_NAME_QUALITY_L1_RADIOMETRIC_SATURATION = "LC08_L1TP_046027_20200316_20200822_02_T1_QA_RADSAT.TIF"
    FILE_NAME_ANGLE_COEFFICIENT = "LC08_L1TP_046027_20200316_20200822_02_T1_ANG.txt"
    FILE_NAME_ANGLE_SENSOR_AZIMUTH_BAND_4 = "LC08_L1TP_046027_20200316_20200822_02_T1_VAA.TIF"
    FILE_NAME_ANGLE_SENSOR_ZENITH_BAND_4 = "LC08_L1TP_046027_20200316_20200822_02_T1_VZA.TIF"
    FILE_NAME_ANGLE_SOLAR_AZIMUTH_BAND_4 = "LC08_L1TP_046027_20200316_20200822_02_T1_SAA.TIF"
    FILE_NAME_ANGLE_SOLAR_ZENITH_BAND_4 = "LC08_L1TP_046027_20200316_20200822_02_T1_SZA.TIF"
    FILE_NAME_METADATA_ODL = "LC08_L1TP_046027_20200316_20200822_02_T1_MTL.txt"
    FILE_NAME_METADATA_XML = "LC08_L1TP_046027_20200316_20200822_02_T1_MTL.xml"
    FILE_NAME_CPF = "LC08CPF_20191231_20200331_02.01"
    FILE_NAME_BPF_OLI = "LO8BPF20200316184343_20200316193132.02"
    FILE_NAME_BPF_TIRS = "LT8BPF20200310060739_20200324104153.01"
    FILE_NAME_RLUT = "LC08RLUT_20150303_20431231_02_01.h5"
    DATA_SOURCE_TIRS_STRAY_LIGHT_CORRECTION = "TIRS"
    DATA_SOURCE_ELEVATION = "GLS2000"
    GROUND_CONTROL_POINTS_VERSION = 5
    GROUND_CONTROL_POINTS_MODEL = 1200
    GEOMETRIC_RMSE_MODEL = 6.579
    GEOMETRIC_RMSE_MODEL_Y = 5.469
    GEOMETRIC_RMSE_MODEL_X = 3.657
    GROUND_CONTROL_POINTS_VERIFY = 524
    GEOMETRIC_RMSE_VERIFY = 5.253
  END_GROUP = LEVEL1_PROCESSING_RECORD
  GROUP = LEVEL1_MIN_MAX_RADIANCE
    RADIANCE_MAXIMUM_BAND_1 = 767.78308
    RADIANCE_MINIMUM_BAND_1 = -63.40377
    RADIANCE_MAXIMUM_BAND_2 = 786.21924
    RADIANCE_MINIMUM_BAND_2 = -64.92624
    RADIANCE_MAXIMUM_BAND_3 = 724.49426
    RADIANCE_MINIMUM_BAND_3 = -59.82898
    RADIANCE_MAXIMUM_BAND_4 = 610.93439
    RADIANCE_MINIMUM_BAND_4 = -50.45116
    RADIANCE_MAXIMUM_BAND_5 = 373.86139
    RADIANCE_MINIMUM_BAND_5 = -30.87359
    RADIANCE_MAXIMUM_BAND_6 = 92.97593
    RADIANCE_MINIMUM_BAND_6 = -7.67798
    RADIANCE_MAXIMUM_BAND_7 = 31.33786
    RADIANCE_MINIMUM_BAND_7 = -2.58789
    RADIANCE_MAXIMUM_BAND_8 = 691.40961
    RADIANCE_MINIMUM_BAND_8 = -57.09683
    RADIANCE_MAXIMUM_BAND_9 = 146.11351
    RADIANCE_MINIMUM_BAND_9 = -12.06610
    RADIANCE_MAXIMUM_BAND_10 = 22.00180
    RADIANCE_MINIMUM_BAND_10 = 0.10033
    RADIANCE_MAXIMUM_BAND_11 = 22.00180
    RADIANCE_MINIMUM_BAND_11 = 0.10033
  END_GROUP = LEVEL1_MIN_MAX_RADIANCE
  GROUP = LEVEL1_MIN_MAX_REFLECTANCE
    REFLECTANCE_MAXIMUM_BAND_1 = 1.210700
    REFLECTANCE_MINIMUM_BAND_1 = -0.099980
    REFLECTANCE_MAXIMUM_BAND_2 = 1.210700
    REFLECTANCE_MINIMUM_BAND_2 = -0.099980
    REFLECTANCE_MAXIMUM_BAND_3 = 1.210700
    REFLECTANCE_MINIMUM_BAND_3 = -0.099980
    REFLECTANCE_MAXIMUM_BAND_4 = 1.210700
    REFLECTANCE_MINIMUM_BAND_4 = -0.099980
    REFLECTANCE_MAXIMUM_BAND_5 = 1.210700
    REFLECTANCE_MINIMUM_BAND_5 = -0.099980
    REFLECTANCE_MAXIMUM_BAND_6 = 1.210700
    REFLECTANCE_MINIMUM_BAND_6 = -0.099980
    REFLECTANCE_MAXIMUM_BAND_7 = 1.210700
    REFLECTANCE_MINIMUM_BAND_7 = -0.099980
    REFLECTANCE_MAXIMUM_BAND_8 = 1.210700
    REFLECTANCE_MINIMUM_BAND_8 = -0.099980
    REFLECTANCE_MAXIMUM_BAND_9 = 1.210700
    REFLECTANCE_MINIMUM_BAND_9 = -0.099980
  END_GROUP = LEVEL1_MIN_MAX_REFLECTANCE
  GROUP = LEVEL1_MIN_MAX_PIXEL_VALUE
    QUANTIZE_CAL_MAX_BAND_1 = 65535
    QUANTIZE_CAL_MIN_BAND_1 = 1
    QUANTIZE_CAL_MAX_BAND_2 = 65535
    QUANTIZE_CAL_MIN_BAND_2 = 1
    QUANTIZE_CAL_MAX_BAND_3 = 65535
    QUANTIZE_CAL_MIN_BAND_3 = 1
    QUANTIZE_CAL_MAX_BAND_4 = 65535
    QUANTIZE_CAL_MIN_BAND_4 = 1
    QUANTIZE_CAL_MAX_BAND_5 = 65535
    QUANTIZE_CAL_MIN_BAND_5 = 1
    QUANTIZE_CAL_MAX_BAND_6 = 65535
    QUANTIZE_CAL_MIN_BAND_6 = 1
    QUANTIZE_CAL_MAX_BAND_7 = 65535
    QUANTIZE_CAL_MIN_BAND_7 = 1
    QUANTIZE_CAL_MAX_BAND_8 = 65535
    QUANTIZE_CAL_MIN_BAND_8 = 1
    QUANTIZE_CAL_MAX_BAND_9 = 65535
    QUANTIZE_CAL_MIN_BAND_9 = 1
    QUANTIZE_CAL_MAX_BAND_10 = 65535
    QUANTIZE_CAL_MIN_BAND_10 = 1
    QUANTIZE_CAL_MAX_BAND_11 = 65535
    QUANTIZE_CAL_MIN_BAND_11 = 1
  END_GROUP = LEVEL1_MIN_MAX_PIXEL_VALUE
  GROUP = LEVEL1_RADIOMETRIC_RESCALING
    RADIANCE_MULT_BAND_1 = 1.2683E-02
    RADIANCE_MULT_BAND_2 = 1.2988E-02
    RADIANCE_MULT_BAND_3 = 1.1968E-02
    RADIANCE_MULT_BAND_4 = 1.0092E-02
    RADIANCE_MULT_BAND_5 = 6.1760E-03
    RADIANCE_MULT_BAND_6 = 1.5359E-03
    RADIANCE_MULT_BAND_7 = 5.1768E-04
    RADIANCE_MULT_BAND_8 = 1.1422E-02
    RADIANCE_MULT_BAND_9 = 2.4137E-03
    RADIANCE_MULT_BAND_10 = 3.3420E-04
    RADIANCE_MULT_BAND_11 = 3.3420E-04
    RADIANCE_ADD_BAND_1 = -63.41646
    RADIANCE_ADD_BAND_2 = -64.93923
    RADIANCE_ADD_BAND_3 = -59.84094
    RADIANCE_ADD_BAND_4 = -50.46125
    RADIANCE_ADD_BAND_5 = -30.87977
    RADIANCE_ADD_BAND_6 = -7.67952
    RADIANCE_ADD_BAND_7 = -2.58841
    RADIANCE_ADD_BAND_8 = -57.10825
    RADIANCE_ADD_BAND_9 = -12.06851
    RADIANCE_ADD_BAND_10 = 0.10000
    RADIANCE_ADD_BAND_11 = 0.10000
    REFLECTANCE_MULT_BAND_1 = 2.0000E-05
    REFLECTANCE_MULT_BAND_2 = 2.0000E-05
    REFLECTANCE_MULT_BAND_3 = 2.0000E-05
    REFLECTANCE_MULT_BAND_4 = 2.0000E-05
    REFLECTANCE_MULT_BAND_5 = 2.0000E-05
    REFLECTANCE_MULT_BAND_6 = 2.0000E-05
    REFLECTANCE_MULT_BAND_7 = 2.0000E-05
    REFLECTANCE_MULT_BAND_8 = 2.0000E-05
    REFLECTANCE_MULT_BAND_9 = 2.0000E-05
    REFLECTANCE_ADD_BAND_1 = -0.100000
    REFLECTANCE_ADD_BAND_2 = -0.100000
    REFLECTANCE_ADD_BAND_3 = -0.100000
    REFLECTANCE_ADD_BAND_4 = -0.100000
    REFLECTANCE_ADD_BAND_5 = -0.100000
    REFLECTANCE_ADD_BAND_6 = -0.100000
    REFLECTANCE_ADD_BAND_7 = -0.100000
    REFLECTANCE_ADD_BAND_8 = -0.100000
    REFLECTANCE_ADD_BAND_9 = -0.100000
  END_GROUP = LEVEL1_RADIOMETRIC_RESCALING
  GROUP = LEVEL1_THERMAL_CONSTANTS
    K1_CONSTANT_BAND_10 = 774.8853
    K2_CONSTANT_BAND_10 = 1321.0789
    K1_CONSTANT_BAND_11 = 480.8883
    K2_CONSTANT_BAND_11 = 1201.1442
  END_GROUP = LEVEL1_THERMAL_CONSTANTS
  GROUP = LEVEL1_PROJECTION_PARAMETERS
    MAP_PROJECTION = "UTM"
    DATUM = "WGS84"
    ELLIPSOID = "WGS84"
    UTM_ZONE = 10
    GRID_CELL_SIZE_PANCHROMATIC = 15.00
    GRID_CELL_SIZE_REFLECTIVE = 30.00
    GRID_CELL_SIZE_THERMAL = 30.00
    ORIENTATION = "NORTH_UP"
    RESAMPLING_OPTION = "CUBIC_CONVOLUTION"
  END_GROUP = LEVEL1_PROJECTION_PARAMETERS
END_GROUP = LANDSAT_METADATA_FILE
END

MOD09GA

md5sum

md5sum MOD09GA.A2022224.h09v04.061.2022226033159.hdf

00ea1272e7b3d3a9ac2831d03405a056  MOD09GA.A2022224.h09v04.061.2022226033159.hdf

Metadata

gdalinfo MOD09GA.A2022224.h09v04.061.2022226033159.hdf

Driver: HDF4/Hierarchical Data Format Release 4
Files: MOD09GA.A2022224.h09v04.061.2022226033159.hdf
Size is 512, 512
Metadata:
  ADDITIONALLAYERS1KM=12
  ADDITIONALLAYERS500M=1
  ASSOCIATEDINSTRUMENTSHORTNAME.1=MODIS
  ASSOCIATEDPLATFORMSHORTNAME.1=Terra
  ASSOCIATEDSENSORSHORTNAME.1=MODIS
  AUTOMATICQUALITYFLAG.1=Passed
  AUTOMATICQUALITYFLAGEXPLANATION.1=No automatic quality assessment is performed in the PGE
  CHARACTERISTICBINANGULARSIZE1KM=30.0
  CHARACTERISTICBINANGULARSIZE500M=15.0
  CHARACTERISTICBINSIZE1KM=926.625433055556
  CHARACTERISTICBINSIZE500M=463.312716527778
  CLOUDOPTION=MOD09 internally-derived
  COVERAGECALCULATIONMETHOD=volume
  COVERAGEMINIMUM=0.00999999977648258
  DATACOLUMNS1KM=1200
  DATACOLUMNS500M=2400
  DATAROWS1KM=1200
  DATAROWS500M=2400
  DAYNIGHTFLAG=Day
  DEEPOCEANFLAG=Yes
  DESCRREVISION=6.1
  EASTBOUNDINGCOORDINATE=-104.421704737634
  EQUATORCROSSINGDATE.1=2022-08-12
  EQUATORCROSSINGDATE.2=2022-08-12
  EQUATORCROSSINGDATE.3=2022-08-12
  EQUATORCROSSINGLONGITUDE.1=-110.871169674779
  EQUATORCROSSINGLONGITUDE.2=-135.592908318006
  EQUATORCROSSINGLONGITUDE.3=-160.314090231185
  EQUATORCROSSINGTIME.1=17:40:17.711427
  EQUATORCROSSINGTIME.2=19:19:10.698437
  EQUATORCROSSINGTIME.3=20:58:03.616566
  EXCLUSIONGRINGFLAG.1=N
  FIRSTLAYERSELECTIONCRITERIA=order of input pointer
  GEOANYABNORMAL=False
  GEOESTMAXRMSERROR=50.0
  GLOBALGRIDCOLUMNS1KM=43200
  GLOBALGRIDCOLUMNS500M=86400
  GLOBALGRIDROWS1KM=21600
  GLOBALGRIDROWS500M=43200
  GRANULEBEGINNINGDATETIME=2022-08-12T17:25:00.000000Z
  GRANULEBEGINNINGDATETIMEARRAY=2022-08-12T17:25:00.000000Z, 2022-08-12T19:00:00.000000Z, 2022-08-12T19:05:00.000000Z, 2022-08-12T20:40:00.000000Z, 2022-08-12T20:45:00.000000Z
  GRANULEDAYNIGHTFLAG=Day
  GRANULEDAYNIGHTFLAGARRAY=Day, Day, Day, Day, Day
  GRANULEDAYOFYEAR=224
  GRANULEENDINGDATETIME=2022-08-12T20:45:00.000000Z
  GRANULEENDINGDATETIMEARRAY=2022-08-12T17:30:00.000000Z, 2022-08-12T19:05:00.000000Z, 2022-08-12T19:10:00.000000Z, 2022-08-12T20:45:00.000000Z, 2022-08-12T20:50:00.000000Z
  GRANULENUMBERARRAY=211, 230, 231, 250, 251, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1
  GRANULEPOINTERARRAY=0, -1, 1, 2, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1
  GRINGPOINTLATITUDE.1=39.7342308150748, 49.9394187999602, 50.1159178280076, 39.8623890159424
  GRINGPOINTLONGITUDE.1=-117.746445975456, -140.795234672207, -124.615349244084, -104.235445821904
  GRINGPOINTSEQUENCENO.1=1, 2, 3, 4
  HDFEOSVersion=HDFEOS_V2.19
  HORIZONTALTILENUMBER=9
  identifier_product_doi=10.5067/MODIS/MOD09GA.061
  identifier_product_doi=10.5067/MODIS/MOD09GA.061
  identifier_product_doi_authority=http://dx.doi.org
  identifier_product_doi_authority=http://dx.doi.org
  INPUTPOINTER=MOD09GST.A2022224.h09v04.061.2022226032927.hdf, MOD09GHK.A2022224.h09v04.061.2022226033012.hdf, MOD09GQK.A2022224.h09v04.061.2022226032957.hdf, MODPT1KD.A2022224.h09v04.061.2022226032741.hdf, MODPTHKM.A2022224.h09v04.061.2022226032741.hdf, MODPTQKM.A2022224.h09v04.061.2022226032741.hdf, MODMGGAD.A2022224.h09v04.061.2022226032745.hdf, MODTBGD.A2022224.h09v04.061.2022226033019.hdf, MODOCGD.A2022224.h09v04.061.2022226033026.hdf, MOD10L2G.A2022224.h09v04.061.2022226032851.hdf, DEM_SN_H.h09v04.006_0.hdf, MCDLCHKM.A2010001.h09v04.051.2014287174137.hdf
  KEEPALL=No
  L2GSTORAGEFORMAT1KM=compact
  L2GSTORAGEFORMAT500M=compact
  l2g_storage_format_1km=compact
  l2g_storage_format_500m=compact
  LOCALGRANULEID=MOD09GA.A2022224.h09v04.061.2022226033159.hdf
  LOCALVERSIONID=6.0.9
  LONGNAME=MODIS/Terra Surface Reflectance Daily L2G Global 1km and 500m SIN Grid
  MAXIMUMOBSERVATIONS1KM=13
  MAXIMUMOBSERVATIONS500M=2
  maximum_observations_1km=13
  maximum_observations_500m=2
  MAXOUTPUTRES=QKM
  NADIRDATARESOLUTION1KM=1km
  NADIRDATARESOLUTION500M=500m
  NORTHBOUNDINGCOORDINATE=49.9999999955098
  NumberLandWater1km=23171, 958159, 16958, 9742, 0, 1, 43681, 388288, 0
  NumberLandWater500m=46342, 1916318, 33916, 19484, 0, 2, 87362, 776576, 0
  NUMBEROFGRANULES=1
  NUMBEROFINPUTGRANULES=5
  NUMBEROFORBITS=3
  NUMBEROFOVERLAPGRANULES=3
  ORBITNUMBER.1=120481
  ORBITNUMBER.2=120482
  ORBITNUMBER.3=120483
  ORBITNUMBERARRAY=120481, -1, 120482, 120483, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1
  PARAMETERNAME.1=MOD09G
  PERCENTCLOUDY=55
  PERCENTLAND=62
  PERCENTLANDSEAMASKCLASS=4, 62, 5, 1, 0, 0, 2, 27
  PERCENTLOWSUN=0
  PERCENTPROCESSED=100
  PERCENTSHADOW=5
  PGEVERSION=6.1.9
  PROCESSINGCENTER=MODAPS
  PROCESSINGENVIRONMENT=Linux minion20223 5.4.0-122-generic #138-Ubuntu SMP Wed Jun 22 15:00:31 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
  PROCESSVERSION=6.0.9
  PRODUCTIONDATETIME=2022-08-14T03:31:59.000Z
  QAPERCENTGOODQUALITY=99
  QAPERCENTINTERPOLATEDDATA.1=0
  QAPERCENTMISSINGDATA.1=0
  QAPERCENTNOTPRODUCEDCLOUD=0
  QAPERCENTNOTPRODUCEDOTHER=0
  QAPERCENTOTHERQUALITY=1
  QAPERCENTOUTOFBOUNDSDATA.1=0
  QAPERCENTPOOROUTPUT500MBAND1=1
  QAPERCENTPOOROUTPUT500MBAND2=1
  QAPERCENTPOOROUTPUT500MBAND3=1
  QAPERCENTPOOROUTPUT500MBAND4=0
  QAPERCENTPOOROUTPUT500MBAND5=1
  QAPERCENTPOOROUTPUT500MBAND6=0
  QAPERCENTPOOROUTPUT500MBAND7=0
  QUALITYCLASSPERCENTAGE500MBAND1=99, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0
  QUALITYCLASSPERCENTAGE500MBAND2=98, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0
  QUALITYCLASSPERCENTAGE500MBAND3=99, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0
  QUALITYCLASSPERCENTAGE500MBAND4=100, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
  QUALITYCLASSPERCENTAGE500MBAND5=86, 0, 0, 0, 0, 0, 0, 9, 4, 0, 0, 0, 0, 1, 0, 0
  QUALITYCLASSPERCENTAGE500MBAND6=100, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
  QUALITYCLASSPERCENTAGE500MBAND7=100, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
  RANGEBEGINNINGDATE=2022-08-12
  RANGEBEGINNINGTIME=00:00:00
  RANGEENDINGDATE=2022-08-12
  RANGEENDINGTIME=23:59:59
  RANKING=No
  REPROCESSINGACTUAL=processed once
  REPROCESSINGPLANNED=further update is anticipated
  RESOLUTIONBANDS1AND2=500
  SCIENCEQUALITYFLAG.1=Not Investigated
  SCIENCEQUALITYFLAGEXPLANATION.1=See http://landweb.nascom.nasa.gov/cgi-bin/QA_WWW/qaFlagPage.cgi?sat=terra for the product Science Quality status.
  SHORTNAME=MOD09GA
  SOUTHBOUNDINGCOORDINATE=39.9999999964079
  SPSOPARAMETERS=2015
  SYSTEMFILENAME=MOD09GST.A2022224.h09v04.061.2022226032927.hdf, MOD09GHK.A2022224.h09v04.061.2022226033012.hdf, MOD09GQK.A2022224.h09v04.061.2022226032957.hdf, MODPT1KD.A2022224.h09v04.061.2022226032741.hdf, MODPTHKM.A2022224.h09v04.061.2022226032741.hdf, MODPTQKM.A2022224.h09v04.061.2022226032741.hdf, MODMGGAD.A2022224.h09v04.061.2022226032745.hdf, MODTBGD.A2022224.h09v04.061.2022226033019.hdf, MODOCGD.A2022224.h09v04.061.2022226033026.hdf, MOD10L2G.A2022224.h09v04.061.2022226032851.hdf
  TileID=51009004
  TOTALADDITIONALOBSERVATIONS1KM=3325893
  TOTALADDITIONALOBSERVATIONS500M=1437070
  TOTALOBSERVATIONS1KM=4765893
  TOTALOBSERVATIONS500M=7197060
  total_additional_observations_1km=3325893
  total_additional_observations_500m=1437070
  VERSIONID=61
  VERTICALTILENUMBER=4
  WESTBOUNDINGCOORDINATE=-140.015144391787
  ZONEIDENTIFIER=Universal Transverse Mercator UTM
Subdatasets:
  SUBDATASET_1_NAME=HDF4_EOS:EOS_GRID:"MOD09GA.A2022224.h09v04.061.2022226033159.hdf":MODIS_Grid_1km_2D:num_observations_1km
  SUBDATASET_1_DESC=[1200x1200] num_observations_1km MODIS_Grid_1km_2D (8-bit integer)
  SUBDATASET_2_NAME=HDF4_EOS:EOS_GRID:"MOD09GA.A2022224.h09v04.061.2022226033159.hdf":MODIS_Grid_1km_2D:state_1km_1
  SUBDATASET_2_DESC=[1200x1200] state_1km_1 MODIS_Grid_1km_2D (16-bit unsigned integer)
  SUBDATASET_3_NAME=HDF4_EOS:EOS_GRID:"MOD09GA.A2022224.h09v04.061.2022226033159.hdf":MODIS_Grid_1km_2D:SensorZenith_1
  SUBDATASET_3_DESC=[1200x1200] SensorZenith_1 MODIS_Grid_1km_2D (16-bit integer)
  SUBDATASET_4_NAME=HDF4_EOS:EOS_GRID:"MOD09GA.A2022224.h09v04.061.2022226033159.hdf":MODIS_Grid_1km_2D:SensorAzimuth_1
  SUBDATASET_4_DESC=[1200x1200] SensorAzimuth_1 MODIS_Grid_1km_2D (16-bit integer)
  SUBDATASET_5_NAME=HDF4_EOS:EOS_GRID:"MOD09GA.A2022224.h09v04.061.2022226033159.hdf":MODIS_Grid_1km_2D:Range_1
  SUBDATASET_5_DESC=[1200x1200] Range_1 MODIS_Grid_1km_2D (16-bit unsigned integer)
  SUBDATASET_6_NAME=HDF4_EOS:EOS_GRID:"MOD09GA.A2022224.h09v04.061.2022226033159.hdf":MODIS_Grid_1km_2D:SolarZenith_1
  SUBDATASET_6_DESC=[1200x1200] SolarZenith_1 MODIS_Grid_1km_2D (16-bit integer)
  SUBDATASET_7_NAME=HDF4_EOS:EOS_GRID:"MOD09GA.A2022224.h09v04.061.2022226033159.hdf":MODIS_Grid_1km_2D:SolarAzimuth_1
  SUBDATASET_7_DESC=[1200x1200] SolarAzimuth_1 MODIS_Grid_1km_2D (16-bit integer)
  SUBDATASET_8_NAME=HDF4_EOS:EOS_GRID:"MOD09GA.A2022224.h09v04.061.2022226033159.hdf":MODIS_Grid_1km_2D:gflags_1
  SUBDATASET_8_DESC=[1200x1200] gflags_1 MODIS_Grid_1km_2D (8-bit unsigned integer)
  SUBDATASET_9_NAME=HDF4_EOS:EOS_GRID:"MOD09GA.A2022224.h09v04.061.2022226033159.hdf":MODIS_Grid_1km_2D:orbit_pnt_1
  SUBDATASET_9_DESC=[1200x1200] orbit_pnt_1 MODIS_Grid_1km_2D (8-bit integer)
  SUBDATASET_10_NAME=HDF4_EOS:EOS_GRID:"MOD09GA.A2022224.h09v04.061.2022226033159.hdf":MODIS_Grid_1km_2D:granule_pnt_1
  SUBDATASET_10_DESC=[1200x1200] granule_pnt_1 MODIS_Grid_1km_2D (8-bit unsigned integer)
  SUBDATASET_11_NAME=HDF4_EOS:EOS_GRID:"MOD09GA.A2022224.h09v04.061.2022226033159.hdf":MODIS_Grid_500m_2D:num_observations_500m
  SUBDATASET_11_DESC=[2400x2400] num_observations_500m MODIS_Grid_500m_2D (8-bit integer)
  SUBDATASET_12_NAME=HDF4_EOS:EOS_GRID:"MOD09GA.A2022224.h09v04.061.2022226033159.hdf":MODIS_Grid_500m_2D:sur_refl_b01_1
  SUBDATASET_12_DESC=[2400x2400] sur_refl_b01_1 MODIS_Grid_500m_2D (16-bit integer)
  SUBDATASET_13_NAME=HDF4_EOS:EOS_GRID:"MOD09GA.A2022224.h09v04.061.2022226033159.hdf":MODIS_Grid_500m_2D:sur_refl_b02_1
  SUBDATASET_13_DESC=[2400x2400] sur_refl_b02_1 MODIS_Grid_500m_2D (16-bit integer)
  SUBDATASET_14_NAME=HDF4_EOS:EOS_GRID:"MOD09GA.A2022224.h09v04.061.2022226033159.hdf":MODIS_Grid_500m_2D:sur_refl_b03_1
  SUBDATASET_14_DESC=[2400x2400] sur_refl_b03_1 MODIS_Grid_500m_2D (16-bit integer)
  SUBDATASET_15_NAME=HDF4_EOS:EOS_GRID:"MOD09GA.A2022224.h09v04.061.2022226033159.hdf":MODIS_Grid_500m_2D:sur_refl_b04_1
  SUBDATASET_15_DESC=[2400x2400] sur_refl_b04_1 MODIS_Grid_500m_2D (16-bit integer)
  SUBDATASET_16_NAME=HDF4_EOS:EOS_GRID:"MOD09GA.A2022224.h09v04.061.2022226033159.hdf":MODIS_Grid_500m_2D:sur_refl_b05_1
  SUBDATASET_16_DESC=[2400x2400] sur_refl_b05_1 MODIS_Grid_500m_2D (16-bit integer)
  SUBDATASET_17_NAME=HDF4_EOS:EOS_GRID:"MOD09GA.A2022224.h09v04.061.2022226033159.hdf":MODIS_Grid_500m_2D:sur_refl_b06_1
  SUBDATASET_17_DESC=[2400x2400] sur_refl_b06_1 MODIS_Grid_500m_2D (16-bit integer)
  SUBDATASET_18_NAME=HDF4_EOS:EOS_GRID:"MOD09GA.A2022224.h09v04.061.2022226033159.hdf":MODIS_Grid_500m_2D:sur_refl_b07_1
  SUBDATASET_18_DESC=[2400x2400] sur_refl_b07_1 MODIS_Grid_500m_2D (16-bit integer)
  SUBDATASET_19_NAME=HDF4_EOS:EOS_GRID:"MOD09GA.A2022224.h09v04.061.2022226033159.hdf":MODIS_Grid_500m_2D:QC_500m_1
  SUBDATASET_19_DESC=[2400x2400] QC_500m_1 MODIS_Grid_500m_2D (32-bit unsigned integer)
  SUBDATASET_20_NAME=HDF4_EOS:EOS_GRID:"MOD09GA.A2022224.h09v04.061.2022226033159.hdf":MODIS_Grid_500m_2D:obscov_500m_1
  SUBDATASET_20_DESC=[2400x2400] obscov_500m_1 MODIS_Grid_500m_2D (8-bit integer)
  SUBDATASET_21_NAME=HDF4_EOS:EOS_GRID:"MOD09GA.A2022224.h09v04.061.2022226033159.hdf":MODIS_Grid_500m_2D:iobs_res_1
  SUBDATASET_21_DESC=[2400x2400] iobs_res_1 MODIS_Grid_500m_2D (8-bit unsigned integer)
  SUBDATASET_22_NAME=HDF4_EOS:EOS_GRID:"MOD09GA.A2022224.h09v04.061.2022226033159.hdf":MODIS_Grid_500m_2D:q_scan_1
  SUBDATASET_22_DESC=[2400x2400] q_scan_1 MODIS_Grid_500m_2D (8-bit unsigned integer)
Corner Coordinates:
Upper Left  (    0.0,    0.0)
Lower Left  (    0.0,  512.0)
Upper Right (  512.0,    0.0)
Lower Right (  512.0,  512.0)
Center      (  256.0,  256.0)

Test data creation

The following commands create the test data subset for rapid testing. Additional testing can always be done on the complete scene.

Workflow is:

• Subset to a X by Y pixel subset
• Remove all bands and files not used by SCAG
• Edit the MTL JSON file to match X by Y
• Re-tar so it presents as a ‘normal’ LANDSAT tarball to the SCAG workflow

Landsat 8

rm -fR LC08_L2SP_046027_20200316_20200822_02_T1
mkdir -p LC08_L2SP_046027_20200316_20200822_02_T1
tar xvf ../0_upstream/LC08_L2SP_046027_20200316_20200822_02_T1.tar -C LC08_L2SP_046027_20200316_20200822_02_T1

cd LC08_L2SP_046027_20200316_20200822_02_T1

# remove unused files
rm *_ANG.txt *_{QA_PIXEL,QA_RADSAT,SR_QA_AEROSOL}.TIF *_T1_ST*.TIF *_thumb_*.jpeg *_stac.json

# crop remaining TIFF files
xs=1500
ys=10
x0=593924
y0=5189256
x1=$(( 593924 + ${xs}*30 ))
y1=$(( 5189256 + ${ys}*30 ))
for f in *.TIF; do
  gdalwarp -te ${x0} ${y0} ${x1} ${y1}  ${f} tmp.tif
  mv tmp.tif ${f}
done

# Adjust MTL JSON file used by BIPifier
cat LC08_L2SP_046027_20200316_20200822_02_T1_MTL.json \
    | jq '.LANDSAT_METADATA_FILE.PROJECTION_ATTRIBUTES.REFLECTIVE_LINES = '"${ys}"  \
    | jq '.LANDSAT_METADATA_FILE.PROJECTION_ATTRIBUTES.REFLECTIVE_SAMPLES = '"${xs}" \
    | jq '.LANDSAT_METADATA_FILE.PROJECTION_ATTRIBUTES.CORNER_UL_PROJECTION_X_PRODUCT = '"${x0}" \
    | jq '.LANDSAT_METADATA_FILE.PROJECTION_ATTRIBUTES.CORNER_UL_PROJECTION_Y_PRODUCT = '"${y1}" \
    | jq '.LANDSAT_METADATA_FILE.PROJECTION_ATTRIBUTES.CORNER_LR_PROJECTION_X_PRODUCT = '"${x1}" \
    | jq '.LANDSAT_METADATA_FILE.PROJECTION_ATTRIBUTES.CORNER_LR_PROJECTION_Y_PRODUCT = '"${y0}" \
	 > tmp.json
mv tmp.json LC08_L2SP_046027_20200316_20200822_02_T1_MTL.json

tar cvf LC08_L2SP_046027_20200316_20200822_02_T1_subset_${xs}x${ys}.tar *
mv LC08_L2SP_046027_20200316_20200822_02_T1_subset_${xs}x${ys}.tar ../
cd ../

LC08_L2SP_046027_20200316_20200822_02_T1_ANG.txt
LC08_L2SP_046027_20200316_20200822_02_T1_MTL.json
LC08_L2SP_046027_20200316_20200822_02_T1_MTL.txt
LC08_L2SP_046027_20200316_20200822_02_T1_MTL.xml
LC08_L2SP_046027_20200316_20200822_02_T1_QA_PIXEL.TIF
LC08_L2SP_046027_20200316_20200822_02_T1_QA_RADSAT.TIF
LC08_L2SP_046027_20200316_20200822_02_T1_SR_B1.TIF
LC08_L2SP_046027_20200316_20200822_02_T1_SR_B2.TIF
LC08_L2SP_046027_20200316_20200822_02_T1_SR_B3.TIF
LC08_L2SP_046027_20200316_20200822_02_T1_SR_B4.TIF
LC08_L2SP_046027_20200316_20200822_02_T1_SR_B5.TIF
LC08_L2SP_046027_20200316_20200822_02_T1_SR_B6.TIF
LC08_L2SP_046027_20200316_20200822_02_T1_SR_B7.TIF
LC08_L2SP_046027_20200316_20200822_02_T1_SR_QA_AEROSOL.TIF
LC08_L2SP_046027_20200316_20200822_02_T1_SR_stac.json
LC08_L2SP_046027_20200316_20200822_02_T1_ST_ATRAN.TIF
LC08_L2SP_046027_20200316_20200822_02_T1_ST_B10.TIF
LC08_L2SP_046027_20200316_20200822_02_T1_ST_CDIST.TIF
LC08_L2SP_046027_20200316_20200822_02_T1_ST_DRAD.TIF
LC08_L2SP_046027_20200316_20200822_02_T1_ST_EMIS.TIF
LC08_L2SP_046027_20200316_20200822_02_T1_ST_EMSD.TIF
LC08_L2SP_046027_20200316_20200822_02_T1_ST_QA.TIF
LC08_L2SP_046027_20200316_20200822_02_T1_ST_TRAD.TIF
LC08_L2SP_046027_20200316_20200822_02_T1_ST_URAD.TIF
LC08_L2SP_046027_20200316_20200822_02_T1_ST_stac.json
LC08_L2SP_046027_20200316_20200822_02_T1_thumb_large.jpeg
LC08_L2SP_046027_20200316_20200822_02_T1_thumb_small.jpeg
Creating output file that is 1500P x 10L.
Processing LC08_L2SP_046027_20200316_20200822_02_T1_SR_B1.TIF [1/1] : 0Using internal nodata values (e.g. 0) for image LC08_L2SP_046027_20200316_20200822_02_T1_SR_B1.TIF.
Copying nodata values from source LC08_L2SP_046027_20200316_20200822_02_T1_SR_B1.TIF to destination tmp.tif.
...10...20...30...40...50...60...70...80...90...100 - done.
Creating output file that is 1500P x 10L.
Processing LC08_L2SP_046027_20200316_20200822_02_T1_SR_B2.TIF [1/1] : 0Using internal nodata values (e.g. 0) for image LC08_L2SP_046027_20200316_20200822_02_T1_SR_B2.TIF.
Copying nodata values from source LC08_L2SP_046027_20200316_20200822_02_T1_SR_B2.TIF to destination tmp.tif.
...10...20...30...40...50...60...70...80...90...100 - done.
Creating output file that is 1500P x 10L.
Processing LC08_L2SP_046027_20200316_20200822_02_T1_SR_B3.TIF [1/1] : 0Using internal nodata values (e.g. 0) for image LC08_L2SP_046027_20200316_20200822_02_T1_SR_B3.TIF.
Copying nodata values from source LC08_L2SP_046027_20200316_20200822_02_T1_SR_B3.TIF to destination tmp.tif.
...10...20...30...40...50...60...70...80...90...100 - done.
Creating output file that is 1500P x 10L.
Processing LC08_L2SP_046027_20200316_20200822_02_T1_SR_B4.TIF [1/1] : 0Using internal nodata values (e.g. 0) for image LC08_L2SP_046027_20200316_20200822_02_T1_SR_B4.TIF.
Copying nodata values from source LC08_L2SP_046027_20200316_20200822_02_T1_SR_B4.TIF to destination tmp.tif.
...10...20...30...40...50...60...70...80...90...100 - done.
Creating output file that is 1500P x 10L.
Processing LC08_L2SP_046027_20200316_20200822_02_T1_SR_B5.TIF [1/1] : 0Using internal nodata values (e.g. 0) for image LC08_L2SP_046027_20200316_20200822_02_T1_SR_B5.TIF.
Copying nodata values from source LC08_L2SP_046027_20200316_20200822_02_T1_SR_B5.TIF to destination tmp.tif.
...10...20...30...40...50...60...70...80...90...100 - done.
Creating output file that is 1500P x 10L.
Processing LC08_L2SP_046027_20200316_20200822_02_T1_SR_B6.TIF [1/1] : 0Using internal nodata values (e.g. 0) for image LC08_L2SP_046027_20200316_20200822_02_T1_SR_B6.TIF.
Copying nodata values from source LC08_L2SP_046027_20200316_20200822_02_T1_SR_B6.TIF to destination tmp.tif.
...10...20...30...40...50...60...70...80...90...100 - done.
Creating output file that is 1500P x 10L.
Processing LC08_L2SP_046027_20200316_20200822_02_T1_SR_B7.TIF [1/1] : 0Using internal nodata values (e.g. 0) for image LC08_L2SP_046027_20200316_20200822_02_T1_SR_B7.TIF.
Copying nodata values from source LC08_L2SP_046027_20200316_20200822_02_T1_SR_B7.TIF to destination tmp.tif.
...10...20...30...40...50...60...70...80...90...100 - done.
LC08_L2SP_046027_20200316_20200822_02_T1_MTL.json
LC08_L2SP_046027_20200316_20200822_02_T1_MTL.txt
LC08_L2SP_046027_20200316_20200822_02_T1_MTL.xml
LC08_L2SP_046027_20200316_20200822_02_T1_SR_B1.TIF
LC08_L2SP_046027_20200316_20200822_02_T1_SR_B2.TIF
LC08_L2SP_046027_20200316_20200822_02_T1_SR_B3.TIF
LC08_L2SP_046027_20200316_20200822_02_T1_SR_B4.TIF
LC08_L2SP_046027_20200316_20200822_02_T1_SR_B5.TIF
LC08_L2SP_046027_20200316_20200822_02_T1_SR_B6.TIF
LC08_L2SP_046027_20200316_20200822_02_T1_SR_B7.TIF

MOD09GA

This is easier to subset than the Landsat tarballs because of the HDF (NetCDF) file format.

import numpy as np
import xarray as xr
ds = xr.open_dataset('./0_upstream/MOD09GA.A2022224.h09v04.061.2022226033159.hdf' , engine='netcdf4')
v = [_ for _ in ds.variables if (('sur_refl_b' not in _) & ('SolarZenith_1' not in _))]
ds = ds.drop_vars(v)
ds = ds.isel({'YDim:MODIS_Grid_500m_2D':np.arange(100),
              'XDim:MODIS_Grid_500m_2D':np.arange(150)})

ds.to_netcdf('./1_subset/MOD09GA.A2022224.h09v04.061.2022226033159_subset.hdf')

md5sum MOD09GA.A2022224.h09v04.061.2022226033159_subset.hdf

ad976133ab597327a4d38d567307ddf0  MOD09GA.A2022224.h09v04.061.2022226033159_subset.hdf

BIPified data

Generate SCAG BIP data

eval "$(conda shell.bash hook)"
conda activate scag

# LC08
~/projects/NSIDC/scag/scripts/BIPifier.py \
  -i ./1_subset/LC08_L2SP_046027_20200316_20200822_02_T1 \
  -o ./2_scag_bip/LC08_L2SP_046027_20200316_20200822_02_T1.bip

# MOD09
~/projects/NSIDC/scag/scripts/BIPifier.py \
  -i ./1_subset/MOD09GA.A2022224.h09v04.061.2022226033159_subset.hdf \
  -o ./2_scag_bip/MOD09GA.A2022224.h09v04.061.2022226033159.bip

SCAG BIP md5sum hashes

md5sum 2_scag_bip/*

SCAG BIP contents

for f in *meta; do
  echo ${f}
  cat -n ${f}
  echo ""
done

LC08_L2SP_046027_20200316_20200822_02_T1.bip.meta
     1	SOURCE_FILE=/home/kdm/projects/NSIDC/scag_test_data/1_subset/LC08_L2SP_046027_20200316_20200822_02_T1
     2	SENSOR=OLI
     3	NLINES=10
     4	NSAMPLES=1500
     5	NBANDS=6
     6	SUN_ZENITH=51.84235926
     7	PROJ_STRING="+proj=utm +zone=10 +datum=WGS84 +units=m +no_defs +ellps=WGS84"
     8	ZONE_NUMBER=10
     9	ELLIPSOID=WGS84
    10	DATUM=WGS84
    11	GRID_CELL_SIZE_REFLECTIVE=30.0
    12	CORNER_UL_PROJECTION_X_PRODUCT=593909.0
    13	CORNER_UL_PROJECTION_Y_PRODUCT=5189571.0
    14	CORNER_LR_PROJECTION_X_PRODUCT=638939.0
    15	CORNER_LR_PROJECTION_Y_PRODUCT=5189241.0

MOD09GA.A2022224.h09v04.061.2022226033159.bip.meta
     1	SOURCE_FILE=/home/kdm/projects/NSIDC/scag_test_data/1_subset/MOD09GA.A2022224.h09v04.061.2022226033159_subset.hdf
     2	SENSOR=MODIS
     3	NLINES=100
     4	NSAMPLES=150
     5	NBANDS=7
     6	SUN_ZENITH=0.3270000076293945
     7	PROJ_STRING="+proj=sinu +R=6371007.181 +nadgrids=@null +wktext"
     8	ZONE_NUMBER=h09v04
     9	ELLIPSOID=6371007.181
    10	DATUM=WGS84
    11	GRID_CELL_SIZE_REFLECTIVE=463.312716527778
    12	CORNER_UL_PROJECTION_X_PRODUCT=-10007554.677
    13	CORNER_UL_PROJECTION_Y_PRODUCT=5559752.598333
    14	CORNER_LR_PROJECTION_X_PRODUCT=-8895604.157333
    15	CORNER_LR_PROJECTION_Y_PRODUCT=4447802.078667

SCAG outputs

Run SCAG

• This is presumed to be run from the scag home folder.
• We save the BIN files in case a gdal upgrade changes the GeoTIFF md5sum hash

SCAG_TEST_DATA=$(readlink -f .)
SRC=${SCAG_TEST_DATA}/2_scag_bip/LC08_L2SP_046027_20200316_20200822_02_T1.bip
DEST=${SCAG_TEST_DATA}/3_output/LC08/
cd ../scag
./scag.sh -i ${SRC} -o ${DEST}  -w tmp --debug
cp ./tmp/*.bin  ${SCAG_TEST_DATA}/3_output/LC08/

SCAG output md5sum hashes

md5sum 3_output/LC08/*

gdalinfo -mm LC08_L2SP_046027_20200316_20200822_02_T1.snow.tif

Driver: GTiff/GeoTIFF
Files: LC08_L2SP_046027_20200316_20200822_02_T1.snow.tif
Size is 1500, 10
Coordinate System is:
PROJCRS["WGS 84 / UTM zone 10N",
    BASEGEOGCRS["WGS 84",
        DATUM["World Geodetic System 1984",
            ELLIPSOID["WGS 84",6378137,298.257223563,
                LENGTHUNIT["metre",1]]],
        PRIMEM["Greenwich",0,
            ANGLEUNIT["degree",0.0174532925199433]],
        ID["EPSG",4326]],
    CONVERSION["UTM zone 10N",
        METHOD["Transverse Mercator",
            ID["EPSG",9807]],
        PARAMETER["Latitude of natural origin",0,
            ANGLEUNIT["degree",0.0174532925199433],
            ID["EPSG",8801]],
        PARAMETER["Longitude of natural origin",-123,
            ANGLEUNIT["degree",0.0174532925199433],
            ID["EPSG",8802]],
        PARAMETER["Scale factor at natural origin",0.9996,
            SCALEUNIT["unity",1],
            ID["EPSG",8805]],
        PARAMETER["False easting",500000,
            LENGTHUNIT["metre",1],
            ID["EPSG",8806]],
        PARAMETER["False northing",0,
            LENGTHUNIT["metre",1],
            ID["EPSG",8807]]],
    CS[Cartesian,2],
        AXIS["(E)",east,
            ORDER[1],
            LENGTHUNIT["metre",1]],
        AXIS["(N)",north,
            ORDER[2],
            LENGTHUNIT["metre",1]],
    USAGE[
        SCOPE["Engineering survey, topographic mapping."],
        AREA["Between 126°W and 120°W, northern hemisphere between equator and 84°N, onshore and offshore. Canada - British Columbia (BC); Northwest Territories (NWT); Nunavut; Yukon. United States (USA) - Alaska (AK)."],
        BBOX[0,-126,84,-120]],
    ID["EPSG",32610]]
Data axis to CRS axis mapping: 1,2
Origin = (593909.000000000000000,5189571.000000000000000)
Pixel Size = (30.020000000000000,-33.000000000000000)
Metadata:
  AREA_OR_POINT=Area
Image Structure Metadata:
  COMPRESSION=DEFLATE
  INTERLEAVE=BAND
Corner Coordinates:
Upper Left  (  593909.000, 5189571.000) (121d46' 5.30"W, 46d51'10.98"N)
Lower Left  (  593909.000, 5189241.000) (121d46' 5.54"W, 46d51' 0.29"N)
Upper Right (  638939.000, 5189571.000) (121d10'39.75"W, 46d50'42.62"N)
Lower Right (  638939.000, 5189241.000) (121d10'40.11"W, 46d50'31.93"N)
Center      (  616424.000, 5189406.000) (121d28'22.56"W, 46d50'52.83"N)
Band 1 Block=1500x5 Type=Byte, ColorInterp=Gray
    Computed Min/Max=0.000,100.000
  NoData Value=255