zyayi / qiandi_pmcode Goto Github PK
View Code? Open in Web Editor NEWThis project forked from pmmodeling/qiandi_pmcode
code about PM modeling
This project forked from pmmodeling/qiandi_pmcode
code about PM modeling
################################################## ################ Dictionary ###################### ################################################## *Raw Data/Raw File* “raw data� or “raw file� refers the any kinds of data we downloaded/obtained from external sources. They can be any format: raster, grid, NetCDF, shapefile, etc. They may have different spatial, temporal resolutions; may be contaminated by missing values etc; If raw data are daily measurements, they may have multiple measurement for the same location within a single day, or no measurements at all in a single day. *Aggregated Data/Aggregated File* We processed raw data and matched data to consistent grid cells, and call the processed data “aggregated data�. *Input Data/Input File* We interpolated aggregated data to monitoring sites or grid cells. Files are stored in separated files by variables. *Assemble Data/Assemble File* We put input data from different variables together and stored the merged file, which is ready to do model training. *Trained Model and Model Training* We fed input data into machine learning algorithms to train the model. The model after training process (such as neural network with weights updated) is called “trained model�. This process is called “model training�. *Grid cells/Grids* We call the location of monitoring sites or 1 km grid cells to make predictions at “grid cells� or “grids�. Grid cells are presented by a table with id number, latitude, and longitude. The latitude and longitude values dependent on geographic coordinate system. ################################################## ######### Folder structure ####################### ################################################## ./raw_data/ folder for raw data, aggregate data, and relevant code ./raw_data/matlab_to_aggregate/ code files that process raw data to obtain aggregate data ./raw_data/matlab_to_interpolate/ code files that process aggregate data to obtain input data ./raw_data/download_scripts/ script that download raw data ./raw_data/data/unprocessed/ folder that store raw data ./raw_data/data/aggregate/ folder that store aggregate data ./raw_data/data/shapefile/ folder that store relevant shape files ./processed_data/ folder for input data ./processed_data/AQRVPM25/ folder for input data at PM2.5 monitoring site ./processed_data/EPACastNetOzone/ folder for input data at ozone monitoring site ./processed_data/EPANO2/ folder for input data at NO2 monitoring site ./processed_data/TempData/ folder for temporarily files for data imputation ./processed_data/DataProcessRecorder/ folder for temporarily files for obtaining input data ./processed_data/matlab_to_assemble/ code files that merge input data together to obtain assemble data ./assembled_data/ folder for assemble data ./predictions/ folder for prediction outputs ################################################## ################ Flow Chart ###################### ################################################## For flowchart, please refer to flowchart.pdf ################################################## ################# Manuals ######################## ################################################## Step 1: Download Raw Data Download raw data from different sources Run: run a download makefile Step 2: Process Raw Data to Obtain Aggregate Data Process raw data to obtain aggregate data, making data align to consistent grid cells. Aggregate data are daily or annual data files at consistent grid cells; or raster files; Run: Run_AggregateDataSummary(year) Step 3: Process Aggregate Data to Obtain Input Data Interpolate aggregate data to obtain input data at grid cells of interests (e.g., at monitoring site, or 1 km grid cells); data are stored as separate files by variable. Run: Run_InterpolateDataSummary('AQRVPM25', [datenum(year,1,1),datenum(year,12,31)],0,nan) Run_InterpolateDataSummary('EPACastNetOzone', [datenum(year,1,1),datenum(year,12,31)],0,nan) Run_InterpolateDataSummary('EPANO2', [datenum(year,1,1),datenum(year,12,31)],0,nan) Step 4: Merge Input data from different variables together to obtain assemble data Run: Run_AssembleDataSummary(1,99941,2008,'All_round3','PM25','AllRecords',''); Run_AssembleDataSummary(1,99941,2008,'All_round3','Ozone','AllRecords',''); Run_AssembleDataSummary(1,99941,2008,'All_round3','NO2','AllRecords',''); Step 5: run model training; Run: Analysis_RunModel(1,99941,year ,'All_round3','PM25','AllRecords',''); Analysis_RunModel(1,99941,year ,'All_round3','Ozone','AllRecords',''); Analysis_RunModel(1,99941,year ,'All_round3','NO2','AllRecords',''); Note Assemble data are in matlab file format and will be stored at ./assembled_data/, by different subfolders. Use testH20_convert.R to convert them into rds format. ################################################## ################# About Python Code############### ################################################## *Note:* it is worth mentioning that not all raw data were processed by matlab files; some of them were processed by python files. All Landuse variables, including NLCD landuse data, NLCD impervious, NLCD tree canopy, Restaurant density, road density, elevation (all raw data under the /raw_data/data/unprocessed/LANDUSE/) were processed by python code (see github https://github.com/NSAPH/PythonCode) ######################################################################################################################################## ################################################## ###########Laundry List of Files ################# ################################################## ################################################### ### control files, files that are actually running: ################################################### Run_AggregateDataSummary • Controller file process raw data • upload to . /raw_data/matlab_to_aggregate/ • file dependence: o USMERRA2aerSite.mat and USMERRA2aerSite_North_America_Equidistant_Conic.mat at ./raw_data/data/aggregate /MERRA2aer/ o USMOD11A1Site.mat and USMOD11A1Site_North_America_Equidistant_Conic.mat at ./raw_data/data/aggregate/MOD11A1/; o USMOD04L2Site.mat and USMOD04L2Site_North_America_Equidistant_Conic.mat at ./raw_data/data/aggregate/ MOD04L2/; o USMAIACUS5kmSite_North_America_Equidistant_Conic.mat, USMAIACUS5kmSite.mat, USMAIACUS1kmSite_North_America_Equidistant_Conic.mat, USMAIACUS1kmSite.mat at ./ raw_data/data/aggregate/MAIACUS/ o USMAIACUS5kmSite_North_America_Equidistant_Conic.mat, USMAIACUS5kmSite.mat, USMAIACUS1kmSite_North_America_Equidistant_Conic.mat, USMAIACUS1kmSite.mat at ./ raw_data/data/aggregate/MAIACUS/ o USOMAERUVdSite.mat and USOMAERUVdSite_North_America_Equidistant_Conic.mat at ./raw_data/data/aggregate/OMAERUVd/ o USOMAEROeSite.mat and USOMAEROeSite_North_America_Equidistant_Conic.mat at ./raw_data/data/aggregate/OMAEROe/; o USOMNO2dSite.mat and USOMNO2dSite_North_America_Equidistant_Conic.mat at ./raw_data/data/aggregate/OMNO2d/; o USOMSO2eSite.mat and USOMSO2eSite_North_America_Equidistant_Conic.mat at ./raw_data/data/aggregate/OMSO2e/; o USOMTO3eSite.mat and USOMTO3eSite_North_America_Equidistant_Conic.mat at ./raw_data/data/aggregate/OMTO3e/; o USOMUVBdSite.mat and USOMUVBdSite_North_America_Equidistant_Conic.mat at ./raw_data/data/aggregate/OMUVBd/ o USOMO3PRSite.mat and USOMO3PRSite_North_America_Equidistant_Conic.mat at ./raw_data/data/aggregate/OMO3PR/ o USCMAQSite.mat and USCMAQSite_North_America_Equidistant_Conic.mat at ./raw_data/data/aggregate/CMAQ/; o USGFEDSite.mat and USGFEDSite_North_America_Equidistant_Conic.mat at ./raw_data/data/aggregate/CFED/; Run_InterpolateDataSummary.m • controller file that interpolates from aggregated data and produces input data and stored the output by variable • upload to ./raw_data/ matlab_to_interpolate/ • file dependence: grid cells files placed under ./processed_data/your_grid_cell_name/Location/; Monitoring data placed at ./processed_data/your_grid_cell_name/Monitor/ • Output folder: input data at /processed_data/Your_grid_cell_name/ Run_AssembleDataSummary.m • The controller file that assembles input data separated by variables together • upload to ./processed_data/matlab_to_assemble/ Analysis_RunModel.m • controller file that assemble input data separated by variable together to produce an assemble file, then do model training; • upload to ./processed_data/matlab_to_assemble/ • file dependence: VariableList_[IDNUM].csv ready, IDNUM is the model ID number to distinguish different models; • output: Assembled data at ./assembled_data/, stored by folder ########################################## #### files called when processing raw data ########################################### Read_RestaurantData.m • Process raw restaurant data; • Called by Run_AggregateDataSummary • upload to ./raw_data/matlab_to_aggregate/ ReadMERRA2aer.m • Process MERRA 2 data set to obtain aggregate data; • Called by Run_AggregateDataSummary; • upload to ./raw_data/matlab_to_aggregate/ ReadMOD11A1_Main.m • Process MOD11A1 (surface radiation) data set to obtain aggregate data; • Called by Run_AggregateDataSummary; • upload to ./raw_data/matlab_to_aggregate/ ReadMOD04L2_Main • Process MOD04L2 (550 nm AOD data) data set to obtain aggregate data; • Called by Run_AggregateDataSummary; • upload to ./raw_data/matlab_to_aggregate/ ReadMAIAC_Main.m • Process MAIAC AOD data set to obtain aggregate data; • Called by Run_AggregateDataSummary; • upload to ./raw_data/matlab_to_aggregate/ ReadMODIS_Main.m: • Process MOD13A2 (NDVI) and MOD09A1 (surface reflectance) data set to obtain aggregate data; • Called by Run_AggregateDataSummary; • upload to ./raw_data/matlab_to_aggregate/ ReadOMIAIData_Main.m • Process OMAERUVd, OMAEROe, OMNO2d, OMSO2e, OMTO3e, OMUVBd data set to obtain aggregate data; • Called by Run_AggregateDataSummary; • upload to ./raw_data/matlab_to_aggregate/ ReadOMO3PR_Main.m • Process OMO3PR (ozone vertical profile) data set to obtain aggregate data; • Called by Run_AggregateDataSummary; • upload to ./raw_data/matlab_to_aggregate/ ReadCMAQSite.m • Process CMAQ simulations to obtain aggregate data; • Called by Run_AggregateDataSummary; • upload to ./raw_data/matlab_to_aggregate/ ReadForestfire.m; • Process forest fire data set to obtain aggregate data; • Called by Run_AggregateDataSummary; • upload to ./raw_data/matlab_to_aggregate/ ReadMODIS_function1.m • read MODIS data and extract value of interests • called by ReadMOD11A1_Main, ReadMODIS_Main • upload to ./raw_data/matlab_to_aggregate/ ReadOMIAIData_function.m • read OMI aerosol index data • called by ReadOMIAIData_Main • upload to ./raw_data/matlab_to_aggregate/ LoadData_function.m • load matlab files • called by ReadCMAQSite, ReadForestfire, ReadMERRA2aer, ReadMOD04L2_Main • upload to ./raw_data/matlab_to_aggregate/ Visualization_USResult_1.m • make maps • Called by ReadCMAQSite, ReadForestfire, ReadMAIAC_Main, ReadMERRA2aer, ReadMOD04L2_Main, ReadMODIS_Main, ReadOMIAIData_Main, ReadOMO3PR_Main • upload to ./raw_data/matlab_to_aggregate/ InterpMyData_2.m • 2D interpolation • Called by ReadCMAQSite, ReadMOD04L2_Main, ReadOMO3PR_Main • upload to ./raw_data/matlab_to_aggregate/ ################################################ #### files called when processing aggregate data ################################################# InterpMyData_2.m • 2D interpolation • Called by Interpolate_3hourPressureLevelReanalysis_Main.m, Interpolate_3hourReanalysis_Main.m • upload to ./assembled_data/matlab_to_interpolate/ Interpolate_OMI_Main.m • Extract values from OMI data set; it can also extract values from MOD04L2, forest fire, CMAQ output • Called by Run_InterpolateDataSummary • upload to ./raw_data/matlab_to_interpolate/ Interpolate_NearbyMonitor_Main.m • calculate averaged monitoring data • Called by Run_InterpolateDataSummary • upload to ./raw_data/matlab_to_interpolate/ Interpolate_MOD11A1_Main.m • main function to extract values from MOD11A1; it can also extract value from MAIAC AOD aggregate files • Called by Run_InterpolateDataSummary • upload to ./raw_data/matlab_to_interpolate/ Interpolate_Generic_Main.m • extract data from MOD13A2 and MOD09A1 MODIS data • Called by Run_InterpolateDataSummary • upload to ./raw_data/matlab_to_interpolate/ Interpolate_MCD12Q1_Main.m • extract value for grid cells of interests from all types of MCD12Q1 • Called by Run_InterpolateDataSummary • upload to ./raw_data/matlab_to_interpolate/ Interpolate_LandUse_Main.m • extract value for grid cells of interests from land use raster files • Called by Run_InterpolateDataSummary • upload to ./raw_data/matlab_to_interpolate/ Interpolate_3hourPressureLevelReanalysis_Main.m • read 3-hour reanalysis data at vertical level and interpolate to locations • Called by Run_InterpolateDataSummary • upload to ./raw_data/matlab_to_interpolate/ Interpolate_3hourReanalysis_Main.m • read 3-hour reanalysis data and interpolate to locations • Called by Run_InterpolateDataSummary • upload to ./raw_data/matlab_to_interpolate/ Interpolate_3hourReanalysis_function.m • a local function called while reading meteorological data remove missing, filling, max, min values from meteorological files • Called by Run_InterpolateDataSummary • upload to ./raw_data/matlab_to_interpolate/ MultipleWeightMatrix.m and MultipleWeightMatrix_1.m • matrix multiplication while ignoring nan values • Called by Interpolate_Generic_Main.m, Interpolate_MOD11A1_Main.m, Interpolate_NearbyMonitor_Main.m, Interpolate_OMI_Main.m • upload to ./assembled_data/matlab_to_interpolate/ LoadData_function.m • load matlab files • Called by Interpolate_3hourPressureLevelReanalysis_Main.m, Interpolate_3hourReanalysis_Main.m, Interpolate_Generic_Main.m, Interpolate_LandUse_Main.m, Interpolate_MCD12Q1_Main.m, Interpolate_MOD11A1_Main.m, Interpolate_NearbyMonitor_Main.m, Interpolate_OMI_Main.m • upload to ./assembled_data/matlab_to_interpolate/ Visualization_USResult_1.m • make maps • Called by Interpolate_3hourPressureLevelReanalysis_Main.m, Interpolate_3hourReanalysis_Main.m, Interpolate_Generic_Main.m, Interpolate_LandUse_Main.m, Interpolate_MCD12Q1_Main.m, Interpolate_MOD11A1_Main.m, Interpolate_NearbyMonitor_Main.m, Interpolate_OMI_Main.m • upload to ./assembled_data/matlab_to_interpolate/ LagData.m • take lagged value • called by Interpolate_NearbyMonitor_Main • upload to ./assembled_data/matlab_to_interpolate/ ####################################### #### files called when assembling data ####################################### VariableList_99941.csv and VariableList_999411.csv • Configuration file that specify which variables to be used as input variables • used by Run_AssembleDataSummary • upload to ./processed_data/matlab_to_assemble/ Analysis_InputDataDescriptiveAnalysis.m • descriptive stat of input variables • called by Run_AssembleDataSummary • upload to ./processed_data/matlab_to_assemble/ Analysis_ImputationInputData.m • data imputation • called by Run_AssembleDataSummary • upload to ./processed_data/matlab_to_assemble/ Analysis_ReadInputData.m • Read input variable and assemble together • called by Run_AssembleDataSummary • upload to ./processed_data/matlab_to_assemble/ LoadData_function.m • load matlab files • called by Run_AssembleDataSummary,Analysis_InputDataDescriptiveAnalysis,Analysis_ImputationInputData,Analysis_ReadInputData • upload to ./processed_data/matlab_to_assemble/ ####################################### #### files used to train model ####################################### Analysis_ResultDescriptiveAnalysis.m • calculate R2 between fitted value and monitored value • called by Analysis_FitModel, Analysis_FitModel_function1, Analysis_ResultDescriptiveAnalysis, Analysis_RunModel • upload to ./assembled_data/code_to_train/ Analysis_InputDataDescriptiveAnalysis.m • descriptive stat of input variables • called by Analysis_FitModel_function1, Analysis_InputDataDescriptiveAnalysis, Analysis_RunModel • upload to ./assembled_data/code_to_train/ Analysis_TwoStep.m, Analysis_CalculateSpatialAverage.m • Calculate averaged fitted value from neighboring days and nearby monitoring site as input variables • called by Analysis_FitModel_function1, Analysis_RunModel • upload to ./assembled_data/code_to_train/ Analysis_Transform_fucntion1.m • transform the data; and de-transform data • called by Analysis_RunModel • upload to ./assembled_data/code_to_train/ Analysis_ReadInputData.m • Read input variable and assemble together • called by Analysis_RunModel • upload to ./assembled_data/code_to_train/ Analysis_NeuralNetwork.m • fit neural network and other machine learning algorithms • called by Analysis_FitModel, Analysis_FitModel_function1 • upload to ./assembled_data/code_to_train/ Analysis_ImputationInputData.m • data imputation • called by Analysis_RunModel • upload to ./assembled_data/code_to_train/ Analysis_FitModel_function1.m • update neighboring days and nearby monitoring site as input variables in a two-stage setting • called by Analysis_FitModel • upload to ./assembled_data/code_to_train/ Analysis_FitModel_function2.m • return the records to be used in a cross-validation setting • called by Analysis_FitModel • upload to ./assembled_data/code_to_train/ Analysis_FitModel.m • fit model either with 100% training data or in a cross-validated way • called by Analysis_RunModel • upload to ./assembled_data/code_to_train/ isnan_matrix.m • Return the index of missing value inside a matrix • called by Analysis_FitModel_function2, Analysis_InputDataDescriptiveAnalysis • upload to ./assembled_data/code_to_train/ CalculateRsquare.m • Calculation R2 • called by Analysis_FitModel_function1, Analysis_ResultDescriptiveAnalysis • upload to ./assembled_data/code_to_train/ MultipleWeightMatrix.m and MultipleWeightMatrix_1.m • matrix multiplication while ignoring nan values • upload to ./assembled_data/code_to_train/ VariableList_99941.csv and VariableList_999411.csv • Configuration file that specify which variables to be used as input variables • upload to ./assembled_data/code_to_train/ InterpMyData_2.m • 2D interpolation • Called by Analysis_RunModel • upload to ./assembled_data/code_to_train/ Visualization_USResult_1.m • make maps • Called by Analysis_RunModel • upload to ./assembled_data/code_to_train/ LoadData_function.m • load matlab files • called by Analysis_ImputationInputData, Analysis_ReadInputData, Analysis_RunModel, Analysis_TwoStep, • upload to . • upload to ./assembled_data/code_to_train/ VariableList_99941.csv and VariableList_999411.csv • Configuration file that specify which variables to be used as input variables • used by Run_AssembleDataSummary • upload to ./assembled_data/code_to_train/ ####################################### ######### output examples ############# ####################################### Below information was excerpted from the full output. This excerpted output was used to notify users what the output looked like when processing the data for the first time. ########################################## ## Output from Run_AggregateDataSummary### ########################################## # processing Merra 2 data; example output, print out file name: processing...MERRA2_400.inst3_3d_aer_Nv.20150101.SUB.nc4 processing...MERRA2_400.inst3_3d_aer_Nv.20150102.SUB.nc4 processing...MERRA2_400.inst3_3d_aer_Nv.20150103.SUB.nc4 processing...MERRA2_400.inst3_3d_aer_Nv.20150104.SUB.nc4 processing...MERRA2_400.inst3_3d_aer_Nv.20150105.SUB.nc4 processing...MERRA2_400.inst3_3d_aer_Nv.20150106.SUB.nc4 processing...MERRA2_400.inst3_3d_aer_Nv.20150107.SUB.nc4 processing...MERRA2_400.inst3_3d_aer_Nv.20150108.SUB.nc4 processing...MERRA2_400.inst3_3d_aer_Nv.20150109.SUB.nc4 processing...MERRA2_400.inst3_3d_aer_Nv.20150110.SUB.nc4 # processing MOD11A1 data; example output, print out processed file name by tile by day: 01-Jan-2015 processing..2015-01-01 MOD11A1.A2015001.h08v04.006.2016212060159.hdf MOD11A1.A2015001.h08v05.006.2016212060205.hdf MOD11A1.A2015001.h08v06.006.2016212060205.hdf MOD11A1.A2015001.h09v04.006.2016212060214.hdf MOD11A1.A2015001.h09v05.006.2016212060216.hdf MOD11A1.A2015001.h09v06.006.2016212060207.hdf MOD11A1.A2015001.h10v04.006.2016212060215.hdf MOD11A1.A2015001.h10v05.006.2016212060209.hdf MOD11A1.A2015001.h10v06.006.2016212060207.hdf MOD11A1.A2015001.h11v04.006.2016212060216.hdf MOD11A1.A2015001.h11v05.006.2016212060213.hdf MOD11A1.A2015001.h11v06.006.2016212060208.hdf MOD11A1.A2015001.h12v04.006.2016212060229.hdf MOD11A1.A2015001.h12v05.006.2016212060218.hdf MOD11A1.A2015001.h13v04.006.2016212060217.hdf 02-Jan-2015 processing..2015-01-02 MOD11A1.A2015002.h08v04.006.2016212061654.hdf MOD11A1.A2015002.h08v05.006.2016212061701.hdf MOD11A1.A2015002.h08v06.006.2016212061657.hdf MOD11A1.A2015002.h09v04.006.2016212061657.hdf MOD11A1.A2015002.h09v05.006.2016212061656.hdf MOD11A1.A2015002.h09v06.006.2016212061651.hdf MOD11A1.A2015002.h10v04.006.2016212061657.hdf MOD11A1.A2015002.h10v05.006.2016212061650.hdf MOD11A1.A2015002.h10v06.006.2016212061650.hdf MOD11A1.A2015002.h11v04.006.2016212061650.hdf MOD11A1.A2015002.h11v05.006.2016212061653.hdf MOD11A1.A2015002.h11v06.006.2016212061645.hdf MOD11A1.A2015002.h12v04.006.2016212061723.hdf MOD11A1.A2015002.h12v05.006.2016212061718.hdf MOD11A1.A2015002.h13v04.006.2016212061718.hdf # processing MOD04L2 data, print out processed file by day by tile; also print out the percentage of missing values 2015-01-01 MOD04_L2.A2015001.1535.006.2015034063228.hdf nan:0.974713 MOD04_L2.A2015001.1540.006.2015034063702.hdf nan:0.952454 MOD04_L2.A2015001.1715.006.2015034062347.hdf nan:0.973107 MOD04_L2.A2015001.1720.006.2015033150936.hdf nan:0.932348 MOD04_L2.A2015001.1850.006.2015034062326.hdf nan:0.998796 MOD04_L2.A2015001.1855.006.2015034063022.hdf nan:0.908606 MOD04_L2.A2015001.2030.006.2015034062230.hdf nan:1.000000 2015-01-02 MOD04_L2.A2015002.1440.006.2015035115904.hdf nan:1.000000 MOD04_L2.A2015002.1615.006.2015035120001.hdf nan:1.000000 MOD04_L2.A2015002.1620.006.2015035120129.hdf nan:0.985185 MOD04_L2.A2015002.1625.006.2015035120420.hdf nan:0.947893 MOD04_L2.A2015002.1755.006.2015035120156.hdf nan:0.996278 MOD04_L2.A2015002.1800.006.2015035120120.hdf nan:0.906003 MOD04_L2.A2015002.1935.006.2015035120013.hdf nan:0.989637 MOD04_L2.A2015002.1940.006.2015035115923.hdf nan:0.993140 # processing Aqua MAIAC data; print out processed file by day by tile; 01-Jan-2015 processing..2015-01-01 MAIACAAOT.h01v03.20150012035.hdf MAIACAAOT.h01v03.20150012215.hdf MAIACAAOT.h01v04.20150012035.hdf MAIACAAOT.h01v04.20150012215.hdf MAIACAAOT.h01v05.20150012030.hdf MAIACAAOT.h02v03.20150011900.hdf MAIACAAOT.h02v03.20150012035.hdf MAIACAAOT.h02v04.20150011855.hdf MAIACAAOT.h02v04.20150012035.hdf MAIACAAOT.h02v05.20150012030.hdf MAIACAAOT.h03v03.20150011720.hdf MAIACAAOT.h03v03.20150011900.hdf MAIACAAOT.h03v03.20150012035.hdf MAIACAAOT.h03v04.20150011855.hdf MAIACAAOT.h03v04.20150012035.hdf MAIACAAOT.h03v05.20150011855.hdf MAIACAAOT.h03v05.20150012030.hdf MAIACAAOT.h04v03.20150011720.hdf MAIACAAOT.h04v03.20150011900.hdf MAIACAAOT.h04v04.20150011720.hdf MAIACAAOT.h04v04.20150011855.hdf MAIACAAOT.h04v05.20150011855.hdf MAIACAAOT.h05v03.20150011540.hdf MAIACAAOT.h05v03.20150011720.hdf 02-Jan-2015 processing..2015-01-02 MAIACAAOT.h01v03.20150022120.hdf MAIACAAOT.h01v03.20150022300.hdf MAIACAAOT.h01v04.20150022120.hdf MAIACAAOT.h01v05.20150022115.hdf MAIACAAOT.h02v03.20150021940.hdf MAIACAAOT.h02v03.20150022120.hdf MAIACAAOT.h02v04.20150021940.hdf MAIACAAOT.h02v04.20150022120.hdf MAIACAAOT.h02v05.20150021935.hdf MAIACAAOT.h02v05.20150022115.hdf MAIACAAOT.h03v03.20150021800.hdf MAIACAAOT.h03v03.20150021940.hdf MAIACAAOT.h03v04.20150021800.hdf MAIACAAOT.h03v04.20150021940.hdf MAIACAAOT.h03v05.20150021935.hdf MAIACAAOT.h04v03.20150021625.hdf MAIACAAOT.h04v03.20150021800.hdf MAIACAAOT.h04v03.20150021940.hdf MAIACAAOT.h04v04.20150021800.hdf MAIACAAOT.h04v04.20150021940.hdf MAIACAAOT.h04v05.20150021800.hdf MAIACAAOT.h04v05.20150021935.hdf MAIACAAOT.h05v03.20150021625.hdf MAIACAAOT.h05v03.20150021800.hdf # processing Terra MAIAC data; print out processed file by day by tile; 01-Jan-2015 processing..2015-01-01 MAIACTAOT.h01v03.20150011855.hdf MAIACTAOT.h01v03.20150012035.hdf MAIACTAOT.h01v04.20150011855.hdf MAIACTAOT.h01v04.20150012035.hdf MAIACTAOT.h01v05.20150011900.hdf MAIACTAOT.h02v03.20150011715.hdf MAIACTAOT.h02v03.20150011855.hdf MAIACTAOT.h02v04.20150011720.hdf MAIACTAOT.h02v04.20150011855.hdf MAIACTAOT.h02v05.20150011720.hdf MAIACTAOT.h02v05.20150011900.hdf MAIACTAOT.h03v03.20150011715.hdf MAIACTAOT.h03v03.20150011850.hdf MAIACTAOT.h03v04.20150011715.hdf MAIACTAOT.h03v05.20150011720.hdf MAIACTAOT.h04v03.20150011535.hdf MAIACTAOT.h04v03.20150011715.hdf MAIACTAOT.h04v04.20150011540.hdf MAIACTAOT.h04v04.20150011715.hdf MAIACTAOT.h04v05.20150011540.hdf MAIACTAOT.h04v05.20150011720.hdf MAIACTAOT.h05v03.20150011400.hdf MAIACTAOT.h05v03.20150011535.hdf 02-Jan-2015 processing..2015-01-02 MAIACTAOT.h01v03.20150021800.hdf MAIACTAOT.h01v03.20150021940.hdf MAIACTAOT.h01v03.20150022115.hdf MAIACTAOT.h01v04.20150021800.hdf MAIACTAOT.h01v04.20150021940.hdf MAIACTAOT.h01v05.20150021805.hdf MAIACTAOT.h01v05.20150021940.hdf MAIACTAOT.h02v03.20150021800.hdf MAIACTAOT.h02v03.20150021935.hdf MAIACTAOT.h02v04.20150021800.hdf MAIACTAOT.h02v04.20150021935.hdf MAIACTAOT.h02v05.20150021805.hdf MAIACTAOT.h03v03.20150021620.hdf MAIACTAOT.h03v03.20150021800.hdf MAIACTAOT.h03v04.20150021620.hdf MAIACTAOT.h03v04.20150021800.hdf MAIACTAOT.h03v05.20150021625.hdf MAIACTAOT.h03v05.20150021805.hdf MAIACTAOT.h04v03.20150021440.hdf MAIACTAOT.h04v03.20150021620.hdf MAIACTAOT.h04v03.20150021755.hdf MAIACTAOT.h04v04.20150021620.hdf MAIACTAOT.h04v05.20150021625.hdf MAIACTAOT.h05v03.20150021440.hdf MAIACTAOT.h05v03.20150021620.hdf ## processing MOD13A2; print out processed file by day by tile; not all days have MOD13A2 measurements; # also print out the number and percentage of missing values; 2015-01-01 17 processing..2015-01-01 MOD13A2.A2015001.h08v04.006.2015295101249.hdf count and percentage of nan data:313768 9.921700e-01 MOD13A2.A2015001.h08v05.006.2015295101251.hdf count and percentage of nan data:1420824 9.271611e-01 MOD13A2.A2015001.h08v06.006.2015295100648.hdf count and percentage of nan data:864000 8.854107e-01 MOD13A2.A2015001.h09v04.006.2015295101621.hdf count and percentage of nan data:1183076 8.292160e-01 MOD13A2.A2015001.h09v05.006.2015295101252.hdf count and percentage of nan data:1440000 7.464091e-01 MOD13A2.A2015001.h09v06.006.2015295100645.hdf count and percentage of nan data:864000 7.271550e-01 MOD13A2.A2015001.h10v03.006.2015295101300.hdf count and percentage of nan data:282135 7.115178e-01 MOD13A2.A2015001.h10v04.006.2015295101254.hdf count and percentage of nan data:1440000 6.289266e-01 MOD13A2.A2015001.h10v05.006.2015295100658.hdf count and percentage of nan data:1440000 5.475146e-01 MOD13A2.A2015001.h10v06.006.2015295100643.hdf count and percentage of nan data:864000 5.390605e-01 MOD13A2.A2015001.h11v03.006.2015295100653.hdf count and percentage of nan data:288000 5.224884e-01 MOD13A2.A2015001.h11v04.006.2015295100650.hdf count and percentage of nan data:1440000 4.430197e-01 MOD13A2.A2015001.h11v05.006.2015295100640.hdf count and percentage of nan data:1440000 3.935406e-01 MOD13A2.A2015001.h11v06.006.2015295100642.hdf count and percentage of nan data:864000 3.933252e-01 MOD13A2.A2015001.h12v04.006.2015295101251.hdf count and percentage of nan data:1440000 3.272660e-01 MOD13A2.A2015001.h12v05.006.2015295100644.hdf count and percentage of nan data:1090678 3.243732e-01 MOD13A2.A2015001.h13v04.006.2015295101246.hdf count and percentage of nan data:691575 2.975893e-01 percentage of nan:0.297589 max:10000.000000 mean:2509.421703 min:-2000.000000 2015-01-02 0 2015-01-03 0 2015-01-04 0 2015-01-05 0 2015-01-06 0 2015-01-07 0 2015-01-08 0 2015-01-09 0 2015-01-10 0 2015-01-11 0 2015-01-12 0 2015-01-13 0 2015-01-14 0 2015-01-15 0 2015-01-16 0 2015-01-17 17 processing..2015-01-17 MOD13A2.A2015017.h08v04.006.2015295120233.hdf count and percentage of nan data:313768 9.921640e-01 MOD13A2.A2015017.h08v05.006.2015295120224.hdf count and percentage of nan data:1420824 9.271465e-01 MOD13A2.A2015017.h08v06.006.2015295115637.hdf count and percentage of nan data:864000 8.854013e-01 MOD13A2.A2015017.h09v04.006.2015295120405.hdf count and percentage of nan data:1183076 8.291317e-01 MOD13A2.A2015017.h09v05.006.2015295120218.hdf count and percentage of nan data:1440000 7.463162e-01 MOD13A2.A2015017.h09v06.006.2015295115832.hdf count and percentage of nan data:864000 7.271007e-01 MOD13A2.A2015017.h10v03.006.2015295120029.hdf count and percentage of nan data:282135 7.113305e-01 MOD13A2.A2015017.h10v04.006.2015295115842.hdf count and percentage of nan data:1440000 6.286606e-01 MOD13A2.A2015017.h10v05.006.2015295120225.hdf count and percentage of nan data:1440000 5.472454e-01 MOD13A2.A2015017.h10v06.006.2015295120028.hdf count and percentage of nan data:864000 5.388123e-01 MOD13A2.A2015017.h11v03.006.2015295120231.hdf count and percentage of nan data:288000 5.222376e-01 MOD13A2.A2015017.h11v04.006.2015295120234.hdf count and percentage of nan data:1440000 4.422270e-01 MOD13A2.A2015017.h11v05.006.2015295120234.hdf count and percentage of nan data:1440000 3.927448e-01 MOD13A2.A2015017.h11v06.006.2015295115832.hdf count and percentage of nan data:864000 3.925366e-01 MOD13A2.A2015017.h12v04.006.2015295120417.hdf count and percentage of nan data:1440000 3.255894e-01 MOD13A2.A2015017.h12v05.006.2015295115630.hdf count and percentage of nan data:1090678 3.226951e-01 MOD13A2.A2015017.h13v04.006.2015295115224.hdf count and percentage of nan data:691575 2.959314e-01 percentage of nan:0.295931 max:10000.000000 mean:2702.462716 min:-2000.000000 ## processing MOD09A1; print out processed file by day by tile; not all days have MOD09A1 measurements; # also print out the number and percentage of missing values; 2015-01-01 17 processing..2015-01-01 MOD09A1.A2015001.h08v04.006.2015295093200.hdf count and percentage of nan data:5760000 9.411765e-01 MOD09A1.A2015001.h08v05.006.2015295093104.hdf count and percentage of nan data:5760000 8.823530e-01 MOD09A1.A2015001.h08v06.006.2015295091732.hdf count and percentage of nan data:5760000 8.235294e-01 MOD09A1.A2015001.h09v04.006.2015295093427.hdf count and percentage of nan data:5760000 7.647059e-01 MOD09A1.A2015001.h09v05.006.2015295092207.hdf count and percentage of nan data:5760000 7.058824e-01 MOD09A1.A2015001.h09v06.006.2015295094219.hdf count and percentage of nan data:5760000 6.470588e-01 MOD09A1.A2015001.h10v03.006.2015295092257.hdf count and percentage of nan data:5760000 5.882353e-01 MOD09A1.A2015001.h10v04.006.2015295092334.hdf count and percentage of nan data:5760000 5.294118e-01 MOD09A1.A2015001.h10v05.006.2015295092733.hdf count and percentage of nan data:5760000 4.705882e-01 MOD09A1.A2015001.h10v06.006.2015295092601.hdf count and percentage of nan data:5760000 4.117647e-01 MOD09A1.A2015001.h11v03.006.2015295093010.hdf count and percentage of nan data:5760000 3.529412e-01 MOD09A1.A2015001.h11v04.006.2015295092420.hdf count and percentage of nan data:5760000 2.941177e-01 MOD09A1.A2015001.h11v05.006.2015295092240.hdf count and percentage of nan data:5760000 2.352941e-01 MOD09A1.A2015001.h11v06.006.2015295091800.hdf count and percentage of nan data:5760000 1.764706e-01 MOD09A1.A2015001.h12v04.006.2015295092433.hdf count and percentage of nan data:5760000 1.176471e-01 MOD09A1.A2015001.h12v05.006.2015295092430.hdf count and percentage of nan data:5760000 5.882354e-02 MOD09A1.A2015001.h13v04.006.2015295091822.hdf count and percentage of nan data:5760000 1.021242e-08 percentage of nan:0.000000 max:16000.000000 mean:1886.316808 min:-100.000000 2015-01-02 0 2015-01-03 0 2015-01-04 0 2015-01-05 0 2015-01-06 0 2015-01-07 0 2015-01-08 0 2015-01-09 17 processing..2015-01-09 MOD09A1.A2015009.h08v04.006.2015295140838.hdf count and percentage of nan data:5760000 9.411765e-01 MOD09A1.A2015009.h08v05.006.2015295140619.hdf count and percentage of nan data:5760000 8.823529e-01 MOD09A1.A2015009.h08v06.006.2015295140309.hdf count and percentage of nan data:5760000 8.235294e-01 MOD09A1.A2015009.h09v04.006.2015295141141.hdf count and percentage of nan data:5760000 7.647059e-01 MOD09A1.A2015009.h09v05.006.2015295140529.hdf count and percentage of nan data:5760000 7.058824e-01 MOD09A1.A2015009.h09v06.006.2015295140015.hdf count and percentage of nan data:5760000 6.470588e-01 MOD09A1.A2015009.h10v03.006.2015295140621.hdf count and percentage of nan data:5760000 5.882353e-01 MOD09A1.A2015009.h10v04.006.2015295140526.hdf count and percentage of nan data:5760000 5.294118e-01 MOD09A1.A2015009.h10v05.006.2015295140101.hdf count and percentage of nan data:5760000 4.705882e-01 MOD09A1.A2015009.h10v06.006.2015295140033.hdf count and percentage of nan data:5760000 4.117647e-01 MOD09A1.A2015009.h11v03.006.2015295140326.hdf count and percentage of nan data:5760000 3.529412e-01 MOD09A1.A2015009.h11v04.006.2015295140230.hdf count and percentage of nan data:5760000 2.941176e-01 MOD09A1.A2015009.h11v05.006.2015295140137.hdf count and percentage of nan data:5760000 2.352941e-01 MOD09A1.A2015009.h11v06.006.2015295140059.hdf count and percentage of nan data:5760000 1.764706e-01 MOD09A1.A2015009.h12v04.006.2015295140531.hdf count and percentage of nan data:5760000 1.176471e-01 MOD09A1.A2015009.h12v05.006.2015295140124.hdf count and percentage of nan data:5760000 5.882353e-02 MOD09A1.A2015009.h13v04.006.2015295140952.hdf count and percentage of nan data:5760000 0 percentage of nan:0.000000 max:15999.000000 mean:1839.268639 min:1.000000 ## processing OMAERUVd; print out processed file names by day; each day has measurements available; # also print out some descriptive information; 2015-01-01 OMI-Aura_L3-OMAERUVd_2015m0101_v003-2017m0821t145637.he5 UVAerosolIndex--- nan percentage:0.494519 max:2.259600 mean:0.292568 min:-1.051500 FinalAerosolSingleScattAlb500--- nan percentage:0.943704 max:0.995800 mean:0.925068 min:0.855200 2015-01-02 OMI-Aura_L3-OMAERUVd_2015m0102_v003-2017m0821t145632.he5 UVAerosolIndex--- nan percentage:0.487407 max:1.715000 mean:0.321844 min:-1.548400 FinalAerosolSingleScattAlb500--- nan percentage:0.964444 max:0.984100 mean:0.915727 min:0.856900 2015-01-03 OMI-Aura_L3-OMAERUVd_2015m0103_v003-2017m0821t145629.he5 UVAerosolIndex--- nan percentage:0.487704 max:1.928300 mean:0.309876 min:-0.584000 FinalAerosolSingleScattAlb500--- nan percentage:0.950222 max:0.994100 mean:0.922689 min:0.853400 2015-01-04 OMI-Aura_L3-OMAERUVd_2015m0104_v003-2017m0821t145650.he5 UVAerosolIndex--- nan percentage:0.489185 max:1.975800 mean:0.296028 min:-0.957900 FinalAerosolSingleScattAlb500--- nan percentage:0.944000 max:0.999900 mean:0.935069 min:0.856100 2015-01-05 OMI-Aura_L3-OMAERUVd_2015m0105_v003-2017m0821t145649.he5 UVAerosolIndex--- nan percentage:0.483852 max:1.800200 mean:0.247434 min:-1.205500 FinalAerosolSingleScattAlb500--- nan percentage:0.913778 max:0.989500 mean:0.921102 min:0.859700 2015-01-06 OMI-Aura_L3-OMAERUVd_2015m0106_v003-2017m0821t145646.he5 UVAerosolIndex--- nan percentage:0.487407 max:2.725600 mean:0.300256 min:-1.037800 FinalAerosolSingleScattAlb500--- nan percentage:0.908444 max:0.998200 mean:0.922873 min:0.854000 2015-01-07 OMI-Aura_L3-OMAERUVd_2015m0107_v003-2017m0821t145649.he5 UVAerosolIndex--- nan percentage:0.480296 max:2.370000 mean:0.305356 min:-1.277600 FinalAerosolSingleScattAlb500--- nan percentage:0.922963 max:0.997600 mean:0.923805 min:0.845900 2015-01-08 OMI-Aura_L3-OMAERUVd_2015m0108_v003-2017m0821t145645.he5 UVAerosolIndex--- nan percentage:0.480889 max:1.878500 mean:0.331206 min:-1.110100 FinalAerosolSingleScattAlb500--- nan percentage:0.929778 max:0.999400 mean:0.922369 min:0.865300 2015-01-09 OMI-Aura_L3-OMAERUVd_2015m0109_v003-2017m0821t145647.he5 UVAerosolIndex--- nan percentage:0.470815 max:2.199000 mean:0.321346 min:-1.459400 FinalAerosolSingleScattAlb500--- nan percentage:0.938370 max:0.999600 mean:0.925506 min:0.857000 2015-01-10 OMI-Aura_L3-OMAERUVd_2015m0110_v003-2017m0821t145645.he5 UVAerosolIndex--- nan percentage:0.474370 max:2.569900 mean:0.332940 min:-0.911100 FinalAerosolSingleScattAlb500--- nan percentage:0.946963 max:0.999300 mean:0.923544 min:0.858300 ## processing OMAEROe; print out processed file names by day; each day has measurements available; # also print out some descriptive information for all variables of interests; 2015-01-01 OMI-Aura_L3-OMAEROe_2015m0101_v003-2015m0103t013855.he5 VISAerosolIndex--- nan percentage:0.398833 max:12.620000 mean:-6.129567 min:-36.450001 UVAerosolIndex--- nan percentage:0.398833 max:3.230000 mean:0.524181 min:-2.600000 ViewingZenithAngle--- nan percentage:0.398833 max:69.478996 mean:52.554844 min:22.679001 SolarZenithAngle--- nan percentage:0.398833 max:87.041000 mean:64.731852 min:39.271999 2015-01-02 OMI-Aura_L3-OMAEROe_2015m0102_v003-2015m0104t021202.he5 VISAerosolIndex--- nan percentage:0.400574 max:13.600000 mean:-4.860347 min:-32.860001 UVAerosolIndex--- nan percentage:0.400574 max:3.110000 mean:0.487008 min:-3.370000 ViewingZenithAngle--- nan percentage:0.400574 max:69.477997 mean:51.139909 min:20.511999 SolarZenithAngle--- nan percentage:0.400574 max:87.069000 mean:63.951587 min:39.195000 2015-01-03 OMI-Aura_L3-OMAEROe_2015m0103_v003-2015m0105t012111.he5 VISAerosolIndex--- nan percentage:0.364815 max:12.730000 mean:-6.083580 min:-35.049999 UVAerosolIndex--- nan percentage:0.364815 max:2.900000 mean:0.494084 min:-3.220000 ViewingZenithAngle--- nan percentage:0.364815 max:69.482002 mean:51.930168 min:20.513000 SolarZenithAngle--- nan percentage:0.364815 max:85.233002 mean:64.908281 min:39.113998 2015-01-04 OMI-Aura_L3-OMAEROe_2015m0104_v003-2015m0106t020427.he5 VISAerosolIndex--- nan percentage:0.398315 max:14.050000 mean:-4.423788 min:-33.689999 UVAerosolIndex--- nan percentage:0.364056 max:2.890000 mean:0.583598 min:-3.390000 ViewingZenithAngle--- nan percentage:0.364056 max:69.480003 mean:51.909455 min:20.513000 SolarZenithAngle--- nan percentage:0.364056 max:85.056999 mean:64.704791 min:38.932999 2015-01-05 OMI-Aura_L3-OMAEROe_2015m0105_v003-2015m0107t010310.he5 VISAerosolIndex--- nan percentage:0.403241 max:15.030000 mean:-5.567245 min:-34.869999 UVAerosolIndex--- nan percentage:0.369463 max:2.890000 mean:0.461060 min:-3.940000 ViewingZenithAngle--- nan percentage:0.369463 max:69.482002 mean:51.798910 min:20.513000 SolarZenithAngle--- nan percentage:0.369463 max:84.837997 mean:64.640499 min:38.851002 2015-01-06 OMI-Aura_L3-OMAEROe_2015m0106_v003-2015m0108t021302.he5 VISAerosolIndex--- nan percentage:0.419167 max:13.830000 mean:-6.968605 min:-36.990002 UVAerosolIndex--- nan percentage:0.386037 max:3.300000 mean:0.590036 min:-4.670000 ViewingZenithAngle--- nan percentage:0.386037 max:69.480003 mean:52.239100 min:20.511999 SolarZenithAngle--- nan percentage:0.386037 max:85.878998 mean:64.736840 min:38.626999 2015-01-07 OMI-Aura_L3-OMAEROe_2015m0107_v003-2015m0109t005520.he5 VISAerosolIndex--- nan percentage:0.392426 max:13.660000 mean:-6.845780 min:-34.209999 UVAerosolIndex--- nan percentage:0.358056 max:3.800000 mean:0.510464 min:-4.410000 ViewingZenithAngle--- nan percentage:0.358056 max:69.480003 mean:51.634381 min:20.513000 SolarZenithAngle--- nan percentage:0.358056 max:84.553001 mean:64.110015 min:38.532001 2015-01-08 OMI-Aura_L3-OMAEROe_2015m0108_v003-2015m0110t013153.he5 VISAerosolIndex--- nan percentage:0.377889 max:14.600000 mean:-5.874112 min:-36.349998 UVAerosolIndex--- nan percentage:0.377889 max:4.040000 mean:0.587620 min:-2.330000 ViewingZenithAngle--- nan percentage:0.377889 max:69.477997 mean:51.972835 min:20.513000 SolarZenithAngle--- nan percentage:0.377889 max:84.357002 mean:64.254899 min:38.344002 2015-01-09 OMI-Aura_L3-OMAEROe_2015m0109_v003-2015m0111t021252.he5 VISAerosolIndex--- nan percentage:0.344333 max:13.920000 mean:-6.588794 min:-36.720001 UVAerosolIndex--- nan percentage:0.344333 max:2.830000 mean:0.564176 min:-3.460000 ViewingZenithAngle--- nan percentage:0.344333 max:69.475998 mean:51.670961 min:20.513000 SolarZenithAngle--- nan percentage:0.344333 max:84.598999 mean:63.786703 min:38.153000 2015-01-10 OMI-Aura_L3-OMAEROe_2015m0110_v003-2015m0112t012323.he5 VISAerosolIndex--- nan percentage:0.369204 max:14.180000 mean:-6.294938 min:-38.480000 UVAerosolIndex--- nan percentage:0.369204 max:2.930000 mean:0.603543 min:-1.950000 ViewingZenithAngle--- nan percentage:0.369204 max:69.480003 mean:51.959729 min:20.513000 SolarZenithAngle--- nan percentage:0.369204 max:85.752998 mean:64.011795 min:38.077999 ## processing OMNO2d; print out processed file names by day; each day has measurements available; # also print out some descriptive information; 2015-01-01 OMI-Aura_L3-OMNO2d_2015m0101_v003-2016m0826t232248.he5 ColumnAmountNO2StratoCloudScreened--- nan percentage:0.724222 max:1942158625996800.000000 mean:1482698997298852.250000 min:144879187394560.000000 2015-01-02 OMI-Aura_L3-OMNO2d_2015m0102_v003-2016m0826t231724.he5 ColumnAmountNO2StratoCloudScreened--- nan percentage:0.763722 max:1950894740471808.000000 mean:1445959341753817.250000 min:183516952264704.000000 2015-01-03 OMI-Aura_L3-OMNO2d_2015m0103_v003-2016m0826t232255.he5 ColumnAmountNO2StratoCloudScreened--- nan percentage:0.734167 max:1984085845082112.000000 mean:1558957165866453.000000 min:229958370721792.000000 2015-01-04 OMI-Aura_L3-OMNO2d_2015m0104_v003-2016m0826t232246.he5 ColumnAmountNO2StratoCloudScreened--- nan percentage:0.792481 max:1904979862355968.000000 mean:1604516284287846.750000 min:385991663681536.000000 2015-01-05 OMI-Aura_L3-OMNO2d_2015m0105_v003-2016m0826t232753.he5 ColumnAmountNO2StratoCloudScreened--- nan percentage:0.728241 max:2015287138320384.000000 mean:1652790333697303.000000 min:440046175911936.000000 2015-01-06 OMI-Aura_L3-OMNO2d_2015m0106_v003-2016m0826t232319.he5 ColumnAmountNO2StratoCloudScreened--- nan percentage:0.707630 max:1994209821196288.000000 mean:1627984685156601.000000 min:310324003602432.000000 2015-01-07 OMI-Aura_L3-OMNO2d_2015m0107_v003-2016m0826t231719.he5 ColumnAmountNO2StratoCloudScreened--- nan percentage:0.698019 max:2105262173323264.000000 mean:1573561271019012.250000 min:567253384822784.000000 2015-01-08 OMI-Aura_L3-OMNO2d_2015m0108_v003-2016m0826t232228.he5 ColumnAmountNO2StratoCloudScreened--- nan percentage:0.737185 max:1962767036710912.000000 mean:1498446703722200.000000 min:583084256788480.000000 2015-01-09 OMI-Aura_L3-OMNO2d_2015m0109_v003-2016m0826t232242.he5 ColumnAmountNO2StratoCloudScreened--- nan percentage:0.731463 max:2016281389694976.000000 mean:1598146597825557.750000 min:635362095398912.000000 2015-01-10 OMI-Aura_L3-OMNO2d_2015m0110_v003-2016m0826t232819.he5 ColumnAmountNO2StratoCloudScreened--- nan percentage:0.739093 max:1914293498937344.000000 mean:1597013613633975.000000 min:742670898561024.000000 # processing OMSO2e; print out processed file names by day; each day has measurements available; # also print out some descriptive information; 2015-01-01 OMI-Aura_L3-OMSO2e_2015m0101_v003-2015m0226t075925.he5 ColumnAmountSO2_PBL--- nan percentage:0.797537 max:2.153913 mean:0.011180 min:-2.310705 2015-01-02 OMI-Aura_L3-OMSO2e_2015m0102_v003-2015m0226t075946.he5 ColumnAmountSO2_PBL--- nan percentage:0.851667 max:8.046083 mean:0.021500 min:-2.502771 2015-01-03 OMI-Aura_L3-OMSO2e_2015m0103_v003-2015m0226t075933.he5 ColumnAmountSO2_PBL--- nan percentage:0.798537 max:1.577025 mean:0.033366 min:-1.239236 2015-01-04 OMI-Aura_L3-OMSO2e_2015m0104_v003-2015m0226t075928.he5 ColumnAmountSO2_PBL--- nan percentage:0.835093 max:2.217980 mean:0.031315 min:-1.288870 2015-01-05 OMI-Aura_L3-OMSO2e_2015m0105_v003-2015m0226t080053.he5 ColumnAmountSO2_PBL--- nan percentage:0.806167 max:2.284446 mean:0.019609 min:-2.417970 2015-01-06 OMI-Aura_L3-OMSO2e_2015m0106_v003-2015m0226t075816.he5 ColumnAmountSO2_PBL--- nan percentage:0.759611 max:2.635092 mean:0.027202 min:-1.571885 2015-01-07 OMI-Aura_L3-OMSO2e_2015m0107_v003-2015m0226t075957.he5 ColumnAmountSO2_PBL--- nan percentage:0.800130 max:4.006037 mean:0.019285 min:-2.265734 2015-01-08 OMI-Aura_L3-OMSO2e_2015m0108_v003-2015m0226t075743.he5 ColumnAmountSO2_PBL--- nan percentage:0.816963 max:2.652568 mean:0.030481 min:-1.751842 2015-01-09 OMI-Aura_L3-OMSO2e_2015m0109_v003-2015m0226t075914.he5 ColumnAmountSO2_PBL--- nan percentage:0.803463 max:3.802536 mean:0.013612 min:-2.169395 2015-01-10 OMI-Aura_L3-OMSO2e_2015m0110_v003-2015m0226t075803.he5 ColumnAmountSO2_PBL--- nan percentage:0.813759 max:2.516983 mean:0.010742 min:-1.772840 # processing OMTO3e; print out processed file names by day; each day has measurements available; # also print out some descriptive information by variable of interests; 2015-01-01 OMI-Aura_L3-OMTO3e_2015m0101_v003-2015m0108t113901.he5 ColumnAmountO3--- nan percentage:0.335889 max:430.000000 mean:293.134373 min:239.100006 ViewingZenithAngle--- nan percentage:0.335889 max:69.195999 mean:48.575689 min:16.243000 2015-01-02 OMI-Aura_L3-OMTO3e_2015m0102_v003-2015m0108t113904.he5 ColumnAmountO3--- nan percentage:0.353500 max:411.899994 mean:290.124760 min:240.500000 ViewingZenithAngle--- nan percentage:0.353500 max:69.175003 mean:48.280896 min:16.243000 2015-01-03 OMI-Aura_L3-OMTO3e_2015m0103_v003-2015m0108t113912.he5 ColumnAmountO3--- nan percentage:0.313944 max:460.500000 mean:295.338637 min:236.600006 ViewingZenithAngle--- nan percentage:0.313944 max:69.188004 mean:48.634783 min:16.245001 2015-01-04 OMI-Aura_L3-OMTO3e_2015m0104_v003-2015m0108t113851.he5 ColumnAmountO3--- nan percentage:0.304241 max:429.000000 mean:299.570043 min:218.800003 ViewingZenithAngle--- nan percentage:0.304241 max:69.195000 mean:48.932308 min:16.243000 2015-01-05 OMI-Aura_L3-OMTO3e_2015m0105_v003-2015m0108t113857.he5 ColumnAmountO3--- nan percentage:0.309537 max:497.700012 mean:308.813700 min:219.399994 ViewingZenithAngle--- nan percentage:0.309537 max:69.193001 mean:48.808518 min:16.243999 2015-01-06 OMI-Aura_L3-OMTO3e_2015m0106_v003-2015m0108t113853.he5 ColumnAmountO3--- nan percentage:0.314222 max:513.000000 mean:306.342606 min:217.300003 ViewingZenithAngle--- nan percentage:0.314222 max:69.195000 mean:48.701108 min:16.243000 2015-01-07 OMI-Aura_L3-OMTO3e_2015m0107_v003-2015m0109t002750.he5 ColumnAmountO3--- nan percentage:0.294426 max:536.099976 mean:314.851720 min:224.000000 ViewingZenithAngle--- nan percentage:0.294426 max:69.195999 mean:49.110231 min:16.243000 2015-01-08 OMI-Aura_L3-OMTO3e_2015m0108_v003-2015m0110t010055.he5 ColumnAmountO3--- nan percentage:0.297185 max:549.599976 mean:322.763464 min:230.300003 ViewingZenithAngle--- nan percentage:0.297185 max:69.195000 mean:49.126122 min:16.243999 2015-01-09 OMI-Aura_L3-OMTO3e_2015m0109_v003-2015m0111t014241.he5 ColumnAmountO3--- nan percentage:0.288315 max:570.599976 mean:332.564138 min:227.300003 ViewingZenithAngle--- nan percentage:0.288315 max:69.193001 mean:49.341615 min:16.243999 2015-01-10 OMI-Aura_L3-OMTO3e_2015m0110_v003-2015m0112t005224.he5 ColumnAmountO3--- nan percentage:0.291074 max:562.299988 mean:333.829648 min:223.199997 ViewingZenithAngle--- nan percentage:0.291074 max:69.195999 mean:49.198040 min:16.243999 # processing OMUVBd; print out processed file names by day; each day has measurements available; # also print out some descriptive information by variable of interests; 2015-01-01 OMI-Aura_L3-OMUVBd_2015m0101_v003-2016m0804t100041.he5 OMI-Aura_L3-OMUVBd_2015m0101_v003-2016m0804t100041.he5.1 UVindex--- nan percentage:0.349037 max:8.687814 mean:2.352308 min:0.008913 ViewingZenithAngle--- nan percentage:0.349037 max:69.188515 mean:51.069711 min:20.522678 SolarZenithAngle--- nan percentage:0.349037 max:82.710838 mean:63.299079 min:40.162312 2015-01-02 OMI-Aura_L3-OMUVBd_2015m0102_v003-2016m0804t102256.he5 OMI-Aura_L3-OMUVBd_2015m0102_v003-2016m0804t102256.he5.1 UVindex--- nan percentage:0.361778 max:8.711970 mean:2.412372 min:0.104606 ViewingZenithAngle--- nan percentage:0.361778 max:69.174545 mean:50.746142 min:20.522230 SolarZenithAngle--- nan percentage:0.361778 max:82.690384 mean:62.820181 min:40.078392 2015-01-03 OMI-Aura_L3-OMUVBd_2015m0103_v003-2016m0804t104458.he5 OMI-Aura_L3-OMUVBd_2015m0103_v003-2016m0804t104458.he5.1 UVindex--- nan percentage:0.330667 max:9.249155 mean:2.359278 min:0.099427 ViewingZenithAngle--- nan percentage:0.330667 max:69.187408 mean:51.042830 min:20.522699 SolarZenithAngle--- nan percentage:0.330667 max:82.929222 mean:63.550171 min:39.995792 2015-01-04 OMI-Aura_L3-OMUVBd_2015m0104_v003-2016m0804t110705.he5 OMI-Aura_L3-OMUVBd_2015m0104_v003-2016m0804t110705.he5.1 UVindex--- nan percentage:0.318815 max:9.161878 mean:2.294281 min:0.095895 ViewingZenithAngle--- nan percentage:0.318815 max:69.194679 mean:51.509275 min:20.522421 SolarZenithAngle--- nan percentage:0.318815 max:83.350426 mean:63.850062 min:39.800209 2015-01-05 OMI-Aura_L3-OMUVBd_2015m0105_v003-2016m0804t112911.he5 OMI-Aura_L3-OMUVBd_2015m0105_v003-2016m0804t112911.he5.1 UVindex--- nan percentage:0.323556 max:9.194166 mean:2.389001 min:0.099325 ViewingZenithAngle--- nan percentage:0.323556 max:69.193230 mean:51.367308 min:20.522968 SolarZenithAngle--- nan percentage:0.323556 max:83.544846 mean:63.395875 min:39.689663 2015-01-06 OMI-Aura_L3-OMUVBd_2015m0106_v003-2016m0804t115114.he5 OMI-Aura_L3-OMUVBd_2015m0106_v003-2016m0804t115114.he5.1 UVindex--- nan percentage:0.330667 max:9.170545 mean:2.561278 min:0.097484 ViewingZenithAngle--- nan percentage:0.330667 max:69.194778 mean:51.292197 min:20.523355 SolarZenithAngle--- nan percentage:0.330667 max:83.285896 mean:63.421688 min:39.499649 2015-01-07 OMI-Aura_L3-OMUVBd_2015m0107_v003-2016m0804t121318.he5 OMI-Aura_L3-OMUVBd_2015m0107_v003-2016m0804t121318.he5.1 UVindex--- nan percentage:0.309333 max:9.202747 mean:2.554112 min:0.083071 ViewingZenithAngle--- nan percentage:0.309333 max:69.195259 mean:51.848855 min:20.523193 SolarZenithAngle--- nan percentage:0.309333 max:83.829124 mean:63.520453 min:39.377087 2015-01-08 OMI-Aura_L3-OMUVBd_2015m0108_v003-2016m0804t123533.he5 OMI-Aura_L3-OMUVBd_2015m0108_v003-2016m0804t123533.he5.1 UVindex--- nan percentage:0.310519 max:9.021791 mean:2.340278 min:0.088377 ViewingZenithAngle--- nan percentage:0.310519 max:69.193031 mean:51.914574 min:20.523550 SolarZenithAngle--- nan percentage:0.310519 max:83.682640 mean:63.551738 min:39.195133 2015-01-09 OMI-Aura_L3-OMUVBd_2015m0109_v003-2016m0804t125746.he5 OMI-Aura_L3-OMUVBd_2015m0109_v003-2016m0804t125746.he5.1 UVindex--- nan percentage:0.304000 max:8.909929 mean:2.421541 min:0.099284 ViewingZenithAngle--- nan percentage:0.304000 max:69.192192 mean:52.148439 min:20.523149 SolarZenithAngle--- nan percentage:0.304000 max:83.522102 mean:63.461630 min:39.042801 2015-01-10 OMI-Aura_L3-OMUVBd_2015m0110_v003-2016m0804t131951.he5 OMI-Aura_L3-OMUVBd_2015m0110_v003-2016m0804t131951.he5.1 UVindex--- nan percentage:0.309333 max:8.655426 mean:2.341785 min:0.099157 ViewingZenithAngle--- nan percentage:0.309333 max:69.195183 mean:51.834389 min:20.523298 SolarZenithAngle--- nan percentage:0.309333 max:83.772453 mean:63.203806 min:38.900963 # processing OMO3PR; print out processed file names by day; each day has multiple measurements; # also print out some descriptive information by variable of interests; 2015-01-01 5 OMI-Aura_L2-OMO3PR_2015m0101t1441-o55663_v003-2015m0103t012306.he5 nan:0.575833 max:0.001199 min:0.000685 20150101 OMI-Aura_L2-OMO3PR_2015m0101t1620-o55664_v003-2015m0103t012225.he5 nan:0.500444 max:0.001424 min:0.000532 20150101 OMI-Aura_L2-OMO3PR_2015m0101t1759-o55665_v003-2015m0103t025655.he5 nan:0.077611 max:0.001586 min:0.000415 20150101 OMI-Aura_L2-OMO3PR_2015m0101t1938-o55666_v003-2015m0103t042913.he5 nan:0.150222 max:0.001601 min:0.000438 20150101 OMI-Aura_L2-OMO3PR_2015m0101t2117-o55667_v003-2015m0103t055513.he5 nan:0.348611 max:0.001668 min:0.000349 20150101 nan:0.001481 0.001380 0.000525 -60.125000 -134.875000 59.875000 15.125000 2015-01-02 4 OMI-Aura_L2-OMO3PR_2015m0102t1524-o55678_v003-2015m0104t000211.he5 nan:0.462611 max:0.001427 min:0.000553 20150102 OMI-Aura_L2-OMO3PR_2015m0102t1703-o55679_v003-2015m0104t032933.he5 nan:0.342870 max:0.001591 min:0.000544 20150102 OMI-Aura_L2-OMO3PR_2015m0102t1842-o55680_v003-2015m0104t053504.he5 nan:0.150426 max:0.001610 min:0.000512 20150102 OMI-Aura_L2-OMO3PR_2015m0102t2021-o55681_v003-2015m0104t065954.he5 nan:0.228426 max:0.001558 min:0.000457 20150102 nan:0.008500 0.001558 0.000519 -60.125000 -134.875000 59.875000 15.125000 2015-01-03 5 OMI-Aura_L2-OMO3PR_2015m0103t1429-o55692_v003-2015m0105t010138.he5 nan:0.611056 max:0.001259 min:0.000762 20150103 OMI-Aura_L2-OMO3PR_2015m0103t1608-o55693_v003-2015m0105t012617.he5 nan:0.362333 max:0.001548 min:0.000596 20150103 OMI-Aura_L2-OMO3PR_2015m0103t1746-o55694_v003-2015m0105t024217.he5 nan:0.088907 max:0.001625 min:0.000455 20150103 OMI-Aura_L2-OMO3PR_2015m0103t1925-o55695_v003-2015m0105t041712.he5 nan:0.127796 max:0.001575 min:0.000431 20150103 OMI-Aura_L2-OMO3PR_2015m0103t2104-o55696_v003-2015m0105t074347.he5 nan:0.307944 max:0.001417 min:0.000494 20150103 nan:0.000000 0.001271 0.000431 -60.125000 -134.875000 59.875000 15.125000 2015-01-04 4 OMI-Aura_L2-OMO3PR_2015m0104t1512-o55707_v003-2015m0106t014236.he5 nan:0.497296 max:0.001420 min:0.000714 20150104 OMI-Aura_L2-OMO3PR_2015m0104t1651-o55708_v003-2015m0106t032145.he5 nan:0.234537 max:0.001768 min:0.000547 20150104 OMI-Aura_L2-OMO3PR_2015m0104t1830-o55709_v003-2015m0106t051538.he5 nan:0.057722 max:0.001671 min:0.000438 20150104 OMI-Aura_L2-OMO3PR_2015m0104t2009-o55710_v003-2015m0106t050710.he5 nan:0.200667 max:0.001507 min:0.000502 20150104 nan:0.000000 0.001372 0.000595 -60.125000 -134.875000 59.875000 15.125000 # processing CMAQ data. print out processed file by day. Processing a single time is very time consuming; TEMPLATE:WRFV381_CMAQ_DEVv5_2Gamma_14Mar2017_cb6r3_ae6nvPOA_aq_2014fb.12US1.35L.cmaq.conc.$Date$ processing...WRFV381_CMAQ_DEVv5_2Gamma_14Mar2017_cb6r3_ae6nvPOA_aq_2014fb.12US1.35L.cmaq.conc.20140101 Elapsed time is 109.064443 seconds. processing...WRFV381_CMAQ_DEVv5_2Gamma_14Mar2017_cb6r3_ae6nvPOA_aq_2014fb.12US1.35L.cmaq.conc.20140102 Elapsed time is 96.606053 seconds. processing...WRFV381_CMAQ_DEVv5_2Gamma_14Mar2017_cb6r3_ae6nvPOA_aq_2014fb.12US1.35L.cmaq.conc.20140103 Elapsed time is 103.158008 seconds. # processing GFED data set; print out processed file names by year; each year has one file; reading...../data/unprocessed/GFED/GFED4.1s_2014.hdf5 reading...../data/unprocessed/GFED/GFED4.1s_2015.hdf5 reading...../data/unprocessed/GFED/GFED4.1s_2016.hdf5 saving...../data/aggregate/GFED/GFEDFireCarbon_USGFED_2015_2015.mat ############################################ ## Output from Run_InterpolateDataSummary### ############################################ # for meteorological reanalysis data: this controller file only prints out option used, year, input variable used, also tells users what it is doing (saving, removing missing values, taking mean values...) SiteName:AQRVPM25,Year:2010,Variable Name:apcp,Output format:By-Year,convolutional option:DailyMean ../data/unprocessed/Data_3HourNCEP/apcp/apcp.2010.nc AQRVPM25 2010 apcp removing Fill Value...9.969210e+36 removing missing Value...-9.969210e+36 removing values outside valid range...0 100 processing...taking...DailyMean removing Fill Value...9.969210e+36 removing missing Value...-9.969210e+36 removing values outside valid range...0 100 saving...2010 ************************* 0 ************************* SiteName:AQRVPM25,Year:2010,Variable Name:dlwrf,Output format:By-Year,convolutional option:DailyMean ../data/unprocessed/Data_3HourNCEP/dlwrf/dlwrf.2010.nc AQRVPM25 2010 dlwrf removing Fill Value...9.969210e+36 removing missing Value...-9.969210e+36 removing values outside valid range...-30 650 processing...taking...DailyMean removing Fill Value...9.969210e+36 removing missing Value...-9.969210e+36 removing values outside valid range...-30 650 saving...2010 # MOD09A1:since not all days have MOD09A1 measurements, the controller file only prints out file name when measurement is available. # the similar also holds for MOD13A2 ************************* 0 ************************* 03-Jan-2010 reading A...20100101 reading B...20100109 temporal interpolating...20100103 04-Jan-2010 reading A...20100101 reading B...20100109 temporal interpolating...20100104 05-Jan-2010 reading A...20100101 reading B...20100109 temporal interpolating...20100105 06-Jan-2010 reading A...20100101 reading B...20100109 temporal interpolating...20100106 07-Jan-2010 reading A...20100101 reading B...20100109 temporal interpolating...20100107 08-Jan-2010 reading A...20100101 reading B...20100109 temporal interpolating...20100108 09-Jan-2010 reading...../data/aggregate/MOD09A1/MOD09A1_20100109_20100109.mat 10-Jan-2010 reading A...20100109 reading B...20100117 temporal interpolating...20100110 11-Jan-2010 reading A...20100109 reading B...20100117 temporal interpolating...20100111 12-Jan-2010 reading A...20100109 reading B...20100117 temporal interpolating...20100112 13-Jan-2010 reading A...20100109 reading B...20100117 temporal interpolating...20100113 14-Jan-2010 reading A...20100109 reading B...20100117 temporal interpolating...20100114 15-Jan-2010 reading A...20100109 reading B...20100117 temporal interpolating...20100115 # MOD11A1:the controller prints file names while reading: SiteName:AQRVPM25,Year:2010,Variable Name:MOD11A1,Output format:By-Year,convolutional option:NaN creating neighourhood files... reading...20100101 MOD11A1_USMOD11A1_20100101_20100101.mat reading...20100102 MOD11A1_USMOD11A1_20100102_20100102.mat reading...20100103 MOD11A1_USMOD11A1_20100103_20100103.mat reading...20100104 MOD11A1_USMOD11A1_20100104_20100104.mat reading...20100105 MOD11A1_USMOD11A1_20100105_20100105.mat reading...20100106 MOD11A1_USMOD11A1_20100106_20100106.mat reading...20100107 MOD11A1_USMOD11A1_20100107_20100107.mat reading...20100108 MOD11A1_USMOD11A1_20100108_20100108.mat reading...20100109 MOD11A1_USMOD11A1_20100109_20100109.mat reading...20100110 MOD11A1_USMOD11A1_20100110_20100110.mat reading...20100111 MOD11A1_USMOD11A1_20100111_20100111.mat reading...20100112 MOD11A1_USMOD11A1_20100112_20100112.mat #For MAIAC AOD data, the controller file prints out file names while reading: SiteName:AQRVPM25,Year:2010,Variable Name:MAIACUSAqua,Output format:By-Year,convolutional option:NaN creating neighourhood files... reading...20100101 MAIACUSAqua_USMAIACUS5km_20100101_20100101.mat reading...20100102 MAIACUSAqua_USMAIACUS5km_20100102_20100102.mat reading...20100103 MAIACUSAqua_USMAIACUS5km_20100103_20100103.mat reading...20100104 MAIACUSAqua_USMAIACUS5km_20100104_20100104.mat reading...20100105 MAIACUSAqua_USMAIACUS5km_20100105_20100105.mat reading...20100106 MAIACUSAqua_USMAIACUS5km_20100106_20100106.mat reading...20100107 MAIACUSAqua_USMAIACUS5km_20100107_20100107.mat reading...20100108 MAIACUSAqua_USMAIACUS5km_20100108_20100108.mat reading...20100109 MAIACUSAqua_USMAIACUS5km_20100109_20100109.mat reading...20100110 MAIACUSAqua_USMAIACUS5km_20100110_20100110.mat # for all OMI data set, the controller file prints option used, the data set being read, and output file name: # the output for CMAQ is similar ************************* SiteName:AQRVPM25,Year:2010,Variable Name:OMAERUVd_UVAerosolIndex,Output format:By-Year,convolutional option:NaN creating neighourhood files... reading...../data/aggregate/OMAERUVd/OMAERUVd_UVAerosolIndex_USOMAERUVd_2010_2010.mat saving...../../processed_data/AQRVPM25/OMAERUVd/OMAERUVd_UVAerosolIndex_Mean_Thres200000_AQRVPM25_20100101_20101231.mat ************************* 0 ************************* SiteName:AQRVPM25,Year:2010,Variable Name:OMAEROe_UVAerosolIndex,Output format:By-Year,convolutional option:NaN creating neighourhood files... reading...../data/aggregate/OMAEROe/OMAEROe_UVAerosolIndex_USOMAEROe_2010_2010.mat saving...../../processed_data/AQRVPM25/OMAEROe/OMAEROe_UVAerosolIndex_Mean_Thres100000_AQRVPM25_20100101_20101231.mat ************************* 0 ************************* SiteName:AQRVPM25,Year:2010,Variable Name:OMAEROe_VISAerosolIndex,Output format:By-Year,convolutional option:NaN reading...../data/aggregate/OMAEROe/OMAEROe_VISAerosolIndex_USOMAEROe_2010_2010.mat saving...../../processed_data/AQRVPM25/OMAEROe/OMAEROe_VISAerosolIndex_Mean_Thres100000_AQRVPM25_20100101_20101231.mat ************************* 0 ************************* SiteName:AQRVPM25,Year:2010,Variable Name:CMAQ_PM25_Vertical,Output format:By-Year,convolutional option:NaN creating neighourhood files... reading...../data/aggregate/CMAQ/CMAQ_PM25_Vertical_CMAQ_2010_2010.mat saving...../../processed_data/AQRVPM25/CMAQ/CMAQ_PM25_Vertical_Mean_Thres50000_AQRVPM25_20100101_20101231.mat ************************* 0 ************************* SiteName:AQRVPM25,Year:2010,Variable Name:CMAQ_PM25_TOT,Output format:By-Year,convolutional option:NaN reading...../data/aggregate/CMAQ/CMAQ_PM25_TOT_CMAQ_2010_2010.mat saving...../../processed_data/AQRVPM25/CMAQ/CMAQ_PM25_TOT_Mean_Thres50000_AQRVPM25_20100101_20101231.mat ************************* 0 ************************* SiteName:AQRVPM25,Year:2010,Variable Name:CMAQ_PM25_SO4,Output format:By-Year,convolutional option:NaN reading...../data/aggregate/CMAQ/CMAQ_PM25_SO4_CMAQ_2010_2010.mat saving...../../processed_data/AQRVPM25/CMAQ/CMAQ_PM25_SO4_Mean_Thres50000_AQRVPM25_20100101_20101231.mat ************************* 0 ************************* # nearby monitoring data: the controller file prints option used, the data set being read, and output file name: ************************* 0 ************************* SiteName:AQRVPM25,Year:2010,Variable Name:NO2,Output format:Peak2_Lag0,convolutional option:NaN creating neighourhood files... reading...../../processed_data/EPANO2/Monitor/MONITOR_NO2_EPANO2_2010_2010.mat saving...../../processed_data/AQRVPM25/Nearby/MONITOR_Peak2Lag0_ThresInf_NO2_AQRVPM25_20100101_20101231.mat ************************* 0 ************************* SiteName:AQRVPM25,Year:2010,Variable Name:NO2,Output format:Peak2_Lag1,convolutional option:NaN reading...../../processed_data/EPANO2/Monitor/MONITOR_NO2_EPANO2_2010_2010.mat saving...../../processed_data/AQRVPM25/Nearby/MONITOR_Peak2Lag1_ThresInf_NO2_AQRVPM25_20100101_20101231.mat ************************* 0 ************************* SiteName:AQRVPM25,Year:2010,Variable Name:NO2,Output format:Peak2_Lag3,convolutional option:NaN reading...../../processed_data/EPANO2/Monitor/MONITOR_NO2_EPANO2_2010_2010.mat saving...../../processed_data/AQRVPM25/Nearby/MONITOR_Peak2Lag3_ThresInf_NO2_AQRVPM25_20100101_20101231.mat ## land use data: the controller file prints option used, the data set being read, and output file name; ## if the controller file read two land use variables and did some interpolation, it prints names of both files. ************************* 0 ************************* SiteName:AQRVPM25,Year:2010,Variable Name:RoadDensity_prisecroads1000,Output format:By-Year,convolutional option:NaN reading...../data/aggregate/LANDUSE/RoadDensity_prisecroads1000_2010_2010.tif ************************* 0 ************************* SiteName:AQRVPM25,Year:2010,Variable Name:RoadDensity_prisecroads10000,Output format:By-Year,convolutional option:NaN reading...../data/aggregate/LANDUSE/RoadDensity_prisecroads10000_2010_2010.tif ************************* 0 ************************* SiteName:AQRVPM25,Year:2010,Variable Name:RoadDensity_roads1000,Output format:By-Year,convolutional option:NaN reading...../data/aggregate/LANDUSE/RoadDensity_roads1000_2010_2010.tif ************************* 0 ************************* SiteName:AQRVPM25,Year:2010,Variable Name:NLCD_Water100,Output format:By-Year,convolutional option:NaN reading A...../data/aggregate/LANDUSE/NLCD_Water100_2006_2006.tif reading B...../data/aggregate/LANDUSE/NLCD_Water100_2011_2011.tif temporal interpolating...2010 ************************* 0 ************************* ########################################## ## Output from Run_AssembleDataSummary### ########################################## Run_AssembleDataSummary(1,99941,2008,'All_round3','Ozone','AllRecords',''); reading ./VariableList_99941.csv variables to be read... MonitorData MOD13A2_Nearest4 USElevation_dsc10000 USElevation_max100 USElevation_max10000 USElevation_mea100 USElevation_mea10000 USElevation_med100 USElevation_med10000 USElevation_min100 USElevation_min10000 USElevation_std100 USElevation_std10000 USElevation_bln100 USElevation_bln10000 NLCD_Barren100 NLCD_Barren10000 NLCD_Developed100 NLCD_Developed10000 NLCD_Herbaceous100 NLCD_Herbaceous10000 NLCD_Planted100 NLCD_Planted10000 NLCD_Shrubland100 NLCD_Shrubland10000 NLCD_Water100 NLCD_Water10000 NLCD_Wetlands100 NLCD_Wetlands10000 NLCD_Impervious100 NLCD_Impervious10000 NLCD_canopy100 NLCD_canopy10000 RoadDensity_primaryroads1000 RoadDensity_primaryroads10000 RoadDensity_prisecroads1000 RoadDensity_prisecroads10000 RoadDensity_roads1000 Business_Restaurant1000 REANALYSIS_hpbl_DailyMax REANALYSIS_shum_2m_DailyMax REANALYSIS_windspeed_10m_DailyMax REANALYSIS_prate_DailyMax REANALYSIS_vis_DailyMax REANALYSIS_apcp_DailyMean REANALYSIS_dlwrf_DailyMean REANALYSIS_dswrf_DailyMean REANALYSIS_evap_DailyMean REANALYSIS_hpbl_DailyMean REANALYSIS_gflux_DailyMean REANALYSIS_lhtfl_DailyMean REANALYSIS_shtfl_DailyMean REANALYSIS_shum_2m_DailyMean REANALYSIS_snowc_DailyMean REANALYSIS_soilm_DailyMean REANALYSIS_tcdc_DailyMean REANALYSIS_ulwrf_DailyMean REANALYSIS_omega_DailyMean REANALYSIS_windspeed_10m_DailyMean REANALYSIS_weasd_DailyMean REANALYSIS_prate_DailyMean REANALYSIS_vis_DailyMean REANALYSIS_hpbl_DailyMin REANALYSIS_shum_2m_DailyMin REANALYSIS_windspeed_10m_DailyMin REANALYSIS_prate_DailyMin REANALYSIS_vis_DailyMin REANALYSIS_hpbl_1Day REANALYSIS_shum_2m_1Day REANALYSIS_windspeed_10m_1Day REANALYSIS_prate_1Day REANALYSIS_vis_1Day MOD11A1_LST_Day_1km_Nearest4 MOD11A1_LST_Night_1km_Nearest4 MOD11A1_Clear_day_cov_Nearest4 MOD11A1_Clear_night_cov_Nearest4 MAIACUS_Optical_Depth_047_Aqua_Nearest4 MAIACUS_Optical_Depth_055_Aqua_Nearest4 MAIACUS_Optical_Depth_047_Terra_Nearest4 MAIACUS_Optical_Depth_055_Terra_Nearest4 MAIACUS_cosVZA_Aqua_Nearest MAIACUS_cosVZA_Terra_Nearest REANALYSIS_air_sfc_DailyMin REANALYSIS_air_sfc_DailyMean REANALYSIS_air_sfc_DailyMax REANALYSIS_air_sfc_1Day Other_Lat Other_Lon CalendarDay Spatial_Lagged_1 Spatial_Lagged_2 Spatial_Lagged_3 Temporal_Lagged_1 Temporal_Lagged_2 Temporal_Lagged_3 GFEDFireCarbon CMAQ_NO2 CMAQ_NO2_Vertical CMAQ_Ozone CMAQ_Ozone_Vertical CMAQ_PM25_TOT CMAQ_PM25_Vertical CMAQ_PM25_NO3 CMAQ_PM25_SO4 OMAEROe_UVAerosolIndex_Mean OMAEROe_VISAerosolIndex_Mean OMAERUVd_UVAerosolIndex_Mean OMNO2d_ColumnAmountNO2StratoCloudScreened_Mean OMSO2e_ColumnAmountSO2_PBL_Mean OMUVBd_UVindex_Mean OMTO3e_ColumnAmountO3 OMO3PR MOD04L2_550 MOD09A1 PM25_Region Ozone_Region NO2_Region Nearby_Peak2_NO2 Nearby_Peak2_PM25 Nearby_Peak2_Ozone Nearby_Peak2Lag1_NO2 Nearby_Peak2Lag1_PM25 Nearby_Peak2Lag1_Ozone Nearby_Peak2Lag3_NO2 Nearby_Peak2Lag3_PM25 Nearby_Peak2Lag3_Ozone
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.