Comments (9)
giswqs
commented on July 18, 2024
1
BTW, I am excited to see that you can load raster datasets into DuckDB! That would be a game changer. Can't wait to try it out when it becomes available
from maxar-open-data.
giswqs
commented on July 18, 2024
Can you share the asset path to the image so that I can look into it? I wondered if this is an outlier (artifact). If many images have the same issue, we need to report it to Maxar
from maxar-open-data.
marklit
commented on July 18, 2024
I used a bunch of Python to extract TIFF Stack metadata, dumped that to JSON and loaded the JSON into DuckDB.
This is the image I ran through convert: https://maxar-opendata.s3.amazonaws.com/events/New-Zealand-Flooding23/ard/60/213311212330/2023-02-08/10300100E23B4300-visual.tif
Every image across all ~20 datasets is built in the same way, an RGB image at the top of the stack followed by a lot of other, unlabeled images.
from maxar-open-data.
giswqs
commented on July 18, 2024
This issue seems caused by the conversion you did. The origial image renders properly in QGIS and file explorer without QGIS. The original image contains only RGB bands.
from maxar-open-data.
giswqs
commented on July 18, 2024
This is the output from gdalinfo 10300100E23B4300-visual.tif
Driver: GTiff/GeoTIFF
Files: 10300100E23B4300-visual.tif
Size is 17408, 17408
Coordinate System is:
PROJCRS["WGS 84 / UTM zone 60S",
BASEGEOGCRS["WGS 84",
DATUM["World Geodetic System 1984",
ELLIPSOID["WGS 84",6378137,298.257223563,
LENGTHUNIT["metre",1]]],
PRIMEM["Greenwich",0,
ANGLEUNIT["degree",0.0174532925199433]],
ID["EPSG",4326]],
CONVERSION["UTM zone 60S",
METHOD["Transverse Mercator",
ID["EPSG",9807]],
PARAMETER["Latitude of natural origin",0,
ANGLEUNIT["degree",0.0174532925199433],
ID["EPSG",8801]],
PARAMETER["Longitude of natural origin",177,
ANGLEUNIT["degree",0.0174532925199433],
ID["EPSG",8802]],
PARAMETER["Scale factor at natural origin",0.9996,
SCALEUNIT["unity",1],
ID["EPSG",8805]],
PARAMETER["False easting",500000,
LENGTHUNIT["metre",1],
ID["EPSG",8806]],
PARAMETER["False northing",10000000,
LENGTHUNIT["metre",1],
ID["EPSG",8807]]],
CS[Cartesian,2],
AXIS["(E)",east,
ORDER[1],
LENGTHUNIT["metre",1]],
AXIS["(N)",north,
ORDER[2],
LENGTHUNIT["metre",1]],
USAGE[
SCOPE["Navigation and medium accuracy spatial referencing."],
AREA["Between 174°E and 180°E, southern hemisphere between 80°S and equator, onshore and offshore. New Zealand."],
BBOX[-80,174,0,180]],
ID["EPSG",32760]]
Data axis to CRS axis mapping: 1,2
Origin = (289843.750000000000000,5930156.250000000000000)
Pixel Size = (0.305175781250000,-0.305175781250000)
Metadata:
AREA_OR_POINT=Area
Image Structure Metadata:
COMPRESSION=YCbCr JPEG
INTERLEAVE=PIXEL
JPEGTABLESMODE=1
JPEG_QUALITY=75
SOURCE_COLOR_SPACE=YCbCr
Corner Coordinates:
Upper Left ( 289843.750, 5930156.250) (174d38'45.24"E, 36d45' 3.90"S)
Lower Left ( 289843.750, 5924843.750) (174d38'39.97"E, 36d47'56.16"S)
Upper Right ( 295156.250, 5930156.250) (174d42'19.32"E, 36d45' 8.08"S)
Lower Right ( 295156.250, 5924843.750) (174d42'14.18"E, 36d48' 0.35"S)
Center ( 292500.000, 5927500.000) (174d40'29.68"E, 36d46'32.13"S)
Band 1 Block=512x512 Type=Byte, ColorInterp=Red
Overviews: 8704x8704, 4352x4352, 2176x2176, 1088x1088, 544x544, 272x272
Mask Flags: PER_DATASET
Overviews of mask band: 8704x8704, 4352x4352, 2176x2176, 1088x1088, 544x544, 272x272
Band 2 Block=512x512 Type=Byte, ColorInterp=Green
Overviews: 8704x8704, 4352x4352, 2176x2176, 1088x1088, 544x544, 272x272
Mask Flags: PER_DATASET
Overviews of mask band: 8704x8704, 4352x4352, 2176x2176, 1088x1088, 544x544, 272x272
Band 3 Block=512x512 Type=Byte, ColorInterp=Blue
Overviews: 8704x8704, 4352x4352, 2176x2176, 1088x1088, 544x544, 272x272
Mask Flags: PER_DATASET
Overviews of mask band: 8704x8704, 4352x4352, 2176x2176, 1088x1088, 544x544, 272x272
from maxar-open-data.
giswqs
commented on July 18, 2024
@drwelby works on Maxar Open Data releases. He might be able to help here.
from maxar-open-data.
marklit
commented on July 18, 2024
I'm able to open the GeoTIFFs in QGIS and see them just fine. It's the first of the 14 images inside of the GeoTIFF that we can see there. Half of the other images extracted from the GeoTIFF render fine as well.
My question is what are these 14 layers? Why do ~7 of them not render in Explorer? Are they meta data? Masks? Are these 14 layers within the GeoTIFF documented anywhere?
from maxar-open-data.
giswqs
commented on July 18, 2024
Those layers are probably image pyramids. You can see that each image layer width decreases in half from the preceeding layer.
tiffinfo 10300100E23B4300-visual.tifTIFFReadDirectory: Warning, Unknown field with tag 33550 (0x830e) encountered.
TIFFReadDirectory: Warning, Unknown field with tag 33922 (0x8482) encountered.
TIFFReadDirectory: Warning, Unknown field with tag 34735 (0x87af) encountered.
TIFFReadDirectory: Warning, Unknown field with tag 34737 (0x87b1) encountered.
=== TIFF directory 0 ===
TIFF Directory at offset 0x8 (8)
Image Width: 17408 Image Length: 17408
Tile Width: 512 Tile Length: 512
Bits/Sample: 8
Sample Format: unsigned integer
Compression Scheme: JPEG
Photometric Interpretation: YCbCr
YCbCr Subsampling: 2, 2
Samples/Pixel: 3
Planar Configuration: single image plane
Reference Black/White:
0: 0 255
1: 128 255
2: 128 255
Tag 33550: 0.305176,0.305176,0.000000
Tag 33922: 0.000000,0.000000,0.000000,289843.750000,5930156.250000,0.000000
Tag 34735: 1,1,0,7,1024,0,1,1,1025,0,1,1,1026,34737,22,0,2049,34737,7,22,2054,0,1,9102,3072,0,1,32760,3076,0,1,9001
Tag 34737: WGS 84 / UTM zone 60S|WGS 84|
JPEG Tables: (142 bytes)
=== TIFF directory 1 ===
TIFF Directory at offset 0x2682 (9858)
Subfile Type: transparency mask (4 = 0x4)
Image Width: 17408 Image Length: 17408
Tile Width: 512 Tile Length: 512
Bits/Sample: 1
Sample Format: unsigned integer
Compression Scheme: AdobeDeflate
Photometric Interpretation: transparency mask
Samples/Pixel: 1
Planar Configuration: single image plane
Predictor: none 1 (0x1)
=== TIFF directory 2 ===
TIFF Directory at offset 0x22aabb0 (36350896)
Subfile Type: reduced-resolution image (1 = 0x1)
Image Width: 8704 Image Length: 8704
Tile Width: 512 Tile Length: 512
Bits/Sample: 8
Sample Format: unsigned integer
Compression Scheme: JPEG
Photometric Interpretation: YCbCr
YCbCr Subsampling: 2, 2
Samples/Pixel: 3
Planar Configuration: single image plane
Reference Black/White:
0: 0 255
1: 128 255
2: 128 255
JPEG Tables: (142 bytes)
=== TIFF directory 3 ===
TIFF Directory at offset 0x22ab648 (36353608)
Subfile Type: reduced-resolution image (1 = 0x1)
Image Width: 4352 Image Length: 4352
Tile Width: 512 Tile Length: 512
Bits/Sample: 8
Sample Format: unsigned integer
Compression Scheme: JPEG
Photometric Interpretation: YCbCr
YCbCr Subsampling: 2, 2
Samples/Pixel: 3
Planar Configuration: single image plane
Reference Black/White:
0: 0 255
1: 128 255
2: 128 255
JPEG Tables: (142 bytes)
=== TIFF directory 4 ===
TIFF Directory at offset 0x22aba60 (36354656)
Subfile Type: reduced-resolution image (1 = 0x1)
Image Width: 2176 Image Length: 2176
Tile Width: 512 Tile Length: 512
Bits/Sample: 8
Sample Format: unsigned integer
Compression Scheme: JPEG
Photometric Interpretation: YCbCr
YCbCr Subsampling: 2, 2
Samples/Pixel: 3
Planar Configuration: single image plane
Reference Black/White:
0: 0 255
1: 128 255
2: 128 255
JPEG Tables: (142 bytes)
=== TIFF directory 5 ===
TIFF Directory at offset 0x22abcb8 (36355256)
Subfile Type: reduced-resolution image (1 = 0x1)
Image Width: 1088 Image Length: 1088
Tile Width: 512 Tile Length: 512
Bits/Sample: 8
Sample Format: unsigned integer
Compression Scheme: JPEG
Photometric Interpretation: YCbCr
YCbCr Subsampling: 2, 2
Samples/Pixel: 3
Planar Configuration: single image plane
Reference Black/White:
0: 0 255
1: 128 255
2: 128 255
JPEG Tables: (142 bytes)
=== TIFF directory 6 ===
TIFF Directory at offset 0x22abe90 (36355728)
Subfile Type: reduced-resolution image (1 = 0x1)
Image Width: 544 Image Length: 544
Tile Width: 512 Tile Length: 512
Bits/Sample: 8
Sample Format: unsigned integer
Compression Scheme: JPEG
Photometric Interpretation: YCbCr
YCbCr Subsampling: 2, 2
Samples/Pixel: 3
Planar Configuration: single image plane
Reference Black/White:
0: 0 255
1: 128 255
2: 128 255
JPEG Tables: (142 bytes)
=== TIFF directory 7 ===
TIFF Directory at offset 0x22ac040 (36356160)
Subfile Type: reduced-resolution image (1 = 0x1)
Image Width: 272 Image Length: 272
Tile Width: 512 Tile Length: 512
Bits/Sample: 8
Sample Format: unsigned integer
Compression Scheme: JPEG
Photometric Interpretation: YCbCr
YCbCr Subsampling: 2, 2
Samples/Pixel: 3
Planar Configuration: single image plane
Reference Black/White:
0: 0 255
1: 128 255
2: 128 255
JPEG Tables: (142 bytes)
=== TIFF directory 8 ===
TIFF Directory at offset 0x22ac1d0 (36356560)
Subfile Type: reduced-resolution image/transparency mask (5 = 0x5)
Image Width: 8704 Image Length: 8704
Tile Width: 512 Tile Length: 512
Bits/Sample: 1
Sample Format: unsigned integer
Compression Scheme: AdobeDeflate
Photometric Interpretation: transparency mask
Samples/Pixel: 1
Planar Configuration: single image plane
Predictor: none 1 (0x1)
=== TIFF directory 9 ===
TIFF Directory at offset 0x22acb86 (36359046)
Subfile Type: reduced-resolution image/transparency mask (5 = 0x5)
Image Width: 4352 Image Length: 4352
Tile Width: 512 Tile Length: 512
Bits/Sample: 1
Sample Format: unsigned integer
Compression Scheme: AdobeDeflate
Photometric Interpretation: transparency mask
Samples/Pixel: 1
Planar Configuration: single image plane
Predictor: none 1 (0x1)
=== TIFF directory 10 ===
TIFF Directory at offset 0x22acebc (36359868)
Subfile Type: reduced-resolution image/transparency mask (5 = 0x5)
Image Width: 2176 Image Length: 2176
Tile Width: 512 Tile Length: 512
Bits/Sample: 1
Sample Format: unsigned integer
Compression Scheme: AdobeDeflate
Photometric Interpretation: transparency mask
Samples/Pixel: 1
Planar Configuration: single image plane
Predictor: none 1 (0x1)
=== TIFF directory 11 ===
TIFF Directory at offset 0x22ad032 (36360242)
Subfile Type: reduced-resolution image/transparency mask (5 = 0x5)
Image Width: 1088 Image Length: 1088
Tile Width: 512 Tile Length: 512
Bits/Sample: 1
Sample Format: unsigned integer
Compression Scheme: AdobeDeflate
Photometric Interpretation: transparency mask
Samples/Pixel: 1
Planar Configuration: single image plane
Predictor: none 1 (0x1)
=== TIFF directory 12 ===
TIFF Directory at offset 0x22ad128 (36360488)
Subfile Type: reduced-resolution image/transparency mask (5 = 0x5)
Image Width: 544 Image Length: 544
Tile Width: 512 Tile Length: 512
Bits/Sample: 1
Sample Format: unsigned integer
Compression Scheme: AdobeDeflate
Photometric Interpretation: transparency mask
Samples/Pixel: 1
Planar Configuration: single image plane
Predictor: none 1 (0x1)
=== TIFF directory 13 ===
TIFF Directory at offset 0x22ad1f6 (36360694)
Subfile Type: reduced-resolution image/transparency mask (5 = 0x5)
Image Width: 272 Image Length: 272
Tile Width: 512 Tile Length: 512
Bits/Sample: 1
Sample Format: unsigned integer
Compression Scheme: AdobeDeflate
Photometric Interpretation: transparency mask
Samples/Pixel: 1
Planar Configuration: single image plane
Predictor: none 1 (0x1)
from maxar-open-data.
marklit
commented on July 18, 2024
That's what it is. Thanks for pointing that out. I missed that even though I had a column stating 'reduced-resolution image'.
from maxar-open-data.
Related Issues (9)
- ~60 datasets from 2010 - mid-2021? HOT 3
- ValueError: '10300500E4F91900' is not in list HOT 1
- KeyError: 'tiles'
- Incorrect URL in streamlit_app.py HOT 2
- Visualizing the Morrocco "after" datasets HOT 1
- How to get longitude/Latitudes as well as images for these events? HOT 4
- Any plans to include metadata URLs alongside the visual.tif URLs? HOT 1
- New-Zealand-Flooding22 images have gone missing HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from maxar-open-data.