nilsmagnus / grib Goto Github PK
View Code? Open in Web Editor NEWGolang GRIB2 parser
License: Other
Golang GRIB2 parser
License: Other
When reducing file, the grid is left untouched even if the filter-options for coordinates is used.
Fix: update latitude/longitude start/stop when reducing on coordinates.
Seems like the data in Data3 is parsed incorrectly. Have a look at how it is done in https://github.com/Unidata/thredds/blob/5.0.0/grib/src/main/java/ucar/nc2/grib/grib2/Grib2DataReader.java#L374
and re-implement the parsing.
Seems like parts of bitreader.go
is copy-paste from the golang bzip2 package. Because of this, license and creditation needs to be updated.
There is duplicate code, fix this by using 1 of the implementations.
add godoc to github
I'm working with the library to decode JPEG2000 data as spec'd here. I'd like to put together a pull request with my work but am unsure how the maintainers wish to handle decoding of the JPEG2000 format.
It doesn't appear that anyone has put together a native Go JPEG2000 decoder and as such I've been making use of gographic's Imagick bindings to convert the data into a native Image
instance.
Would this work still be of value to the project given that it introduces two dependencies (the Imagick binding and the imagemagick binary suite)?
panic: runtime error: index out of range
goroutine 1 [running]:
github.com/nilsmagnus/grib/griblib.ParseData3(0x848a20, 0xc42908a060, 0x22eb, 0xc42004d080, 0x5, 0xc42d3c1438, 0x7000022f0)
/home/larsgard/go/src/github.com/nilsmagnus/grib/griblib/data.go:215 +0x10fd
github.com/nilsmagnus/grib/griblib.ReadSection7(0x848a20, 0xc42908a060, 0x22eb, 0x40000000, 0x6258d19a01010004, 0x6deffffffff, 0x100000300, 0x6a00000001, 0x10208, 0x0, ...)
/home/larsgard/go/src/github.com/nilsmagnus/grib/griblib/sections.go:341 +0x4d
github.com/nilsmagnus/grib/griblib.readMessage(0x848a20, 0xc42908a060, 0x202000047524942, 0x23ba, 0x202000047524942, 0x23ba, 0x1010200000007, 0xc0b0c07e1, 0x100, 0x0, ...)
/home/larsgard/go/src/github.com/nilsmagnus/grib/griblib/sections.go:117 +0x537
github.com/nilsmagnus/grib/griblib.ReadMessage(0x849720, 0xc42000e040, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
/home/larsgard/go/src/github.com/nilsmagnus/grib/griblib/sections.go:86 +0x2d9
github.com/nilsmagnus/grib/griblib.ReadMessages(0x849720, 0xc42000e040, 0x0, 0x0, 0x0, 0x1f96, 0x7ffdf9daecfc)
/home/larsgard/go/src/github.com/nilsmagnus/grib/griblib/sections.go:51 +0xa9
main.main()
/home/larsgard/go/src/github.com/nilsmagnus/grib2influx/main.go:47 +0x17a
how about grib1
panic: runtime error: makeslice: len out of range
goroutine 1 [running]:
github.com/nilsmagnus/grib/griblib.ReadMessage(0x177be20, 0xc420336060, 0x0, 0x17e6fc8, 0x0)
github.com/nilsmagnus/grib/griblib/sections.go:80 +0xb4
github.com/nilsmagnus/grib/griblib.ReadMessages(0x177be20, 0xc420336060, 0xc42004f520, 0x55ec6e, 0x7fa8dea7a160, 0xc420382060, 0xc42004f520)
github.com/nilsmagnus/grib/griblib/sections.go:58 +0x9b
I downloaded the gribb file from weather.gov and I get this error when
Input parameters : {"operation":"parse","discipline":-1,"dataExport":true,"category":-1,"filePath":"latest.grib2","reduceFilePath":"reduced.grib2","exportType":3,"maximumNumberOfMessages":2147483647,"geoFilter":{"minLat":0,"maxLat":36000000,"minLong":-9000000,"maxLong":9000000},"surfaceFilter":{"type":0,"scale":0,"value":0}}
Error when parsing a message, Template number not supported: 41
[
]
Hello,
I apologize in advance, I'm falling into the grib files recently, to replace an old Python import which I understand nothing. So I try to understand but I admit that it's a bit complicated ^^
I downloaded a grb file from the NOAA FTP site, but when I try to export to json, I have these following errors.
I am trying to extract meteorological data from grib files for a weather site so I am trying to do a complete redesign.
Thanks for your help
$ go version
go version go1.6.2 linux/amd64
$ grib -export 3 -file pgb174.grb
Input parameters : griblib.Options{Operation:"parse", Discipline:-1, DataExport:true, Category:-1, Filepath:"pgb174.grb", ReduceFilePath:"reduced.grib2", ExportType:3, MaximumNumberOfMessages:2147483647, GeoFilter:griblib.GeoFilter{MinLat:-90000000, MaxLat:90000000, MinLong:0, MaxLong:360000000}, Surface:griblib.Surface{Type:0xff, Scale:0x0, Value:0x0}}
Corrupt message "runtime error: index out of range"
Corrupt message "runtime error: index out of range"
Corrupt message "runtime error: index out of range"
Corrupt message "runtime error: index out of range"
Corrupt message "runtime error: index out of range"
Corrupt message "runtime error: index out of range"
Corrupt message "runtime error: index out of range"
Corrupt message "runtime error: index out of range"
Corrupt message "runtime error: index out of range"
Corrupt message "runtime error: index out of range"
Corrupt message "runtime error: index out of range"
Corrupt message "runtime error: index out of range"
Corrupt message "runtime error: index out of range"
Corrupt message "runtime error: index out of range"
Corrupt message "runtime error: index out of range"
Corrupt message "runtime error: index out of range"
Corrupt message "runtime error: index out of range"
Corrupt message "runtime error: index out of range"
Corrupt message "runtime error: index out of range"
Corrupt message "runtime error: index out of range"
Corrupt message "runtime error: index out of range"
Corrupt message "runtime error: index out of range"
Corrupt message "runtime error: index out of range"
Corrupt message "runtime error: index out of range"
Corrupt message "runtime error: index out of range"
Corrupt message "runtime error: index out of range"
Corrupt message "runtime error: index out of range"
Corrupt message "runtime error: index out of range"
Corrupt message "runtime error: index out of range"
Corrupt message "runtime error: index out of range"
Corrupt message "runtime error: index out of range"
Corrupt message "runtime error: index out of range"
Corrupt message "runtime error: index out of range"
Corrupt message "runtime error: index out of range"
Corrupt message "runtime error: index out of range"
Corrupt message "runtime error: index out of range"
Corrupt message "runtime error: index out of range"
Corrupt message "runtime error: index out of range"
Corrupt message "runtime error: index out of range"
Corrupt message "runtime error: index out of range"
Corrupt message "runtime error: index out of range"
Corrupt message "runtime error: index out of range"
Corrupt message "runtime error: index out of range"
Corrupt message "runtime error: index out of range"
Corrupt message "runtime error: index out of range"
Corrupt message "runtime error: index out of range"
Corrupt message "runtime error: index out of range"
Corrupt message "runtime error: index out of range"
Corrupt message "runtime error: index out of range"
Corrupt message "runtime error: index out of range"
Corrupt message "runtime error: index out of range"
Corrupt message "runtime error: index out of range"
Corrupt message "runtime error: index out of range"
Corrupt message "runtime error: index out of range"
Corrupt message "runtime error: index out of range"
Corrupt message "runtime error: index out of range"
Corrupt message "runtime error: index out of range"
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
Using GeoFilter %#V {-90000000 90000000 0 360000000}
panic: runtime error: index out of range
goroutine 1 [running]:
panic(0x56cea0, 0xc82000e0c0)
/usr/lib/go-1.6/src/runtime/panic.go:481 +0x3e6
github.com/nilsmagnus/grib/griblib.filterValuesFromGeoFilter(0xc82f3d9a00, 0x55d4a80faa2b580, 0x15752a0000000000, 0x35, 0x0, 0x0)
/home/jewome62/go/src/github.com/nilsmagnus/grib/griblib/filters.go:97 +0x2b7
github.com/nilsmagnus/grib/griblib.Filter(0xc8ccba1000, 0x1a1, 0x200, 0x5a1dc8, 0x5, 0xffffffffffffffff, 0x1, 0xffffffffffffffff, 0x7fff9a5faff8, 0xa, ...)
/home/jewome62/go/src/github.com/nilsmagnus/grib/griblib/filters.go:41 +0x47d
main.parse(0x7f65fa73d770, 0xc820090078, 0x5a1dc8, 0x5, 0xffffffffffffffff, 0x1, 0xffffffffffffffff, 0x7fff9a5faff8, 0xa, 0x5a8cc0, ...)
/home/jewome62/go/src/github.com/nilsmagnus/grib/main.go:121 +0x1e4
main.main()
/home/jewome62/go/src/github.com/nilsmagnus/grib/main.go:72 +0x490
Can you please create a new release, so that the latest version will be taken into account when using dep (https://golang.github.io/dep/docs/daily-dep.html) ?
Thanks
data3 is not parsed correctly
on the second message, data should be:
-1.09462
-1.03462
-0.964617
-0.904617
-0.834617
-0.774617
-0.714617
-0.644617
-0.574617
-0.514617
-0.444617
But I get:
-1.094617
-1.034617
14.255383
14.235383
14.255383
14.235383
14.245383
14.255383
14.245383
14.235383
14.255383
I'm attempting to decode the attached grib file, which is grid0 with an Ni
= 1440 and Nj
= 721. I've dealt with similar grib files using griblib in the past, but I'm seeing this particular file return a single message with a Section7.Data
length of 468056 where I would typically expect 1038240 (Ni * Nj
).
I've opened the grib file with an independent viewer and the data is listed there as expected (Grid: 1440x721=1038240 points
). I've also confirmed that the Section5.DataTemplateNumber
is 3, which I would expect.
Is this a decoding issue, or some grib peculiarity that I'm missing?
Here is my script to get wind data from NOAA
#!/bin/bash
DATE=`date +%Y%m%d`
echo "GET ${DATE}"
curl "http://nomads.ncep.noaa.gov/cgi-bin/filter_gfs_1p00.pl?file=gfs.t00z.pgrb2.1p00.f000&lev_10_m_above_ground=on&var_UGRD=on&var_VGRD=on&dir=%2Fgfs.${DATE}00" -o gfs.t00z.pgrb2.1p00.f000
With this command I get integers instead of floats
./grib -file gfs.t00z.pgrb2.1p00.f000 -latMin -90000000 -latMax 90000000 -longMin 0 -longMax 359000000 -export 3
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.