rwcarlsen / goexif Goto Github PK
View Code? Open in Web Editor NEWDecode embedded EXIF meta data from image files.
License: BSD 2-Clause "Simplified" License
Decode embedded EXIF meta data from image files.
License: BSD 2-Clause "Simplified" License
I'm trying to track down how to reproduce this error, but:
panic: division by zero
goroutine 14 [running]:
panic(0x2d48a0, 0xc42120e7a0)
/usr/local/go/src/runtime/panic.go:500 +0x1a1
math/big.(*Rat).SetFrac64(0xc420d99740, 0x0, 0x0, 0x13)
/usr/local/go/src/math/big/rat.go:311 +0x147
math/big.NewRat(0x0, 0x0, 0x0)
/usr/local/go/src/math/big/rat.go:25 +0x4f
github.com/rwcarlsen/goexif/tiff.(*Tag).Rat(0xc4207d4000, 0x0, 0x34b1b7, 0xb, 0xc420253dd8)
/Users/matt/Dev/src/github.com/rwcarlsen/goexif/tiff/tag.go:329 +0x7a
So this line must mean that d
is 0.
My calling code:
// altitude
rawAlt, err := x.Get(exif.GPSAltitude)
if err != nil {
return nil, fmt.Errorf("getting altitude from EXIF: %v", err)
}
alt, err := rawAlt.Rat(0) // *boom*
if err != nil {
return nil, fmt.Errorf("converting altitude value: %v", err)
}
altFlt, _ := alt.Float64()
file, err := os.Open("/tmp/rotation.jpeg")
assert.Nil(t, err)
defer file.Close()
_, err = exif.Decode(file)
assert.Nil(t, err)
I use goexif to extract and parse the maker notes for apple device created jpegs. I had PR #62 opened last year. That PR is broken now so I closed it.
It seems extracting exif.MakerNote
is broken in general
I wrote this little testing program
package main
import (
"flag"
"fmt"
"os"
"github.com/rwcarlsen/goexif/exif"
)
func main() {
flag.Parse()
filename := flag.Arg(0)
f, err := os.Open(filename)
if err != nil {
fmt.Println("File error: ", err.Error())
return
}
x, err := exif.Decode(f)
if err != nil {
fmt.Println("Decode error: ", err.Error())
return
}
m, err := x.Get(exif.MakerNote)
if err != nil {
fmt.Println("Error: ", err.Error())
} else {
fmt.Printf("OK. Extracted %d bytes\n\n%x\n", len(m.Val), m.Val)
}
}
With goexif@9b9b2b
$ go run ./test-extract.go testdata/apple_contentid.jpg
OK. Extracted 1000 bytes
4170706c6 ... (the rest of the data)
With goexif@76e334
$ go run ./test-extract.go testdata/apple_contentid.jpg
OK. Extracted 0 bytes
Here is the testing image ... the makenotes where copied into it from another file with exiftool.
call Int(0)
I have another crashing image.
When decoding the exif tags, we end up in an infinite loop decoding the following tags:
t.Type:1 valLen:164 t.Count:164 t.ValOffset:0
t.Type:20 valLen:0 t.Count:144 t.ValOffset:0
t.Type:1 valLen:164 t.Count:164 t.ValOffset:0
t.Type:20 valLen:0 t.Count:144 t.ValOffset:0
plz help
Could you think about supporting WebP if that is not already supported yet?
Problem
All Nikon files from exif-samples/jpg/gps are failing with error: tiff: short read of tag value
Would be nice to know what the exact tag is, because on Mac Inspector everything seems fine.
Reproduction
Test file: DSCN0010.jpg
Result Expected
To continue getting the valid fields, like here:
I've started running https://github.com/dvyukov/go-fuzz against goexif. One panic I've seen is:
panic: runtime error: index out of range
goroutine 1 [running]:
github.com/rwcarlsen/goexif/exif.(*parser).Parse(0x2b3f60, 0x20832e510, 0x0, 0x0)
/var/folders/69/cstplpp51jz6f9_5m_m__3gh0000gq/T/go-fuzz-build329778221/src/github.com/rwcarlsen/goexif/exif/exif.go:147 +0x77a
github.com/rwcarlsen/goexif/exif.Decode(0x2208307a30, 0x208312080, 0x2b3d58, 0x0, 0x0)
/var/folders/69/cstplpp51jz6f9_5m_m__3gh0000gq/T/go-fuzz-build329778221/src/github.com/rwcarlsen/goexif/exif/exif.go:287 +0xec4
github.com/joeshaw/goexif-fuzz.Fuzz(0x22084b6000, 0x1a45, 0x200000, 0x1)
/var/folders/69/cstplpp51jz6f9_5m_m__3gh0000gq/T/go-fuzz-build329778221/src/github.com/joeshaw/goexif-fuzz/fuzz.go:10 +0x133
github.com/dvyukov/go-fuzz/go-fuzz-dep.Main(0x21c378)
/Users/joeshaw/src/gosrc/src/github.com/dvyukov/go-fuzz/go-fuzz-dep/main.go:44 +0x13d
main.main()
/var/folders/69/cstplpp51jz6f9_5m_m__3gh0000gq/T/go-fuzz-build329778221/src/go-fuzz-main/main.go:10 +0x2a
This is because x.Tiff.Dirs
has length 0. The exact data fed in is:
"II*\x00\x00\x00\x00\x00"
I'm not sure about what the right level to fix this is. I see a few options:
tiff.Tiff
with len(t.Dirs) == 0
? If not, an error should probably be thrown from tiff.Decode()
.exif.parser.Parse()
should check len(x.Tiff.Dirs)
before accessing the first element and error out.Detailed Panic Stack Trace -
goroutine 5 [running]:
runtime/debug.Stack(0xc42004f208, 0xb71060, 0x1222960)
/usr/local/go/src/runtime/debug/stack.go:24 +0xa7
Upload-Image-API/services/controllers/uploadimage.UploadImages.func1()
/home3/indiamart/public_html/go/src/Upload-Image-API/services/controllers/uploadimage/uploadimage.go:28 +0x48
panic(0xb71060, 0x1222960)
/usr/local/go/src/runtime/panic.go:491 +0x283
github.com/rwcarlsen/goexif/mknote.(*nikonV3).Parse(0x1289980, 0xc4201adb90, 0x0, 0x0)
/home3/indiamart/public_html/go/src/github.com/rwcarlsen/goexif/mknote/mknote.go:58 +0x2f4
github.com/rwcarlsen/goexif/exif.Decode(0x1232ae0, 0xc42000e1a0, 0x2, 0x0, 0x0)
/home3/indiamart/public_html/go/src/github.com/rwcarlsen/goexif/exif/exif.go:301 +0x666
Upload-Image-API/services/utils.GetMetaDataOfImage(0xc420076600, 0x77, 0xb26720, 0xc420019310, 0xc420151888)
/home3/indiamart/public_html/go/src/Upload-Image-API/services/utils/utils.go:204 +0x1ed
Upload-Image-API/services/controllers/uploadimage.UploadImages(0x12397e0, 0xc42015c000, 0xc4200fc500)
/home3/indiamart/public_html/go/src/Upload-Image-API/services/controllers/uploadimage/uploadimage.go:170 +0x1c33
net/http.HandlerFunc.ServeHTTP(0xc518b8, 0x12397e0, 0xc42015c000, 0xc4200fc500)
/usr/local/go/src/net/http/server.go:1918 +0x44
github.com/gorilla/mux.(*Router).ServeHTTP(0xc4201c1f80, 0x12397e0, 0xc42015c000, 0xc4200fc500)
/home3/indiamart/public_html/go/src/github.com/gorilla/mux/mux.go:162 +0xed
github.com/gorilla/handlers.(*cors).ServeHTTP(0xc4201a9830, 0x12397e0, 0xc42015c000, 0xc4200fc300)
/home3/indiamart/public_html/go/src/github.com/gorilla/handlers/cors.go:52 +0xa77
net/http.serverHandler.ServeHTTP(0xc4201ae4e0, 0x12397e0, 0xc42015c000, 0xc4200fc300)
/usr/local/go/src/net/http/server.go:2619 +0xb4
net/http.(*conn).serve(0xc420158000, 0x1239e20, 0xc42005aac0)
/usr/local/go/src/net/http/server.go:1801 +0x71d
created by net/http.(*Server).Serve
/usr/local/go/src/net/http/server.go:2720 +0x288
Hello!
In the
switch tag.Format() {
case tiff.RatVal:
the set tag.format struct value doesn't seem to be correct. (exif.go L: 482).
My images, which contains valid EXIF data, is not being seen as RatVal or StringVal, so the default case catches it, and returns an error. If i insert the logic from the tiff.RatVal case, the output of LatLng is correct.
I'm not one to make suggestions, but i'm guessing there's a minor bug here.
I can provide code/pictures if needed.
Thank you for a good lib!
Is it possible?
https://github.com/rwcarlsen/goexif/blob/go1/mknote/mknote.go#L58 when not len(m.Val) < 6.
@hullerob reported here: perkeep/perkeep#612 that newAppSec can panic with at least this .xcf file:
https://drive.google.com/file/d/0B5TLCARP-3XkQ1lUX1lpRHdRUkk
which I can confirm.
Fix incoming.
I took a movie with my iPhone SE and the tool fails to decode EXIF data with the error mentioned above.
Hello,
Sorry to open a ticket for this question but I am lost.
I am reading exif specification to calculate FStop but I am wrong...
aperture, _ := x.Get(exif.ApertureValue)
numer, denom, _ = aperture.Rat2(0)
fmt.Printf("\n%v", ...)
No way to calculate it. Do you have any idea to convert A / B to a readable F-STOP ?
Thanks a lot !
Best regards,
The current way of encoding Undefined
is to try to print any utf-8 rune. I believe it would make sense to serialize Undefined
as []byte through json.Marshal and let it encode it as base64. That way you could also remove the nullString
function if #36 is merged.
We're seeing OOMs quite frequently in production, but haven't yet isolated a root cause. Was wondering if you had any thoughts, since it always originates in the exif code:
Images are always < 5MB on disk and JPG or JPEG.
Thanks in advance.
runtime.throw(0xbd8337)
/home/ec2-user/infra/packages/internal/go/go/src/pkg/runtime/panic.c:464 +0x69 fp=0x7f9ca16dde88
runtime.SysMap(0xc51fc40000, 0x82700000, 0xbeadd8)
/home/ec2-user/infra/packages/internal/go/go/src/pkg/runtime/mem_linux.c:131 +0xfe fp=0x7f9ca16ddeb8
runtime.MHeap_SysAlloc(0xbf4d20, 0x82700000)
/home/ec2-user/infra/packages/internal/go/go/src/pkg/runtime/malloc.goc:473 +0x10a fp=0x7f9ca16ddef8
MHeap_Grow(0xbf4d20, 0x82700)
/home/ec2-user/infra/packages/internal/go/go/src/pkg/runtime/mheap.c:241 +0x5d fp=0x7f9ca16ddf38
MHeap_AllocLocked(0xbf4d20, 0x826fe, 0x0)
/home/ec2-user/infra/packages/internal/go/go/src/pkg/runtime/mheap.c:126 +0x305 fp=0x7f9ca16ddf78
runtime.MHeap_Alloc(0xbf4d20, 0x826fe, 0x100000000, 0x1)
/home/ec2-user/infra/packages/internal/go/go/src/pkg/runtime/mheap.c:95 +0x7b fp=0x7f9ca16ddfa0
runtime.mallocgc(0x826fd370, 0x74fd61, 0x0)
/home/ec2-user/infra/packages/internal/go/go/src/pkg/runtime/malloc.goc:89 +0x484 fp=0x7f9ca16de010
cnew(0x74fd60, 0x104dfa6e, 0xc200000001)
/home/ec2-user/infra/packages/internal/go/go/src/pkg/runtime/malloc.goc:718 +0xc1 fp=0x7f9ca16de030
runtime.cnewarray(0x74fd60, 0x104dfa6e)
/home/ec2-user/infra/packages/internal/go/go/src/pkg/runtime/malloc.goc:731 +0x3a fp=0x7f9ca16de050
makeslice1(0x6c3c40, 0xd0b2ebf, 0x104dfa6e, 0x7f9ca16de108)
/home/ec2-user/infra/packages/internal/go/go/src/pkg/runtime/slice.c:57 +0x4d fp=0x7f9ca16de068
growslice1(0x6c3c40, 0xc4995a0000, 0xd0b2ebf, 0xd0b2ebf, 0xd0b2ec0, ...)
/home/ec2-user/infra/packages/internal/go/go/src/pkg/runtime/slice.c:113 +0x58 fp=0x7f9ca16de098
runtime.growslice(0x6c3c40, 0xc4995a0000, 0xd0b2ebf, 0xd0b2ebf, 0x1, ...)
/home/ec2-user/infra/packages/internal/go/go/src/pkg/runtime/slice.c:80 +0x9d fp=0x7f9ca16de0e0
github.com/rwcarlsen/goexif/tiff.Decode(0x7f9cabfe1268, 0xc2108aef30, 0x0, 0x0, 0x0)
/media/ephemeral0/var-local/posadero/jenkins-workspace/GoMiro_1_Compile/src/github.com/rwcarlsen/goexif/tiff/tiff.go:87 +0x806 fp=0x7f9ca16de210
github.com/rwcarlsen/goexif/exif.Decode(0x7f9cac000a30, 0xc2115a5280, 0x892000, 0x7f9cabff89e8, 0xc21e867d20)
/media/ephemeral0/var-local/posadero/jenkins-workspace/GoMiro_1_Compile/src/github.com/rwcarlsen/goexif/exif/exif.go:163 +0xb59 fp=0x7f9ca16de410
<snip>
This is so that when using a dependency manager, we can lock to a version and get predictable builds
Otherwise, we're sticking to a commit sha or always pulling master.
For example, using glide, we can set it to only pull certain patch/minor versions based on our comfort.
I am trying to use goexif to grab and decode GPS location exif tags in
jpeg images. This doesn't seem to work, I always get 0 rather than the
actual values.
I think the problem is in the interaction of the DecodeTag() and
Rat2() logic in tiff.go. The GPSLatitude and GPSLongitude tags are
made up of 3 rational values. This puts valLen over 4 so DecodeTag
sets Val to the raw bytes rather than using PutUvarint. The Rat2
method on the other hand tries to interpret Val as a varint and
because the high byte of the GPS coordinates tends to be 0
binary.Uvarint stops processing when it encounters this initial 0 and
returns 0.
Attempting to extract GPS metadata from an image known to contain it causes the following error to be logged:
loading EXIF sub-IFD: exif: sub-IFD ExifIFDPointer decode failed: zero length tag value
Go version:
go version go1.12.6 linux/amd64
Test code (basically copy-paste of sample):
package main
import (
"fmt"
"log"
"os"
"github.com/rwcarlsen/goexif/exif"
"github.com/rwcarlsen/goexif/mknote"
)
func main() {
fname := "osaka.jpg"
f, err := os.Open(fname)
if err != nil {
log.Fatal(err)
}
// Optionally register camera makenote data parsing - currently Nikon and
// Canon are supported.
exif.RegisterParsers(mknote.All...)
x, err := exif.Decode(f)
if err != nil {
log.Fatal(err)
}
camModel, _ := x.Get(exif.Model) // normally, don't ignore errors!
fmt.Println(camModel.StringVal())
focal, _ := x.Get(exif.FocalLength)
numer, denom, _ := focal.Rat2(0) // retrieve first (only) rat. value
fmt.Printf("%v/%v", numer, denom)
// Two convenience functions exist for date/time taken and GPS coords:
tm, _ := x.DateTime()
fmt.Println("Taken: ", tm)
lat, long, _ := x.LatLong()
fmt.Println("lat, long: ", lat, ", ", long)
}
Image file: https://github.com/FooSoft/goldsmith-components/raw/master/plugins/exif/testdata/source/osaka.jpg
In loadSubDir the sub IFD tag's have overlap and result in overwrites in the exif struct's global map[tagid]exifval map.
Based on apple's reply, https://forums.developer.apple.com/thread/89759, seems like HEIF stores EXIF into mp4 box/atom container. Just wonder if you have any plan to support it?
Given
57 15' 23.31" N as latitude, converted to decimal should output 57.256475 not 57.256474
only when i converted back to GPS coordinates the problem becomes clear:
57.256475 == 57 15' 23.31"
57.256474 == 57 15' 23.3"
(converted using http://www.onlineconversion.com/map_decimaldegrees.htm)
seems you are taking in account only the first digit after the .
Same issue on Longitude.
Can you please fix it, to have the same output as exiftool?
Get runtime panic out of memory
when I try to extract EXIF metadata from a large, empty file.
Given a file generated with this:
dd if=/dev/zero of=output.dat bs=5000000 count=1000
Obviously I don't expect this to work, but I don't expect it to consume all system resources either.
10.10.5 (14F27)
Hi there,
I would like to contribute to the project but I am not sure if the PR would even be read. There are a lot of PRs open (some more than three years old).
Is the project still maintained or should I go for the goexif2 forks (despite the fact that the original one of these is looking for a new maintainer)?
If this project is still maintained: Why are the PRs not being merged or closed?
Best Regards
Thanks for making such a useful library. seems like currently it mainly supports JPG. Just wonder if you have any plan to support PNG. such as https://superuser.com/questions/219642/what-software-can-i-use-to-read-png-metadata
goexif throws a panic for the attached image. Despite the GPS tags are malformed, the package should recover itself and simply return an error.
Quote from https://code.google.com/p/go-wiki/wiki/PanicAndRecover#Usage_in_a_Package:
"By convention, no explicit panic() should be allowed to cross a package boundary. Indicating error conditions to callers should be done by returning error value."
Quote from http://blog.golang.org/defer-panic-and-recover:
"The convention in the Go libraries is that even when a package uses panic internally, its external API still presents explicit error return values."
The relevant panic log:
panic: Tag format is not 'rational'
goroutine 495 [running]:
runtime.panic(0x9c5460, 0x1f132958)
c:/go/src/pkg/runtime/panic.c:279 +0xe9
camlistore.org/third_party/github.com/camlistore/goexif/tiff.(*Tag).Rat2(0x1fa9e9b0, 0x0, 0x0, 0x743a66, 0x9c0de0, 0x1fa17440)
c:/Users/dsde012/go/src/camlistore.org/tmp/build-gopath- nosqlite/src/camlistore.org/third_party/github.com/camlistore/goexif/tiff/tag.go:238 +0x6f
camlistore.org/third_party/github.com/camlistore/goexif/exif.tagDegrees(0x1fa9e9b0, 0xb6c538, 0xe)
c:/Users/dsde012/go/src/camlistore.org/tmp/build-gopath-nosqlite/src/camlistore.org/third_party/github.com/camlistore/goexif/exif/exif.go:232 +0x3e
camlistore.org/third_party/github.com/camlistore/goexif/exif.(*Exif).LatLong(0x1f132930, 0x0, 0x0, 0x0, 0x0, 0x0)
c:/Users/dsde012/go/src/camlistore.org/tmp/build-gopath-nosqlite/src/camlistore.org/third_party/github.com/camlistore/goexif/exif/exif.go:254 +0x147
I was searching for a function which can give you the image in orientation 1. This is needed when you deal with images which has been created with smartphones, since those are mostly shown with wrong orientation.
Since your solution only reads exif it was not able to correct the exif data after transfer, but since the copy has no exif at all the images are now shown correctly.
Maybe you would like to integrate this function somehow? Its quite nice, since go does not provide support for orientation out of the box. Here is the solution:
https://github.com/Macilias/go-images-orientation
This might relate to issue #72 I getting a a slice bounds out of range when trying to decode exif
This was working about 2 weeks ago but failed when I updated today.
For reference here one of the images that caused the error https://drive.google.com/file/d/1FfKz6CqIS-3XEMeulHyj1uhLmGZiQWNJ/view?usp=sharing
github.com/robrotheram/gogallery/vendor/github.com/rwcarlsen/goexif/mknote.(*nikonV3).Parse(0x107f000, 0xc00e4024c0, 0x0, 0x0)
/home/travis/gopath/src/github.com/robrotheram/gogallery/vendor/github.com/rwcarlsen/goexif/mknote/mknote.go:57 +0x2b0
github.com/robrotheram/gogallery/vendor/github.com/rwcarlsen/goexif/exif.LazyDecode(0xba1ce0, 0xc00e400570, 0xc00dacc470, 0x10513c0, 0x20)
/home/travis/gopath/src/github.com/robrotheram/gogallery/vendor/github.com/rwcarlsen/goexif/exif/exif.go:297 +0x255
github.com/robrotheram/gogallery/vendor/github.com/rwcarlsen/goexif/exif.Decode(0xba23c0, 0xc00e408088, 0x2, 0x19, 0x10513c0)
/home/travis/gopath/src/github.com/robrotheram/gogallery/vendor/github.com/rwcarlsen/goexif/exif/exif.go:222 +0x6e
github.com/robrotheram/gogallery/datastore.(*Picture).CreateExif(0xc00e3fedd0)
/home/travis/gopath/src/github.com/robrotheram/gogallery/datastore/pictures.go:106 +0x182
github.com/robrotheram/gogallery/datastore.ScanPath.func1(0xc00e39b830, 0x22, 0xbab740, 0xc00e3fed00, 0x0, 0x0, 0xc, 0xc00ad53cc8)
/home/travis/gopath/src/github.com/robrotheram/gogallery/datastore/scan.go:83 +0x4c1
path/filepath.walk(0xc00e39b830, 0x22, 0xbab740, 0xc00e3fed00, 0xb1a680, 0x0, 0x0)
/home/travis/.gimme/versions/go1.11.6.linux.amd64/src/path/filepath/path.go:358 +0x41c
path/filepath.walk(0xc00e404420, 0x15, 0xbab740, 0xc00e3feb60, 0xb1a680, 0x0, 0x0)
/home/travis/.gimme/versions/go1.11.6.linux.amd64/src/path/filepath/path.go:382 +0x2fe
path/filepath.walk(0xc000024017, 0xd, 0xbab740, 0xc00013c0d0, 0xb1a680, 0x0, 0xc005531f18)
/home/travis/.gimme/versions/go1.11.6.linux.amd64/src/path/filepath/path.go:382 +0x2fe
path/filepath.Walk(0xc000024017, 0xd, 0xb1a680, 0xd, 0x0)
/home/travis/.gimme/versions/go1.11.6.linux.amd64/src/path/filepath/path.go:404 +0x105
github.com/robrotheram/gogallery/datastore.ScanPath(0xc000024017, 0xd, 0x0, 0x0, 0x0)
/home/travis/gopath/src/github.com/robrotheram/gogallery/datastore/scan.go:105 +0x160
main.main.func1()
/home/travis/gopath/src/github.com/robrotheram/gogallery/main.go:59 +0x40
created by main.main
/home/travis/gopath/src/github.com/robrotheram/gogallery/main.go:58 +0x2d5
Get error panic: EOF from decode
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.