F-spot crashes on import Fotos with too many files open about f-spot HOT 11 CLOSED

@dkoeb can you take a look at this?

from f-spot.

That's quite likely a bug I introduced with the latest pull request :-(
@codingdave if you need a working build for now, go back to e1265e2
I'll try to fix it within the next days.

from f-spot.

@codingdave I just did a quick test with a quite large list of files (tens of thousands) but could not reproduce the exception. Could you share some more details on your environment? (e.g, number of files in the folder you tried to import, file types)

from f-spot.

@dkoeb sure.
git describe
0.8.0-644-ga6b4d190

uname -a
Linux UX32VD 4.8.0-40-generic #43-Ubuntu SMP Thu Feb 23 16:01:19 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

find ~/Fotos/ -type f -iname "*.JPG" -exec ls {} ; | wc -l
17757

find ~/Fotos/ -type f -not -iname "*.JPG" -exec ls {} ; | wc -l
3

find ~/Fotos/ -type d | wc -l
1504

The is reproducible with every try on my side. What else can I provide?

from f-spot.

I reverted to the revision mentioned above. It also crashes. Interestingly it seems that f-spot also has trouble with my exif data and at some point starts to create xmp files for all following jpgs. Maybe this is related?

david@UX32VD:~/w/f-spot.git|stable⚡*?
➤ ~/unstable/f-spot/bin/f-spot
[Info 20:38:29.379] Initializing Mono.Addins
Cairo.Surface is leaking, programmer is missing a call to Dispose
Set MONO_CAIRO_DEBUG_DISPOSE to track allocation traces
Cairo.Surface is leaking, programmer is missing a call to Dispose
Set MONO_CAIRO_DEBUG_DISPOSE to track allocation traces
Cairo.Surface is leaking, programmer is missing a call to Dispose
Set MONO_CAIRO_DEBUG_DISPOSE to track allocation traces
Cairo.Surface is leaking, programmer is missing a call to Dispose
Set MONO_CAIRO_DEBUG_DISPOSE to track allocation traces

(f-spot:23877): GdkPixbuf-WARNING **: GdkPixbufLoader finalized without calling gdk_pixbuf_loader_close() - this is not allowed. You must explicitly end the data stream to the loader before dropping the last reference.

** (f-spot:23877): WARNING **: Invalid borders specified for theme pixmap:
/usr/share/themes/Breeze/gtk-2.0/../assets/line-h.png,
borders don't fit within the image
[Warn 20:38:36.026] Metadata of file file:///home/david/Fotos/2004/01/01/STA60133.JPG may be corrupt, refusing to write to it, falling back to XMP sidecar.
[Warn 20:38:36.546] Metadata of file file:///home/david/Fotos/2004/01/03/STA60356.JPG may be corrupt, refusing to write to it, falling back to XMP sidecar.
[Warn 20:38:38.122] Metadata of file file:///home/david/Fotos/2007/09/26/DSC_0127.JPG may be corrupt, refusing to write to it, falling back to XMP sidecar.
[Warn 20:38:38.644] Metadata of file file:///home/david/Fotos/2007/09/26/DSC_0126.JPG may be corrupt, refusing to write to it, falling back to XMP sidecar.
[Warn 20:38:39.160] Metadata of file file:///home/david/Fotos/2007/09/26/DSC_0125.JPG may be corrupt, refusing to write to it, falling back to XMP sidecar.
[Warn 20:38:39.676] Metadata of file file:///home/david/Fotos/2007/09/26/DSC_0124.JPG may be corrupt, refusing to write to it, falling back to XMP sidecar.
[Warn 20:38:40.199] Metadata of file file:///home/david/Fotos/2007/09/26/DSC_0123.JPG may be corrupt, refusing to write to it, falling back to XMP sidecar.
[Warn 20:38:40.717] Metadata of file file:///home/david/Fotos/2007/09/26/DSC_0122.JPG may be corrupt, refusing to write to it, falling back to XMP sidecar.
[Warn 20:38:41.236] Metadata of file file:///home/david/Fotos/2007/09/26/DSC_0121.JPG may be corrupt, refusing to write to it, falling back to XMP sidecar.
[Warn 20:38:41.754] Metadata of file file:///home/david/Fotos/2007/09/26/DSC_0120.JPG may be corrupt, refusing to write to it, falling back to XMP sidecar.
[Warn 20:38:42.266] Metadata of file file:///home/david/Fotos/2007/09/26/DSC_0119.JPG may be corrupt, refusing to write to it, falling back to XMP sidecar.
[Warn 20:38:42.778] Metadata of file file:///home/david/Fotos/2007/09/26/DSC_0118.JPG may be corrupt, refusing to write to it, falling back to XMP sidecar.
[Warn 20:38:43.290] Metadata of file file:///home/david/Fotos/2007/09/26/DSC_0117.JPG may be corrupt, refusing to write to it, falling back to XMP sidecar.
[Warn 20:38:43.803] Metadata of file file:///home/david/Fotos/2007/09/26/DSC_0116.JPG may be corrupt, refusing to write to it, falling back to XMP sidecar.
[Warn 20:38:44.320] Metadata of file file:///home/david/Fotos/2007/09/26/DSC_0115.JPG may be corrupt, refusing to write to it, falling back to XMP sidecar.
[Warn 20:38:44.838] Metadata of file file:///home/david/Fotos/2007/09/26/DSC_0113.JPG may be corrupt, refusing to write to it, falling back to XMP sidecar.
[Warn 20:38:45.351] Metadata of file file:///home/david/Fotos/2007/09/26/DSC_0112.JPG may be corrupt, refusing to write to it, falling back to XMP sidecar.
[Warn 20:38:45.879] Metadata of file file:///home/david/Fotos/2007/09/26/DSC_0111.JPG may be corrupt, refusing to write to it, falling back to XMP sidecar.
[Warn 20:38:46.392] Metadata of file file:///home/david/Fotos/2007/09/26/DSC_0109.JPG may be corrupt, refusing to write to it, falling back to XMP sidecar.
[Warn 20:38:46.904] Metadata of file file:///home/david/Fotos/2007/09/26/DSC_0108.JPG may be corrupt, refusing to write to it, falling back to XMP sidecar.
[Warn 20:38:47.418] Metadata of file file:///home/david/Fotos/2007/09/26/DSC_0107.JPG may be corrupt, refusing to write to it, falling back to XMP sidecar.
[Warn 20:38:47.740] Caught an exception - Hyena.Data.Sqlite.SqliteException: Sqlite error 14: unable to open database file (SQL: INSERT INTO rolls (time) VALUES (1488393527)) (in `Hyena.Data.Sqlite')
at Hyena.Data.Sqlite.Connection.CheckError (System.Int32 errorCode, System.String sql) [0x00042] in :0
at Hyena.Data.Sqlite.Connection.Execute (System.String sql) [0x00021] in :0
at Hyena.Data.Sqlite.HyenaSqliteCommand.Execute (Hyena.Data.Sqlite.HyenaSqliteConnection hconnection, Hyena.Data.Sqlite.Connection connection) [0x000a8] in :0

Unhandled Exception:
Hyena.Data.Sqlite.SqliteException: Sqlite error 14: unable to open database file (SQL: INSERT INTO rolls (time) VALUES (1488393527))
at Hyena.Data.Sqlite.HyenaSqliteCommand.WaitForResult (Hyena.Data.Sqlite.HyenaSqliteConnection conn) [0x00049] in :0
at Hyena.Data.Sqlite.HyenaSqliteConnection.Execute (Hyena.Data.Sqlite.HyenaSqliteCommand command) [0x0000e] in :0
at FSpot.Database.RollStore.Create (System.DateTime time_in_utc) [0x00027] in <36ec76427fb34676bb6062e735cd2151>:0
at FSpot.Database.RollStore.Create () [0x00007] in <36ec76427fb34676bb6062e735cd2151>:0
at FSpot.Import.ImportController.DoImport (FSpot.Database.IDb db, FSpot.Core.IBrowsableCollection photos, System.Collections.Generic.IList1[T] tagsToAttach, System.Boolean duplicateDetect, System.Boolean copyFiles, System.Boolean removeOriginals, System.Action2[T1,T2] reportProgress, System.Threading.CancellationToken token) [0x00047] in <0bae3e55f1fe4a58babeb31bb9c1aa48>:0
at FSpot.Import.ImportDialogController.DoImport (System.Threading.CancellationToken token) [0x0005e] in <4736f7bb28f0409f81cfbe58844a260e>:0
at FSpot.Import.ImportDialogController.m__0 () [0x0000c] in <4736f7bb28f0409f81cfbe58844a260e>:0
at System.Threading.ThreadHelper.ThreadStart_Context (System.Object state) [0x00017] in :0
at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) [0x0008d] in :0
at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) [0x00000] in :0
at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state) [0x00031] in :0
at System.Threading.ThreadHelper.ThreadStart () [0x0000b] in :0
[ERROR] FATAL UNHANDLED EXCEPTION: Hyena.Data.Sqlite.SqliteException: Sqlite error 14: unable to open database file (SQL: INSERT INTO rolls (time) VALUES (1488393527))
at Hyena.Data.Sqlite.HyenaSqliteCommand.WaitForResult (Hyena.Data.Sqlite.HyenaSqliteConnection conn) [0x00049] in :0
at Hyena.Data.Sqlite.HyenaSqliteConnection.Execute (Hyena.Data.Sqlite.HyenaSqliteCommand command) [0x0000e] in :0
at FSpot.Database.RollStore.Create (System.DateTime time_in_utc) [0x00027] in <36ec76427fb34676bb6062e735cd2151>:0
at FSpot.Database.RollStore.Create () [0x00007] in <36ec76427fb34676bb6062e735cd2151>:0
at FSpot.Import.ImportController.DoImport (FSpot.Database.IDb db, FSpot.Core.IBrowsableCollection photos, System.Collections.Generic.IList1[T] tagsToAttach, System.Boolean duplicateDetect, System.Boolean copyFiles, System.Boolean removeOriginals, System.Action2[T1,T2] reportProgress, System.Threading.CancellationToken token) [0x00047] in <0bae3e55f1fe4a58babeb31bb9c1aa48>:0
at FSpot.Import.ImportDialogController.DoImport (System.Threading.CancellationToken token) [0x0005e] in <4736f7bb28f0409f81cfbe58844a260e>:0
at FSpot.Import.ImportDialogController.m__0 () [0x0000c] in <4736f7bb28f0409f81cfbe58844a260e>:0
at System.Threading.ThreadHelper.ThreadStart_Context (System.Object state) [0x00017] in :0
at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) [0x0008d] in :0
at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) [0x00000] in :0
at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state) [0x00031] in :0
at System.Threading.ThreadHelper.ThreadStart () [0x0000b] in :0

from f-spot.

well, this now is a different issue, an SQLite one. This also happened for all my tries (2 of 2).

from f-spot.

btw, the trouble with the exif info is also with the latest revision

from f-spot.

It's the number of folder, not the number of files that causes this issue.
@codingdave please, let me know if #82 fixes the issue on your system and I will merge it to master.

Your second observation is not related. Would you mind opening a separate issue?

from f-spot.

@dkoeb thanks for the fix. This really was the reason. I was able to import all my files without f-spot crashing:

david@UX32VD:~/w/f-spot.git|users/dkoeb/issue-81⚡*?
➤ ~/unstable/f-spot/bin/f-spot
[Info 23:40:46.399] Initializing Mono.Addins
Cairo.Surface is leaking, programmer is missing a call to Dispose
Set MONO_CAIRO_DEBUG_DISPOSE to track allocation traces
Cairo.Surface is leaking, programmer is missing a call to Dispose
Set MONO_CAIRO_DEBUG_DISPOSE to track allocation traces
Cairo.Surface is leaking, programmer is missing a call to Dispose
Set MONO_CAIRO_DEBUG_DISPOSE to track allocation traces
Cairo.Surface is leaking, programmer is missing a call to Dispose
Set MONO_CAIRO_DEBUG_DISPOSE to track allocation traces

(f-spot:20939): GdkPixbuf-WARNING **: GdkPixbufLoader finalized without calling gdk_pixbuf_loader_close() - this is not allowed. You must explicitly end the data stream to the loader before dropping the last reference.

** (f-spot:20939): WARNING **: Invalid borders specified for theme pixmap:
/usr/share/themes/Breeze/gtk-2.0/../assets/line-h.png,
borders don't fit within the image
[Warn 23:40:52.598] Metadata of file file:///home/david/Fotos/2003/10/31/03-11-08_Freitag_20.JPG may be corrupt, refusing to write to it, falling back to XMP sidecar.
[Warn 23:40:53.124] Metadata of file file:///home/david/Fotos/2004/01/01/STA60158.JPG may be corrupt, refusing to write to it, falling back to XMP sidecar.
...
hundreds of xmp issues of the same kind (I'll file a new case about it)
...
(f-spot:20939): Gdk-WARNING **: losing last reference to undestroyed window

[Info 00:03:38.243] Exiting...

(f-spot:20939): GLib-CRITICAL **: Source ID 73138 was not found when attempting to remove it

(f-spot:20939): GLib-CRITICAL **: Source ID 61772 was not found when attempting to remove it

(f-spot:20939): GLib-CRITICAL **: Source ID 52458 was not found when attempting to remove it

(f-spot:20939): GLib-CRITICAL *: Source ID 66891 was not found when attempting to remove it
david@UX32VD:~/w/f-spot.git|users/dkoeb/issue-81⚡?

from f-spot.

please merge to master ;-)

from f-spot.

merged e83d49c

from f-spot.