qutecoacoustics / audio-analysis Goto Github PK

View Code? Open in Web Editor NEW

52.0 22.0 12.0 72.22 MB

The audio analysis code (AnalysisPrograms.exe) for the QUT Ecoacoustics Research Group

Home Page: https://ap.qut.ecoacoustics.info/

License: Apache License 2.0

C# 95.20% F# 3.86% PowerShell 0.85% HTML 0.01% Dockerfile 0.03% Vim Snippet 0.06%

analysis acoustics tests recognizers indices csharp

audio-analysis's Introduction

audio-analysis

The source code for the QUT Ecoacoustics AnalysisPrograms.exe (AP.exe) program. Documentation (in progress) can be found at https://ap.qut.ecoacoustics.info/.

Quick links

Ask questions or start a discussion with us in the discussions 🙂
See the docs for instructions on
- Downloading AnalysisPrograms.exe
- Running the program
- Understanding concepts
See the Issues list for
- reporting bugs
- requesting new features
See the Contributing guidelines if you want to
- Compile the code yourself
- Make a contribution
Our documentation can be found at https://ap.qut.ecoacoustics.info/
- The source files are in the docs folder and instructions for editing them are in docs/README.md
Let us know if you're using AP.exe

Description

QUT Ecoacoustics Analysis Programs is a software package that can perform a suite of analyses on audio recordings of the environment. Although the analyses are intended for long-duration recordings (1 – 24 hours), in fact they can be performed on any audio file in a format supported by the software. Analysis Programs can:

calculate of summary and spectral acoustic indices at variable resolutions
produce long-duration, false-colour, multi-index spectrograms
calculate critical statistics of annotations downloaded from an Acoustic Workbench
run various acoustic event recognizers

All the analyses are performed by a single executable file, AnalysisPrograms.exe.

Citation

This citation should be used in all publications that use data, concepts, or results generated by AnalysisPrograms.exe or from this code base:

Michael Towsey, Anthony Truskinger, Mark Cottman-Fields, & Paul Roe. (2020, November 15). QutEcoacoustics/audio-analysis: Ecoacoustics Audio Analysis Software v20.11.2.0 (Version v20.11.2.0). Zenodo. http://doi.org/10.5281/zenodo.4274299

Additionally, depending on the analysis that was run, extra work may be required to be cited. Any such additional citations will printed in the console and in the log file.

Build status

Weekly versions of AnalysisPrograms.exe are now built automatically at midnight Monday night. You can get copies from the Releases page.

Per-commit (the very latest) builds can be found from the Actions page.

License

This project is very old. We've released the full history for the sake of maintainability and transparency. Unfortunately this means all code before our open source release is not open sourced.

In practice this should never be a problem. We never use our old code, except for historical purposes, and you never should need to either. If you do need to use any of this code, just raise an issue and we'll make an exeption for your use case.

Newer code - All commits after 2018-01-30

All code after a275d0bc5744ba43096b43de2ef2aee32dc14c18 (2018-01-30T06:38:46Z) are licensed under the Apache License 2.0

Older code - All commits before 2018-01-30

All commits before a275d0bc5744ba43096b43de2ef2aee32dc14c18 (2018-01-30T06:38:46Z) are not licensed under an open source license. All rights and copyright are retained, however, the public has permission to view, link to, and cite the code on GitHub.

audio-analysis's People

Contributors

Stargazers

Watchers

Forkers

stewartmacdonald mazenimadjaber gitter-badger ninascarpelli gwae1 tubbz-alt shyneforce jagakala andrew-1234 jjkenned ybayle meenautas

audio-analysis's Issues

Feedback-errors can happen due to sample rate of recordings

When running any application that needs to pick up specific sample rate (e.g. 22050, 16000 etc) for recordings, please remember to check whether the current value is the one you want. It gave Xueyan error and it may cause problem to you.

Currently, the sample rate for recordings from 'http://baw.ecosounds.org/' are all formatted to 44100 at the moment.

This is just a kind reminder from Xueyan.

Parallelisation bug mono/file system

For accessing the IndexProperties.yml there is a Concurrent dictionary that should allow parallel and coordinated reading and caching of the properties.

Works on windows, fails on mono.

System.IO.IOException: Sharing violation on path /home/ubuntu/bioacoustics/workers/analysis/AP/ConfigFiles/IndexPropertiesConfig.yml
at System.IO.FileStream..ctor (string,System.IO.FileMode,System.IO.FileAccess,System.IO.FileShare,int,bool,System.IO.FileOptions) <IL 0x00338, 0x00607>
at System.IO.FileStream..ctor (string,System.IO.FileMode,System.IO.FileAccess,System.IO.FileShare) <IL 0x0000d, 0x0004b>
at (wrapper remoting-invoke-with-check) System.IO.FileStream..ctor (string,System.IO.FileMode,System.IO.FileAccess,System.IO.FileShare) <IL 0x0001e, 0x00067>
at System.IO.FileInfo.Open (System.IO.FileMode,System.IO.FileAccess,System.IO.FileShare) <IL 0x00009, 0x00053>

Create spectrogram generator

Implement as IAnalyzer
Handle day long
Output CSV files as well
Question: return results at all?

Reduce the number of times the IndexProperties config is resolved in acoustic

Currently, when doing analyse long recordings, IndexProperties.Find is called 4 + 2n times, where n is the number of blocks of audio analyzed (n = originalRecording.Duration / IndexCalculationDuration)

See if we can reduce this overhead (or add shortcuts to Find function so less checking is done by default`)

Convert AED into IAnalyser2

Address #8 and #9 at the same time

Running AP.exe with command line parameters that include paths are not parsed properly

A differnt parameter style has to be used. The problem is in the PowerArgs library.

This

mono "/home/ubuntu/bioacoustics/workers/worker-dummy/AP/AnalysisPrograms.exe" audio2csv -source "/home/ubuntu/bioacoustics/workers/worker-dummy/_original_audio/5d/5d6280ce-0f64-4f0c-9628-c2c10d57d959_100815-1200.mp3" -config "/home/ubuntu/bioacoustics/workers/worker-dummy/AP/ConfigFiles/Towsey.Acoustic.yml" -tempdir "/home/ubuntu/bioacoustics/workers/worker-dummy/tmp/5d6280ce-0f64-4_2014_11_20t05_30_50z" -output "/home/ubuntu/bioacoustics/workers/worker-dummy/_cached_analysis_jobs/5d/5d6280ce-0f64-4f0c-9628-c2c10d57d959"

has to be changed to this

mono "/home/ubuntu/bioacoustics/workers/worker-dummy/AP/AnalysisPrograms.exe" audio2csv /source:"/home/ubuntu/bioacoustics/workers/worker-dummy/_original_audio/5d/5d6280ce-0f64-4f0c-9628-c2c10d57d959_100815-1200.mp3" /config:"/home/ubuntu/bioacoustics/workers/worker-dummy/AP/ConfigFiles/Towsey.Acoustic.yml" /tempdir:"/home/ubuntu/bioacoustics/workers/worker-dummy/tmp/5d6280ce-0f64-4_2014_11_20t05_30_50z" /output:"/home/ubuntu/bioacoustics/workers/worker-dummy/_cached_analysis_jobs/5d/5d6280ce-0f64-4f0c-9628-c2c10d57d959"

AED sometimes draw inaccurate boundary around events

Sometimes, the boundary of the events detected by AED is not accurate.

This case is more likely to happen when there are overlapped events.

One example is as follow:

As can be seen in the small events, there is blind space between hit pixels (transparent pink points) and the boundary of the events.

Remove all traces of the piddling custom CSV code

Will need to replace with methods that adequately reproduce the intended code.

embed current git hash into ap.exe meta data on build

Notes:

http://richarddingwall.name/2012/04/30/recording-the-git-commit-hash-in-your-assembly-with-teamcity/
http://stackoverflow.com/questions/15141338/embed-git-commit-hash-in-a-net-dll
- git rev-parse HEAD
http://blog.mshmelev.com/2014/01/embedding-git-commit-hash-to-assembly.html
git describe
Do not wan't a file committed for each change/build

SaveSonogramFiles switch in config not working

Images aren't produced even when switch is set to true

AP.exe log file is created with incorrect path separator

Logs created by AP.exe are a file with the name 'Logs\log.txt', when it should be 'Logs/log.txt'.

Subsegment calculation in AcousticIndices analyser may be wrong

Typically we expect 1 result per minute returned when ICD=60.0.

Inaccuracies in audio cutting and the floor operation currently used to calculate the required number of segments can cause this precondition to fail.

This problem is relatively easy to fix at ICD=60.0 - just Math.max(subsegments, 1)

However, at different scales, the subsegments calculated can be no longer viable for processing.

e.g. at ICD=0.2, for 60 seconds of audio, we expect 300 subsegments. The last subsegment (at 200ms) long can easily fall with the error margin for audio cutting inaccuracy. Even a partial overlap with the end segment can make the end segment no longer viable... less than 8 frames of spectrogram data is not enough to calculate statistics with.

Problem compounds as results are aggregated. Outputting matrices of spectral indices with inconsistent run-rates is really confusing. At all scales it means we could be missing data that is largely viable due to the current floor calculation (e.g. 20ms inaccuracy, at ICD=1.0, means we lose the whole second).

Proposed solution one:
Use perfect numbers to calculate subsegments. Always output the expected number of results from the analyzer per segment (e.g. ICD=0.2, we always output 300 results).
The issue is how do we pad the non-viable sections? Empty result set? Maybe an error flag on empty results object to mark explicit no data rows/tuples/arrays?
At most resolutions, there will be no unviable results and the summary statistics calculated will cover slightly less data than normal. At lower resolutions, if a susbsegment is required to work on less than 8 spectral frames of audio data, that is the only case that will produce an empty/error results...

Alternate solution:
Keep existing behaviour and special case the matrix dumping code to write the output CSVs at consistent run-rates.
How is this different from error results?
Different abstractions, no dummy data in program. Is it a better abstraction?
Simpler maybe?
Does not apply to all scales?
Maybe the minimum viable threshold applies in this case as well?
@cofiem were there other arguments of yours I missed?

Problems killing sox

2014-05-12T20:33:06.2045528+10:00 [1] INFO  LoggedConsole - QUT Bioacoustic Analysis Program - version 14.05.0.0 (RELEASE build, 6/05/2014 10:03) 
Copyright QUT 2014
2014-05-12T20:33:06.2565938+10:00 [1] INFO  LogFileOnly - Executable called with these arguments: 
"C:\Users\Administrator\Desktop\Sensors Analysis\ParallelExecutables\4\AnalysisPrograms.exe"  audio2csv -source "Y:\Owl Victoria\Mosaic17_Sensor23\OWL7_20120909_215104.wav" -config "C:\Users\Administrator\Desktop\Sensors Analysis\Towsey.Acoustic.cfg" -output "Y:\Results\2014May12-111458\Owl Victoria\Mosaic17_Sensor23\OWL7_20120909_215104.wav" -tempdir F:\4

2014-05-12T20:33:08.3716594+10:00 [1] INFO  LoggedConsole - # PROCESS LONG RECORDING
2014-05-12T20:33:08.3716594+10:00 [1] INFO  LoggedConsole - # DATE AND TIME: 12/05/2014 20:33:08
2014-05-12T20:33:08.3716594+10:00 [1] INFO  LoggedConsole - # Recording file:     OWL7_20120909_215104.wav
2014-05-12T20:33:08.3726615+10:00 [1] INFO  LoggedConsole - # Configuration file: C:\Users\Administrator\Desktop\Sensors Analysis\Towsey.Acoustic.cfg
2014-05-12T20:33:08.3726615+10:00 [1] INFO  LoggedConsole - # Output folder:      Y:\Results\2014May12-111458\Owl Victoria\Mosaic17_Sensor23\OWL7_20120909_215104.wav
2014-05-12T20:33:08.3726615+10:00 [1] INFO  LoggedConsole - # Temp File Directory:      F:\4
2014-05-12T20:33:08.4106925+10:00 [1] INFO  LoggedConsole - STARTING ANALYSIS ...
2014-05-12T20:36:36.7157697+10:00 [23] WARN  Acoustics.Tools.Audio.SoxAudioUtility - [2014-05-12T10:36:09 UTC] sox.exe with args  -q -V4 "Y:\Owl Victoria\Mosaic17_Sensor23\OWL7_20120909_215104.wav" "F:\4\Towsey.Acoustic\n0jd1e53.wav" trim 1380 60 rate -v -s -a 17640   running in F:\4\Towsey.Acoustic. Waited for 00:03:00. Process had already terminated after timeout.

Stdout:  not

Stderr: 
2014-05-12T20:36:37.0740612+10:00 [28] WARN  Acoustics.Tools.Audio.SoxAudioUtility - [2014-05-12T10:36:09 UTC] sox.exe with args  -q -V4 "Y:\Owl Victoria\Mosaic17_Sensor23\OWL7_20120909_215104.wav" "F:\4\Towsey.Acoustic\ok3upaln.wav" trim 1320 60 rate -v -s -a 17640   running in F:\4\Towsey.Acoustic. Waited for 00:03:00. Process had already terminated after timeout.

Stdout:  not

Stderr: 
2014-05-12T20:36:44.7120398+10:00 [27] WARN  Acoustics.Tools.Audio.SoxAudioUtility - [2014-05-12T10:36:26 UTC] sox.exe with args  -q -V4 "Y:\Owl Victoria\Mosaic17_Sensor23\OWL7_20120909_215104.wav" "F:\4\Towsey.Acoustic\q4j42oap.wav" trim 2340 60 rate -v -s -a 17640   running in F:\4\Towsey.Acoustic. Waited for 00:03:00. Process had already terminated after timeout.

Stdout:  not

Stderr: 
2014-05-12T20:37:02.4392825+10:00 [1] FATAL AnalysisPrograms.MainEntry - Unhandled exception ->
System.AggregateException: One or more errors occurred. ---> System.InvalidOperationException: Cannot kill the current process! tried 3 times. Process Name: C:\Users\Administrator\Desktop\Sensors Analysis\ParallelExecutables\4\audio-utils\sox\sox.exe. Arguments:  -q -V4 "Y:\Owl Victoria\Mosaic17_Sensor23\OWL7_20120909_215104.wav" "F:\4\Towsey.Acoustic\nlidpxjw.wav" trim 900 60 rate -v -s -a 17640  . ---> System.AggregateException: One or more errors occurred. ---> System.ComponentModel.Win32Exception: Access is denied
   at System.Diagnostics.Process.Kill()
   at Acoustics.Shared.ProcessRunner.KillProcess() in c:\Work\GitHub\audio-analysis\Acoustics\Acoustics.Shared\ProcessRunner.cs:line 201
   --- End of inner exception stack trace ---
   --- End of inner exception stack trace ---
   at Acoustics.Shared.ProcessRunner.KillProcess() in c:\Work\GitHub\audio-analysis\Acoustics\Acoustics.Shared\ProcessRunner.cs:line 214
   at Acoustics.Shared.ProcessRunner.PrepareRun(String arguments, String workingDirectory) in c:\Work\GitHub\audio-analysis\Acoustics\Acoustics.Shared\ProcessRunner.cs:line 152
   at Acoustics.Shared.ProcessRunner.RunTaskReaders(String arguments, String workingDirectory, Int32 retryCount) in c:\Work\GitHub\audio-analysis\Acoustics\Acoustics.Shared\ProcessRunner.cs:line 272
   at Acoustics.Shared.ProcessRunner.ProcessTimeout(String arguments, String workingDirectory, Int32 retryCount, Action`3 retryMethod) in c:\Work\GitHub\audio-analysis\Acoustics\Acoustics.Shared\ProcessRunner.cs:line 347
   at Acoustics.Shared.ProcessRunner.RunTaskReaders(String arguments, String workingDirectory, Int32 retryCount) in c:\Work\GitHub\audio-analysis\Acoustics\Acoustics.Shared\ProcessRunner.cs:line 288
   at Acoustics.Tools.Audio.AbstractUtility.RunExe(ProcessRunner processRunner, String arguments, String workingDirectory) in c:\Work\GitHub\audio-analysis\Acoustics\Acoustics.Tools\Audio\AbstractUtility.cs:line 368
   at Acoustics.Tools.Audio.AbstractAudioUtility.Modify(FileInfo source, String sourceMediaType, FileInfo output, String outputMediaType, AudioUtilityRequest request) in c:\Work\GitHub\audio-analysis\Acoustics\Acoustics.Tools\Audio\AbstractAudioUtility.cs:line 92
   at Acoustics.Tools.Audio.MasterAudioUtility.ConvertAndSegmentUsingSox(FileInfo source, String sourceMimeType, AudioUtilityRequest request) in c:\Work\GitHub\audio-analysis\Acoustics\Acoustics.Tools\Audio\MasterAudioUtility.cs:line 487
   at Acoustics.Tools.Audio.MasterAudioUtility.Modify(FileInfo source, String sourceMediaType, FileInfo output, String outputMediaType, AudioUtilityRequest request) in c:\Work\GitHub\audio-analysis\Acoustics\Acoustics.Tools\Audio\MasterAudioUtility.cs:line 255
   at Acoustics.Tools.AudioFilePreparer.PrepareFile(DirectoryInfo outputDirectory, FileInfo fiSource, String outputMediaType, AudioUtilityRequest request, DirectoryInfo temporaryFilesDirectory) in c:\Work\GitHub\audio-analysis\Acoustics\Acoustics.Tools\AudioFilePreparer.cs:line 62
   at AnalysisRunner.LocalSourcePreparer.PrepareFile(DirectoryInfo outputDirectory, FileInfo source, String outputMediaType, TimeSpan startOffset, TimeSpan endOffset, Int32 targetSampleRateHz, DirectoryInfo temporaryFilesDirectory) in c:\Work\GitHub\audio-analysis\AudioAnalysis\AnalysisRunner\LocalSourcePreparer.cs:line 266
   at AnalysisBase.AnalysisCoordinator.PrepareFileAndRunAnalysis(FileSegment fileSegment, IAnalyser analyser, AnalysisSettings settings) in c:\Work\GitHub\audio-analysis\AudioAnalysis\AnalysisBase\AnalysisCoordinator.cs:line 343
   at AnalysisBase.AnalysisCoordinator.<>c__DisplayClass8.<RunParallel>b__6(FileSegment item, ParallelLoopState state, Int64 index) in c:\Work\GitHub\audio-analysis\AudioAnalysis\AnalysisBase\AnalysisCoordinator.cs:line 223
   at System.Threading.Tasks.Parallel.<>c__DisplayClassf`1.<ForWorker>b__c()
   at System.Threading.Tasks.Task.InnerInvokeWithArg(Task childTask)
   at System.Threading.Tasks.Task.<>c__DisplayClass10.<ExecuteSelfReplicating>b__f(Object param0)
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
   at System.Threading.Tasks.Task.Wait()
   at System.Threading.Tasks.Parallel.ForWorker[TLocal](Int32 fromInclusive, Int32 toExclusive, ParallelOptions parallelOptions, Action`1 body, Action`2 bodyWithState, Func`4 bodyWithLocal, Func`1 localInit, Action`1 localFinally)
   at System.Threading.Tasks.Parallel.ForEachWorker[TSource,TLocal](IEnumerable`1 source, ParallelOptions parallelOptions, Action`1 body, Action`2 bodyWithState, Action`3 bodyWithStateAndIndex, Func`4 bodyWithStateAndLocal, Func`5 bodyWithEverything, Func`1 localInit, Action`1 localFinally)
   at System.Threading.Tasks.Parallel.ForEach[TSource](IEnumerable`1 source, ParallelOptions parallelOptions, Action`3 body)
   at AnalysisBase.AnalysisCoordinator.RunParallel(IEnumerable`1 analysisSegments, IAnalyser analysis, AnalysisSettings settings) in c:\Work\GitHub\audio-analysis\AudioAnalysis\AnalysisBase\AnalysisCoordinator.cs:line 239
   at AnalysisBase.AnalysisCoordinator.Run(IEnumerable`1 fileSegments, IAnalyser analysis, AnalysisSettings settings) in c:\Work\GitHub\audio-analysis\AudioAnalysis\AnalysisBase\AnalysisCoordinator.cs:line 164
   at AnalysisPrograms.AnalyseLongRecording.Execute(Arguments arguments) in c:\Work\GitHub\audio-analysis\AudioAnalysis\AnalysisPrograms\AnalyseLongRecording.cs:line 275
   at System.Dynamic.UpdateDelegates.UpdateAndExecuteVoid2[T0,T1](CallSite site, T0 arg0, T1 arg1)
   at PowerArgs.ArgAction`1.Invoke() in c:\Work\Software Dev\Github\PowerArgs\PowerArgs\ArgAction.cs:line 38
   at AnalysisPrograms.MainEntry.Main(String[] args) in c:\Work\GitHub\audio-analysis\AudioAnalysis\AnalysisPrograms\MainEntry.cs:line 66
---> (Inner Exception #0) System.InvalidOperationException: Cannot kill the current process! tried 3 times. Process Name: C:\Users\Administrator\Desktop\Sensors Analysis\ParallelExecutables\4\audio-utils\sox\sox.exe. Arguments:  -q -V4 "Y:\Owl Victoria\Mosaic17_Sensor23\OWL7_20120909_215104.wav" "F:\4\Towsey.Acoustic\nlidpxjw.wav" trim 900 60 rate -v -s -a 17640  . ---> System.AggregateException: One or more errors occurred. ---> System.ComponentModel.Win32Exception: Access is denied
   at System.Diagnostics.Process.Kill()
   at Acoustics.Shared.ProcessRunner.KillProcess() in c:\Work\GitHub\audio-analysis\Acoustics\Acoustics.Shared\ProcessRunner.cs:line 201
   --- End of inner exception stack trace ---
   --- End of inner exception stack trace ---
   at Acoustics.Shared.ProcessRunner.KillProcess() in c:\Work\GitHub\audio-analysis\Acoustics\Acoustics.Shared\ProcessRunner.cs:line 214
   at Acoustics.Shared.ProcessRunner.PrepareRun(String arguments, String workingDirectory) in c:\Work\GitHub\audio-analysis\Acoustics\Acoustics.Shared\ProcessRunner.cs:line 152
   at Acoustics.Shared.ProcessRunner.RunTaskReaders(String arguments, String workingDirectory, Int32 retryCount) in c:\Work\GitHub\audio-analysis\Acoustics\Acoustics.Shared\ProcessRunner.cs:line 272
   at Acoustics.Shared.ProcessRunner.ProcessTimeout(String arguments, String workingDirectory, Int32 retryCount, Action`3 retryMethod) in c:\Work\GitHub\audio-analysis\Acoustics\Acoustics.Shared\ProcessRunner.cs:line 347
   at Acoustics.Shared.ProcessRunner.RunTaskReaders(String arguments, String workingDirectory, Int32 retryCount) in c:\Work\GitHub\audio-analysis\Acoustics\Acoustics.Shared\ProcessRunner.cs:line 288
   at Acoustics.Tools.Audio.AbstractUtility.RunExe(ProcessRunner processRunner, String arguments, String workingDirectory) in c:\Work\GitHub\audio-analysis\Acoustics\Acoustics.Tools\Audio\AbstractUtility.cs:line 368
   at Acoustics.Tools.Audio.AbstractAudioUtility.Modify(FileInfo source, String sourceMediaType, FileInfo output, String outputMediaType, AudioUtilityRequest request) in c:\Work\GitHub\audio-analysis\Acoustics\Acoustics.Tools\Audio\AbstractAudioUtility.cs:line 92
   at Acoustics.Tools.Audio.MasterAudioUtility.ConvertAndSegmentUsingSox(FileInfo source, String sourceMimeType, AudioUtilityRequest request) in c:\Work\GitHub\audio-analysis\Acoustics\Acoustics.Tools\Audio\MasterAudioUtility.cs:line 487
   at Acoustics.Tools.Audio.MasterAudioUtility.Modify(FileInfo source, String sourceMediaType, FileInfo output, String outputMediaType, AudioUtilityRequest request) in c:\Work\GitHub\audio-analysis\Acoustics\Acoustics.Tools\Audio\MasterAudioUtility.cs:line 255
   at Acoustics.Tools.AudioFilePreparer.PrepareFile(DirectoryInfo outputDirectory, FileInfo fiSource, String outputMediaType, AudioUtilityRequest request, DirectoryInfo temporaryFilesDirectory) in c:\Work\GitHub\audio-analysis\Acoustics\Acoustics.Tools\AudioFilePreparer.cs:line 62
   at AnalysisRunner.LocalSourcePreparer.PrepareFile(DirectoryInfo outputDirectory, FileInfo source, String outputMediaType, TimeSpan startOffset, TimeSpan endOffset, Int32 targetSampleRateHz, DirectoryInfo temporaryFilesDirectory) in c:\Work\GitHub\audio-analysis\AudioAnalysis\AnalysisRunner\LocalSourcePreparer.cs:line 266
   at AnalysisBase.AnalysisCoordinator.PrepareFileAndRunAnalysis(FileSegment fileSegment, IAnalyser analyser, AnalysisSettings settings) in c:\Work\GitHub\audio-analysis\AudioAnalysis\AnalysisBase\AnalysisCoordinator.cs:line 343
   at AnalysisBase.AnalysisCoordinator.<>c__DisplayClass8.<RunParallel>b__6(FileSegment item, ParallelLoopState state, Int64 index) in c:\Work\GitHub\audio-analysis\AudioAnalysis\AnalysisBase\AnalysisCoordinator.cs:line 223
   at System.Threading.Tasks.Parallel.<>c__DisplayClassf`1.<ForWorker>b__c()
   at System.Threading.Tasks.Task.InnerInvokeWithArg(Task childTask)
   at System.Threading.Tasks.Task.<>c__DisplayClass10.<ExecuteSelfReplicating>b__f(Object param0)<---

2014-05-12T20:37:02.4963280+10:00 [1] FATAL AnalysisPrograms.MainEntry - 

==> Inner exception:
System.InvalidOperationException: Cannot kill the current process! tried 3 times. Process Name: C:\Users\Administrator\Desktop\Sensors Analysis\ParallelExecutables\4\audio-utils\sox\sox.exe. Arguments:  -q -V4 "Y:\Owl Victoria\Mosaic17_Sensor23\OWL7_20120909_215104.wav" "F:\4\Towsey.Acoustic\nlidpxjw.wav" trim 900 60 rate -v -s -a 17640  . ---> System.AggregateException: One or more errors occurred. ---> System.ComponentModel.Win32Exception: Access is denied
   at System.Diagnostics.Process.Kill()
   at Acoustics.Shared.ProcessRunner.KillProcess() in c:\Work\GitHub\audio-analysis\Acoustics\Acoustics.Shared\ProcessRunner.cs:line 201
   --- End of inner exception stack trace ---
   --- End of inner exception stack trace ---
   at Acoustics.Shared.ProcessRunner.KillProcess() in c:\Work\GitHub\audio-analysis\Acoustics\Acoustics.Shared\ProcessRunner.cs:line 214
   at Acoustics.Shared.ProcessRunner.PrepareRun(String arguments, String workingDirectory) in c:\Work\GitHub\audio-analysis\Acoustics\Acoustics.Shared\ProcessRunner.cs:line 152
   at Acoustics.Shared.ProcessRunner.RunTaskReaders(String arguments, String workingDirectory, Int32 retryCount) in c:\Work\GitHub\audio-analysis\Acoustics\Acoustics.Shared\ProcessRunner.cs:line 272
   at Acoustics.Shared.ProcessRunner.ProcessTimeout(String arguments, String workingDirectory, Int32 retryCount, Action`3 retryMethod) in c:\Work\GitHub\audio-analysis\Acoustics\Acoustics.Shared\ProcessRunner.cs:line 347
   at Acoustics.Shared.ProcessRunner.RunTaskReaders(String arguments, String workingDirectory, Int32 retryCount) in c:\Work\GitHub\audio-analysis\Acoustics\Acoustics.Shared\ProcessRunner.cs:line 288
   at Acoustics.Tools.Audio.AbstractUtility.RunExe(ProcessRunner processRunner, String arguments, String workingDirectory) in c:\Work\GitHub\audio-analysis\Acoustics\Acoustics.Tools\Audio\AbstractUtility.cs:line 368
   at Acoustics.Tools.Audio.AbstractAudioUtility.Modify(FileInfo source, String sourceMediaType, FileInfo output, String outputMediaType, AudioUtilityRequest request) in c:\Work\GitHub\audio-analysis\Acoustics\Acoustics.Tools\Audio\AbstractAudioUtility.cs:line 92
   at Acoustics.Tools.Audio.MasterAudioUtility.ConvertAndSegmentUsingSox(FileInfo source, String sourceMimeType, AudioUtilityRequest request) in c:\Work\GitHub\audio-analysis\Acoustics\Acoustics.Tools\Audio\MasterAudioUtility.cs:line 487
   at Acoustics.Tools.Audio.MasterAudioUtility.Modify(FileInfo source, String sourceMediaType, FileInfo output, String outputMediaType, AudioUtilityRequest request) in c:\Work\GitHub\audio-analysis\Acoustics\Acoustics.Tools\Audio\MasterAudioUtility.cs:line 255
   at Acoustics.Tools.AudioFilePreparer.PrepareFile(DirectoryInfo outputDirectory, FileInfo fiSource, String outputMediaType, AudioUtilityRequest request, DirectoryInfo temporaryFilesDirectory) in c:\Work\GitHub\audio-analysis\Acoustics\Acoustics.Tools\AudioFilePreparer.cs:line 62
   at AnalysisRunner.LocalSourcePreparer.PrepareFile(DirectoryInfo outputDirectory, FileInfo source, String outputMediaType, TimeSpan startOffset, TimeSpan endOffset, Int32 targetSampleRateHz, DirectoryInfo temporaryFilesDirectory) in c:\Work\GitHub\audio-analysis\AudioAnalysis\AnalysisRunner\LocalSourcePreparer.cs:line 266
   at AnalysisBase.AnalysisCoordinator.PrepareFileAndRunAnalysis(FileSegment fileSegment, IAnalyser analyser, AnalysisSettings settings) in c:\Work\GitHub\audio-analysis\AudioAnalysis\AnalysisBase\AnalysisCoordinator.cs:line 343
   at AnalysisBase.AnalysisCoordinator.<>c__DisplayClass8.<RunParallel>b__6(FileSegment item, ParallelLoopState state, Int64 index) in c:\Work\GitHub\audio-analysis\AudioAnalysis\AnalysisBase\AnalysisCoordinator.cs:line 223
   at System.Threading.Tasks.Parallel.<>c__DisplayClassf`1.<ForWorker>b__c()
   at System.Threading.Tasks.Task.InnerInvokeWithArg(Task childTask)
   at System.Threading.Tasks.Task.<>c__DisplayClass10.<ExecuteSelfReplicating>b__f(Object param0)
2014-05-12T20:37:02.4983356+10:00 [1] INFO  LogFileOnly - ERRORLEVEL: 1000

Koala and Canetoad Recognisers may be faulty

Canetoad detector is giving inconsistent results
The KoalaMale detector is not producing any events

Source problems may be caused by a recent refactor to Oscillations.cs (now know as Ocillations2012.cs) or they may be different problems.

Test ap.exe on linux

Also investigate running analyses of DNVM (CoreCLR)

Implemented canetoad/koalaMale analysers to the rest JCU recordings

Continue to run the canetoad analysers to the rest JCU recordings, then send all the results to Kiyomi.

Double Summary Index Row

A double row is being reported in the Summary Indices CSV output. Usually at the start (seems to be a 0/1min error)

Contract failure in AP.exe (audio2csv)

Version: 249d1a5

Source file: /home/ubuntu/bioacoustics/qcif_storage/data_stag/original_audio/3b/3b6064e2-ddc0-419d-ac91-83bf8ab28d05_070314-2330.wav

Command: /mnt/workers/staging/enqueue/runs/system_69_2015_07_03t13_45_34z/programs/AP/AnalysisPrograms.exe audio2csv /source:/home/ubuntu/bioacoustics/qcif_storage/data_stag/original_audio/3b/3b6064e2-ddc0-419d-ac91-83bf8ab28d05_070314-2330.wav /config:/mnt/workers/staging/enqueue/runs/system_69_2015_07_03t13_45_34z/run.config /tempdir:/mnt/workers/staging/enqueue/runs/system_69_2015_07_03t13_45_34z/temp /output:/home/ubuntu/bioacoustics/qcif_storage/data_stag/analysis_results/system/3b/3b6064e2-ddc0-419d-ac91-83bf8ab28d05

Rest of log:

2015-07-03T13:45:35.4462920+00:00 [1] INFO  LoggedConsole - # PROCESS LONG RECORDING
2015-07-03T13:45:35.4466380+00:00 [1] INFO  LoggedConsole - # DATE AND TIME: 7/3/2015 1:45:35 PM
2015-07-03T13:45:35.4467490+00:00 [1] INFO  LoggedConsole - # Recording file:      /home/ubuntu/bioacoustics/qcif_storage/data_stag/original_audio/3b/3b6064e2-ddc0-419d-ac91-83bf8ab28d05_070314-2330.wav
2015-07-03T13:45:35.4468260+00:00 [1] INFO  LoggedConsole - # Configuration file:  /mnt/workers/staging/enqueue/runs/system_69_2015_07_03t13_45_34z/run.config
2015-07-03T13:45:35.4468960+00:00 [1] INFO  LoggedConsole - # Output folder:       /home/ubuntu/bioacoustics/qcif_storage/data_stag/analysis_results/system/3b/3b6064e2-ddc0-419d-ac91-83bf8ab28d05
2015-07-03T13:45:35.4469620+00:00 [1] INFO  LoggedConsole - # Temp File Directory: /mnt/workers/staging/enqueue/runs/system_69_2015_07_03t13_45_34z/temp
2015-07-03T13:45:35.5430700+00:00 [1] INFO  AnalysisPrograms.AnalyseLongRecordings.AnalyseLongRecording - Minimum event threshold has been set to 0.2
2015-07-03T13:45:35.5443010+00:00 [1] WARN  AnalysisPrograms.AnalyseLongRecordings.AnalyseLongRecording - Neither start nor end segment offsets provided. Therefore ignored
2015-07-03T13:45:35.6682110+00:00 [1] INFO  AnalysisPrograms.Acoustic - Image tiling is NOT enabled
2015-07-03T13:45:35.6706870+00:00 [1] INFO  LoggedConsole - STARTING ANALYSIS ...
2015-07-03T13:45:35.8791110+00:00 [1] INFO  AnalysisBase.AnalysisCoordinator - Analysis started in parallel.
2015-07-03T13:45:36.6369610+00:00 [Threadpool worker] WARN  AnalysisPrograms.Acoustic - The IndexCalculationDuration (60) does not fit well into the provided segment (30). This means a partial result has been added, 0 results will be calculated
2015-07-03T13:45:36.6400500+00:00 [Threadpool worker] INFO  AnalysisBase.AnalysisCoordinator - Completed segment 1/1 - roughly 1 completed
2015-07-03T13:45:36.6404950+00:00 [1] INFO  LoggedConsole - 
2015-07-03T13:45:36.6405590+00:00 [1] INFO  LoggedConsole - START PROCESSING RESULTS ...
2015-07-03T13:45:36.6437280+00:00 [1] WARN  AudioAnalysisTools.ResultsTools - No events or summary indices were produced, events cannot be made into indices
2015-07-03T13:45:36.9160720+00:00 [1] FATAL AnalysisPrograms.MainEntry - Unhandled exception ->
System.Diagnostics.Contracts.__ContractsRuntime+ContractException: Precondition failed: items != null
at System.Diagnostics.Contracts.__ContractsRuntime.TriggerFailure (System.Diagnostics.Contracts.ContractFailureKind,string,string,string,System.Exception) <0x00067>
at System.Diagnostics.Contracts.__ContractsRuntime.ReportFailure (System.Diagnostics.Contracts.ContractFailureKind,string,string,System.Exception) <0x00053>
at System.Diagnostics.Contracts.__ContractsRuntime.Requires (bool,string,string) <0x0002f>
at System.EnumerableExtensions.ToTwoDimensionalArray<double, AnalysisBase.ResultBases.SpectralIndexBase> (System.Collections.Generic.IList`1<AnalysisBase.ResultBases.SpectralIndexBase>,System.Collections.Generic.Dictionary`2<string, System.Func`2<AnalysisBase.ResultBases.SpectralIndexBase, double[]>>,Acoustics.Shared.TwoDimensionalArray) <0x000d3>
at AnalysisPrograms.Acoustic.SummariseResults (AnalysisBase.AnalysisSettings,AnalysisBase.FileSegment,AnalysisBase.ResultBases.EventBase[],AnalysisBase.ResultBases.SummaryIndexBase[],AnalysisBase.ResultBases.SpectralIndexBase[],AnalysisBase.AnalysisResult2[]) <0x007b7>
at AnalysisPrograms.AnalyseLongRecordings.AnalyseLongRecording.Execute (AnalysisPrograms.AnalyseLongRecordings.AnalyseLongRecording/Arguments) <0x027a3>
at (wrapper dynamic-method) object.CallSite.Target (System.Runtime.CompilerServices.Closure,System.Runtime.CompilerServices.CallSite,object,object) <0x000d0>
at System.Dynamic.UpdateDelegates.UpdateAndExecuteVoid2<object, object> (System.Runtime.CompilerServices.CallSite,object,object) <0x00553>
at PowerArgs.ArgAction`1<AnalysisPrograms.Production.MainEntryArguments>.Invoke () <0x004ac>
at AnalysisPrograms.MainEntry.Execute (PowerArgs.ArgAction`1<AnalysisPrograms.Production.MainEntryArguments>) <0x00093>
at AnalysisPrograms.MainEntry.Main (string[]) <0x00197>

Enable tiling of secondary false color spectrogram on standard indices run

These tiles will be useful when we do our mass generation - especially for detecting Koala bellows it seems.

Summary Indices not being properly rendered for Event Recognisers

For example,

For Towsey.KoalaMale, despite events being found, the black and white summary indices image is not rendering any of the relevant tracks. The RankOrder track is being rendered, but none of the data is visible/rendered within the track.

Reconfigure AED to optionally utilize Briggs noise removal

Needed by @J3xie

Depends on #7

Mp3splt error output not parsed properly

Mp3splt writes error information to standard output that is currently not captured (and exceptions are not thrown).

The result is that mp3splt fails and no file is written, resulting is a FileNotFound error later in the program

Error: begin point out of file

Seperate large events should work on horizontally as well

Work on existing AED algorithm to separate large events for Xueyan's purpose: Split events along frames.

Should use the existing separateLargeEvents function and execute it a second time with transposed input?
Ensure well covered by unit tests

Example:

1   1
1 1 1
1   1

Should produce:

1
1
1

1
1
1

Regression in audiocutter

Audiocutter now processes only first ~8 segments, then fails with file not found error. See log below.

2014-05-20T15:28:36.0610453+10:00 [1] INFO LoggedConsole - QUT Bioacoustic Analysis Program - version 14.05.0.0 (RELEASE build, 20/05/2014 3:27 PM)
Copyright QUT 2014
2014-05-20T15:28:36.0940486+10:00 [1] INFO LogFileOnly - Executable called with these arguments:
"C:\Work\GitHub\audio-analysis\AudioAnalysis\AnalysisPrograms\bin\Release\AnalysisPrograms.exe" audiocutter -n -inputfile "C:\Jie\data\Feb-Palmetum\120214.MP3" -outputdir "C:\Jie\data\Highquality_segment_01"

2014-05-20T15:28:37.9782370+10:00 [1] INFO LoggedConsole - Started segmenting at 20/05/2014 3:28:37 PM in parallel: C:\Jie\data\Feb-Palmetum\120214.MP3.
2014-05-20T15:28:41.1155507+10:00 [6] INFO LoggedConsole - Created segment 181 of 721: 120214_180min.wav
2014-05-20T15:28:41.2075599+10:00 [13] INFO LoggedConsole - Created segment 631 of 721: 120214_630min.wav
2014-05-20T15:28:41.2205612+10:00 [1] INFO LoggedConsole - Created segment 1 of 721: 120214_0min.wav
2014-05-20T15:28:41.2465638+10:00 [10] INFO LoggedConsole - Created segment 451 of 721: 120214_450min.wav
2014-05-20T15:28:41.2465638+10:00 [9] INFO LoggedConsole - Created segment 361 of 721: 120214_360min.wav
2014-05-20T15:28:41.2555647+10:00 [11] INFO LoggedConsole - Created segment 271 of 721: 120214_270min.wav
2014-05-20T15:28:41.2765668+10:00 [8] INFO LoggedConsole - Created segment 91 of 721: 120214_90min.wav
2014-05-20T15:28:41.2785670+10:00 [7] INFO LoggedConsole - Created segment 541 of 721: 120214_540min.wav
2014-05-20T15:28:41.2815673+10:00 [1] FATAL AnalysisPrograms.MainEntry - Unhandled exception ->
System.AggregateException: One or more errors occurred. ---> System.ArgumentException: File does not exist: C:\Users\n8788952\AppData\Local\Temp\j0lipa3j.mp3
Parameter name: file
at Acoustics.Tools.Audio.AbstractUtility.CheckFile(FileInfo file) in c:\Work\GitHub\audio-analysis\Acoustics\Acoustics.Tools\Audio\AbstractUtility.cs:line 421
at Acoustics.Tools.Audio.AbstractAudioUtility.Modify(FileInfo source, String sourceMediaType, FileInfo output, String outputMediaType, AudioUtilityRequest request) in c:\Work\GitHub\audio-analysis\Acoustics\Acoustics.Tools\Audio\AbstractAudioUtility.cs:line 98
at Acoustics.Tools.Audio.MasterAudioUtility.SegmentMp3(FileInfo source, String sourceMimeType, AudioUtilityRequest request) in c:\Work\GitHub\audio-analysis\Acoustics\Acoustics.Tools\Audio\MasterAudioUtility.cs:line 455
at Acoustics.Tools.Audio.MasterAudioUtility.Modify(FileInfo source, String sourceMediaType, FileInfo output, String outputMediaType, AudioUtilityRequest request) in c:\Work\GitHub\audio-analysis\Acoustics\Acoustics.Tools\Audio\MasterAudioUtility.cs:line 234
at Acoustics.Tools.AudioFilePreparer.PrepareFile(DirectoryInfo outputDirectory, FileInfo fiSource, String outputMediaType, AudioUtilityRequest request, DirectoryInfo temporaryFilesDirectory) in c:\Work\GitHub\audio-analysis\Acoustics\Acoustics.Tools\AudioFilePreparer.cs:line 62
at AnalysisRunner.LocalSourcePreparer.PrepareFile(DirectoryInfo outputDirectory, FileInfo source, String outputMediaType, TimeSpan startOffset, TimeSpan endOffset, Int32 targetSampleRateHz, DirectoryInfo temporaryFilesDirectory) in c:\Work\GitHub\audio-analysis\AudioAnalysis\AnalysisRunner\LocalSourcePreparer.cs:line 266
at AnalysisPrograms.AudioCutter.CreateSegment(ISourcePreparer sourcePreparer, FileSegment fileSegment, AnalysisSettings settings, Arguments arguments, Int32 itemNumber, Int32 itemCount) in c:\Work\GitHub\audio-analysis\AudioAnalysis\AnalysisPrograms\AudioCutter.cs:line 223
at AnalysisPrograms.AudioCutter.<>c__DisplayClass3.b__2(FileSegment item, ParallelLoopState state, Int64 index) in c:\Work\GitHub\audio-analysis\AudioAnalysis\AnalysisPrograms\AudioCutter.cs:line 209
at System.Threading.Tasks.Parallel.<>c__DisplayClassf1.<ForWorker>b__c() at System.Threading.Tasks.Task.InnerInvokeWithArg(Task childTask) at System.Threading.Tasks.Task.<>c__DisplayClass11.<ExecuteSelfReplicating>b__10(Object param0) --- End of inner exception stack trace --- at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken) at System.Threading.Tasks.Task.Wait() at System.Threading.Tasks.Parallel.ForWorker[TLocal](Int32 fromInclusive, Int32 toExclusive, ParallelOptions parallelOptions, Action1 body, Action2 bodyWithState, Func4 bodyWithLocal, Func1 localInit, Action1 localFinally)
at System.Threading.Tasks.Parallel.ForEachWorker[TSource,TLocal](IEnumerable1 source, ParallelOptions parallelOptions, Action1 body, Action2 bodyWithState, Action3 bodyWithStateAndIndex, Func4 bodyWithStateAndLocal, Func5 bodyWithEverything, Func1 localInit, Action1 localFinally)
at System.Threading.Tasks.Parallel.ForEach[TSource](IEnumerable1 source, Action3 body)
at AnalysisPrograms.AudioCutter.RunParallel(List1 fileSegments, ISourcePreparer sourcePreparer, AnalysisSettings settings, Arguments arguments) in c:\Work\GitHub\audio-analysis\AudioAnalysis\AnalysisPrograms\AudioCutter.cs:line 202 at AnalysisPrograms.AudioCutter.Execute(Arguments arguments) in c:\Work\GitHub\audio-analysis\AudioAnalysis\AnalysisPrograms\AudioCutter.cs:line 179 at System.Dynamic.UpdateDelegates.UpdateAndExecuteVoid2[T0,T1](CallSite site, T0 arg0, T1 arg1) at PowerArgs.ArgAction1.Invoke() in c:\Work\Software Dev\Github\PowerArgs\PowerArgs\ArgAction.cs:line 38
at AnalysisPrograms.MainEntry.Main(String[] args) in c:\Work\GitHub\audio-analysis\AudioAnalysis\AnalysisPrograms\MainEntry.cs:line 66
---> (Inner Exception #0) System.ArgumentException: File does not exist: C:\Users\n8788952\AppData\Local\Temp\j0lipa3j.mp3
Parameter name: file
at Acoustics.Tools.Audio.AbstractUtility.CheckFile(FileInfo file) in c:\Work\GitHub\audio-analysis\Acoustics\Acoustics.Tools\Audio\AbstractUtility.cs:line 421
at Acoustics.Tools.Audio.AbstractAudioUtility.Modify(FileInfo source, String sourceMediaType, FileInfo output, String outputMediaType, AudioUtilityRequest request) in c:\Work\GitHub\audio-analysis\Acoustics\Acoustics.Tools\Audio\AbstractAudioUtility.cs:line 98
at Acoustics.Tools.Audio.MasterAudioUtility.SegmentMp3(FileInfo source, String sourceMimeType, AudioUtilityRequest request) in c:\Work\GitHub\audio-analysis\Acoustics\Acoustics.Tools\Audio\MasterAudioUtility.cs:line 455
at Acoustics.Tools.Audio.MasterAudioUtility.Modify(FileInfo source, String sourceMediaType, FileInfo output, String outputMediaType, AudioUtilityRequest request) in c:\Work\GitHub\audio-analysis\Acoustics\Acoustics.Tools\Audio\MasterAudioUtility.cs:line 234
at Acoustics.Tools.AudioFilePreparer.PrepareFile(DirectoryInfo outputDirectory, FileInfo fiSource, String outputMediaType, AudioUtilityRequest request, DirectoryInfo temporaryFilesDirectory) in c:\Work\GitHub\audio-analysis\Acoustics\Acoustics.Tools\AudioFilePreparer.cs:line 62
at AnalysisRunner.LocalSourcePreparer.PrepareFile(DirectoryInfo outputDirectory, FileInfo source, String outputMediaType, TimeSpan startOffset, TimeSpan endOffset, Int32 targetSampleRateHz, DirectoryInfo temporaryFilesDirectory) in c:\Work\GitHub\audio-analysis\AudioAnalysis\AnalysisRunner\LocalSourcePreparer.cs:line 266
at AnalysisPrograms.AudioCutter.CreateSegment(ISourcePreparer sourcePreparer, FileSegment fileSegment, AnalysisSettings settings, Arguments arguments, Int32 itemNumber, Int32 itemCount) in c:\Work\GitHub\audio-analysis\AudioAnalysis\AnalysisPrograms\AudioCutter.cs:line 223
at AnalysisPrograms.AudioCutter.<>c__DisplayClass3.b__2(FileSegment item, ParallelLoopState state, Int64 index) in c:\Work\GitHub\audio-analysis\AudioAnalysis\AnalysisPrograms\AudioCutter.cs:line 209
at System.Threading.Tasks.Parallel.<>c__DisplayClassf`1.b__c()
at System.Threading.Tasks.Task.InnerInvokeWithArg(Task childTask)
at System.Threading.Tasks.Task.<>c__DisplayClass11.b__10(Object param0)<---

2014-05-20T15:28:41.3635755+10:00 [1] FATAL AnalysisPrograms.MainEntry -

==> Inner exception:
System.ArgumentException: File does not exist: C:\Users\n8788952\AppData\Local\Temp\j0lipa3j.mp3
Parameter name: file
at Acoustics.Tools.Audio.AbstractUtility.CheckFile(FileInfo file) in c:\Work\GitHub\audio-analysis\Acoustics\Acoustics.Tools\Audio\AbstractUtility.cs:line 421
at Acoustics.Tools.Audio.AbstractAudioUtility.Modify(FileInfo source, String sourceMediaType, FileInfo output, String outputMediaType, AudioUtilityRequest request) in c:\Work\GitHub\audio-analysis\Acoustics\Acoustics.Tools\Audio\AbstractAudioUtility.cs:line 98
at Acoustics.Tools.Audio.MasterAudioUtility.SegmentMp3(FileInfo source, String sourceMimeType, AudioUtilityRequest request) in c:\Work\GitHub\audio-analysis\Acoustics\Acoustics.Tools\Audio\MasterAudioUtility.cs:line 455
at Acoustics.Tools.Audio.MasterAudioUtility.Modify(FileInfo source, String sourceMediaType, FileInfo output, String outputMediaType, AudioUtilityRequest request) in c:\Work\GitHub\audio-analysis\Acoustics\Acoustics.Tools\Audio\MasterAudioUtility.cs:line 234
at Acoustics.Tools.AudioFilePreparer.PrepareFile(DirectoryInfo outputDirectory, FileInfo fiSource, String outputMediaType, AudioUtilityRequest request, DirectoryInfo temporaryFilesDirectory) in c:\Work\GitHub\audio-analysis\Acoustics\Acoustics.Tools\AudioFilePreparer.cs:line 62
at AnalysisRunner.LocalSourcePreparer.PrepareFile(DirectoryInfo outputDirectory, FileInfo source, String outputMediaType, TimeSpan startOffset, TimeSpan endOffset, Int32 targetSampleRateHz, DirectoryInfo temporaryFilesDirectory) in c:\Work\GitHub\audio-analysis\AudioAnalysis\AnalysisRunner\LocalSourcePreparer.cs:line 266
at AnalysisPrograms.AudioCutter.CreateSegment(ISourcePreparer sourcePreparer, FileSegment fileSegment, AnalysisSettings settings, Arguments arguments, Int32 itemNumber, Int32 itemCount) in c:\Work\GitHub\audio-analysis\AudioAnalysis\AnalysisPrograms\AudioCutter.cs:line 223
at AnalysisPrograms.AudioCutter.<>c__DisplayClass3.b__2(FileSegment item, ParallelLoopState state, Int64 index) in c:\Work\GitHub\audio-analysis\AudioAnalysis\AnalysisPrograms\AudioCutter.cs:line 209
at System.Threading.Tasks.Parallel.<>c__DisplayClassf`1.b__c()
at System.Threading.Tasks.Task.InnerInvokeWithArg(Task childTask)
at System.Threading.Tasks.Task.<>c__DisplayClass11.b__10(Object param0)
2014-05-20T15:28:41.3975789+10:00 [1] INFO LogFileOnly - ERRORLEVEL: 1000

Show absolute datetime when present in images

We have time scales now on false color spectrograms.

Adapt the code so that the absolute time is shown in one of the axes when the absolute time is present. When not present, simply show recording relative time as per normal.

The place to get this time is in the FileSegment class. Parsing of the date from file names is automatic.

The method:
https://github.com/QutBioacoustics/audio-analysis/blob/master/AudioAnalysis/AnalysisPrograms/AcousticIndices.cs#L500

The value inputFileSegment.OriginalFileStartDate should be passed to LDSpectrogramRGB.DrawSpectrogramsFromSpectralIndices (add an extra parameter (default value should be null) to that method.

This is for @YvonneP

Analysis of last 1 minute segment

When one hour recordings are analysed, the first file is exactly 60 min long and hence the whole file is analysed and appears in the false colour spectrograms and indices, but all subsequent wave files are 59minutes 58 seconds. The analysis is not analysing that last "minute" and therefore there is only 59 minutes of analysis and 59 minutes in the false colour spectrograms. I have got around this by determining a time reference from the start of the recording that takes this into consideration.

Add very verbose output to AP.exe

Add a -vv or very verbose output command line switch to AP.exe. Also add a Trace logging function (extension method) to ILog so we can log at finer details.

In particular move all stdout printing code (particularly for executable tools) over to logging to trace by default.

SaveIntermediateWavFiles saves files in the temp folder

Correct behaviour: Minute long wav files should be saved in the output directory when SaveIntermediateFiles=true.

All other files (typically true temp file with random names) should always be saved in the temp dir, if it exists.

Add IndexProperties for Event fields so SummaryIndices graph is printed properly

Separate out TileImage and reading file date from filename functionality

mp3split sometimes erroneously gives 'error: begin point out of file' for mp3 file

Realted to: QutEcoacoustics/baw-audio-tools#8

2015-04-16T00:51:05.7538926+10:00 [1] DEBUG AnalysisBase.AnalysisCoordinator - Starting segment 86/726
2015-04-16T00:51:05.7538926+10:00 [1] DEBUG AnalysisBase.AnalysisCoordinator - Starting item 2: 101212_20121210_190000+1000.mp3 (12:05:46.5520000) 01:26:00 - 01:27:00.
2015-04-16T00:51:05.7851601+10:00 [1] DEBUG Acoustics.Tools.Audio.MasterAudioUtility - Segmenting mp3 file Y:\Jie Frogs\Recorder monitoring data\2012\HR-Freestun3\Dec\101212_20121210_190000+1000.mp3 to F:\0\Towsey.Canetoad\xklu5q0btsvbhhlcvos5hr2bqrahzmfj.mp3 using mp3splt. Settings: Request starts at 1 hour 26 minutes (5160000ms) and finishes at 1 hour 27 minutes (5220000ms) (1 minute 60000ms).
2015-04-16T00:51:20.0198889+10:00 [1] DEBUG Acoustics.Tools.Audio.Mp3SpltAudioUtility - Executed mp3splt.exe in working directory F:\0\Towsey.Canetoad. Took 14 seconds 205 ms (14205.9291ms).
2015-04-16T00:51:20.0198889+10:00 [1] DEBUG Acoustics.Tools.Audio.Mp3SpltAudioUtility - Process runner output for mp3splt.exe:
Arguments:  -q  -d "F:\0\Towsey.Canetoad"  -o "xklu5q0btsvbhhlcvos5hr2bqrahzmfj"   "Y:\Jie Frogs\Recorder monitoring data\2012\HR-Freestun3\Dec\101212_20121210_190000+1000.mp3"  0086.00.00  0087.00.00 
Error output:  error: begin point out of file


Standard output:  warning: output format ambiguous (@t or @n missing)
 Processing file 'Y:\Jie Frogs\Recorder monitoring data\2012\HR-Freestun3\Dec\101212_20121210_190000+1000.mp3' ...
 info: file matches the plugin 'mp3 (libmad)'
 info: MPEG 1 Layer 3 - 32000 Hz - Mono - 64 Kb/s - Total time: 725m.46s
 info: starting normal split



2015-04-16T00:51:20.0355139+10:00 [1] DEBUG Acoustics.Tools.Audio.Mp3SpltAudioUtility - Source File exists (332.2 mb): Y:\Jie Frogs\Recorder monitoring data\2012\HR-Freestun3\Dec\101212_20121210_190000+1000.mp3
2015-04-16T00:51:20.0355139+10:00 [1] DEBUG Acoustics.Tools.Audio.Mp3SpltAudioUtility - Output File not found: F:\0\Towsey.Canetoad\xklu5q0btsvbhhlcvos5hr2bqrahzmfj.mp3
2015-04-16T00:51:20.0355139+10:00 [1] FATAL AnalysisPrograms.MainEntry - Unhandled exception ->
System.ArgumentException: File does not exist: F:\0\Towsey.Canetoad\xklu5q0btsvbhhlcvos5hr2bqrahzmfj.mp3
Parameter name: file
   at Acoustics.Tools.Audio.AbstractUtility.CheckFile(FileInfo file) in c:\Work\GitHub\audio-analysis\Acoustics\Acoustics.Tools\Audio\AbstractUtility.cs:line 421
   at Acoustics.Tools.Audio.AbstractAudioUtility.Modify(FileInfo source, String sourceMediaType, FileInfo output, String outputMediaType, AudioUtilityRequest request) in c:\Work\GitHub\audio-analysis\Acoustics\Acoustics.Tools\Audio\AbstractAudioUtility.cs:line 98
   at Acoustics.Tools.Audio.MasterAudioUtility.SegmentMp3(FileInfo source, String sourceMimeType, AudioUtilityRequest request) in c:\Work\GitHub\audio-analysis\Acoustics\Acoustics.Tools\Audio\MasterAudioUtility.cs:line 431
   at Acoustics.Tools.Audio.MasterAudioUtility.Modify(FileInfo source, String sourceMediaType, FileInfo output, String outputMediaType, AudioUtilityRequest request) in c:\Work\GitHub\audio-analysis\Acoustics\Acoustics.Tools\Audio\MasterAudioUtility.cs:line 242
   at Acoustics.Tools.AudioFilePreparer.PrepareFile(DirectoryInfo outputDirectory, FileInfo fiSource, String outputMediaType, AudioUtilityRequest request, DirectoryInfo temporaryFilesDirectory) in c:\Work\GitHub\audio-analysis\Acoustics\Acoustics.Tools\AudioFilePreparer.cs:line 62
   at AnalysisRunner.LocalSourcePreparer.PrepareFile(DirectoryInfo outputDirectory, FileInfo source, String outputMediaType, TimeSpan startOffset, TimeSpan endOffset, Int32 targetSampleRateHz, DirectoryInfo temporaryFilesDirectory, Nullable`1 mixDownToMono) in c:\Work\GitHub\audio-analysis\AudioAnalysis\AnalysisRunner\LocalSourcePreparer.cs:line 267
   at AnalysisBase.AnalysisCoordinator.PrepareFileAndRunAnalysis(FileSegment fileSegment, IAnalyser2 analyser, AnalysisSettings localCopyOfSettings, Boolean parallelised) in c:\Work\GitHub\audio-analysis\AudioAnalysis\AnalysisBase\AnalysisCoordinator.cs:line 353
   at AnalysisBase.AnalysisCoordinator.ProcessItem(FileSegment item, IAnalyser2 analysis, AnalysisSettings clonedSettings, Boolean parallelised) in c:\Work\GitHub\audio-analysis\AudioAnalysis\AnalysisBase\AnalysisCoordinator.cs:line 624
   at AnalysisBase.AnalysisCoordinator.RunSequential(IEnumerable`1 analysisSegments, IAnalyser2 analysis, AnalysisSettings clonedSettings) in c:\Work\GitHub\audio-analysis\AudioAnalysis\AnalysisBase\AnalysisCoordinator.cs:line 290
   at AnalysisBase.AnalysisCoordinator.Run(IEnumerable`1 fileSegments, IAnalyser2 analysis, AnalysisSettings settings) in c:\Work\GitHub\audio-analysis\AudioAnalysis\AnalysisBase\AnalysisCoordinator.cs:line 190
   at AnalysisPrograms.AnalyseLongRecordings.AnalyseLongRecording.Execute(Arguments arguments) in c:\Work\GitHub\audio-analysis\AudioAnalysis\AnalysisPrograms\AnalyseLongRecordings\AnalyseLongRecording.cs:line 220
   at System.Dynamic.UpdateDelegates.UpdateAndExecuteVoid2[T0,T1](CallSite site, T0 arg0, T1 arg1)
   at PowerArgs.ArgAction`1.Invoke() in c:\Work\Software Dev\Github\PowerArgs\PowerArgs\ArgAction.cs:line 38
   at AnalysisPrograms.MainEntry.Main(String[] args) in c:\Work\GitHub\audio-analysis\AudioAnalysis\AnalysisPrograms\MainEntry.cs:line 53
2015-04-16T00:51:20.0511393+10:00 [1] INFO  LogFileOnly - ERRORLEVEL: 1000

Add new output options for FalseColorSpectrograms generated via audio2csv

False color images need to be modified for use on the website.
They need to be:

tiled into hour long blocks
aligned to the hour in absolute time
each tile should include absolute time in string (matching the filename date format from QutBioacoustics/baw-workers)
be stripped of all labels / grid lines
ensure the clipping track is not overlayed and are saved separately
ensure the ribbon spectrograms are saved separately
any stripped, relevant, data is saved in a machine readable format.

This alternate file output behaviour should not be default and only enabled when required.

SoX hangs on some segments

Problem only reproducible under high, concurrent, load. AED for example.

Commits that have been addressing this issue:
fe90900

1266324

Add variable length index calculation to AcousticIndices.cs

Intro

We want to calculate indices at different resolutions (zoom levels).

It has been decided that this will be achieved by adding a parameter to Towsey.Acoustic.yml named IndexCalculationDuration. The value will be in seconds and by default be set to 60.0. When set to 60.0 the current behaviour of audio2csv -c Towsey.Acoustic.yml will remain unchanged.

When the IndexCalculationDuration is set to values other than 60.0 we'll be able to produce indices at different resolutions. These different resolution indices will be reported exactly the same as normal - as results in the AnalysisResults2 object. The files that are output will also stay the same.

Changes

The Analyse method will need to change to:

Calculate the noise profile for the entire audio segment
- And pass the noise profile into IndexCalculate.Analysis when needed
Wrap the call to IndexCalculate.Analysis (lines 350-359) in a for loop
- where the number of iterations is equal to Math.Ceil(recording.Duration()) / IndexCalculationDuration
- where the relevant sub-section of the audio-recoding samples are passed into IndexCalculate.Analysis
- and where the results are appended onto the SummaryIndices and SpectralIndices arrays

Other changes

Refactoring out noise removal and pulling samples from WAV out of IndexCalculate.Analysis
Ensuring no assumptions are made about index length in IndexCalculate.Analysis
Ensuring other result properties are correctly updated (see invariants below)

Lastly, it may be worth considering turning off image generation for analysis runs where IndexCalculationDuration != 60.0. Print a warning when this happens.

Invariants

If these conditions are false, the program is in an invalid state... an exception should be thrown for most.

IndexCalculationDuration must be less than or equal to SegmentDuration × 60.0
IndexCalculationDuration must be set to 60.0 when the value is missing or null in the config file
The AnalysisResults2 object returned from Analyse should have the following properties:
- analysisResults.SummaryIndices.Length == (Math.Ceil(recording.Duration()) / IndexCalculationDuration)
- analysisResults.SpectralIndices.Length == (Math.Round(recording.Duration()) / IndexCalculationDuration)
The StartOffset property attached to both Spectral and Summary index results will need to be set correctly. It Should be currentAudioSegmentOffset + (currentFragmentIndex * IndexCalculationDuration)
The SegmentDuration property attached to both Spectral and Summary index results will need to be set correctly. It Should be SegmentDuration = IndexCalculationDuration.
The number of rows (including the header) in any of the output CSV files should be: Math.Ceil(audioRecordingDurationSeconds / IndexCalculationDuration) + 1
- e.g. A 24hr file with IndexCalculationDuration == 60.0 should produce 1441 rows
- e.g. A 24hr file with IndexCalculationDuration == 1.0 should produce 86400 rows

Files from Tin Can Bay

Hi Anthony,

Could you please upload and analyse the data supplied. Thanks.

Yvonne

GARGH - Ensure the AudioCutter actually uses the MixDownToMonoArgument

At the moment, it would seem it is ignored 😠

See https://github.com/QutBioacoustics/audio-analysis/blob/master/AudioAnalysis/AnalysisPrograms/AudioCutter.cs#L90

Check canetoad & koala command line entry for config file correctness

Check the config file is being provided properly to the analysis method, when Execute is called

SmallAreaThreshold in AED is not appropriate.

There is a threshold called smallAreaThreshold in AED algorithm, it is used for filtering small events. However, it doesn't work in a correct way.

Here is an example below:

The smallAreaThreshold is set to 10 but the small events in the image below are not removed.

In my opinion, the actual area of the events in the image are 4 pixels, and they should be filtered out.

Then, tried the smallAreaThreshold = 20, it does filter the small events indicated in the above image.

Maybe there is an issue on smallAreaThreshold.

OpenCV

Two issues

It is huge!
Does our current solution work on mono and ARM platforms?

Possible solution: builds that optionally include OpenCV (and ones that do not!)

Possible bug with IndexCalculate

System.OverflowException: Arithmetic operation resulted in an overflow. at TowseyLibrary.DataTools.Subarray(Double[] A, Int32 start, Int32 length)

The rain and cicada indices were disabled because of the same exception (see #53) .

See https://github.com/QutBioacoustics/audio-analysis/blob/master/AudioAnalysis/AudioAnalysisTools/Indices/IndexCalculate.cs#L193

Exception stack trace as follows:

2015-05-08T18:24:46.8292571+10:00 [22] DEBUG AnalysisBase.AnalysisCoordinator - Item 2806 finished analysing 7a667c05-825e-4870-bc4b-9cec98024f5a_101013-0000_791min.wav, took 00:00:30.2583556.
2015-05-08T18:24:46.8292571+10:00 [22] DEBUG AnalysisBase.AnalysisCoordinator - Item 2806 deleted file F:\1\Towsey.Acoustic\7a667c05-825e-4870-bc4b-9cec98024f5a_101013-0000_791min.wav.
2015-05-08T18:24:46.8292571+10:00 [22] INFO  AnalysisBase.AnalysisCoordinator - Completed segment 792/1435 - roughly 1402 completed
2015-05-08T18:24:46.8917582+10:00 [67] DEBUG AnalysisBase.AnalysisCoordinator - Item 2807 finished analysing 7a667c05-825e-4870-bc4b-9cec98024f5a_101013-0000_989min.wav, took 00:00:30.2113229.
2015-05-08T18:24:46.9073840+10:00 [67] DEBUG AnalysisBase.AnalysisCoordinator - Item 2807 deleted file F:\1\Towsey.Acoustic\7a667c05-825e-4870-bc4b-9cec98024f5a_101013-0000_989min.wav.
2015-05-08T18:24:46.9073840+10:00 [67] INFO  AnalysisBase.AnalysisCoordinator - Completed segment 990/1435 - roughly 1403 completed
2015-05-08T18:24:47.5167741+10:00 [45] FINE  AudioAnalysisTools.Indices.IndexCalculate - Backtracking to fill missing data from imperect audio cuts because not enough samples available. 2134 samples overlap.
2015-05-08T18:24:47.5636516+10:00 [45] DEBUG AnalysisBase.AnalysisCoordinator - Item 2808 finished analysing 7a667c05-825e-4870-bc4b-9cec98024f5a_101013-0000_923min.wav, took 00:00:30.9676932.
2015-05-08T18:24:47.5636516+10:00 [45] DEBUG AnalysisBase.AnalysisCoordinator - Item 2808 deleted file F:\1\Towsey.Acoustic\7a667c05-825e-4870-bc4b-9cec98024f5a_101013-0000_923min.wav.
2015-05-08T18:24:47.5636516+10:00 [45] INFO  AnalysisBase.AnalysisCoordinator - Completed segment 924/1435 - roughly 1404 completed
2015-05-08T18:24:47.7355313+10:00 [81] FINE  AudioAnalysisTools.Indices.IndexCalculate - Backtracking to fill missing data from imperect audio cuts because not enough samples available. 2134 samples overlap.
2015-05-08T18:24:47.7980324+10:00 [81] DEBUG AnalysisBase.AnalysisCoordinator - Item 2814 finished analysing 7a667c05-825e-4870-bc4b-9cec98024f5a_101013-0000_1165min.wav, took 00:00:26.1896542.
2015-05-08T18:24:47.7980324+10:00 [81] DEBUG AnalysisBase.AnalysisCoordinator - Item 2814 deleted file F:\1\Towsey.Acoustic\7a667c05-825e-4870-bc4b-9cec98024f5a_101013-0000_1165min.wav.
2015-05-08T18:24:47.7980324+10:00 [81] INFO  AnalysisBase.AnalysisCoordinator - Completed segment 1166/1435 - roughly 1405 completed
2015-05-08T18:24:47.8449086+10:00 [46] FINE  AudioAnalysisTools.Indices.IndexCalculate - Backtracking to fill missing data from imperect audio cuts because not enough samples available. 2710 samples overlap.
2015-05-08T18:24:47.9074093+10:00 [54] FINE  AudioAnalysisTools.Indices.IndexCalculate - Backtracking to fill missing data from imperect audio cuts because not enough samples available. 2134 samples overlap.
2015-05-08T18:24:47.9074093+10:00 [46] DEBUG AnalysisBase.AnalysisCoordinator - Item 2813 finished analysing 7a667c05-825e-4870-bc4b-9cec98024f5a_101013-0000_1296min.wav, took 00:00:26.3540497.
2015-05-08T18:24:47.9074093+10:00 [46] DEBUG AnalysisBase.AnalysisCoordinator - Item 2813 deleted file F:\1\Towsey.Acoustic\7a667c05-825e-4870-bc4b-9cec98024f5a_101013-0000_1296min.wav.
2015-05-08T18:24:47.9074093+10:00 [46] INFO  AnalysisBase.AnalysisCoordinator - Completed segment 1297/1435 - roughly 1406 completed
2015-05-08T18:24:47.9542888+10:00 [54] DEBUG AnalysisBase.AnalysisCoordinator - Item 2812 finished analysing 7a667c05-825e-4870-bc4b-9cec98024f5a_101013-0000_1362min.wav, took 00:00:27.2714285.
2015-05-08T18:24:47.9542888+10:00 [54] DEBUG AnalysisBase.AnalysisCoordinator - Item 2812 deleted file F:\1\Towsey.Acoustic\7a667c05-825e-4870-bc4b-9cec98024f5a_101013-0000_1362min.wav.
2015-05-08T18:24:47.9542888+10:00 [54] INFO  AnalysisBase.AnalysisCoordinator - Completed segment 1363/1435 - roughly 1407 completed
2015-05-08T18:24:47.9699168+10:00 [1] FATAL AnalysisPrograms.MainEntry - Unhandled exception ->
System.AggregateException: One or more errors occurred. ---> System.OverflowException: Arithmetic operation resulted in an overflow.
   at TowseyLibrary.DataTools.Subarray(Double[] A, Int32 start, Int32 length) in c:\Work\GitHub\audio-analysis\AudioAnalysis\TowseyLibrary\DataTools.cs:line 134
   at AudioAnalysisTools.Indices.IndexCalculate.Analysis(AudioRecording recording, AnalysisSettings analysisSettings, TimeSpan subsegmentOffsetTimeSpan, TimeSpan indexCalculationDuration, TimeSpan bgNoiseNeighborhood, FileInfo indicesPropertiesConfig) in c:\Work\GitHub\audio-analysis\AudioAnalysis\AudioAnalysisTools\Indices\IndexCalculate.cs:line 194
   at AnalysisPrograms.Acoustic.Analyze(AnalysisSettings analysisSettings) in c:\Work\GitHub\audio-analysis\AudioAnalysis\AnalysisPrograms\AcousticIndices.cs:line 384
   at AnalysisBase.AnalysisCoordinator.PrepareFileAndRunAnalysis(FileSegment fileSegment, IAnalyser2 analyser, AnalysisSettings localCopyOfSettings, Boolean parallelised) in c:\Work\GitHub\audio-analysis\AudioAnalysis\AnalysisBase\AnalysisCoordinator.cs:line 400
   at AnalysisBase.AnalysisCoordinator.<>c__DisplayClass4.<RunParallel>b__3(FileSegment item, ParallelLoopState state, Int64 index) in c:\Work\GitHub\audio-analysis\AudioAnalysis\AnalysisBase\AnalysisCoordinator.cs:line 258
   at System.Threading.Tasks.Parallel.<>c__DisplayClassf`1.<ForWorker>b__c()
   at System.Threading.Tasks.Task.InnerInvokeWithArg(Task childTask)
   at System.Threading.Tasks.Task.<>c__DisplayClass10.<ExecuteSelfReplicating>b__f(Object param0)
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
   at System.Threading.Tasks.Task.Wait()
   at System.Threading.Tasks.Parallel.ForWorker[TLocal](Int32 fromInclusive, Int32 toExclusive, ParallelOptions parallelOptions, Action`1 body, Action`2 bodyWithState, Func`4 bodyWithLocal, Func`1 localInit, Action`1 localFinally)
   at System.Threading.Tasks.Parallel.ForEachWorker[TSource,TLocal](IEnumerable`1 source, ParallelOptions parallelOptions, Action`1 body, Action`2 bodyWithState, Action`3 bodyWithStateAndIndex, Func`4 bodyWithStateAndLocal, Func`5 bodyWithEverything, Func`1 localInit, Action`1 localFinally)
   at System.Threading.Tasks.Parallel.ForEach[TSource](IEnumerable`1 source, ParallelOptions parallelOptions, Action`3 body)
   at AnalysisBase.AnalysisCoordinator.RunParallel(List`1 analysisSegments, IAnalyser2 analysis, AnalysisSettings clonedSettings) in c:\Work\GitHub\audio-analysis\AudioAnalysis\AnalysisBase\AnalysisCoordinator.cs:line 268
   at AnalysisBase.AnalysisCoordinator.Run(IEnumerable`1 fileSegments, IAnalyser2 analysis, AnalysisSettings settings) in c:\Work\GitHub\audio-analysis\AudioAnalysis\AnalysisBase\AnalysisCoordinator.cs:line 193
   at AnalysisPrograms.AnalyseLongRecordings.AnalyseLongRecording.Execute(Arguments arguments) in c:\Work\GitHub\audio-analysis\AudioAnalysis\AnalysisPrograms\AnalyseLongRecordings\AnalyseLongRecording.cs:line 185
   at System.Dynamic.UpdateDelegates.UpdateAndExecuteVoid2[T0,T1](CallSite site, T0 arg0, T1 arg1)
   at PowerArgs.ArgAction`1.Invoke() in c:\Work\Software Dev\Github\PowerArgs\PowerArgs\ArgAction.cs:line 38
   at AnalysisPrograms.MainEntry.Main(String[] args) in c:\Work\GitHub\audio-analysis\AudioAnalysis\AnalysisPrograms\MainEntry.cs:line 53
---> (Inner Exception #0) System.OverflowException: Arithmetic operation resulted in an overflow.
   at TowseyLibrary.DataTools.Subarray(Double[] A, Int32 start, Int32 length) in c:\Work\GitHub\audio-analysis\AudioAnalysis\TowseyLibrary\DataTools.cs:line 134
   at AudioAnalysisTools.Indices.IndexCalculate.Analysis(AudioRecording recording, AnalysisSettings analysisSettings, TimeSpan subsegmentOffsetTimeSpan, TimeSpan indexCalculationDuration, TimeSpan bgNoiseNeighborhood, FileInfo indicesPropertiesConfig) in c:\Work\GitHub\audio-analysis\AudioAnalysis\AudioAnalysisTools\Indices\IndexCalculate.cs:line 194
   at AnalysisPrograms.Acoustic.Analyze(AnalysisSettings analysisSettings) in c:\Work\GitHub\audio-analysis\AudioAnalysis\AnalysisPrograms\AcousticIndices.cs:line 384
   at AnalysisBase.AnalysisCoordinator.PrepareFileAndRunAnalysis(FileSegment fileSegment, IAnalyser2 analyser, AnalysisSettings localCopyOfSettings, Boolean parallelised) in c:\Work\GitHub\audio-analysis\AudioAnalysis\AnalysisBase\AnalysisCoordinator.cs:line 400
   at AnalysisBase.AnalysisCoordinator.<>c__DisplayClass4.<RunParallel>b__3(FileSegment item, ParallelLoopState state, Int64 index) in c:\Work\GitHub\audio-analysis\AudioAnalysis\AnalysisBase\AnalysisCoordinator.cs:line 258
   at System.Threading.Tasks.Parallel.<>c__DisplayClassf`1.<ForWorker>b__c()
   at System.Threading.Tasks.Task.InnerInvokeWithArg(Task childTask)
   at System.Threading.Tasks.Task.<>c__DisplayClass10.<ExecuteSelfReplicating>b__f(Object param0)<---

2015-05-08T18:24:48.0480373+10:00 [1] FATAL AnalysisPrograms.MainEntry - 

==> Inner exception:
System.OverflowException: Arithmetic operation resulted in an overflow.
   at TowseyLibrary.DataTools.Subarray(Double[] A, Int32 start, Int32 length) in c:\Work\GitHub\audio-analysis\AudioAnalysis\TowseyLibrary\DataTools.cs:line 134
   at AudioAnalysisTools.Indices.IndexCalculate.Analysis(AudioRecording recording, AnalysisSettings analysisSettings, TimeSpan subsegmentOffsetTimeSpan, TimeSpan indexCalculationDuration, TimeSpan bgNoiseNeighborhood, FileInfo indicesPropertiesConfig) in c:\Work\GitHub\audio-analysis\AudioAnalysis\AudioAnalysisTools\Indices\IndexCalculate.cs:line 194
   at AnalysisPrograms.Acoustic.Analyze(AnalysisSettings analysisSettings) in c:\Work\GitHub\audio-analysis\AudioAnalysis\AnalysisPrograms\AcousticIndices.cs:line 384
   at AnalysisBase.AnalysisCoordinator.PrepareFileAndRunAnalysis(FileSegment fileSegment, IAnalyser2 analyser, AnalysisSettings localCopyOfSettings, Boolean parallelised) in c:\Work\GitHub\audio-analysis\AudioAnalysis\AnalysisBase\AnalysisCoordinator.cs:line 400
   at AnalysisBase.AnalysisCoordinator.<>c__DisplayClass4.<RunParallel>b__3(FileSegment item, ParallelLoopState state, Int64 index) in c:\Work\GitHub\audio-analysis\AudioAnalysis\AnalysisBase\AnalysisCoordinator.cs:line 258
   at System.Threading.Tasks.Parallel.<>c__DisplayClassf`1.<ForWorker>b__c()
   at System.Threading.Tasks.Task.InnerInvokeWithArg(Task childTask)
   at System.Threading.Tasks.Task.<>c__DisplayClass10.<ExecuteSelfReplicating>b__f(Object param0)
2015-05-08T18:24:48.0480373+10:00 [1] INFO  LogFileOnly - ERRORLEVEL: 1000

Microsoft.CSharp.RuntimeBinder.RuntimeBinderException: Cannot convert type `YamlDotNet.Dynamic.DynamicYaml' to `double'

ubuntu@worker-1:~/bioacoustics/workers/worker-dummy/AP$ mono --debug "/home/ubuntu/bioacoustics/workers/worker-dummy/AP/AnalysisPrograms.exe" audio2csv /source:"/home/ubuntu/bioacoustics/workers/worker-dummy/_original_audio/5d/5d6280ce-0f64-4f0c-9628-c2c10d57d959_100815-1200.mp3" /config:"/home/ubuntu/bioacoustics/workers/worker-dummy/AP/ConfigFiles/Towsey.Acoustic.yml" /tempdir:"/home/ubuntu/bioacoustics/workers/worker-dummy/tmp/5d6280ce-0f64-4_2014_11_20t05_30_50z" /output:"/home/ubuntu/bioacoustics/workers/worker-dummy/_cached_analysis_jobs/5d/5d6280ce-0f64-4f0c-9628-c2c10d57d959"
QUT Bioacoustic Analysis Program - version 14.11.0.0 (DEBUG build, 11/16/2014 10:42 PM) 
Copyright QUT 2014
Do you wish to debug? Attach now or press [Y] to attach. Press any key other key to continue.
# PROCESS LONG RECORDING
# DATE AND TIME: 11/20/2014 7:38:07 AM
# Recording file:      5d6280ce-0f64-4f0c-9628-c2c10d57d959_100815-1200.mp3
# Configuration file:  /home/ubuntu/bioacoustics/workers/worker-dummy/AP/ConfigFiles/Towsey.Acoustic.yml
# Output folder:       /home/ubuntu/bioacoustics/workers/worker-dummy/_cached_analysis_jobs/5d/5d6280ce-0f64-4f0c-9628-c2c10d57d959
# Temp File Directory: /home/ubuntu/bioacoustics/workers/worker-dummy/tmp/5d6280ce-0f64-4_2014_11_20t05_30_50z
[2014-11-20T07:38:07.1355220+00:00] WARN  - IndexProperties config can not be found! This will result in an excpetion if it is needed later on.
[2014-11-20T07:38:07.1438150+00:00] FATAL - Unhandled exception ->
Microsoft.CSharp.RuntimeBinder.RuntimeBinderException: Cannot convert type `YamlDotNet.Dynamic.DynamicYaml' to `double'
at (wrapper dynamic-method) object.CallSite.Target (System.Runtime.CompilerServices.Closure,System.Runtime.CompilerServices.CallSite,object) <IL 0x000a2, 0x002ac>
at System.Dynamic.UpdateDelegates.UpdateAndExecute1<object, double> (System.Runtime.CompilerServices.CallSite,object) <IL 0x0012a, 0x00468>
at AnalysisPrograms.AnalyseLongRecordings.AnalyseLongRecording.Execute (AnalysisPrograms.AnalyseLongRecordings.AnalyseLongRecording/Arguments) [0x0064f] in c:\Work\GitHub\audio-analysis\AudioAnalysis\AnalysisPrograms\AnalyseLongRecordings\AnalyseLongRecording.cs:102
at (wrapper dynamic-method) object.CallSite.Target (System.Runtime.CompilerServices.Closure,System.Runtime.CompilerServices.CallSite,object,object) <IL 0x00052, 0x000d0>
at System.Dynamic.UpdateDelegates.UpdateAndExecuteVoid2<object, object> (System.Runtime.CompilerServices.CallSite,object,object) <0x00500>
at PowerArgs.ArgAction`1<AnalysisPrograms.Production.MainEntryArguments>.Invoke () <0x00484>
at AnalysisPrograms.MainEntry.Execute (PowerArgs.ArgAction`1<AnalysisPrograms.Production.MainEntryArguments>) [0x00055] in c:\Work\GitHub\audio-analysis\AudioAnalysis\AnalysisPrograms\Production\MainEntryUtilities.cs:173
at AnalysisPrograms.MainEntry.Main (string[]) [0x0005e] in c:\Work\GitHub\audio-analysis\AudioAnalysis\AnalysisPrograms\MainEntry.cs:44

Mono for windows not reading machine.config properly

This means it requires configSections in the app.config that are usually in the machine.config

Disable rain and cicada indexes

Both are broken

Further, the rain index is throwing exceptions:

2015-05-05T10:24:47.1371207+10:00 [1] FATAL AnalysisPrograms.MainEntry - Unhandled exception ->
System.AggregateException: One or more errors occurred. ---> System.OverflowException: Arithmetic operation resulted in an overflow.
   at TowseyLibrary.DataTools.Subarray(Double[] A, Int32 start, Int32 length) in c:\Work\GitHub\audio-analysis\AudioAnalysis\TowseyLibrary\DataTools.cs:line 134
   at AudioAnalysisTools.Indices.RainIndices.GetIndices(Double[] signalEnvelope, TimeSpan audioDuration, TimeSpan frameStepDuration, Double[,] spectrogram, Int32 lowFreqBound, Int32 midFreqBound, Double binWidth) in c:\Work\GitHub\audio-analysis\AudioAnalysis\AudioAnalysisTools\Indices\RainIndices.cs:line 93
 at AudioAnalysisTools.Indices.IndexCalculate.Analysis(AudioRecording recording, AnalysisSettings analysisSettings) in c:\Work\GitHub\audio-analysis\AudioAnalysis\AudioAnalysisTools\Indices\IndexCalculate.cs:line 328
   at AnalysisPrograms.Acoustic.Analyse(AnalysisSettings analysisSettings) in c:\Work\GitHub\audio-analysis\AudioAnalysis\AnalysisPrograms\AcousticIndices.cs:line 286

On a set of 86 recordings, 77 failed with that exception

Reconfigure AED to optionally output hit areas

Either CSV or JSON?

Either bitmap, polygon representation, point list, on PNG?

Need by @J3xie.

Depends on #7

audio2csv run with same parameters and settings on two different machines give different results

See .zip file in Y:\temp

Conver other Analysers to IAnalyser2

Analysers to convert (priority):

canetoad
koala
... others should be added here

Fix AnalyseLongRecordings

Modify canetoad recognizer to detect release calls

Sox duration estimate wrong

Fix: check the output info (particularly the duration stat) matches between audio tools.

An exception should be thrown if they do not match.

The root cause for this seems to be a corrupted MP3 stored on availae (Y:\Backups\CorruptMP3s)

Error output when encoding file https://gist.github.com/antmt/2a718da1976a2cfda338

Invesitgate spectrogram generation

Anecdotally the spectrograms we generate seem to be a of 'lower' quality than those produced by other programs (like Audacity or SoX).

We should systematically investigate these differences (e.g. same sample rate (and same resampling rate), same window, overlap, function, same contrast/intensity settings.

Also verify if we can use the newer version of the Math.Net spectrogram generator.