addictedcs / soundfingerprinting Goto Github PK

With this current version 2.0, is it possible to detect a particular song within a single stream or a long audio file of various songs by knowing the start and end time of when the song was matched in the single stream of various songs

Remove theoretical documents from the repository and create links to them on http://addictedcs.github.io/soundfingerprinting resource.

I'm concerned about their legal status, thus its better to get the potential heat out of the project.

http://semver.org/

Adjust the Interop classes in order to load both x64 and x86 native libraries.
Within the FFTWService make the call to the appropriate library.

This will allow others installing and using the library much more efficiently and easily.

Currently it is not clear how one should use the library in his/her custom application.
The description page should contain details about database initialization, fingerprinting and querying mechanisms.

Sound fingerprints are obtained for master list of songs, and stored in database using Fill database functionality.

To recognize the songs from Android, possible to match it through api ? How to compute audio samples from android and match against existing fingerprints in database.

NAudio service currently reads at a different sample format from specified URL.

The issue has be described here: http://stackoverflow.com/questions/22385783/aac-stream-resampled-incorrectly.

Waiting for any reliable resolution on that.

Dependency Resolver should have a default resolver that will wire up all the necessary dependencies, thus not asking the client for interfaces he is not aware of.

IAudioServie is defining a contract for too much work. It has to be split into a couple of different interfaces: IWaveFileUtility, IMicrophoneRecordingService, IURLStreamingService.

Read bass license key from configuration file. In case if it is not present use the defaults.

startAt parameter does not work well for NAudioService. It seeks only to the middle of the startAt parameter.

Everytime fingerprints are created, log frequency ranges are generated. Even though it is not an expensive operation, there is no need to do repetitive work.

Affected classes: LogUtility

The following 2 methods from NAudioService have to be implemented

public float[] ReadMonoFromURL(string urlToResource, int sampleRate, int secondsToDownload)
{
    throw new NotImplementedException("Use Bass.NET");
}

public float[] RecordFromMicrophoneToFile(string pathToFile, int sampleRate, int secondsToRecord)
{
     throw new NotImplementedException("Use Bass.NET");
}

Current backend, MSSQL, has its limits in terms of performance and scalability. Storing large number of songs in a database requires a data storage which is easily scalable horizontally, and has a good performance index.

After studying a large number of NoSQL solutions, would recommend MongoDB for the winner. REDIS would fit perfect for a caching layer, as it seems to perform not as well as other NoSQL database systems when stored set exceeds the amount of RAM available on dedicated machine. Possible amount of songs and fingerprints will definitely exceed RAM capacity. CouchDB seems very similar to Mongo. Because we don't really need master - master replication, as the data will be mostly inserted (with no frequent updates), would go for Mongo as well. Other database system has also been analyzed, though mainly these 3 have been considered as good candidates.

• Use Aggregation capabilities in order to perform main query on HashTable - HashBin
• Add complex index on HashTable, HashBin in order to burst performance
• Map data models to the same structure as in MSSQL (connect entities via references)
• Would store Album Title as a part of Track to minimize lookups for Album. In case if the album is not known, do not store untitled
• Sharding (if required) will be performed over the HashTables (25 elements fit perfectly)

Your code is awesome.

When i try to find a song, your algo uses Hamming Distance calculation which in turn uses fingerprints from original song (binaries). Is there a way to find the song accurately using hashes (int []). If so kindly provide code for it.

Thanks

If we have 2 audio files A1 and A2 which could be drawed as following :
A1 : XXXXXYYYYZZZZZZZZZZZZZZ (where XXXXXXXXX, YYYYY, ZZZZZZZZZZZ indicate 3 audio regions of differents lengths)
A2 : SSSSSSSSSYYYYYTTTTTTTTT (where SSSSSSSSSS, YYYYY, TTTT indicate 3 audio regions of differents lengths)

then we could call a similarity fonction which will tell us that A1 and A2 have a "similar" region of length N seconds which starts at time t1 in A1 and t2 in A2.

For the moment I'm working myself on this, but feel free to help/contribute if you wish.

Thanks,
Valeriu

Fingerprint Service method should return strongly typed object (E.g. Fingerprint) instead of bool []. Don't know yet, whether this is critical.

public interface IFingerprintService
{
   List<bool[]> CreateFingerprints(float[] samples, IFingerprintingConfiguration fingerprintingConfiguration);
}

For FingerprintUnit

RunAlgorithm - return Fingerprints
RunAlgorithmWithHashing - return SubFingerprint

Hi,

When I download the exe published in the code-project article (http://www.codeproject.com/Articles/206507/Duplicates-detector-via-audio-fingerprinting?msg=4829666#xx4829666xx) and evaluate on 250 songs, it is giving accurate results. But when I download the source code from here and test on the same songs, it is not giving the same results. Can I get the source code for the one published in code-project? I want to work on it...

Thanks,
Sateesh

Perf tests are required to profile how fast/accurate the algorithm performs, comparing to various versions of the application.

Following code was throwing an memory exception in my application

[File Name : BassAudioService.cs]
// get re-sampled/mono data
int bytesRead = Bass.BASS_ChannelGetData(mixerStream, buffer, buffer.Length * 4);
if (bytesRead == 0)
{
break;
}

bytes read was returning -1 when the limit was reached but the while loop was running for long time and chunks.Add(chunk) was creating memory exception. Do i need to change

if (bytesRead == -1)
{
break;
}
Am i missing something ?

If we have 2 audio files A1 and A2 which could be drawed as following :
A1 : XXXXXYYYYZZZZZZZZZZZZZZ (where XXXXXXXXX, YYYYY, ZZZZZZZZZZZ indicate 3 audio regions of differents lengths)
A2 : SSSSSSSSSYYYYYTTTTTTTTT (where SSSSSSSSSS, YYYYY, TTTT indicate 3 audio regions of differents lengths)

then we could call a similarity fonction which will tell us that A1 and A2 have a "similar" region of length N seconds which starts at time t1 in A1 and t2 in A2.

For the moment I'm working myself on this, but feel free to help/contribute if you wish.

Thanks,
Valeriu

soundfingerprinting repository has grown immensely over the last years. It's time to split it up according to responsabilities:

• soundfingerprinting
  • soundfingerprinting.audio
  • soundfingerprinting.audio.naudio // as the default audio engine
  • soundfingerprinting.infrastructure
  • soundfingerprinting.math
  • soundfingerprinting.minhash.permutations
  • soundfingerprinting.fft.exocortex

List of dependent libraries on soundfingerprinting core (with their own release planning etc.):

• soundfingerprinting.audio.bass
• soundfingerprinting.sql
• soundfingerprinting.mongodb
• soundfingerprinting.neuralhasher
• soundfingerprinting.soundtools
• soundfingerprinting.duplicatesdetector

Dependent libraries should contain their own build cmdlets, as well as unit test suites.

AudioService, StreamingUrlReader, SoundCaptureService provide similar functionality which can be abstracted under the same interface, or at least combined under the same naming convention: AudioFileReader, StreamingUrlReader, AudioDeviceReader. This will help develop commands that will fingerprint/query using different underlying readers (source).

Public interfaces should not contain any default bindings s.t. they will be indadvertedly used without the caller knowing about it.

Only internal interfaces should have them.

2 adjacent fingerprints from the same audio snippet are usually very similar in terms of their top 200 wavelet locations. In order to take advantage of this characteristic, we can keep track of indexes of previously sorted array, and reassign them when FingerprintDescriptor.ExtractTopWavelets is called.

A good implementation of Timsort algorithm can be found here.

Below method has to be modified:

public bool[] ExtractTopWavelets(float[][] frames, int topWavelets)
{
       int rows = frames.GetLength(0); /*128*/
       int cols = frames[0].Length; /*32*/
       float[] concatenated = new float[rows * cols]; /* 128 * 32 */
       for (int row = 0; row < rows; row++)
       {
            Buffer.BlockCopy(frames[row], 0, concatenated, row * frames[row].Length * 4, frames[row].Length * 4);
       }

       int[] indexes = Enumerable.Range(0, concatenated.Length).ToArray();
       Array.Sort(concatenated, indexes, absComparator); // TODO Use Timsort
       bool[] result = EncodeFingerprint(concatenated, indexes, topWavelets);
       return result;
}

Hi.
I'm trying to run the build.cmd but ends with the following error:

D:\Documentos\soundfingerprinting-master\build\Build.proj(33,2): error MSB3073: O comand
o "..\tools\MSTest.exe /testcontainer:Release\UnitTests\SoundFingerprinting.Tests.dll" concluiu com o código -532462766.

What can be the reason?
My system is windows 8.1 x64.

Thanks and sorry by the poor english.

If we have 2 audio files A1 and A2 which could be drawed as following :
A1 : XXXXXYYYYZZZZZZZZZZZZZZ (where XXXXXXXXX, YYYYY, ZZZZZZZZZZZ indicate 3 audio regions of differents lengths)
A2 : SSSSSSSSSYYYYYTTTTTTTTT (where SSSSSSSSSS, YYYYY, TTTT indicate 3 audio regions of differents lengths)

then we could call a similarity fonction which will tell us that A1 and A2 have a "similar" region of length N seconds which starts at time t1 in A1 and t2 in A2.

For the moment I'm working myself on this, but feel free to help/contribute if you wish.

Thanks,
Valeriu

DirectSound service has to be deleted from the project. It is useless in terms of its usability (reading only .wav files).
Substitute DirectSound with NAudio in unit test project and delete DirectSound (amen).

SupportedFormats has to return a string list of formats that AudioService is supporting.

Extend FingerprintCommand so that fingerprints can be generated from StreamingUrlReader. This will allow fingerprint generation directly from a URL resource.

Album model can be safely embedded into Track as it represents mainly read only information (it will be updated only in case if an error is introduced, which is unlikely). Having a typical read only model, would suggest denormalizing it into Track:

• Album name
• Album release year (is it really useful?)

Mark those fields as nullable (in case when album information is not known).

Hi,

I tried to use your library to build a "Shazam like" app for myself. Only for testing purpose, I recorded an audio sample with my smartphone, imported it on my computer and query the database with it.

Unfortunately, with the default configurations, I never get a correct matching of the song. The record is not too bad, there's no significant noise since it was recorded at home in a quiet room from my main speakers system.

Still, default configurations work great to find songs duplicates, so I'm just wondering if there's any configuration parameter that can be specifically tune to reach my goal.

Thanks

Xavier Q.

Permutations don't have to be stored in the database. They rarely (actually never) get changed.

Remove database permutations related code from the SoundFingerprinting project. Create a patch to delete them from the database (as well as their associated stored procedures).

Add an option of querying the underlying datasource with fingerprints generated directly from a streaming url.

In case if seconds to query is bigger than audio snippet length, the algorithm has to query only the amount which it can generate the fingerprints from.

Consider any other freely accessible library that will be able to read and down-sample audio files of different format (mandatory .mp3 format).

NAudio seems a good candidate - http://naudio.codeplex.com

New service should implement methods defined in IAudioService and IExtendedAudioService.

IModelService contains methods which expose underlying database structure. In order to seamlessly integrate NoSQL data support, underlying model structure has to be hidden.
IModelService has to contain only the following methods:

{
    IModelReference InsertTrack(TrackData track);
    void InsertHashDataForTrack(IEnumerable<HashData> hashes, IModelReference trackReference);
    IList<SubFingerprintData> ReadSubFingerprintDataByHashBucketsWithThreshold(long[] buckets, int threshold);
}

Rename ReadSubFingerprintDataByHashBucketsWithThreshold method to better depict underlying functionality.

Currently, permutations are read either from database or from a local file. In order to have a resilient library, include a class with predefined permutations declared in a static array. This will allow deploying the dlls without accompanying files.

Unsure how to test v2 of this library. I attempted to test using the Nuget version, but that seems to be 1.0.1

How do I use version 2.0?

-- Just realised that the github code trunk is currently at version 2...

Windows (Hann, Hamming, etc.). did not introduce any benefit in applying them, meaning that they are a useless component for the system.

Removing them will burst performance on any of the FFT services.

Normalizing audio signal have to be performed transparently, out of SpectrumService. Move it to a separate step within the transformation cycle.

A variety of songs can be downloaded from http://incompetech.com/. The songs are available under a creative commons licence : http://incompetech.com/music/royalty-free/licenses/

I would suggest that adding a note to the readme.md file would be sufficient to attribute the songs.

I'd suggest going to http://incompetech.com/music/royalty-free/ and using the search tools to pick a song to include.

Rename duplicates detector, sound tools namespace as well.
Do not forget Readme file and NuGet package description.

Create a description page on http://addictedcs.github.io/soundfingerprinting resource with details about all algorithms that have been used in the implementation. Also link all third party libraries that are linked to the project, with links to their pages.

Hi, When generating fingerprints for a song. the subfingerprint table is getting filled correctly. But when querying for a best match, the code is looking for entries in the 25 Hash tables which are all empty. so a match is never found .

Is the code supposed to add rows to the 25 hash tables during creation of a fingerprint ?
Thanks in advance

Getting rid of all StyleCop violations will rather improve code readability. No violations has to be added to existent ones.

Profile MongoDbModelService. Time required to fingerprint data from an audio file is unfeasible. This is strongly related to groupby functionality used by the algorithm.

Similar audio files have same almost same duration. In order to decrease false negatives, before declaring a song to be a duplicate, check for its duration (+- delta).