streambinder / spotitube Goto Github PK

Synchronize your Spotify collections downloading from external providers

Home Page: https://davidepucci.it/doc/spotitube

Go 99.54% Dockerfile 0.46%

golang go spotify spotify-api spotify-library music music-library shell

spotitube's Introduction

Systems Engineer at @Meta

Email	Website	Resume	LinkedIn	Twitter	Spotify	Instagram

spotitube's People

Contributors

Stargazers

Watchers

Forkers

mewbak subsevenx dclambert coul33t omrisnyk justforkin 6180 ichoake ppartarr

spotitube's Issues

Replace artwork download method

This is needed to isolate spotify's API stuff from Track logic, making Track.Image's URL field a string object, instead of Image.

Refer to this: http://surf.readthedocs.io/overview/#downloading

Catch user's manual kill

Ctrl+C could be detected, and tmp files deleted.

Problem right after authenticating

After executing:
spotitube -folder Music/ -log

I get a message in a window of my firefox browser saying: Login completed. However, right after that, there is a problem:

[2018-01-26 17:15:33] Something went wrong while fetching album info: Something gone wrong in 0th chunk [[]] of albums: invalid id..

Any idea what might be the problem?

Downloading playlist containing local file(s)

I tried to download both a playlist with only a local file and a playlist with local file + normal Spotify song and it states

[2019-05-20 09:01:23] Version youtube-dl: 2019.05.11
[2019-05-20 09:01:23] Version ffmpeg: 3.4.6
[2019-05-20 09:01:23] Outside authentication enabled: assure "spotitube.local" points to this machine.
[2019-05-20 09:01:23] Indexing started...
[2019-05-20 09:01:24] Indexing finished: 3 tracks indexed and 3 linked.
[2019-05-20 09:01:25] Authentication URL: http://tinyurl.com/y4tekybq
[2019-05-20 09:01:30] Authentication completed.
[2019-05-20 09:01:30] Session user: meltedexistence
[2019-05-20 09:01:30] Fetching playlist data...
[2019-05-20 09:01:30] Something went wrong while fetching playlist info.

Using ./spotitube.bin -authenticate-outside -folder /downloads -playlist https://open.spotify.com/user/meltedexistence/playlist/1PYnNE4ZzwFxVM07v4wfbT?si=_eIN2hk5QhiMLvvP4btRTQ -disable-playlist-file -log as my command. There is no log file in the spotitube directory.

Youtube Search seems to be broken

It seems the youtube.go provider is having trouble finding the urls
to download.

[2020-07-14 18:12:09] Folder: ~/Music
[2020-07-14 18:12:10] youtube-dl version: 2020.03.24
[2020-07-14 18:12:10] ffmpeg version: 4.2.2
[2020-07-14 18:12:10] Version: 27
[2020-07-14 18:12:10] Date: 2020-07-14 18:12:10
[2020-07-14 18:12:10] License: GPL-3.0
[2020-07-14 18:12:10] Authentication URL: *redacted*
[2020-07-14 18:12:10] Waiting for automatic login process. If wait is too long, manually open that URL.
[2020-07-14 18:12:13] Session user: Samuel Aldis
[2020-07-14 18:12:13] Library cache: 3 minutes / 30 minutes
[2020-07-14 18:12:13] Authentication completed.
[2020-07-14 18:12:13] Songs online: 61
[2020-07-14 18:12:13] Songs offline: 0
[2020-07-14 18:12:13] Songs missing: 61
[2020-07-14 18:12:13] 61 will be downloaded, 61 flushed and 0 ignored
[2020-07-14 18:12:13] 1/61: "Matroda - The Movement"
[2020-07-14 18:12:13] Searching entries on YouTube provider
[2020-07-14 18:12:13] No entry to download has been found.
[2020-07-14 18:12:13] 2/61: "Chris Webby - Our Planet (ft Bria Lee)"
[2020-07-14 18:12:13] Searching entries on YouTube provider
[2020-07-14 18:12:14] No entry to download has been found.
[2020-07-14 18:12:14] 3/61: "Chris Webby - Money"
[2020-07-14 18:12:14] Searching entries on YouTube provider
[2020-07-14 18:12:14] No entry to download has been found.
[2020-07-14 18:12:14] Searching entries on YouTube provider
[2020-07-14 18:12:14] 4/61: "Chris Webby - Chemically Imbalanced"
[2020-07-14 18:12:15] No entry to download has been found.
[2020-07-14 18:12:15] 5/61: "Adventure Club - Rebellious (ft Yuna)"
[2020-07-14 18:12:15] Searching entries on YouTube provider
[2020-07-14 18:12:16] No entry to download has been found.
[2020-07-14 18:12:16] 6/61: "Justin Stone - Next (ft Emilio Rojas)"
[2020-07-14 18:12:16] Searching entries on YouTube provider
[2020-07-14 18:12:16] No entry to download has been found.
[2020-07-14 18:12:16] 7/61: "Machine Gun Kelly - Dopeman"
[2020-07-14 18:12:16] Searching entries on YouTube provider
[2020-07-14 18:12:16] No entry to download has been found.
[2020-07-14 18:12:16] 8/61: "Rusko - Everyday"
[2020-07-14 18:12:16] Searching entries on YouTube provider
[2020-07-14 18:12:17] No entry to download has been found.
[2020-07-14 18:12:17] 9/61: "Machine Gun Kelly - RAP DEVIL"

Add playlist support instead of library

Please do it for me

Cannot build "extra characters at the end of p command"

I am trying to build this application on macOS 10.13.6. This is the error I receive:

λ ~/r/spotitube/ master SPOTIFY_ID=my_id SPOTIFY_KEY=my_key make

WARNING: GENIUS_TOKEN environment variable is not set. You won't be able to fetch Genius lyrics.
-en Building...
sed: 1: "/Users/myname/r/sp ...": extra characters at the end of p command
mv: rename /Users/myname/r/spotitube/src/track/constant.go.orig to /Users/myname/r/spotitube/src/track/constant.go: No such file or directory
make: *** [bin] Error 1

Any ideas what could be wrong?

Check dependencies at runtime

Add preventive checks about runtime dependencies, such ffmpeg and youtube-dl.

Add -clean-junks flag

Use this flag to batch remove temporary files that kept existing for any unattended runtime error.

Something went wrong while fetching playlist info

It authenticates correctly but when it tries to fetch the playlist fails. I was trying with one of my playlist and I also tried with the example you have in the gif:

/usr/local/sbin/spotitube -folder Music/ -playlist spotify:user:'Davide Pucci':playlist:TMG

Both with the same result

--help

Please display some usage or people will go mad.

Also, think about a more instinctive -f/--folder option instead of --music.

Add documentation

Problem authenticating using latest version

I pulled the latest code and when authenticating, my browser opens a tab with a lot of HTML text: https://hastebin.com/carisobeya.xml

Then it fails with:

runtime error: index out of range
goroutine 1 [running]: 
    panic(0x83c24c0, 0x1892a028)    /usr/lib64/go/src/runtime/panic.go:500 +0x331
    spotify.(*Spotify).ParsePlaylistUri(0x1892a708, 0xffb05148, 0x24, 0xffb05156, 0x16, 0x0, 0x0)    /home/manuel/spotitube/src/spotify/spotify.go:149 +0xd6
    spotify.(*Spotify).Playlist(0x1892a708, 0xffb05148, 0x24, 0x80, 0x0, 0x0)    /home/manuel/spotitube/src/spotify/spotify.go:81 +0x3f
    main.main()    /home/manuel/spotitube/src/main/main.go:226 +0x70ab

Keep track of YouTube track URL on metadata

Add metadata value to every song to keep track of its YouTube origin.

Add net check at runtime

Authentication failing after completing once

Running on Mac (Mojave 10.14.3), built from .bin

I had trouble getting authentication redirect to work, but once I successfully authenticated, spotitube continues to tell me "Unable to authenticate to spotify" in both GUI and headless.

The log file shows that it's returning "Authentication completed." afterward, but it's not progressing to a download state.

Screenshot: http://s.skybox1.com/9VHLzb

I'm using command: ./spotitube -folder ~/Music -playlist spotify:user:jhgough:playlist:52STzrIiqvc7Z8EP0JW2oW -disable-gui -log -debug

(Global command doesn't work - not found, I had to chmod -x the main file)

Log:

WARNING: Unknown GENIUS_TOKEN: please, export SPOTIFY_KEY enviroment variable, if you wan't to fetch lyrics from Genius provider.
Folder: ~/Music
Log: spotitube_2019-05-15_15.37.56.log
Version: 24
Caller: installed
Date: 2019-05-15 15:37:56
URL: https://github.com/streambinder/spotitube
License: GPLv2
Version youtube-dl: 2019.05.11
Version ffmpeg: 4.1.3
Actual version 24, online version 24.
// I removed index here //

Authentication URL: http://tinyurl.com/y64ems4r
Waiting for automatic login process. If wait is too long, manually open that URL.
Unable to authenticate to spotify.
Authentication completed.
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x8 pc=0x2db58d]

goroutine 1 [running]:
github.com/zmb3/spotify.(*Client).CurrentUser(0x0, 0x0, 0x3b2500, 0x11d460a0)
	/Users/johngough/Desktop/spotitube/src/github.com/zmb3/spotify/user.go:87 +0x2d
spotify.(*Spotify).User(0x11d46190, 0x43991d, 0x19, 0x80, 0x0)
	/Users/johngough/Desktop/spotitube/src/spotify/interface.go:79 +0x1d
main.mainFetch()
	/Users/johngough/Desktop/spotitube/src/main/main.go:229 +0x167
main.main()
	/Users/johngough/Desktop/spotitube/src/main/main.go:215 +0x10bb

Introduce packaging scripts

Include source code for snapcraft packaging and start rolling release using snap format.
For more informations: https://docs.snapcraft.io/build-snaps/your-first-snap

Issue downloading thousands-tracks collections

Using command ./spotitube.bin -authenticate-outside -folder /downloads -disable-playlist-file -log attempting to sync my library of just about 4.5k songs, after sitting for about 5 minutes I get this message.

[2019-05-20 08:26:02] Version youtube-dl: 2019.05.11
[2019-05-20 08:26:02] Version ffmpeg: 3.4.6
[2019-05-20 08:26:03] Outside authentication enabled: assure "spotitube.local" points to this machine.
[2019-05-20 08:26:03] Indexing started...
[2019-05-20 08:26:03] Indexing finished: 3 tracks indexed and 3 linked.
[2019-05-20 08:26:05] Authentication URL: http://tinyurl.com/y47fvupp
[2019-05-20 08:26:09] Authentication completed.
[2019-05-20 08:26:09] Session user: meltedexistence
[2019-05-20 08:26:09] Unable to load tracks cache: open /home/stlrc/.cache/spotitube/meltedexistence_library.gob: no such file or directory
[2019-05-20 08:26:09] Fetching music library...
[2019-05-20 08:30:08] Something went wrong while fetching tracks from library: Something gone wrong while reading 8th chunk of tracks: Get https://api.spotify.com/v1/me/tracks?limit=50&offset=400: read tcp 192.168.1.99:46940->35.186.224.53:443: read: connection reset by peer..

If it changes anything, the previous time I ran this it said 70th chunk of tracks.

edit: found the log file, it's in my -folder lol

Artworks get not properly applied

Downloaded artworks on final mp3 contain green artifacts.

Save playlist order

When the -playlist option is requested, save the tracks order in a m3u playlist file.
This will let users easily import playlists with their favourite music player.

Integrate surf module to fully replace browser authentication

The intention is to fully replace an active login flow via a xdg-open calls to the default system browser with a fully automated one, using input passed credential.

Add songs duration validation on YouTube result check

Add song duration filter, checking YouTube result video duration is in song duration + some seconds tolerance.

Add a proper readme

Answer to the following:

What is this?
What do I need for it to run?
How do I build it?
How do I use it?
Enjoy?

Spotify playlist fetch failure, web browser shows raw markup

Not sure why this happens but when spotitube opens a browser page it shows markup but doesn't actually render it. Results in spotitube saying "Something went wrong while fetching playlist info".

CLIENT_INVALID error

Trying to validate using the tiny url provided by the app.

Even though I am on the correct machine, it throws CLIENT_INVALID error.

I have created my own app and gave the api keys while compiling.

What am I missing, or what causes this error?

Great app anyway, Thank you,

Songs volume normalization not working as expected to

Actually the commit - 55a7c5a - which introduced songs volume normalization (increasing every songs maximum volume by +5dB) was thought around the fact ffmpeg max_volume field can reach at maximum 0.0dB: they only report a maximum of 0.0dB, but the command which increases its decibels can make it go a lot over that value (e.g., the case a song has already a max_volume value of 0.0dB will reach 5.0dB, but namely still 0.0dB).

ffmpeg flag -af "volumedetect" can give several informations about those values, and will be used to fetch the exact needed value to normalize (intended as increase till - and not more than - 0.0dB) volume value.

Add songs year metadata

Сreating a folder by playlist name

Add ability to retrigger download of already downloaded songs

This would allow to redownload songs if the download URL would change, compared to the one of the song already downloaded (depends on #19)

More verbose output for make

The first make call pulls missing external libraries, leaving the user alone with no output.
Add some output to let the user see it's actually doing something.

Suggest which was the automated decision in interactive mode

Print a simple boolean-like value into interactive mode prompt, which suggests if spotify-dl would choose that song.

what is GENIUS_TOKEN

I am getting error for setting up genius token. what is this error? looks like it is for lyrics from genius. How can I disable it or get token?

Introduce versioning

Use this value to check updates and update packages.

Limit number of parallel calls

If you have a very large library (or playlist) and decide to just -flush-metadata, lot of parallel calls would be done. As the parallelized part contains lot of files opening/closures, it could happens the SO kick the whole process out due to an eccessive amount of opened fds.

Add fixed Accept-Language header to YouTube requests

This should restrict the cases of page differences between different countries.

Last song never gets metadata applied

Enter Spotitube folder using GnuWin32

How to enter Spotitube folder using GnuWin32?

Handle featurings via 'vs' keyword

The parsing process of songs titles if featuring is reported using vs keyword doesn't work properly.
Example:

Title: Katchi
Artist: Ofenbach
Featuring: Nick Waterhouse

Song gets parsed like this:

Ofenbach - Katchi (Ofenbach vs Nick Waterhouse) (ft Nick Waterhouse)

No Internet detected on Windows 7

Running on Windows 7, self-built.

When I call the program with spotitube <Spotify playlist URI> -disable-gui -disable-browser-opening, it starts with a warning about a missing Genius token. I'm not too concerned with that functionality unless it's the root of the problem here.
Then it goes on to say that no tracks were indexed, and when I visit the tinyurl link it responds with an error message that says "INVALID_CLIENT: Invalid redirect URI". 0 tracks are indexed and 0 are linked. Any ideas what I might be doing wrong?

SoundCloud as a fallback download provider

One more idea for the night: if the song is not found on YouTube, could it be searched for on SoundCloud? A lot of my songs are failing to be found on YouTube but come up with one SoundCloud search. youtube-dl will still download from SoundCloud :)

Refactorization of packages/code

Refactoring code is actually really urgent: it's organized in a really bad and illogical way.

unable to go-get: unrecognized import path "logger"

Thanks for publishing the spotitube command!

I tried to go-get spotitube, but got stuck at the following error:

package github.com/streambinder/spotitube/src/gui
	imports logger: unrecognized import path "logger" (import path does not begin with hostname)
package github.com/streambinder/spotitube/src/gui
	imports system: unrecognized import path "system" (import path does not begin with hostname)

Similar errors are given for:

package github.com/streambinder/spotitube/src/main
	imports gui: unrecognized import path "gui" (import path does not begin with hostname)
package github.com/streambinder/spotitube/src/main
	imports spotify: unrecognized import path "spotify" (import path does not begin with hostname)
package github.com/streambinder/spotitube/src/main
	imports track: unrecognized import path "track" (import path does not begin with hostname)
package github.com/streambinder/spotitube/src/main
	imports youtube: unrecognized import path "youtube" (import path does not begin with hostname)

Follow web-browser authentication flow on different machine

I'm running Ubuntu 18.04.2 headless on a VM, this is currently Go-lang 1.7.6, but I tried the latest 1.12.5 as well. ffmpeg and youtube-dl are installed, it seems like it's failing trying to open the web server for authentication. Not sure what to do, using the prebuilt .deb. Thanks!

stlrc@spotitubevm:~/spotitube$ spotitube -playlist spotify:user:meltedexistence:playlist:6J3Jhg7vKC3PHurFUyigx3 -disable-gui
Folder: ~/spotitube
Version: 24
Caller: installed
Date: 2019-05-14 03:35:56
URL: https://github.com/streambinder/spotitube
License: GPLv2

Version youtube-dl: 2019.05.11

Version ffmpeg: 3.4.6
Authentication URL: http://tinyurl.com/y4mctlf2
Unable to authenticate to spotify.
Authentication completed.
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x8 pc=0x82f6893]

goroutine 1 [running]:
github.com/zmb3/spotify.(*Client).CurrentUser(0x0, 0x19, 0x83c3ec0, 0xa2a6070)
        /home/streambinder/Workspace/spotitube/src/github.com/zmb3/spotify/user.go:87 +0x33
spotify.(*Spotify).User(0xa06f4d0, 0x844711c, 0x19, 0x80, 0x0)
        /home/streambinder/Workspace/spotitube/src/spotify/interface.go:78 +0x23
main.mainFetch()
        /home/streambinder/Workspace/spotitube/src/main/main.go:218 +0x18d
main.main()
        /home/streambinder/Workspace/spotitube/src/main/main.go:204 +0xf5b