Giter VIP home page Giter VIP logo

krateng / maloja Goto Github PK

View Code? Open in Web Editor NEW
1.1K 17.0 63.0 7.68 MB

Self-hosted music scrobble database to create personal listening statistics and charts

Home Page: https://maloja.krateng.ch

License: GNU General Public License v3.0

Python 57.83% JavaScript 10.57% HTML 0.20% Shell 0.69% Dockerfile 0.46% Jinja 24.34% CSS 5.90%
self-hosted scrobble music statistics listening-habits charts

maloja's Introduction

Maloja

Simple self-hosted music scrobble database to create personal listening statistics.

screenshot

You can check my own Maloja page as an example instance.

Table of Contents

Features

  • Self-hosted: You will always be able to access your data in an easily-parseable format. Your library is not synced with any public or official music database, so you can follow your own tagging schema.
  • Associated Artists: Compare different artists' popularity in your listening habits including subunits, collaboration projects or solo performances by their members. Change these associations at any time without losing any information.
  • Multi-Artist Tracks: Some artists often collaborate with others or are listed under "featuring" in the track title. Instead of tracking each combination of artists, each individual artist competes in your charts.
  • Custom Images: Don't rely on the community to select the best pictures for your favorite artists. Upload your own so that your start page looks like you want it to look.
  • Proxy Scrobble: No need to fully commit or set up every client twice - you can configure your Maloja server to forward your scrobbles to other services.
  • Standard-compliant API: Use existing, mature apps or extensions to scrobble to your Maloja server.
  • Manual Scrobbling: Listening to vinyl or elevator background music? Simply submit a scrobble with the web interface.
  • Keep it Simple: Unlike Last.fm and similar alternatives, Maloja doesn't have social networking, radios, recommendations or any other gimmicks. It's a tool to keep track of your listening habits over time - and nothing more.

How to install

Requirements

Maloja should run on any x86 or ARM machine that runs Python.

It is highly recommended to use Docker or Podman.

Your CPU should have a single core passmark score of at the very least 1500. 500 MB RAM should give you a decent experience, but performance will benefit greatly from up to 2 GB.

Docker / Podman

Pull the latest image or check out the repository and use the included Containerfile.

Of note are these settings which should be passed as environmental variables to the container:

  • MALOJA_SKIP_SETUP -- Make the server setup process non-interactive. Maloja will not work properly in a container without this variable set. This is done by default in the provided Containerfile.
  • MALOJA_FORCE_PASSWORD -- Set an admin password for Maloja. You only need this on the first run.
  • MALOJA_DATA_DIRECTORY -- Set the directory in the container where configuration folders/files should be located
    • Mount a volume to the specified directory to access these files outside the container (and to make them persistent)

You must publish a port on your host machine to bind to the container's web port (default 42010). The container uses IPv4 per default.

An example of a minimum run configuration to access maloja via localhost:42010:

	docker run -p 42010:42010 -v $PWD/malojadata:/mljdata -e MALOJA_DATA_DIRECTORY=/mljdata krateng/maloja

Linux Host

NOTE: If you are using rootless containers with Podman this DOES NOT apply to you.

If you are running Docker on a Linux Host you should specify user:group ids of the user who owns the folder on the host machine bound to MALOJA_DATA_DIRECTORY in order to avoid docker file permission problems. These can be specified using the environmental variables PUID and PGID.

To get the UID and GID for the current user run these commands from a terminal:

  • id -u -- prints UID (EX 1000)
  • id -g -- prints GID (EX 1001)

The modified run command with these variables would look like:

	docker run -e PUID=1000 -e PGID=1001 -p 42010:42010 -v $PWD/malojadata:/mljdata -e MALOJA_DATA_DIRECTORY=/mljdata krateng/maloja

PyPI

You can install Maloja with

	pip install malojaserver

To make sure all dependencies are installed, you can also use one of the included scripts in the install folder.

From Source

Clone this repository and enter the directory with

	git clone https://github.com/krateng/maloja
	cd maloja

Then install all the requirements and build the package, e.g.:

	sh ./install/install_dependencies_alpine.sh
	pip install -r requirements.txt
	pip install .

Extras

  • If you'd like to display images, you will need API keys for Last.fm and Spotify. These are free of charge!

  • Put your server behind a reverse proxy for SSL encryption. Make sure that you're proxying to the IPv6 or IPv4 address according to your settings.

How to use

Basic control

When not running in a container, you can run the application with maloja run. You can also run it in the background with maloja start and maloja stop, but this might not be supported in the future.

Data

If you would like to import your previous scrobbles, use the command maloja import *filename*. This works on:

⚠️ Never import your data while maloja is running. When you need to do import inside docker container start it in shell mode instead and perform import before starting the container as mentioned above.

	docker run -it --entrypoint sh -v $PWD/malojadata:/mljdata -e MALOJA_DATA_DIRECTORY=/mljdata krateng/maloja
	cd /mljdata
	maloja import my_last_fm_export.csv

To backup your data, run maloja backup, optional with --include_images.

Customization

  • Have a look at the available settings and specifiy your choices in /etc/maloja/settings.ini. You can also set each of these settings as an environment variable with the prefix MALOJA_ (e.g. MALOJA_SKIP_SETUP).

  • If you have activated admin mode in your web interface, you can upload custom images for artists or tracks by simply dragging them onto the existing image on the artist or track page. You can also manage custom images directly in the file system - consult images.info in the /var/lib/maloja/images folder.

  • To specify custom rules, consult the rules.info file in /etc/maloja/rules. You can also apply some predefined rules on the /admin_setup page of your server.

How to scrobble

You can set up any amount of API keys in the file apikeys.yml in your configuration folder (or via the web interface). It is recommended to define a different API key for every scrobbler you use.

Some scrobbler clients support Maloja's native API. You can also use any scrobbler that allows you to set a custom Listenbrainz or GNUFM server. See API.md for details.

If you're the maintainer of a music player or server and would like to implement native Maloja scrobbling, feel free to reach out!

If you can't automatically scrobble your music, you can always do it manually on the /admin_manual page of your Maloja server.

How to extend

If you'd like to implement anything on top of Maloja, visit /api_explorer.

maloja's People

Contributors

aaron-em avatar alim4r avatar badlandspray avatar bpepple avatar brendantwh avatar chrisnewtonrevium avatar da2x avatar derkades avatar duckfromdiscord avatar foxxmd avatar ictman1076 avatar jan666 avatar kirinokirino avatar krateng avatar krkk avatar marcobuster avatar northys avatar p182 avatar sirmartin avatar tdemin avatar thinkchaos avatar timkicker avatar velocidensity avatar zackboe avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

maloja's Issues

Funny thing but I cannot login

Why instead of "user" and "password" there is "password" and "response"?
I have to say such details discourage me from using such software as I don't know what "response" is.
I'll try to build my own solution, it take ages but at least it gives me peace of mind.

Docker update?

I am no expert by any means but I wondered if the Docker needed tweaking to always update the dependencies and install from the latest versions ..

FROM python:3-alpine

WORKDIR /usr/src/app

# update / dependencies  
RUN apk --update upgrade \
&& apk --no-cache --no-progress add gcc libxml2-dev libxslt-dev py3-pip libc-dev linux-headers \
&& rm -rf /var/cache/apk/*

# install maloja
RUN pip3 install --no-cache-dir psutil malojaserver

EXPOSE 42010

ENTRYPOINT maloja run

Cannot rebuild library (WinError 183 FileExistsError)

Clicking on "rebuild your library" in /admin_issues , without even touching anything, will redirect to the /wait page and stay there. Tabbing out to the console window (maloja debug) shows this error:

Traceback (most recent call last):
  File "c:\users\owo\appdata\local\programs\python\python38\lib\site-packages\bottle.py", line 868, in _handle
    return route.call(**args)
  File "c:\users\owo\appdata\local\programs\python\python38\lib\site-packages\bottle.py", line 1748, in wrapper
    rv = callback(*a, **ka)
  File "c:\users\owo\appdata\local\programs\python\python38\lib\site-packages\nimrodel\_api.py", line 151, in route
    result = self.handle(nodes,reqmethod,keys,headers)
  File "c:\users\owo\appdata\local\programs\python\python38\lib\site-packages\nimrodel\_apifunction.py", line 79, in handle
    return func(**querykeys,**pathkeys)
  File "c:\users\owo\appdata\local\programs\python\python38\lib\site-packages\doreah\auth.py", line 209, in newfunc
    return func(*args,**kwargs)
  File "c:\users\owo\appdata\local\programs\python\python38\lib\site-packages\maloja\apis\native_v1.py", line 260, in rebuild
    from .proccontrol.tasks.fixexisting import fix
ModuleNotFoundError: No module named 'maloja.apis.proccontrol'

This can be fixed by changing this line

from .proccontrol.tasks.fixexisting import fix

to

from ..proccontrol.tasks.fixexisting import fix

However, doing that causes another problem: in trying to rename an imported last.fm csv file, it doesn't delete the old one, erroring out and not redirecting from the /wait page (which I think is supposed to be happening)

Traceback (most recent call last): 
  File "c:\users\owo\appdata\local\programs\python\python38\lib\site-packages\bottle.py", line 868, in _handle
    return route.call(**args)
  File "c:\users\owo\appdata\local\programs\python\python38\lib\site-packages\bottle.py", line 1748, in wrapper
    rv = callback(*a, **ka)
  File "c:\users\owo\appdata\local\programs\python\python38\lib\site-packages\nimrodel\_api.py", line 151, in route
    result = self.handle(nodes,reqmethod,keys,headers)
  File "c:\users\owo\appdata\local\programs\python\python38\lib\site-packages\nimrodel\_apifunction.py", line 79, in handle
    return func(**querykeys,**pathkeys)
  File "c:\users\owo\appdata\local\programs\python\python38\lib\site-packages\doreah\auth.py", line 209, in newfunc
    return func(*args,**kwargs)
  File "c:\users\owo\appdata\local\programs\python\python38\lib\site-packages\maloja\apis\native_v1.py", line 261, in rebuild
    fix()
  File "c:\users\owo\appdata\local\programs\python\python38\lib\site-packages\maloja\proccontrol\tasks\fixexisting.py", line 68, in fix
    os.rename(datadir("scrobbles",filename_new),datadir("scrobbles",filename))
FileExistsError: [WinError 183] Cannot create a file when that file already exists: 'C:\\Users\\owo\\HOME\\.local/share/maloja\\scrobbles\\lastfmimport.tsv_new' -> 'C:\\Users\\owo\\HOME\\.local/share/maloja\\scrobbles\\lastfmimport.tsv'

Running:

  • Maloja 2.10.5
  • CPython 3.8.3
  • Windows 10 2004

Slow database build on startup and progress indication

I have 70k scrobbles in my database and run maloja in a docker container.

After rebuilding the image database building can take up to 5 minutes:

// logs/database.log
2020/11/16 17:50:12  Starting database...
2020/11/16 17:50:12  Building database...
2020/11/16 17:55:55  Database fully built!
2020/11/16 17:55:56  Authenticated Machines: web-scrobbler
2020/11/16 17:55:56  Database reachable!

The duration isn't really the issue though -- the issue is that the web server isn't started(?) until after the DB is built so during that 5 minutes it looks like the server is "down". Additionally there is no output to stdout/err so nothing shows up in the docker log output, which is a bit unnerving. I know now to look in database.log now but its not obvious and wouldn't be clear to a new user.

allow speficying separate directories for configuration, state, cache, logs

Currently configuration, state (e.g. the database), cache, and logs all reside under the DATA_DIR, which defaults to /etc/maloja. It would be nice to be able to have these default to /etc/maloja, /var/lib/maloja, /var/cache/maloja, and /var/log/maloja respectively instead and be able to configure them separately to be abe to use things like https://www.freedesktop.org/software/systemd/man/systemd.exec.html#RuntimeDirectory=

Musicbrainz Metadata API Support

Musicbrainz has an API for getting metadata from their database. Would it be possible to add support for their information catalog to compliment Last.FM/Spotify? I've found that between all 3 sources (Musicbrainz, Last.FM and Spotify) I rarely am missing metadata.

Development instructions or jumpstarter?

I know your time is limited right now and I apologize for making so many issues!

I'd like to contribute at a deeper level with Maloja by actually submitting PRs for the fixes and stuff I've been asking for but I am not a python developer and don't really have a clue or where to start when jumping into development for Maloja.

I have 8 years of professional development experience in JS, PHP, and C# and have the tooling I probably need (Jetbrains IDE with python plugin) so I don't need a complete beginner's guide but any information you can point to or give me for a jumping off point to get Maloja running in a way I can debug it would be very helpful.

EDIT: I stumbled my way into it but would still appreciate input. I will make a PR to the readme (or a developing.md file) after I have something more complete to add:

The directory specified by the env MALOJA_DATA_DIRECTORY must be populated with all folders from maloja/data_files. By default on linux this is ~/.local/share/maloja

pydevd.py --module ... --file maloja.server

/post.audioscrobbler.com to ? for Slimserver8 (Working)

Hi there

I am trying to tweak an old Slimserver plugin to get it to scrobble to my maloja box and wondered if I could just replace the http://post.audioscrobbler.com with my maloja url.

It doesn't seem to work and I wondered if someone could point me too the url I need to get the same api as at
post.audioscrobbler.com?

I also tried http://my-maloja/apis/audioscrobbler which didn't seem to work either :-/

use constant HANDSHAKE_URL => 'http://post.audioscrobbler.com/';
	my $url = HANDSHAKE_URL
		. '?hs=true&p=1.2'
		. '&c=' . CLIENT_ID
		. '&v=' . CLIENT_VER
		. '&u=' . $params->{username}
		. '&t=' . $time
		. '&a=' . md5_hex( $params->{password} . $time );

Thanks for any help

Edit: Notice the '1.2' and wondered if that was related to -> #35

Package from PyPi not working

If the package is installed on amd64 only by typing pip3 install malojaserver, which is the recommended way in the docs and install_alpine.sh and install_ubuntu.sh do not say otherwise, I get a traceback:

$ maloja run
Traceback (most recent call last):
  File "/home/temrix/Downloads/maloja/venv/bin/maloja", line 10, in <module>
    sys.exit(main())
  File "/home/temrix/Downloads/maloja/venv/lib/python3.7/site-packages/doreah/control.py", line 44, in wrapper
    return func(*args,**kwargs)
  File "/home/temrix/Downloads/maloja/venv/lib/python3.7/site-packages/maloja/controller.py", line 195, in main
    if action in actions: actions[action](*args,**kwargs)
  File "/home/temrix/Downloads/maloja/venv/lib/python3.7/site-packages/maloja/controller.py", line 169, in direct
    from . import server
  File "/home/temrix/Downloads/maloja/venv/lib/python3.7/site-packages/maloja/server.py", line 12, in <module>
    from . import database
  File "/home/temrix/Downloads/maloja/venv/lib/python3.7/site-packages/maloja/database.py", line 4, in <module>
    from .cleanup import CleanerAgent, CollectorAgent
  File "/home/temrix/Downloads/maloja/venv/lib/python3.7/site-packages/maloja/cleanup.py", line 2, in <module>
    from . import utilities
  File "/home/temrix/Downloads/maloja/venv/lib/python3.7/site-packages/maloja/utilities.py", line 150, in <module>
    cacheage = settings.get_settings("CACHE_EXPIRE_POSITIVE") * 24 * 3600
TypeError: unsupported operand type(s) for *: 'NoneType' and 'int'

However, on armhf, using this method, it is installing just fine. I noticed pkg-resources==0.0.0 as an additional package being installed on amd64.

Comma (,) not detected as artist seperator

As in the title.

Viktor Vaughn feat. Lord Sear, Ben Grimm, Rodan & Louis Logic

results in:

Viktor Vaughn
Lord Sear, Ben Grimm, Rodan
Louis Logic

as 3 seperate artists.

Is this an issue with my tags or something that could be implemented in Maloja? I would be happy to change them within reason, (in the case of artists that are currently billed as Artist 1, Artist 2, I would be willing to change to Artist 1 feat. Artist 2) however I wouldn't want to go with something that is gramatically incorrect such as Viktor Vaughn feat. Lord Sear & Ben Grimm & Rodan & Louis Logic

Great work on this. I'd be happy to take a look if you could point me in the right direction in terms of running this for development (currently learning Python)

Multiple users

Hi!
I wanted to use docker image on my RPi but it's amd64 only, so I was wondering if I install using pip install malojaserver how can I support multiple users? I wanted to use this with multi-scrobble but I'm not sure how I can set everything up for 2 users.

Thank you!

[Admin UI][Enhancement] Key management

Would it be possible to add a key management page under the admin UI?

I'd like to be able to host multiple instances of maloja for friends/family and it'd be very handy if users could manage their keys without having access to the server.

Ability to delete / remove an artist and/or a track?

Hi there

Forgot and left my music running and played some tracks I'd prefer had not gotten scrobbled :)

Christmas tracks and the like that I'd prefer to remove from my scrobbles history.

Is there a gui way to remove them at all please?

Thanks in advance.

Thanks for sharing this with me on Reddit but

However I would like to have possibility to turn off "this week", e.g.radio program is weekly so it does not have much sense :)
As I understand this is for one-person use so for different users I need to run 2 servers?
And why some intresting project like this are in python? πŸ‘Ž pip is horrible to update dependencies, and tbh I don't see python as a web server tool. Nodejs would have more sense to me or even Swift.

/issues doesnt work, Error 500 after rebuild database

Hi, I have problem where the /issues page doesnt work.
Whenever I click something, it will show on debug the error below:

[debug] Generated page issues in 1.3312759399414062s (Python+HTML)
Traceback (most recent call last):
  File "/home/joshua/.local/bin/bottle.py", line 868, in _handle
    return route.call(**args)
  File "/home/joshua/.local/bin/bottle.py", line 1748, in wrapper
    rv = callback(*a, **ka)
  File "/home/joshua/.local/lib/python3.7/site-packages/nimrodel/_api.py", line 134, in route
    result = self.handle(nodes,reqmethod,keys,headers)
  File "/home/joshua/.local/lib/python3.7/site-packages/nimrodel/_apifunction.py", line 79, in handle
    return func(**querykeys,**pathkeys)
  File "/home/joshua/.local/lib/python3.7/site-packages/maloja/database.py", line 720, in newrule
    tsv.add_entry("rules/webmade.tsv",[k for k in keys])
  File "/home/joshua/.local/lib/python3.7/site-packages/doreah/_internal.py", line 27, in newfunc
    return func(*args,**realargs)
  File "/home/joshua/.local/lib/python3.7/site-packages/doreah/tsv.py", line 111, in add_entry
    create(filename)
  File "/home/joshua/.local/lib/python3.7/site-packages/doreah/tsv.py", line 101, in create
    open(filename,"w").close()
FileNotFoundError: [Errno 2] No such file or directory: 'rules/webmade.tsv'

I tried to touch /home/joshua/.local/share/maloja/rules/webmade.tsv
Still same result.

Before enabling debug, here is the symptom that lead me to troubleshoot with debug:
After I done my clicks on /issues pages, after I rebuild the database, I can no longer visit the page properly, it will give me ERROR 500, and i have to restart maloja server to get it to work again

Below is the error of that in error log

2020/05/26 11:17:28  Error in website generation: (<class 'IndexError'>, IndexError('list index out of range'), <traceback object at 0x7fd8d67f44c8>)

I run Debian and run maloja by pip installing it, on user joshua, and not using any virtual env. This container is just for this service alone.

Use stdout for logging

When running in a docker container, usually log messages should be handled by docker instead of logged to a file by the application.

For right now I've done this, but it feels hacky:

LOGS="${MALOJA_DATA_DIRECTORY}/logs"
mkdir -p ${LOGS}
ln -s /dev/stdout "${LOGS}/debug.log"
ln -s /dev/stdout "${LOGS}/maloja.server.log"
ln -s /dev/stdout "${LOGS}/maloja.database.log"
ln -s /dev/stdout "${LOGS}/maloja.utilities.log"

Is there an option (environment variable) to disable all file logging and use stdout instead?

All pages showing things like \num1,

All pages (/, /track, /charts_tracks, etc...) shows \num1, \num22 escapes instead of their respective symbols. The console shows this too, but I assume this is intentional.

Docker support

Would love to see this dockerized so I can run it as a service on my unraid server

Can't scrobble from android with Simple Scrobbler

The readme says that Maloja is compatible with Simple Scrobbler for android, but I cant get it to work. Im running Maloja through nginx for SSL, but when putting the details for the server in the GNU-FM server, SS instantly says "wrong username/password/token". Is there something else I should do? Ive installed the nginx certificate on my phone, port forwarded, and I can access the main page through Chrome android with https no problem.

Running:

  • Maloja 2.12.5
  • CPython 3.8.5
  • Ubuntu 20.04 with WSL 2 on Windows 10 20H2
  • Android 10
  • Simple Scrobbler 1.7.0

Update from 2.6.4 to 2.7.8 appears to break database maintenance

URL Affected: [maloja instance]/admin_issues

Page returned: 500

maloja debug output:

Traceback (most recent call last):
  File "/usr/local/bin/bottle.py", line 868, in _handle
    return route.call(**args)
  File "/usr/local/bin/bottle.py", line 1748, in wrapper
    rv = callback(*a, **ka)
  File "/home/maloja/.local/lib/python3.7/site-packages/doreah/auth.py", line 197, in newfunc
    return func(*args,**kwargs)
  File "/home/maloja/.local/lib/python3.7/site-packages/maloja/server.py", line 232, in static_html_private
    return static_html(name)
  File "/home/maloja/.local/lib/python3.7/site-packages/maloja/server.py", line 263, in static_html
    res = template.render(**LOCAL_CONTEXT)
  File "/home/maloja/.local/lib/python3.7/site-packages/jinja2/environment.py", line 1090, in render
    self.environment.handle_exception()
  File "/home/maloja/.local/lib/python3.7/site-packages/jinja2/environment.py", line 832, in handle_exception
    reraise(*rewrite_traceback_stack(source=source))
  File "/home/maloja/.local/lib/python3.7/site-packages/jinja2/_compat.py", line 28, in reraise
    raise value.with_traceback(tb)
  File "/home/maloja/.local/lib/python3.7/site-packages/maloja/web/jinja/admin_issues.jinja", line 35, in top-level template code
    {% set issuedata = dbp.issues() %}
  File "/home/maloja/.local/lib/python3.7/site-packages/maloja/web/jinja/abstracts/admin.jinja", line 1, in top-level template code
    {% extends "abstracts/base.jinja" %}
  File "/home/maloja/.local/lib/python3.7/site-packages/maloja/web/jinja/abstracts/base.jinja", line 21, in top-level template code
    {% block content %}
  File "/home/maloja/.local/lib/python3.7/site-packages/maloja/web/jinja/abstracts/admin.jinja", line 47, in block "content"
    {% block maincontent %}
  File "/home/maloja/.local/lib/python3.7/site-packages/maloja/web/jinja/admin_issues.jinja", line 61, in block "maincontent"
    <td>{{ artistLink(issue[0]) }} sounds like the combination of {{ len(issue[1]) }} artists:
jinja2.exceptions.UndefinedError: 'artistLink' is undefined

I'll leave my Maloja database as it is (save for scrobbling new tracks) if you need it for debugging

Today lost all my Top Artist cover art from the top left?

After clearing the cache directory of the docker and adding spotify, I managed to get all of the top left artists populated and life was good πŸ‘

Today I went to my maloja page and all of the artist cover art are gone?

I have checked the thirdparty.log and nothing jumps out, the meta providers register and proceed to get data, some can't find coverart some can, but no other errors?

2020/10/28 13:15:16  LastFM registered as metadata provider
2020/10/28 13:15:16  Spotify registered as metadata provider
2020/10/28 13:15:17  MusicBrainz registered as metadata provider

Not sure if spotify / lastfm has blocked me, but don't think so as some fetches succeed...?

Could not set up andoid phone to work with Maloja

Hi!
I've tried to use both two applications: Pano Scrobbler and Simple Scrobbler.
I tried to use http and https. Set up with and without ssl support, with and withouth "http://" and "https://" prefix. Tried several tokens, but still no success.
The only success was with Simple Scrobbler - I set up custom ListenBrainz server with "https://" prefix and disabled ssl.

maloja debug show message:
`API request: ['listenbrainz', '1', 'submit-listens']
Keys:
listen_type : playing_now

     payload : [{'track_metadata': {'artist_name': 'Benassi, Benny - Best of', 'track_name': 'Satisfaction (Original Album Version)'}}]

     Host : maloja website

     Authorization : token secrettoken

     Content-Type : application/json; charset=UTF-8

     User-Agent : Dalvik/2.1.0 (Linux; U; Android 10; LYA-L29 Build/HUAWEILYA-L29)

     Accept-Encoding : gzip

     X-Forwarded-For : secret

     X-Forwarded-Host : secret

     X-Forwarded-Server : secret

     Connection : Keep-Alive

     Content-Length : 156

     ['submit-listens']

     Response: {'code': 200, 'status': 'ok'}`

But it didn't appear in the Scrobbles page in Maloja.

May be I missed some instruction, of something else?
Thank you in advance!

No API Secrets results in TypeError: unsupported operand type(s) for +: 'NoneType' and 'str'

When I don't specify any API secrets I get the following error:

Various external services can be used to display images. If not enough of them are set up, only local images will be used.
	Currently not using a Last.fm API Key for image display.
	Currently not using a Fanart.tv API Key for image display.
	Currently not using a Spotify Client ID for image display.
	Currently not using a Spotify Client Secret for image display.
Traceback (most recent call last):
  File "/tmp/env/bin/maloja", line 10, in <module>
    sys.exit(main())
  File "/tmp/env/lib/python3.7/site-packages/doreah/control.py", line 44, in wrapper
    return func(*args,**kwargs)
  File "/tmp/env/lib/python3.7/site-packages/maloja/proccontrol/control.py", line 91, in main
    if action in actions: actions[action](*args,**kwargs)
  File "/tmp/env/lib/python3.7/site-packages/maloja/proccontrol/control.py", line 73, in direct
    from .. import server
  File "/tmp/env/lib/python3.7/site-packages/maloja/server.py", line 14, in <module>
    from . import database
  File "/tmp/env/lib/python3.7/site-packages/maloja/database.py", line 5, in <module>
    from .cleanup import CleanerAgent, CollectorAgent
  File "/tmp/env/lib/python3.7/site-packages/maloja/cleanup.py", line 2, in <module>
    from . import utilities
  File "/tmp/env/lib/python3.7/site-packages/maloja/utilities.py", line 18, in <module>
    from . import thirdparty
  File "/tmp/env/lib/python3.7/site-packages/maloja/thirdparty/__init__.py", line 202, in <module>
    from . import *
  File "/tmp/env/lib/python3.7/site-packages/maloja/thirdparty/spotify.py", line 6, in <module>
    class Spotify(MetadataInterface):
  File "/tmp/env/lib/python3.7/site-packages/maloja/thirdparty/__init__.py", line 71, in __init_subclass__
    s = cls()
  File "/tmp/env/lib/python3.7/site-packages/maloja/thirdparty/__init__.py", line 67, in __init__
    self.authorize()
  File "/tmp/env/lib/python3.7/site-packages/maloja/thirdparty/spotify.py", line 31, in authorize
    "Authorization":"Basic " + b64(utf(self.settings["apiid"] + ":" + self.settings["secret"])).decode("utf-8")
TypeError: unsupported operand type(s) for +: 'NoneType' and 'str'

manual scrobbling always split artist

Hi,

I added some belongtogether rules which works via api. But when I manually scrobble some tracks in the setting menu the artist is always split.

Local images not showing up

Hi. I've been trying to get Maloja to show local images in every way I could think of but it doesn't seem to want to work.

Things I tried:

(Let's pretend my top artist is called Band Name and my top track is called Song Title)

  • Putting BandName.jpg in ~/.local/share/maloja/images
  • Putting bandname.jpg in ~/.local/share/maloja/images
  • Making a folder called BandName in ~/.local/share/maloja/images and putting the picture inside it.
  • Making a folder called bandname in ~/.local/share/maloja/images and putting the picture inside it.
  • Doing the same thing with a png file instead of a jpg one.
  • Doing the same thing with BandName_SongTitle.jpg, bandname_songtitle.jpg, BandName_SongTitle.png and bandname_songtitle.png
  • Doing the same thing with a different band/different song.
  • Enabling admin mode then dragging and dropping pictures on the artist/song tiles (no matter where I drop a picture the browser just opens it instead of adding it to the page.)

Now I'm at my wits' end. Either there's something really obvious that I'm completely overlooking or I'm doing things right and it's just a weird bug.
I did try shutting down and restarting the server in between each change to the images folder and I did try deleting the cache.
I did add USE_LOCAL_IMAGES = true and CHARTS_DISPLAY_TILES = true to my settings.ini file.
I am running the latest version of Maloja.

I'd rather handle everything locally so I haven't tried adding a last.fm/spotify/fanart.tv API yet and I'd rather avoid doing so, but I guess I could try if it's the absolute last resort. Thanks in advance.

Unpredictable redirects to http from http

Hi!
Issue could be reproduced, when maloja is working behind Apache2, which is creating https-host for it.
For example - at this page "https://domain-name.tld/admin_overview" UI is trying to get data from "https://domain-name.tld/api/serverinfo", after it - it is redirected to "http://domain-name.tld/apis/mlj_1/serverinfo". And because of "mixed content" this api-call failed.
I'm not python dev, but I'm assuming, that the reason is here:

redirect("/apis/" + canonical + "/" + pth + "?" + urlencode(request.query))

Bottle.redirect is trying to redirect call and, as it is working on "http" behind the apache - it is redirecting call to the http.

May be possibility to run whole maloja on "https", or setting param to bottle to redirect always to https could solve this.

Or may be I just missed something :)

Native browser extension not scrobbling from Chromium

Chromium 86 (Linux) with maloja web extension v1.7 doesn't scrobble with Deezer in the browser?

I checked the plugin inspect popup and nothing, as well as the plugin config is green and ticking now πŸ‘

I first tried only allowing selected urls to be tracked, which didn't work, so I tried allowing everything in the plugin settings but that doesn't scrobble either.

Top Artists part of page never shows any images.

Hi. I'm a relatively new user running commit 3c12462 on my server. I'm trying to figure out why, on the Top Artists part of the page there are never any images displayed. I'm running Maloja behind Nginx, which I suspect may have something to do with it but I'm not sure.

In case you want to take a look at what I'm seeing, here's my running install: https://chrysoprax.drwho.virtadpt.net/

I haven't had much time to mess with it lately so the only place you'd see anything is on the "All Time" part.

Native support via Tauon Music Player

Hi all

I use Tauon music player (which is excellent and well supported) and I have started using a selfhosted maloja instance very recently (thanks btw)

The Tauon dev @Taiko2k kindly started to develop support for maloja, but I am not being much help!

I am trying to test and am getting an error when submitting of error : "not allowed" and I wondered whether I have misconfigured my instance?

I ask as my admin page states to use the following api link, but I am not clear if that is correct?

If you use another browser or another music player, you could try to code your own extension. The API is super simple! Just send a POST HTTP request to http://192.168.10.17:42010/apis/mlj_1/newscrobble (make sure to use the public URL) with the key-value-pairs

Any help for the native api url would be a great help please

Thanks in advance

Not accessible outside localhost

Installed Maloja on a fresh Debian 9 VPS, installed the dependencies with pip3 install -r requirements.txt, and ran ./maloja start as according to the instructions. I can access it on localhost on the VPS, but I can't access it from any outside devices. UFW is disabled, and the port seems to be open, and nothing relevant is in the system logs.

Invalid scrobble protocol using listenbrainz endpoint

Sorry I can't get more specific -- I'm not too familiar with debugging python so I can only report what I'm seeing from logs.

Using the listenbrainz endpoint I get Response: {'error': 'Invalid scrobble protocol'} even though the request payload looks valid according to the listenbrainz docs.

API request: ['listenbrainz']
Keys:
	 listen_type : playing_now
	 payload : [{'track_metadata': {'artist_name': 'Molly Burch', 'track_name': 'Nothing to Say', 'additional_info': {}, 'release_name': 'First Flower'}}]
	 Host : localhost:42010
	 Connection : keep-alive
	 Content-Length : 171
	 Authorization : Token qqdE9wei94kBEyLUZLQWRXQ1iErLfDI5vYKQaAyrjGUx3YAbXlPqOasb7d8oaQo3
	 User-Agent : Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.122 Safari/537.36
	 Content-Type : application/json; charset=UTF-8
	 Accept : */*
	 Origin : chrome-extension://dcmeiocjdadiecpnjkdlnicodhnneolh
	 Sec-Fetch-Site : none
	 Sec-Fetch-Mode : cors
	 Sec-Fetch-Dest : empty
	 Accept-Encoding : gzip, deflate, br
	 Accept-Language : en-GB,en;q=0.9,en-US;q=0.8,zh;q=0.7,zh-CN;q=0.6
	 Cookie : _ga=GA1.1.2101764908.1587748442; _gid=GA1.1.185754272.1588003795
Response: {'error': 'Invalid scrobble protocol'}


API request: ['listenbrainz']
Keys:
	 listen_type : single
	 payload : [{'listened_at': 1588086589, 'track_metadata': {'artist_name': 'Molly Burch', 'track_name': 'Nothing to Say', 'additional_info': {}, 'release_name': 'First Flower'}}]
	 Host : localhost:42010
	 Connection : keep-alive
	 Content-Length : 191
	 Authorization : Token qqdE9wei94kBEyLUZLQWRXQ1iErLfDI5vYKQaAyrjGUx3YAbXlPqOasb7d8oaQo3
	 User-Agent : Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.122 Safari/537.36
	 Content-Type : application/json; charset=UTF-8
	 Accept : */*
	 Origin : chrome-extension://dcmeiocjdadiecpnjkdlnicodhnneolh
	 Sec-Fetch-Site : none
	 Sec-Fetch-Mode : cors
	 Sec-Fetch-Dest : empty
	 Accept-Encoding : gzip, deflate, br
	 Accept-Language : en-GB,en;q=0.9,en-US;q=0.8,zh;q=0.7,zh-CN;q=0.6
	 Cookie : _ga=GA1.1.2101764908.1587748442; _gid=GA1.1.185754272.1588003795
Response: {'error': 'Invalid scrobble protocol'}

Is this because payload is an array with a json object, rather than just the json object? I'm testing integration for web-scrobbler/web-scrobbler#1986

add wand to requirements.txt

Wand is not in requirements.txt

output from maloja start:

Server is not running
The following python modules are highly recommended, some features will not work without them:
wand
You can install them with pip install -r requirements.txt or Maloja can try to install them automatically. For this, you need to run this script as a root user.

fixed by adding wand>=0.5.4 to requirements.txt and rerunning pip3 install

Airsonic / AirsonicAdvanced Scrobble Support

It looks like Airsonic Advanced is sending a problematic payload to the Maloja ListenBrainz api endpoint.

Sample Log Data


API request: ['listenbrainz']
Keys:
         payload : [{'track_metadata': {'release_name': 'Ten Thousand Fists', 'artist_name': 'Disturbed', 'additional_info'
: {'recording_mbid': '71241320-4121-4f5c-958f-feec521267eb', 'tracknumber': 1, 'release_mbid': '22584981-79a0-3300-bd26-72d
6b24c05e3'}, 'track_name': 'Ten Thousand Fists'}}]
         listen_type : playing_now
         Host : my.hostname.here
         User-Agent : Apache-HttpClient/4.5.12 (Java/14.0.1)
         Content-Length : 304
         Accept-Encoding : gzip,deflate
         Authorization : token 1234567890ABCDEF
         Content-Type : application/json; charset=utf-8
         X-Forwarded-For : my.hostname.here
         X-Forwarded-Host : my.hostname.here
         X-Forwarded-Port : 443
         X-Forwarded-Proto : https
         X-Forwarded-Server : 3dd9864aa4f3
         X-Real-Ip : my.hostname.here
Response: {'error': 'Invalid scrobble protocol'}

API request: ['listenbrainz']
Keys:
         payload : [{'listened_at': 1595608698, 'track_metadata': {'release_name': 'Indestructible', 'artist_name': 'Distur
bed', 'additional_info': {'recording_mbid': '14b5e624-5322-4a4e-a4f5-8ead6b15cf22', 'tracknumber': 1, 'release_mbid': '8888
12c2-d0b4-45b8-b14f-1a7d5a12e872'}, 'track_name': 'Indestructible'}}]
         listen_type : single
         Host : my.hostname.here
         User-Agent : Apache-HttpClient/4.5.12 (Java/14.0.1)
         Content-Length : 316
         Accept-Encoding : gzip,deflate
         Authorization : token 1234567890ABCDEF
         Content-Type : application/json; charset=utf-8
         X-Forwarded-For : my.hostname.here
         X-Forwarded-Host : my.hostname.here
         X-Forwarded-Port : 443
         X-Forwarded-Proto : https
         X-Forwarded-Server : 3dd9864aa4f3
         X-Real-Ip : my.hostname.here
Response: {'error': 'Invalid scrobble protocol'}


working scrobbler clients

Hey, great project. Thanks.

I saw in the Readme that you wanted info about working clients.

So far, I have:

  • Simple Scrobbler (Android). Works using the listenbrainz API
  • DeaDBeeF (Linux). No success. I tried all the APIs as well as any permutation I could think of. Not sure what logs would show errors that would help finding the exact problem.

Improve unicode handling on CSV import

Trying to import my last.fm list is getting the following error:

UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 4256
This is the result of the artist Jack Ü

https://stackoverflow.com/questions/23396069/best-way-to-convert-unicode-in-csv-to-plain-text

The following file can be used as an example to break it.
tmp.txt

Edit:
The follow apostrophes appear to break it too.
Vigiland,We’re The Same,We’re The Same,07 Apr 2020 23:24

Allow specifying custom data directory

I'm trying to make a "proper" docker image for this application so I'd like to store data in /data instead of ~/.local/share/maloja.
Right now it seems pretty much hard coded, I can only change the parent directory:

try:
	DATA_DIR = os.environ["XDG_DATA_HOME"].split(":")[0]
	assert os.path.exists(DATA_DIR)
except:
	DATA_DIR = os.path.join(os.environ["HOME"],".local/share/")

DATA_DIR = os.path.join(DATA_DIR,"maloja")

Could you add a DATA_DIR environment variable?

belongtogether does not apply to existing records

Hi, I added some rules under the rules directory, but it does not take effect.

I've tried to restart the process, or use the rebuild library function in admin panel. Looking into the logs nothing seems relevant. Somehow I'm sure the syntax is correct (copy&pasted).

the version is 2.11.0

Pagination for scrobbles endpoint?

I'm building a native scrobble client and want to return X number of recent scrobbles so I can verify the track hasn't already been scrobbled by another client.

I'm trying to use this url GET myUrl/apis/mlj_1/scrobbles?max=10, and it looks like it should work since max is in the pagination keys for uri_to_internal, but all scrobbles are returned by the response.

Am I doing something wrong here?

Edit: Additionally, I'm having a hard time finding any documentation on how I am supposed to include my api key in the newscrobble request

Weighted Metadata Searches / Multiple Providers Used

Would it be possible to assign a 'weight' to the different metadata providers with a cascade through them?

I Just discovered some artists have cover art and other metadata on different providers and Maloja doesn't seem to move onto a different provider when one fails.

I'd also like to order the providers by my personal preference as I've found certain providers have more 'vanilla' cover art than others.

See top albums?

The front page only shows top tracks and top artists, but no way to see what your top album are. I'd really like to see my most listened albums and their stats, but the only way to do that now is to check on last.fm.

[Admin UI][Enhancement] LastFM/Spotify Key management

Would it be possible to add upstream api key management to the admin UI?

I would like to host maloja for friends/family but don't necessarily want to use my own last.fm or spotify api keys for their instances. Having an option in the Admin UI for them to setup their own keys would be handy.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.