All versions since 0.16.1
0.16.1
Changelog
🐛 Bug Fixes
- (No Category) Add missing
configureAsfor ENV parsed components #661
0.16.2
Changelog
🐛 Bug Fixes
- (source): Fix missing id/uid on cached recent plays #662 #665
0.16.3
Changelog
🐛 Bug Fixes
- Fix incorrect prefix not being used for env key collection #668
- (apple music) include library-songs in recent tracks #667
📚 Documentation
- Improve scenario guidance for lfm/lz endpoints
0.16.4
The jumble of patches released this week in the aftermath of 0.16.0 were indicative of a poor release.
I take pride in making sure Multi-scrobbler is not one of those projects you are hesitant to upgrade because you don’t know if “this will be the one that breaks unexpectedly”. 0.16.0 was not up to that quality and for that, I apologize.
This release should hopefully make things right.
What’s New?
ENV Testing and Stability
The ENV improvements made in 0.16.0 are here to stay but to address the quality issues mentioned above this release focused on building a proper test suite to validate and test all ENV keys with real, valid data:
- tests were added for all sources/clients that
- generate all ENVs for each source/client with real data derived from the ENV schema shape, using zocker
- assign these ENVs to the actual environment and then build the actual source/client, verifying no config errors and that exactly one source/client of that type exist
This now puts ENV testing on the same level of testing as file/aio config and should catch any errors, like those fixed in 0.16.1 - 0.16.3, before they get to the release stage.
Full Changelog
⛰️ Features
- (No Category) Relax configureAs requirements for AIO configs
- (No Category) Strongly-type internal events #666
- (No Category) Add ENV config test suite
- (No Category) Update json schema to draft-2020-12
- (No Category) Normalize domain:port url input
- (No Category) Better validation for normalized config urls
🐛 Bug Fixes
- (discord) Fix discord ENV parsing validation
- (No Category) Check for ENV prefix with startsWith instead of includes #668
- (No Category) Narrow env config key collection to include prefix underscore #668
- (No Category) Hand-rolled env data fixtures for some edge cases
- (No Category) Hand-roll applemusic env fixture data
📚 Documentation
- (No Category) Add more guidance for File config example/editor
- (No Category) Hand-roll discord ENV updates
0.16.5
This is a small release due to needing to get Listenbrainz’s new request requirements out to stable! A larger release with new UI functionality will be coming soon.
What’s New?
Listenbrainz User-Agent
A recent botnet attack has forced Listenbrainz to require all traffic to include a User-Agent and implement retry backoffs.
Multi-Scrobbler already implements backoffs but did not have a proper User-Agent in place. This release introduces a UA for all requests that should restore Listenbrainz connectivity.
teal.fm production lexicon
teal.fm has moved their lexicon to the production namespace and so this release of multi-scrobbler updates the repository records to use the new namespace.
Other Changes
- @halkeye fixed the code for generating “Edit this page” links in the docs :tada:
0.17.0
What’s New?
Preview UI Feature Parity
The new UI has feature parity with the existing UI, plus so many more improvements. :tada:
0.17.0 will be the last release before the new UI replaces the existing one. If you have design feedback or things aren’t working as expected please report them in this issue. Also would be nice if you left a comment letting me know that everything is working as expected and you like the new design! Feedback either way is good.
Listenbrainz User-Agent Changes
Listenbrainz has updated their User-Agent requirements to require a point of contact, similar to how Musicbrainz works.
The 0.16.5 release that addressed Listenbrainz User-Agent updates is no longer sufficient. Use the updated Listenbrainz for multi-scrobbler to update your configs to include a contact.
Use the 0.17.2 release. Your listenbrainz configuration does not need to be modified.
Improved Source Play Processing
The base Source class has been restructured to use a queue workflow, similar to the base Client class, that enables new functionality and improves all existing Play processing. Some highlights:
- Backlog play processing no longer blocks Source startup
- Ingress based Sources can now send batch scrobbles and get an immediate response without needing to wait for all Plays to process
- Source plays that fail to process can now be retried (same way as Client plays)
- Sources can process existing, queued Plays even if polling has failed
Optimized Existing Play Checks
When polling Listening History based Sources MS parses all returned history, every time, to look for changes in the list.
Previously, if you had preCompare transforms these needed to run for all history before MS could check if each Play already existed.
Now, MS stores a hash of the original data, before transforms, and uses this to check for existing Plays before transforming. This optimization can potentially save hundreds of API calls to musicbrainz and wasted CPU cycles transforming Plays that would then be thrown away immediately.
Play Events
The events shown on the Timeline for a Play in the Preview UI were previously hacked together from many different locations in a Play’s data.
This data has now been restructured as a first-class data structure persisted to the database. What this means for you is that the Timeline will now show a more complete, user-friendly list of events and that future event types are trivial to create.
Api Validation and Docs
Thanks to @Mini-Sylar ’s excellent express-typed-router library MS now has fully-typed api routes.
Ingress-based Sources like Webscrobbler and Listenbrainz/Last.fm Endpoints get exhaustive request body/query validation with same kind of descriptive errors configuration reports.
And all api endpoints, including ingress, are now self-documenting and added to the main docs website. If you’ve ever wanted to build on top of the data MS creates, you now can!
Note: Other than ingress-based endpoints, the APIs are documented but not fully stable yet. They will be stabilized once the preview ui is released.
BrainzMash Musicbrainz Server
For users who are using Musicbrainz for enhancing their scrobbles: a new, public Musicbrainz mirror is detailed in the docs under the BrainzMash Musicbrainz Server expandable section.
BrainzMash is a community-run pool of self-hosted, read-only hybrid Musicbrainz/Lidarr metadata servers. Requests are made to one, public endpoint that load-balances traffic out to all participating, private servers.
Supplementing your config with this additional Musicbrainz server can increase your transformer speed and throughput.
Security Documentation
A Security section has been added to the docs that details the risks of public access to your Multi-Scrobbler and what you can do to secure it.
Thank to @gnosticJade for the Caddy example and corrections on protected routes!
Other Changelog Highlights
⛰️ Features
- (lastfm) Use p-retry for request retries
- (librefm) Implement rate limiting
- (No Category) Use better rate limiting implementation for musicbrainz
🐛 Bug Fixes
- (atproto) Guard against null rate limit header parsing #689
- (atproto) Detect dupe rkey in batch before persist #689
- (atproto) Implement proper error handling for xrpc calls #689
- (source) Check for player repeat using closed listening session fallback #691
- (tests) Improve network mocking to compensate for libraries with frozen imports
- (ui) Log settings route schema and middleware #693
- (transform): Catch and log errors during console diff generation #684
📚 Documentation
- (No Category) Add security guidance
- (No Category) Adjust caddy snippet with better guidance
- (No Category) Cleanup atproto admonitions
- (No Category) Add BrainzMash musicbrainz guidance
- (No Category) Add openapi docs
🚜 Refactor
- (lastfm) Simplify api error reporting
🧪 Testing
- (lastfm) Test rate limiting and error handling
0.17.1
Changelog
⛰️ Features
- (api) Bump express-typed-router and add global validation failure logging
🐛 Bug Fixes
- (api) Make optional query params optional in route schemas #697 thanks @MFA-G
- (lastfm) Source missing options when initing client #699
- (webscrobbler) Loosen scrobble schema for real-world usage #700
- (No Category) Fix route mounting for /api/callback #699
📚 Documentation
- (No Category) Update BrainzMash language
0.17.2
What’s New?
Listenbrainz User Agent Correction
A misreading of the new Listenbrainz User-Agent requirements introduced in 0.17.0 has been clarified: contact is not required for regular users, only developers.
The contact (File/AIO) and LZ_CONTACT (ENV) config options have been made completely optional with updated language. You should only use these options if you are using a forked version of multi-scrobbler.
Changelog
⛰️ Features
- (listenbrainz) Remove contact guidance and provide default developer info #701
- (source) Remove noisy logging when dropping backlog queue plays
- (No Category) Increase lfm/lz source min polling time to 30 seconds
🐛 Bug Fixes
- (deezer) Add missing parsedFrom
- (lastfm) Add missing parsedFrom to enforce pre-transform/queue check on history sources #702
- (No Category) Fix component id param for auth url route
0.18.0 - New UI
What’s New?
New UI :tada:
Multi-Scrobbler has a new UI that has been completely rewritten from scratch!
This is the same UI that has been in preview for awhile. It now completely replaces the old UI. In conjunction with replacing the UI, all the old UI code and associated API endpoints have been ripped out, removing 4400 lines of code and 15 dependencies.
The UI refresh has been a long time coming: #500 was opened March 24 and was the impetus for both the database and queue overhaul implementation. The kinds of features I wanted to see in the UI would not have been possible without these prerequisites so it has spurred much of the quality upgrades for MS over the last half year.
A HUGE thank you goes out to everyone who has helped test the new UI and all its features, giving me feedback on design and exposing bugs. Especially @4rft5 :crown:
There is still much to improve with the new UI so look out for new features coming soon!
Musicbrainz User Agent
Like with the Listenbrainz User Agent change from 0.17.2, contact is no longer required for configuring the Musicbrainz API for transformer use. If you are using this API you do not need to make any changes to continue using it, but you can remove contact if you want to.
If you only have the contact field for your aio confg and want to remove it, replace it with "enable": true.
Musicbrainz Rate Limit Improvements
0.17.0 introduced BrainzMash as a recommended, supplementary Musicbrainz server to increase transformer throughput but the API rate limiting functionality in Multi-Scrobbler still enforced it, and other non-official servers, with the same rate limits as the official Musicbrainz server.
This functionality has been overhauled to be both optionally configurable and load balanced based on rate limit capacity.
Configuration
Set the max requests within seconds-based time frame in your MB API config:
"apis": [ { // uses default Musicbrainz server https://musicbrainz.org // hard-limited to 1req/s "enable": true }, { // non-official musicbrainz server "url": "my.mb.server", "rate": { "requests": 6, // max number of requests "perTime": 1 // can be made within X number of seconds } }]- Official Musicbrainz server is hard-limited to max 1req/s
- Brainzmash is hard-limited to max 4req/s
- Other, custom Musicbrainz servers default to 1req/s but have no hard limit
Load Balancing
Based on the rate limits you set, or that are hardcoded based on the above, Mutli-Scrobbler will automatically load balance requests between your servers based on which has the most number of requests left for the time span.
Subpath Deployments
For users deploying Multi-Scrobbler behind a reverse proxy, subpaths are now supported using BASE_URL env.
To deploy to https://mydomain.com/multi-scrobbler:
services: multi-scrobbler: # ... environment: - BASE_URL=https://mydomain.com/multi-scrobblerFurther setup is required if you also need self-hosted docs, see Base URL guidance in the installation docs.
Other Changelog Highlights
⛰️ Features
- (db) Check for existing play by parent hash
- (No Category) Add hash regeneration migration
🐛 Bug Fixes
- (dedup) Match backlog plays against live-tracked duration window #709 @plttn
- (dedup) Anchor the duration/listenedFor window on playDate, not the SOC-adjusted date #709 @plttn
- (listenbrainz) Try to coerce tracknumber to number #705
- (source) Don’t mark oldest of set of duplicate inputs as a duplicate of the newer inputs #707
- (spotify) Backfill missing ISRC from real-time polling endpoints #703 @plttn
- (No Category) Regenerate play hash and mbid columns on play entity update db operaiton
📚 Documentation
- (endpointlfm) Add API docs callout
- (endpointlz) Add api doc callouts and summaries to popular apps
- (endpointz) Add tidal hi-fi guidance
- (webscrobbler) Add api docs callout
- (No Category) Remove contact from listenbrainz example config file
- (No Category) Update express-typed-router for pathExample handling
- (No Category) Update development docs to reflect current architecture and implementation
🚜 Refactor
- (No Category) Consolidate repo hydration and existing play function
- (No Category) Consolidate play queuing into abstract component
- (No Category) Consolidate ingress queue pre/post processing into abstract component
🧪 Testing
- (spotify) Add test case for skipping enirch when ISRC is present #703 @plttn
0.18.1
🐛 Bug Fixes
- (plex) Re-add missing component artwork api route #718
- (ui) Add duped to common play state so its available for source play filtering
0.19.0
This is huge release that brings a ton of new Play/Scrobble modification options to MS!
What’s New?
Simpler Config For Teal/Rocksky Sources
App Password has been removed from required config for Teal.fm and Rocksky Sources. It still required for Clients (scrobblers).
Simpler Config for Musicbrainz Transformer
Due to the lack of required config since the change to the Musicbrainz API in 0.18.0, explicit setup of the Musicbrainz transformer is no longer required. It is now built-in to MS with a default configuration that can be used without needing to specify apis or anything in your AIO/ENV config.
You can still configure rate limiting for the default server, or add additional servers, using the same config. It’s just not required to explicitly set anything anymore.
Example
If you currently have this in your config.json
{ // ... "transformers": [ // your empty musicbrainz transform { "type": "musicbrainz", "name": "MyMB", "data": { "apis": [ { "enable": true } ] }, }, // ...maybe more transforms here ], // ...}You can remove the musicbrainz object entirely. If it’s the only entry in your transforms array then you can remove that too.
Then, in your Source/Client transform hooks change this
"playTransform": { "preCompare": [ { "type": "musicbrainz", "name": "MyMB" } ] }to this
"playTransform": { "preCompare": [ { "type": "musicbrainz", "name": "MSDefault" } ] }or entirely remove "name": "MSDefault" and MS will use whatever is configured first.
Rocksky PDS Client
The Rocksky Client has been rewritten to use the sdk agent for scrobbling rather than scrobbling through the Rocksky server.
This change, along with using the new Rocksky matchSong metadata API, means Multi-Scrobbler can now:
- give you immediate feedback on when a scrobble does not meet Rocksky’s standard of quality and
- directly write scrobbles to your PDS repository
This is a huge improvement in user experience over the old “submit scrobble to rocksky server and find out later if it was accepted” approach. Use this in conjunction with the new Rocksky Transformer to get (essentially) guaranteed scrobbles to Rocksky with verifiable writes.
An additional benefit of the PDS client is that MS now supports Now Playing for Rocksky.
Rocksky Transformer
A new scrobble enhancement stage is available, the Rocksky Transformer.
This stage uses Rocksky’s metadata API to match multiple providers (deezer, spotify, etc…) and returns an enriched metadata result that Multi-Scrobbler can use to add Musicbrainz Recording MBID, ISRC, artists, etc… similar to how the Musicbrainz Stage works.
Rocksky’s metadata API is still a work-in-progress so the resulting transformation is limited for now, but it is extremely fast and provides good results!
Use this in conjunction with the Rocksky PDS Client to guarantee scrobbles for Rocksky or as a fast-and-cheap alternative to Musicbrainz.
Spotify Transformer
A new scrobble enhancement stage is available, the Spotify Transformer. All of the research, config options, and initial implementation was contributed by @DigitalStarSys :tada:
This stage works similarly to Rocksky and Musicbrainz transformers in that it is useful for fixing/enriching your scrobble data.
It is advantageous to use, over Musicbrainz, if you prefer a simpler setup with fast API responses and formatted data that is familiar to a mainstream audience (vs. having to configure Musicbrainz and filter through all the different release types).
Note: This transform requires a premium Spotify account (same as the Spotify Source) to use the Spotify API.
Cover Art Archive Transformer and Art Transforms
A new scrobble enhancement stage is available, the Cover Art Archive Transformer. This is standalone version of the Cover Art Archive functionality found in Discord.
When this Stage is used, if your scrobble data has Musicbrainz’ MBIDs for album or release-group and is missing album art, MS will use the CAA API to fetch an album art URL.
Art In Other Transformers
As part of this improvement, both the Rocksky and Spotify Transformer can also populate album art URLs when they are missing. This is enabled automatically as part of Stage Rules and can be explicitly disabled with a rule "art": false.
Device Info for Listenbrainz-Compatible Scrobblers
Thanks to @rpschube reported device info from Sources can now be safely submitted as scrobble data to listenbrainz-compatible clients (Listenbrainz and Koito) without leaking any PII by using an allowlist of scrubbed identifiers to match with real device ids.
See the Scrobbling Device Data expandable section on Listenbrainz and Koito clients to see how this works.
Reduced Docker Build Size
More cleanup done after 0.18.0 revealed a large number of files in docker file build layers that could be easily pruned. A minor rewrite of the Dockerfile has eliminated these and improved build times as well. The cleaned up docker image is 20% smaller than 0.18.1.
Typescript Strict Mode
The entire project has been refactored to compile with typescript strict mode on. This “enables a wide range of type checking behavior that results in stronger guarantees of program correctness” which in laymen’s terms means that ongoing development of Multi-Scrobbler is less likely to introduce new bugs. Just moving to strict mode fixed at least 30+ undiscovered bugs!
Dynamic Transformer Loading and Usage
The logic that loads code for tranformers has been refactored so that all transformers and their dependencies loaded just before a Source/Client actually uses them instead of on startup. This should result in less memory usage for all users but especially those that don’t use any transformers at all.
Additionally, the names of built-in transformers and the order they are used in has been changed to be deterministic and give you more options for which you would like to use, between built-in, ENV configured, or file/aio configured. See the Breaking Changes section below for these changes.
Breaking Changes
Rocksky Auth API Key Removal and Token Deprecation
- The previously-deprecated API Key authentication for Rocksky has been removed
- Access Token authentication for Rocksky is now deprecated in favor of handle/app password auth that uses the new functionality from above.
- Review the Rocksky Client docs for how to set this up
Discord Cover Art Archive Deprecation
The automatic Cover Art Archive usage currently found in the Discord Scrobbler is now deprecated and should be replaced with a Cover Art Archive Transformer.
Example
For Discord Scrobbler with ENV add this to your environmental variables:
DISCORD_TRANSFORMS=coverartarchiveFor File/AIO add the example playTransform block to your config options (or just add the coverartarchive stage config if you already have hooks configured):
{ "name": "MyDiscord", "data": { /* ... */}, // add this "options": { "playTransform": { "preCompare": [ { "type": "coverartarchive", } ] } } }A new Kitchensink example shows a more complex example, here.
Transform Names and Fallbacks
Transform stages are specified by type and name when used in a Source/Client, however the name is optional.
Example
[ { "name": "MySubsonic", "data": { /* ... */}, "options": { "playTransform": { "preCompare": [ { "type": "musicbrainz", "name": "MyMB" // name is optional } ] } } }]Previously when name was omitted MS used the first non-default transformer, then tried the default.
Now, names and order of selection (when name is omitted in configuration) looks like this:
- MS looks for a non-default, non-ENV config first
- MS falls back to ENV config (name is
MSEnv) if it exists - MS falls back to default config (name is
MSDefault) if it exists
Steps to migrate:
- If you only use
named configs that aren’tMSDefaultthere are no changes necessary - If you only use ENV configuration there are no changes necessary
- If you use the
nameMSDefaultbut do not mix ENV transforms with AIO/File config, no changes are necessary - If you use the
nameMSDefaultand also have ENV transforms configured (musicbrainz)- Rename
nametoMSEnv
- Rename
See more about Stage Names and Defaults in this new section in the docs.
Logs Directory for Non-Docker Users
More attention has gone into the Data Directory Standardization work that began in 0.15.0. Now, the directory logs are written to has been standardized, alongside the existing data and config directories.
This change only affects non-docker users.
The logs directory is now determined by, in order of priority:
LOGS_DIRENVLOGS_DIRECTORYENV (used by systemd)- OS-specific standardized path
You can see OS-specific paths here in the documentation.
If you are running multi-scrobbler locally with nodejs you’ll likely want to set LOGS_DIR=./config/logs
Other Changelog Highlights
- fix(endpointlz): Missing ‘import’ enum for listen_type on submit payload #732
- fix: Update dbcomponent at runtime when name changes #721
0.19.1
Changelog
⛰️ Features
- (musicbrainz) Disallow audiobook and music videos with config options #734
🐛 Bug Fixes
- (transform) Add missing musicbrainz default transform #737
0.19.2
Changelog
🐛 Bug Fixes
- (transform) Don’t try to initialize transformers that are already initializing, reduces noisy logging on startup
- (transform) Fix async transform initialization race condition and type/name key case sensitivity #739
v0.20.0
What’s New?
Play Editing and Search
Plays can now (finally) be edited! If your scrobbles fail you can now edit the data to correct it manually before resending it.
A plain text mode is initially used where you can edit everything related to fields and metadata ids
Additionally a search mode is available to search all of the metadata providers you have setup. Musicbrainz, Spotify, and Rocksky can all be searched concurrently. Selecting a result fills in that field, any associated fields (track can set artists + album), and meta ids.
Musicbrainz and Rocksky providers are automatically setup. All providers automatically inherit from your existing transformers. You can also set up preferred providers to only use some of them, or use with alternative configuration.
See the docs for more information.
Tuneshine Client
Tuneshine, a wifi-connected hardware device with an aesthetic, low-count LED display for showcasing the album art of the music you are currently listening to, is now integrated as a Scrobble Client.
Like Discord, it can be used to display Now Playing activity from any of your Sources.
Mixcloud Source
Mixcloud is now integrated as a Scrobble Source thanks to @anandghegde
This source records the mixes you listen to as scrobbles. It supports both free and Pro Mixcloud accounts.
Metadata Overhaul
Previously, metadata like track identifiers from Spotify/Musicbrainz/etc… and album art, were stored alongside a Play but they were not strongly associated with specific parts of the Play. This lead to situations where some functionality would require assertions like “Artist 1 has metadata artist ID 1” but it was not possible to guarantee this.
Now, Play data has been restructured to tightly associate all of this metadata to the field it belongs to. This change does not affect any end-user functionality but it dramatically simplifies developer experience. The Play Editing functionality above would have been much more difficult without this change, for example.
Changelog Highlights
⛰️ Features
- (endpointlz) Allow disabling artist extraction from ingress payload #750 #751
- (deezer) Add ISRC and optional title variant to parsed data #752 #754
- (koito) Add includePlayPosition to koito scrobbler implementation #741 #742
- (spotify) Periodically backfill songs from spotify history api for missed live playback state #735 thanks to @sudbrack
- (No Category) Consolidate SSE for logs into events to reduce connection pool usage for HTTP/1.1
- (No Category) Consolidate global SSE connections with coordinator
🐛 Bug Fixes
- (docs) Resource static asset link broken by trailingSlash #573
- (listenbrainz) Only use original artist_name in naive play if no artists credits were extracted #750
- (stats) Persist lifetime discovered/scrobbled counts across restarts #745 @dylangovender
- (No Category) Improve detection of repeat plays when listen range is close to end of a position player
📚 Documentation
- (discord) Update kasmweb/discord service example
- (metadata) Add editing and metadata search
- (No Category) Add reverse proxy recommendation and tips
🚜 Refactor
- (No Category) Decouple transform manager init
- (No Category) Circular dependency elimination
⚙️ Miscellaneous Tasks
- (No Category) Add docs and devcontainer modifications to ease discord IPC development with kasmcord
v0.20.1
This release bumps the Nodejs version used in the docker image to 24.22.1 in order to include the october security release fixes in the distributed docker image.
You should upgrade as soon as possible in any case, but users who run MS in a way that exposes it to the public internet (API or UI accessible from a non-authenticated URL) are particularly vulnerable as this fixes an exploit for remote code execution. Do not delay upgrading if you are in this scenario.
v0.20.2
This is another security fix that removes compromised eMudhra certificates from the base docker image. #759
v0.20.3 Latest
What’s New?
Docs Moved to Starlight
Documentation has been migrated from Docusaurus to Astro ✨Starlight✨
Docusaurus has been great but Starlight, and Astro being a general static site build system, is more flexible and lighter. Along with the already-strong community plugin support it has enabled more functionality for the MS docs that was not possible with Docusaurus.
The docs now have a fresh look with clearer/more readable styling. Particularly awesome new features:
- Code blocks get walkthroughs, footnotes, inline linking, stylized focus, dynamic placeholders, and much more thanks to starlight-codeblocks
- Changelogs are now built into the public site
Additionally, the developer experience for docs is greatly simplified. Same build commands but many things, like og image generation and open api doc generation, are all built-in now.
Improved Shutdown Behavior
MS now handles all exit signals (SIGNIT and SIGTERM) instead of just SIGINT. On exit signal it now tries to gracefully stop all Source/Clients by letting all currently processing Plays/scrobbles finish before closing the database and shutting down.