Skip to content
Last.fm is disabling API Keys for non-paid accounts December 1st.Learn More

Changelog

0.18.0 - New UI

What’s New?

New UI :tada:

Multi-Scrobbler has a new UI that has been completely rewritten from scratch!

This is the same UI that has been in preview for awhile. It now completely replaces the old UI. In conjunction with replacing the UI, all the old UI code and associated API endpoints have been ripped out, removing 4400 lines of code and 15 dependencies.

The UI refresh has been a long time coming: #500 was opened March 24 and was the impetus for both the database and queue overhaul implementation. The kinds of features I wanted to see in the UI would not have been possible without these prerequisites so it has spurred much of the quality upgrades for MS over the last half year.

A HUGE thank you goes out to everyone who has helped test the new UI and all its features, giving me feedback on design and exposing bugs. Especially @4rft5 :crown:

There is still much to improve with the new UI so look out for new features coming soon!

image

Musicbrainz User Agent

Like with the Listenbrainz User Agent change from 0.17.2, contact is no longer required for configuring the Musicbrainz API for transformer use. If you are using this API you do not need to make any changes to continue using it, but you can remove contact if you want to.

If you only have the contact field for your aio confg and want to remove it, replace it with "enable": true.

Musicbrainz Rate Limit Improvements

0.17.0 introduced BrainzMash as a recommended, supplementary Musicbrainz server to increase transformer throughput but the API rate limiting functionality in Multi-Scrobbler still enforced it, and other non-official servers, with the same rate limits as the official Musicbrainz server.

This functionality has been overhauled to be both optionally configurable and load balanced based on rate limit capacity.

Configuration

Set the max requests within seconds-based time frame in your MB API config:

"apis": [
{
// uses default Musicbrainz server https://musicbrainz.org
// hard-limited to 1req/s
"enable": true
},
{
// non-official musicbrainz server
"url": "my.mb.server",
"rate": {
"requests": 6, // max number of requests
"perTime": 1 // can be made within X number of seconds
}
}
]
  • Official Musicbrainz server is hard-limited to max 1req/s
  • Brainzmash is hard-limited to max 4req/s
  • Other, custom Musicbrainz servers default to 1req/s but have no hard limit

Load Balancing

Based on the rate limits you set, or that are hardcoded based on the above, Mutli-Scrobbler will automatically load balance requests between your servers based on which has the most number of requests left for the time span.

Subpath Deployments

For users deploying Multi-Scrobbler behind a reverse proxy, subpaths are now supported using BASE_URL env.

To deploy to https://mydomain.com/multi-scrobbler:

services:
multi-scrobbler:
# ...
environment:
- BASE_URL=https://mydomain.com/multi-scrobbler

Further setup is required if you also need self-hosted docs, see Base URL guidance in the installation docs.

Other Changelog Highlights

⛰️ Features

  • (db) Check for existing play by parent hash
  • (No Category) Add hash regeneration migration

🐛 Bug Fixes

  • (dedup) Match backlog plays against live-tracked duration window #709 @plttn
  • (dedup) Anchor the duration/listenedFor window on playDate, not the SOC-adjusted date #709 @plttn
  • (listenbrainz) Try to coerce tracknumber to number #705
  • (source) Don’t mark oldest of set of duplicate inputs as a duplicate of the newer inputs #707
  • (spotify) Backfill missing ISRC from real-time polling endpoints #703 @plttn
  • (No Category) Regenerate play hash and mbid columns on play entity update db operaiton

📚 Documentation

  • (endpointlfm) Add API docs callout
  • (endpointlz) Add api doc callouts and summaries to popular apps
  • (endpointz) Add tidal hi-fi guidance
  • (webscrobbler) Add api docs callout
  • (No Category) Remove contact from listenbrainz example config file
  • (No Category) Update express-typed-router for pathExample handling
  • (No Category) Update development docs to reflect current architecture and implementation

🚜 Refactor

  • (No Category) Consolidate repo hydration and existing play function
  • (No Category) Consolidate play queuing into abstract component
  • (No Category) Consolidate ingress queue pre/post processing into abstract component

🧪 Testing

  • (spotify) Add test case for skipping enirch when ISRC is present #703 @plttn

0.17.2

What’s New?

Listenbrainz User Agent Correction

A misreading of the new Listenbrainz User-Agent requirements introduced in 0.17.0 has been clarified: contact is not required for regular users, only developers.

The contact (File/AIO) and LZ_CONTACT (ENV) config options have been made completely optional with updated language. You should only use these options if you are using a forked version of multi-scrobbler.

Changelog

⛰️ Features

  • (listenbrainz) Remove contact guidance and provide default developer info #701
  • (source) Remove noisy logging when dropping backlog queue plays
  • (No Category) Increase lfm/lz source min polling time to 30 seconds

🐛 Bug Fixes

  • (deezer) Add missing parsedFrom
  • (lastfm) Add missing parsedFrom to enforce pre-transform/queue check on history sources #702
  • (No Category) Fix component id param for auth url route

0.17.1

Changelog

⛰️ Features

  • (api) Bump express-typed-router and add global validation failure logging

🐛 Bug Fixes

  • (api) Make optional query params optional in route schemas #697 thanks @MFA-G
  • (lastfm) Source missing options when initing client #699
  • (webscrobbler) Loosen scrobble schema for real-world usage #700
  • (No Category) Fix route mounting for /api/callback #699

📚 Documentation

  • (No Category) Update BrainzMash language

0.17.0

What’s New?

Preview UI Feature Parity

The new UI has feature parity with the existing UI, plus so many more improvements. :tada:

0.17.0 will be the last release before the new UI replaces the existing one. If you have design feedback or things aren’t working as expected please report them in this issue. Also would be nice if you left a comment letting me know that everything is working as expected and you like the new design! Feedback either way is good.

image

Listenbrainz User-Agent Changes

Listenbrainz has updated their User-Agent requirements to require a point of contact, similar to how Musicbrainz works.

The 0.16.5 release that addressed Listenbrainz User-Agent updates is no longer sufficient. Use the updated Listenbrainz for multi-scrobbler to update your configs to include a contact.

Use the 0.17.2 release. Your listenbrainz configuration does not need to be modified.

Improved Source Play Processing

The base Source class has been restructured to use a queue workflow, similar to the base Client class, that enables new functionality and improves all existing Play processing. Some highlights:

  • Backlog play processing no longer blocks Source startup
  • Ingress based Sources can now send batch scrobbles and get an immediate response without needing to wait for all Plays to process
  • Source plays that fail to process can now be retried (same way as Client plays)
  • Sources can process existing, queued Plays even if polling has failed

Optimized Existing Play Checks

When polling Listening History based Sources MS parses all returned history, every time, to look for changes in the list.

Previously, if you had preCompare transforms these needed to run for all history before MS could check if each Play already existed.

Now, MS stores a hash of the original data, before transforms, and uses this to check for existing Plays before transforming. This optimization can potentially save hundreds of API calls to musicbrainz and wasted CPU cycles transforming Plays that would then be thrown away immediately.

Play Events

The events shown on the Timeline for a Play in the Preview UI were previously hacked together from many different locations in a Play’s data.

This data has now been restructured as a first-class data structure persisted to the database. What this means for you is that the Timeline will now show a more complete, user-friendly list of events and that future event types are trivial to create.

Api Validation and Docs

Thanks to @Mini-Sylar ’s excellent express-typed-router library MS now has fully-typed api routes.

Ingress-based Sources like Webscrobbler and Listenbrainz/Last.fm Endpoints get exhaustive request body/query validation with same kind of descriptive errors configuration reports.

And all api endpoints, including ingress, are now self-documenting and added to the main docs website. If you’ve ever wanted to build on top of the data MS creates, you now can!

Note: Other than ingress-based endpoints, the APIs are documented but not fully stable yet. They will be stabilized once the preview ui is released.

BrainzMash Musicbrainz Server

For users who are using Musicbrainz for enhancing their scrobbles: a new, public Musicbrainz mirror is detailed in the docs under the BrainzMash Musicbrainz Server expandable section.

BrainzMash is a community-run pool of self-hosted, read-only hybrid Musicbrainz/Lidarr metadata servers. Requests are made to one, public endpoint that load-balances traffic out to all participating, private servers.

Supplementing your config with this additional Musicbrainz server can increase your transformer speed and throughput.

Security Documentation

A Security section has been added to the docs that details the risks of public access to your Multi-Scrobbler and what you can do to secure it.

Thank to @gnosticJade for the Caddy example and corrections on protected routes!

Other Changelog Highlights

⛰️ Features

  • (lastfm) Use p-retry for request retries
  • (librefm) Implement rate limiting
  • (No Category) Use better rate limiting implementation for musicbrainz

🐛 Bug Fixes

  • (atproto) Guard against null rate limit header parsing #689
  • (atproto) Detect dupe rkey in batch before persist #689
  • (atproto) Implement proper error handling for xrpc calls #689
  • (source) Check for player repeat using closed listening session fallback #691
  • (tests) Improve network mocking to compensate for libraries with frozen imports
  • (ui) Log settings route schema and middleware #693
  • (transform): Catch and log errors during console diff generation #684

📚 Documentation

  • (No Category) Add security guidance
  • (No Category) Adjust caddy snippet with better guidance
  • (No Category) Cleanup atproto admonitions
  • (No Category) Add BrainzMash musicbrainz guidance
  • (No Category) Add openapi docs

🚜 Refactor

  • (lastfm) Simplify api error reporting

🧪 Testing

  • (lastfm) Test rate limiting and error handling

0.16.5

This is a small release due to needing to get Listenbrainz’s new request requirements out to stable! A larger release with new UI functionality will be coming soon.

What’s New?

Listenbrainz User-Agent

A recent botnet attack has forced Listenbrainz to require all traffic to include a User-Agent and implement retry backoffs.

Multi-Scrobbler already implements backoffs but did not have a proper User-Agent in place. This release introduces a UA for all requests that should restore Listenbrainz connectivity.

teal.fm production lexicon

teal.fm has moved their lexicon to the production namespace and so this release of multi-scrobbler updates the repository records to use the new namespace.

Other Changes

  • @halkeye fixed the code for generating “Edit this page” links in the docs :tada:

0.16.4

The jumble of patches released this week in the aftermath of 0.16.0 were indicative of a poor release.

I take pride in making sure Multi-scrobbler is not one of those projects you are hesitant to upgrade because you don’t know if “this will be the one that breaks unexpectedly”. 0.16.0 was not up to that quality and for that, I apologize.

This release should hopefully make things right.

What’s New?

ENV Testing and Stability

The ENV improvements made in 0.16.0 are here to stay but to address the quality issues mentioned above this release focused on building a proper test suite to validate and test all ENV keys with real, valid data:

  • tests were added for all sources/clients that
    • generate all ENVs for each source/client with real data derived from the ENV schema shape, using zocker
    • assign these ENVs to the actual environment and then build the actual source/client, verifying no config errors and that exactly one source/client of that type exist

This now puts ENV testing on the same level of testing as file/aio config and should catch any errors, like those fixed in 0.16.1 - 0.16.3, before they get to the release stage.

Full Changelog

⛰️ Features

  • (No Category) Relax configureAs requirements for AIO configs
  • (No Category) Strongly-type internal events #666
  • (No Category) Add ENV config test suite
  • (No Category) Update json schema to draft-2020-12
  • (No Category) Normalize domain:port url input
  • (No Category) Better validation for normalized config urls

🐛 Bug Fixes

  • (discord) Fix discord ENV parsing validation
  • (No Category) Check for ENV prefix with startsWith instead of includes #668
  • (No Category) Narrow env config key collection to include prefix underscore #668
  • (No Category) Hand-rolled env data fixtures for some edge cases
  • (No Category) Hand-roll applemusic env fixture data

📚 Documentation

  • (No Category) Add more guidance for File config example/editor
  • (No Category) Hand-roll discord ENV updates

0.16.3

Changelog

🐛 Bug Fixes

  • Fix incorrect prefix not being used for env key collection #668
  • (apple music) include library-songs in recent tracks #667

📚 Documentation

  • Improve scenario guidance for lfm/lz endpoints

0.16.2

Changelog

🐛 Bug Fixes

  • (source): Fix missing id/uid on cached recent plays #662 #665

0.16.1

Changelog

🐛 Bug Fixes

  • (No Category) Add missing configureAs for ENV parsed components #661

0.16.0

What’s New?

Reduced Memory Usage

Moving to Zod in 0.15.0 provided a big reduction in memory footprint that has been further improved in this release by moving individual Source/Client validation to be dynamically loaded. This means MS will not load validation/types for Source/Clients you do not use.

Users with a minimal number of Sources/Clients should see the largest impact from this.

Benchmarks for MS with no Sources/Clients

Startup

Version Runtime Validation Context % Memory Usage Change
0.14.2 tsx ts gen + ajv -
0.15.0 node zod + eager validation -13%
0.16.0 node zod + dynamic validation -14%

Stabilized (after 4 minutes of running)

Version Runtime Validation Context % Memory Usage Change
0.14.2 tsx ts gen + ajv -
0.15.0 node zod + eager validation -22%
0.16.0 node zod + dynamic validation -27%

ENV Config UX Improvements

The move to Zod in 0.15.0 also opened the door for big improvements to ENV Config experience for the end-user:

Self-Documenting

ENV variables now share, or inherit, schema from file/aio config so ENV descriptions match those found elsewhere. The ENV config for Source/Clients in the docs is now self-documenting (generated from schema) to prevent typos and make common ENV easier to extend.

Improved Validation

ENV validation at startup is now more straightforward and useful. Instead of mapping variables to file (KOITO_TOKEN => data.token) and then printing an error

✖ Invalid input: expected string, received undefined
→ at data.token

Variables are now directly validated to give you a clearer idea of what is missing or wrong

✖ Invalid input: expected string, received undefined
→ at KOITO_TOKEN

Source/Client ID Enforcement

To further improve DX/UX the ID requirements first introduced in 0.14.0 are now mandatory.

Previously, MS would log a warning and use your Source/Client name as the ID, as noted in the 0.14.0 upgrade guide. Now, it reports a configuration error if no ID is present.

To migrate, fully complete the Configuring IDs section from the 0.14.0 migration guide.

Other Changelog Highlights

⛰️ Features

  • (lfm endpoint) Implement better api emulation #656
  • (source) Add Ingress SOT type
  • (now playing) Buffer valid updates for endpoint sources #659
  • (ui) Improve UX for theme switcher
  • (ui) Implement basic settings page
  • (No Category) Improve location wording for config errors

🐛 Bug Fixes

  • (source): Fix player cleanup for ingress sources #652 #591

📚 Documentation

  • (No Category) Use italicize-bold text style for required env properties instead of separate column
  • (No Category) Update env descriptions for id, name, and enable
  • (No Category) Update now playing for clients
  • (No Category) Fix config.json example
  • (No Category) Add lastfm endpoint alternative guidance
  • (No Category) Update configuration types language