Scrobble Modification
Multi-scrobbler configs support the ability to modify scrobble data in an automated fashion by matching and replacing strings in title, artists, and album at many different times in multi-scrobbler's lifecycle.
Why?
You may need to "clean up" data from a Source or before sending to a scrobble Client due to any number of reasons:
- ID3 tags in your music collection are dirty or have repeating garbage IE
[YourMusicSource.com] My Artist - My Title - A Source's service often incorrectly adds data to some field IE
My Artist - My Title (Album Version)when the title should just beMy Title - An Artist you listen to often is spelled different between a Source and a Client which causes duplicate scrobbles
In any scenario where a repeating pattern can be found in the data it would be nice to be able to fix it before the data gets downstream or to help prevent duplicate scrobbling. Multi-scrobbler can help you do this.
Overview
Journey of a Scrobble
First, let's recap the lifecycle of a scrobble in multi-scrobbler:
Sources are the beginning of the journey for a Play (song you've listened to long enough to be scrobblable)
- A Source finds a new valid Play
- The Source compares this new Play to all the other Plays it has already seen, if the Play is unique (title/artist/album/listened datetime) then...
- The Source discovers the Play, adds it to Plays it has seen already, and broadcasts the Play should be scrobbled to all Clients
Scrobble Clients listen for discovered Plays from Sources, then...
- A Client receives a Play from a Source
- The Client compares this Play to all the other scrobbles it has already seen, if the Play is unique (title/artist/album/listened datetime) then...
- The Client scrobbles the Play downstream to the scrobble service and adds it as a Scrobble it has seen already
Lifecyle Hooks
You'll notice there is a pattern above that looks like this:
- Before data is compared
- Data is compared
- After data is compared
These points, during both Source and Client processes, are when you can hook into the scrobble lifecycle and modify it.
TLDR
In more concrete terms this is the structure of hooks within a configuration (can be used in any Source or Client):
[
{
"name": "myLastFm",
"enable": true,
"configureAs": "source",
"data": {
// ...
},
"options": {
"playTransform": {
"preCompare": {/* ... */},
"compare": {/* ... */},
"postCompare": {/* ... */}
}
}
}
]
Hook
For Sources:
preCompare- modify Play data immediately when receivedcompare- temporarily modify Play data when it is being compared to see if Play was already discoveredpostCompare- modify Play data before sending to scrobble Clients
For Clients:
preCompare- modify Play data immediately when receivedcompare- temporarily modify Play data when it is being compared to see if it was already scrobbledpostCompare- modify Play data before scrobbling it to downstream service and adding to already seen scrobbles
Keep in mind that modifying Scrobble/Play data earlier in the lifecycle will affect that data at all times later in the lifecycle.
For example, to modify the track so it's the same anywhere it is processed in multi-scrobbler you only need to modify it in the Source's preCompare hook because all later processes will receive the data with the modified track.
Modification Parts
Each hook (preCompare etc...) is an object that specifies what part of the Play to modify:
{
"title": [/* ... */],
"artists": [/* ... */],
"album": [/* ... */]
}
Expression
and then a list what pattern/replacements (expressions) to use for the modification by using either simple strings or search-replace objects:
[
"badTerm", // remove all instances of 'badTerm'
{
"search": "anotherBadTerm", // and also match all instances of 'anotherBadTerm'
"replace": "goodTerm" // replace with the string 'goodTerm'
}
]
Putting it all together:
[
{
"name": "myLastFm",
"enable": true,
"configureAs": "source",
"data": {
// ...
},
"options": {
"playTransform": {
"preCompare": {
"title": [
[
"badTerm",
{
"search": "badTerm",
"replace": "goodTerm"
}
]
]
},
}
}
}
]
If the value of the field (title, an artist, album) is an empty string after transforming then the field is removed.
Modifications can also be applied to all Sources or all Clients when using the AIO Config config.json by setting playTransform in sourceDefaults or clientDefaults:
Example
{
"sourceDefaults": { // will apply playTransform to all sources
"playTransform": {
"preCompare": {
"title": [
"(Album Version)"
]
}
}
},
"sources": [/* ... */],
"clients": [/* ... */]
}
Compare Hook
The compare hook is slightly different than preCompare and postCompare. It consists of an object where you define which side(s) of the comparison should be modified. It also does not modify downstream data! Instead, the modifications are made only for use in the comparison.
[
{
"name": "myLastFm",
// ...
"options": {
"playTransform": {
"compare": {
"candidate": {/* ... */}, // modify the "new" Play being compared
"existing": {/* ... */}, // modify all "existing" Play/Scrobbles the new Play is being compared against
},
}
}
}
]
Regular Expressions
In addition to plain strings expressions that are matched and removed you can also use Regular Expressions. Write your regex like you normally would, but as a string, and it'll automatically be parsed:
[
"/^\(\w+.com)/i", // matches any string that starts with '(YourMusic.com)' and removes it
{
"search": "/^\(\w+.com)/i", // matches any string that starts with '(YourMusic.com)'
"replace": "[MySite.com]" // replace with the string '[MySite.com]'
}
]
The replace property uses javascript's replace() function and so can use any special string characters.
Conditional Modification
"When" Condition
Top-level hooks and individual rules also support a when key for testing if they should be run.
The when key is similar to a normal modification except:
- the keys accept a single string instead of an array
- the
whenkey data is an array instead of a single object
All parts of an individual when clause must test true to "pass" but if any when clauses pass the hook/rule is processed. Example when data:
{
"when": [
{
"artist": "Elephant Gym", // both of these must match the Play object (AND)
"album": "Dreams" // both of these must match the Play object (AND)
},
// OR
{
"title": "/(Remastered)$/", // both of these must match the Play object (AND)
"album": "Various Artists" // both of these must match the Play object (AND)
}
]
}
More succinctly:
- All parts (
artistalbumtitle) of awhenareANDconditions - All part-objects in the
whenarray areORconditions
Example of top-level hook with when condition
{
// IF the artist is Elephant Gym
// THEN Run preCompare hook ELSE skip this hook
//
// Run search-replace on album
// Run regex title remove
"sourceDefaults": {
"playTransform": {
"preCompare": {
"when": [
{
"artist": "/Elephant Gym/"
}
],
"album": [
{
"search": "Dreams",
"replace": "夢境"
}
],
"title": ["/\s\-\s滾石40\s滾石撞樂隊\s40團拚經典(.+)$/i"]
},
}
}
}
Example of individual rule with when condition
{
// Always run preCompare
//
// On search-replace in title...
// IF artist matches "Elephant Gym"
// THEN Run regex search-replace ELSE skip this rule
//
// Run live|remastered regex remove
"sourceDefaults": {
"playTransform": {
"preCompare": {
"title": [
{
"search": "/\\s\\-\\s滾石40\\s滾石撞樂隊\\s40團拚經典(.+)$/i",
"replace": "",
"when": [
{
"artist": "/Elephant Gym/"
}
]
},
"/(\\s\\-\\s|\\s)(feat\\.(.+)|live|remastered(.+))$/i"
],
}
}
}
}
Top-level Hook array
Top-level hooks can also be an array of hooks. This makes creating multiple scenarios for top-level when-gated hooks easier. All hooks in the array will be run (assuming their when's pass, if they exist) and their input will be the Play object output of the previous hook in the array.
Example
{
"sourceDefaults": {
"playTransform": {
"preCompare": [
// first lifecycle hook of preCompare to run
{
"title": [
{
"search": "something",
"replace": "else unique"
}
]
},
// second lifecycle hook of preCompare to run
{
"title": [
{
"search": "else unique",
"replace": "very demure"
}
]
},
]
}
}
}
Logging
MS can log the output of hook transformations if/when they occur. In the playTransform object of a Source/Client config use log:
"log": true=> Output original play + final transformed output of last hook in the array"log": "all"=> Output original play + final transformed output of each hook in the array
{
"name": "myThing",
"data": {/*...*/},
"options": {
"playTransform": {
"preCompare": {/*...*/},
"log": true
}
}
}
Examples
Remove phrase from Title in all new Plays
Removes the phrase (Album Version) from the Title of a Play
Example
{
"sourceDefaults": {
"playTransform": {
"preCompare": {
"title": [
"(Album Version)"
]
}
}
}
}
Remove all parenthesized content from the end of a title
Example
[
{
"name": "myLastFm",
// ...
"options": {
"playTransform": {
"compare": {
"candidate": {
"title": [
"/(\(.+\))\s*$/"
]
},
"existing": {
"title": [
"/(\(.+\))\s*$/"
]
},
},
}
}
}
]
Rename misspelled artist in all new Plays
Example
{
"sourceDefaults": {
"playTransform": {
"preCompare": {
"artists": [
{
"search": "Boz Skaggs",
"replace": "Boz Scaggs"
}
]
}
}
}
}
Remove "Various Artists" albums in all new Plays
Example
{
"sourceDefaults": {
"playTransform": {
"preCompare": {
"album": [
{
"search": "Various Artists",
"replace": ""
}
]
}
}
}
}