Last.fm duplicate scrobble deleter

lastfm-remove-duplicates.js

var elements = Array.from(document.querySelectorAll('.js-link-block'))
elements.map(function (element) {
  var nameElement = element.querySelector('.chartlist-name')
  return nameElement && nameElement.textContent.replace(/\s+/g, ' ').trim()
}).forEach(function (name, i, names) {
  if (name !== names[i + 1]) return
  var deleteButton = elements[i].querySelector('[data-ajax-form-sets-state="deleted"]')
  if (deleteButton) deleteButton.click()
  location.reload()
})

lastfm-remove-duplicates.md

doesn't work anymore and i deleted my last.fm - maybe someone posted a working version in the comments below!

---

Last.fm duplicate scrobble deleter

This script serves to delete duplicate scrobbles (i.e. the same song scrobbled multiple times in a row) from your Last.fm library. To use it, paste the script into your browser's console (or the address bar, but prefix the script with javascript:) while logged in and in the own library.

Why would I need this?

Your scrobbler might just have decided to scrobble every song hundreds of times and you can't really remove those scrobbles efficiently. Or, if you're like me, you might have accidentally installed multiple scrobbler extensions at the same time - wondering why multiple scrobbles appear for every song played at a time - and you want to clear them after finding the issue.

Using this script still doesn't necessarily make the process quick, since Last.fm only shows a specific number of scrobbles which can be removed on each page in your library.

How-to (create a bookmarklet)

1. Copy the following script URL into your clipboard

javascript:var elements=Array.from(document.querySelectorAll('.js-link-block'));elements.map(function(a){var b=a.querySelector('.chartlist-name');return b&&b.textContent.replace(/\s+/g,' ').trim()}).forEach(function(a,b,c){if(a===c[b+1]){var d=elements[b].querySelector('[data-ajax-form-sets-state="deleted"]');d&&d.click(),location.reload()}});

2. Right-click your browser's bookmark bar and click "Add page..."
3. Give the bookmark a name, like "Remove duplicates"
4. Paste the script you copied in step 1 into the bookmark's URL.
5. Save the bookmark
6. Open your Last.fm account's library while being logged in (https://www.last.fm/user/_/library).
7. Opening your bookmark will trigger the script to execute.
8. Repeat clicking the bookmark as long as there are no duplicates left.

How-to (alternative, one-time way)

1. Copy the script

var elements=Array.from(document.querySelectorAll('.js-link-block'));elements.map(function(a){var b=a.querySelector('.chartlist-name');return b&&b.textContent.replace(/\s+/g,' ').trim()}).forEach(function(a,b,c){if(a===c[b+1]){var d=elements[b].querySelector('[data-ajax-form-sets-state="deleted"]');d&&d.click(),location.reload()}});

2. Open your Last.fm account's library while being logged in (https://www.last.fm/user/_/library).
3. Type javascript: into the address bar and paste the script directly after it. Or, open the dev tools and paste the script into the console.
4. Press enter. This will remove all duplicates on the current page.
5. Let the site reload (invoked by the script).
6. Repeat pasting the script and pressing enter if more duplicates appear at the bottom.
7. If needed, go to the next page of your library repeat the steps as of step 3.

Why do I need to repeat executing the script?

The script will only remove what's visible on the current library page. After entries were deleted, more duplicates may appear at the bottom. This might happen multiple times. Once one page is finally duplicate-free, the process can be repeated for next pages.

lastfm-remove-duplicates.min.js

var elements=Array.from(document.querySelectorAll('.js-link-block'));elements.map(function(a){var b=a.querySelector('.chartlist-name');return b&&b.textContent.replace(/\s+/g,' ').trim()}).forEach(function(a,b,c){if(a===c[b+1]){var d=elements[b].querySelector('[data-ajax-form-sets-state="deleted"]');d&&d.click(),location.reload()}});

This is really helpful. My 2 issues thus far though are:

1. This doesn't delete scrobbles that aren't consecutively listed in the library. For instance, back when I was using my iPod I had instances of Song A being scrobbled at a certain timestamp and at another timestamp + x minutes later. I believe this occurred when I sync my iPod to my laptop. Since I was also listening to other songs, what's listed in my library would be Song A - timestamp, Song B - timestamp + x. Song A - timestamp + y, etc.

2. Unfortunately, I accidentally deleted my scrobbles of my all-time played track. I was just testing if this would delete the scrobbles with the same timestamp. But since there were no other listed tracks there (because I visited the scrobbles of a specific track), it deleted everything in that page. =/

Anyway, kudos to you @sk22. I had to create an account here in github to personally acknowledge your efforts, especially that I've been a last.fm user for more than a decade now.

http://meaw.kagechiyo.net/web/RemoveDuplicatedTrack.eng.html

My version of this. If it finds any duplicates, it will wait 5s before reloading the page. Otherwise, it will go to the previous page of results so that you can run it again. Ideally, you'd start somewhere in the middle/end of of your play history and work backwards.

var found = 0; var num = 5; var els = Array.from(document.querySelectorAll('.js-link-block'));
els.map(function (el) { var nmEl = el.querySelector('.chartlist-name'); return nmEl && nmEl.textContent.replace(/\s+/g, ' ').trim(); }).forEach(function (name, i, names) {
    if (!names.slice(i + 1, i + 1 + num).includes(name)) return;
    var delBtn = els[i].querySelector('[data-ajax-form-sets-state="deleted"]');
    if (delBtn) { delBtn.click(); found++; }
});
if (found > 0) setTimeout(function() { location.reload(); }, 5000);
else window.location = window.location.pathname + replaceQueryParam('page', gup('page', window.location.href) - 1, window.location.search)

function replaceQueryParam(param, newval, search) { var regex = new RegExp("([?;&])" + param + "[^&;]*[;&]?"); var query = search.replace(regex, "$1").replace(/&$/, ''); return (query.length > 2 ? query + "&" : "?") + (newval ? param + "=" + newval : ''); }
function gup( name, url ) { if (!url) url = location.href; name = name.replace(/[\[]/,"\\\[").replace(/[\]]/,"\\\]"); var regexS = "[\\?&]"+name+"=([^&#]*)"; var regex = new RegExp( regexS ); var results = regex.exec( url ); return results == null ? null : results[1]; }

@gms8994 thank you for your version!!

@sk22 I'm trying to get your original script working and having no luck with either the bookmarklet or one-timer. I'm wondering if you can help. Using Chrome Version 71.0.3578.98 -- could it be setting in newer Chrome builds that I need to look at?

Duplicate scrobbling is out-of-control recently!

This useful script seems to have stopped working :-(

It works, but I think you reload the page too soon, so some of the click events do not have time to be executed and stuff that was marked as deleted is still there after reloading the page. Maybe add a delay before reloading the page? The best would be to use Promises, but I don't know how it would work with Last.fm click events.

So what should I copy into the bookmark's URL if I want it to delete duplicates sometimes scrobbled in an alternating pattern?

This broke, right? document.querySelectorAll('.js-link-block') doesn't find anything anymore.

I fixed it (included additions from @huw):

var num = 5
var sections = Array.from(document.getElementsByTagName("tbody"))
sections.forEach(function (section) {
    var elements = Array.from(section.rows)
    var names = elements.map(function (element) {
      var nameElement = element.querySelector('.chartlist-name')
      return nameElement && nameElement.textContent.replace(/\s+/g, ' ').trim()
    })
    
    names.forEach(function (name, i, names) {
      if (!names.slice(i + 1, i + 1 + num).includes(name)) return
      var deleteButton = elements[i].querySelector('[data-ajax-form-sets-state="deleted"]')
      if (deleteButton) deleteButton.click()
      location.reload()
    })
})

Working as on today
(bookmark option)

javascript:jQuery('.dropdown-menu-clickable-item[data-ajax-form-sets-state="deleted"]').each(function(_, b) {b.click();});location.reload();

Omg thankkk youuu for this!! Duplicate scrobbles have been a bane for awhile now, as an OCD last.fm user and someone that hardly ever listens to tracks on repeat. This is working great. Discovered this via ViolentMonkey's "Find scripts for this site" search feature.

This script helped me a lot! Thanks.

glad it did, @7ma7X. you're welcome!

Changes needed with the last.fm-redesign:

var num = 5;
var sections = Array.from(document.getElementsByTagName("tbody"));
sections.forEach(function (section) {
    var elements = Array.from(section.rows);
    var titles = elements.map(function (element) {
      var nameElement = element.querySelector('.chartlist-name');
      var artistElement = element.querySelector('.chartlist-artist');
      return nameElement && artistElement && nameElement.textContent.replace(/\s+/g, ' ').trim()+':'+artistElement.textContent.replace(/\s+/g, ' ').trim();
    });

    titles.forEach(function (title, i, titles) {
      if (!titles.slice(i + 1, i + 1 + num).includes(title)) return;
      var deleteButton = elements[i].querySelector('[data-ajax-form-sets-state="deleted"]');
      if (deleteButton) deleteButton.click();
    });
});

The last one doesn#t work on Chrome :/

sadly i don't use last.fm anymore (because i was concerned about the publicity of my data), so i can't really help anymore. i hope that you people can help each other and share updated scripts etc. sorry!

That last one worked for me using the Console feature from Chrome's Inspect:

var num = 5
var sections = Array.from(document.getElementsByTagName("tbody"))
sections.forEach(function (section) {
var elements = Array.from(section.rows)
var titles = elements.map(function (element) {
var nameElement = element.querySelector('.chartlist-name')
var artistElement = element.querySelector('.chartlist-artist')
return nameElement && artistElement && nameElement.textContent.replace(/\s+/g, ' ').trim()+':'+artistElement.textContent.replace(/\s+/g, ' ').trim()
})

titles.forEach(function (title, i, titles) {
  if (!titles.slice(i + 1, i + 1 + num).includes(title)) return
  var deleteButton = elements[i].querySelector('[data-ajax-form-sets-state="deleted"]')
  if (deleteButton) deleteButton.click()
})

})

Hello everyone,

I've been using your scripts for a long time now. After trying out the more automated version from @gms8994 I was more than happy. A few weeks ago though the script didn't work anymore. The updated version from @CennoxX of the former script from @mattsson is working however. I've decided to merge these two variants together so that the script is updated and working and also automated, so that you don't have to control and reload manually.

There wasn't much further development needed, so big thanks to the original authors of these scripts and also @sk22

It would be great if this could be implemented into a chrome (and firefox) extension but I'm not experienced enough to create something like that.

var found = 0;
var num = 5;
var sections = Array.from(document.getElementsByTagName("tbody"));

function replaceQueryParam(param, newval, search) {
	var regex = new RegExp("([?;&])" + param + "[^&;]*[;&]?");
	var query = search.replace(regex, "$1").replace(/&$/, '');
	return (query.length > 2 ? query + "&" : "?") + (newval ? param + "=" + newval : '');
};
 
function gup( name, url ) {
	if (!url) url = location.href;
	name = name.replace(/[\[]/,"\\\[").replace(/[\]]/,"\\\]");
	var regexS = "[\\?&]"+name+"=([^&#]*)";
	var regex = new RegExp( regexS );
	var results = regex.exec( url );
	return results == null ? null : results[1];
};

sections.forEach(function (section) {
	var els = Array.from(section.rows);
	var names = els.map(function (el) {
		var nmEl = el.querySelector('.chartlist-name');
		var artEl = el.querySelector('.chartlist-artist');
		return nmEl && artEl && nmEl.textContent.replace(/\s+/g, ' ').trim()+':'+artEl.textContent.replace(/\s+/g, ' ').trim();
	});
	
	names.forEach(function (name, i, names) {
		if (!names.slice(i + 1, i + 1 + num).includes(name)) return;
		var delBtn = els[i].querySelector('[data-ajax-form-sets-state="deleted"]');
		if (delBtn) { delBtn.click(); found++; };
	});
});

if (found > 0) setTimeout(function() {
	location.reload();
}, 5000);
else window.location = window.location.pathname + replaceQueryParam('page', gup('page', window.location.href) - 1, window.location.search);

As a one liner for use in a bookmark: (just save this URL as a bookmark)

javascript:var found = 0; var num = 5; var sections = Array.from(document.getElementsByTagName("tbody"));  function replaceQueryParam(param, newval, search) { var regex = new RegExp("([?;&])" + param + "[^&;]*[;&]?"); var query = search.replace(regex, "$1").replace(/&$/, ''); return (query.length > 2 ? query + "&" : "?") + (newval ? param + "=" + newval : ''); };   function gup( name, url ) { if (!url) url = location.href; name = name.replace(/[\[]/,"\\\[").replace(/[\]]/,"\\\]"); var regexS = "[\\?&]"+name+"=([^&#]*)"; var regex = new RegExp( regexS ); var results = regex.exec( url ); return results == null ? null : results[1]; };  sections.forEach(function (section) { var els = Array.from(section.rows); var names = els.map(function (el) { var nmEl = el.querySelector('.chartlist-name'); var artEl = el.querySelector('.chartlist-artist'); return nmEl && artEl && nmEl.textContent.replace(/\s+/g, ' ').trim()+':'+artEl.textContent.replace(/\s+/g, ' ').trim(); });  names.forEach(function (name, i, names) { if (!names.slice(i + 1, i + 1 + num).includes(name)) return; var delBtn = els[i].querySelector('[data-ajax-form-sets-state="deleted"]'); if (delBtn) { delBtn.click(); found++; }; }); });  if (found > 0) setTimeout(function() { location.reload(); }, 5000); else window.location = window.location.pathname + replaceQueryParam('page', gup('page', window.location.href) - 1, window.location.search);

Edit: Added semicolons for bookmark usage compatibility. Of course this still works in the console.

@muhdiboy thank you!!

@muhdiboy Thank you much. The bookmark is working like a champ

How do I use this script?

@RoelJanssens well, mine not at all, but maybe @muhdiboy's is still working:

As a one liner for use in a bookmark: (just save this URL as a bookmark)

javascript:var found = 0; var num = 5; var sections = Array.from(document.getElementsByTagName("tbody"));  function replaceQueryParam(param, newval, search) { var regex = new RegExp("([?;&])" + param + "[^&;]*[;&]?"); var query = search.replace(regex, "$1").replace(/&$/, ''); return (query.length > 2 ? query + "&" : "?") + (newval ? param + "=" + newval : ''); };   function gup( name, url ) { if (!url) url = location.href; name = name.replace(/[\[]/,"\\\[").replace(/[\]]/,"\\\]"); var regexS = "[\\?&]"+name+"=([^&#]*)"; var regex = new RegExp( regexS ); var results = regex.exec( url ); return results == null ? null : results[1]; };  sections.forEach(function (section) { var els = Array.from(section.rows); var names = els.map(function (el) { var nmEl = el.querySelector('.chartlist-name'); var artEl = el.querySelector('.chartlist-artist'); return nmEl && artEl && nmEl.textContent.replace(/\s+/g, ' ').trim()+':'+artEl.textContent.replace(/\s+/g, ' ').trim(); });  names.forEach(function (name, i, names) { if (!names.slice(i + 1, i + 1 + num).includes(name)) return; var delBtn = els[i].querySelector('[data-ajax-form-sets-state="deleted"]'); if (delBtn) { delBtn.click(); found++; }; }); });  if (found > 0) setTimeout(function() { location.reload(); }, 5000); else window.location = window.location.pathname + replaceQueryParam('page', gup('page', window.location.href) - 1, window.location.search);

just save this as a bookmark (that is, create a bookmark of whatever page and change the url to the text above) - opening the bookmark will trigger the script.
or, you can copy everything after the javascript: part into your browser console. (press ctrl + shift + j to open it)

@RoelJanssens well, mine not at all, but maybe @muhdiboy's is still working:

As a one liner for use in a bookmark: (just save this URL as a bookmark)

javascript:var found = 0; var num = 5; var sections = Array.from(document.getElementsByTagName("tbody"));  function replaceQueryParam(param, newval, search) { var regex = new RegExp("([?;&])" + param + "[^&;]*[;&]?"); var query = search.replace(regex, "$1").replace(/&$/, ''); return (query.length > 2 ? query + "&" : "?") + (newval ? param + "=" + newval : ''); };   function gup( name, url ) { if (!url) url = location.href; name = name.replace(/[\[]/,"\\\[").replace(/[\]]/,"\\\]"); var regexS = "[\\?&]"+name+"=([^&#]*)"; var regex = new RegExp( regexS ); var results = regex.exec( url ); return results == null ? null : results[1]; };  sections.forEach(function (section) { var els = Array.from(section.rows); var names = els.map(function (el) { var nmEl = el.querySelector('.chartlist-name'); var artEl = el.querySelector('.chartlist-artist'); return nmEl && artEl && nmEl.textContent.replace(/\s+/g, ' ').trim()+':'+artEl.textContent.replace(/\s+/g, ' ').trim(); });  names.forEach(function (name, i, names) { if (!names.slice(i + 1, i + 1 + num).includes(name)) return; var delBtn = els[i].querySelector('[data-ajax-form-sets-state="deleted"]'); if (delBtn) { delBtn.click(); found++; }; }); });  if (found > 0) setTimeout(function() { location.reload(); }, 5000); else window.location = window.location.pathname + replaceQueryParam('page', gup('page', window.location.href) - 1, window.location.search);

just save this as a bookmark (that is, create a bookmark of whatever page and change the url to the text above) - opening the bookmark will trigger the script.
or, you can copy everything after the javascript: part into your browser console. (press ctrl + shift + j to open it)

Does it delete duplicates if they play like 10 times in the same minute or just any duplicates like if I loop a song all day will it delete all of them as well?

Does it delete duplicates if they play like 10 times in the same minute or just any duplicates like if I loop a song all day will it delete all of them as well?

Yes, this will delete those, even if they are a day apart.
It is possible though to implement a check on date. I even tried that before I posted my version, but I didn't have the necessary skills for that.

@muhdiboy thank you!

But this script only removes the duplicates from the page that is currently loaded right? So I must click ALL my history pages one by one to get them all removed, that will take a looooong time.

But this script only removes the duplicates from the page that is currently loaded right? So I must click ALL my history pages one by one to get them all removed, that will take a looooong time.

@RoelJanssens This script will load the next page (in terms of newer page) automatically if it can't find another duplicate. So you just have to start executing this script on the page you want to start deleting duplicates. Then after reload you execute it again and so forth. For an easy execution I recommend creating a bookmark as described above.

Of course it is still possible to set this script up in an environment to automate the whole process.

I have more than 300 pages of history and I don't have the knowlegde to automate your script to go through my whole archive.

I still think it's unbelievable that Last.FM doesn't detect duplicates automatically.

Hello everyone,

I've saved up more than 100 pages of history and decided to implement an automation to this process. With the help of Tampermonkey (Chrome/Chromium) or Greasemonkey (Firefox) you can execute the duplicate deletion automatically on page load.

The Gist of this UserScript is on my profile and not a comment here, because it doesn't suit this Gist anymore. It is based on my latest version of the script.
Of course many thanks to all contributors on this Gist and @sk22. I've decided to include everyone involved in the description of the UserScript.

I've tested this UserScript with Tampermonkey and implemented some features to stop the script, because otherwise it will endlessly run.

Check out here: https://gist.github.com/muhdiboy/a293cbff355af750e3b8f45ec816d1f1
I'm open to any suggestions or improvements. To be clear though, I'm not a developer and quite the novice in terms of Javascript.

The Gist is currently missing a install and user guide, I will add it tomorrow. It's not rocket science, just install your choice of UserScript Manager/Tool and import the script, either manually or directly with the link: https://gist.github.com/muhdiboy/a293cbff355af750e3b8f45ec816d1f1#file-lastfm-automated-remove-duplicates-js
After that enable the UserScript, go to your last.fm library, head to the page where to start the deletion and finally reload the page to initiate the automation. At the end of the loop you will receive a pop-up to remind you to disable the script.

Kind regards to everyone and have fun!

---

fyi. @RoelJanssens