Identify & fix Duplicate IDs in Statamic 3.

readme.md

Duplicate ID Identifier & Fixer for Statamic 3

Instructions:

• Copy the contents of this gist's console.php into routes/console.php
• Run php artisan identify-duplicates

At the start of the script, it will ask you if you want to replace any identied duplicate IDs. If you choose to do so, new Stache IDs will be generated and the Laravel cache will be cleared.

console.php

<?php // copy everything after this line

Artisan::command('identify-duplicates', function () {
    $duplicates = [];
    $shouldFixDuplicates = $this->confirm('Should Duplicate IDs be replaced with fresh IDs?');

    $items = collect(\Illuminate\Support\Facades\File::allFiles(__DIR__.'/../content'))
        ->filter(function ($file) {
            return $file->isFile();
        })
        ->map(function (SplFileInfo $file) {
            $data = \Statamic\Facades\YAML::parse(file_get_contents($file->getRealPath()));

            return [
                'path' => $file->getRealPath(),
                'stache_id' => isset($data['id']) ? $data['id'] : null,
            ];
        })
        ->reject(function ($item) {
            return is_null($item['stache_id']);
        });

    $items->each(function ($item) use (&$duplicates, $items) {
        $itemsWithCurrentId = $items->where('stache_id', $item['stache_id']);

        if ($itemsWithCurrentId->count() > 1) {
            if (array_key_exists($item['stache_id'], $duplicates)) {
                $duplicates[$item['stache_id']] = array_merge($duplicates[$item['stache_id']], [
                    $item['path'],
                ]);
            } else {
                $duplicates[$item['stache_id']] = [
                    $item['path'],
                ];
            }
        }
    });

    foreach ($duplicates as $id => $items) {
        $this->error("Duplicate ID Found: {$id}");

        foreach ($items as $item) {
            $this->line($item);

            if ($shouldFixDuplicates) {
                $contents = file_get_contents($item);
                $contents = str_replace($id, \Statamic\Facades\Stache::generateId(), $contents);
                file_put_contents($item, $contents);
            }
        }

        $this->line('');
    }

    if ($shouldFixDuplicates) {
        Artisan::call('cache:clear');
        $this->info('Duplicate IDs have been replaced and your Cache has been cleared. Have a good day!');
    }
});

Hmm, this tool won't fix your collection's yaml file. It'll only fix the IDs in the entry files themselves. I could look into it if I have enough time this weekend.

its ok, I found the dupes.

I tweaked this slightly to also check for duplicate users:

Artisan::command('identify-duplicates', function () {
    $duplicates = [];
    $shouldFixDuplicates = $this->confirm('Should Duplicate IDs be replaced with fresh IDs?');

    $items = collect()
        ->merge(\Illuminate\Support\Facades\File::allFiles(__DIR__.'/../content'))
        ->merge(\Illuminate\Support\Facades\File::allFiles(__DIR__.'/../users'))
        ->filter(function ($file) {
            return $file->isFile();
        })
        ->map(function (SplFileInfo $file) {
            $data = \Statamic\Facades\YAML::parse(file_get_contents($file->getRealPath()));

            return [
                'path' => $file->getRealPath(),
                'stache_id' => isset($data['id']) ? $data['id'] : null,
            ];
        })
        ->reject(function ($item) {
            return is_null($item['stache_id']);
        });

    $items->each(function ($item) use (&$duplicates, $items) {
        $itemsWithCurrentId = $items->where('stache_id', $item['stache_id']);

        if ($itemsWithCurrentId->count() > 1) {
            if (array_key_exists($item['stache_id'], $duplicates)) {
                $duplicates[$item['stache_id']] = array_merge($duplicates[$item['stache_id']], [
                    $item['path'],
                ]);
            } else {
                $duplicates[$item['stache_id']] = [
                    $item['path'],
                ];
            }
        }
    });

    foreach ($duplicates as $id => $items) {
        $this->error("Duplicate ID Found: {$id}");

        foreach ($items as $item) {
            $this->line($item);

            if ($shouldFixDuplicates) {
                $contents = file_get_contents($item);
                $contents = str_replace($id, \Statamic\Facades\Stache::generateId(), $contents);
                file_put_contents($item, $contents);
            }
        }

        $this->line('');
    }

    if ($shouldFixDuplicates) {
        Artisan::call('cache:clear');
        $this->info('Duplicate IDs have been replaced and your Cache has been cleared. Have a good day!');
    }
});

Ah thanks! I'll add that to the main version of the script after work.

I am trying this today on a multisite and it produces this error:

 Statamic\Yaml\ParseException

  The YAML value does not appear to be valid UTF-8.

  at storage/statamic/tmp/yaml/d985620e8891be4a9502b9a1b190e805:-1

Is this script still actual?

It probably means you have some invalid characters in one of your YAML files. This script is no longer relevant since you can now resolve duplicate IDs in the Control Panel.