Normalize unicode file names (converts UTF-8 NFD to NFC). Required by macOS clients through AFP/NFS/SMB. Tested on Synology DSM 6.2 with built-in Python 2.7.12.

README.md

NFCFN.py

Normalize unicode file names (converts UTF-8 NFD to NFC).
Required by macOS clients through AFP/NFS/SMB.

Tested on Synology DSM 6.2 with built-in Python 2.7.12.

Usage

# 1. Activate SSH on your NAS

# 2. On your computer, open a new console/terminal and connect to your server:
ssh [email protected]

# 3. Go to the directory where you want saving the `nfcfn.py` script:
cd /volume1/YourSharedFolder/PathToScript

# 4. Download the latest version:
wget https://gist.githubusercontent.com/xthezealot/9a65fac2c7b916c4d84e66188bf06bec/raw/nfcfn.py

# 5. Run it with Python to check the result:
python nfcfn.py -cr /volume1/YourSharedFolder

# 6. When you are sure, add the `-p` flag to effectively rename the files:
python nfcfn.py -crp /volume1/YourSharedFolder

nfcfn.py

#!/usr/bin/env python
# -*- coding: utf-8 -*-
"""Normalize unicode file names."""

from __future__ import unicode_literals

from argparse import ArgumentParser
from os import rename, walk
from os.path import exists, isfile, join, split
from sys import version_info
from unicodedata import normalize


def bytes_saved(old, new):
    """Print difference of bytes between old an new string."""
    diff = len(new) - len(old)
    s = "[\033["
    if diff < 0:
        s += "32m" + str(diff)
    elif diff > 0:
        s += "31m+" + str(diff)
    else:
        s += "34m="
    s += " byte"
    if abs(diff) > 1:
        s += "s"
    return s + "\033[0m]"


def norm(root, file, form, proceed):
    """Do the normalization."""
    normed = (
        normalize(form, file).replace("/", "／").replace("\\", "＼").replace(":", "：")
    )
    if file != normed:
        old = join(root, file)
        new = join(root, normed)
        if exists(new):
            print("%s \033[31mcannot be renamed as\033[0m %s \033[31malready exists\033[0m" % (old, normed))
        else:
            print("%s  ▶︎  %s  %s" % (old, normed, bytes_saved(file, normed)))
            if proceed:
                rename(old, new)


def main():
    """Normalize unicode file names."""
    parser = ArgumentParser(description="Normalize unicode file names.")
    parser.add_argument("source", help="the source file or directory")
    parser.add_argument(
        "-c",
        "--compatibility",
        action="store_true",
        help='normalize with compatibility (ex: "fi"' ' becomes "fi")',
    )
    parser.add_argument("-p", "--proceed", action="store_true", help="rename files")
    parser.add_argument(
        "-r",
        "--recursive",
        action="store_true",
        help="go through directories recursively",
    )
    args = parser.parse_args()

    if version_info < (3,):
        args.source = unicode(args.source, "utf8")

    norm_form = "NFKC" if args.compatibility else "NFC"

    # Source is a file
    if isfile(args.source):
        head, tail = split(args.source)
        norm(head, tail, norm_form, args.proceed)
    # Source is a directory
    else:
        for root, dirs, files in walk(args.source):
            for d in dirs:
                norm(root, d, norm_form, args.proceed)
            for f in files:
                norm(root, f, norm_form, args.proceed)
            if not args.recursive:
                break


if __name__ == "__main__":
    main()

I am not familiar with python. Sorry that I did not read your code.
After reading your code, I now understand that I can avoid the problem by not adding -c option to the argument now.

The compatible version, NFKA normalization, may introduce invalid characters. I think NFC normalization is a better option for file name conversion.
There should be at least 3 characters that are commonly used in Asian languages will become problematic after converted to NFKC form. They are:

• ／
• ＼
• ：

I would recommend omitting option -c from the arguments to avoid possible problems.

I added exceptions for these 3 characters.

But --compatibility is still useful for some of us who just don't like having ambiguous special chars in their filenames.

Thx a lot, you saved my day (even my week)

Thx a lot, you saved my day (even my week)

Mais avec plaisir ! 👍

Arthur, you are my hero! After more than one year search to solve these sticky problem I found your script (and the description how to treat) – and it works brilliant. Just one point took me time to figure out:

• If there is a folder with a name needing to be normalized, the script do, but stopp to look up its content. For every level of subfolders the script needs to be startet again.

This is not a problem, because it's quickly done (and I have mostly just three levels of subfolders). Just you have to be aware of this point.
So, thank you very much…

Just an other observation: If a folder after normalization will get the same name as an allready existing one, the script will stop and I get the message:

Traceback (most recent call last):
File "nfcfn.py", line 82, in
main()
File "nfcfn.py", line 74, in main
norm(root, d, norm_form, args.proceed)
File "nfcfn.py", line 39, in norm
rename(old, new)
OSError: [Errno 39] Directory not empty

thanks man. you literally saved me one week of my life. why on earth is it that difficult to have a universal standard for filename parameters in the year 2021... pfff
thanks

Thanks !!!!! This is the end of "error 43" :D

Please help. I get this error:
Traceback (most recent call last):
File "/volume1/homes/Wenner Peter/Drive/Script/nfcfn.py", line 82, in
main()
File "/volume1/homes/Wenner Peter/Drive/Script/nfcfn.py", line 67, in main
if isfile(args.source):
File "/usr/lib/python2.7/genericpath.py", line 37, in isfile
st = os.stat(path)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xf3' in position 37: ordinal not in range(128)

i want to run this in synology task scheduler daily

which python should i install?
in the package center i see:

• Python Module - TRIED
• Python3 - NOW INSTALLED

Community:

• Python - TRIED
• Phyton 3.8
• Python3

Any suggestion?

Thanks for the feedback.

@mauthner, going recursively after renaming a directory would require another strategy which I don't actually have time to code. Sorry.
For the "already existing directory", I added a warning. The script won't fail anymore over this one.

@rennewpeter, the built-in Python version (2.7.18 if your DSM is up to date) should be sufficient. Python 3 is not required.
The error simply shows that you provided a directory path that cannot be parsed by Python.
Maybe your terminal or your Synology uses an uncommon encoding. It's difficult to investigate without details.
But I already edited the script to use the unicode function instead of decode. You can redownload it and retry.

@ArthurWhite thx. i tested the script with python3 and dsm7 in task scheduler and its worked. The script run daily. So you saved my life, i have 25TB of albums, with this synology drive search working again. Respect

Szívesen! 👍

@ArthurWhite

I have recently run the script and noticed some more side effect of the -c option which I think most people do not want.
For example:

• /volume2/Media/shared/Movies/Alien³ (1992) ▶︎ Alien3 (1992) [= byte]
• /volume2/Media/shared/Movies/ＤＥＳＴＩＮＹ (2017) ▶︎ DESTINY (2017) [= byte]

I am not challenging your recommendation. I am genuinely curious and would like to learn more. Could you enlighten me about why the -c option is a good idea?

Simply because with some fonts, different strings may look exactly the same although they are in fact not "equal".

So for example, if you have a file that contains fi (1 char) in its name, and you search for fi (2 chars), you may not find this file.
And different versions of a same OS can have a different behaviors.
If you're not aware of this kind of subtlety, it can be tricky. It's not always as obvious as ³ vs. 3.

That's why I recommend sticking as close as possible to ASCII characters for file names.
Even spaces in a file path can cause problems with some apps, I experienced it, but that's too much of a rule…

So yes, it's an old-school recommendation but still a "good practice" when heterogenous OS and file systems share the same data, which is the purpose of a NAS.

But if only you have access to your NAS and always through the same device, OK, the -c option is not a big win.

Thank you very much for your explanation. I understand now. 👍🏼

I think I better omit the -c option in my use case though. It causes confusion to a few tools I use.