find illegal unicode charaters in a csv file

gistfile1.txt

Relevant unicode characters reference:  https://www.unicode.org/charts/PDF/U2000.pdf

how to type an unicode character in Linux:
    CTRL + Shift + U2014 Enter
how to type an unico character in Windoze:  (not sure here !)
    NumLock ON  Alt Down 0151 

file_with_illegal_characters.csv
--------------------------------

c1,c2,c3,c4
a,b,c,a-b
a,b,x—x,c
a,—x,b,bb
rr,rr,rr,rr
t t,t t,tt,t

find_illegal_characters_in_csv.py
---------------------------------

import argparse
import csv
from typing import Dict

def find_illegal_characters(file_obj, targets: Dict[str, str]):
    """
    Searches a CSV file for specific target characters in a single pass.
    
    :param file_obj: An open file object
    :param targets: A dictionary mapping { 'character': 'Display Name' }
    """
    reader = csv.reader(file_obj)
    try:
        headers = next(reader)
    except StopIteration:
        print("The file is empty.")
        return

    found_any = False

    for row_num, row in enumerate(reader, start=2): # Start at 2 (row 1 is headers)
        for col_idx, cell in enumerate(row):
            """
            If a row in the CSV has fewer columns than the header row (jagged CSV rows),
            headers[col_idx] would throw an IndexError  and crash the script.
            I added a safe fallback: headers[col_idx] if col_idx < len(headers) else f"Column {col_idx + 1}
            """
            col_name = headers[col_idx] if col_idx < len(headers) else f"Column {col_idx + 1}"
            
            for char, name in targets.items():
                if char in cell:
                    found_any = True
                    print(f"Found {name} in Row {row_num}, Column '{col_name}'")
                    print(f"  Cell content: {cell}\n")
                    
    if not found_any:
        target_names = ", ".join(targets.values())
        print(f"No {target_names} found in the file.")

def main():
    parser = argparse.ArgumentParser(description="File to be investigated for illegal characters")
    parser.add_argument("filename", type=str, help="Name of the file to read")
    args = parser.parse_args()

    # Define characters to search for here
    targets = {
        '\u2014': 'EM DASH',
        '\u00A0': 'NO-BREAK SPACE'
    }

    try:
        """
        Explicit UTF-8 Encoding: Added encoding="utf-8" to the open() function. On Windows, Python defaults to cp1252 encoding.
        If a non-breaking space is saved in a specific way, Windows might fail to read it properly or throw an error.
        Forcing UTF-8 makes the script behave consistently across all operating systems
        """
        with open(args.filename, "r", encoding="utf-8") as fin:
            find_illegal_characters(fin, targets)
        print("[*]Done!")
            
    except FileNotFoundError:
        print(f"Error: File not found: {args.filename}")
    except UnicodeDecodeError:
        print(f"Error: Could not read '{args.filename}'. Ensure it is a valid UTF-8 text file.")

if __name__ == '__main__':
    main()

run the script
--------------

boring@rat:~/Desktop$ python3 find_illegal_characters_in_csv.py file_with_illegal_characters.csv
Found em dash in Row 3, Column 'c3'
  Cell content: x—x

Found em dash in Row 4, Column 'c2'
  Cell content: —x

Found NO-BREAK SPACE in Row 6, Column 'c1'
  Cell content: t t

Notes
-----

1. Are you sure it's an Em Dash?
Sometimes what looks like an em dash is actually a different character. If the code above returns "No em dashes found" but you can clearly see a long dash in Excel, you might be dealing with:

     En dash: – (U+2013) - Slightly shorter than an em dash.
     Horizontal Bar: ― (U+2015)
     Minus Sign: − (U+2212)