Skip to content

Instantly share code, notes, and snippets.

@do-me
Created November 24, 2023 16:18
Show Gist options
  • Select an option

  • Save do-me/28cb01fbb34dd371471ed67787eb0a11 to your computer and use it in GitHub Desktop.

Select an option

Save do-me/28cb01fbb34dd371471ed67787eb0a11 to your computer and use it in GitHub Desktop.
Pandas function to generate unique hash for each row
import hashlib
# Function to generate hash for each row
def generate_hash(row):
hash_object = hashlib.sha256()
# Convert each element in the row to a string and update the hash object
for value in row.values:
hash_object.update(str(value).encode('utf-8'))
# Return the hexadecimal representation of the hash
return hash_object.hexdigest()[:30]
# Apply the function to each row and create a new column 'hash'
df['hash'] = df.apply(generate_hash, axis=1)
df
# Useful: check duplicate rows
df[df.duplicated(keep="first")]
# Keep only first duplicate
df.drop_duplicates("OI",keep="first").reset_index(drop=True)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment