Skip to content

Instantly share code, notes, and snippets.

@LouisAmon
Last active November 4, 2023 08:34
Show Gist options
  • Save LouisAmon/181c874ad4ee06f600f424045d508d8d to your computer and use it in GitHub Desktop.
Save LouisAmon/181c874ad4ee06f600f424045d508d8d to your computer and use it in GitHub Desktop.
Convert a `pandas.DataFrame` to a generator of ElasticSearch actions to be passed to a bulk helper
def pandas_to_elasticsearch(df):
"""
Generator that converts `pandas.DataFrame` rows into ElasticSearch actions
"""
# ElasticSearch will raise an exception on `NaN` values
df = df.dropna()
for i, row in df.iterrows():
action = {
'_op_type': 'index',
'_index': 'my_index',
'_type': 'my_doctype',
#'_id': row['some_pk'],
**row.to_dict()
}
yield action
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment