Skip to content

Instantly share code, notes, and snippets.

View bthaman's full-sized avatar

Bill Thaman bthaman

  • HDR
  • Austin
View GitHub Profile
@bthaman
bthaman / df_to_sql_fast.py
Created January 17, 2022 19:08
High-performance Pandas dataframe to SQL Server - uses pyodbc executemany with fast_executemany = True. This is an alternative to out-of-the-box Pandas df_to_sql, which is slow for larger dataframes.
def df_to_sql_fast(df, table_name, numeric_columns, date_columns, append_or_replace, conn):
"""
Appends or overwrites a SQL Server table
using data from a Pandas DataFrame.
Submits df records at once for faster performance
compared to df_to_sql.
Parameters:
df (DataFrame): df used to create/append table
table_name (str): Name of existing SQL Server table