AG mzpqnxow

This small subclass of the Pandas sqlalchemy-based SQL support for reading/storing tables uses the Postgres-specific "COPY FROM" method to insert large amounts of data to the database. It is much faster that using INSERT. To acheive this, the table is created in the normal way using sqlalchemy but no data is inserted. Instead the data is saved to a temporary CSV file (using Pandas' mature CSV support) then read back to Postgres using Psychopg2 support for COPY FROM STDIN.

	"""
	Patched version to support PostgreSQL
	(original version: https://github.com/pydata/pandas/blob/v0.13.1/pandas/io/sql.py)

	Adapted functions are:
	- added _write_postgresql
	- updated table_exist
	- updated get_sqltype
	- updated get_schema

	#!/usr/bin/env python3
	#
	# Parse https://testssl.sh/openssl-iana.mapping.html or a copy/paste of
	# the content from a browser, drop it in a line-based JSON file
	#
	# This lets you quickly translate from IANA<->OpenSSL<->int16 which is
	# useful for unusual people :>
	#
	# Requirements: lxml, pandas>=1.2
	# If Pandas is too old, you will get:

	#!/usr/bin/env python3
	"""Small function to use PostgreSQL MASKLEN() with SQLALchemy

	If you work regularly with SQLAlchemy and know a cleaner way to do this, please
	leave a comment on this Gist! Thanks

	(C) 2019, mzpqnxow, BSD 3-Clause
	"""

	from typing import Union

	import logging
	import cStringIO
	import csv

	DEBUG = False

	def data2csv(data):
	si = cStringIO.StringIO()
	cw = csv.writer(si, delimiter='\t',lineterminator="\n")
	for row in data:

	import logging.handlers
	import lzma
	import os
	"""
	This can be done a few different ways. You can also implement it as a standalone
	function, instnatiate a standard logging.handlers.RotatingFileHandler, and then
	assign your own rotator() function to to the `rotator` attribute of the instance

	I think it's more convenient encapsulating it like this, though
	-AG

	def kv_pair_list_to_map(self, obj, name_key, value_key='value'):
	"""Return a dictionary from a list of fixed name/value keyed dicts

	This handles the common "xml-style" storage of values in JSON
	It also merges into lists as appropriate
	Example Usage (for below input)
	===============================

	map_obj = kv_pair_list_to_map(obj, 'var_name', value_key='value')

	def crackdn(dn_str):
	"""Crack a DN string into a dict

	'C=US, ST=New Jersey, L=Newark, O=The Dump, Inc., CN=www.thedump.newark'

	... Becomes ...

	{
	"C": "US",
	"ST": "New Jersey",

	import codecs
	import json
	import datetime
	from uuid import UUID

	class UniversalEncoder(json.JSONEncoder):
	def default(self, obj):

	if isinstance(obj, (datetime.datetime, datetime.date, datetime.time)):
	return obj.isoformat()