Md. Sany Ahmed sany2k8

Document stores in PostgreSQL

Notes for software engineering meeting presentation

What

mid to late 2000s: appearance of Document stores / NoSQL databases such as Mongo, Couch
Relational DBs now have support for document data: JSON in MySQL, JSON and JSONB in PostgreSQL
Focus on JSONB in Postgres (most full featured)

PostgreSQL is Enough

Background and Cron Jobs

Commit Message Format

Each commit message consists of a header, a body and a footer. The header has a special format that includes a type, a scope and a subject:

<type>(<scope>): <subject>
<BLANK LINE>
<body>
<BLANK LINE>

Usage

(Create a symlink pytest for py.test)

pytest [options] [file_or_dir] [file_or_dir] ...

Help:

Upload files to AWS

Make sure you already have S3 bucket, access key and secret key before go through this notes.

How to connect to AWS?

Boto3 allows Python developers to write software that makes use of services like Amazon S3 and Amazon EC2.

Step 1: Install `boto3` with pip

pip install boto3

	import phonenumbers
	from pydantic.validators import strict_str_validator

	class PhoneNumber(str):
	"""Phone Number Pydantic type, using google's phonenumbers"""

	@classmethod
	def __get_validators__(cls):
	yield strict_str_validator
	yield cls.validate

	from typing import Any
	from typing import Dict
	from typing import Set
	from typing import Type

	from pydantic import SecretStr
	from pydantic.utils import update_not_none


	class Password(SecretStr):

	# Add in ~/.bashrc or ~/.bash_profile
	function parse_git_branch () {
	git branch 2> /dev/null \| sed -e '/^[^]/d' -e 's/ \(.*\)/(\1)/'
	}

	RED="\[\033[01;31m\]"
	YELLOW="\[\033[01;33m\]"
	GREEN="\[\033[01;32m\]"
	BLUE="\[\033[01;34m\]"
	NO_COLOR="\[\033[00m\]"

	import asyncio
	import aiohttp
	import time


	async def gather_with_concurrency(n, *tasks):
	semaphore = asyncio.Semaphore(n)

	async def sem_task(task):
	async with semaphore:

	# best practice: linux
	nano ~/.pgpass
	:5432::username:password
	chmod 0600 ~/.pgpass

	# best practice: windows
	edit %APPDATA%\postgresql\pgpass.conf
	:5432::username:password

	# linux