Sandeep Shantharam machbio

PostgreSQL is Enough

Background and Cron Jobs

Lecture 1: Introduction to Research — [📝Lecture Notebooks] [▶️Video]
Lecture 2: Introduction to Python — [📝Lecture Notebooks] [▶️Video]
Lecture 3: Introduction to NumPy — [📝Lecture Notebooks] [▶️Video]
Lecture 4: Introduction to pandas — [📝Lecture Notebooks] [▶️Video]
Lecture 5: Plotting Data — [📝Lecture Notebooks] [[▶️Vide

There are a myriad of benefits of smaller PRs with smaller line change deltas:

easier for reviewers, not only in sheer amount of code to review, but also in the ability to provide useful feedback or start a discussion around different architecture patterns
lower chances of bugs creeping in
simpler to refactor and iterate upon
encourages the code author to think more iteratively (e.g. what is the API I am trying to add right now? What will it look like when I'm done? Is what I'm doing extensible? Will it be suitable for future extensions, etc.)
more synonymous with a CI flow; it is better to ship multiple small things that can easily be cherry-picked or reverted than it is to ship big PRs that are harder to debug and roll back

One pitfall to small PRs is people feeling like they can't move ahead or build on top of code that is up for review.

Rationale

This snippet is a sample showing how to implement CloudWatch Logs streaming to ElasticSearch using terraform. I wrote this gist because I didn't found a clear, end-to-end example on how to achieve this task. In particular, I understood the resource "aws_lambda_permission" "cloudwatch_allow" part by reading a couple of bug reports plus this stackoverflow post.

The js file is actually the Lambda function automatically created by AWS when creating this pipeline through the web console. I only added a endpoint variable handling so it is configurable from terraform.

kops: https://github.com/kubernetes/kops
1. Getting Started Guide: https://github.com/kubernetes/kops/blob/master/docs/aws.md
2. Installing Kubernetes on AWS with kops: https://kubernetes.io/docs/getting-started-guides/kops/
3. Mulit-master Kubernetes Cluster on AWS with kops: http://blog.arungupta.me/multimaster-kubernetes-cluster-amazon-kops/
4. Booting Kubernetes on Amazon Elastic Compute with kops: https://deis.com/docs/workflow/quickstart/provider/aws/boot/
5. Setting up an HA Kubernetes Cluster in AWS with private topology with kops 1.5.1: https://www.nivenly.com/kops-1-5-1/
6. Kubernetes on AWS: https://daemonza.github.io/2017/01/15/kubernetes-on-aws/
7. Your 2nd day with Kubernetes on AWS: https://www.nivenly.com/2nd-hour/
Tectonic (Terraform): http://github.com/coreos/tectonic-installer
1. Graphical installer: https://coreos.com/tectonic/docs/latest/install/aws/

Download docker-compose.yml to dir named sentry
Change SENTRY_SECRET_KEY to random 32 char string
Run docker-compose up -d
Run docker-compose exec sentry sentry upgrade to setup database and create admin user
(Optional) Run docker-compose exec sentry pip install sentry-slack if you want slack plugin, it can be done later
Run docker-compose restart sentry
Sentry is now running on public port 9000

https://docs.docker.com/compose/compose-file/#env_file

	const { spawn } = require('child_process');

	const videoUrls = ["YOUR-TIKTOK-VIDEO-THAT-YOU-WANT-TO-DOWNLOAD"];

	async function downloadVideo(videoUrl) {
	return new Promise((resolve, reject) => {
	const ytDlpProcess = spawn('yt-dlp', [videoUrl]);

	ytDlpProcess.stdout.on('data', (data) => {
	console.log(`yt-dlp stdout: ${data}`);

	pipeline {
	agent { node { label 'swarm-ci' } }

	environment {
	TEST_PREFIX = "test-IMAGE"
	TEST_IMAGE = "${env.TEST_PREFIX}:${env.BUILD_NUMBER}"
	TEST_CONTAINER = "${env.TEST_PREFIX}-${env.BUILD_NUMBER}"
	REGISTRY_ADDRESS = "my.registry.address.com"

	SLACK_CHANNEL = "#deployment-notifications"