Liem Le Hoang Duc liemle3893

LInux

Three system configuration parameters must be set to support a large number of open files and TCP connections with large bursts of messages. Changes can be made using the /etc/rc.d/rc.local or /etc/sysctl.conf script to preserve changes after reboot.

1. /proc/sys/fs/file-max: The maximum number of concurrently open files.

fs.file-max = 1000000

2. /proc/sys/net/ipv4/tcp_max_syn_backlog: Maximum number of remembered connection requests, which are still did not receive an acknowledgment from connecting client. The default value is 1024 for systems with more than 128Mb of memory, and 128 for low memory machines.

net.ipv4.tcp_max_syn_backlog = 3240000

3. /proc/sys/net/core/somaxconn: Limit of socket listen() backlog, known in userspace as SOMAXCONN. Defaults to 128.

net.core.somaxconn = 3240000

PSQL

Magic words:

psql -U postgres

Some interesting flags (to see all, use -h or --help depending on your psql version):

-E: will describe the underlaying queries of the \ commands (cool for learning!)
-l: psql will list all databases and then exit (useful if the user you connect with doesn't has a default database, like at AWS RDS)

Initialize

gitflow	git
`git flow init`	`git init`
	`git commit --allow-empty -m "Initial commit"`
	`git checkout -b develop master`

Connect to the remote repository

Optimizing postgresql table for more than 100K inserts per second

Create UNLOGGED table. This reduces the amount of data written to persistent storage by up to 2x.
Set WITH (autovacuum_enabled=false) on the table. This saves CPU time and IO bandwidth on useless vacuuming of the table (since we never DELETE or UPDATE the table).
Insert rows with COPY FROM STDIN. This is the fastest possible approach to insert rows into table.
Minimize the number of indexes in the table, since they slow down inserts. Usually an index on time timestamp with time zone is enough.
Add synchronous_commit = off to postgresql.conf.
Use table inheritance for fast removal of old data:

The repository for the assignment is public and Github does not allow the creation of private forks for public repositories.

The correct way of creating a private frok by duplicating the repo is documented here.

For this assignment the commands are:

Create a bare clone of the repository. (This is temporary and will be removed so just do it wherever.)

git clone --bare [email protected]:usi-systems/easytrace.git

MicroService Proxy Gateway Solutions

Kong, Traefik, Caddy, Linkerd, Fabio, Vulcand, and Netflix Zuul seem to be the most common in microservice proxy/gateway solutions. Kubernetes Ingress is often a simple Ngnix, which is difficult to separate the popularity from other things.

Github Star Trend:

This is just a picture of this link from March 2, 2019

Originally, I had included some other solution

HOWTO: Clean Up Nomad Data Directory

Issue

When attempting to remove all of the data in the Nomad data directory, several directories and files are unable to be deleted. Many messages are logged to the console like:

rm: cannot remove ‘alloc/736f61b9-d7dc-cb73-0dd1-76b1b2ba032d/nomad-ui/secrets’: Device or resource busy
rm: cannot remove ‘alloc/ddcf5a78-5497-f4a4-a101-221fc4e0180b/fabio/alloc’: Device or resource busy
rm: cannot remove ‘alloc/ddcf5a78-5497-f4a4-a101-221fc4e0180b/fabio/secrets’: Device or resource busy

Fluent-bit rocks

A short survey of log collection options and why you picked the wrong one. 😜

Who am I? Where am I from?

I'm Steve Coffman and I work at Ithaka. We do JStor (academic journals) and other stuff. How big is it?

Number	what it means
101,332,633	unique visitors in 2017

	var mediaJSON = { "categories" : [ { "name" : "Movies",
	"videos" : [
	{ "description" : "Big Buck Bunny tells the story of a giant rabbit with a heart bigger than himself. When one sunny day three rodents rudely harass him, something snaps... and the rabbit ain't no bunny anymore! In the typical cartoon tradition he prepares the nasty rodents a comical revenge.\n\nLicensed under the Creative Commons Attribution license\nhttp://www.bigbuckbunny.org",
	"sources" : [ "http://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4" ],
	"subtitle" : "By Blender Foundation",
	"thumb" : "images/BigBuckBunny.jpg",
	"title" : "Big Buck Bunny"
	},
	{ "description" : "The first Blender Open Movie from 2006",
	"sources" : [ "http://commondatastorage.googleapis.com/gtv-videos-bucket/sample/ElephantsDream.mp4" ],

	# === Optimized my.cnf configuration for MySQL/MariaDB (on cPanel/WHM servers) ===
	#
	# by Fotis Evangelou, developer of Engintron (engintron.com)
	#
	# ~ Updated September 2024 ~
	#
	#
	# The settings provided below are a starting point for a 8-16 GB RAM server with 4-8 CPU cores.
	# If you have different resources available you should adjust accordingly to save CPU, RAM & disk I/O usage.
	#