jshen28

HTTP Streaming (or Chunked vs Store & Forward)

The standard way of understanding the HTTP protocol is via the request reply pattern. Each HTTP transaction consists of a finitely bounded HTTP request and a finitely bounded HTTP response.

However it's also possible for both parts of an HTTP 1.1 transaction to stream their possibly infinitely bounded data. The advantages is that the sender can send data that is beyond the sender's memory limit, and the receiver can act on

README

The example below creates a TCP server listening on a stream (i.e. SOCK_STREAM) socket. A similar approach can be followed to create a UDP server on a datagram (i.e. SOCK_DGRAM) socket. See man systemd.socket for details.

An example server

Create an simple echo server at /opt/foo/serve.py.

Python 3 extension example

Build

python3 setup.py build

Output: build/lib.macosx-10.11-x86_64-3.5/hello.cpython-35m-darwin.so

Run

README

The example below creates a TCP server listening on a stream (i.e. SOCK_STREAM) socket. A similar approach can be followed to create a UDP server on a datagram (i.e. SOCK_DGRAM) socket. See man systemd.socket for details.

An example server

Create an simple echo server at ~/tmp/foo/serve.py.

QEMU arm64 cloud server emulation

This is basically a rehash of an original post on CNXSoft - all credit (particularly for the Virtio device arguments used below) belongs to the author of that piece.

Download the latest uefi1.img image. E.g. ubuntu-16.04-server-cloudimg-arm64-uefi1.img from https://cloud-images.ubuntu.com/releases/16.04/release/

Download the UEFI firmware image QEMU_EFI.fd from https://releases.linaro.org/components/kernel/uefi-linaro/latest/release/qemu64/

Determine your current username and get your current ssh public key:

How to launch QEMU from command line without libvirt with macvtap and vhost support

This sets up a host local bridge with a macvlan interface for VM to host communication. The macvtap is setup with vhost support.

The command line options to note are

-netdev tap,fd=3,id=hostnet0,vhost=on,vhostfd=4 3<>$"$tapdev" 4<>/dev/vhost-net -device virtio-net-pci,netdev=hostnet0,id=net0,mac=$(< /sys/class/net/testtap/address)

This sets up two different fd's. The first for macvtap and the second for vhost-net.

Simple vsock setup for QEMU

Configuration

Host Kernel: rawhide 4.13.0-0.rc6.git4.2.fc28.x86_64 (on Fedora 24)

QEMU is mainline built from sources: QEMU emulator version 2.10.50 (v2.10.0-105-g223cd0e)

Guest: clear-17460-kvm.img (which has vsock support)

	$ erlc processes.erl
	$ erl -noshell -s processes test -s init stop
	the max processes is 262144
	the proecess time is 2.0:3.57
	$ erl -noshell -s processes test2 100000 -s init stop
	the max processes is 262144
	{"init terminating in do_boot",{system_limit,[{erlang,spawn,[erlang,apply,[#Fun<processes.2.26012808>,[]]],[]},{erlang,spawn,1,[]},{processes,for,3,[{file,"processes.erl"},{line,24}]},{processes,for,3,[{file,"processes.erl"},{line,24}]},{processes,max,1,[{file,"processes.erl"},{line,10}]},{init,start_it,1,[]},{init,start_em,1,[]}]}}

	# program freeze here...
	# Ctrl+C not work

	[main]
	plugins=ifupdown,keyfile

	[ifupdown]
	managed=false

	[keyfile]
	unmanaged-devices=interface-name:docker0;interface-name:vmnet1;interface-name:vmnet8;interface-name:vboxnet0

	#include <stdio.h>
	#include <unistd.h>
	#include <memory.h>
	#include <sys/types.h>
	#include <sys/socket.h>
	#include <linux/types.h>
	#include <linux/netlink.h>
	#include <linux/rtnetlink.h>

	#define BUF_SIZE 8192