Kong API Gateway

Architecture

Built on nginx + openresty
LUA as scripting language
API abstraction over nginx
Plugin architecture
Available with Docker
Configuration in Cassandra/Postgresql

Start Kong (Docker/Linux)

docker run -d --name kong-database \
              -p 5432:5432 \
              -e "POSTGRES_USER=kong" \
              -e "POSTGRES_DB=kong" \
              postgres:9.4
			  
docker run -d --name kong1 \
              --link kong-database:kong-database \
              -e "KONG_DATABASE=postgres" \
              -e "KONG_CASSANDRA_CONTACT_POINTS=kong-database" \
              -e "KONG_PG_HOST=kong-database" \
              -p 8000:8000 \
              -p 8443:8443 \
              -p 8001:8001 \
              -p 7946:7946 \
              -p 7946:7946/udp \
              kong:0.10.1
			  
curl localhost:8001
docker ps --format "table {{.ID}}\t{{.Image}}\t{{.Names}}"

Start Kong (Docker/Windows)

docker run -d --name kong-database ^
              -p 5432:5432 ^
              -e "POSTGRES_USER=kong" ^
              -e "POSTGRES_DB=kong" ^
              postgres:9.4		
			  
docker run -d --name kong1 ^
              --link kong-database:kong-database ^
              -e "KONG_DATABASE=postgres " ^
              -e "KONG_CASSANDRA_CONTACT_POINTS=kong-database" ^
              -e "KONG_PG_HOST=kong-database" ^
              -p 8000:8000 ^
              -p 8443:8443 ^
              -p 8001:8001 ^
              -p 7946:7946 ^
              -p 7946:7946/udp ^
              kong:0.10.1
			  
start http://localhost:8001
docker ps --format "table {{.ID}}^t{{.Image}}^t{{.Names}}"

Exploring the database (linux)

docker exec -it kong-database bin/bash #(or /bin/sh from windows)

$> psql -U kong -W #(pass: kong)
=# \l
=# \c kong
=# \dt
=# select * from nodes;
=# select * from apis;
=# \q

Kong CLI

docker exec -it kong1 bin/bash

$> kong version
$> kong health
$> cat /etc/kong/kong.conf.default
$> kong reload

Configuration

Any envvar starting with KONG_ will overwrite a value
Includes nginx config
proxy_port 8000 <-- traffic from clients
proxy_ssl_port 8443 <-- https
admin_api_port: 8001 <-- admin

Clustering

docker run -d --name kong2 ^
              --link kong-database:kong-database ^
              -e "KONG_DATABASE=postgres" ^
              -e "KONG_CASSANDRA_CONTACT_POINTS=kong-database" ^
              -e "KONG_PG_HOST=kong-database" ^
              kong:0.10.1

Shares the database (autodiscovering in a table)
cluster_listen address port in config
kong cluster reachability -> to check the cluster
kong quit -> to remove the cluster
kong cluster force-leave -> to force the removal

Playground box

docker run -it --rm --link kong1:kong1 --link kong2:kong2 bash
$> apk add --update python py-pip ca-certificates
$> pip install httpie httpie-unixsocket 
$> apk add jq

Creating an API

http POST kong1:8001/apis 
http kong1:8001/apis
http POST kong1:8001/apis name=demo \
	   hosts=demo.com upstream_url=http://httpbin.org
  
http kong1:8001/apis/demo
http kong2:8001/apis/demo 
http kong1:8000/headers Host:demo.com

Key authentication

http kong1:8000/headers Host:demo.com
http POST kong1:8001/apis/demo/plugins name=key-auth
http kong1:8000/headers Host:demo.com

http POST kong1:8001/consumers username=alice  \
	   [email protected]
http POST kong1:8001/consumers/alice/key-auth key=1111
http POST kong1:8001/consumers/alice/key-auth key=2222
http kong1:8000/headers Host:demo.com apikey:1111
http DELETE kong1:8001/consumers/alice/key-auth/1111

Additional x-forwarded-* headers added
Additional x-consumir-id header forwarded
Multiple credentials can be provided for the same consumer

Anonymous users

# Check authenticated access
http head kong1:8000 Host:demo.com apikey:2222
# Check anonymous access (fails)
http head kong1:8000/headers Host:demo.com

# Get key-auth plugin id
plugin=$(http kong1:8001/apis/demo/plugins | \
	jq -r '.data[] | select(.name=="key-auth") | .id')
# Create a user and get its ID
http POST kong1:8001/consumers username=anonymous_user  
user=$(http kong1:8001/consumers/anonymous_user | jq -r ".id")
http PATCH kong1:8001/apis/demo/plugins/$plugin config.anonymous=$user

# Check again anonymous access (success)
http kong1:8000/headers Host:demo.com

JWT Authentication

http POST kong1:8001/apis/demo/plugins name=jwt config.uri_param_names=jwt
http POST kong1:8001/consumers/alice/jwt algorithm=HS256 \
	   key=alice-key-1  secret=mysecret
http kong1:8000/headers/?jwt=xxxxxxxxx (jwt token)
http DELETE kong1:8001/consumers/alice/jwt/alice-key-1

Rate limiting

# API level
http POST kong1:8001/apis/demo/plugins name=rate-limiting \
	   config.second=1 config.hour=100
# Consumer level
user=$(http kong1:8001/consumers/anonymous_user | jq -r ".id")
http POST kong1:8001/apis/demo/plugins name=rate-limiting  \ 
	   consumer_id=$user config.second=1 config.hour=100

Request size limiting

Important for preventing flood attacks
Size in MB, decimals allowed

http POST kong1:8001/apis/demo/plugins name=request-size-limiting \
	   config.allowed_payload_size=0.5

Request correlation

Very important for distributed systems
Enables traceability
Format: ip-port-nginx_pid-connection-connection_counter-timestamp

http POST kong1:8001/apis/demo/plugins \
	   name=correlation-id config.header_name=X-Kong-ID \ 
	   config.generator=tracker
http kong1:8000/headers Host:demo.com

Logs

Kong errors are in the nginx log (/usr/local/kong/logs/*)
Logging plugins are logs of forwarded request/responses
Asynchronous
Can act over specific consumers
/tmp may not be the best path

http POST kong1:8001/apis/demo/plugins name=file-log config.path=/tmp/request.log
http kong1:8000/headers Host:demo.com

docker exec -it kong1 /bin/bash
cat /tmp/request.log

Monitoring with Datadog

Create a DataDog account
Update kong instance:

docker exec -it kong1 /bin/bash

$>yum install sysstat
$>DD_API_KEY=<API_KEY> sh -c \
    "$(curl -L https://raw.githubusercontent.com/DataDog/dd-agent/master/packaging/datadog-agent/source/setup_agent.sh)"

Install datadog plugin:

http POST kong1:8001/apis/demo/plugins name=datadog \
       config.host=127.0.0.1 config.port=8125 config.timeout=1000

Generate metrics with

http kong1:8000/headers Host:demo.com

Open https://app.datadoghq.com/metric/explorer
- Select "kong.demo.request.count"
- Make the chart XL

Monitoring alternatives

Prometheus

Custom configuration

Not the best part of Kong

docker exec -it kong1 /bin/bash

$> vi /usr/local/share/lua/5.1/kong/templates/nginx_kong.lua

Add add_header X-Javi washere; after location / {

Cache

Uses nginx proxy_cache module
Blog post

Kong on mesos (1/4)

Official support
Launch the cluster using the portal
Stablish the tunnel with

ssh -L 80:localhost:80 -N \    
       <username>@<clusterprefix>mgmt.westeurope.cloudapp.azure.com -p 2200

Kong on mesos (2/4)

Check the portal with opening http://localhost:80
Download and install the dc-os CLI
Deploy everything!

dcos config set core.dcos_url http://localhost
dcos auth login
git clone https://github.com/<username>/kong-dist-mesos
cd kong-dist-mesos
dcos marathon app add postgres.json
dcos marathon app add kong_postgres.json
dcos marathon task list

Kong on mesos (3/4)

Check the deployment

$local> ssh-agent
# set SSH_AUTH_SOCK & SSH_AGENT_PID variables
# No, really, set them.
$local> ssh-add .ssh\id_rsa
#Jump to master
$local> ssh <username>@<clusterprefix>mgmt.westeurope.cloudapp.azure.com -A -p 2200
# check connectivity and services
$master> curl marathon-lb.marathon.mesos:10001
$master> curl marathon-lb.marathon.mesos:10002
#stablish tunnel from master:8001 to lb:10002
$master> ssh -L 8001:localhost:10002 -N marathon-lb.marathon.mesos
#stablisht unnel from local:8001 to master:8001
$local> ssh -L 8001:localhost:8001 -N <username>@<clusterprefix>mgmt.westeurope.cloudapp.azure.com -p 2200
#test local access
$local> http localhost:8001
#setup kong api
$local> http POST localhost:8001/apis name=demo \
     hosts=<clusterprefix>agents.westeurope.cloudapp.azure.com \
     upstream_url=http://httpbin.org
#jump to master again and test the API
$local> ssh <username>@<clusterprefix>mgmt.westeurope.cloudapp.azure.com -p 2200
$master> apt-get install httpie
$master> http marathon-lb.marathon.mesos:10001/headers \
     Host:<clusterprefix>agents.westeurope.cloudapp.azure.com

Kong on mesos (4/4)

Update the ACS public load balancer configuration:
- Create probe probe marathon-lb-10001 to port 10001
- Modify LbRuleHTTP: backend port 80 -> 10001, probe marathon-lb-10001
- Add a rule to dcos-agent-public-nsg allowing traffic to 10001
- Wait. Wait more.
- Open http://<clusterprefix>agents.westeurope.cloudapp.azure.com

jjamor/kong-api-gateway.md

Kong API Gateway

Architecture

Start Kong (Docker/Linux)

Start Kong (Docker/Windows)

Exploring the database (linux)

Kong CLI

Configuration

Clustering

Playground box

Creating an API

Key authentication

Anonymous users

JWT Authentication

Rate limiting

Request size limiting

Request correlation

Logs

Monitoring with Datadog

Monitoring alternatives

Custom configuration

Cache

Kong on mesos (1/4)

Kong on mesos (2/4)

Kong on mesos (3/4)

Kong on mesos (4/4)

Additional tools and resources

End of the module