FWIW: I (@rondy) am not the creator of the content shared here, which is an excerpt from Edmond Lau's book. I simply copied and pasted it from another location and saved it as a personal note, before it gained popularity on news.ycombinator.com. Unfortunately, I cannot recall the exact origin of the original source, nor was I able to find the author's name, so I am can't provide the appropriate credits.
# referecing: # https://www.digitalocean.com/community/tutorials/how-to-install-apache-kafka-on-ubuntu-14-04 # https://chongyaorobin.wordpress.com/2015/07/08/step-by-step-of-install-apache-kafka-on-ubuntu-standalone-mode/
$ sudo useradd kafka -m
bin/kafka-topics.sh --zookeeper localhost:2181 --list
bin/kafka-topics.sh --zookeeper localhost:2181 --describe --topic mytopic
bin/kafka-topics.sh --zookeeper localhost:2181 --alter --topic mytopic --config retention.ms=1000
... wait a minute ...
Create a template service file at /etc/systemd/system/[email protected]. The template parameter will correspond to the name
of target host:
[Unit]
Description=Setup a secure tunnel to %I
After=network.target
In Kafka, the responsibility of what has been consumed is the responsibility of the consumer and this is also one of the main reasons why Kafka has such great horizontal scalability.
Using the high level consumer API will automatically do this for you by committing consumed offsets in Zookeeper (or a more recent configuration option is using by a special Kafka topic to keep track of consumed messages).
The simple consumer API make you deal with how and where to keep track of consumed messages yourself.
Purging of messages in Kafka is done automatically by either specifying a retention time for a topic or by defining a disk quota for it so for your case of one 5GB file, this file will be deleted after the retention period you define has passed, regardless of if it has been consumed or not.
This method avoids merge conflicts if you have periodically pulled master into your branch. It also gives you the opportunity to squash into more than 1 commit, or to re-arrange your code into completely different commits (e.g. if you ended up working on three different features but the commits were not consecutive).
Note: You cannot use this method if you intend to open a pull request to merge your feature branch. This method requires committing directly to master.
Switch to the master branch and make sure you are up to date:
If your system is running slowly, perhaps a process is using too much CPU time and won't let other processes run smoothly. To find out which processes are taking up a lot of CPU time, you can use Apple's Activity Monitor.
The CPU pane shows how processes are affecting CPU (processor) activity:
