ShankarShastri shankarshastri

How to Write Hygienic Rust Macros

Macro hygiene is the concept of macros that work in all contexts; they don't affect and aren't affected by anything around them. Ideally all macros would be fully hygienic, but there are lots of pitfalls and traps that make it all too easy to accidentally write unhygienic macros. This guide attempts to provide a comprehensive resource for writing the most hygienic macros.

Understanding the Module System

First, a little aside on the details of Rust's module system, and specifically paths; it is

Avro serialization

There are 4 possible serialization format when using avro:

Avro Json encoding
Avro Data Serialization (https://avro.apache.org/docs/current/spec.html#Data+Serialization) Binary format with an header that contains the full schema, this is the format usually used when writing Avro files
Avro Single Object Encoding (https://avro.apache.org/docs/current/spec.html#single_object_encoding) Binary format with an header with only the fingerprint/id of the schema, this it the format used by Kafka (see this
Avro Binary Encoding (https://avro.apache.org/docs/current/spec.html#binary_encoding)

How to create an HTTPS certificate for localhost domains

This focuses on generating the certificates for loading local virtual hosts hosted on your computer, for development only.

Do not use self-signed certificates in production ! For online certificates, use Let's Encrypt instead (tutorial).

About

Example on how to run locally an AWS Lambda via API Gateway using localstack.

Based on...

Usage

The problem

I have some data which has adjacent entries that I want to group together and perform actions on. I know roughly that fs2.Pull can be used to "step" through a stream and do more complicated logic than the built in combinators allow. I don't know how to write one though!

In the end we should have something like

def combineAdjacent[F[_], A](
 shouldCombine: (A, A) =&gt; Boolean,

Operation	Input	Result	Notes
map	`F[A]` , `A => B`	`F[B]`	Functor
apply	`F[A]` , `F[A => B]`	`F[B]`	Applicative
(fa, fb, ...).mapN	`(F[A], F[B], ...)` , `(A, B, ...) => C`	`F[C]`	Applicative
(fa, fb, ...).tupled	`(F[A], F[B], ...)`	`F[(A, B, ...)]`	Applicative
flatMap	`F[A]` , `A => F[B]`	`F[B]`	Monad
traverse	`F[A]` , `A => G[B]`	`G[F[A]]`	Traversable; `fa.traverse(f)` == `fa.map(f).sequence`; "foreach with effects"
sequence	`F[G[A]]`	`G[F[A]]`	Same as `fga.traverse(identity)`
attempt	`F[A]`	`F[Either[E, A]]`	Given `ApplicativeError[F, E]`

Kafka 0.11.0.0 (Confluent 3.3.0) added support to manipulate offsets for a consumer group via cli kafka-consumer-groups command.

List the topics to which the group is subscribed

kafka-consumer-groups --bootstrap-server <kafkahost:port> --group <group_id> --describe

Note the values under "CURRENT-OFFSET" and "LOG-END-OFFSET". "CURRENT-OFFSET" is the offset where this consumer group is currently at in each of the partitions.

Reset the consumer offset for a topic (preview)

Thread Pools

Thread pools on the JVM should usually be divided into the following three categories:

CPU-bound
Blocking IO
Non-blocking IO polling

Each of these categories has a different optimal configuration and usage pattern.

	/*
	* Self-Explanatory Protocol Buffer Lang Guide
	*/


	/*
	* Why Protocol Buffers?
	* Protocol buffers are Google's language-neutral, platform-neutral, extensible mechanism for serializing structured data – think XML, but smaller, faster, and simpler.
	* You define how you want your data to be structured once, then you can use special generated source code to easily write and read your structured data to and from a variety of data streams and using a variety of languages.
	* Protocol Buffers are Schema Of Messages. They are language agnostic.

	import { async, ComponentFixture, TestBed } from '@angular/core/testing';
	import { Component, NO_ERRORS_SCHEMA, DebugElement } from '@angular/core';
	import { By } from '@angular/platform-browser';

	import { CharacterDetailComponent } from './character-detail.component';
	import { Character, DataService } from '../../core';
	import * as testing from '../../../testing';

	@Component({
	template: '<ro-character-detail [character]="selectedCharacter"></ro-character-detail>'