mrsimpson

Project Serializer

Purpose

A flexible shell function to serialize source code directories into a single text stream, useful for preprocessing code for LLM analysis or code review.

Features

• Serialize multiple project directories
• Include/exclude file patterns
• Skip binary files
• Verbose mode for debugging

mrsimpson

Apache Flink

Apache Flink ist eine Open-Source-Stream-Processing-Engine, die für verteilte, hochparallele und fehlertolerante Datenströme entwickelt wurde. Es ermöglicht die Verarbeitung von kontinuierlichen Datenströmen in Echtzeit.

Integration mit Apache Kafka

Apache Flink integriert sich nahtlos mit Apache Kafka, dem de-facto-Standard für Datenströme:

• Flink kann Kafka-Topics als Datenquellen nutzen und Ergebnisse in Kafka-Topics schreiben[1][4]
• Flink unterstützt die exakt-einmalige Verarbeitung von Ereignissen, selbst bei Systemausfällen[4]

mrsimpson

package de.fermata.flink.app;

import java.time.Instant;
import java.util.Random;
import java.util.stream.StreamSupport;
import org.apache.flink.api.common.state.BroadcastState;
import org.apache.flink.api.common.state.MapStateDescriptor;
import org.apache.flink.api.common.state.ReadOnlyBroadcastState;
import org.apache.flink.api.java.tuple.Tuple2;
import org.apache.flink.configuration.Configuration;

mrsimpson

package de.fermata.flink.app;

import java.time.Duration;
import java.util.stream.StreamSupport;
import org.apache.flink.api.common.eventtime.*;
import org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator;
import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
import org.apache.flink.streaming.api.functions.sink.PrintSinkFunction;
import org.apache.flink.streaming.api.functions.windowing.ProcessWindowFunction;
import org.apache.flink.streaming.api.windowing.assigners.TumblingProcessingTimeWindows;

mrsimpson

import java.sql.Timestamp;
import java.util.Comparator;
import java.util.Optional;
import java.util.stream.Collectors;
import java.util.stream.Stream;
import java.util.stream.StreamSupport;
import org.apache.flink.api.common.state.ListState;
import org.apache.flink.api.common.state.ListStateDescriptor;
import org.apache.flink.api.common.typeinfo.TypeInformation;
import org.apache.flink.configuration.Configuration;

mrsimpson

package de.db.lightgate.streaming.functions;

import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ConcurrentMap;
import java.util.concurrent.CountDownLatch;
import org.apache.flink.streaming.api.functions.source.RichParallelSourceFunction;
import org.apache.flink.streaming.api.watermark.Watermark;

mrsimpson

Yet another Kafka compose

This Gist bundles a Kafka with useful tools and a UI to maintain it.

CAUTION Kafka is very picky about non-cleaned-up state. Thus, when stopping it, remove the volumes

docker compose down --volumes

mrsimpson

mrsimpson

function generatePassword() {
    const groups = ["ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz",
        "0123456789",
        "!§$%&/()=?*+-"];

    return [12, 2, 1]
        .map(function (len, i) {
            return Array(len)
                .fill(groups[i]).map(function (x) {
                    return x[Math.floor(Math.random() * x.length)]

mrsimpson

name: CI/CD

on:
  push:
    paths-ignore:
      - '**.md'

jobs:
  test:
    runs-on: ubuntu-latest