Skip to content

Instantly share code, notes, and snippets.

View pabloem's full-sized avatar
🐢

Pablo pabloem

🐢
  • Seattle, WA
View GitHub Profile
@pabloem
pabloem / README.md
Last active July 5, 2023 17:50
An Apache Beam Wordle with Python async calls

An Apache Beam Wordle

This project demonstrates how to put together a list of words for Apache Beam. This list of words is meant to be used to launch a Beam wordle game. The project has the following pieces:

  • A Beam Pipeline that scrapes the Beam website in scrape-beam-words.py. The outputs of this pipeline are:
    • beam_wordle_words.txt*: A long list of words, separated by newline.
  • beam_wordle_histogram.txt*: A file detailing the frequency in word
@pabloem
pabloem / message_queue.c
Last active December 16, 2022 17:42
A message queue made by6 ChatGPT
#include <pthread.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#define MAX_QUEUE_SIZE 10
typedef struct {
int tag;
char *message;
@pabloem
pabloem / expected_outcome_script.txt
Created November 8, 2022 21:22
expected outcome from first run of image
This file has been truncated, but you can view the full file.
prebuilt_beam_sdk/beam_python_prebuilt_sdk
IMAGES FOR REPO prebuilt_beam_sdk/beam_python_prebuilt_sdk
IMAGES FOR REPO beam-sdk
IMAGES FOR REPO beam_portability
gcr.io/apache-beam-testing/prebuilt_beam_sdk/beam_python_prebuilt_sdk/cachegcr.io/apache-beam-testing/beam-sdk/beam_go_sdk gcr.io/apache-beam-testing/beam-sdk/beam_java11_sdk gcr.io/apache-beam-testing/beam-sdk/beam_java17_sdk gcr.io/apache-beam-testing/beam-sdk/beam_java8_sdk gcr.io/apache-beam-testing/beam-sdk/beam_python3.10_sdk gcr.io/apache-beam-testing/beam-sdk/beam_python3.6_sdk gcr.io/apache-beam-testing/beam-sdk/beam_python3.7_sdk gcr.io/apache-beam-testing/beam-sdk/beam_python3.8_sdk gcr.io/apache-beam-testing/beam-sdk/beam_python3.9_sdkgcr.io/apache-beam-testing/beam_portability/beam_flink1.10_job_server gcr.io/apache-beam-testing/beam_portability/beam_flink1.12_job_server gcr.io/apache-beam-testing/beam_portability/beam_flink1.13_job_server gcr.io/apache-beam-testing/beam_portability/beam_go_sdk gcr.io/apache-beam-testing/beam_portability/b
@pabloem
pabloem / README.md
Last active November 2, 2022 21:25
Calculating Pi with a Monte Carlo algorithm and using Beam Batched DoFns

Calculating Pi using Beam and Batched DoFns

This is a set of two sample Beam Pipelines that we use to calculate Pi with a monte carlo algorithm that relies on generating random points in an n-dimentional cube, and validating whether they are located within a circle in that space.

These examples are meant to showcase the use of Beam's Batched DoFns developed by @TheNeuralBit.

@pabloem
pabloem / README.md
Created August 23, 2021 18:38
The Tail at Scale - Seminario de Sistemas Distribuidos

Tail At Scale

Codigo para demostrar distribuciones de latencia en un servicio de software con distintos algoritmos.

Para instalar el entorno en Linux/Unix:

virtualenv venv
@pabloem
pabloem / requirements.txt
Created July 9, 2021 11:59
Extract tar and move to external storage
apache-beam[aws,gcp,test]
ipython
## -*- coding: utf-8 -*-
import json
import sys
import time
import apache_beam as beam
from apache_beam.io.fileio import CompressionTypes as ct
from apache_beam.utils.options import PipelineOptions
from beam_utils.sources import CsvFileSource
@pabloem
pabloem / README.md
Created July 29, 2016 02:48
Problema de LNPP

.

import subprocess
tables = [
"Dictionary_codi",
"Dictionary_item",
"NFC_hits",
"album",
"apply",
"banner",
"board_comment",
#!/usr/bin/env python3
import sys
import json
import pickle
def decode(dic, bitstr):
res = []
length = bitstr.bit_length() - 1
if bitstr >> length != 1: