Skip to content

Instantly share code, notes, and snippets.

View valencik's full-sized avatar
🍍

Andrew Valencik valencik

🍍
View GitHub Profile
@valencik
valencik / JsonlPayloadPipe.scala
Created July 9, 2024 20:19
Elasticsearch Bulk Indexing with a stream of JSON strings
import cats.effect.IO
import fs2.{Pipe, Pull, Stream}
object JsonlPayloadPipe {
/** Builds a pipe that transforms a stream of JSON strings into payloads for the Elasticsearch Bulk
* API. Each input string represents a JSON document to be sent to the Bulk API. The JSONs are
* gathered, interspersed with `delimiter`, and packaged into payload strings up to a maximum
* size of `batchMax` bytes.
*
@valencik
valencik / lucene-quick.sc
Created April 7, 2023 12:27
Quick Lucene InMemory example
//> using scala "2.13.10"
//> using lib "org.apache.lucene:lucene-core:9.5.0"
//> using lib "org.apache.lucene:lucene-analysis-common:9.5.0"
//> using lib "org.apache.lucene:lucene-memory:9.5.0"
//> using lib "org.apache.lucene:lucene-queryparser:9.5.0"
import org.apache.lucene.analysis.standard.StandardAnalyzer
import org.apache.lucene.index.memory.MemoryIndex
import org.apache.lucene.queryparser.flexible.standard.StandardQueryParser
<li class="indented0" name="fs2.Stream.InvariantOps#observe" group="Ungrouped" fullcomment="yes" data-isabs="false" visbl="pub">
<a id="observe(p:fs2.Pipe[F,O,Nothing])(implicitF:cats.effect.Concurrent[F]):fs2.Stream[F,O]" class="anchorToMember"></a><a id="observe(Pipe[F,O,Nothing])(Concurrent[F]):Stream[F,O]" class="anchorToMember"></a>
<span class="permalink">
<a href="../fs2/Stream$$InvariantOps.html#observe(p:fs2.Pipe[F,O,Nothing])(implicitF:cats.effect.Concurrent[F]):fs2.Stream[F,O]" title="Permalink"><i class="material-icons"></i></a>
</span>
<span class="modifier_kind"><span class="modifier"></span> <span class="kind">def</span></span>
<span class="symbol">
<span class="name">observe</span>
<span class="params">
(
@valencik
valencik / cats-hierarchy.md
Created December 29, 2022 01:09
A very simplified view of the cats typeclass hierarchy building to IO

Cats

The Typelevel ecosystem of libraries for functional programming in Scala provides several fundamental type classes for representing the capabilities of different computations.

Functor

Effect: Transforming the value inside an effect by mapping over it

[📚 Functor docs][docs-functor]

@valencik
valencik / gpt-parallel-requests-typelevel.md
Created December 2, 2022 01:16
Asking ChatGPT how to use Typelevel libraries for some parallel fetching.

I'm writing purely functional code in Scala, using the ecosystem of Typelevel libraries. Specifically I'm using cats-effect, fs2, and http4s.

I have a list of URLs, how do make parallel GET requests to them and write their bytes to disk?


Here is one way you can do it using the cats-effect, fs2, and http4s libraries in Scala:

Import the necessary libraries:

@valencik
valencik / aoc-2022-day1-gpt3.md
Created December 1, 2022 13:06
Getting GPT-3 (text-davinci-003) to solve Advent of Code day one in python
# Previously Seen Queries
- can i cancel a refund
- how can i cancel an order
- cancel my account
- can i dropship using paypal
- paypal payments
- paypal providers
@valencik
valencik / README.md
Created September 25, 2015 22:21 — forked from anonymous/README.md
Quick Gist: The self replicating chrome extension
  • Download this gist as a zip and extract
  • Open chrome://extensions
  • Enable Developer mode
  • Click Load unpacked extension... and select the extracted folder
  • 💵 Profit 💵

Created by the SMU CS Society

@valencik
valencik / units-2.11-compile-error.log
Created September 21, 2014 17:42
Error compiling units-2.11 on Mac OS X
AndrewMBP:units-2.11 andrew$ ./configure
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
@valencik
valencik / vimdeck-presentation
Created September 7, 2014 18:12
Vimdeck multiple lists issue.
$ls -l presentation/
total 16
-rw-r--r-- 1 andrew admin 153 7 Sep 14:58 script.vim
-rw-r--r-- 1 andrew admin 360 7 Sep 14:58 slide001.md
-----
$cat script.vim
set nonumber
set hidden
noremap <PageUp> :bp<CR>