chrisdone

This is a small experiment to see whether one can:

1. Lex a file efficiently, retaining line/column and indentation information.
2. Consuming no or little memory (aside from the input size itself), and
3. Still have the flexibility to perform zero-cost operations like folds (counting tokens), doing nothing (a simple pass), or printing. SAX-style.

This proves that one could, e.g., run in ST and write to a mutable Storable vector. Allowing the caller to process the set of tokens later. But the cost/calculation of figuring out line/col/indentation of each token has already been figured out.

The input file is war-and-peace.txt which is 6MB. Simply reading the file takes 27ms. Counting all words (non-space) in the file takes 36ms. So let's say about 9ms, in the absense of more rigorous gauge-based benchmarking. There are 1,132,619 "words" in the file.

chrisdone

{-# LANGUAGE GADTs, LambdaCase #-}

import Control.Monad

-- Model the problem with a simple GADT:

data Foo a where
  GetInput :: Foo String
  WriteOutput :: String -> Foo ()
  Pure :: a -> Foo a

chrisdone

{-# OPTIONS_GHC -fno-warn-type-defaults #-}
{-# LANGUAGE TypeFamilies #-}
{-# LANGUAGE GADTs #-}
{-# LANGUAGE ExtendedDefaultRules #-}
{-# LANGUAGE LambdaCase #-}
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE TypeApplications #-}
{-# LANGUAGE TypeOperators #-}
{-# LANGUAGE ScopedTypeVariables #-}
{-# LANGUAGE StandaloneDeriving #-}

chrisdone

Change the mx.chrisdone.com mentions to your own domain.

Spin up a Droplet on DigitalOcean with docker-machine.

$ time docker-machine create \
  --driver digitalocean \
  --digitalocean-access-token $(cat ~/.do-token) \
  --digitalocean-monitoring \
  --digitalocean-region "lon1" \

--digitalocean-size "s-1vcpu-1gb" \

chrisdone

-- | A currency-less integral monetary value which cannot be further
-- subdivided. E.g. cent, penny, satoshi.
--
-- For lack of a better name:
-- <https://money.stackexchange.com/questions/85562/generic-name-for-the-smallest-unit-of-currency>
newtype IntegralMoney = IntegralMoney Int
  deriving (Eq, Ord, Integral, Num, Enum, Real, Show)
instance PersistFieldSql IntegralMoney where
  sqlType _ = SqlString
instance PersistField IntegralMoney where

chrisdone

{-# LANGUAGE CPP #-}
-- |

module Data.FileEmbed2
  ( makeRelativeToProject2
  , module Data.FileEmbed
  ) where

import Data.FileEmbed
import Language.Haskell.TH

chrisdone

{-# LANGUAGE FlexibleContexts #-}
{-# LANGUAGE ExistentialQuantification #-}
{-# LANGUAGE LambdaCase #-}
{-# LANGUAGE GADTs #-}

-- | A restricted web type.

module Web
  ( Web(..)
  , runWebHandler

chrisdone

{-# LANGUAGE BangPatterns #-}

import Control.DeepSeq
import Data.List hiding (foldl)
import Prelude hiding (foldl)

foldl = \f accumulator list ->
  case list of
    [] -> accumulator
    x:xs -> foldlS f (f accumulator x) xs

chrisdone

Duet's folds example

Duet has this folding example:

data List a = Nil | Cons a (List a)
foldr = \f z l ->
  case l of
    Nil -> z
    Cons x xs -> f x (foldr f z xs)

chrisdone

-- | Execute the steps.
execute :: [SomeStep] -> RIO MainEnv ()
execute steps = do
  resources <- newIORef mempty
  mapRIO
    (\MainEnv {logger} -> ExecuteEnv {logger = logger . ExecuteLog, resources})
    (mapM_ executeSomeStep steps)