brisbane-elixir-2015-07.md

Brisbane Elixir

July 23, 2015

General

Welcome newcomers!

Introducing Card Shark

Last meetup Mark walked us through creating a RESTful database application in Phoenix, from scratch. We played with the HTML and JSON generators that ship with Phoenix to create a simple CRUD app. We want to keep building on the Card Shark idea as a project we can all hack on to learn elixir and Phoenix.

If you missed the intro last meetup, don't worry - it's never too late to jump in, and there's never any assumed knowledge or skill level...we're all learning here :)

Project: http://github.com/brisbane-elixir/cardshark

Gitter Chat: https://gitter.im/brisbane-elixir/cardshark

Post a message on Gitter and join the project! Create an issue for a proposed feature...pick up an issue and open some PRs!

And heeeeeres Mark!

• Integrating React.js with phoenix

Testing Elixir/Phoenix

After Mark's Card Shark kickstarter, I started to think about what were the first features that needed building. If this were a professional project, I would always start with some tests - so I decided to take a deeper dive into the testing options in elixir.

Elixir as ExUnit built in, so it is usually the default. However, inspired by more exotic testing frameworks in other languages, so too in elixir are there choices!

ExUnit

Basic unit testing framework that ships with elixir core.

Default Phoenix generated tests are in ExUnit.

defmodule AssertionTest do
  use ExUnit.Case, async: true

  test "the truth" do
    assert true
  end
end

Doctests

Write executable examples in your elixir docs that can be verified at build time.

Rspec-like

Espec

ESpec is inspired by RSpec and the main idea is to be close to its perfect DSL

defmodule SomeSpec do
  use ESpec

  example_group do
    context "Some context" do
      it do: expect("abc").to match(~r/b/)
    end

    describe "Some another context with opts", focus: true do
      it do: 5 |> should be_between(4,6)
    end
  end
end

espec_phoenix

Pavlov

A BDD framework for your Elixir projects. Think of it as RSpec's little Elixir-loving brother.

Shouldi

elixir testing with shouldi

defmodule MyFatTest do

  with "necessary_key" do
    setup context do
      Dict.put context, :necessary_key, :necessary_value
    end

    should( "have necessary key", context ) do
      assert context.necessary_key == :necessary_value
    end
  end

  with "sometimes_necessary_key" do
    setup context do
      Dict.put context, :sometimes_necessary_key, :sometimes_necessary_value
    end

    should( "have necessary key", context ) do
      assert context.sometimes_necessary_key == :sometimes_necessary_value
    end
  end
end

Amrita

A polite, well mannered and thoroughly upstanding testing framework for Elixir

White bread

Feature: Serve coffee
  Coffee should not be served until paid for
  Coffee should not be served until the button has been pressed
  If there is no coffee left then money should be refunded

  Scenario: Buy last coffee
    Given there are 1 coffees left in the machine
    And I have deposited £1
    When I press the coffee button
    Then I should be served a coffee

defmodule SunDoe.CoffeeShopContext do
  use WhiteBread.Context

  given_ "there are 1 coffees left in the machine", fn state ->
    {:ok, state |> Dict.put(:coffees, 1)}
  end

  given_ ~r/^I have deposited £(?<pounds>[0-9]+)$/, fn state, %{pounds: pounds} ->
    {:ok, state |> Dict.put(:pounds, pounds)}
  end
end

Browser Automation

Hound

  test "the truth", meta do
    navigate_to("http://example.com/guestbook.html")

    element_id = find_element(:name, "message")
    fill_field(element_id, "Happy Birthday ~!")
    submit_element(element_id)

    assert page_title() == "Thank you"
  end

Tuco Tuco

Similar browser driver.

How do you choose?

Look at

• how long ago it was last committed to? (has it been abandoned?)
• how many contributors does it have? does it have an active community, or just 1 guy?
• who are they? If elixir core team members are contributing...that's a good sign.
• Does it have docs? Has any effort been made to help people us it?

Default Phoenix Tests

The code generators that ship with Phoenix create some tests for you. Let's see what they do.

Model Tests

use CardShark.ModelCase Default ExUnit tests can run in parallel, but Database tests do not by default. Every test runs inside a transaction which is reset at the beginning of the test.

Imports from Ecto convenience functions.

Controller Tests

use CardShark.ConnCase Conveniences for testing phoenix endpoints

• manages test database transactions as per model tests
• import route helpers
• get, post, etc helper functions against app endpoint
• tests against plug - elixir's equivalent of rack (ruby), wsgi (python), servlets (Java)
• Not entire middleware stack applied (e.g. phoenix_ecto error handlers) - look into this.

Semaphore

Having tests is good. Running them is better. https://semaphoreci.com/blog/2015/05/12/elixir-on-semaphore.html

Event Driven Architecture

Event Sourcing is the new hot sauce - so, let's play with it!

A ticket management system like Card Shark seems like a perfect case for storing changes as a series of events. Some obvious features include the ability to view a change history for a ticket or a project. Some more interesting features could be things like animating the movement tickets throughout a sprint, to assess whether the most important tickets were picked up in the best order, etc.

Storing Events in Postgres

Phoenix/Ecto already supports Postgres as it's default database. An obvious for storing events is something like a JSON document. We need something with a flexible, nested structure - different events will need to store different sets of properties, and these may need to be some kind of nested structure. Postgres natively supports JSON types, so this is easiest path forward.

Note, Ecto supports other databases such as MySQL and MS SQL Server, but it isn't limited to relation databases. A Mongo DB adapter is in progress, and other NoSQL DBs are sure to follow.

The trick for our event store is that the existing Postgres adapter doesn't support JSON types yet. Oh no. But wait!

The beauty of a functional approach is that things are much more easily extensible. It's pretty trivial to extend ecto ourselves to add JSON support - with a little help from Google!

https://medium.com/@alanpeabody/embedding-elixir-structs-in-ecto-models-8f4fcbc06baa

Basically, all we need to do is create a Postgres.Extension. Here it is:

defmodule CardShark.Extensions.Json do
  alias Postgrex.TypeInfo

  @behaviour Postgrex.Extension
  @json ["json", "jsonb"]

  def init(_parameters, opts),
    do: Keyword.fetch!(opts, :library)

  def matching(_library),
    do: [type: "json", type: "jsonb"]

  def format(_library),
    do: :binary

  def encode(%TypeInfo{type: type}, map, _state, library) when type in @json,
    do: library.encode!(map)

  def decode(%TypeInfo{type: type}, json, _state, library) when type in @json,
    do: library.decode!(json)
end

We then configure our DB Repo to use it:

config :card_shark, CardShark.Repo,
  extensions: [{CardShark.Extensions.Json, library: Poison}]

We also can tell ecto to construct a custom type when it pulls values from the DB:

 schema "events" do
   field :name, :string
   field :payload, CardShark.EventPayload.Type
   field :timestamp, Ecto.DateTime, default: Ecto.DateTime.utc
 end

Now let's store events

Whenever we create or update a model, we want to persist an event for it.

card = Repo.insert!(changeset)
card |> Event.card_created |> Event.store

  def card_created(card) do
      payload = card
      |> Map.take([:id, :summary, :detail, :estimate, :assignee, :project_id])

      %Event{
        name: "card_created",
        payload: payload
      }
  end

Thats a great start, but it's not event driven enough yet.

Commands

CQRS (Command Query Responsibilty Separation) is a key pattern related to event driven architechtures.

For the command part, the Command pattern from OO programming can still be applied in a non-OO context. The key idea is that requests to the application are presented as a 'command' data structure. Commands are given to an Executor. On successful execution, an would be fired, notifying interested parties that something interesting happened.

One key thing this drives is separating your business logic from the controllers. We probably want to issue similar commands from controllers and Channels, for example.

Here are some ideas.

Existing controller code for card creation:

def create(conn, %{"card" => card_params}) do
    changeset = Card.changeset(%Card{}, card_params)

    if changeset.valid? do
      card = Repo.insert!(changeset)
      card |> Event.card_created |> Event.store

      CardShark.Endpoint.broadcast! "stream", "cardevent", %{event: "created", card: card}
      render(conn, "show.json", card: card)
    else
      conn
      |> put_status(:unprocessable_entity)
      |> render(CardShark.ChangesetView, "error.json", changeset: changeset)
    end
  end
end

Possible new controller, creating a command and giving it to an executor:

  def create(conn, %{"card" => card_params}) do
    %Commands.CreateCard{card_params: card_params}
    |> CommandExecutor.execute
    |> render_create_result(conn)
  end

  def render_create_result({:ok, card}, conn) do
    render(conn, "show.json", card: card)
  end

  def render_create_result({:error, changeset}, conn) do
      conn
      |> put_status(:unprocessable_entity)
      |> render(CardShark.ChangesetView, "error.json", changeset: changeset)
  end

The logic for creating cards is in the executor

defmodule CardShark.CommandExecutor do
  def execute(command = %CreateCard{}) do
    changeset = Card.changeset(%Card{}, command.card_params)

    if changeset.valid? do
      card = Repo.insert!(changeset)
      event = card |> Event.card_created |> Event.publish
      {:ok, card}
    else
      {:error, changeset}
    end
  end
end

And events can be subscribed to in order to handle things like broadcasting on Channels:

    subscribe_to Commands.CardCreated, fn event ->
      CardShark.Endpoint.broadcast! "stream", "cardevent", %{event: "created", card: event.payload}
    end

Help me experiment to see if this is a helpful pattern for Card Shark....if not, we'll learn something :)

General

Submit topics you want to hear about, or talk about!

Resources

• The Elixir Fountain - Pod cast
• Elixir Radar - Email news

Interesting Projects

• Typed Elixir http://wiki.cfcl.com/Projects/Elixir/TE/WebHome
• Elixir Script