Ecto Morph blog post

ecto_morph.ex

response = %{"meat_type" => "medium rare", "pickles" => false, "collection_date" => "2019-11-04"}

# We want a struct so that we can do things like implement protocols for it
defmodule SteamedHam do
  defstruct [:meat_type, :pickles, :collection_date]
end

# With no !, the struct function selects only the fields defined in the schema.
steamed_ham = struct(SteamedHam, response)

# Now we have our struct, we write some code like this:

def expiry_date(%SteamedHam{collection_date: collection_date}) do
  Date.add(collection_date, 3)
end

# Expect when we call it:

SteamedHam.expiry_date(steamed_ham)

# Boom! ** (FunctionClauseError) no function clause matching in Date.add/2

# This is a perfectly reasonable error, we are trying to use Date.add on a string. What we want
# really is a way to specify up front the types of the steamed ham struct fields, That way we can
# rely on those types throughout the rest of our program. So we use Ecto:

defmodule SteamedHam do
  use Ecto.Schema

  embedded_schema do
    field(:meat_type, :string)
    field(:pickles, :boolean)
    field(:collection_date, :date)
  end

  # def expiry_date(%SteamedHam{collection_date: collection_date}) do
  #   Date.add(collection_date, 3)
  # end
  def new(data = %{collection_date: collection_date}) do
    struct(SteamedHam, %{data | collection_date: Date.from_iso8601!(collection_date)})
  end
end

# This is a nice signal to other developers, but it doesn't actually enforce anything;
response = %{"meat_type" => "medium rare", "pickles" => false, "collection_date" => "2019-11-04"}
steamed_ham = struct(SteamedHam, response)

# As you can see the date is still a date. What we really want to do is coerce the type to a date.
# Okay so we cab write a new function:

def new(data = %{collection_date: collection_date}) do
  struct(SteamedHam, %{data | collection_date: Date.from_iso8601!(collection_date)})
end

# This is okay, especially with just one field, or just a few fields that need coercion. But imagine
# if we had lots of structs and lots of data - we'd need to keep track of which fields in each of
# the structs need coercing. Worse than that we've already defined exactly what we want each field to
# be in the definition of the Ecto Schema!

# So what we could do is use that schema to figure our dynamically which fields should be what.
# Ecto allows us to introspect the schema (reflection they call it) like this:

# This function returns us a list of fields that we defined in the schema.
SteamedHam.__schema__(:fields)

# This function returns us the type of the given field.
SteamedHam.__schema__(:type, :collection_date)

# That means we can combine the two and write a function like this:
type_mappings =
  for field <- SteamedHam.__schema__(:fields), into: %{} do
    {field, SteamedHam.__schema__(:type, field)}
  end

# which returns a map with the field name as the key and the type as a value:

# Now we can use that map to define casting functions for any type we might write in our Ecto Schema:

casted_data =
  for {key, value} <- response, into: %{} do
    atomised_key = String.to_existing_atom(key)
    {atomised_key, cast_value(value, type_mappings[atomised_key])}
  end

def cast_value(value, :date) when is_binary(value), do: Date.from_iso8601!(value)

# Let's put all of this code in a module so we can run it:

defmodule EctoHelper do
  def create_struct(data, schema) do
    casted_map =
      for {key, value} <- data, into: %{} do
        atomised_key = String.to_existing_atom(key)

        {atomised_key, cast_value(value, type_mappings(schema)[atomised_key])}
      end

    struct(schema, casted_map)
  end

  defp cast_value(value, :date) when is_binary(value), do: Date.from_iso8601!(value)
  defp cast_value(value, _), do: value

  defp type_mappings(schema) do
    for field <- schema.__schema__(:fields), into: %{} do
      {field, schema.__schema__(:type, field)}
    end
  end
end

EctoHelper.create_struct(response, SteamedHam)

# Okay that worked well. We know have a pretty generic function to enable automatic casting of
# data to what our schema defines. Let's test it with another schema:

defmodule AuroraBorealis do
  use Ecto.Schema

  embedded_schema do
    field(:location, :string)
    field(:probability, :float)
    field(:actually_a_fire?, :boolean)
  end
end

response = %{"location" => "Kitchen", "probability" => 1.3, "actually_a_fire?" => true}
EctoHelper.create_struct(response, AuroraBorealis)

# So we could now extend our `cast()` function to cater for all of the ecto types that we can define.
# For example, if our schema defines a field as a number, but our API response says it's a
# string, we could do this:
def cast(value, :integer) when is_binary(value), do: String.to_integer(value)

# Okay this all works and it's kind of clever, but hopefully by now you are thinking,
# BUT WHY WOULD YOU WANT TO DO THAT?! Introspecting Ecto Schemas feels a bit weird - and it is unnecessary.
# Ecto gives us all of this power for free, with changesets!

# Let's look at the exact same idea, but using changesets:

response = %{"meat_type" => "medium rare", "pickles" => false, "collection_date" => "2019-11-04"}
Ecto.Changeset.cast(%SteamedHam{}, response, SteamedHam.__schema__(:fields))

%Ecto.Changeset{
  action: nil,
  changes: %{collection_date: ~D[2019-11-04], meat_type: "medium rare"},
  errors: [],
  data: %SteamedHam{},
  valid?: false
}

response = %{"meat_type" => "medium rare", "pickles" => 10, "collection_date" => "2019-11-04"}
Ecto.Changeset.cast(%SteamedHam{}, response, SteamedHam.__schema__(:fields))

%Ecto.Changeset{
  action: nil,
  changes: %{collection_date: ~D[2019-11-04], meat_type: "medium rare"},
  errors: [pickles: {"is invalid", [type: :integer, validation: :cast]}],
  data: %SteamedHam{},
  valid?: false
}

# The date gets coerced to a date automatically, and any invalid values get put into the changeset as errors.
# This is super awesome because we can use that to decide what we want to do in each case. For example:

response = %{"meat_type" => "medium rare", "pickles" => 10, "collection_date" => "2019-11-04"}

Ecto.Changeset.cast(%SteamedHam{}, response, SteamedHam.__schema__(:fields))
|> make_struct()

defp make_struct(changeset = %{errors: []}) do
  {:ok, Ecto.Changeset.apply_changes(changeset)}
end

defp make_struct(changeset) do
  {:error, changeset}
end

# Okay so this is really good for simple fields, but now let's look at relations. Imagine we define
# the following schema:

defmodule DinnerGuest do
  use Ecto.Schema

  embedded_schema do
    field(:name, :string)
    embeds_many(:steamed_hams, SteamedHam)
    embeds_one(:aurora_borealis, AuroraBorealis)
  end
end

# This schema says we have dinner guests which have many steamed_hams and one aurora_borealis. An
# example might look like this:

%DinnerGuest{
  name: "Super Nintendo Chalmers",
  steamed_hams: [
    %SteamedHam{pickles: false, meat_type: "Rare", collection_date: ~D[2019-05-05]},
    %SteamedHam{pickles: true, meat_type: "burnt", collection_date: ~D[2019-05-05]}
  ],
  aurora_borealis: %AuroraBorealis{location: "Kitchen", probability: 1, actually_a_fire?: true}
}

# Now we have to be a bit careful because the embedded relations need to be treated differently from
# the usual fields. If we want the same casting behaviour as before for our relations, we need to use
# `cast_embed`. cast_embed does the same thing as the `Ecto.Changeset.cast` function above, but, you
# guessed it, for embedded_schemas. Let's try using it now:

response = %{
  "name" => "Super Nintendo Chalmers",
  "steamed_hams" => [
    %{"meat_type" => "medium rare", "pickles" => false, "collection_date" => "2019-11-04"},
    %{"meat_type" => "rare", "pickles" => true, "collection_date" => "2019-11-04"}
  ],
  "aurora_borealis" => %{
    "location" => "Kitchen",
    "probability" => 1.3,
    "actually_a_fire?" => true
  }
}

Ecto.Changeset.cast(%DinnerGuest{}, response, DinnerGuest.__schema__(:fields))

# This blows up because that last argument to `cast` is a list of fields that we want to allow inside
# the response, during casting. Above we have said, let all the fields through, but we don't want all
# the fields, we want only all of the fields that are _not_ embeds, in this case the name field:

Ecto.Changeset.cast(%DinnerGuest{}, response, [:name])

# Now this hasn't failed, but it has also ignored the embedded fields completely, which is not awesome.
# To help that we can do this:

Ecto.Changeset.cast(%DinnerGuest{}, response, [:name])
|> Ecto.Changeset.cast_embed(:steamed_hams,
  with: fn steamed_ham = %{__struct__: schema}, data ->
    Ecto.Changeset.cast(steamed_ham, data, schema.__schema__(:fields))
  end
)
|> Ecto.Changeset.cast_embed(:aurora_borealis,
  with: fn aurora_borealis = %{__struct__: schema}, data ->
    Ecto.Changeset.cast(aurora_borealis, data, schema.__schema__(:fields))
  end
)

# There is a lot happening here, but essentially we are taking each of the embedded fields and calling
# `Ecto.Changeset.cast` on them. cast_embed takes a `with` option as the last argument which allows us
# to define for ourselves exactly how we want our embedded struct to be `cast`ed. In this case we want
# to just call Ecto.Changeset.cast the same way that we did when we just had it on our own.
# If we pipe that all into `Ecto.Changeset.apply_changes` we get:

%DinnerGuest{
  aurora_borealis: %AuroraBorealis{
    actually_a_fire?: true,
    location: "Kitchen",
    probability: 1.3
  },
  name: "Super Nintendo Chalmers",
  steamed_hams: [
    %SteamedHam{
      collection_date: ~D[2019-11-04],
      meat_type: "medium rare",
      pickles: false
    },
    %SteamedHam{
      collection_date: ~D[2019-11-04],
      meat_type: "rare",
      pickles: true
    }
  ]
}

# Amazing! Now we have the benefits of cast for all of our associations.
# All that's left to do is make this whole process general enough to work on any Ecto schema. I've
# done that, and packaged it up into a library called EctoMorph. Check it out on hex and github here
# https://github.com/Adzz/ecto_morph.