Request Validation with Akka-HTTP and Cats

akka_http_and_cats_validation.md

Request Validation with Akka-HTTP and Cats

Greg Beech wrote an article on Akka HTTP Entity Validation. This post is on the same topic, except we are going to use typeclass instances, as opposed to subtyping, for form validations.

Sample Problem: User Regsitration

To start with, let us suppose that we are working on some sort of user management service using Scala/Akka. However, here we are only going to focus on the registration part, in which the client is expected to send a user data, in JSON format; and the server needs to validate them, and save the user if the validations succeed, or reject the request with all the errors detected. That said, we should keep in mind that our validations may be used through out the entire codebase, not just in the user registration endpoint.

The rules for our validation are as follows:

1. All fields are required.
2. Each field must have the correct type.
3. All strings must be non-empty.
4. Classic password validations (which we are not going to emphasize much).
5. A user should have at least one hobby (see #1).
6. Only people of ages 18 or above are allowed to register (it's not for kids).

Note that in a real-world project, registration validations are usually more complicated than this list, but for our current purposes, this should do.

Data Model and Route Implementation

Our user model has the usual attributes present in most registration forms:

final case class User(
  username: String,
  password: String,
  firstName: String, 
  lastName: String,
  age: Int, 
  hobbies: Vector[String])

In order to parse the JSON data and convert it into a User instance and vice-versa, we need to tell Akka-HTTP how to (un)marshall between JSON and User:

object RegistrationJsonSupport extends SprayJsonSupport with DefaultJsonProtocol {
  implicit val userFormat = jsonFormat6(User)
  // ... more formats here ...
}

Then we bring that into the scope and use it in our route:

import RegistrationJsonSupport._

val route = post {
  entity(as[User]) { user =>
    // For simplicity, let us agree that this operation never fails.
    saveUser(user)
    
    complete(s"User ${user.username} has been successfully registered.")
  }
}

This has added advantages: Akka-HTTP ensures that non-optional fields are provided by the client, and each field will have the correct type as specified in the model. Any requests that do not comply with these structural rules will be rejected. This means our current code already got the first 2 validation rules above covered.

Fields Validation

The next thing we need to do is implement the different validations for the fields.

If you are interested only in the form validations and how to integrate them with Akka-HTTP, you can skip this section.

The remaining validation rules are all about checking for non-emptiness and minimum values, and password validations. We can write our own utilities to generalize some of these things:

trait FieldValidator {
  trait Required[F] extends (F => Boolean)
  
  trait Minimum[F] extends ((F, Int) => Boolean)
  
  def required[F](field: F)(implicit req: Required[F]): Boolean =
    req(field)

  def minimum[F](field: F, limit: Int)(implicit min: Minimum[F]): Boolean =
    min(field, limit)

  implicit val minimumStringLength: Minimum[String] = _.length >= _

  implicit val minimumInteger: Minimum[Int] = _ >= _

  implicit val requiredString: Required[String] = _.nonEmpty

  // Warning: I haven't tried compiling this part.
  implicit val requireVector: Required[Vector[String]] = _.forall(required[String])
}

If that looks like a bunch of typeclasses and their instances, that's because it is. I've got two main motivations for suggesting this approach:

1. Type-safety. Having minimum and required take Any values on which to pattern match to determine the specific type of input is not type-safe.
2. Ad-hoc Polymorphism. You do not have to invoke different functions for the same type of validations (minimumInt, minimumString, etc.). If you want to check for, say, a minimum, you can just call the minimum function regardless of the type of field, providing, of course, that the typeclass has an (implicit) instance for that field in scope. Sometimes you might need to specify the type of the field to check (e.g. mimimum[Int], required[String]).

We can then use these utilities in our field validation functions:

import cats.data.ValidatedNec
import cats.implicits._

trait FieldValidator {
  // previous code goes here...
  
  def validateRequired[F: Required](field: F, fieldName: String): ValidationResult[F] =
    Either.cond(
      required(field),
      field,
      EmptyField(fieldName)).toValidatedNec

  def validateMinimum[F: Minimum](field: F, fieldName: String, limit: Int): ValidationResult[F] =
    Either.cond(
      minimum(field, limit),
      field,
      BelowMinimum(fieldName, limit)).toValidatedNec
      
  def validatePassword(password: String): ValidationResult[String] = ...    
      
  // more validations here
}

In case you didn't know what [F: Required] indicates, it's a Scala feature known as context bounds. Put simply, the signatures desugar to:

def validateRequired[F](field: F, fieldName: String)(implicit required: Required[F]): ValidationResult[F] = ???
def validateMinimum[F](field: F, fieldName: String, limit: Int)(implicit min: Minimum[F]): ValidationResult[F] = ???

We use Either.cond to handle the results of the validations. EmptyField and BelowMinimum are case classes representing errors:

sealed trait RequestValidation {
  def errorMessage: String
}

object RequestValidation {
  final case class EmptyField(fieldName: String) extends RequestValidation {
    override def errorMessage = s"$fieldName is empty"
  }

  final case class BelowMinimum(fieldName: String, limit: Int) extends RequestValidation {
    override def errorMessage = s"$fieldName is below the minimum of $limit"
  }
  
  // ... more error messages here...
}

Let's go back to our field validation functions. You can see that the resulting Eithers from the Either.conds call toValidatedNec. This method turns an Either into a ValidatedNec from Cats. ValidatedNec is just like Either, but far more useful when accumulating multiple errors. However, if you look at the signature of the functions, the results are actually VaidationResults, a type that we'll talk about in the next section.

Form Validation

Our form validator will try to validate all the fields in a given request. It can produce two possible outcomes: the validated object indicating that the validation went successfully, or an accumulation of errors. For this, we've got two options: use Either or Cats's ValidatedNec. Cats documentation for Validated recommends to use Validated (and consequently, ValidatedNec) for the accumulation of errors. In this post, we are going for ValidatedNec.

Just like Either, ValidatedNec has left and right projections, with left used to represent errors. We already know the type of errors we want, and we don't want to keep on specifying it, so we'll make a type alias that will fill in ValidatedNec's left projection with RequestValidation.

And that's where the ValidationResult we saw in the previous section comes in:

type ValidationResult[A] = ValidatedNec[RequestValidation, A]

Let's also add a FormValidation type alias for a function that lifts a form (of any type) to ValidationResult (which would allow us to do point-free style programming later on):

type FormValidation[F] = F => ValidationResult[F]

Now that we have the types in place, we can write a generic form validator:

def validateForm[F, A](form: F)(f: ValidationResult[F] => A)
    (implicit formValidation: FormValidation[F]): A =
  f(formValidation(form))

I forgot to tell you that FormValidation can be treated as a typeclass. And that means we can apply validateForm to any form of type F as long as there is a FormValidation instance for that form in scope.

Here's our form validator, with the typeclass instances:

trait FormValidator {
  type ValidationResult[A] = ValidatedNec[RequestValidation, A]
  type FormValidation[F] = F => ValidationResult[F]

  def validateForm[F, A](form: F)(f: ValidationResult[F] => A)
      (implicit formValidation: FormValidation[F]): A =
    f(formValidation(form))

  implicit lazy val registrationFormValidation: FormValidation[User] = {
    case User(username, password, firstName, lastName, address, age) => (
      validateRequired(username, "Username"),
      validatePassword(password, "Password"),
      validateRequired(firstName, "First Name",
      validateRequired(lastName, "Last Name"),
      validateRequired(age, "Age"),
      validateRequired(hobbies, "Hobbies")
      validateMinimum(age, "Age", 18))).mapN(User)
  }
  
  // ...other implicit typeclass instances here...
  implicit lazy val userUpdateFormValidation: FormValidation[User] = ???
  implicit lazy val userDeletionFormValidation: FormValdiation[String] = ???
}

mapN would take our chain of validations and accumulate all the errors found in the process, wrapping them all in the Invalid projection (ValidatedNec's equivalent to Either's Left). If no errors are found, it will return a Valid result (ValidatedNec's equivalent to Either's Right) with a user instance as the content.

We can then incorporate it into our route:

val route = post {
  entity(as[User]) { user =>
    validateForm(user) { 
      case Valid(_) =>
        saveUser(user)
        complete(s"User ${user.username} has been successfully registered.")
      case Invalid(failures) => complete(internalError(
          failures.iterator.map(_.errorMessage).mkString("\n")))
    } 
  }
}

The only remaining thing to do is the removal of that extra call to validateForm, which adds another layer of indentation, and explicit handling of the validation results.

To do that we may have to implement our custom unmashaller that does the validation. The problem is, Greg's custom unmarshaller, implemented using implicit conversions, wouldn't work in our case because our validations are not part of the entity. Also we will need to declare implicit parameters for the instances of our FormValidation typeclasses.

Fortunately, there is another way to circumvent the issue, and that is to add extension methods to the default unmarshaller:

implicit class ValidationRequestMarshaller[A](um: FromRequestUnmarshaller[A]) {
  def validate(implicit validation: FormValidation[A]) = um.flatMap { _ => _ => entity =>
    validateForm(entity) {
      case Valid(_) => Future.successful(entity)
      case Invalid(failures) =>
        Future.failed(new IllegalArgumentException(
          failures.iterator.map(_.errorMessage).mkString("\n")))
    }
  }
}

We are using implicit class instead of direct implicit conversion (implicit def). This implicit class effectively adds a validate method to a FromRequestUnmarshaller. Note that validate also contains the implicit parameter I mentioned above.

Here's the final, much better version of our route:

import validators._   // perhaps a package object that extends both FieldValidator and FormValidator

...

val route = post {
  entity(as[User].validate) { user =>
    saveUser(user)
    complete(s"User ${user.username} has been successfully registered.")
  }
}

@rcardin Yes. For compile-time values, refined types are even better because your code won't compile if the values are invalid. For runtime values, you might want to use refineV, which returns an Either and can be integrated with Cats validation.

Hey, I managed to use your approach successfully. However, I've got a question. What if I need to return a JSON error in case of a validation error? Is it possible?

Hey, I managed to use your approach successfully. However, I've got a question. What if I need to return a JSON error in case of a validation error? Is it possible?

Hi @rcardin. Yes, it is. All you have to do is transform the request validation to json. One way to do it is to use json encoder. In fact, the advantage of returning an algebraic data type is the ability to interpret it in different ways in the end.

Maybe I missed the point.

implicit class ValidationRequestMarshaller[A](um: FromRequestUnmarshaller[A]) {
  def validate(implicit validation: FormValidation[A]) = um.flatMap { _ => _ => entity =>
    validateForm(entity) {
      case Valid(_) => Future.successful(entity)
      case Invalid(failures) =>
        Future.failed(new IllegalArgumentException(
          failures.iterator.map(_.errorMessage).mkString("\n")))
    }
  }
}

The validate method can only return a Future, and futures completed exceptionally can only wrap an exception.

Maybe I missed the point.

implicit class ValidationRequestMarshaller[A](um: FromRequestUnmarshaller[A]) {
  def validate(implicit validation: FormValidation[A]) = um.flatMap { _ => _ => entity =>
    validateForm(entity) {
      case Valid(_) => Future.successful(entity)
      case Invalid(failures) =>
        Future.failed(new IllegalArgumentException(
          failures.iterator.map(_.errorMessage).mkString("\n")))
    }
  }
}

The validate method can only return a Future, and futures completed exceptionally can only wrap an exception.

Yes you can use spray json to encode your results to json. I'm on mobile so it's difficult to provide an example right now. I'll get back to you on this when I get access to my laptop.

Thanks. An example will be very appreciated :)

Sorry if I ping you, but did you manage to create the example we talked about?

Sorry if I ping you, but did you manage to create the example we talked about?

Sorry I got really busy. I will find some time for it during this weekend.

Maybe I missed the point.

implicit class ValidationRequestMarshaller[A](um: FromRequestUnmarshaller[A]) {
  def validate(implicit validation: FormValidation[A]) = um.flatMap { _ => _ => entity =>
    validateForm(entity) {
      case Valid(_) => Future.successful(entity)
      case Invalid(failures) =>
        Future.failed(new IllegalArgumentException(
          failures.iterator.map(_.errorMessage).mkString("\n")))
    }
  }
}

The validate method can only return a Future, and futures completed exceptionally can only wrap an exception.

Hi. I just got the chance to answer this. I think I misunderstood your point before. I thought you were asking about encoding the entity to JSON (which is already what was happening in the code). I even forgot my suggestions about using SprayJson is already in the article (which only shows that time has really passed).

To answer your question, it might depend on how you want the Json to be returned. The reason we are returning an IllegalArgumentException within the future is for Akka's directive to interpret it as a validation error in the payload itself. However, you can put the stringified Json inside the message of that exception if you want.

Or you can also write your own handlers as show here: https://stackoverflow.com/questions/35165376/return-json-errors-in-akka-http-api.

Unfortunately, the latter is a very involved process.