Standard design patterns in scala recommend the cake pattern to help compose larger programs from smaller ones. Generally, for simple cake layers, this works okay. Boner's article suggests using it to compose repository and service layers and his focus is on DI-type composition. As you abstract more of your IO layers however, you realize that you the cake pattern as described does not abstract easily and usage becomes challenging. As the dependencies mount, you create mixin traits that express those dependence and perhaps they use self-types to ensure they are mixed in correctly.
Then at the end of the world, you have to mix in many different traits to get all the components. In addition, perhaps you have used existential types and now you must have a val/object somewhere (i.e. a well defined path) in order to import the types within the service so you can write your program. Existential types within the cake layer, say within the service definition itself, require path dependent types to access the types for use in the methods of that layer. This is standard scala. You don't have to use path dependent types but if you want to avoid mixing objects together then path-dependent types can help you.
Generally, I do not think the value proposition for scala and the cake patterns/compositional patterns are the key value proposition that scala brings to the table. In may ways, the container-less approach with scala gives you some more options and capabilities, but I'm not sure they are 100x type improvements over not having containers. Where scala does provide tremendous value is allowing you to refine, extend and also integrate these components together. For me, this was the important part of the value proposition in this blog versus a specific compositional approach. I have used spring containers extensively and found them easy to use. When software gets complex, whether its spring or scala-oriented techniques, there are always issues to be solved. The option of using scala in the ways described in this blog and that do not involve proxies or byte-code generation is of value to the software I am concerned about.
Scala provides a variety of ways to handle object composition and the "constructor/member" "setting/injection" dilemma with implicits, constructor args, objects, vals and cake patterns but mixing them altogether can be tough at times.
Let's say you define a ''notes'' service that saves notes in a specialized data store (note that this is also a cake layer by itself). The rest of the database support needed in the application is handled by other database techniques. Let's just focus on the NotesService.
trait NotesService {
type NoteId
type FindNoteInfo <: FindNoteInfoLike
type Note <: NoteLike
type NoteSource <: NoteSourceLike[_]
/**
* Find a note based on search criteria.
*/
def find(info: FindNoteInfo): Option[Note]
/**
* Obtain the actual note based on the id.
*/
def get(id: NoteId): Option[Note]
/**
* Save or update the note. Use the returned Note. The
* input Note is no longer valid.
*/
def saveOrUpdate(note: Note): Note
/**
* Delete a note. This may or may not result in a physical
* deletion.
*/
def delete(id: NoteId)
/**
* The source of a note when a note needs to be read into the
* service or out of the service.
*/
trait NoteSourceLike[T] {
def source: T
}
/**
* Search criteria info for finder methods.
*/
trait FindNoteInfoLike
/**
* The actual note object for use by the application.
* The content can be obtained in a lazy fashion.
*/
trait NoteLike {
def id: NoteId
def content: Option[NoteSource]
}
/**
* Used to indicate that a note has not been saved yet.
*/
val UnassignedNoteId: NoteId
}You realize that to use this service, you'll need to use path dependent types in your code and there's alot of refinement that will go on for both the TraitService "container" as well as those existential types. In addition, as you subclass the trait to specialize it for specific backing stores, you realize that you want to keep this NotesService trait separate from a specific cake pattern e.g. NotesServiceComponent because it is either owned by someone else er you do not want to constrain how this service is used.
But lets say you want to use this service in a cake layer and embed it directly in the cake. So you write the standard definition recognizing that subclasses must also mixin the specific backend (e.g. a RDBMS backend configuration)required by that subclass.
For example, doing a standard "DI component" cake layer:
trait NotesServiceComponent {
val notesService: NotesService
trait NotesService { ...see text above...}
}This is rather restricting in that you are mixing in the NotesService with a specification of how you want that object to be declared. As you go through the layers and subclasses, you start using self-types and suddenly you are forcing your clients to use a very specific, and perhaps tangled, way of using your NotesService. I've found that once you start using the DI-component cake layer model, everything has to start staying in the cake layer because it becomes difficult to evolve both the "component" cake layer and the "service" component over time. So using the cake pattern directly for the trait limits how it is consumed. We would like to keep it separate but still allow us to access those internal, path-dependent types easily to write our code. This means we have to statically declare the value somewhere so that it has a well-formed path.
Let's say you are handed the NotesService class along with a subclass:
trait RdbmsNotesService extends NotesService {
/**
* Subclasses must defined the actual id structure.
*/
type RdbmsId
/**
* Subclasses must define what an unassigned id looks like.
*/
val UnassignedNoteId: NoteId
/**
* To find note, enter some string criteria.
*/
case class FindNoteInfo(criteria: String) extends FindNoteInfoLike
/**
* To put or get the actual note, input streams are used.
* This means notes can be binary objects as well or a
* picture or ... alright, just use a string for now.
*/
case class NoteSource(source: String) extends NoteSourceLike[String]
val EmptyNoteSource = NoteSource(null)
/**
* Represent a node id by a parameterized type.
*
* You cannot use a value class here due to a variety of
* language restrictions.
*/
case class NoteId(val id: RdbmsId)
/**
* A note has a MIME type as well as the content.
*/
trait Note extends NoteLike {
def mimeType: String
}
}You may need to make a specific implementation of this service. The standard approach is to define a subclass of the service. Let's assume that you want a slick backend:
trait ProfileAware {
import slick.driver.JdbcProfile
implicit protected val profile: JdbcProfile
}
trait DatabaseAware extends ProfileAware {
implicit def database: profile.simple.Database
}
trait NotesComponents { self: ProfileAware =>
import profile.simple._
val html = "text/html"
val plain = "text/plain"
case class Mime(name: String, description: Option[String])
class Mimes(tag: Tag) extends Table[Mime](tag, "Mimes") {
def mime = column[String]("mime", O.PrimaryKey)
def description = column[Option[String]]("description")
def * = (mime, description) <> (Mime.tupled, Mime.unapply)
}
class Notes(tag: Tag) extends Table[(Long, String, Option[String])](tag, "Notes") {
def id = column[Long]("id", O.PrimaryKey, O.AutoInc)
def mime = column[String]("mime")
def content = column[Option[String]]("content")
def * = (id, mime, content)
def mimeFk = foreignKey("mimeFK", mime, Mimes)(_.mime)
}
lazy val Mimes = TableQuery[Mimes]
lazy val Notes = TableQuery[Notes]
}
/**
* Notes service using slick, and hence JDBC, for the backend.
* Since the {{{NotesService}}} has path-dependent types, you can
* only access the {{{Note, NoteId, ...}}} types through the
* instance.
*/
trait SlickNotesService extends RdbmsNotesService {
self: DatabaseAware with NotesComponents =>
type RdbmsId = Long
override val UnassignedNoteId: NoteId = NoteId(-1)
import profile.simple._
case class Note(id: NoteId = UnassignedNoteId, mimeType: String,
content: NoteSource) extends super.Note
/**
* Import this to obtain some helper functions.
*/
object Helpers {
implicit def long2NoteId(id: Long): NoteId = new NoteId(id)
implicit def string2NoteSource(source: Option[String]): NoteSource =
source match {
case None => EmptyNoteSource
case Some(v) => NoteSource(v)
}
def noteSource2String(ns: NoteSource): Option[String] =
ns match {
case EmptyNoteSource => None
case ns: NoteSource => Option(ns.source)
case _ => None
}
}
import Helpers._
def find(info: FindNoteInfo): Option[Note] = {
database.withTransaction {
implicit session =>
None
}
}
def delete(id: NoteId): Unit = {
database.withTransaction {
implicit session =>
getNoteById(id.id).delete
}
}
private def getNoteById(id: Long) =
for (n <- Notes if n.id === id) yield n
def get(id: NoteId): Option[Note] = {
database.withTransaction { implicit session =>
getNoteById(id.id).firstOption match {
case Some(n) =>
Some(Note(id = NoteId(n._1), mimeType = n._2,
content = Helpers.noteSource2String(n._3)))
case _ => None
}
}
}
/**
* @todo perform versioning and save previous version
*/
def saveOrUpdate(note: Note): Note = {
database.withTransaction { implicit session =>
val id = note.id.id
// Find existing note, if it exists
getNoteById(id).firstOption match {
case Some(n) =>
// update it
Notes.update((id, note.mimeType, noteSource2String(note.content)))
note
case _ =>
// create a new one
val newId = (Notes returning Notes.map(_.id)) insert
(id, note.mimeType, noteSource2String(note.content))
new Note(NoteId(newId), note.mimeType, note.content)
}
}
}
}Now we have a slick backend, but it has a dependency. It expects to be mixed in with a DatabaseAware trait that holds the actual database. Its rather prescriptive here in how it used self-types and force the mixing. NoteService was fairly open in its usage but now its a bit more restricted in how it can be used. If the SlickNotesService had been dependent on types inside the actual backend driver, the def databatase may need to be a val, but a def database means that the database can be created when the application is ready to create it and it does not need to be created statically. That's the right model for an application that configures itself at the start.
Now we can use our driver very easily like below:
object TestNotesService {
import slick.driver.H2Driver
object Service extends SlickNotesService
with DatabaseAware with NotesComponents {
val profile = slick.driver.H2Driver.profile
val database = profile.simple.Database.forURL(url = "jdbc:h2:mem:test;DB_CLOSE_DELAY=-1",
driver = "org.h2.Driver")
}
import Service._
import profile.simple._
import Service.Helpers._
def main(args: Array[String]): Unit = {
database.withSession { implicit session =>
val ddl = Notes.ddl ++ Mimes.ddl
println("ddl: " + ddl.createStatements.mkString(";\n"))
ddl.create
Mimes ++= Seq(Mime(html, Some("html notes")),
Mime(plain, Some("text notes")))
require(Mimes.filter(_.mime === plain).firstOption.isDefined)
}
var note = new Note(mimeType = plain, content = EmptyNoteSource)
note = saveOrUpdate(note)
println("Initial Note: " + note)
var newNote = note.copy(content = NoteSource("Updated note!"))
newNote = saveOrUpdate(newNote)
println("Updated Note: " + newNote)
val aNoteId = newNote.id
println("Note id: " + aNoteId)
val foundNote = get(aNoteId)
println("Found updated Note: " + foundNote)
}
}We may also realize that it would be nice to add some helper methods so we created the Helpers object inside the service.
But where should these go? How do you keep these helper methods together? Do we always have to create them inside the class? What if we want to add more helpers? I created a nice anchored object in the above test program but do I always have to specify everything at program start? What's the best way to combine this service with 10 other services I need so its easy to figure out all the dependent objects you need for your application?
Again, if everything is statically known, including the actual database connection parameters at the start of the program, then this is a mute point. I mostly encounter designs where the database link is configured at runtime in some way and not from static config files only. And its clear that the incoming data components may have very different design that force a specific way of managing your dependencies and objects. If everything is statically known, we could change the def database into a val and define it directly.
Some applications components are simple and are provided to you in a simple DI-dependency cake like boner describer, some look like NotesService and some may come in with specific implementation designs such as the self-type in SlickNotesDriver. You'll need to be able to handle all the different types to fuse them together.
To use the SlickNotesService is not hard. But let's look at what happens with some variations of application configuration. We'll assume that we need to initialize the database in some way and that the NotesService subclasses like SlickNotesService are not dependent on the actual backend types that require path-dependent type resolution. Slick is a stateless backend that lifts queries into its driver environment. So it is fairly flexible in its design.
object ObjectRegistry extends SlickNotesService with DatabaseAware with TableComponents with TableQueryComponents {
import H2Driver.profile._
val profile: profile
// create a singleton database object for the app, but not statically defined since we need to use slick to create the Database
def database: profile.simple.Database = _database
private var _database: profile.simple.Database
def init(url: String): Unit = { _database = Database.forUrl(url) }
}
...in another program area...
import ObjectRegistry._
var newNote = Note(UnAssignedNoteId, plain, Option(EmptyNoteSource))
newNote = save(newNote)
There are multiple issues here including making ObjectRegistry inherit from many different components which is distracting as well as that the ObjectRegistry is actually now a NotesService object itself! That may not be bad, but it looks goofy. Also, we are importing the contents of the entire ObjectRegistry in to get easier access to the path-dependent types which is not good.
Now we want to use composition a bit more:
object ObjectRegistry {
object notesService extends SlicksNotesService with DatabaseAware with TableComponents with TableQueryComponents {
import H2Driver.profile._
val profile: profile
def database: profile.simple.Database = _database
private var _database: profile.simple.Database
def init(url: String): Unit = { _database = Database.forUrl(url) }
}
}
....in another program area...
import ObjectRegistry._
notesService.init(yourUrl)
...
import notesService._
var newNote = Note(UnAssignedNoteId, plain, Option(EmptyNoteSource))
newNote = save(newNote)
...Its around this point, you think it is easier to just add a Database constructor parameter for the database, or perhaps an implicit constructor value. You can make the giant object extension signature go away by using an intermediate definition so its much shorter but the real issue is that you have to have very specific knowledge about how to use the provided NotesService.
Let's still keep the NotesService separate but define a small cake layer to handle how the mixin occurs.
trait NotesServiceComponent {
val notesService: NotesService
}
object ObjectRegistry with NotesServiceComponent {
val notesService: SlickNotesService = new SlickNotesService with ... { ... }
}
...or...
object ObjectRegistry with NotesServiceComponent {
object notesService extends SlickNotesService with ... { ... }
}So with this approach, again keeping NotesService separate, we are now in the same place we were before except we did indicate that the ObjectRegistry is indeed a provider of a NotesService object. So this did not buy us too much.
We can also create a separate, configured object with everything we need and then set members into this object that are needed for configuration (or if the configuration object is not shared, created directly in the separate object). This essentially creates the notes service object externally and is not much different than the other options above.
...some other file...
object MyNotesService extends SlickNotesService with TableComponents with TableQueryComponents with DatabaseAware
{
import H2Driver.profile._
val profile: profile
def database: profile.simple.Database = _database
private var _database: profile.simple.Database
def init(url: String): Unit = { _database = Database.forUrl(url) }
}
....in your main app...
object ObjectRegistery {
val notesService = MyNotesService
}
The nice thing about this approach is that it is all kept separate. However, if you have a few services that have the same dependencies, you'll have to grab those dependencies wherever they are created and pull them back into ObjectRegistry--that's not a scalabale way to do this if you have several services with several dependencies. What is nice about this approach is MyNotesService plays the role of a "driver" in the sense that it pulls together the service and the dependencies all in one place.
Another approach is to create a separate shared configuration object and use more existential types to provide a centralized configuration point within a central, well known location like the ObjectRegistry. You have a couple of ways to do this but the example below shows this approach with two services one that is externally given to you that you cannot change how it has expressed its dependencies (NotesService) and one that you have control over the code (AnotherService). ConfigNotesService could, if it wanted to, inherit from SlickNotesService as well. There are a few variations you can make on the below that also make sense.
// This looks alot like the NotesService! It's the same idea.
trait ConfigComponent {
trait Config
val config: Config
}
trait ConfigNotesService extends ConfigComponent {
type Config <: ConfigDef
trait ConfigDef extends super.Config {
val profile: JdbcProfile
def database: profile.simple.Database = _database
}
val notesService: SlickNotesService
}
trait ConfigAnotherService extends AnotherService with ConfigComponent ... {
type Config <: ConfigDef
trait ConfigDef extends super.config {
val moreConfig: String
}
val anotherService: AnotherService
trait AnotherService extends super.AnotherService {
...something that is designed to use ConfigDef directly...
}
}
object ObjectRegistry extends ConfigNotesService with ConfigAnotherService {
type Config = config.type
object config extends super[ConfigNotesService].ConfigDef with super[ConfigAnotherService].ConfigDef {
val moreConfig = "aValue"
import H2Driver.profile._
val profile: profile
val database = Database.fromDataSource("jdbc2:h2:mem:app") // or use a def and private val as needed
}
val notesService = new SlickNotesService with TableQueriesComponent with TableComponents with DatabaseAware {
// Some boilerplate...
val profile = config.profile
def database = config.database
}
val anotherService: new AnotherService { } // service defined within the cake layer ConfigAnotherService
// more registry objects here
}This is actually an interesting approach in that we are using a separate "configuration" object that creates well-known paths for the application to work from. But depending on how your service is defined, you may have to have some supporting declarations (boilerplate) as we needed for the NotesService. AnotherService had already been adapted to work with the config object so there was no boilerplate for that service. The use of a config object forced, at least, the dependencies to be created even if the NotesService required some boilerplate to make sure the scope of the dependencies was available to the NotesService instance.
We can also once again see that if you have control over your code, then like AnotherService, you can bake in your dependency approach quite easily. But if the code is given to you and you cannot change it, as in NotesService, you have to adapt around it.
The key lesson from this option is that you can define the service instance, you can define some type of separate dependency object (in this case ConfigComponent) and you can have some behavior that initializes (creates or configs) the dependencies. So that's three separate parts that are needed to form a general solution to managing dependencies regardless of whether you control the service codebase or not.
#Using Something Else to Handle Composition and Abstracting So its pretty clear that to use the NotesService in traditional ways, is a bit tough. Not overly hard or impossible by any stretch and you have very fine-grained choices on how to mix it in. It's important to realize that by keeping NotesServices separate, you preserve those choices for the client. That's good.
But we have also seen where if we have different requirements and needs, it gets difficult to know how to create the mix. It is highly dependent on what you have coming in the door and their specific formulation. So if we look at the different scenarios we may encounter:
- Services may be statically defined completely at the start or must be "configured" during runtime.
- Services may use complex types that require path-dependent types to be used, hence, there must be a well-formed path.
- Services have different configuration requirements (e.g. constructor, implicits, self-types)
- Services may have dependencies (and many of them) that are statically known at the start or must be "configured" during runtime
- Services may have helper methods that are provided or not provided to you and they have to be easy to find and use.
- Services may be created by you or someone else and you have to adapt them to your application in many different ways.
- Services may need to be composed into other objects or inherited in
That's alot of variety. It an be said however, that all problems can be solved through enough layers of abstraction...we need to
- Find a way to force an object, like ObjectRegistry, to declare a service
- Find a way to declare a service that sometimes requires well-formed paths to help with path-dependent types
- Find a way to declare a set of dependencies for that service
- Find a way to allow dependencies to be shared, when needed, across services
- Find a way to initialize/change those dependencies to be ready for use
- Find a way to allow helper objects and definitions be available
- Find a way to bundle all the above together in a coherent, consistent approach that also balances DRY principles
More on this in the next blog. Stay tuned!