Skip to content

Instantly share code, notes, and snippets.

@bmc
Last active September 3, 2021 20:08
Show Gist options
  • Save bmc/2db513245a4d7213ba7aba4f67723d12 to your computer and use it in GitHub Desktop.
Save bmc/2db513245a4d7213ba7aba4f67723d12 to your computer and use it in GitHub Desktop.
I needed code to serve static files from an Akka HTTP server. I wanted to use fs2 to read the static file. Michael Pilquist recommended using streamz to convert from an fs2 Task to an Akka Source. This is what I came up with. (It does actually work.)
object StaticFile {
// Various necessary imports. Notes:
//
// 1. fs2 is necessary. See https://github.com/functional-streams-for-scala/fs2
// 2. streamz is necessary. See https://github.com/krasserm/streamz
// 3. Apache Tika is used to infer MIME types from file names, because it's more reliable and
// fully-featured than using java.nio.file.Files.probeContentType().
//
// If using SBT, you'll want these library dependencies and resolvers:
//
// resolvers += "krasserm at bintray" at "http://dl.bintray.com/krasserm/maven"
// libraryDependencies ++= Seq(
// "com.typesafe.akka" %% "akka-http" % "10.0.0",
// "com.typesafe.akka" %% "akka-http-spray-json" % "10.0.0",
// "com.github.krasserm" %% "streamz-akka-stream" % "0.5.1",
// "org.apache.tika" % "tika-core" % "1.14",
// "co.fs2" %% "fs2-core" % "0.9.2",
// "co.fs2" %% "fs2-io" % "0.9.2"
// )
import akka.NotUsed
import akka.http.scaladsl.server.Directives._
import akka.stream.scaladsl.{Source => AkkaSource}
import akka.http.scaladsl.model._
import akka.util.ByteString
import akka.stream.{Graph, SourceShape}
import streamz.akka.stream._
import fs2._
import org.apache.tika.Tika
import scala.concurrent.{ExecutionContext, Future}
import scala.util.control.NonFatal
import java.nio.file.{AccessDeniedException, NoSuchFileException, Paths}
val BaseDir = "/path/to/static/base/dir"
val tika = new Tika
/** Serve up a static file, using fs2. Here's how to use this function inside
* an Akka HTTP route:
*
* {{{
* pathPrefix("static" / Remaining) { urlPath =>
* get {
* complete(StaticFile.serve(s"static/$urlPath"))
* }
* } ~
* path("/") {
* get {
* complete(StaticFile.serve("index.html"))
* }
* } ~ // etc
* }}}
*
* @param urlPath the URL path, as extracted by Akka.
* @param ctx the execution context (presumably from Akka, e.g., `ActorSystem("server").dispatcher`)
*
* @return An Akka `HttpResponse`, wrapped in a `Future`.
*/
def serve(urlPath: String)(implicit ctx: ExecutionContext): Future[HttpResponse] = {
Future {
// In this case, the base directory is hard-coded, and we need
// to use it to determine the actual on-disk path. There are other
// approaches one could use (e.g., reading the base directory from
// a config, allowing an environment variable override, etc.).
val pathPieces = BaseDir :+ urlPath.split("/")
val path = Paths.get(pathPieces.mkString(java.io.File.separator))
// Use Apache Tika to get the MIME type from the path. This isn't
// wrapped in an Option because it always seems to return *something*.
val mime = tika.detect(path)
// Parse the resulting MIME type string into an Akka HTTP ContentType.
ContentType.parse(mime) match {
case Right(contentType) =>
// We got a valid MIME type, so it's time to try to read the file.
// fs2 will throw exceptions on error; those are explicitly handled
// in the Future's recover() block, below.
// Note: I have added explicit types for clarity. Obviously, you can
// leave them off and let the compiler infer them.
val source: Graph[SourceShape[ByteString], NotUsed] =
io.file. // We want to use fs2 to read a file.
readAll[Task](path, 8192). // Read the whole file as an effectful stream, in 8K chunks
chunks. // Get the chunks
map { ch: NonEmptyChunk[Byte] =>
// We need to map the fs2 chunks into Akka ByteString objects, since
// that's what Akka expects in its streams.
ByteString(ch.toArray)
}.
// Convert the FS2 Stream[A, ByteString] to an Akka Stream
// Graph[SourceShape[ByteString], NotUsed]. As Martin Krasser
// notes, the actual FS2 -> Akka Stream conversion happens here,
// via streamz implicits.
toSource
// Map the Akka Stream Graph object into an Akka Source.
val akkaSource: AkkaSource[ByteString, NotUsed] = AkkaSource.fromGraph(source)
// We can return the Akka Source wrapped in the HttpResponse. Note that
// NO data has been read from the file yet! When Akka HTTP materializes its Source,
// to stream the bytes up to the browser, it will implicitly materialize the
// underlying fs2 stream. The "edge" here is inside Akka HTTP's browser-delivery
// logic.
HttpResponse(StatusCodes.OK, entity = HttpEntity(contentType, akkaSource))
case Left(errors) =>
// We failed to determine the MIME type.
HttpResponse(StatusCodes.InternalServerError, entity = s"Can't determine MIME type: ${errors.mkString(" ")}")
}
}
.recover {
// Here's where we handle any errors that occur. Note that fs2 I/O errors will
// throw java.nio exceptions. We map them explicitly into HTTP result codes.
case e: NoSuchFileException =>
HttpResponse(StatusCodes.NotFound, entity = "Not found.")
case e: AccessDeniedException =>
HttpResponse(StatusCodes.Forbidden, entity = "Permission denied.")
case NonFatal(e) =>
HttpResponse(StatusCodes.InternalServerError, entity = s"Read failure: $e")
}
}
}
@bmc
Copy link
Author

bmc commented Nov 30, 2016

This Gist incorporates suggestions from both @mpilquist and @krasserm.

@hseuming
Copy link

Hi,
Lines 65 & 66 seem a bit odd. From the REPL of Scala 2.12.6, there is the following:

==========================================================
Welcome to Scala 2.12.6 (OpenJDK 64-Bit Server VM, Java 1.8.0_171).
Type in expressions for evaluation. Or try :help.

scala> import java.nio.file.{AccessDeniedException, NoSuchFileException, Paths}
import java.nio.file.{AccessDeniedException, NoSuchFileException, Paths}

scala> val urlPath = "index.html"
urlPath: String = index.html

scala> val BaseDir = "/path/to/some/dir"
BaseDir: String = /path/to/some/dir

scala> val pathPieces = BaseDir :+ urlPath.split("/")
pathPieces: scala.collection.immutable.IndexedSeq[Any] = Vector(/, p, a, t, h, /, t, o, /, s, o, m, e, /, d, i, r, Array(index.html))

scala> val path = Paths.get(pathPieces.mkString(java.io.File.separator))
path: java.nio.file.Path = /p/a/t/h/t/o/s/o/m/e/d/i/r/[Ljava.lang.String;@2bb717d7

The path does not look right. Am I missing something here?

Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment