Last active
September 3, 2021 20:08
-
-
Save bmc/2db513245a4d7213ba7aba4f67723d12 to your computer and use it in GitHub Desktop.
I needed code to serve static files from an Akka HTTP server. I wanted to use fs2 to read the static file. Michael Pilquist recommended using streamz to convert from an fs2 Task to an Akka Source. This is what I came up with. (It does actually work.)
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
object StaticFile { | |
// Various necessary imports. Notes: | |
// | |
// 1. fs2 is necessary. See https://github.com/functional-streams-for-scala/fs2 | |
// 2. streamz is necessary. See https://github.com/krasserm/streamz | |
// 3. Apache Tika is used to infer MIME types from file names, because it's more reliable and | |
// fully-featured than using java.nio.file.Files.probeContentType(). | |
// | |
// If using SBT, you'll want these library dependencies and resolvers: | |
// | |
// resolvers += "krasserm at bintray" at "http://dl.bintray.com/krasserm/maven" | |
// libraryDependencies ++= Seq( | |
// "com.typesafe.akka" %% "akka-http" % "10.0.0", | |
// "com.typesafe.akka" %% "akka-http-spray-json" % "10.0.0", | |
// "com.github.krasserm" %% "streamz-akka-stream" % "0.5.1", | |
// "org.apache.tika" % "tika-core" % "1.14", | |
// "co.fs2" %% "fs2-core" % "0.9.2", | |
// "co.fs2" %% "fs2-io" % "0.9.2" | |
// ) | |
import akka.NotUsed | |
import akka.http.scaladsl.server.Directives._ | |
import akka.stream.scaladsl.{Source => AkkaSource} | |
import akka.http.scaladsl.model._ | |
import akka.util.ByteString | |
import akka.stream.{Graph, SourceShape} | |
import streamz.akka.stream._ | |
import fs2._ | |
import org.apache.tika.Tika | |
import scala.concurrent.{ExecutionContext, Future} | |
import scala.util.control.NonFatal | |
import java.nio.file.{AccessDeniedException, NoSuchFileException, Paths} | |
val BaseDir = "/path/to/static/base/dir" | |
val tika = new Tika | |
/** Serve up a static file, using fs2. Here's how to use this function inside | |
* an Akka HTTP route: | |
* | |
* {{{ | |
* pathPrefix("static" / Remaining) { urlPath => | |
* get { | |
* complete(StaticFile.serve(s"static/$urlPath")) | |
* } | |
* } ~ | |
* path("/") { | |
* get { | |
* complete(StaticFile.serve("index.html")) | |
* } | |
* } ~ // etc | |
* }}} | |
* | |
* @param urlPath the URL path, as extracted by Akka. | |
* @param ctx the execution context (presumably from Akka, e.g., `ActorSystem("server").dispatcher`) | |
* | |
* @return An Akka `HttpResponse`, wrapped in a `Future`. | |
*/ | |
def serve(urlPath: String)(implicit ctx: ExecutionContext): Future[HttpResponse] = { | |
Future { | |
// In this case, the base directory is hard-coded, and we need | |
// to use it to determine the actual on-disk path. There are other | |
// approaches one could use (e.g., reading the base directory from | |
// a config, allowing an environment variable override, etc.). | |
val pathPieces = BaseDir :+ urlPath.split("/") | |
val path = Paths.get(pathPieces.mkString(java.io.File.separator)) | |
// Use Apache Tika to get the MIME type from the path. This isn't | |
// wrapped in an Option because it always seems to return *something*. | |
val mime = tika.detect(path) | |
// Parse the resulting MIME type string into an Akka HTTP ContentType. | |
ContentType.parse(mime) match { | |
case Right(contentType) => | |
// We got a valid MIME type, so it's time to try to read the file. | |
// fs2 will throw exceptions on error; those are explicitly handled | |
// in the Future's recover() block, below. | |
// Note: I have added explicit types for clarity. Obviously, you can | |
// leave them off and let the compiler infer them. | |
val source: Graph[SourceShape[ByteString], NotUsed] = | |
io.file. // We want to use fs2 to read a file. | |
readAll[Task](path, 8192). // Read the whole file as an effectful stream, in 8K chunks | |
chunks. // Get the chunks | |
map { ch: NonEmptyChunk[Byte] => | |
// We need to map the fs2 chunks into Akka ByteString objects, since | |
// that's what Akka expects in its streams. | |
ByteString(ch.toArray) | |
}. | |
// Convert the FS2 Stream[A, ByteString] to an Akka Stream | |
// Graph[SourceShape[ByteString], NotUsed]. As Martin Krasser | |
// notes, the actual FS2 -> Akka Stream conversion happens here, | |
// via streamz implicits. | |
toSource | |
// Map the Akka Stream Graph object into an Akka Source. | |
val akkaSource: AkkaSource[ByteString, NotUsed] = AkkaSource.fromGraph(source) | |
// We can return the Akka Source wrapped in the HttpResponse. Note that | |
// NO data has been read from the file yet! When Akka HTTP materializes its Source, | |
// to stream the bytes up to the browser, it will implicitly materialize the | |
// underlying fs2 stream. The "edge" here is inside Akka HTTP's browser-delivery | |
// logic. | |
HttpResponse(StatusCodes.OK, entity = HttpEntity(contentType, akkaSource)) | |
case Left(errors) => | |
// We failed to determine the MIME type. | |
HttpResponse(StatusCodes.InternalServerError, entity = s"Can't determine MIME type: ${errors.mkString(" ")}") | |
} | |
} | |
.recover { | |
// Here's where we handle any errors that occur. Note that fs2 I/O errors will | |
// throw java.nio exceptions. We map them explicitly into HTTP result codes. | |
case e: NoSuchFileException => | |
HttpResponse(StatusCodes.NotFound, entity = "Not found.") | |
case e: AccessDeniedException => | |
HttpResponse(StatusCodes.Forbidden, entity = "Permission denied.") | |
case NonFatal(e) => | |
HttpResponse(StatusCodes.InternalServerError, entity = s"Read failure: $e") | |
} | |
} | |
} |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hi,
Lines 65 & 66 seem a bit odd. From the REPL of Scala 2.12.6, there is the following:
==========================================================
Welcome to Scala 2.12.6 (OpenJDK 64-Bit Server VM, Java 1.8.0_171).
Type in expressions for evaluation. Or try :help.
scala> import java.nio.file.{AccessDeniedException, NoSuchFileException, Paths}
import java.nio.file.{AccessDeniedException, NoSuchFileException, Paths}
scala> val urlPath = "index.html"
urlPath: String = index.html
scala> val BaseDir = "/path/to/some/dir"
BaseDir: String = /path/to/some/dir
scala> val pathPieces = BaseDir :+ urlPath.split("/")
pathPieces: scala.collection.immutable.IndexedSeq[Any] = Vector(/, p, a, t, h, /, t, o, /, s, o, m, e, /, d, i, r, Array(index.html))
scala> val path = Paths.get(pathPieces.mkString(java.io.File.separator))
path: java.nio.file.Path = /p/a/t/h/t/o/s/o/m/e/d/i/r/[Ljava.lang.String;@2bb717d7
The path does not look right. Am I missing something here?
Thanks.