Skip to content

Instantly share code, notes, and snippets.

@bszwej

bszwej/mongodb-s3-stream-compression.scala

Created July 31, 2017 16:24
Show Gist options
  • Save bszwej/1ac9e74e2b47c976d71259a74df6b1cf to your computer and use it in GitHub Desktop.
Save bszwej/1ac9e74e2b47c976d71259a74df6b1cf to your computer and use it in GitHub Desktop.
Download ZIP
Raw
mongodb-s3-stream-compression.scala
Source.fromPublisher(collection.find(): Publisher[org.bson.Document])
.map((doc: org.bson.Document) ⇒ doc.toJson())
.map((json: String) ⇒ ByteString(json))
.via(Compression.gzip: Flow[ByteString, ByteString, NotUsed])
.runWith(s3Sink)
val s3Sink: Sink[ByteString, Future[MultipartUploadResult]] =
s3Client.multipartUpload(
bucket = "bucketWithCookies",
key = "CookieCollectionBackup.json"
)
The full runnable code can be found here:
https://github.com/bszwej/mongodb-s3-stream-example
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment