Skip to content

Instantly share code, notes, and snippets.

@lilith
Last active August 29, 2015 14:20
Show Gist options
  • Select an option

  • Save lilith/12144f3b8f98f6e3ee18 to your computer and use it in GitHub Desktop.

Select an option

Save lilith/12144f3b8f98f6e3ee18 to your computer and use it in GitHub Desktop.
ImageResizer HTTP pipeline

ImageResizer has traditionally taken a very minimalist approach to touching the HTTP pipeline.

  1. Only handle PostAuthorizeRequest and PreSendRequestHeaders events.

  2. PostAuthorizeRequest only takes action if both (a) there are image processing commands, and (b) the file has an image-type file extension.

  3. We relied on StaticFileModule to deliver cached, static assets after we called RewritePath during PostAuthorizeRequest. Our cache-miss path used a naive HttpHandler without byte-range support.

If a custom VirtualPathProvider was registered, we relied on StaticFileHandler to serve unmodifed versions of those files. Modified versions would be served either by StatciFileModule after being written to disk, or by a naive handler upon cache-miss.

An existing weak point was that we did not help out users that were using fake extensions - i.e, image.jpg.ashx - when there was no query. I.e, image.jpg.ashx?width=200 and image.jpg would respond, but image.jpg.ashx would fail.

New problems via IIS hotfixes and .NET 4.0

  • StaticFileModule is frequently not installed.
  • When there is a VirtualPathProvider providing image assets, it is ignored, and 404s are provided. StaticFileHandler must be registered for the image extensions individually. RAMMFAR no longer resolves this.
  • HttpHeaders applied to the response during PreSendRequestHeaders and less and less-frequently honored.

Async

  • StaticFileHandler is not async, and does not leverage async methods on the streams provided by a VirtualPathProvider
  • Application of response filters is not async, despite Stream->Stream

Goal: Retain compatibility with CMSes and VirtualPathProviders, yet provide an async IHttpModule that works consistently regardless of environment configuration.

Users expect ImageResizer to be able to access existing image assets, process them, (optionally) cache them, and serve them with high throughput. Asset sizes range from 1kb to 100mb. Underlying data stores can be highly latent, particularly if used as an image proxy.

Users expect to declare event handlers that can adjust Expires: headers, rewrite URLs, and provide access control.


Questions

  1. Is there a modern StaticFileHandler that we can leverage? (Async, can take a stream OR a path)
  2. Is there any way in which we can provide 'output-filtering' that is disk cachable? As systems (wisely) ditch VirtualPathProvider (but have no replacement interface), the need for output-filtering-like behavior increases.

ImageResizer tries to be middleware, but given the platform state this seems increasingly infeasible.

Scenario walk-throughs

http://imageresizing.net/docs/v4/plugins/diskcache

Caches: DiskCache, SourceDiskCache, MemCache, SourceMemCache

https://github.com/imazen/resizer/blob/develop/Core/Plugins/Basic/NoCache.cs https://github.com/imazen/resizer/blob/develop/Core/Plugins/Basic/NoCacheAsyncHandler.cs

We have a Content-type for every request (calculated from conversion commands or source URL file extension). This can, of course, be wrong, but we don't claim to support extensionless blobs or incorrect file extenions well. If the image is processed it will always be correct.

Backends

  • HTTP proxy. Remote URL contains original asset. When DiskCache plugin is installed, we permacache the result, and we don't involve a modified date from the remote server. We use the last-modified date of the cached file. Remote HTTP resources can't change/be invalidated ATM. Without DiskCache, the work is re-performed every time.
  • Amazon S3, Azure: Users can optionally enable source file modified date checking, but this roundtrip is usually an unacceptable performance loss.
  • MongoReader: like http proxy, no modified date.
  • SqlReader: We have low(ish)-latency access to a modified date.

To note: we have no metadata database. Cache entries (files) use the right file extension, and use the cache time for last-modifed dates sent to the client. We have users with > 10 million images, so any metadata store would need to outperform the NTFS filesystem exists check (not too hard, but still a high bar).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment