Encapsulate your requests

I've written a lot of code that accesses other web APIs so here are some of my thoughts about how to do it the right way in C#.

HttpClient

Let's take a closer look at the .NET HttpClient interface. Are you surprised? This design looks like they took too much of a good thing —convenience— and turned it into a bad thing: the burden of choice.

interface HttpClient : HttpMessageInvoker
{
    Task<HttpResponseMessage> DeleteAsync (string? requestUri);
    Task<HttpResponseMessage> DeleteAsync (Uri? requestUri);
    Task<HttpResponseMessage> DeleteAsync (string? requestUri, CancellationToken cancellationToken);
    Task<HttpResponseMessage> DeleteAsync (Uri? requestUri, CancellationToken cancellationToken);
    Task<HttpResponseMessage> GetAsync (string? requestUri);
    Task<HttpResponseMessage> GetAsync (Uri? requestUri);
    Task<HttpResponseMessage> GetAsync (string? requestUri, HttpCompletionOption completionOption);
    Task<HttpResponseMessage> GetAsync (string? requestUri, CancellationToken cancellationToken);
    Task<HttpResponseMessage> GetAsync (Uri? requestUri, HttpCompletionOption completionOption);
    Task<HttpResponseMessage> GetAsync (Uri? requestUri, CancellationToken cancellationToken);
    Task<HttpResponseMessage> GetAsync (string? requestUri, HttpCompletionOption completionOption, CancellationToken cancellationToken);
    Task<HttpResponseMessage> GetAsync (Uri? requestUri, HttpCompletionOption completionOption, CancellationToken cancellationToken);
    Task<byte[]> GetByteArrayAsync (string? requestUri);
    Task<byte[]> GetByteArrayAsync (Uri? requestUri);
    Task<byte[]> GetByteArrayAsync (string? requestUri, CancellationToken cancellationToken);
    Task<byte[]> GetByteArrayAsync (Uri? requestUri, CancellationToken cancellationToken);
    Task<Stream> GetStreamAsync (Uri? requestUri, CancellationToken cancellationToken);
    Task<Stream> GetStreamAsync (string? requestUri, CancellationToken cancellationToken);
    Task<Stream> GetStreamAsync (Uri? requestUri);
    Task<Stream> GetStreamAsync (string? requestUri);
    Task<string> GetStringAsync (string? requestUri);
    Task<string> GetStringAsync (Uri? requestUri);
    Task<string> GetStringAsync (string? requestUri, CancellationToken cancellationToken);
    Task<string> GetStringAsync (Uri? requestUri, CancellationToken cancellationToken);
    Task<HttpResponseMessage> PatchAsync (string? requestUri, HttpContent? content);
    Task<HttpResponseMessage> PatchAsync (Uri? requestUri, HttpContent? content);
    Task<HttpResponseMessage> PatchAsync (string? requestUri, HttpContent? content, CancellationToken cancellationToken);
    Task<HttpResponseMessage> PatchAsync (Uri? requestUri, HttpContent? content, CancellationToken cancellationToken);
    Task<HttpResponseMessage> PostAsync (string? requestUri, HttpContent? content, CancellationToken cancellationToken);
    Task<HttpResponseMessage> PostAsync (Uri? requestUri, HttpContent? content, CancellationToken cancellationToken);
    Task<HttpResponseMessage> PostAsync (string? requestUri, HttpContent? content);
    Task<HttpResponseMessage> PostAsync (Uri? requestUri, HttpContent? content);
    Task<HttpResponseMessage> PutAsync (string? requestUri, HttpContent? content);
    Task<HttpResponseMessage> PutAsync (Uri? requestUri, HttpContent? content);
    Task<HttpResponseMessage> PutAsync (string? requestUri, HttpContent? content, CancellationToken cancellationToken);
    Task<HttpResponseMessage> PutAsync (Uri? requestUri, HttpContent? content, CancellationToken cancellationToken);
    [UnsupportedOSPlatform("browser")]
    HttpResponseMessage Send (HttpRequestMessage request);
    [UnsupportedOSPlatform("browser")]
    HttpResponseMessage Send (HttpRequestMessage request, HttpCompletionOption completionOption);
    [UnsupportedOSPlatform("browser")]
    HttpResponseMessage Send (HttpRequestMessage request, CancellationToken cancellationToken);
    [UnsupportedOSPlatform("browser")]
    HttpResponseMessage Send (HttpRequestMessage request, HttpCompletionOption completionOption, CancellationToken cancellationToken);
    Task<HttpResponseMessage> SendAsync (HttpRequestMessage request, HttpCompletionOption completionOption, CancellationToken cancellationToken);
    Task<HttpResponseMessage> SendAsync (HttpRequestMessage request);
    Task<HttpResponseMessage> SendAsync (HttpRequestMessage request, HttpCompletionOption completionOption);
    Task<HttpResponseMessage> SendAsync (HttpRequestMessage request, CancellationToken cancellationToken);
}

public interface HttpMessageInvoker
{
    [UnsupportedOSPlatform("browser")]
    HttpResponseMessage Send (HttpRequestMessage request, CancellationToken cancellationToken);
    Task<HttpResponseMessage> SendAsync (HttpRequestMessage request, CancellationToken cancellationToken);
}

As a programmer who just wants to get shit done, you mutter a few curses and then use the easiest method that does everything you need.

using var httpClient = new HttpClient();
var jsonText = await httpClient.GetStringAsync("https://apiserver/resource");

I can hardly blame you for writing code like this. This is good enough at first. Sadly everything you need is always a moving target. Suppose you want to add gzip compression, you can't do it with GetStringAsync. You will have to rewrite this code and dig deeper into all the available options. Or you can just follow along, because I just did it for you.

Dissecting the interface

I can see overloads that do the same thing but act on different representation of a URI.

Task<HttpResponseMessage> GetAsync (string? requestUri);
Task<HttpResponseMessage> GetAsync (Uri? requestUri);

Next they added cancellation support. Normally that means you have just 1 new overload, but in this case there are 2.

Task<HttpResponseMessage> GetAsync (string? requestUri);
Task<HttpResponseMessage> GetAsync (string? requestUri, CancellationToken cancellationToken); // NEW
Task<HttpResponseMessage> GetAsync (Uri? requestUri);
Task<HttpResponseMessage> GetAsync (Uri? requestUri, CancellationToken cancellationToken); // NEW

The same also applies to overloads for other HTTP verbs.

Task<HttpResponseMessage> PostAsync (string? requestUri, HttpContent? content);
Task<HttpResponseMessage> PostAsync (string? requestUri, HttpContent? content, CancellationToken cancellationToken);
Task<HttpResponseMessage> PostAsync (Uri? requestUri, HttpContent? content, CancellationToken cancellationToken);
Task<HttpResponseMessage> PostAsync (Uri? requestUri, HttpContent? content);

I'm already starting to feel anxious about the first design choice (adding overloads for string and for Uri). Instead of 2 methods, there are now 4 methods. Add another optional argument to each overload and there will be 8 methods that do almost the same thing slightly differently.

This is indeed what happened. You can specify how soon the returned Task should complete: as soon as the response headers are received, or after the entire content is buffered, by passing the HttpCompletionOption.

The number of available methods is quickly escalating. Yikes!

Task<HttpResponseMessage> GetAsync (string? requestUri);
Task<HttpResponseMessage> GetAsync (string? requestUri, HttpCompletionOption completionOption);
Task<HttpResponseMessage> GetAsync (string? requestUri, CancellationToken cancellationToken);
Task<HttpResponseMessage> GetAsync (string? requestUri, HttpCompletionOption completionOption, CancellationToken cancellationToken);
Task<HttpResponseMessage> GetAsync (Uri? requestUri);
Task<HttpResponseMessage> GetAsync (Uri? requestUri, HttpCompletionOption completionOption);
Task<HttpResponseMessage> GetAsync (Uri? requestUri, CancellationToken cancellationToken);
Task<HttpResponseMessage> GetAsync (Uri? requestUri, HttpCompletionOption completionOption, CancellationToken cancellationToken);

We really can't add any more optional arguments to this. The number of methods would explode. Yet we still want the ability to set request headers like Authorization, Accept-Language, Accept-Encoding, Cache-Control and much more.

Conversely, it's also worth noting that they did not add HttpCompletionOption to PostAsync even though it could be useful.

Clearly this design isn't scalable and potentially frustrating to work with. A better solution is to introduce a Parameter Object. Which is actually sort of what they did. If you squint hard enough, the HttpRequestMessage class can be considered a Parameter Object.

Task<HttpResponseMessage> SendAsync (HttpRequestMessage request);
Task<HttpResponseMessage> SendAsync (HttpRequestMessage request, CancellationToken cancellationToken);
Task<HttpResponseMessage> SendAsync (HttpRequestMessage request, HttpCompletionOption completionOption);
Task<HttpResponseMessage> SendAsync (HttpRequestMessage request, HttpCompletionOption completionOption, CancellationToken cancellationToken);

The SendAsync method does not provide the same level of convenience, but it does provide all the flexibility you could ever need. You can fully customize the request and also specify the HttpCompletionOption behavior of the HttpClient, with optional cancellation, across all HTTP verbs.

For these reasons, I recommend that you only ever use SendAsync and ignore the convenience methods. Your future self will be grateful.

Making SendAsync easier to use

Now that we've established you should only ever use SendAsync, your code from before has taken a left turn.

using var httpClient = new HttpClient();
using var request = new HttpRequestMessage(HttpMethod.Get, "https://apiserver/resource")
{
    Headers =
    {
        AcceptEncoding = { new StringWithQualityHeaderValue("gzip") }
    }
};
using var response = await httpClient.SendAsync(request);
response.EnsureSuccessStatusCode();
var contentStream = await response.Content.ReadAsStreamAsync();
if (response.Content.Headers.ContentEncoding.Contains("gzip"))
{
    contentStream = new GZipStream(contentStream, CompressionMode.Decompress);
}

var charset = Encoding.GetEncoding(response.Content.Headers.ContentType?.CharSet ?? "utf-8");
using var reader = new StreamReader(contentStream, charset);
var jsonText = await reader.ReadToEndAsync();

What used to be a one-liner is now some very dense garbage and no amount of documentation can ease the emotional pain one experiences from reading this code.

Anyway the problem is we've forgotten to apply the core principle of OOP: bundling data and behavior that acts upon it. As it currently stands, the data is in HttpRequestMessage while the behavior is spread across HttpClient and your own code.

Luckily it is a problem that can be rectified by adding a new bundle (class).

The way you design your class is more of an art than a science, but I propose standardizing on the following interface for all requests. The HttpClient is passed in as an argument so you retain control over its object lifetime.

interface IHttpRequest<T>
{
    Task<T> SendAsync(HttpClient httpClient);
    Task<T> SendAsync(HttpClient httpClient, CancellationToken cancellationToken);
}

Create a class that implements it and move all those ugly, ugly request details from before into its SendAsync method.

class JsonHttpRequest : IHttpRequest<string>
{
    private readonly string requestUri;

    public JsonHttpRequest(string requestUri)
    {
        this.requestUri = requestUri;
    }

    public Task<string> SendAsync(HttpClient httpClient)
    {
        return SendAsync(httpClient, CancellationToken.None);
    }

    public Task<string> SendAsync(HttpClient httpClient, CancellationToken cancellationToken)
    {
        using var request = new HttpRequestMessage(HttpMethod.Get, requestUri)
        {
            Headers =
            {
                AcceptEncoding = { new StringWithQualityHeaderValue("gzip") }
            }
        };
        using var response = await httpClient.SendAsync(request);
        response.EnsureSuccessStatusCode();
        var contentStream = await response.Content.ReadAsStreamAsync();
        if (response.Content.Headers.ContentEncoding.Contains("gzip"))
        {
            contentStream = new GZipStream(contentStream, CompressionMode.Decompress);
        }

        var charset = Encoding.GetEncoding(response.Content.Headers.ContentType?.CharSet ?? "utf-8");
        using var reader = new StreamReader(contentStream, charset);
        return await reader.ReadToEndAsync();
    }
}

Now you can clean up your original code again.

using var httpClient = new HttpClient();
var request = new JsonHttpRequest("https://apiserver/resource");
var jsonText = await request.SendAsync(httpClient);

Neat! From the perspective of this code, there is almost perfect encapsulation. It would be even better if we didn't have to pass in the HttpClient but then we lose the ability to reuse it across multiple requests. Creating and destroying the HttpClient for each request is not recommended.

You can add optional properties to your request class to further configure its behavior. Maybe some requests require an access token.

class JsonHttpRequest : IHttpRequest<string>
{
+    public string? AccessToken { get; set; }

    public Task<string> SendAsync(HttpClient httpClient, CancellationToken cancellationToken)
    {
        using var request = new HttpRequestMessage(HttpMethod.Get, requestUri)
        {
            Headers =
            {
                AcceptEncoding = { new StringWithQualityHeaderValue("gzip") }
            }
        };

+        if (!string.IsNullOrEmpty(AccessToken))
+        {
+            request.Headers.Authorization = new AuthenticationHeaderValue("Bearer", AccessToken);
+        }

        using var response = await httpClient.SendAsync(request);
        response.EnsureSuccessStatusCode();
        var contentStream = await response.Content.ReadAsStreamAsync();
        if (response.Content.Headers.ContentEncoding.Contains("gzip"))
        {
            contentStream = new GZipStream(contentStream, CompressionMode.Decompress);
        }

        var charset = Encoding.GetEncoding(response.Content.Headers.ContentType?.CharSet ?? "utf-8");
        using var reader = new StreamReader(contentStream, charset);
        return await reader.ReadToEndAsync();
    }
}

Usage

using var httpClient = new HttpClient();
var request = new JsonHttpRequest("https://apiserver/resource")
{
    AccessToken = "a real token"
};
var jsonText = await request.SendAsync(httpClient);

Don't fall into the trap of creating a single Request class that does everything. It's recommendable to create highly specialized request classes. You can reuse some behaviors by composing requests.

As an example, you could create request classes that return materialized objects from JSON, one class for each materialized thing.

class MaterializedThing
{
    public int Id { get; set; }
    public string? Name { get; set; }
    public string? LongDescription { get; set; }
}

class MaterializedThingRequest : IHttpRequest<MaterializedThing>
{
    public string? AccessToken { get; set; }

    public Task<string> SendAsync(HttpClient httpClient)
    {
        return SendAsync(httpClient, CancellationToken.None);
    }

    public Task<MaterializedThing> SendAsync(HttpClient httpClient, CancellationToken cancellationToken)
    {
        var request = new JsonHttpRequest("https://apiserver/resource")
        {
            AccessToken = AccessToken
        };

        var jsonText = await request.SendAsync(httpClient, cancellationToken);
        using var jsonDocument = JsonDocument.Parse(jsonText);
        return new MaterializedThing
        {
            Id = jsonDocument.RootElement.GetProperty("id").GetInt32(),
            Name = jsonDocument.RootElement.GetProperty("name").GetString(),
            LongDescription = jsonDocument.RootElement.GetProperty("long_desc").GetString()
        }
    }
}

Usage

using var httpClient = new HttpClient();
var request = new MaterializedThingRequest()
{
    AccessToken = "a real token"
};
var materializedThing = await request.SendAsync(httpClient);

You would create many of these request classes, one for every representation of a resource. Where possible you should reuse the generic JsonHttpRequest but it's also perfectly fine to make it a one-off implementation.

One final note is that you can override some defaults of the HttpClient such as DefaultRequestHeaders, but I urge you not to do that. Firstly it is not thread-safe. Secondly it means your request class no longer encapsulates all details about the request.

Don't do it!

using var httpClient = new HttpClient();

// Don't do it!
// Especially don't do it if your HttpClient is injected from a Dependency Injection provider!!
httpClient.DefaultRequestHeaders.Authorization = 
    new AuthenticationHeaderValue("Bearer", "a real token");
var request = new MaterializedThingRequest();
var materializedThing = await request.SendAsync(httpClient);

Summary

Ignore convenience methods of the HttpClient, only use SendAsync
Bundle request options and behaviors in custom request classes
Reuse behaviors by composing request classes
Avoid overriding defaults of the HttpClient

sliekens/encapsulated-requests.md

Encapsulate your requests

HttpClient

Dissecting the interface

Making SendAsync easier to use

Summary