Efficiently serving Blobs from Azure Storage in ASP.NET Core using custom ActionResult

Efficiently serving Blobs from Azure Storage in ASP.NET Core using custom ActionResult

You already may be using Azure Blob Storage to store some files in your application. Do you return those files to the user efficiently? There are a couple of things that are commonly done.

Anonymous public access

This is quite easy and requires little effort. As per the official documentation, you would enable anonymous public access on your blobs in the container. After uploading the file, you would put the URL of the blob in the response e.g. https://contoso.blob.core.windows.net/container1/dir1/file1.pdf

Taking this a step further, you could enable a CDN on the storage account, optionally prefixing the files. You would set up a CDN profile with an endpoint pointing to your blob storage account. The official documentation explains it a more simplistic way with images for the steps. You would then modify the URL of the blob before writing it into your response. For our blob above, you would end up with something like https://contoso.azureedge.net/container1/dir1/file1.pdf and if you add a prefix to the CDN endpoint origin, you can omit some path segments. For example, you can set the path to /container1/dir1 so that the resulting URL would be https://contoso.azureedge.net/file1.pdf. Using a CDN provides better a user experience and can help save costs on Blob Storage but that is not the focus of this post. There’s tons of information out there on this.

How about authentication?

So you have built your app but you want to decide if someone gets access to certain files. For example, in a muti-tenant application where the storage account is shared. Azure Blob Storage supports authentication using keys and Azure AD. Keys are meant for your application. For users in your organization (the tenant that owns the Azure Subscription in which the storage account exists), Azure AD may be sufficient to control permissions or using a more enterprise solution like SharePoint would be better. Unfortunately, this is not always the case, at least in my line of work.

The very first and fairly reliable option is to use ASP.NET Core, where you can control user access to your liking and in the process, hide away the intrinsics of where your files are stored from your end user. In most cases, you would either download the file to memory or local disk before serving it in your response.

public class SampleController : ControllerBase
{
    private readonly BlobServiceClient blobServiceClient;
 
    public SampleController(BlobServiceClient blobServiceClient)
    {
        this.blobServiceClient = blobServiceClient ?? throw new ArgumentNullException(nameof(blobServiceClient));
    }
 
    [HttpGet]
    public async Task<IActionResult> DownloadAsync()
    {
        // getting blob storage url from database or other sources, omitted for simplicity
        var containerClient = blobServiceClient.GetContainerClient("container1");
        var blobClient = containerClient.GetBlobClient("dir1/file1.pdf");
        using var ms = new MemoryStream();
        await blobClient.DownloadToAsync(ms);
        ms.Seek(0, SeekOrigin.Begin);
        return File(ms, "application/pdf");
    }
}

Look familiar? Yes! Why? It is the easiest way to do so, and if it works, why change it? Unfortunately, this tends to be inefficient in at least two scenarios I can think of:

  1. You want to support HTTP HEAD verb not just HTTP GET without having to download the file from Azure Blob Storage.
  2. You need clients to perform ranging when downloading the contents.

In both of these situations, you need to decide whether to download the contents of the file. The second situation particularly is useful for large files. Storage/Memory (volatile and non-volatile) is expensive, you do not always have enough resources to use for these downloads.

Solution? Give me the code

The solution to this is make use of the Azure.Storage.Blobs library with all the features it offers, including steaming download. If the file is large, a HTTP HEAD request will help the HTTP client (e.g. the browser) to make appropriate preparations before downloading such as checking for space, planning parallel downloads (e.g. download manager).

First, we create a custom implementation of ActionResult because the inbuilt FileStreamResult and FileContentResult expect you to have the complete stream or byte array with you. Basically, they only support local files or small files whose contents you can keep in memory. Here we create BlobStorageResult :

using Microsoft.AspNetCore.Mvc.Infrastructure;
using Microsoft.Extensions.DependencyInjection;
using System;
using System.Threading.Tasks;
 
namespace Microsoft.AspNetCore.Mvc
{
    /// <summary>
    /// Represents an <see cref="ActionResult"/> that when executed
    /// will write a file from a blob to the response.
    /// </summary>
    public class BlobStorageResult : FileResult
    {
        public BlobStorageResult(string blobUrl, string contentType) : base(contentType)
        {
            if (string.IsNullOrWhiteSpace(BlobUrl = blobUrl))
            {
                throw new ArgumentException($"'{nameof(blobUrl)}' cannot be null or whitespace.", nameof(blobUrl));
            }
        }
 
        /// <summary>Gets the URL for the block blob to be returned.</summary>
        public string BlobUrl { get; }
 
        /// <summary>Gets or sets the <c>Content-Type</c> header if the value is already known.</summary>
        public long? ContentLength { get; set; }
 
        /// <summary>
        /// Gets or sets whether blob properties should be retrieved before downloading.
        /// Useful when the file size is not known in advance
        /// </summary>
        public bool GetPropertiesBeforeDownload { get; set; }
 
        /// <inheritdoc/>
        public override Task ExecuteResultAsync(ActionContext context)
        {
            if (context == null) throw new ArgumentNullException(nameof(context));
 
            var executor = context.HttpContext.RequestServices.GetRequiredService<IActionResultExecutor<BlobStorageResult>>();
            return executor.ExecuteAsync(context, this);
        }
    }
}

To use it, we would simplify our controller to:

public class SampleController : ControllerBase
{
    [HttpGet]
    [HttpHead]
    public IActionResult DownloadAsync()
    {
        // getting blob storage url from database or other sources, omitted for simplicity
        var blobUrl = "https://contoso.blob.core.windows.net/container1/dir1/file1.pdf";
        return new BlobStorageResult(blobUrl, "application/pdf");
    }
}

Simple, right? Not so quick. We still need to implement IActionResultExecutor<BlobStorageResult> before it can be resolved in ExecuteResultAsync(…). Luckily that is not so hard. Given that BlobStorageResult inherits from FileResult, we can make use of the inbuilt FileResultExecutorBase. A blob is a file after all (well, most of the times).

using Azure;
using Azure.Storage.Blobs;
using Microsoft.AspNetCore.Http;
using Microsoft.Extensions.Logging;
using Microsoft.Net.Http.Headers;
using System;
using System.IO;
using System.Threading.Tasks;
 
namespace Microsoft.AspNetCore.Mvc.Infrastructure
{
    public class BlobStorageResultExecutor : FileResultExecutorBase, IActionResultExecutor<BlobStorageResult>
    {
        private readonly BlobServiceClient blobServiceClient;
 
        public BlobStorageResultExecutor(BlobServiceClient blobServiceClient, ILoggerFactory loggerFactory)
            : base(CreateLogger<BlobStorageResultExecutor>(loggerFactory))
        {
            this.blobServiceClient = blobServiceClient ?? throw new ArgumentNullException(nameof(blobServiceClient));
        }
 
        /// <inheritdoc/>
        public async Task ExecuteAsync(ActionContext context, BlobStorageResult result)
        {
            if (context == null) throw new ArgumentNullException(nameof(context));
            if (result == null) throw new ArgumentNullException(nameof(result));
 
            var cancellationToken = context.HttpContext.RequestAborted;
            var bub = new BlobUriBuilder(new Uri(result.BlobUrl));
            var containerClient = blobServiceClient.GetBlobContainerClient(bub.BlobContainerName);
            var client = containerClient.GetBlobClient(bub.BlobName);
 
            Logger.ExecutingBlobStorageResult(result);
 
            if (HttpMethods.IsHead(context.HttpContext.Request.Method))
            {
                // if values are not set, pull them from blob properties
                if (result.ContentLength is null || result.LastModified is null || result.EntityTag is null)
                {
                    // Get the properties of the blob
                    var response = await client.GetPropertiesAsync(cancellationToken: cancellationToken);
                    var properties = response.Value;
                    result.ContentLength ??= properties.ContentLength;
                    result.LastModified ??= properties.LastModified;
                    result.EntityTag ??= MakeEtag(properties.ETag);
                }
 
                SetHeadersAndLog(context: context,
                                 result: result,
                                 fileLength: result.ContentLength,
                                 enableRangeProcessing: result.EnableRangeProcessing,
                                 lastModified: result.LastModified,
                                 etag: result.EntityTag);
            }
            else
            {
                // if values are not set, pull them from blob properties
                if (result.GetPropertiesBeforeDownload)
                {
                    // Get the properties of the blob
                    var arp = await client.GetPropertiesAsync(cancellationToken: cancellationToken);
                    var properties = arp.Value;
                    result.ContentLength ??= properties.ContentLength;
                    result.LastModified ??= properties.LastModified;
                    result.EntityTag ??= MakeEtag(properties.ETag);
                }
 
                var (range, rangeLength, serveBody) = SetHeadersAndLog(context: context,
                                                                       result: result,
                                                                       fileLength: result.ContentLength,
                                                                       enableRangeProcessing: result.EnableRangeProcessing,
                                                                       lastModified: result.LastModified,
                                                                       etag: result.EntityTag);
 
                if (!serveBody)
                {
                    return;
                }
 
                // Download the blob in the specified range
                var hr = range is not null ? new HttpRange(range.From!.Value, rangeLength) : default;
                var response = await client.DownloadStreamingAsync(hr, cancellationToken: cancellationToken);
 
                // if LastModified and ETag are not set, pull them from streaming result
                var bdr = response.Value;
                var details = bdr.Details;
                if (result.LastModified is null || result.EntityTag is null)
                {
                    var httpResponseHeaders = context.HttpContext.Response.GetTypedHeaders();
                    httpResponseHeaders.LastModified = result.LastModified ??= details.LastModified;
                    httpResponseHeaders.ETag = result.EntityTag ??= MakeEtag(details.ETag);
                }
 
                var stream = bdr.Content;
                using (stream)
                {
                    await WriteAsync(context, stream);
                }
            }
        }
 
        /// <summary>
        /// Write the contents of the <see cref="BlobStorageResult"/> to the response body.
        /// </summary>
        /// <param name="context">The <see cref="ActionContext"/>.</param>
        /// <param name="stream">The <see cref="Stream"/> to write.</param>
        protected virtual Task WriteAsync(ActionContext context, Stream stream)
        {
            if (context == null) throw new ArgumentNullException(nameof(context));
            if (stream == null) throw new ArgumentNullException(nameof(stream));
 
            return WriteFileAsync(context: context.HttpContext,
                                  fileStream: stream,
                                  range: null, // prevent seeking
                                  rangeLength: 0);
        }
 
        private static EntityTagHeaderValue MakeEtag(ETag eTag) => new(eTag.ToString("H"));
 
    }
 
    internal static partial class Log
    {
        private static readonly Action<ILogger, string, string, Exception?> _executingFileResultWithNoFileName
            = LoggerMessage.Define<string, string>(
                eventId: new EventId(1, nameof(ExecutingBlobStorageResult)),
                logLevel: LogLevel.Information,
                formatString: "Executing {FileResultType}, sending file with download name '{FileDownloadName}'");
 
        private static readonly Action<ILogger, Exception?> _writingRangeToBody
            = LoggerMessage.Define(
                eventId: new EventId(17, nameof(WritingRangeToBody)),
                logLevel: LogLevel.Debug,
                formatString: "Writing the requested range of bytes to the body.");
 
        public static void ExecutingBlobStorageResult(this ILogger logger, BlobStorageResult result)
        {
            if (logger.IsEnabled(LogLevel.Information))
            {
                var resultType = result.GetType().Name;
                _executingFileResultWithNoFileName(logger, resultType, result.FileDownloadName, null);
            }
        }
 
        public static void WritingRangeToBody(this ILogger logger)
        {
            if (logger.IsEnabled(LogLevel.Debug))
            {
                _writingRangeToBody(logger, null);
            }
        }
    }
}

When I wrote this, I looked at the inbuilt implementation of FileStreamResultExecutor. At first, I thought I did not even need to make use of FileResult because it would be different till I realized using that would allow me to use the FileResultExecutorBase which does a lot of the boiler plate such as:

  1. Validating ranges provided in the request header and deciding if we should write the body.
  2. Setting the appropriate response headers (Content-Length, Content-Type, If-Range, Range, Content-Range, etc) and response code (200, 206 or 416).

What does the proposed approach achieve?

  1. Allow the client to request a range and when they do so, we only download the range from Azure Blob Storage using DownloadStreamingAsync(…)
  2. If we have details required for the HTTP HEAD request (e.g. from the database), we can avoid our call to Azure Blob Storage.
  3. When we do not have the details required for the HTTP HEAD request we can fetch the blob properties only which is faster and smaller.

Finally, setting up the services is easy-peasy:

using Azure.Storage.Blobs;
using Microsoft.AspNetCore.Mvc;
using Microsoft.AspNetCore.Mvc.Infrastructure;
using Microsoft.Extensions.Configuration;
using System;
 
namespace Microsoft.Extensions.DependencyInjection
{
    public static class IServiceCollectionExtensions
    {
        public static IServiceCollection AddBlobStorage(this IServiceCollection services, IConfiguration configuration)
        {
            if (services == null) throw new ArgumentNullException(nameof(services));
 
            // Endpoint should look like "https://{account_name}.blob.core.windows.net" except for emulator
            var endpoint = configuration.GetValue<string>("Endpoint") ?? "UseDevelopmentStorage=true;";
            var containerName = configuration.GetValue<string>("ContainerName");
 
            services.AddSingleton<BlobServiceClient>(provider =>
            {
                // Create the credential, you can use connection string instead
                var credential = new DefaultAzureCredential();
                /*
                 * TokenCredential cannot be used with non https scheme.
                 * So we have to check if we are local and use a connection string instead.
                 * https://github.com/Azure/azure-sdk/issues/2195
                 */
                var isLocal = endpoint.StartsWith("http://127.0.0.1") || endpoint.Contains("UseDevelopmentStorage");
                return isLocal
                        ? new BlobServiceClient(connectionString: "UseDevelopmentStorage=true;")
                        : new BlobServiceClient(serviceUri: new Uri(endpoint), credential: credential);
            });
 
            services.AddScoped<BlobContainerClient>(provider =>
            {
                var serviceClient = provider.GetRequiredService<BlobServiceClient>();
                return serviceClient.GetBlobContainerClient(containerName);
            });
 
            services.AddSingleton<IActionResultExecutor<BlobStorageResult>, BlobStorageResultExecutor>();
 
            return services;
        }
    }
}

You can then call services.AddBlobStorage(...) in your Startup.cs file.

Even though this may cover most scenarios, it does not support Page blobs (or may be it does by mistake? Let me know), or passing of ContainerName and BlobName instead of BlobUrl but that you can modify in your application.