NOTE: This is tutorial is based upon the Next Steps from a previous post.

Requirements

As an Architect, I need to design an application that would receive a Document with a file from Form-Data posted from a Web Application. The Azure Cosmos DB Document is to be created and the file is to be inserted as an attachment. The file needs to be accessible from all the Azure Locations.
The files are too be used to update IoT devices. The contentType is
application/zip.

INFO: This application is part of a Azure IoT Devices Software Update Micro Service.
The file sizes are small, usually under 10 K. Once the Devices have been successfully updated, the attachments are archived to Azure Blog Storage.

Architecture

Architecture

Use Case

  • The Web Application is part of an IoT solution.
  • The user would either:
    1. Select a document from a drop-down list. The value would be the body of the JSON document for the Attachment.
    2. Fill-in the form fields. The values would become the body of the JSON document for the Attachment.
  • The user would select a zip file that contains the software update for a device.
  • The user would Send the form-data to a Function App.
  • The function app would return the StatusCode and the Resource data of the attachment.
  • If there was an error, the StatusCode and the error description would be returned.

NOTE: There are two ways to create an Attachment for a Document.

  1. Upload the fie to an external blob service and store the metadata for the Attachment in Azure Cosmos DB. This works well for large files, since there is a size limitation for a Cosmos DB Account. The total size of all your Attachments cannot exceed 2 GB.
  2. Store the Attachment media/blog with the Document so it’s managed by Cosmos DB.

INFO: You can read more about Attachments and Media Resource here.

Solution

Saving the file to a Azure Blob service would not meet the requirements. Azure Blob Storage provides replication limited to two regions. Also it would require my client to manage the files. We would have the same issues using OneDrive, Dropbox, or any other blob service.

I decided to have the files stored as Cosmos DB Attachments. Cosmos DB uses Azure Blob Storage internally. Because Cosmos DB provides for global distribution , the Attachments are available in more than thirty geographical regions.

Development

INFO: Rather than "reinvent the wheel" I referenced the Http Multipart Parser, developed by Jake Woods in the Function App.

The following is the source code for the Function App

namespace CosmosDBFormData
{
   public static class FormData
   {
      private static readonly string Endpoint = ConfigurationManager.AppSettings["endpoint"];
      private static readonly string AuthKey = ConfigurationManager.AppSettings["authKey"];
      private static readonly string Database = ConfigurationManager.AppSettings["database"];
      private static readonly string Collection = ConfigurationManager.AppSettings["collection"];

      private static readonly DocumentClient Client = new DocumentClient(new Uri(Endpoint), AuthKey,
         new ConnectionPolicy
         {
            MaxConnectionLimit = 60,
            ConnectionMode = ConnectionMode.Direct,
            ConnectionProtocol = Protocol.Tcp
         }
      );

      [FunctionName("FormData")]
      public static async Task<HttpResponseMessage> Run(
         [HttpTrigger(AuthorizationLevel.Function, "get", "post", Route = null)]
         HttpRequestMessage req, TraceWriter log)
      {
         var data = req.Content.ReadAsStreamAsync();

         var parser = new MultipartFormDataParser(data.Result);
         var document = parser.GetParameterValue("doc");
         var parsed = JsonConvert.DeserializeObject(document);

         var file = parser.Files.FirstOrDefault();
         if (file == null) return req.CreateResponse(HttpStatusCode.BadRequest);
         var fileName = file.FileName;
         var attachment = file.Data;

         var mediaOptions = new MediaOptions
         {
            Slug = fileName,
            ContentType = file.ContentType
         };

         try
         {
            var doc = await Client.CreateDocumentAsync(UriFactory.CreateDocumentCollectionUri(Database, Collection), parsed);

            try
            {

               var resp = await Client.CreateAttachmentAsync(UriFactory.CreateDocumentUri(Database, Collection, doc.Resource.Id),


                  attachment, mediaOptions);

               return req.CreateResponse(HttpStatusCode.OK, resp.Resource);

            }
            catch (DocumentClientException ex)
            {
               var statusCode = ex.StatusCode;
               return req.CreateErrorResponse((HttpStatusCode) statusCode, ex.Message);
            }

         }
         catch (DocumentClientException ex)
         {
            var statusCode = ex.StatusCode;
            return req.CreateErrorResponse((HttpStatusCode) statusCode, ex.Message);
         }
      }
   }
}

To test the function app, I used Postman, as shown in the following figure.

I used the following for the doc key value shown in the figure above.

{"secret": 1337, "trash": "' value='"}
The following is an example of Success response.
{
  "contentType": "application/zip",
  "media": "/media/jdUKAO35uQABAAAAAAAAAHejG+kB",
  "id": "success.zip",
  "_rid": "jdUKAO35uQABAAAAAAAAAHejG+k=",
  "_self": "dbs/jdUKAA==/colls/jdUKAO35uQA=/docs/jdUKAO35uQABAAAAAAAAAA==/attachments/jdUKAO35uQABAAAAAAAAAHejG+k=",
  "_ts": 1520007641,
  "_etag": "\"12000b2e-0000-0000-0000-5a9979d90000\""
}
The following is a example of an Error response

Failed

if we view the Cosmos DB Document, we can see that the attachment has been added.

Results

Summary

  • A Function App can be used to receive Form-Data
  • Using the Http Multipart/form-data parser eliminates custom code.
  • Ability to create a Cosmos DB Document and save an Attachment from a Function App.
  • Inserting a Cosmos DB Attachment as an embedded resource provides access to the attachment from any Azure Location.
  • Our file Attachments are managed by Cosmos DB.

Next Steps

  • Continue design and development of our IoT Software Update Micro Service.