The Azure Cosmos DB Blog

All about Azure Cosmos DB and Azure

Considering Software as an IoT Device

An Azure IoT Hub can store just about any type of data from a Device.

There is support for:

  • Sending Device to Cloud messages.
  • Invoking direct methods on a device
  • Uploading files from a device
  • Managing Device Identities
  • Scheduling Jobs on single for multiple devices

The following is the List of of built-in endpoints

Custom Endpoints can also be created.

IoT Hub currently supports the following Azure services as additional endpoints:

  • Azure Storage containers
  • Event Hubs
  • Service Bus Queues
  • Service Bus Topics

Architecture

If we look through the documentation on the Azure Architecture Center, we can see a list of Architectural Styles.

If we were to design an IoT Solution, we would want to follow Best Practices. We can do this by using the Azure Architectural Style of Event Driven Architecture. Event-driven architectures are central to IoT solutions.

Merging Event Driven Architecture with Microservices can be used to separate the IoT Business Services.
These services include:

  • Provisioning
  • Management
  • Software Updating
  • Security
  • Logging and Notifications
  • Analytics

Creating our services

To create these services, we start by selecting our Compute Options.

App Services

The use of Azure Functions is becoming commonplace. They are an excellent replacement for API Applications. And they can be published to Azure Api Management.

We are able to create a Serverless API, or use Durable Functions that allow us to create workflows and maintain state in a serverless environment.

Logic Apps provide us with the capability of building automated scalable workflows.

Data Store

Having a single data store is usually not the best approach. Instead, it’s often better to store different types of data in different data stores, each focused towards a specific workload or usage pattern. These stores include Key/value stores, Document databases, Graph databases, Column-family databases, Data Analytics, Search Engine databases, Time Series databases, Object storage, and Shared files.

This may hold true for other Architectural Styles. In our Event-driven Architecture, it is ideal to store all data related to IoT Devices in the IoT Hub. This data includes results from all events within the Logic Apps, Function Apps, and Durable Functions.


Which brings us back to our topic… Considering Software as an IoT Device

Since Azure IoT supports the TransportType.Http1 protocol, we can use the Microsoft.Azure.Devices.ClientLibrary to send Event data to our IoT Hub from any type of software. We also have the capability of receiving configuration data from the IoT Hub.

The following is the source code for our SendEvent Function App.

SendEvent Function App

#region Information

//  
//  MIT License
//  
//  Copyright (c) 2018  Howard Edidin
//  
//  Permission is hereby granted, free of charge, to any person obtaining a copy
//  of this software and associated documentation files (the "Software"), to deal
//  in the Software without restriction, including without limitation the rights
//  to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
//  copies of the Software, and to permit persons to whom the Software is
//  furnished to do so, subject to the following conditions:
//  
//  The above copyright notice and this permission notice shall be included in all
//  copies or substantial portions of the Software.
//  
//  THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
//  IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
//  FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
//  AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
//  LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
//  OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
//  SOFTWARE.

#endregion

#region

using System;
using System.Collections.Generic;
using System.Configuration;
using System.Data.Services.Client;
using System.Net;
using System.Net.Http;
using System.Text;
using System.Threading.Tasks;
using Microsoft.Azure.Devices.Client;
using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Extensions.Http;
using Microsoft.Azure.WebJobs.Host;
using Newtonsoft.Json;
using TransportType = Microsoft.Azure.Devices.Client.TransportType;

#endregion

namespace IoTHubClient
{
    public static class SendEvent
    {
        private static readonly string IotHubUri = ConfigurationManager.AppSettings["hubEndpoint"];

        [FunctionName("SendEventToHub")]
        public static async Task<HttpResponseMessage> Run(
            [HttpTrigger(AuthorizationLevel.Function, "post", Route = "device/{id}/{key:guid}")]
            HttpRequestMessage req, string id, Guid key, TraceWriter log)
        {
            log.Info("C# HTTP trigger function processed a request.");


            // Get request body
            dynamic data = await req.Content.ReadAsAsync<object>();

            var deviceId = id;
            var deviceKey = key.ToString();

            if (string.IsNullOrEmpty(deviceKey) || string.IsNullOrEmpty(deviceId))
                return req.CreateResponse(HttpStatusCode.BadRequest, "Please pass a deviceid and deviceKey in the Url");

            var telemetry = new Dictionary<Guid, object>();


            foreach (var item in data.telemetryData)
            {
                var telemetryData = new TelemetryData
                {
                    MetricId = item.metricId,
                    MetricValue = item.metricValue,
                    MericDateTime = item.metricDateTime,
                    MetricValueType = item.metricValueType
                };

                telemetry.Add(Guid.NewGuid(), telemetryData);
            }


            var deviceData = new DeviceData
            {
                DeviceId = deviceId,
                DeviceName = data.deviceName,
                DeviceVersion = data.deviceVersion,
                DeviceOperation = data.deviceOperation,
                DeviceType = data.deviceType,
                DeviceStatus = data.deviceStatus,
                DeviceLocation = data.deviceLocation,
                SubscriptionId = data.subcriptionId,
                ResourceGroup = data.resourceGroup,
                EffectiveDateTime = new DateTimeOffset(DateTime.Now),
                TelemetryData = telemetry
            };


            var json = JsonConvert.SerializeObject(deviceData);

            var message = new Message(Encoding.ASCII.GetBytes(json));


            try
            {
                var client = DeviceClient.Create(IotHubUri, new DeviceAuthenticationWithRegistrySymmetricKey(deviceId, deviceKey),
                    TransportType.Http1);

                await client.SendEventAsync(message);

                return req.CreateResponse(HttpStatusCode.OK);
            }
            catch (DataServiceClientException e)
            {
                var resp = new HttpResponseMessage
                {
                    StatusCode = (HttpStatusCode) e.StatusCode,
                    Content = new StringContent(e.Message)
                };
                return resp;
            }
        }
    }


    public class DeviceData
    {
        public string DeviceId { get; set; }

        public string DeviceName { get; set; }

        public string DeviceVersion { get; set; }

        public string DeviceType { get; set; }

        public string DeviceOperation { get; set; }

        public string DeviceStatus { get; set; }

        public DeviceLocation DeviceLocation { get; set; }

        public string AzureRegion { get; set; }

        public string ResourceGroup { get; set; }

        public string SubscriptionId { get; set; }

        public DateTimeOffset EffectiveDateTime { get; set; }

        public Dictionary<Guid, object> TelemetryData { get; set; }
    }

    public class TelemetryData
    {
        public string MetricId { get; set; }

        public string MetricValueType { get; set; }

        public string MetricValue { get; set; }

        public DateTime MericDateTime { get; set; }
    }

    public enum DeviceLocation
    {
        Cloud,
        Container,
        OnPremise
    }
}

Software Device Properties

The following values are required in the Url Path

Route = "device/{id}/{key:guid}")

Name Description
id Device Id (String)
key Device Key (Guid)

The following are the properties to be sent in the Post Body
Name Description
deviceName Device Name
deviceVersion Device version number
deviceType Type of Device
deviceOperation Operation name or type
deviceStatus Default: Active
deviceLocation Cloud
Container
OnPremise
subscriptionId Azure Subscription Id
resourceGroup Azure Resource group
azureRegion Azure Region
telemetryData Array
telemetryData.metricId Array item id
telemetryData.metricValueType Array item valueType
telemetryData.metricValue Array item value
telemetryData.metricTimeStamp Array item TimeStamp

Summary

  • We can easily add the capability of sending messages and events to our Function and Logic Apps.
  • Optionally, we can send the data to an Event Grid.
  • We have a single data store for all our IoT events.
  • We can identify performance issues within our services.
  • Having a single data store makes it easier to perform Analytics.
  • We can use a Azure Function App to Send Device to Cloud Messages. In this case our Function App will be also be taking the role of a Device.

Saving Medical Device to Cloud messages as HL7 FHIR Observation resources

In this tutorial we are going to learn how to get our Medical Device to Cloud data.

   Note:
This tutorial is from a chapter in the HL7 FHIR on Azure – Device Framework eBook.

The following are the Requirements.

Requirements

  • Ability to capture our Medical Device output data.
  • Ability to filter the data.
  • Ability to update the Observation Resource document with the output data.
  • Ability to modify the EndPoint Response after updating the Observation Resource document.
  • Ability to parse the Observation Resource document and post it to any EndPoint.

Solution

We are going to create a Function App, with a EventHubTrigger to do the following:

  1. Loop through the EventHub messages.
  2. Create a EventGrid message.
  3. Post the EventGrid message.

Next we will create a Function App, DeviceGrid that handles the updating of the Observation Resource document.

  1. Subscribes to our EventGrid messages
  2. Upserts the Observation Resource document with the data from the EventGrid messages

Leverage Azure API Management Policies.

  1. Publish DeviceGrid Function App to Azure API Management.
  2. Create a policy that subscribes to the EventGrid topic.
  3. Extract specific device data and post it to any EndPoint.

Function Apps

EventHubTrigger – Function App

We are using an EventHubTrigger to loop through the EventHub Messages.
Then we create a EventGrid message and publish it.

Source Code
#region

using System;
using System.Configuration;
using System.Globalization;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Security.Cryptography;
using System.Text;
using System.Threading.Tasks;
using System.Web;
using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Host;
using Microsoft.Azure.WebJobs.ServiceBus;
using Newtonsoft.Json;

#endregion

namespace DeviceMetrics
{
    public static class DeviceToCloud
    {
        private static readonly string TopicEndpoint = ConfigurationManager.AppSettings["topicEndpoint"];
        private static readonly string TopicKey = ConfigurationManager.AppSettings["topicKey"];

        [FunctionName("DeviceToCloud")]
        public static void Run([EventHubTrigger("DeviceToCloud", Connection = " EventHubConnectionString")]
            string[] myEventHubMessages, TraceWriter log)
        {
            log.Info($"C# Event Hub trigger function processed a message: {myEventHubMessages.Length}");

            foreach (var msg in myEventHubMessages)
            {
                dynamic data = JsonConvert.DeserializeObject(msg);

                var eventGridMessage = new GridEvent
                {
                    Id = Guid.NewGuid().ToString(),
                    EventTime = DateTime.UtcNow,
                    EventType = data.Properties.EventType, 
                    Data = data.Properties.Data,
                    Subject = data.Properties.Name,
                    Topic = data.Properties.Topic,
                    DeviceId = data.Id
                };


                try
                {
                    SendEvent(TopicEndpoint, TopicKey, eventGridMessage).ConfigureAwait(false);
                }
                catch (HttpRequestException e)
                {
                    log.Error(e.Message, e);
                }
            }
        }


        public static async Task SendEvent(string topicEndpoint, string topicKey, object data)
        {
            // Create a SAS token for the call to the event grid. We can do this with 
            // the SAS key as well but wanted to show an alternative.
            var sas = CreateEventGridSasToken(topicEndpoint, DateTime.Now.AddDays(1), topicKey);

            // Instantiate an instance of the HTTP client with the 
            // event grid topic endpoint.
            var client = new HttpClient {BaseAddress = new Uri(topicEndpoint)};

            // Configure the request headers with the content type
            // and SAS token needed to make the request.
            client.DefaultRequestHeaders.Accept.Clear();
            client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
            client.DefaultRequestHeaders.Add("aeg-sas-token", sas);

            // Serialize the data
            var json = JsonConvert.SerializeObject(data);
            var stringContent = new StringContent(json, Encoding.UTF8, "application/json");

            // Publish grid event
            await client.PostAsync(string.Empty, stringContent);
        }


        private static string CreateEventGridSasToken(string resourcePath, DateTime expirationUtc, string topicKey)
        {
            const char resource = 'r';
            const char expiration = 'e';
            const char signature = 's';

            // Encode the topic resource path and expiration parameters
            var encodedResource = HttpUtility.UrlEncode(resourcePath);
            var encodedExpirationUtc = HttpUtility.UrlEncode(expirationUtc.ToString(CultureInfo.InvariantCulture));

            // Format the unsigned SAS token
            var unsignedSas = $"{resource}={encodedResource}&{expiration}={encodedExpirationUtc}";

            // Create an HMCASHA256 policy with the topic key
            using (var hmac = new HMACSHA256(Convert.FromBase64String(topicKey)))
            {
                // Encode the signature and create the fully signed URL with the
                // appropriate parameters.
                var bytes = Convert.ToBase64String(hmac.ComputeHash(Encoding.UTF8.GetBytes(unsignedSas)));
                var encodedSignature = HttpUtility.UrlEncode(bytes);
                var signedSas = $"{unsignedSas}&{signature}={encodedSignature}";

                return signedSas;
            }
        }
    }
}

DeviceEventGrid – Function App

Source Code

#region

using System;
using System.Collections.Generic;
using System.Configuration;
using System.Linq;
using System.Net;
using System.Net.Http;
using System.Threading.Tasks;
using Microsoft.Azure.Documents;
using Microsoft.Azure.Documents.Client;
using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Extensions.Http;
using Microsoft.Azure.WebJobs.Host;
using Newtonsoft.Json;

#endregion

namespace DeviceMetrics
{
    public static class DeviceEventGrid
    {
        private static readonly string Endpoint = ConfigurationManager.AppSettings["endpoint"];
        private static readonly string AuthKey = ConfigurationManager.AppSettings["authKey"];
        private static readonly string Database = ConfigurationManager.AppSettings["database"];
        private static readonly string Collection = ConfigurationManager.AppSettings["collection"];

        private static readonly DocumentClient Client = new DocumentClient(new Uri(Endpoint), AuthKey,
            new ConnectionPolicy
            {
                ConnectionMode = ConnectionMode.Direct,
                ConnectionProtocol = Protocol.Tcp
            }
        );


        [FunctionName("DeviceEventGrid")]
        public static async Task<HttpResponseMessage> Run(
            [HttpTrigger(AuthorizationLevel.Function, "post")]
            HttpRequestMessage req,
            TraceWriter log)
        {
            log.Info("C# HTTP trigger function processed a request.");


            var jsonContent = await req.Content.ReadAsStringAsync();
            var gridEvent = JsonConvert.DeserializeObject<List<GridEvent<Dictionary<string, string>>>>(jsonContent)
                ?.SingleOrDefault();

            if (gridEvent == null) return req.CreateErrorResponse(HttpStatusCode.BadRequest, $@"Missing event details");

            var gridEventType = req.Headers.GetValues("Aeg-Event-Type").FirstOrDefault();

            if (gridEventType == "SubscriptionValidation")
            {
                // Retrieve the validation code and echo back.
                var validationCode = gridEvent.Data["validationCode"];
                var validationResponse = JsonConvert.SerializeObject(new
                {
                    validationResponse = validationCode
                });

                return new HttpResponseMessage
                {
                    StatusCode = HttpStatusCode.OK,
                    Content = new StringContent(validationResponse)
                };
            }

            if (gridEventType == "Notification")
            {
                // Get the Observation Resource Document
                dynamic doc = await Client.ReadDocumentAsync(UriFactory.CreateDocumentUri(Database, Collection,
                    gridEvent.Data["deviceId"]));


                var valueQuanity = new ValueQuantity
                {
                    Code = doc.Component.ValueQuantity.Code,
                    System = doc.Component.ValueQuantity.System,
                    Unit = gridEvent.Data["unit"],
                    Value = gridEvent.Data["value"]
                };

                var component = new Component
                {
                    Code = doc.Component.Code,
                    ValueQuantity = valueQuanity
                };


                // partial values 
                var observation = new CompoundNumericObservation
                {
                    Id = doc.Id,
                    Component = {[0] = component},
                    EffectiveDateTime = new DateTimeOffset(DateTime.Parse(gridEvent.Data["timestamp"]))
                };


                try
                {
                    var result =
                        await Client.UpsertDocumentAsync(UriFactory.CreateDocumentUri(Database, Collection, gridEvent.Data["deviceId"]),
                            observation);

                    return req.CreateResponse(result.StatusCode);
                }
                catch (DocumentClientException e)
                {
                    var resp = new HttpResponseMessage
                    {
                        StatusCode = (HttpStatusCode) e.StatusCode,
                        Content = new StringContent(e.Message)
                    };
                    return resp;
                }
            }

            return req.CreateErrorResponse(HttpStatusCode.BadRequest,
                $@"Unknown request type");
        }

        public class GridEvent<T> where T : class
        {
            public string Id { get; set; }
            public string EventType { get; set; }
            public string Subject { get; set; }
            public DateTime EventTime { get; set; }
            public T Data { get; set; }
            public string Topic { get; set; }
        }
    }
}

Subscribing to EventGrid Events Using Azure API Management

After we pubish our Function App to Api Management, we will create a new Policy for our Function App. This allows us to extract specific device data post it to any EndPoint.

The following is an example policy.

<policies>
  <inbound>
    <base />
    <set-variable value="@(context.Request.Headers["Aeg-Event-Type"].Contains("SubscriptionValidation"))" name="isEventGridSubscriptionValidation" />
    <set-variable value="@(context.Request.Headers["Aeg-Event-Type"].Contains("Notification"))" name="isEventGridNotification" />
    <choose>
      <when condition="@(context.Variables.GetValueOrDefault<bool>("isEventGridSubscriptionValidation"))">
        <return-response>
          <set-status code="200" reason="OK" />
          <set-body>@{
            var events = context.Request.Body.As<string>();
            JArray a = JArray.Parse(events);
            var eventGridData = a.First["data"];
            var validationCode = eventGridData["validationCode"];
            var jOutput =
              new JObject(
                new JProperty("validationResponse", validationCode)
                );
            return jOutput.ToString();
          }</set-body>
        </return-response>
      </when>
      <when condition="@(context.Variables.GetValueOrDefault<bool>("isEventGridNotification"))">
        <send-one-way-request mode="new">
          <set-url>https://XXXXXXXXXXXXXXXXXXXXXX}</set-url>
          <set-method>POST</set-method>
          <set-body>@{
            var events = context.Request.Body.As<string>();
            JArray a = JArray.Parse(events);
            var eventGridData = a.First["data"];
            var deviceId = eventGridData["deviceId"];
            var deviceData = eventGridData["deviceData"];
            var patientd = eventGridData["patientd"];
            return new JObject(
                new JProperty("deviceId", deviceId),
                new JProperty("data", deviceData),
                new JProperty("patientd", patientd)                ;
          }</set-body>
        </send-one-way-request>
      </when>
    </choose>
  </inbound>
  <backend>
    <base />
  </backend>
  <outbound>
    <base />
  </outbound>
  <on-error>
    <base />
  </on-error>
</policies>

Summary

  • We are able to leverage the use an Azure EventGrid to create an EventGrid message.
  • We are using a Function App to loop through the EventHub messages.
  • We are able to create and publish EventGrid messages within the Function App.
  • We created another Function App that subscribes to our EventGrid Topic.
  • This Function App upserts the H7 FHIR Observation Resource with the ValueQuantity data.
  • We are using a Azure API management Policy to parse the ValueQuantity data and post it to any EndPoint.

Next Steps

  • We are going to create a Durable Function to orchestrate the message flow
« Older posts
%d bloggers like this:
Skip to toolbar