BenJarvis

Ben Jarvis' Blog

Azure Functions Key Vault Integration

Introduction

A really useful feature that recently came in to public preview for Azure Functions is native integration with Key Vault when retrieving application settings. This feature allows you to source applications settings directly from Key Vault without making any changes to your function code and without having to implement any additional infrastructure code to handle connecting to Key Vault and retrieving secrets. The original blog post announcing this feature can be found at https://azure.microsoft.com/en-us/blog/simplifying-security-for-serverless-and-web-apps-with-azure-functions-and-app-service/.

The Key Vault integration leverages another recent Azure Functions feature called Managed Identities. When creating a function app you can easily create a system assigned managed identity by enabling it through the Azure Portal or including a property within the ARM template (see https://docs.microsoft.com/en-us/azure/app-service/overview-managed-identity#adding-a-system-assigned-identity for more information); this allows you to interact with many Azure services using Azure AD authentication without the need to worry about managing service principals and the inevitable downtime that occurs when their keys expire.

As the feature is currently in preview there is a limitation in that it doesn’t support rotation of the secrets meaning only a single version of the secret is supported however, I’ll run through a workaround for that issue in this post.

Now that we’ve covered the basics of what the feature does I’ll run through a quick demo to demonstrate how it can help us when we’re developing function apps. This post isn’t intended to be an introduction to Azure Functions or Azure Key Vault so I have assumed a basic knowledge of each and only covered the elements of the set up that are related to the Key Vault integration.

Demo

The first step is to create our resources in Azure. For the demo we need to create new resource group that contains a function app with its associated storage account and an instance of Azure Key Vault:

image

Next, we need to enable the system assigned managed identity on our function app, we do this by navigating to the function app in the portal, clicking on the “Platform Features” tab and then clicking on “Identity”:

image

From there we can set the status of the system assigned managed identity to “On” and click save to apply the change:

image

Next, we need to grant our function app permissions to retrieve secrets from Key Vault; we do this by navigating to Key Vault in the portal and clicking on “Access Policies”:

image

Now we need to click on “Add New” and click on the “Select Principal” panel that is displayed, which brings up a new blade where we can search for the managed identity created for our function app in Azure AD:

image

Now we’ve selected our principal we need to give it the appropriate permissions. To retrieve secrets our function app only needs “Get” permissions on secrets so we can select that option and click ok:

image

The Key Vault UI isn’t the most intuitive as most people forget the next step but we need to click the save button to commit the changes to our access policies:

image

Now that we’ve created our Azure resources and set Azure Functions up so that it can communicate with Key Vault we need to add a secret to Key Vault and get our function app to retrieve it. To do this I have created the following script that will add the given secret to Key Vault and then set the required application settings on our function app to allow the value to be retrieved:

Once we run the script our secret will be added to Key Vault and the relevant application setting that acts as a pointer to the secret is added to our function app meaning that if we always use this script to deploy our secrets our function app will always be using the latest version of the secret meaning we are able to work around the current limitation that means the url in the application setting for our function app must include the version of the secret. After running the script we can take a look at the application settings for our function app and see that the reference to the secret is added:

image

Now we’ve added our secret to Key Vault and created the reference to the secret in our function app we can test that our functions are able to retrieve the secret. To do this we can create a new HTTP triggered function that has the following code to use the Environment.GetEnvironmentVariable function to retrieve the value of our application setting:

When we run the function the result returned is as follows, which matches the value we added to Key Vault! Obviously, in the real world we wouldn’t want to expose the value of our secret through outside of our function but this allows us to see the value that was returned by the app setting.

Conclusion

To conclude, we’ve shown how easy it is to integrate Azure Functions with Azure Key Vault. With the newly released integration we can leverage managed identities to access Key Vault without the need to write any additional code or have the overhead of managing service principals, this means we can ensure that our secrets are stored safely in Key Vault thereby improving the security of our serverless application.

Analysis of Spatial Data Using Cosmos DB

Introduction

Recently, while researching Cosmos DB, I came across the in-built capabilities for managing spatial data.

Cosmos DB is Microsoft’s globally distributed, multi-model database. It has the capability to store various types of data, such as document, graph and key-value and can elastically scale to cope with varying needs. The piece of Cosmos DB that this post will be discussing is the spatial capabilities of the document model.

The problem I have chosen to solve using the spatial functionality is working out which airfields are within the range of an aircraft when given its true airspeed and fuel endurance in hours; with the range being calculated by multiplying the airspeed by the endurance.

The Data

The first step was to find a data set containing a list of all of the world’s airfields, this was found on GitHub at https://github.com/mwgg/Airports. The data set contains the details we need, which are:

  • ICAO code – this is the unique identifier for an airport
  • Airport Name
  • Latitude and Longitude of the Airport

The next step was to create a Cosmos DB account in the Azure Portal and write a C# app to do some transformations on the data and load the documents into our Cosmos DB collection.

I first created a C# object that matched the structure of the JSON data:

using Newtonsoft.Json;

namespace LoadAirportData
{
    public class Airport
    {
        [JsonProperty("id")]
        public string Id => this.Icao;

        [JsonProperty("icao")]
        public string Icao { get; set; }

        [JsonProperty("iata")]
        public string Iata { get; set; }

        [JsonProperty("name")]
        public string Name { get; set; }

        [JsonProperty("city")]
        public string City { get; set; }

        [JsonProperty("state")]
        public string State { get; set; }

        [JsonProperty("country")]
        public string Country { get; set; }

        [JsonProperty("elevation")]
        public int Elevation { get; set; }

        [JsonProperty("lat")]
        public double Latitude { get; set; }

        [JsonProperty("lon")]
        public double Longitude { get; set; }

        [JsonProperty("tz")]
        public string TimeZone { get; set; }
    }
}

I then created a C# object that matched the structure of the document I wanted to insert into Cosmos DB. The “Point” data type is used to serialize the latitude and longitude into the GeoJSON structure that Cosmos DB supports:

using Microsoft.Azure.Documents.Spatial;
using Newtonsoft.Json;

namespace LoadAirportData
{
    public class AirportDocument
    {
        public AirportDocument(Airport airport)
        {
            Id = airport.Icao;
            Iata = airport.Iata;
            Name = airport.Name;
            Location = new Point(airport.Longitude, airport.Latitude);
        }

        [JsonProperty("id")]
        public string Id { get; set; }

        [JsonProperty("iata")]
        public string Iata { get; set; }

        [JsonProperty("name")]
        public string Name { get; set; }
        
        [JsonProperty("location")]
        public Point Location { get; set; }       
    }
}

Finally I created a method that dropped the Cosmos DB database, recreated the database and the document collection then loaded the documents into the collection:

using Microsoft.Azure.Documents.Client;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using Microsoft.Azure.Documents;
using Newtonsoft.Json;
using System.Net;
using System.Collections.Concurrent;

namespace LoadAirportData
{
    public class CosmosDbImporter
    {
        private const string ENDPOINT_URL = "<YOUR ENDPOINT>";
        private const string PRIMARY_KEY = "<YOUR KEY>";

        private const string DATABASE_NAME = "airports";
        private const string COLLECTION_NAME = "airportData";

        private const string IMPORT_PATH = @"C:\Temp\Airports.json";

        public async Task ImportData()
        {
            var documentClient = new DocumentClient(new Uri(ENDPOINT_URL), PRIMARY_KEY);

            // Delete the database if it exists
            try
            {
                await documentClient.DeleteDatabaseAsync(UriFactory.CreateDatabaseUri(DATABASE_NAME));
            }
            catch (DocumentClientException ex)
            {
                if (ex.StatusCode != HttpStatusCode.NotFound)
                    throw;
            }

            // Create the Database
            await documentClient.CreateDatabaseIfNotExistsAsync(new Database() { Id = DATABASE_NAME });

            // Create the collection and switch on spatial indexing
            DocumentCollection collection = new DocumentCollection() { Id = COLLECTION_NAME };
            collection.IndexingPolicy = new IndexingPolicy(new SpatialIndex(DataType.Point));

            await documentClient.CreateDocumentCollectionIfNotExistsAsync(UriFactory.CreateDatabaseUri(DATABASE_NAME), collection);

            // Read the file and deserialize to our Airport object
            var data = System.IO.File.ReadAllText(IMPORT_PATH);
            var airports = JsonConvert.DeserializeObject<Dictionary<string, Airport>>(data);

            // Upload documents to CosmosDB            
            await Task.WhenAll(
                from partition in Partitioner.Create(airports.Values).GetPartitions(50)
                select Task.Run(async delegate
                {
                    using (partition)
                    {
                        while (partition.MoveNext())
                        {
                            Console.WriteLine($"Processing {partition.Current.Icao}");

                            var airportDocument = new AirportDocument(partition.Current);
                            await documentClient.CreateDocumentAsync(UriFactory.CreateDocumentCollectionUri(DATABASE_NAME, COLLECTION_NAME), airportDocument);
                        }
                    }
                }));
        }
    }
}

One thing to note is the above code enables spatial indexing when creating the collection, without this enabled performance is extremely poor when performing spatial queries.

The beauty of Cosmos DB is that it is able to elastically scale to the performance level specified by the user through the number of RUs (request units) that are allocated to the collection. While loading the data into Cosmos DB I wanted to scale up my database to take advantage of the multithreading in my C# app and speed up my processing so I just went in to the Azure Portal and adjusted the number of RUs allocated to the collection, the change was almost instant and my process instantly sped up. Once I had finished importing the data I was able to scale my database back down so I’m no longer paying for any unused capacity.

image

Querying

Now the data is in Cosmos DB we can begin to play around with some spatial queries.

To query airfields within a certain distance of a specified point I can run the following query which returns all of the airfields within 25km of Blackbushe airport. As you can see, the SQL syntax for querying Cosmos DB is very similar to T-SQL meaning it’s very easy to re-use your SQL Server skills:

SELECT  airports.id AS ICAO,
        airports.name AS Name,
        ST_DISTANCE(airports.location, {"type": "Point", "coordinates": [-0.8475000262,51.3238983154]}) AS Distance
FROM    airports 
WHERE   ST_DISTANCE(airports.location, {"type": "Point", "coordinates": [-0.8475000262,51.3238983154]}) < 25000

The above query returns the following results, which are the 7 airfields that are within 25km of Blackbushe:

[
    {
        "ICAO": "EGHL",
        "Name": "Lasham Airport",
        "Distance": 19964.7890768588
    },
    {
        "ICAO": "EGVO",
        "Name": "RAF Odiham",
        "Distance": 11985.957064869535
    },
    {
        "ICAO": "EGTF",
        "Name": "Fairoaks Airport",
        "Distance": 20229.321025944442
    },
    {
        "ICAO": "EGLF",
        "Name": "Farnborough Airport",
        "Distance": 7286.035340157135
    },
    {
        "ICAO": "EGLK",
        "Name": "Blackbushe Airport",
        "Distance": 0
    },
    {
        "ICAO": "EGLM",
        "Name": "White Waltham Airfield",
        "Distance": 20312.693531316185
    },
    {
        "ICAO": "EGLP",
        "Name": "Brimpton Airfield",
        "Distance": 23311.94703537874
    }
]

The App

The next step is to create an application that uses the functionality to find the airfields within the range of an aircraft. To do this I created a basic ASP.NET MVC application that has a form with the following fields:

image

When the form is submitted the following C# code is executed:

[HttpPost]
public async Task Index(AirportFinderModel model)
{
	var documentClient = new DocumentClient(new Uri(ENDPOINT_URL), PRIMARY_KEY);

	var baseAirfield = await documentClient.ReadDocumentAsync(UriFactory.CreateDocumentUri(DATABASE_NAME, COLLECTION_NAME, model.BaseAirfield));

	var availableDistanceMeters = (model.EnduranceHours * model.TrueAirspeed) * 1852;

	var result =
		documentClient
		.CreateDocumentQuery(UriFactory.CreateDocumentCollectionUri(DATABASE_NAME, COLLECTION_NAME))
		.Where(a => a.Location.Distance(baseAirfield.Document.Location) <= availableDistanceMeters)
		.ToList();

	return View("Result", new AirportFinderResultModel()
	{
		MapPoints = JsonConvert.SerializeObject(result.Select(a => new MapPoint()
		{
			Title = a.Id,
			Description = a.Name,
			Longitude = a.Location.Position.Longitude,
			Latitude = a.Location.Position.Latitude
		}))
	});
}

The above code connects to Cosmos DB and retrieves the details for the base airfield that was specified, it then calculates the range of the aircraft in meters by multiplying the endurance (in hours) by the true airspeed in knots (nautical miles per hour) and then multiplying that my 1852 (number of meters in a nautical mile). A Linq query is then run against Cosmos DB using the built-in spatial functions to find airfields within the specified distance. The result is then converted into a JSON array that can be understood by the Google Maps API that is being used on the client side.

The client side uses the Google Maps API to plot the airfields on a map, giving us a view like the one below when given a base airfield of Blackbushe (EGLK), a true airspeed of 100kts and an endurance of 4.5 hours:

image

The current functionality of the app is extremely basic but could easily be expanded to make the range calculation more accurate by looking at wind and other factors that can affect range. This could be done by creating a polygon representing our range and then using the ST_WITHIN function to find all airfields within that polygon. The functionality could also be enhanced to take into account other attributes of the airfield by deciding if an airfield is suitable based on runway length and other facilities.

Conclusion

As you can see, using Cosmos DB it is extremely easy to analyse spatial data and the elastic scaling capability allows you to easily increase capacity of your application to cope with increased usage and large amounts of data. The spatial capabilities of Cosmos DB can make it a viable alternative to other databases such as SQL Server that aren’t always as easy to scale on demand and don’t have the flexibility that a document model can give.

As always, if you have any questions or comments then let me know.