Part of my role at Runscope involves me writing OSS libraries or sample projects to share with other developers. I also regularly use 3rd party APIs in the process. This requires the use of API keys and other private data that I'd rather not share. Unfortunately it is all too easy to leave a key in a source code file and accidentally commit it to a public source control repository.
The Stack Overflow Solution
The standard guidance on Stack Overflow is to commit your configuration file with dummy information in it and then tell Git to ignore any future changes to the file. This seems like a reasonable approach as long as you keep your private data out of the standard app or web.config. Once you have committed to using a separate file for private configuration data, the new developer has to be made aware of this settings file.
It isn't a terrible solution, but it felt like there was room for improvement. When someone makes the decision to try your sample application or library, you want the experience to be as painless as possible.
As an aside, I wish API providers would make publicly available API keys that pointed to sample data. Even if the key only allowed read-only access, it would make the education process a whole lot easier.
Automatically Initialize
I decided that I wanted to create a simple HTML form based user interface for supplying private data elements and automatically take care of storing that data in a file somewhere that wouldn't get committed.
My solution to this problem went through various iterations, trying to get the right balance of simplicity and security. I wanted to make sure there was no way that the form could expose previously stored credentials and Jeremy Miller also pointed out that you don't want to allow an external party to inadvertently or maliciously cause the existing credentials to be lost.
In order to avoid having to build a fully authenticated administrative interface, the HTML form is only display once when there is no configuration data file. Once it has been created then the developer must edit the file manually to make changes, or delete the file to trigger the appearance of the HTML form again. This is a simplistic solution ideal for sample applications, but could also be the starting point for something more sophisticated.
The Man in the Middle
The functionality is provided primarily by a piece of middleware. In ASP.NET Web API this is implemented as a class that derives from DelegatingHandler and is added to the MessageHandler pipeline. The same architectural pattern exists in other web frameworks, so I'm sure the the code I am using could be re-implemented on other platforms.
To initialize the piece of middleware using ASP.NET Web API, we create an instance of a new class I created called PrivateDataMessageHandler and pass it a path to a configuration file that will hold my private API keys.
I am using the standard ASP.NET App_Data folder to store the configuration information and I made sure that my source control ignore file is not going to track any files in that folder. You choose any location that your web server can write to.
First Run
Initially that configuration file will not exist and that will cause the MessageHandler to enable itself.
Every request to the system will be routed through this message handler, but unless it is enabled, messages will just be passed right through. This is handy because when you run a sample application for the first time through Visual Studio a web browser opens and hits the root of the API. The message handler will respond with the configuration form.
The path to the configuration file is added to the request properties collection so that controllers can locate the file to read data from it.
First Request
When a sample application is first used we can assume that there are API keys missing to be able to access third party services. As we don't know which resource will necessarily be requested first, we intercept all inbound requests, except for one special one, and return an HTML form that presents input controls for each of the missing API keys.
The _magicPath variable contains the path used when the following form is submitted,
The HTML form is customized to collect whatever private data you need to store.
The names of the input fields will be used as the property names in the configuration file.
Submitting The Private Data Form
The form is submitted back to a unique endpoint /privatedataform that is monitored by the middleware and the information contained in the body is processed and the middleware is disabled.
One annoying issue here is that if you are hosted on IIS and you are using Attribute Routing, it is likely that there will be no matching route for your magicpath so the routing will fail and a 404 will return before the MessageHandlers are fired. This doesn't happen on self-hosted setups and it generally doesn't happen on regular routing because the default route will match. No controller will be found but that's fine because the MessageHandler short circuits the request before controller selection happens.
The submitted form is stored to a JSON configuration file and a simple message is returned to the user.
Accessing the Private Data
In order to get at the data, we need to access the path of the configuration file which the middleware hid away in the Request.Properties dictionary and we need to load the data into an object.
I created an extension method to hide the details.
Now when I am in a controller and I need to access the private data I can just do,
How does this help me?
I've used this on a couple of projects so far. One is my RunscopeMessageHandler that allows you to log requests to a WebAPI up to Runscope's API. I wanted to include a sample application to the project that I could also use for some interactive testing, but didn't want to accidently publish my API key. The other is a SlackBot that I have been playing with that allows you to trigger API tests via Slack commands and the Runscope API.
Due to the fact that the private data you choose to store, and the corresponding HTML form, is custom for each installation, I decided there would be little value in making a library out of these classes. If you think this approach might work for you then feel free to grab the source from either of the two projects I just linked to.
So far it seems to be working well for me. I suspect the whole process could be refined further so I look forward to comments and suggestions from developers who are looking to solve a similar problem.
And with that done, on to the next Yak!
Image Credit:
Keys in lock https://flic.kr/p/4Metz2
Magic Wand https://flic.kr/p/7V1y7c
Men on Bench https://flic.kr/p/oTzP55
Safe https://flic.kr/p/ruAw2
Top Secret https://flic.kr/p/4SCuPK