A while ago, SDN asked me to write an article for them. “Maybe PHP and Windows Azure related?” they added. To be honest, while Windows Azure is pretty new, I’ve had enough of writing “PHP on Azure 101” articles. And that’s the reason why this article will not be about how you can store a file in blob storage. And not about how you can connect with SQL Azure. If you are interested in these topics, browse to http://azurephp.interoperabilitybridges.com/ and have fun. If you are interested in going beyond that, please bear with me. I will introduce you to the wonderful world of Windows Azure and SQL Azure’s Service Management API and take you along a journey of building an “application factory”, a concept that you can reuse when building your own Software-as-a-Service. Sounds intriguing? Let’s go.
Here’s what I want to show you:
- A Software-as-a-Service does not have to be a multi-tenant application. Once can create an application which provisions another application, which is what I’ll be showing you.
- Some less-known features of the Windows Azure platform to automate application packaging and deployment.
Before we start: please take of your “one technology” glasses: while the language used in this article is PHP, the concepts I demonstrate can be leveraged in .NET, Java, Ruby or whatever language you want. The tooling is currently available for .NET and PHP, but if you take some REST (and sleepless nights), you can communicate with the service management API’s in any language of choice.
The datacenter? There’s an API for that!
First of all, let’s explain what I want to achieve. I have created an application, “ShortnR”, which is a fancy and good-looking URL shortening service as you can see in the screenshot below.
In these times of social media and personal branding, it can be interesting to have your own URL shortening service that can increase your personal brand awareness. For example, Belgian magazine www.datanews.be has their URL shortener at http://datane.ws. Much cleaner than tinyurl.com, bit.ly and ow.ly, no? So I decided to make ShortnR available to others, but I want them to run the application on their own database and their own server. Now imagine provisioning a new database and web server for every user in your traditional datacenter or at a traditional hoster… That’s where Windows Azure comes into play: Windows Azure has an API for this: the Service Management API.
Windows Azure and SQL Azure’s Service Management API allow you to interact with the Windows Azure datacenter. Using a simple, SSL-signed REST call into their management endpoints, one can easily do a lot of infrastructure tasks. Some examples of these actions can be:
- Creating a hosted service
- Creating a storage account
- Deploying an application
- Creating a new SQL Azure database server
Combine these with the command-line packaging tools available in the Windows Azure SDK for PHP (http://phpazure.codeplex.com) and you can already imagine some interesting scenarios, such as deploying ShortnR to a new hosted service and database, scripted using some PHP. It’s like yelling at the system administrator that he (or she) should install a new server. Within 5 minutes, because that’s the time it takes in the cloud.
ShortnR in short
Under the hood, a SQL Server database powers the ShortnR application. It contains one table, url, which contains all combinations of original and shortened links. The ShortnR application itself consists of 2 files: index.php, containing the application logic, and setup.php, containing all configuration values. Index.php handles the logic of shortening a URL as well as the inverse operation: whenever the query string contains ?alias=xyz, this alias request variable is used to retrieve the actual URL from the database. That is indeed an ugly URL, but using IIS URL rewriting this is a no-brainer to make prettier.
The setup.php file is the file containing all application configuration details: the database connection string, the username and password for the database as well as the application title (not every application should be named ShortnR, right?) and a tagline (“Mapping long links to short ones since 1875”). Here’s an example:
<?php
date_default_timezone_set('Europe/Brussels');
$siteTitle = "ShortnR";
$tagLine = "Mapping long links to short ones since 1875.";
$connection = new PDO(
'sqlsrv:server=sqlazureserver.database.windows.net;Database=shortnr',
sqluser@sqlazureserver,
'P2ssw0rd1',
array(
'MultipleActiveResultSets' => 0
));
$connection->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
In order to automate deployment of the application, we’ll have to customize setup.php for every installation out there. But that will, of course, not be the only step. There are some more steps to deploying an application on Windows Azure! But before we can go there, the Service Management API should first be configured.
Setting up the Windows Azure Service Management API
The Windows Azure SDK for PHP contains PHP classes for interfacing with both Windows Azure and SQL Azure’s Service Management API. To make a request against the Service Management API, you must first associate a certificate with your subscription and then send the certificate's private key with the request. Since the Service Management API’s are unexplored territory for many, I’ll give you an overview on how to set up things so your PHP scripts can connect to it.
First of all, a pair of PFX and CER (private and public) certificates should be generated. Also, a PEM certificate will be required for connecting to the management API from PHP. Generating the required certificates can be done as follows:
- Download OpenSSL from http://www.openssl.org/related/binaries.html.
- Start a command prompt and navigate to the folder where OpenSSL.exe is installed
- Issue the following statement to generate a PEM certificate. This certificate will be the base certificate used by PHP to connect to the management service API.
openssl req -x509 -nodes -days 9999 -newkey rsa:2048 -keyout mycert.pem -out mycert.pem
- Issue the following command to convert the PEM certificate to a PFX certificate.
openssl pkcs12 -export -out mycert.pfx -in mycert.pem -name "My Certificate"
- Issue the following command to convert the PEM cetificate to a CER certificate.
openssl x509 -in mycert.pem -inform PEM -out mycert.cer -outform DER
The folder where OpenSSL.exe is installed will now contain 3 files: mycert.pem, mycert.pfx and mycert.cer. These files and the passwords entered in the console will be used when scripting a deployment. But before we can go there, the server-side should also be configured and made aware of the certificates generated.
In order for the Windows Azure SDK to be able to connect to the Windows Azure Service Management API, a certificate file should be uploaded to the Windows Azure management portal. This can be done through http://windows.azure.com.
Under Hosted Services, Storage Accounts & CDN, upload the CER file that’s just been created.
Queues as a state machine
If you are a .NET developer, it is pretty common to use Workflow Foundation as a method of creating a state machine, a workflow that “knows” what it has done and should do and is persisted for an undetermined time between two steps. PHP developers do the same using one of the many frameworks out there.
What’s interesting is that you can use a Windows Azure queue as a state machine as well: you put a message in one queue, after processing it you move it to the next queue and so one. Depending on the queue the message is in, the current state in the flow can be determined. What’s even better: a message stays in a queue for 7 days until it is marked as “processed”. This means if one step fails along the road, it will be executed again until it’s removed from the queue (or it expires).
Another reason for doing state machines this way is that every state can be processed by a different script, process or even a different server! Parallelism anyone?
Working with Windows Azure queues in PHP is very straightforward. Once a storage account reference has been created, a queue can be created, messages can be inserted, retrieved and deleted. These five simple commands allow you to create very powerful solutions! Here’s a quick sample on how to do these steps:
$queueClient = new Microsoft_WindowsAzure_Storage_Queue();
$queueClient->createQueueIfNotExists('myqueue');
$queueClient->putMessage('myqueue', 'Hello, world!');
$messages = $queueClient->getMessages($currentQueue, 1);
foreach ($messages as $message) {
echo $message->MessageText;
$queueClient->deleteMessage('myqueue', $message);
}
In our scenario, I’ve introduced 5 queues:
Using different queues, the provisioning of a new ShortnR application instance can be processed in steps. Every queue is a different step that can fail and can be retried if needed.
A request to create a new ShortnR application instance is inserted into the tosetup queue. The provisioning script processes this queue and extracts a Windows Azure solution template to a temporary folder. If that succeeds, the message is removed from the tosetup queue and inserted into the toprovisiondatabase queue.
The toprovisiondatabase queue is processed by the provisioning script and uses the SQL Azure Service Management API to provision a new database server, create a database on it and create the required tables. On success, the message falls down into the toconfigure queue.
A message in the toconfigure queue is processed and the setup.php file is generated, containing all database connection settings. Also, Web.config is altered to setup URL rewriting.
Once a message arrives in the topackage queue, it’s almost time to head into the clouds: the application is packaged into a .cspkg file, which you can compare with a ZIP file containing all application artifcats.
The provisionhostedservice queue is the final step in the process! A hosted service is created on Windows Azure, a storage account is created, the package is uploaded, a deployment is started and… the application requested should be running once that’s finished.
This complete flow is easy to model: a while(true) loop keeps the provisioning process running in which every queue is processed one-by-one, and then over again:
/** Start provisioning loop */
set_time_limit(0);
while (true) {
runStepToSetup();
runStepToProvisionDatabase();
runStepToConfigure();
runStepToPackage();
runStepToProvisionHostedService();
// Sleep a while
sleep(5);
}
Now let’s (finally) look into the concrete actions executed in these actions.
Use the force: scripting a Windows Azure deployment
For the sake of simplicity, I’m not going to copy the full code in this article. I’m also not pasting in the queue read/delete actions, as these are just the control flow on how I’ve implemented this solution. In theory, all commands mentioned above can be executed in one go. The downside of that is that you lose the ability to recover in the middle of the process. But then again: that’s an implementation detail.
Let’s start with the so-called “template”. Whenever you want to create an application running on Windows Azure using PHP, the easiest way to do so is using the command-line tools (now shipping with the Windows Azure SDK for PHP) and “scaffold” a template. This is basically an automated way of generating a file and folder structure in which your application can be hosted. Normally this command is executed from the command line but since it’s a PHP class under the hood why not use that one directly:
$temporaryPath = "c:\\temp\\" . $service->Name;
$packagingTool = new Microsoft_WindowsAzure_CommandLine_Scaffolder();
$packagingTool->runCommand($temporaryPath, null,
array('-DiagnosticsConnectionString:UseDevelopmentStorage=true'));
copy( dirname(__FILE__) . '\\index.php', $temporaryPath . '\\PhpOnAzure.Web\\index.php');
This creates a folder “C:\Temp\<service>” in which, in the last line of the above code snippet, the index.php file for the ShortnR application is copied.
After creating the package, the SQL Azure database will be provisioned. Moreover: a new SQL Azure server will be created in which a database is created. In that database, a table is created for storing all shortened URL data. Here’s how to connect to the SQL Azure Service Management API and provision a new database server (that opens two firewall ports as well, otherwise we will be unable to connect afterwards):
$sqlAzureClient = new Microsoft_SqlAzure_Management_Client(SUBSCRIPTION_ID, MGMT_CERTIFICATE, MGMT_CERTIFICATE_PASS);
$sqlAzureServer = $sqlAzureClient->createServer('sqladm', '@@cool1OO', 'West Europe');
$sqlAzureClient->createFirewallRuleForMicrosoftServices($sqlAzureServer->Name, true);
$sqlAzureClient->createFirewallRule($sqlAzureServer->Name, 'Home', 'your ip address', 'your ip address');
In order to create a database, a connection to the newly created server should be made using the SQL Server Driver for PHP. On this connection, a CREATE DATABASE statement can be issued:
$connection = new PDO($masterConnectionString, $userName, $password,
array(
'MultipleActiveResultSets' => 0
));
$connection->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
$result = $connection->exec("CREATE DATABASE shortnr (EDITION='WEB', MAXSIZE=1GB)");
Once the database is created, we open a connection to it and issue a CREATE TABLE statement:
$connection = new PDO($connectionString, $userName, $password,
array(
'MultipleActiveResultSets' => 0
));
$connection->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
$result = $connection->exec("CREATE TABLE [dbo].[url](
[Id] [int] IDENTITY(1,1) NOT NULL,
[Alias] [nvarchar](6) NOT NULL,
[Url] [nvarchar](1000) NOT NULL,
CONSTRAINT [PrimaryKey_a39d31f1-f51b-40d0-af10-66b64c04f1b2] PRIMARY KEY CLUSTERED ([Id] ASC)
)");
After setting up the database, it’s time to configure the application that will be deployed. This step will consist of copying all SQL Azure connection details into setup.txt (and copying that file to the path where we scaffolded our application’s folder structure). I’m going to let you figure out this step, as it’s basically just reading and writing a file. Feel free to browse the sample code accompanying this article for this.
Packaging the application into a .cspkg file that can be deployed is again abusing the command line tools. The Microsoft_WindowsAzure_CommandLine_Package class handles the Package Create command on the command line, so why not call into it directly?
$packagingTool = new Microsoft_WindowsAzure_CommandLine_Package();
$packagingTool->createPackageCommand($temporaryPath, false, '');
The next step in the deployment process consists of a lot of steps: deploying the application to Windows Azure. First of all, a hosted service should be created. A hosted service is the reservation of a unique DNS name on the Windows Azure platform and specifying in which datacenter your application will live.
$managementClient->createHostedService(strtolower($service->Name), strtolower($service->Name), $service->Name, 'West Europe');
$managementClient->waitForOperation();
In order to be able to upload the .cspkg file we created earlier to Windows Azure, it must be copied onto a storage account. Since the Service Management AI offers the functionality of creating a storage account, we can create a temporary one for accepting the upload:
$managementClient->createStorageAccount(strtolower($service->Name), strtolower($service->Name), $service->Name, 'West Europe');
$managementClient->waitForOperation();
The package should, of course, be uploaded to the newly created storage account. This can be done by having the Service Management client create a reference to the storage account on which we can simply call the putBlob method to upload the package from the local hard disk onto Windows Azure blob storage.
$blobClient = $managementClient->createBlobClientForService(strtolower($service->Name));
$blobClient->createContainerIfNotExists('deployments');
$blobClient->putBlob('deployments', $service->Name . '.cspkg', $temporaryPath . '\\' . $service->Name . '.cspkg');
$packageUrl = 'http://' . strtolower($service->Name) . '.blob.core.windows.net/deployments/' . $service->Name . '.cspkg';
One step left: starting the actual deployment, which consists of creating a virtual machine for our service and deploying the .cspkg file. All of this can be done in one step, handled by the platform:
$managementClient->createDeployment(strtolower($service->Name), 'production', strtolower($service->Name), strtolower($service->Name), $packageUrl, $temporaryPath . '\\ServiceConfiguration.cscfg', true);
$managementClient->waitForOperation();
This step can take a while and will report “Success” once the virtual machine has been provisioned. In the background, it will still be booting but after 15 to 30 minutes, your new instance of ShortnR should be up and running on its dedicated web server and database server.
While waiting for the server to come up, let’s clean up some mess: the storage account we created can safely be removed.
$managementClient->deleteStorageAccount(strtolower($service->Name));
$managementClient->waitForOperation();
If your Software-as-a-Service also contains a “stop subscription” option, it’s perfectly possible to create a script which does the reverse of provisioning: taking a service down can be done using the exact same Service Management API.
Conclusion
In this article I’ve shown that a Software-as-a-Service does not have to be a multi-tenant application. Once can create an application which provisions another application, which is exactly what we did. Imagine offering a Twitter clone to corporations: one can create a provisioning application which sets up a dedicated environment for every customer you have. All of this can be done using some less-known features of the Windows Azure platform: the Windows Azure and SQL Azure Service Management API.
Dowload Sources