Parallelizing Facebook FQL queries to increase performance

This post really doesn’t have anything to do with Azure, but as my blog has a taste of Facebook, I’ll throw it out there anyhow.

Am I Interesting is an application that reads quite a bit of data from Facebook and needs to execute five different queries against Facebook for each user that is rated to have enough data to calculate the final rating. In the original solution, I performed these queries in a synchronous sequence like this:

I now attempted to instead run the queries as much as possible in parallel:

The code I for testing the parallelism, using the .NET 4.0 Task Parallel Library:

I created a simple web page that displays the results and compares the sequential to the parallel processing:

As you can see, this modification cut the total query time in about half!

For now, the performance after the change to parallel query execution is good enough. The next thing to try when/if we require even faster performance is sending off all queries to Facebook in one call using fql.multiquery.

Testing/QA environment in Azure for Facebook application

With ongoing quite extensive modifications of core business rules for Am I Interesting, I need a testing environment where I can deploy my application so that other selected people (my partners in crime) can help test out that everything works the way it should before we release into production.

My initial idea was to do this by deploying to the staging slot of my existing Hosting Service is Azure, but this was not a good solution for me because:

• I need a predictable URL for this environment. When deploying to the staging slot in Azure, a GUID-looking URL is automatically generated that cannot be changed, and this does not work for me since I have to put this URL in the Facebook application settings.
• I am using Facebook C# SDK for authentication from my ASP.NET MVC application that reads all Facebook settings, including the application URL, from web.config, which cannot be modified after deployment. And I don’t know the URL before the application is deployed.
• The staging slot for Azure is really intended for a final smoke test before going into production and should therefore have a production-like physical setup, but for my testing environment I only want/need one Extra Small instance to save cost.

The solution therefore was to create a new Hosted Service and use its production slot, which has a known URL, for my testing environment:

An arguable disadvantage of using a new Hosting Service with a predictable and fixed public URL is that it’s more likely to be hit by users who shouldn’t have access to the testing environment. However, this is not a big issue in my case since the Facebook sandbox mode prevents users, other than those that I have explicitly configured, from getting access to the application.

As any other testing environment, I need storage that’s separated from production, so I have created a new SQL Azure database also setup a new storage account for tables and queues (the “p2piter” account below is the one I use in production):

I have then setup a new Facebook application (in sandbox mode) and pointed that application to the new URL:

To ease deployment, I have added a new Service Configuration with settings specific for the testing environment, like connection strings to Azure Storage and SQL Azure:

Also, I’ve added a new build configuration with a corresponding web.config transformation that contains, among other things, the Facebook application settings that the Facebook C# SDK reads:

When it’s time to deploy, I can now simply choose my new service and build configuration to create the complete deployment package:

Done! 🙂

Using ACS for Facebook Authentication from WP7 (Part 1/2)

On top of the ASP.NET MVC web app for Am I Interesting (AII), I’m also adding a Windows Phone 7 client application. This application needs to first authenticate against Facebook and retrieve a Facebook access token before passing that token when authenticating against the API for AII.

This is a two-part blog post:
Part 1 – Configuring ACS for Facebook Authentication
Part 2 – Implementing ACS in your WP7 app (next post)

For this authentication, I will use Azure AppFabric Access Control Service (ACS) from the WP7 app. ACS offers a convenient way to authenticate against identity providers such as Windows Live, Facebook, Google and Yahoo, and it’s priced at only $1.99 per 100,000 transactions. If you, like me, have an MSDN subscription, you also have 100,000-500,000 transactions per month included free of charge.

To set up ACS, go to the Windows Azure Management Portal, navigate to “Service Bus, Access Control & Caching” and click to create a new service namespace:

Check the Access Control check box and enter a unique namespace, “amiinteresting” in my case.

The next step is to add Facebook as an identity provider to your created service namespace. Click to manage Access Control Service:

Click on Identity providers and then Add:

Enter a display name, your Facebook Application Id and Facebook Application Secret (from the Facebook developer portal), and a comma-separated list of Application permissions that you need. Offline_access is needed if you don’t want your WP7 users to have to enter their login credentials every time they start the app:

I have added two Identity providers for the two different Facebook applications I use for Development (local debugging) and Production with different Application Ids and Application Secrets.

The next step is to tell ACS about the client application (relying party) that will connect to ACS to authenticate users. To do this, navigate to Relying party applications and click on Add:

For a thick client application, that will not use browser redirection, it’s enough to enter a display Name, a URI Realm that the token should be issued for (a bit tricky subject, described here), which in my case is the application URI https://www.amiinteresting.net, and a Token format. I have chosen a Simple Web Token (SWT) which is suitable for tokens that are be passed in the HTTP header or request, which is normally the case when communicating with REST (or REST-alike) API:s:

Further down on the page, choose the Identity provider that was added in a previous step and click to Generate a token signing key for this relying party application:

The last thing you need to do is configure the Rule group that was automatically created when you added the relying party. A rule group is used to define how claims from the identity provider (Facebook) are passed to the relying party (WP7 app). Navigate to Rule groups and click on the automatically added rule group to configure:

On the screen that appears, no rules will have been added yet. The rules can be added individually/manually, or you can click on Generate to automatically generate rules for you:

On the next screen, I have chosen to generate rules for both my Facebook applications (development and production):

I then modified the generated rules by removing the emailaddress claim (I don’t need and therefore didn’t configure the Identity provider to request the email permission from Facebook) and changing the description of the nameidentifier claim to “Facebook User Id” (which it is):

Done! ACS is now configured for Facebook authentication and ready to be used by WP7!

Stay tuned for the next post that covers how to connect the WP7 app to ACS and ultimately get the Facebook access token, which is required to authenticate against the API for Am I Interesting….

Planned blog entries

Below are some of my planned blog posts. Help me prioritize by letting me know if you find anything in partciular extra interesting, or if the list is missing something of interest.

• Facebook applications – the perfect match for Windows Azure (Completed)
• Application architecture for Am I Interesting
• Session state in Azure AppFabric Cache
• SQL Azure and/or Azure Table Storage?
• Generic class to work with Azure Table Storage
• Using own property names instead of PartitionKey and RowKey
• Adding a certificate for SSL/HTTPS
• Using a Web Role background thread to host a worker process
• Using Azure Queues for async processing
• Exponential back-off polling of queues
• Dirty way of realizing transactions over SQL Azure and Azure Table Storage
• Retrying SQL Azure connections
• Repository interface for unit testing
• Using a Startup Task to avoid IIS Idle Timeout
• Activating diagnostics (event logs, trace logs and performance counters)
• Using log4net for trace logging
• Operating cost as a new parameter for product owners to prioritize development

Facebook applications – the perfect match for Windows Azure

First of all – welcome everyone, including myself, to this blog! 🙂

In this first post I want to give some background to the application that I am developing and will be referencing in several coming posts.

The application – called Am I Interesting – is a Facebook app, which to me is a perfect fit for cloud computing and Windows Azure. The application rates users’ activity on Facebook and gives them a percentage rating and a corresponding title, from Facebook Disaster to Facebook Phenomenon. You can rate yourself and your friends and (soon) post an image that compares yourself to selected friends.

This application is a perfect fit for cloud computing and Windows Azure both from a technical and economical perspective.

From a technical perspective, the cloud gives me the ability to architect and host an application that can scale if/when usage increases, which can happen really fast with social apps on Facebook. I’ve seen applications grow to almost a million active users in just a couple of weeks, and with Windows Azure I will have no problems scaling quickly to that volume of usage (as long as my application is properly architected of course ;)). The choice to go with Windows Azure, compared to other cloud platforms, has a lot do to with my .NET background and the ease of getting started. There are many more specific reasons, like the excellent tooling support, and I intend to highlight some of those in coming posts.

From an economical perspective, particularly as I’m developing this as a private educational hobby project, I can start out really small, with just a couple of instances (to secure uptime and the SLA), and get by with a low monthly charge. As a holder of an MSDN Premium subscription, I can get by even cheaper as the subscription includes a good amount of server hours, transactions and traffic volumes. When traffic increases and I need to scale up, I have hopefully worked out a business model that generates at least enough revenue to cover my expenses :).

On the About page, there is some additional information about this blog. My goal is to post a new entry every week, so please let me know if there is something in particular that you would like me to write about!