Comparing the Performance of Azure Table Storage with Other Repositories

I have been using Azure Table Storage—ATS in a couple of my personal projects, and I just love it. It is simple, the performance was decent and the storage quite cheap. A NoSQL key-value store like ATS is just perfect for storing lots of unrelated records like audit, and error. In our case, around 70% of our data fall into this category.

I had the perception that ATS was not that fast, but I did not notice much impact on the performance of the site. Anyway, the audit and error reporting operation were asynchronous. Probably the only major drawback during the project was the ATS poor API – I still cannot conceive the anemic LINQ support.

During the architecture definition of a new web-based project I started to consider some other options for the data storage. This new project required a data model with way more relations between entities – a productive API was key, although I wanted to stick to a NoSQL store for future-proof scalability.

One of the serious options that we started to contemplate was MongoDB. I had some quick experiences in the past with it but nothing serious. I knew that their support of LINQ was phenomenal, the option of growing to a massive scale thanks to sharding and replica sets, but what about performance…? How would ATS performance for read/write operations compare against MongoDB, or Azure SQL?

I built a simple MVC 5 application and deployed it in a Azure Web Role (XS). Using the ATS .NET Storage Client Library 2.2, I built a simple page which reads 100 records in a ATS table, and another page for reading each one of the records in that same ATS table. The average latency of each write and read operation is displayed. I based my application on the tutorials and walkthroughs available  from Microsoft. The idea was to build it with the techniques available, without any tuning.

I did the same thing with Azure SQL, procuring the smallest database I could. I used plain old fast ADO.NET DataReaders for implementing the operations.

For MongoDB, I launched a Extra Small VM with Linux CentOS. This is a 1Ghz CPU, and 786 MB RAM VM.  MongoDB was deployed with default settings.

Naturally, all these resources were located in the same region (US West). This diagram summarizes the topology:

Arqchitecture

And…these are the results:

Perf

Yes, the performance of plain vanilla ATS is just disappointing. After some research I found blog post with similar findings, which indicated how to improve the performance turning-off the nagle before the calls:

public static void InsertRandomEmployeeData()

        {

            string connStr = ConfigurationManager.ConnectionStrings["ConnString"].ConnectionString;

            /// For increased perf Turn off naggle alg

            /// http://alexandrebrisebois.wordpress.com/2013/03/24/why-are-webrequests-throttled-i-want-more-throughput/

            ServicePointManager.UseNagleAlgorithm = false;


            CloudStorageAccount storageAccount = CloudStorageAccount.Parse(connStr);

            CloudTableClient client = storageAccount.CreateCloudTableClient();

            CloudTable table = client.GetTableReference("Employees");


            table.CreateIfNotExists();

            var emp = new EmployeeEntity(GenerateRandomInt(1,10000), GenerateRandomString(0), GenerateRandomDouble(1.0,100000.0));

            TableOperation insertOp = TableOperation.Insert(emp);

            table.Execute(insertOp);

        }

The performance benefit is impressive – I wonder why this is not a default setting. Note how the read operation were not affected by that.

The performance of Azure SQL operations was really good (under 10 msecs on average), but the winner as you can see was MongoDB – impressive, with both operations under 2 msecs!

Well, that was a eye-opener. It is pretty obvious what we are going to use for next projects. Unfortunately, neither Azure nor Amazon Web Services offer a managed MongoDB service at this time, so I would need to setup and maintain my own set of VMs running MongoDB, which is not a big deal, but I would need to pay for this in addition to the storage.

Cheers, see you next time amigos.

Advertisements

Compiling Views in MVC4

By default, any typo or mistake in MVC razor views will only be detected during execution.. However, you can enable compiling of views and detect those errors earlier in the development cycle.

Just open your project file (.csproj) as text (unload your project first), and find the < MvcBuildViews> and change its value to true:

<MvcBuildViews>true</MvcBuildViews>

Beware that compilation time will be increased (almost double in my case). You may also get a weird “Unrecognized attribute ‘xmlns:xdt’on the web.config” error during compilation (especially after switching between build configurations ). To work around it, delete the \obj folder in your project folder, or use the pre-build step described in here.

A pre-build step that worked for me was:

del $(ProjectDir)obj\* /F /S /Q

Dissecting Azure Clouds

What are the challenges in porting your existing ASP.NET/SQL Server applications to Azure?

Bogota 102

If you plan to use SQL Azure then migrate your DB to Azure first. Below you will find some tips to perform this process. Yes, you will find lots of things that do not work on SQL Azure. The good thing is that moving the ASP.NET will be way easier… Once completed, then connect your local application to the DB instance in SQL Azure. This way you debug your application and find additional problems. Remember that you could get the connection string through the SQL Azure portal. Test your app. Does everything work? Then move your ASP.NET app to Azure and publish it. Congratulations, you are now connected to the Cloud!

Here are few points I found migrating my ASP.NET MVC app:

  1. There is no support for Session Affinity (Azure is Stateless) – I’m aware that Azure load balancing doesn’t support Session Affinity – hence the existing web application should be changed if it has session affinity.
  2. If you get this error debugging your app in Azure: "Windows Azure Tools: Failed to initialize the Development Storage service. Unable to start Development Storage. Failed to start Development Storage: the SQL Server instance ‘localhostSQLExpress’ could not be found. Please configure the SQL Server instance for Development Storage using the ‘DSInit’ utility in the Windows Azure SDK." 
    It is because the Dev Store is pointing to a named instance of SQLExpress and if you are using SQL Server like me, then you would need to do like what the error message said.
    Go to where the devstore is installed i.e.
    C:Program FilesWindows Azure SDKv1.0bindevstore
    and type
    dsinit /sqlinstance:.
    Take note of the "." which indicates your current default unnamed SQL Server instance.
    You will then be prompted with a screen that informs you that the installation is successful and the development storage is ready for use.
    You can now start the Development Storage service.
  3. If you get a lovely 403 – Forbidden: Access is denied. You do not have permission to view this directory or page using the credentials that you supplied. It’s because you hit the directory instead of a page.  Check this page out to see how to set a default document in the web.config (web role’s don’t automatically load a default.aspx like you might expect – you have to set it): http://blogs.msdn.com/rakkimk/archive/2007/05/25/iis7-how-to-configure-the-default-document-of-the-website-in-its-web-config.aspx
  4. If your ASP.NET app is based on MVC, ensure that the System.Web.MVC assembly is included in the service package that you deploy to Windows Azure. To do this, for a Visual C# project, expand the References node in Solution Explorer for the MVCAzureStore project, right-click the System.WebMVC assembly and select Properties. Make sure the Copy Local option is set to True.

 

Here there are some tips for moving your DB to SQL Azure:

  1. SQL Azure Database exposes a Tabular Data Stream (TDS) endpoint to databases that are hosted in the cloud. TDS is the same network protocol that on premise SQL Server uses, therefore, a desktop client application can connect to SQL Azure Database in the same way it connects to an on-premise SQL Server instance.
  2. You won’t be able to connect to your remote SQL Azure DB using SQL Server Management Studio (SSMS) 2008.  I guess this will be supported in R2. Currently you can only connect using a “Script Window” to an specific DB. There is no support to “USE <DB>” though.
  3. Remember that SQL Azure only support a subset of the services provided by your on-premise SQL Servers (check out this list of supported SQL commands, or Unsupported Transact-SQL Statements (SQL Azure Database) – MSDN).
  4. There is no GUI-based admin tool, so you will need to create everything (Users, Logins, DB, Tables) using SQL commands and scripts. There is a couple of community provided GUI tools to enable basic operations on your DBs. Check SQL Azure Manager
  5. If you want to migrate one DB from SQL Server 2008 to SQL Azure, and you expect to find some “Attach DB” or “Restore Backup”…forget it, you will need to use your good-old Bulk Copy/BCP, INSERT scripts or  SS Integration Services (SSIS). If you want to give SSIS a try this could help you. Some other handy tools:
    1. I just found a plug-in called SQL Azure Data Sync Tool for SQL Server, but I have not tried it.
    2. SQL Server Migration Wizard http://www.azuresupport.com/2009/12/sql-azure-introduction/2/ . I ended up generating INSERT scripts using this tool.
  6. Be aware of some deprecated features while moving your DB structure and data using SQL scripts:
  • ‘ANSI_NULLS’ is not a recognized SET option.
  • Deprecated feature ‘SET ANSI_PADDING OFF’ is not supported in this version of SQL Server.
  • Deprecated feature ‘More than two-part column name’ is not supported in this version of SQL Server.  This a significant change if your using Schemas.
  • Deprecated feature ‘Data types: text ntext or image’ is not supported in this version of SQL Server.
  • Deprecated feature ‘Table hint without WITH’ is not supported in this version of SQL Server.
    A full list id found at Deprecated Database Engine Features in SQL Server 2008 – MSDN.

 

See you up there!