Large DBs in SQL Azure

Recently, one of my customers asked me that question: “Based on the updated SQL Azure plans, the maximum database size is now 50GB. What if my DB requires more storage?

The first recommendation could be: try to measure how your DB is growing, and (if possible) try to have there only the most relevant information – SSIS is a great option to download all that historic data to your on-premises servers. Another option is Data Sync. Some good articles on measuring your DB size are:

How to Tell If You Are Out of Room – SQL Azure Team Blog – Site Home – MSDN Blogs

CalculatingTheSizeOfYourSQLAzureDatabase

Well, according to Microsoft 50GB is the maximum size, and if you need more space you will need to partition your data (either horizontally or vertically). Unfortunately, SQL Azure won’t help you much with this, and you will need to make some changes in your app logic to handle this. This should be done in your Data Access Layer, and it will not be an easy process to implement, let me warn you. Following articles could give you some insight on the details and limitations of this process:

SQL Azure Horizontal Partitioning- Part 2 – SQL Azure Team Blog – Site Home – MSDN Blogs

Scaling out with SQL Azure – TechNet Articles – Home – TechNet Wiki

Advertisements

Windows Azure… One Year After

We have been using Windows Azure for almost a year, hosting our portal (Hoteles.com.co), and the results have been quite good in general. Compared to the rest of PaaS offerings in the market, this is a great solution if your expertise is around the Microsoft stack – specially .NET and SQL Server.

For v2 we are planning to leverage more of the Azure platform, specially the Azure Storage. We will be servicing Hotel images directly from here. This could enhance the already good response times we have right now. Activating the CDN for this would bring some additional benefits as well.

The other good experience (should I say, the most important for us?? 😉 ) have been around the costs. After some changes, like moving out the Content Admin application from Azure to my regular Hosting provider, we are now paying an average of  60 USD per month. Not bad for a reliable and fast platform like this with access to data in SQL Server.

However, Azure is a new platform, and although it provides a good set of basic services, the portal still lacks of some important services, some available in some other PaaS offers (like AWS), such as:

  • It stills lacks an out-of-the-box UI in the Azure portal that enable Admins to monitor the load of the instances (in term of CPU, RAM, Disk Access, etc) – pretty much the functionality offered by really good tools like Azure Diagnostic Manager by Cerebrata. Why is this important? Well, because either you or the platform need to make decisions base on the load.  Should we allocate another instance to accommodate an increase in traffic?  — this is the base of the elasticity paradigm.  The Azure portal should bring some support to define such rules. The capability to define the number of instances per day of the week (i.e. what if the solution expects more traffic during the weekend..)
  • In addition, there is a lack of traffic statistics reporting in the Azure Portal. We are currently handling this through Google Analytics – but it would be great to have this integrated in the Azure portal.
  • It would be valuable to have access to some “Event Log” window from the Azure Portal with diagnostic information. Sometimes your application has problem and does not start but you cannot get the error info – it is like flying blind.

The good thing is that Azure is really strategic for Microsoft, and I expect to see this functionality shortly as part of the service.

Intro to Vagabonding or “Taking Back your Life”

Have you ever watched a National Geographic documentary or read a book about some gringo-loco spending months in the middle of a lost tribe in Africa, or simply walking from Turkey to Tibet? Did you ever wonder how this people do that? Are they millionaires?, do they simply receive a juicy paycheck every month with no need to work, and plenty of time for planning the next cool adventure? While the rest of humanity, at least the ones lucky enough to have a decent job, are trapped in a cubicle, in an endless 9-to-5 loop of complete frustration and boredom, counting the days for the next paycheck, and saving every penny to “escape”  during a 15-days break to some crowded and overpriced beach resort…well, only if the boss is kind enough to approve the leave…

Prisoner # 215353

I must confess that I love to spend hours in front of my laptop in the office and I am very fortunate to love my job, but I do not want to become another corporate drone. I do not want to climb the corporate ladder, because I have not seen anyone on the top truly enjoying his life, without being a slave of his ego, power, bonus and always watching his back. In the other hand, I am not planning to giving up everything, growing dreadlocks, becoming vegetarian, and visiting 100 countries during years, with no destinations or schedule. No, that’s not my plan, at least not in this life,.. The idea is to be able to take an extended break of some months from work, and dedicate this time on whatever I want, in search of true happiness (travelling, family, volunteering, self-studying, etc.), without ruining my finances, career or family. This is not an extended vacation period, after this time, in some way you are a different person, either because you spend part of this time volunteering, or because you have new skills to bring back to your workplace or because simply you have a different perspective of the world as result of a trip. In Generation X, Douglas Coupland illustrates how some people do this on a regular basis, calling this kind of work as an “anti-sabbatical”—a job approached “with the only purpose of staying for a limited period of time (often a year)… to raise enough funds to partake in another, more personally meaningful activity” 

The message is very clear: “Do not wait for your retirement, plan for the future but enjoy your life now!” This is not about being irresponsible, this is about taking back your life and defining its path based on your own decisions, not your boss’ or your bank’s.

By Travelling around the world you start to meet people who took this path with different motivations and backgrounds. In most of the cases they are coming from first world countries, in Latin America for example, people do not have this option on their minds. You could argue economic reasons, but I am tempted to think that this is not the only one, there are some other causes like: fear, cultural aspects, and insecurity. Newer generations constitute a big part of the work force in these countries;  They got trained in the “race-rat” way of living by their parents, who taught them to work-hard for 30 or 40 years so they can retire and start “enjoying life” , and by a society who defines success by the amount of material goods you own. For this herd of people, having the first car or apartment are well defined milestones on their careers. As they start to climb in the ladder, get better salaries and bonuses, they simply start to spend more – “the more to earn the more you spend”, so the only way to cope with your expenses and credit card bills is simply getting a higher position in the organization (basically: “you want a bigger cheese, don’t you? then start moving faster that wheel”), working harder and adding more frustration. It is very clear that this situation is not sustainable. As I already stated, I love working hard, but you need a clear reason for doing it, and working for your bank is not a very smart one let me tell you.

Is it possible to get out of this non-sense race? Yes, it is possible. I know, because I did it.

IMG_7787

The following tips are based on my personal experience. Books like Rolf Potts’ Vagabonding and Tim Ferris’ The 4-Hour Work Week  helped me a lot on structuring it in my case:

1. Declare your independence: This is the most difficult part and where unfortunately only a few brave ones succeed. Paralysis by analysis is a common disease in this phase. The question of how and when to start this process is not really a question at all. Vagabonding starts now. Even if the practical reality of travel is still months or years away, vagabonding begins the moment you stop making excuses, start saving money, and begin looking at maps with the narcotic tingle of possibility.

2. Define an escape plan: The second thing to do is make an assessment of your financial situation. How much is your total debt? How much money do you really have? If you decide to take this path you will be working every day for your liberation, you will make work serve your interests. Forget about that new faster car, cool phone or exclusive apartment. Reduce your expenses to the minimum and define from now a serious savings plan. How long do you want to take off? 6-months, a year? Your escape budget will be defined according to this. Include in that budget all the money you need to clear your debt.

3. Independence Day has come: The day you are finally debt free – that day you will walk in your office feeling more important than the CEO. From that day, you don’t give a shit about anything. From that same day, all your fears disappear. You are a free man.

4. Be free but not Stupid: you need to define how you are going to leave the cubicle. Unless you are changing positions you would need to come back in the future. Ask your HR department about the company’s sabbatical or unpaid leave policy. If that policy does not exist (as in most companies in Latin America!), the HR lady does not have a clue, or simply you do not want to come back, you can make strategic use of a personal-freedom technique, popularly known as “quitting”. Yes, quit, but do it as a gentleman, this is a small world and you never know.. But, even leaving your job in a more permanent manner need not to be a negative act. You should be even proud of that “hole” in your resume. Enthusiastically include all your positive experiences acquired during this mini-retirement period. Your next employer will be interested and impressed (and a little bit envious). In my case I have so much time to sit down and learn about many software development technologies, methodologies, and I even got an important certification on my industry. If you are going to quit, I-Resign is a valuable resource with advices and templates for diplomatically doing it.

5. Ready, set, go!: ok, today is your last day in the office, from now on you are the only responsible for defining your career path, not your boss, you know exactly where do you want to go, and only you will define your daily agenda. Get your last paycheck, say goodbye, leave the building and do not look back.

6. I am free, now what? from my personal experience I got to tell you that having a clear plan from the beginning of your time-off is very important. Without that, you will be simply loosing your time and savings, going back latter to the marketplace with no extra value and new debts. Define a plan and focus your efforts: a common mistake is getting involved in dozens of different activities because you have plenty of time.

My personal experience: in October 2008 I decided along with my wife to take a break after the upcoming born of Martin, our 1st child. In May 2009, in the middle of the worldwide economic crisis I quitted from my successful position as senior consultant for one of the biggest corporations in the World, to leave the middle-east and come back to my country for a while. Either too brave or very stupid. Fortunately, destiny did not let us down, I have enjoyed so much my time with my wife, home, our baby, and in the mean time I have learned so many things, traveled for weeks, read many books and I got ready to go back in the corporate train with new ideas and more energy. I am happy and I do not regret what I did for a nanosecond.

Room View

Have you ever planned to take a break? How was it? Please leave your comments, experiences, advices and other rants.