The Big data Yarns keep growing...

I work for a company that has been one of the beneficiaries of the whole big data buzz, so why should I be writing an anti-big data buzz blog post. Correct? Well, maybe I’m old fashioned and not a make-a-quick-buck guy or because I believe my clients trust me to advise them right. So when I read this blog-post by a self-proclaimed “Middle East big data influencer and public speaker”, I thought it’s about time I put my USD 0.02 on the topic, break through the hype and give a few reality bytes!

The tough part is the big data business case

This so called “Middle East big data expert”’s blog and linkedin profile said he helped companies in vendor neutral big data technology selection. I’m sorry, but selecting the technology for big data analytics is the easy part, the challenge is in building the right big data business cases for procuring the technology. Here’s an example. Someone right here in the UAE, had been collecting certain data points for nearly 10 years. For various reasons, govt policy decisions were being taken on reports of these data points done on a monthly basis (data points aggregated). We showed them how easy it was to get everything into a near real time scenario with blinking screens and analysis happening in sub seconds. They were thrilled. However, once they were out of that orgasmic high of seeing the technology in action, reality kicked in. “What profound and incrementally improved decision can I take with all this “big data” output at my fingertips?”, That was an tough moment to cross, because no one really knew the answer, as it had never been done before. A world where all that high velocity/ volume/ variable data had been digested and presented to the decision makers in split seconds never existed before, so no one knew what could be the possibilities. They eventually figured it out, but took much longer than it took for them to take a liking towards their preferred technology vendor for the same

Not everyone has a big data problem: 

Am sorry, Mrs or Mr CIO, but just because you’ve deployed a CRM system and have an online customer/citizen transaction system, you’re not yet in Big data la la land. My point is that you don’t need to go hunting around for data in your company, just because it’s a fashionable thing. Is big data, a problem that you even want to entertain technology companies to come and give you stories about? I could suggest so many things to focus on (I seriously think OLAP driven BI is starting to look like a Plastic surgery gone all wrong in front of in-memory data visualization, so maybe that’s something to look at) before you turn your attention to #hadoop and #bigdataanalytics and #highperformanceanalytics. However, if you’re an electricity company which has already rolled out 2 million smart meters giving you 5 minute interval readings (that’s about 20 odd vaiables hurtling into a very expensive database) every day and you’ve been paying a database company top dollar to store that data and not use it, you badly need to figure out that #hadoop is not an elephant species. If you’re a government that gave out 5 million health bands to potential diabetic citizens of your country and capture 5 minute interval data points on a cloud and monitor them, you need to talk to be talking to an #advancedanalytics technology company. On a side note neither the electricity company nor the health ministry exist in the entire middle east (caveat: right now) exist

Help me hire some Big Data Scientists

Once you’ve justified my earlier two points of having a strong business case to exploit your big data, it’s important to turn your attention on hiring the right talent. Every time I’m asked this question “help me hire a few big data scientists, data stewards and data plumbers, we’re seriously planning on setting up a big data analytics lab”, I have to struggle to get my jaw from scraping mud off my shoes”. If you have been tasked with setting up an analytics center of excellence around big data in the Middle East from scratch, Its not an easy one (and worse hiring our Middle East big data influencer and public speaker on a consulting assignment to help you out is not going to help). The list of good data scientists would be so small, I bet I could memorize all the names in it and getting a western expat data scientist to relocate requires you to be doing something really fancy to pull him in or a ridiculous pay package. The eastern expat (i.e India) quality is so suspect that its best to just dump a large portion of the CVs that come through (can’t blame the kids who get conned into taking courses from websites like http://www.bigdatatraining.in/ which offer big data training in 5 days to help you become a big data associate with 100% satisfaction guaranteed (grinning).

Don’t get me wrong, I am not saying “big” is not beautiful here. I’m merely asking you not to get carried away by a pretty cover so many people seem to be touting around “big data” nowadays AND stop believing everything you read on the internet.

(All views are personal)

Of preaching health bands, talking tennis rackets and more…

Advanced Analytics software companies like SAS have been around for over 30 years now, helping corporations and governments analyse data and make better decisions, however it has been rare to have analytics touching the lives of the end customers and helping them take their daily life decisions. With just about every device which we carry and work with; from a cooking range to smart electricity meters capturing user data, analytics as a service will increasingly change our lives without the end consumer ever knowing that he is actually taking an analytically driven decision.

Take the case of the "Smartest Tennis Racket in the Planet"

Babolat is one of the leading tennis racket manufacturer in the world and has just released the Babolat Play Pure Tennis Racket - The world’s first data capturing tennis racket (and btw the same is legally approved by the ITF). From Andy Murray to Nadal, swearing by the power of the racket, it wont be long before this becomes the de-facto standard of the future. 

This smart tennis racket is revolutionizing the tennis world with just the addition of a few small additions  (a small accelerometer and gyro-meter fitted at the edge of the handle, and some more paraphernalia to transmit the data) which can record every stroke and volley done by you in the game. The data is sent to a cloud, where a whole bunch of advanced analytics (from the best analytics company in the world :) runs on it, and the end user can download the analytical insight on his performance on his iphone or android device, as to which strokes were the best, a cluster analysis on what type of shots he placed and the outliers, which shots were the weak ones, after what time did he start losing steam etc. The next version of the rackets may even talk back to him saying “'Hey, you have a weak backhand, and 75 percent of the shots you hit were backhands in the last 10 mins– why do you think you're losing?" Dont believe me as yet,  well, check out this link

Imagine the possibilities of embedding analytics into our daily life technologies:

• Imagine a health band as a patient engagement strategy or to revolutionize the way we handle disease management for patients with CVD risk, that can accurately measure your health indicators and give you back near real time trend or forecast on your health risks (Imagine the possibilities for the UAE, which ranks among the highest in cardio vascular disease risk population)

• Imagine your office chair telling you that its time to go and take a walk in the park or that you have not exercised enough today and have been just sitting around blogging or surfing the internet

Anything that moves and does not, seem to be capturing data and sending it to a cloud. Imagine how much smarter the world is headed to be into the future with analytics deployed on the cloud working on this data. It is definitely the age of analytics and specifically big data analytics, but the difference today is that in today’s world it would not just be corporations and governments using analytics for decision making, it would be end users seamlessly embedding analytics into their daily lives to take the most basic of decisions. As Tom Davenport says in his latest book “Enterprise Analytics”, this is not the age of “software as a service (SaaS), it is the age of “decisions as a service (DaaS)” enabled through software's like SAS.

Welcome to the world of analytics with preaching health bands, gossiping office chairs, talking tennis rackets and more...

Have a good week,

Because the Future is now...

What caught my attention the other day was how large corporations and key govt organizations in the region were not truly unlocking value from their data. If large corporate giants in this region were to unlock their data with powerful predictive analytics, they would probably have  insight into the pulse of a nation, better than possibly anything that the nation’s leadership has ever been used to. Aha! You obviously do not trust me, because that makes no sense (I see that smirk on your face, but I request your patience). Here is one example which I have been chewing on for a while...

Believe it or not :NBAD can predict the financial Health of UAE better than the Central Bank!

Emirates NBD and National Bank of Abu Dhabi are the largest issuer of cards in the UAE. They have such large market shares in the UAE, that their transaction data captured in the core banking system is probably the best way to track the macro economic indicators of this country than any govt department. Imagine if  we were to track the buying pattern of consumers and the usage of cards, and map it against the GDP of the country over time, with some additional variables dropped into the analysis. Predictive analytics could help and identify correlations between consumer spending and GDP and even predict the GDP into the future. After all, GDP or Gross Domestic Product is the nothing but the market value of all officially recognized final goods and services sold within a country in a year.

The advantage of the consumer cards spending data coming from the banking heavy weights in the country is a good enough sample size to be able to predict with relative accuracy the GDP Trend. Consumer spending data (as transactions) is available with the banks all the time. Mining this data for the purpose over time mapped to GDP is not rocket science, but the potential of the insight could be startling. For Starters 1) if Central bank of UAE were to get access to this data, they could track even on a daily basis the consumer spending sentiment and forecast into the future 2) Both these banks have investment arms, imagine the investment bets these banks could make in the stock market on stocks whose prices have a positive correlation with the GDP. 

UAE’s Fiscal policy think tanks may want to sit up and take note that “if you torture the data long enough, it will confess”

My USD 0.02 to end this note : Banks and governments have this rare opportunity to reinvent themselves with data analytics. Ignore the jargons IT companies throw at you like Hadoop, big data and cloud and focus on the goldmine of data you already have access to and mine them with a few smart analysts and some power tools like SAS.

Have a good weekend,