The Big data Yarns keep growing...

I work for a company that has been one of the beneficiaries of the whole big data buzz, so why should I be writing an anti-big data buzz blog post. Correct? Well, maybe I’m old fashioned and not a make-a-quick-buck guy or because I believe my clients trust me to advise them right. So when I read this blog-post by a self-proclaimed “Middle East big data influencer and public speaker”, I thought it’s about time I put my USD 0.02 on the topic, break through the hype and give a few reality bytes!

The tough part is the big data business case

This so called “Middle East big data expert”’s blog and linkedin profile said he helped companies in vendor neutral big data technology selection. I’m sorry, but selecting the technology for big data analytics is the easy part, the challenge is in building the right big data business cases for procuring the technology. Here’s an example. Someone right here in the UAE, had been collecting certain data points for nearly 10 years. For various reasons, govt policy decisions were being taken on reports of these data points done on a monthly basis (data points aggregated). We showed them how easy it was to get everything into a near real time scenario with blinking screens and analysis happening in sub seconds. They were thrilled. However, once they were out of that orgasmic high of seeing the technology in action, reality kicked in. “What profound and incrementally improved decision can I take with all this “big data” output at my fingertips?”, That was an tough moment to cross, because no one really knew the answer, as it had never been done before. A world where all that high velocity/ volume/ variable data had been digested and presented to the decision makers in split seconds never existed before, so no one knew what could be the possibilities. They eventually figured it out, but took much longer than it took for them to take a liking towards their preferred technology vendor for the same

Not everyone has a big data problem: 

Am sorry, Mrs or Mr CIO, but just because you’ve deployed a CRM system and have an online customer/citizen transaction system, you’re not yet in Big data la la land. My point is that you don’t need to go hunting around for data in your company, just because it’s a fashionable thing. Is big data, a problem that you even want to entertain technology companies to come and give you stories about? I could suggest so many things to focus on (I seriously think OLAP driven BI is starting to look like a Plastic surgery gone all wrong in front of in-memory data visualization, so maybe that’s something to look at) before you turn your attention to #hadoop and #bigdataanalytics and #highperformanceanalytics. However, if you’re an electricity company which has already rolled out 2 million smart meters giving you 5 minute interval readings (that’s about 20 odd vaiables hurtling into a very expensive database) every day and you’ve been paying a database company top dollar to store that data and not use it, you badly need to figure out that #hadoop is not an elephant species. If you’re a government that gave out 5 million health bands to potential diabetic citizens of your country and capture 5 minute interval data points on a cloud and monitor them, you need to talk to be talking to an #advancedanalytics technology company. On a side note neither the electricity company nor the health ministry exist in the entire middle east (caveat: right now) exist

Help me hire some Big Data Scientists

Once you’ve justified my earlier two points of having a strong business case to exploit your big data, it’s important to turn your attention on hiring the right talent. Every time I’m asked this question “help me hire a few big data scientists, data stewards and data plumbers, we’re seriously planning on setting up a big data analytics lab”, I have to struggle to get my jaw from scraping mud off my shoes”. If you have been tasked with setting up an analytics center of excellence around big data in the Middle East from scratch, Its not an easy one (and worse hiring our Middle East big data influencer and public speaker on a consulting assignment to help you out is not going to help). The list of good data scientists would be so small, I bet I could memorize all the names in it and getting a western expat data scientist to relocate requires you to be doing something really fancy to pull him in or a ridiculous pay package. The eastern expat (i.e India) quality is so suspect that its best to just dump a large portion of the CVs that come through (can’t blame the kids who get conned into taking courses from websites like http://www.bigdatatraining.in/ which offer big data training in 5 days to help you become a big data associate with 100% satisfaction guaranteed (grinning).

Don’t get me wrong, I am not saying “big” is not beautiful here. I’m merely asking you not to get carried away by a pretty cover so many people seem to be touting around “big data” nowadays AND stop believing everything you read on the internet.

(All views are personal)

Of preaching health bands, talking tennis rackets and more…

Advanced Analytics software companies like SAS have been around for over 30 years now, helping corporations and governments analyse data and make better decisions, however it has been rare to have analytics touching the lives of the end customers and helping them take their daily life decisions. With just about every device which we carry and work with; from a cooking range to smart electricity meters capturing user data, analytics as a service will increasingly change our lives without the end consumer ever knowing that he is actually taking an analytically driven decision.

Take the case of the "Smartest Tennis Racket in the Planet"

Babolat is one of the leading tennis racket manufacturer in the world and has just released the Babolat Play Pure Tennis Racket - The world’s first data capturing tennis racket (and btw the same is legally approved by the ITF). From Andy Murray to Nadal, swearing by the power of the racket, it wont be long before this becomes the de-facto standard of the future. 

This smart tennis racket is revolutionizing the tennis world with just the addition of a few small additions  (a small accelerometer and gyro-meter fitted at the edge of the handle, and some more paraphernalia to transmit the data) which can record every stroke and volley done by you in the game. The data is sent to a cloud, where a whole bunch of advanced analytics (from the best analytics company in the world :) runs on it, and the end user can download the analytical insight on his performance on his iphone or android device, as to which strokes were the best, a cluster analysis on what type of shots he placed and the outliers, which shots were the weak ones, after what time did he start losing steam etc. The next version of the rackets may even talk back to him saying “'Hey, you have a weak backhand, and 75 percent of the shots you hit were backhands in the last 10 mins– why do you think you're losing?" Dont believe me as yet,  well, check out this link

Imagine the possibilities of embedding analytics into our daily life technologies:

• Imagine a health band as a patient engagement strategy or to revolutionize the way we handle disease management for patients with CVD risk, that can accurately measure your health indicators and give you back near real time trend or forecast on your health risks (Imagine the possibilities for the UAE, which ranks among the highest in cardio vascular disease risk population)

• Imagine your office chair telling you that its time to go and take a walk in the park or that you have not exercised enough today and have been just sitting around blogging or surfing the internet

Anything that moves and does not, seem to be capturing data and sending it to a cloud. Imagine how much smarter the world is headed to be into the future with analytics deployed on the cloud working on this data. It is definitely the age of analytics and specifically big data analytics, but the difference today is that in today’s world it would not just be corporations and governments using analytics for decision making, it would be end users seamlessly embedding analytics into their daily lives to take the most basic of decisions. As Tom Davenport says in his latest book “Enterprise Analytics”, this is not the age of “software as a service (SaaS), it is the age of “decisions as a service (DaaS)” enabled through software's like SAS.

Welcome to the world of analytics with preaching health bands, gossiping office chairs, talking tennis rackets and more...

Have a good week,

Because the Future is now...

What caught my attention the other day was how large corporations and key govt organizations in the region were not truly unlocking value from their data. If large corporate giants in this region were to unlock their data with powerful predictive analytics, they would probably have  insight into the pulse of a nation, better than possibly anything that the nation’s leadership has ever been used to. Aha! You obviously do not trust me, because that makes no sense (I see that smirk on your face, but I request your patience). Here is one example which I have been chewing on for a while...

Believe it or not :NBAD can predict the financial Health of UAE better than the Central Bank!

Emirates NBD and National Bank of Abu Dhabi are the largest issuer of cards in the UAE. They have such large market shares in the UAE, that their transaction data captured in the core banking system is probably the best way to track the macro economic indicators of this country than any govt department. Imagine if  we were to track the buying pattern of consumers and the usage of cards, and map it against the GDP of the country over time, with some additional variables dropped into the analysis. Predictive analytics could help and identify correlations between consumer spending and GDP and even predict the GDP into the future. After all, GDP or Gross Domestic Product is the nothing but the market value of all officially recognized final goods and services sold within a country in a year.

The advantage of the consumer cards spending data coming from the banking heavy weights in the country is a good enough sample size to be able to predict with relative accuracy the GDP Trend. Consumer spending data (as transactions) is available with the banks all the time. Mining this data for the purpose over time mapped to GDP is not rocket science, but the potential of the insight could be startling. For Starters 1) if Central bank of UAE were to get access to this data, they could track even on a daily basis the consumer spending sentiment and forecast into the future 2) Both these banks have investment arms, imagine the investment bets these banks could make in the stock market on stocks whose prices have a positive correlation with the GDP. 

UAE’s Fiscal policy think tanks may want to sit up and take note that “if you torture the data long enough, it will confess”

My USD 0.02 to end this note : Banks and governments have this rare opportunity to reinvent themselves with data analytics. Ignore the jargons IT companies throw at you like Hadoop, big data and cloud and focus on the goldmine of data you already have access to and mine them with a few smart analysts and some power tools like SAS.

Have a good weekend,

Jumping into the Gold Rush? Think again!

Undoubtedly, if there were anything like a God of Investments in Greek or Indian Mythology, it would look like a wizened 70 year old man with plastic framed glasses and an old worn out black suit or simply stated -Warren Buffett . If he says something, you’d better listen hard, because 1) he says very little and 2) he says it publicly only once a year (through his letter to the shareholders) and 3) whatever he says seems to make helluva lotta money!

His latest letter to shareholders has some candid confessions (Read Page 4 and 5), amazing insights and importantly some fascinating metaphors which will make you stop and think. Here’s a link to the actual 22 page letter and one among the many things that caught my attention is Warren Buffett’s take on Gold Investments.

He says (and I quote verbatim from Page 19 of his letter )

“Today the world's gold stock is about 170,000 metric tons. If all of this gold were melted together, it would form a cube of about 68 feet per side. (Picture it fitting comfortably within a baseball infield.) At $1,750 per ounce -- gold's price as I write this -- its value would be about $9.6 trillion. Call this cube Pile A.

Let's now create a Pile B costing an equal amount. For that, we could buy all U.S. cropland (400 million acres with output of about $200 billion annually), plus 16 Exxon Mobils (the world's most profitable company, one earning more than $40 billion annually). After these purchases, we would have about $1 trillion left over for walking-around money (no sense feeling strapped after this buying binge). Can you imagine an investor with $9.6 trillion selecting pile A over pile B?

Beyond the staggering valuation given the existing stock of gold, current prices make today's annual production of gold command about $160 billion. Buyers -- whether jewelry and industrial users, frightened individuals, or speculators -- must continually absorb this additional supply to merely maintain an equilibrium at present prices.

A century from now the 400 million acres of farmland will have produced staggering amounts of corn, wheat, cotton, and other crops -- and will continue to produce that valuable bounty, whatever the currency may be. Exxon Mobil will probably have delivered trillions of dollars in dividends to its owners and will also hold assets worth many more trillions (and, remember, you get 16 Exxons). The 170,000 tons of gold will be unchanged in size and still incapable of producing anything. You can fondle the cube, but it will not respond.

My Take
What motivates most gold purchasers (including you and me) is the belief that our worst fears may just prove correct. An economic meltdown of catastrophic proportions will rip through our curreny holdings and we’ll be left with nothing but a sack full of paper worth zilch (al la Zimbabwe). If you are to go by what Buffet says, and also if you're sitting pretty on gold right now, I would also advise you to rebalance your portfolio by selling some gold off and book profits. However, If you’re thinking of investing in Gold right now, looking at the bull run, my advise –Don’t let greed overtake you; stay wise; invest elsewhere...for "what a wise man does in the beginning, a fool does in the end"

Happy Alpha hunting!

Conquering the next frontier in Investment Data Analytics/ mining

All of us in the portfolio/fund management technology world keep wondering, what’s the next technological wave which will sweep and change the tech landscape. The prime reason for this keen interest is because, there is very little innovation and change which happens in the fundamental operations of these businesses. A Fund is a Fund since the birth of the concept and a portfolio is a portfolio, and there are only so many ways in which it can be looked and analyzed, and we've got multiple systems which do it in today's times.

In the early 2000s, everyone wanted an STP front to back office operation, after a few years they wanted to move from thick client architecture to thin client architecture and by 2009-10 the buzz word was outsourced middle and back office operations (a mix of man and machine technology); so the question everyone seems to be asking is what more can technology deliver to us?

I was reading this fascinating article on “Big data” on the WSJ. It spoke about how a handful of cutting edge companies were helping firms comb through terabytes and pentabytes of data with intelligent algos to aggregate and identify hidden pockets of brilliance. I strongly believe this could potentially be a trend if harnessed properly by Fund/ portfolio management firms also.

Specific to the Middle East, and when the topic of discussion is large AUM asset managers, most of the back office operations systems have now been in place for a good decade+ in several organizations. Every year, several thousand transactions go into the back office database from the transaction side. From the CRM side, wealth management pumps into the database another few thousand disjointed demographic information points. Then there’s of historical prices, exchange rates, research notes etc which go into the system from a multitude of sources and reside unused except for the occasional historical unrealized gain loss report which is requested by the CEO or COO.

Here are a few thoughts which I could figure out from the limited reading that I did over the holiday season on self learning and programmable Algorithms.

There are several types of algos which are used by data analytics firms, and some of them could also find use in the asset management industry. There are “Anomaly detection algos” (in simple terms, a string which could be used by companies to identify “Is this investment transaction in pattern, or out of pattern; does it match typical portfolio manager/ trader behaviour, or not?”) and then there are Classification algos (Is this out-of-pattern activity fraudulent, or not?). The potentials are endless. From a no-brainer idea of trying to avoid an "Adobili scandal" , to analyzing your past data to help understand why loss making deals were done in the past , and maybe even learn from the mistakes and avoid such mistakes in the present and future. Now imagine if you could insert such a string into your pre/ post-trade analyses and they could be used by your middle office to comb through fewer transactions thereby reducing your workforce in the middle office compliance function.

Then there are “Clustering algorithms”, these could be used by asset management firms to better understand client behaviour from all the disjointed demographic information stored in the databases. Eg: What other people is this fund investor of yours most like? If you were to mix a Clustering algo with a recommendation algos (k+nearest neighbor algo) you could arrive at even better intelligent decisions like, What fund or ETF would this customer be most likely to accept!

I strongly believe this is the next big leap which all large cap asset/ fund managers would want to make. The technology systems are already out there. They are being cleverly used by firms in the retail FMCG industry,  credit card fraud management industry, automotive industry etc. It is only a matter of intelligently adapting these technologies to suit into our industry.

The fundamental building blocks to reaching this state start with having a strong back office and single book of records. Having multiple back office systems is not a major impediment (as long as you have the money to spend in integration), however a single book of records ensures that these systems can be easily slipped into place when the time is right. 

Ladies and gentlemen, This is our next frontier in the buy side space ...to borrow a phrase from Star Trek "to explore strange new worlds of (unstructured Data), to seek out new (alpha)and new (client needs), to boldly go where no man has gone before"

Wake me up when its over!

Here’s a limerick written by a hotshot asset management company boss that I met recently, which he emailed to his prospective clients

As markets continue to sink,

The end could be here in a blink,

All I ask you, is to stop and think

Do your portfolio managers stink?

Quite relevant, don’t you think? Sometimes having the right portfolio manager, handling your money is probably the best medicine to ensure that you have wonderful night of sleep, but sadly most of the times they just seem to stink.

Here’s a couple of half baked common man questions to test your portfolio manager’s intelligence:


-->Portfolio manager tells me to invest in Gold, but am wondering why he is not investing in Platinum, since its rarer than Gold and has more industrial/ commercial use. Is it because Fort Knox has stocked up Gold that I should also be holding Gold instead of Platinum. Ask your portfolio manager for a convincing answer and see if he doest freeze in front of these headlights.


-->Portfolio Manager explains to me how important asset allocation into Emerging markets is; however can you explain to me why you have not invested any part of my money into rice or wheat futures. With almost 7 Billion in population, am sure feeding them something would be the most valuable commodity in the world ultimately, unless of course you have an explanation that now we’re going to start wheat and rice farms on the moon?


-->The Portfolio Manager told me in the late 80s to buy and hold for long term “Motorola”, Result-Kaput! then in the early nineties you asked me to buy “Nokia” , Result – Kaput! in the turn of the new century he asked me to buy RIM – Result – Kaput! Should I trust him on his Apple stock hold decision or should I sell it?


-->Explain to me why I should not invest in Oil? When the Libyan Oil went away from the market Oil spiked, How long will we have Iran (the second largest supplier of Oil available in the market) before they go the Kim Jong ll route or the Iraq route? Unless you think Obama is going to ask back publicly for his spy drone from Iran/ Syria/ China/ Putin&Co.

Preserving wealth is not all Fun,

Who knows what’s in store in the long run,

Tomorrow there may be a Bank Run,

And then all you’ll need is a Gun!!!!

Happy 2012 everyone! :)

Microsoft Excel on Steroids!

Black has white, bull has bear ..Any guesses on what could be the antithesis of a enterprise wide portfolio management system? well, my take is Microsoft excel. I’m not trying to take credit from excel, its probably the greatest system devised ever for number crunchers and its contribution to the development of world financial markets is probably an untold story.

However what happens when you feed steroids to Microsoft Excel, you get products (Resolver Systems Inc) like there. I was having a discussion with a friend of mine, who was partially responsible for the set up of this start up firm, and the kind of fascinating stuff they have been able to do with excel, is something beyond the realms of reality. In some ways, products like these seem to distort reality. The best systems in the world sometimes fail to help a portfolio manager calculate VaR, Black-Scholes, risk ratios, attribution analysis etc, and here was a lowly excel sheet throwing out all these numbers completely accurately.

I’m not a big fan of systems like these and believe they will always live in the fringe of the financial world, and will always only have some cowboy customers (P.S: I have nothing against the good ol folks at Resolver).

Risk is a risky subject today more than ever, people put serious money based on risk numbers which are computed; would you bet a Million George Washington's on the VaR number from an excel file or would you rather bet it based on the  numbers that come from a proven system.

I'm not trying to sell you a system here, I'm just asking you guys to give it a thought. Accuracy of data, audit issue, a bad case of fat fingers..just be aware of your risks while taking the dose of Microsoft on Steroids..you may just end up with a bad case of BenJohnsonitis!

:)