Big data that’s worth big bucks


Big data that’s worth big bucks

Deepa Kurup

Realising its worth: More and more businesses, even in India, are looking to crunch their large data sets to see what works and what doesn’t. File photo: AP
Realising its worth: More and more busi­nesses, even in India, are look­ing to crunch their large data sets to see what works and what doesn’t. File pho­to: AP

Huge amounts of data are be­ing crunched to cre­ate mean­ing­ful in­form­a­tion

Last week, an of­fi­cial busi­ness meet between the chiefs of so­cial me­dia gi­ant Facebook and re­tail be­hemoth Walmart raised a few eye­brows.

While of­fi­cially it was given to un­der­stand that Walmart, a re­tail ma­jor which lags be­hind oth­ers such as Amazon in on­line re­tail, was look­ing to en­hance its so­cial me­dia pres­ence, tech for­ums de­lib­er­ated on the real pur­pose of the “re­la­tion­ship meet” — data. With over 800 mil­lion users, and need­less to say, a lot of in­tric­ate and of­ten geo-tagged per­son­al data up­loaded by them, Facebook presents a data trove like none oth­er, and Walmart, which has been on the ball as far as tech­no­logy goes, knows that. Just a few months ago, Walmart’s ac­quis­i­tion of ‘Social Calendar’ — a hugely pop­ular Facebook app that people use to track birth­days — was also, ob­vi­ously, about get­ting ac­cess to and us­ing data, mostly per­son­al, to make bet­ter and more cus­tom­ised busi­ness de­cisions.

Today, com­pan­ies, both at home and glob­ally, are wak­ing up to the value of data. The grow­ing in­terest in big data has ob­vi­ously to do with the fact that it is worth big bucks. Driven by the ex­plo­sion of so­cial me­dia, the all-pervasive use of mo­bile net­works and cloud stor­age, data has got­ten big­ger and big­ger, so much so that the term ‘big data’ — used in tech par­lance to refer to data sets that are large and tough to man­age — has come to be known as one that has no pre­scribed up­per lim­it.

As stor­age ca­pa­city, com­put­ing power and par­al­lel pro­cessing cap­ab­il­it­ies ex­pand, the value of data is be­ing real­ised bet­ter. That is, huge amounts of data (this could be data gen­er­ated with­in the en­ter­prise or data on it gen­er­ated on­line or on so­cial me­dia) is be­ing crunched to cre­ate in­sights or mean­ing­ful in­form­a­tion. And in­creas­ingly, this pro­cess, which used to take hours and even days, is now be­ing done in real time. While tools such as Hadoop al­lowed for real-time ana­lys­is of data, Google’s Dremel and oth­er Open Source im­ple­ment­a­tions that are de­vel­op­ing in this eco­sys­tem, al­lows for ad-hoc query­ing of big data in real time.

Around half a dec­ade ago, when ana­lyt­ics was still much in its in­fancy, a pop­ular and pro­voc­at­ive art­icle in asked if ana­lyt­ics sig­nalled the ‘end of the­ory’. In the peta­byte age, the art­icle pondered, will sci­en­ti­fic ana­lys­is based on hy­po­thes­is, mod­el­ling and test­ing be rendered ob­sol­ete? Is the­ory not rel­ev­ant any­more?

Today, big data en­thu­si­asts agree. An ‘ana­lyst’ is more of a “tool ex­pert”, or someone pro­fi­cient in us­ing vari­ous data ana­lyt­ic­al tools, and there is a lot of de­mand in the mar­ket for someone who can do this well, says Rahul Kulkarni, seni­or pro­duct man­ager at Google India.


More and more busi­nesses, even in India, are look­ing to crunch their large data sets to see what works and what doesn’t.

“And people are see­ing the value in that. Earlier, people were not en­thu­si­ast­ic about stor­ing data, but now they know that data con­tains in­sights that can aid cru­cial decision-making,” he ex­plains. Earlier, tak­ing this data and ana­lys­ing it was a two to three week cycle, but now most of this is pos­sible in real time, and the be­ne­fits of that are im­mense, he says.

However, sev­er­al obstacles lim­it their abil­ity to turn this massive amount of un­struc­tured data in­to profit, points out Mitesh Agarwal, Chief Technology Officer and Director, System Solution Consulting, Oracle India. The most prom­in­ent obstacle among them is a lack of un­der­stand­ing on how to add big data cap­ab­il­it­ies to the over­all in­form­a­tion ar­chi­tec­ture to build an all-pervasive big data ar­chi­tec­ture. “When big data is dis­tilled and ana­lysed in com­bin­a­tion with tra­di­tion­al en­ter­prise data, en­ter­prises can de­velop a more thor­ough and in­sight­ful un­der­stand­ing of their busi­ness, which can lead to en­hanced pro­ductiv­ity, a stronger com­pet­it­ive po­s­i­tion and great­er in­nov­a­tion — all of which can have a sig­ni­fic­ant im­pact on the bot­tom line.”

Technology-wise, com­pan­ies are now fo­cus­sing on ways to make the ana­lyt­ics and query in­ter­face as sim­ple as pos­sible. While in­tern­ally Google uses Dremel to do this for its own pro­cesses, for its cli­ents, Google provides ana­lyt­ics as a ser­vice. “What we at­tempt to de­liv­er is ana­lyt­ics in­ter­faces that are so sim­ple that a mar­ket­ing of­ficer can use it to pose ad-hoc quer­ies to the data set, and be able to ex­tract in­form­a­tion that can be used mean­ing­fully,” Mr. Kulkarni ex­plains.


As an emer­ging tech field, sev­er­al Indian com­pan­ies, big and small, have their eyes set on ana­lyt­ics. The big­ger out­sourcers, such as Wipro, TCS and Infosys, are in­to ana­lyt­ics ser­vices; sev­er­al oth­er lar­ger glob­al com­pan­ies across seg­ments ran­ging from auto­mobile to phar­ma­ceut­ic­al, are get­ting their ana­lyt­ics done here.

Apart from them, many smal­ler com­pan­ies and start-ups are in­to ana­lyt­ics ser­vices, and in some sense it is a nat­ur­al pro­gres­sion from busi­ness pro­cess out­sourcing to know­ledge pro­cess out­sourcing to ana­lyt­ics, says S. Anand, Chief Data Scientist at Gramener, a data visu­al­isa­tion com­pany. His com­pany is in­to ana­lyt­ics products and spe­cial­ises in the emer­ging tech field of data visu­al­isa­tion.

“During the nineties, the ser­vices mod­el did well and the products-model in IT did not pick up. That seems to be chan­ging, and in a field like ana­lyt­ics, it now ap­pears we may have the ad­vant­age on both,” he says.

Leave a Reply