Method In Madness

Trump’s very pres­ence is chaos, but a visu­al­iz­a­tion of un­struc­tured data such as the Presidential de­bates sure does help identi­fy the ‘Method in Madness’. An Ivy League pro­fess­or look­ing at Gramener’s visu­al­iz­a­tion of the 2nd Presidential de­bate said, ‘A re­mark­able way to break down and fol­low what of­ten seems chaot­ic and ran­dom.’

presdebate2

At the CNN de­bate, with every fin­ger poin­ted at him for the sex-tape con­tro­ver­sy, Trump tried his be­st to de­vi­ate at­ten­tion us­ing his fa­vor­ite fear mon­ger­ing top­ic – ‘ISIS’. There were at least 19 in­stances where Trump men­tioned ‘ISIS’ while Hillary had men­tioned the same only 5 times or so. For more in­sights do play around with our visu­al­iz­a­tion of the de­bate.

The Las Vegas de­bate on October 20th is do-or-die for Trump. Can Trump do any­thing at all to bring life to his dead cam­paign? Irrespective of the out­come, there’s go­ing to be chaos at the de­bate – lots of it.

“Though this be madness, yet there is method in’t.”

– William Shakespeare, Hamlet

Languages that cities love

We built a small tool that helps us re­cruit. It peri­od­ic­ally pulls data off of Github for de­velopers in India, and shows how they are con­nec­ted. You can watch this 2-minute video to un­der­stand how it works.

codersearch

This data also helps us un­der­stand how pop­ular dif­fer­ent pro­gram­ming lan­guages are across cit­ies. For ex­ample, if we take the top cit­ies, based on the num­ber of users (we’ve been fuzzy about the geo­graphy and in­cluded Colombo and Singapore in­to the mix)…

cities

… and the top pro­gram­ming lan­guages, again based on the num­ber of users …

languages

… it begs the ques­tion: is the pop­ular­ity of lan­guages the same across cit­ies? Or are there cer­tain cit­ies that love or hate cer­tain lan­guages?

This is the dis­tri­bu­tion of pro­gram­mers across these cit­ies:

distribution

This does not read­ily lead to any in­sights. But we could look at this num­ber dif­fer­ently. If all cit­ies had the same dis­tri­bu­tion, then what would these num­bers have looked like? In oth­er words, how many de­velopers of each pro­gram­ming lan­guage would each city have had? That’s shown be­low:

ideal-distribution

So, for ex­ample, Bangalore ac­tu­ally has 321 Javascript de­velopers. But if it had the same per­cent­age of Javascript de­velopers as oth­er cit­ies, it would just have had 263 Javascript de­velopers. So clearly, there are more Javascripters in Bangalore than you’d ex­pect.

The num­bers be­low show the dif­fer­ence between the ex­pec­ted and ac­tu­al num­ber of pro­gram­mers.differences

A few things stand out:

  • If you’re look­ing for Javascript pro­gram­mers, Bangalore and Mumbai would be the two places to vis­it. There are con­sid­er­ably more Javascript pro­gram­mers here than you’d ex­pect.
  • On the oth­er hand, if you’re look­ing for Java pro­gram­mers, you’d be much bet­ter off vis­it­ing Delhi, fol­lowed by Chennai and Bangalore.
  • There’s only one city to vis­it for Python pro­gram­mers – Bangalore. The rest are scattered across the minor cit­ies. (A closer look at the data re­veals that a reas­on­able num­ber are in Kerala.)
  • Colombo, on the oth­er hand, looks primar­ily like a Ruby shop. The fo­cus seems to be server-side de­vel­op­ment. Javascript pro­gram­mers are much rarer than nor­mal.
  • Gurgaon is the primary PHP hub. The city is under-represented in most pop­ular pro­gram­ming lan­guages, but has a thriv­ing group of PHP pro­gram­mers (a lan­guage that Chennai, Bangalore and Mumbai seem to act­ively dis­like.)
  • The biggest hub for iOS de­velopers (Objective-C) is Singapore. Within India, only Pune seems to have a slightly lar­ger than usu­al num­ber of iOS de­velopers – but that’s a mea­gre 20 pro­gram­mers.

Whether you’re a start-up look­ing for your lead de­velopers, or an IT firm re­cruit­ing open source geeks, or just a geek your­self look­ing for friends to hack with, we hope this gives you a idea of which city to vis­it next.

Making Public Service BIG with #BigData

3 Months back while the world was watch­ing, amid­st much hy­pe, a new gov­ern­ment an­nounced its suc­cess with a lot of prom­ise to one and all.

One of those prom­ises was that of min­im­um gov­ern­ment, max­im­um gov­ernance. This state­ment is as au­da­cious as much as it’s suc­cinct. The new gov­ern­ment prom­ises to re­in­vent pub­lic ser­vice, mak­ing it more ef­fi­cient, in­tro­du­cing trans­par­ency and stead­i­er and sus­tain­able growth. With a gov­ern­ment more tech-embracing than ever and the ad­vent of fin­ger­tip tech­no­logy to the people, a lot can be hoped. How much is achieved is yet to be seen.

To solve a prob­lem, know­ing the prob­lem clearly is the key. This key is held by the huge loads of Data that we have hid­den in the or­gan­iz­a­tion­al silos of our gov­ern­ment. At Gramener, we at­tempt to solve this prob­lem through rich­er, bet­ter data-driven in­sights, mak­ing it avail­able to the com­mon Joe. The ad­vent of Big Data in today’s world is not un­known. Big Data is a term that every­one is us­ing today. From board rooms to col­lege canteens, it’s now be­come the buzz in the more priv­ileged world.

Making Public Service Big With Big Data

Fraud Detection

Some stats to put the prob­lem at hand in per­spect­ive:

$314 bil­lion is what India loses from tax eva­sion an­nu­ally, de­priving it of funds for in­vest­ment in roads, ports and power.7 With so little rev­en­ue, the gov­ern­ment must bor­row more to fund a planned $1 tril­lion five-year in­fra­struc­ture pro­gram.

$462 Billion is what India lost due to tax eva­sion, crime and cor­rup­tion post-Independence.

Click here to see more on govt. spend­ing.

The more start­ling fact is that this money is not only from the big-scale frauds that we read about in the dailies. Small, un­re­por­ted frauds add up and form such bizar­re num­bers.

What if we could track these num­bers to their last ru­pee like in the fin­an­cial ser­vices in­dustry? What if we could have sys­tems to de­tect ir­reg­u­lar­it­ies in each micro-transaction? Government should in­vest in the in­fra­struc­ture to cap­ture data from all corners of the gov­ern­ment ma­chinery to one place. Read about how the UK govt saved 33bn a year us­ing Big Data Analytics.

How about Internal Security?

CCTV foot­ages, RFIDs and scan­ner ma­chines and oth­er elec­tron­ic data, al­though un­struc­tured but when used with deft­ness, Wirelessly in­ter­cep­ted in­form­a­tion, Internet brows­ing activ­it­ies can really help ex­tract use­ful in­form­a­tion for ana­lyses to de­tect crime, ter­ror­ist activ­it­ies and track­ing wrong­do­ers faster and easi­er and way more ef­fi­cient.

Law en­force­ment agen­cies need to ad­apt to such prac­tices for the great­er good. This re­quires a con­scious ef­fort to­wards skill ac­quis­i­tion, train­ing etc but it’s worth the ef­fort.

Public Services

Ever thought of filling that form for your PAN-Card purely on­line and get­ting it at your door­step without hassles. Ever wondered what it means to get up­dates about that sky­walk in your neigh­bour­hood, its status real-time on that smart­phone like your Facebook no­ti­fic­a­tion. With mul­tiple sources of data and your de­tails in­teg­rated in­to one place, your up­dates, ser­vices can be more and more per­son­al.

Will there be a time when we real­ize the above is not a hy­po­theses alone? Lets hope our gov­ern­ments real­ize it soon. Watch this space out for some more thoughts on how hav­ing the data is the new ne­ces­sity.