Languages that cities love

We built a small tool that helps us re­cruit. It peri­od­ic­ally pulls data off of Github for de­velopers in India, and shows how they are con­nec­ted. You can watch this 2-minute video to un­der­stand how it works.

codersearch

This data also helps us un­der­stand how pop­ular dif­fer­ent pro­gram­ming lan­guages are across cit­ies. For ex­ample, if we take the top cit­ies, based on the num­ber of users (we’ve been fuzzy about the geo­graphy and in­cluded Colombo and Singapore in­to the mix)…

cities

… and the top pro­gram­ming lan­guages, again based on the num­ber of users …

languages

… it begs the ques­tion: is the pop­ular­ity of lan­guages the same across cit­ies? Or are there cer­tain cit­ies that love or hate cer­tain lan­guages?

This is the dis­tri­bu­tion of pro­gram­mers across these cit­ies:

distribution

This does not read­ily lead to any in­sights. But we could look at this num­ber dif­fer­ently. If all cit­ies had the same dis­tri­bu­tion, then what would these num­bers have looked like? In oth­er words, how many de­velopers of each pro­gram­ming lan­guage would each city have had? That’s shown be­low:

ideal-distribution

So, for ex­ample, Bangalore ac­tu­ally has 321 Javascript de­velopers. But if it had the same per­cent­age of Javascript de­velopers as oth­er cit­ies, it would just have had 263 Javascript de­velopers. So clearly, there are more Javascripters in Bangalore than you’d ex­pect.

The num­bers be­low show the dif­fer­ence between the ex­pec­ted and ac­tu­al num­ber of pro­gram­mers.differences

A few things stand out:

  • If you’re look­ing for Javascript pro­gram­mers, Bangalore and Mumbai would be the two places to vis­it. There are con­sid­er­ably more Javascript pro­gram­mers here than you’d ex­pect.
  • On the oth­er hand, if you’re look­ing for Java pro­gram­mers, you’d be much bet­ter off vis­it­ing Delhi, fol­lowed by Chennai and Bangalore.
  • There’s only one city to vis­it for Python pro­gram­mers – Bangalore. The rest are scattered across the minor cit­ies. (A closer look at the data re­veals that a reas­on­able num­ber are in Kerala.)
  • Colombo, on the oth­er hand, looks primar­ily like a Ruby shop. The fo­cus seems to be server-side de­vel­op­ment. Javascript pro­gram­mers are much rarer than nor­mal.
  • Gurgaon is the primary PHP hub. The city is under-represented in most pop­ular pro­gram­ming lan­guages, but has a thriv­ing group of PHP pro­gram­mers (a lan­guage that Chennai, Bangalore and Mumbai seem to act­ively dis­like.)
  • The biggest hub for iOS de­velopers (Objective-C) is Singapore. Within India, only Pune seems to have a slightly lar­ger than usu­al num­ber of iOS de­velopers – but that’s a mea­gre 20 pro­gram­mers.

Whether you’re a start-up look­ing for your lead de­velopers, or an IT firm re­cruit­ing open source geeks, or just a geek your­self look­ing for friends to hack with, we hope this gives you a idea of which city to vis­it next.

Making Public Service BIG with #BigData

3 Months back while the world was watch­ing, amid­st much hy­pe, a new gov­ern­ment an­nounced its suc­cess with a lot of prom­ise to one and all.

One of those prom­ises was that of min­im­um gov­ern­ment, max­im­um gov­ernance. This state­ment is as au­da­cious as much as it’s suc­cinct. The new gov­ern­ment prom­ises to re­in­vent pub­lic ser­vice, mak­ing it more ef­fi­cient, in­tro­du­cing trans­par­ency and stead­i­er and sus­tain­able growth. With a gov­ern­ment more tech-embracing than ever and the ad­vent of fin­ger­tip tech­no­logy to the people, a lot can be hoped. How much is achieved is yet to be seen.

To solve a prob­lem, know­ing the prob­lem clearly is the key. This key is held by the huge loads of Data that we have hid­den in the or­gan­iz­a­tion­al silos of our gov­ern­ment. At Gramener, we at­tempt to solve this prob­lem through rich­er, bet­ter data-driven in­sights, mak­ing it avail­able to the com­mon Joe. The ad­vent of Big Data in today’s world is not un­known. Big Data is a term that every­one is us­ing today. From board rooms to col­lege canteens, it’s now be­come the buzz in the more priv­ileged world.

Making Public Service Big With Big Data

Fraud Detection

Some stats to put the prob­lem at hand in per­spect­ive:

$314 bil­lion is what India loses from tax eva­sion an­nu­ally, de­priving it of funds for in­vest­ment in roads, ports and power.7 With so little rev­en­ue, the gov­ern­ment must bor­row more to fund a planned $1 tril­lion five-year in­fra­struc­ture pro­gram.

$462 Billion is what India lost due to tax eva­sion, crime and cor­rup­tion post-Independence.

Click here to see more on govt. spend­ing.

The more start­ling fact is that this money is not only from the big-scale frauds that we read about in the dailies. Small, un­re­por­ted frauds add up and form such bizar­re num­bers.

What if we could track these num­bers to their last ru­pee like in the fin­an­cial ser­vices in­dustry? What if we could have sys­tems to de­tect ir­reg­u­lar­it­ies in each micro-transaction? Government should in­vest in the in­fra­struc­ture to cap­ture data from all corners of the gov­ern­ment ma­chinery to one place. Read about how the UK govt saved 33bn a year us­ing Big Data Analytics.

How about Internal Security?

CCTV foot­ages, RFIDs and scan­ner ma­chines and oth­er elec­tron­ic data, al­though un­struc­tured but when used with deft­ness, Wirelessly in­ter­cep­ted in­form­a­tion, Internet brows­ing activ­it­ies can really help ex­tract use­ful in­form­a­tion for ana­lyses to de­tect crime, ter­ror­ist activ­it­ies and track­ing wrong­do­ers faster and easi­er and way more ef­fi­cient.

Law en­force­ment agen­cies need to ad­apt to such prac­tices for the great­er good. This re­quires a con­scious ef­fort to­wards skill ac­quis­i­tion, train­ing etc but it’s worth the ef­fort.

Public Services

Ever thought of filling that form for your PAN-Card purely on­line and get­ting it at your door­step without hassles. Ever wondered what it means to get up­dates about that sky­walk in your neigh­bour­hood, its status real-time on that smart­phone like your Facebook no­ti­fic­a­tion. With mul­tiple sources of data and your de­tails in­teg­rated in­to one place, your up­dates, ser­vices can be more and more per­son­al.

Will there be a time when we real­ize the above is not a hy­po­theses alone? Lets hope our gov­ern­ments real­ize it soon. Watch this space out for some more thoughts on how hav­ing the data is the new ne­ces­sity.

Solving Humanity With Data Science

Part 01 – The be­gin­ning

a perfect world

In the book, A Song of Ice and Fire, George R.R. Martin says “Summer will end soon enough, and child­hood as well”. The quote has an im­pend­ing truth to it, al­most as if he is warn­ing us of a dark end to hu­man­ity it­self. Since the be­gin­ning of time, man has faced ad­versity in myri­ad forms, from the whim­sic­al de­struc­tions of nature to the test of time it­self. The most po­tent en­emy that man has had, how­ever, is him­self. Civilization has helped us grow out of the wild hunters that we were, in­to a spe­cies who are cap­able of design­ing com­plex sys­tems like money, trans­port­a­tion, polit­ics and even send ourselves to out­er space. We as a spe­cies have ex­plored and ex­ploited our world, cre­ated cit­ies with re­mark­able ar­chi­tec­tures and in­ven­ted con­cepts over­time that would have baffled our an­cest­ors. Why is George R.R. Martin then, warn­ing us of an im­min­ent and dark danger? Why can’t we, hu­mans, who are cap­able of any­thing, cre­ate a per­fect world to live in, a uto­pia? The truth is, we can’t, be­cause we’re a troubled spe­cies and a very pro­fane en­emy lurks in our sys­tem; as the pro­verb “A chain is only as strong as its weak­est link” el­eg­antly sum­mar­izes.

The so­ci­ety that we’ve evolved in­to today has some ser­i­ous so­cial flaws, in­clud­ing Gender bi­as, Poverty, Crime, Resource scarcity and I could go on. On December 16, 2012, a hein­ous act by a few in­di­vidu­als in ab­us­ing and caus­ing a girl’s death saw a na­tion protest in uni­son. These crimes are the weak links that scar our so­ci­ety, tu­mours that ter­ri­fy the in­no­cent, and in­cid­ents that make hu­man­ity very far away from cre­at­ing a per­fect world. Barely two years since, on July 02, 2014, a six year old child was ab­used in a school, an in­no­cent child­hood that prob­ably ended too soon. Society is rid­den with such prob­lems, prob­lems that bring us down every time we show­case our achieve­ments, prob­lems that ri­dicule our very be­ing. These prob­lems though, aren’t new, they have a very hu­man ele­ment to them – Anger, Greed, Lust, and the like. In some sense, re­li­gion has an an­swer to these weak­nesses of man­kind. At Gramener, Science is our re­li­gion, and we in­tro­spect today, if one of our greatest achieve­ments (Science) can cre­ate a per­fect world.

Science is a meth­od­ic­al sys­tem, it’s a study of nature’s be­ha­vi­our and can be quan­ti­fied by num­bers. Science helps us cre­ate mod­els with the help of num­bers and equa­tions to rep­res­ent and an­swer some of the most com­plex ques­tions that we have ever faced in our his­tory. We have to use this, our greatest strength, to try and solve the is­sues that are in­hib­it­ing us from evolving in­to a higher, and more civ­il­ized spe­cies. The num­bers that I talk about, the num­bers which can quan­ti­fy situ­ations and sys­tems, is noth­ing but data. There is data, hid­den every­where, in our world today. Data about people, data about weather, data about trans­ac­tions, data about what people are think­ing on on­line me­dia, data every­where. This burst of data has been a nat­ur­al evol­u­tion of the Internet, one of mankind’s greatest achieve­ments. Can these num­bers that are every­where, help us look at a story that is hid­den from the na­ked eye? Can these num­bers help us identi­fy pat­terns of hu­man fail­ures? Can we sci­en­tific­ally provide a solu­tion that can solve hu­man­ity?