(OpenDataMeet is where)… all the data enthusiasts come on board talk about the data initiatives [and] what they’re doing… It might be NGOs, it might be the corporates, enterprises, academicians, everyone. The key goal is: how we can liberate data, get it into the environment so that it’s accessible for everyone; for public use, for enterprise use, for corporate use, for the whole benefit of the society.
We at Gramener, right from day one, [have] been part of Datameet. We are supporting this event. We believe it can make a strong impact to humankind.
So we at Gramener, (introducing to you Gramener: it’s an analytics and visualisation company) we convert numbers into pictures. What that means is: you have tons and tons of data, coming from different places. That might be social media, that might be enteprise, that might be corporate, your own spreadsheets — whatever it might be, (even) your mobile phones generate data. We take all those things from heterogenous data sources, process it in a non-traditional analytics way, and present it out in a visually appealing manner, so that the insights can be derived in the span of a second.
Essentially, (1) it enables you from a decision making perspective. (2) If you’re looking at 10 spreadsheets to make a decision, our target is: can that be made one spreadsheet, (and) still you get more.
The Economist Intelligence Unit report, “The Deciding Factor :Big Data & Decision making,” commissioned by Capgemini, reveals that nine out of ten business leaders believe data is now the fourth factor of production, as fundamental to business as land, labor, and capital. The study, which surveyed more than 600 C-level executives and senior management and IT leaders worldwide, indicates that the use of Big Data has improved businesses’ performance, on average, by 26 percent and that the impact will grow to 41 percent over the next three years. The majority of companies (58 percent) claim they will make a bigger investment in Big Data over the next three years.
Data analysis tools used by banks to determine if you are likely to default on loan repayment, could soon help nab criminals.The National Crime Records Bureau decided to deploy data analytics to not only track criminals but also to predict incidence of crime in a region on the basis of past incidents.
Banks are teaming-up with retailers to target customers by analysing the customer data they collect more closely.
A bank can’t make a customer spend £1,000 that they were not planning to spend. However, if the customer is planning to buy a new TV for £1,000, for example, then there might be ways to encourage the customer to use that bank’s credit card, or to take out a loan.
The use of data to drive management decisions and product design is well known in the financial sector. Today, most commercial banks utilize data analysis to support their decision-making. Grameen Foundation’sMicrosavings Initiative uses these same techniques to help advance the mission of our partner microfinance institutions (MFIs) to effectively bring savings products and services to their poor clients.
Sears (A major retailer in North America) is using big data to help set prices – nearly in real time — and move inventory by giving loyalty shoppers customized coupons.
Sears is using the software to correlate huge quantities of information about everything from product availability in specific stores to competitor prices on specific products to information about local economic conditions in order to set prices.
There’s great opportunity for new business models to emerge. One example is a 9-person firm in Israel that manages an online site that uses analytics to aggregate thousands of social media feeds to provide their audience with a single view of information, research and opinions on various medications on the market. They do not promote one drug or treatment over another. They simply provide a single view of related data for consumers to read what other patients have experienced – in their own words – so they can make their own decision.
No matter the staff size, annual revenue or number of offices, Big Data can be a big advantage for small, medium and large businesses – if they have access to the tools that can help them link relevant information sources while finding useful insights that are buried deep within.
CIO Journal has spotted billboards for Big Data along the side of the road — in Times Square and on Highway 101.
Big Data itself isn’t a competitive differentiator, but your ability to find creative ways of using it very well may be. For instance, at eBay, Big Data is used to identify potential fraud, according to its CTO Mark Carges; at retailers Gilt Groupe and Sears, it’s used to improve product recommendations – but even there, the two companies use different types of Big Data software and have different hardware set-ups to run it.Commonwealth Bank of Australia’s CIO, Michael Harte, tells CIO Journal the bank is using Big Data for real-time pricing and risk assessment, allowing its reps to make credit decisions instantaneously.
Here are top 3 reasons why marketers should let themselves fall in love with data scientists:
1)Drowning in data = bad time management.
Marketer’s time belongs in marketing … not stuck in a spreadsheet with numbers they can’t begin to understand. This critical task is best done by Data scientist.
2)The C-suite craves structure around new marketing methods.
Now, more than ever before, the C-suite demands accountability, and data scientists are best hope for understanding big data, interpreting results and providing solid insights to act on and report.
3)They’re not just machines—they’re part-magician!
Data scientists aren’t robots or computers (though their ability to process information may outstrip your laptop some days). Data scientists know how to deal with facts and ideas, with “definitely” and “maybe.” They pull the big picture from the big data mosaic, and make it make sense.
If you’re a marketer trying to dig your way out of big data, it’s time to admit that you can’t do It alone. Let a data scientist sweep you—and your marketing goals and plans—off your feet.
At 1.8 million words, the Mahabharatha is one of the largest epics – roughly 10 times the size of the Iliad and Odyssey combined. At some level, this represents “big data”. Text is generally considered “unstructured” and therefore tough to analyse. But the growing field of text analytics and text visualisation tell us that there’s a lot more structure to plain text than one might think.
To begin with, a word cloud can tell us a lot about the story.
The story is obviously about a battle between great kings and sons, with the principal characters being Arjuna, Pandu, Bhishma, Bharata, Karna, Duryodhana, Yudhishthira, Vaisampayama, etc. That’s decipherable without having to read the text.
The structure that we gleam out of it arises from a frequency distribution of the words – i.e. a count of which words occur how many times. The word cloud plots the words at a font size proportional to the frequency of occurrence. (Wordle is a good place to create word clouds.)
Now what we know who’re the principal characters, the next questions are: where are they mentioned? Who’re closely related? etc.
Our Mahabharatha browser provides a simple interface to browse the full text of the Mahabharatha and find where the characters appear.
The Mahabharatha is made of 18 books, each with several sections. This visualisation shows each section as a block (the length of the block is proportional to the size of the section.) When you click on a character’s name, the positions in each section where they are mentioned are highlighted
This makes it easy to see where characters speak together (e.g. where does Kunti throw away Karna? Where does she meet him again? Did Draupadi really love Karna before her wedding? Was Arjuna really her favourite? Whom does Krishna favour? etc.) By clicking on the section, you can read the full text of that section.
The second question is, which characters are most closely related? Measuring closeness of characters is a difficult thing to do, even for humans. Fortunately, with text, we can rely on a proxy: how often are two characters found within a few words of each other.
If we take Draupadi as a benchmark character and check how often various people are mentioned within a few words of her, here’s what the picture looks like:
Each row has the name of the character (along with aliases). The first column shows the number of times they’re mentioned within 50 words of her. The next shows how many times they’re mentioned within 100 words of her. And so on. (All within the same section.)
A visual inspection suggests that many characters start fading off at a distance of 200 words, so perhaps 200 might be a reasonable boundary to consider. (This is arbitrary. But based on our subsequent analysis, we find that this parameter does not impact the visual result too much.)
By plotting a network of their closeness, one can get some insights about the structure of the tale.
Yudhishthira is clearly at the centre of the plot. Arjuna, surprisingly, isn’t. Apart from his close relationship with Krishna and Bhishma, his interaction with other characters is not as well spread out (despite his popularity in the epic.) Contrary to popular opinion, Bhima is mentioned quite often, and is fairly well-networked. Nakula and Sahadeva remain peripheral characters. Gandhari is nearly outside of the network, except for her connection with her husband Dhritarashtra, sister-in-law Kunti, and brother-in-law Vidura (with whom she seems to converse much more than with her husband.)
Another way of looking at this picture is through a correlation matrix.
This shows each pair of characters and the number of times they occur within 200 words of each other. The closeness between Nakula and Sahadeva is very obvious; so are Drona & Kripa; Dhritharastra & Vidura; Arjuna & Krishna. Draupadi is mentioned with Dhrishtadhyumna more than anyone else.
You can also see the blocks breaking up into two clusters of sorts – on the bottom right are the primary characters. They interact a lot with each other. In the middle are secondary characters, who again interact amongst themselves; and then there are the narrators on the top left. This is in line with the Mahabharatha discussing several side-plots with secondary characters in parallel with the main plot. The story of Dhrishtadhyumna, of Satyaki, Nakula and Sahadeva’s conversations, etc are examples of these. In fact, in a larger scatterplot, you can see many more tales emerge, such as Nala & Damayanti; Nahusha & Yayati; Uma & Daksha; Vasishta & Vishwamitra; Chitrasena & Vikarna; Virata & Uttara; Dhrishtadhyumna & Shikhandin; Parva & Sambhava; even Ravana & Vali.
If you are interested in seeing the full correlation matrix with all major and minor characters, please reach us at firstname.lastname@example.org.