Introduction to Exploratory Data Analysis – A video lesson

It’s al­ways not ne­ces­sary to use ma­chine learn­ing al­gorithm for ex­tract­ing in­ter­est­ing stor­ies from data. Here is a video by Kathirmani Sukumar, Data Scientist at Gramener, which ex­plains data ana­lys­is sans com­plic­ated ma­chine learn­ing tech­niques. The video les­son will help in learn­ing how to ana­lyse data us­ing a few sim­ple (but power­ful) tech­niques based on Exploratory Data Analysis (EDA) us­ing data from the sport of Cricket. It is use­ful for those who want to do data ana­lys­is, but are not sure of where to start and what to learn from. It also dis­cusses about ba­sics of data types, data muta­tion and uni­vari­ate ana­lys­is, and these tech­niques are do­main ag­nostic. One can ap­ply the same tech­niques on any data from any do­main. It fo­cuses on us­ing Pandas lib­rary for data pro­cessing and plot­ting the res­ults. The Jupyter note­book can be down­loaded from The next video in the series will be a les­son on uni­vari­ate and bivari­ate ana­lys­is.

Dark Patterns in Web Design defines dark pat­terns as designs “that are not mis­takes but care­fully craf­ted with a solid un­der­stand­ing of hu­man psy­cho­logy”. In oth­er words, the ac­tion in­ten­ded by the user is masked un­der an­other design lay­er. This is il­lus­trated be­low us­ing use-cases from three dif­fer­ent soft­ware ap­plic­a­tions.

1) Dark – MySQL down­load

With en­ter­prise and com­munity edi­tions, MySQL is one of the most pop­ular data­base soft­ware that is used across the world. It can be used for desktop ap­plic­a­tions as well as cloud ap­plic­a­tions. Consider a user who in­tends to down­load MySQL soft­ware from and clicks on a op­er­at­ing system-specific down­load link. This prompts a new page with fo­cus on Login or Signup op­tions. A new user might be driv­en to Signup in or­der to down­load the soft­ware. However, if the user scrolls down there is a ‘No thanks, just start my down­load’ link which prompts the down­load.

Hiding MySQL download link after login/signup options
Hiding MySQL down­load link af­ter login/signup op­tions

2) Darker – WhatsApp’s phone num­ber shar­ing with Facebook

WhatsApp is a pop­ular so­cial net­work mes­saging ap­plic­a­tion that boasts of over 600 mil­lion act­ive users. When WhatsApp was ac­quired by Facebook in 2014 its co-founder Jan Koum prom­ised that there will be no data shar­ing with Facebook. WhatsApp rolled end-to-end en­cryp­tion for mes­sages in April 2016. In late fall, the or­gan­iz­a­tion rolled out a pri­vacy up­date that shares a user’s phone num­ber with Facebook.

Image courtesy: EFF
Image cour­tesy: EFF
Image courtesy: EFF
Image cour­tesy: EFF

The screen that prompts an up­date to Terms and Privacy Policy high­lights an op­tion to Agree to the rol­lout. However, there is an op­tion to read more about the key updates to­wards the end of the screen. Upon click­ing that, an op­tion to un-enroll from shar­ing the Whatsapp ac­count in­form­a­tion with Facebook is re­vealed. It is now com­mon know­ledge that most users do not read Terms and Conditions and just ac­cept an application’s up­date. Facebook lever­ages on this and hides this ac­tion by de­fault. Electronic Frontier Foundation (EFF) has lead in spread­ing this aware­ness.

3) Windows 10 – Privacy set­tings

Windows 10 drew a lot of flak for its ques­tion­able de­fault set­tings. Where does the flak ori­gin­ate from? Consider these: it can track your loc­a­tion (Location ser­vices), it sends Microsoft what you write (Getting to know you), in­ab­il­ity to turn off auto­mat­ic Windows up­dates among oth­er set­tings. The Getting to know you fea­ture drew severe cri­ti­cism for send­ing data to the servers in the United States prompt­ing a call in Russia to ban the use of Windows 10 in state agen­cies.

Please refer to the art­icles by LifeHacker and ArsTechnica to know the spe­cific pri­vacy con­cerns and im­plic­a­tions of the dark design pat­terns.

Not a tech-only prob­lem

Dark pat­terns in wide­spread ser­vices are not re­stric­ted only to soft­ware pro­duced by tech­no­logy or­gan­iz­a­tions. Consider the fol­low­ing scen­ari­os. Some of us read through the in­gredi­ents of a bottle of coca-cola, a cook­ie pack­et, a bread pack­et that are avail­able in your loc­al store or an hy­per­mar­ket. How about that bur­ger you buy with a hungry stom­ach in a loc­al bakery or the soda you buy in the fast-food chains? None of these clearly show the im­plic­a­tions of con­sum­ing de­li­cious fatty foods. Research stud­ies have con­sist­ently shown the risks of con­sum­ing high sug­ars. Very few stop to con­sider the im­plic­a­tion of con­sum­ing such foods.

The root of some of the above dark pat­terns is the design of the soft­ware ap­plic­a­tion (MySQL, Whatsapp or Windows 10) and the design of a pack­age (bottle, pack­et) for the rest. In the case of food and bever­ages, a well-meaning cent­ral food and drugs ad­min­is­tra­tion au­thor­ity can re­strict the food on sale de­pend­ing on the in­gredi­ents. Design, when done well, can provide use­ful func­tion­al­ity but has severe im­plic­a­tions when ill-motivated.