The Making of Manhattan’s Coffee Kings

Reading Time: 2 mins

This is part 1 of a 4-part series on The Making of Manhattan’s Coffee Kings.

For the Gramener data storytelling hackathon this month, I planned to tell the story of which coffee store covered the highest population in Manhattan.

This was not an accidental choice. I wanted something that’s map-based. I wanted to merge census data into maps. I also wanted something interesting enough to be marketable.

I read about the fight Starbucks has been facing from the likes of Dunkin’ Donuts and McDonald’s, and decided to write about it.

Planning

I had 4 days before the hackathon, so the planning had to be meticulous. I checked with the hackathon team and confirmed that scraping the data and pre-planning the story was within the rules, as long as I executed the story on the day of the hackathon.

The plan was as follows:

• Get the data (8 July)
  • Get the Manhattan census map
  • Get the Manhattan census population data
  • Get coordinates for coffee shops in Manhattan
• Create the maps (9 July)
  • Create the coffee shop coverage region map
  • Merge census population data into the coverage map
• Write the story (10-11 July)
  • Write down the insights as a storyline
  • Plan a format and design
  • Implement a version of this in Power BI (to showcase at Microsoft Inspire)

I could do all of these except the design and Power BI implementation. I’m not good at either. So I asked for help. My colleagues Devarani & Navya volunteered. Forming the team early helped a lot since we could plan well.

Luckily, we followed the plan almost perfectly, except that “Create the maps” took 2 days instead of 1, so we had only 1 day to “Write the story”. But that didn’t cause a problem.

Get the data

Getting the Manhattan census map and population data didn’t take much time. Both were on NYC.gov.

Scraping the stores’ location data took a bit of time. Using Chrome’s Developer Tools > Network., I found that McDonald’s store locator uses an API at https://www.mcdonalds.com/googleapps/GoogleRestaurantLocAction.do It accepted a radius= and maxResults= parameter. But increasing both, I was able to write a quick script that fetched the 1,000 closest stores within a 50 mile (or is it kilometer?) radius.

Starbucks’ store locator was trickier. The API returns stores around a ?lat= and a ?lng=, but there was no obvious way to increase the number of stores. It also does not allow direct access. It must be accessed via XHR.

Rather than solve this elegantly, I just manually moved the store locator map around to cover all of Manhattan and noted the locations. Then I created a script that collates all of these JSON responses together.

I tried scraping Dunkin’ Donuts’ website but got an Access Denied. It let them know, but it’s still down. So I assumed that they won’t last long, and moved on with just two stores.

At this point, we had all the data and were ready to create the maps.