skf268@nyu.edu
india_map.jpg

Understanding India in one map

Understanding India in one map

The aim of this project is to visualize all the villages and cities in India in one map. Here is the data that inspired this project:

There are 245mn families in India, out of which 180mn live in villages
— NDTV

This figure is from the latest census released by the government in 2011. The first thought that came to my mind when I got to know this figure was, how little do I know about my own country. My best guess would have been 40-60 ratio of people living in villages to people in cities. This figure has changed the entire picture of India that I had in my mind. So I decided to actually paint a new picture of India. I wanted to see what India actually looks like. Hence the project.

Btw this is an example of what you get when you search for such a picture of cities and villages in India, mapped side by side, online now:

Gathering Data

The data that I wanted was basically labelled data of the lat-long of every village and city in the country. I could not find and readymade data source so I decided to build my own. I could get lat-long information from Google’s geolocation api if I know the name of every village and city in the country. No such unified list of names of cities and villages exists but the data from census’ website can be parsed and combined to make that list so I did just that.

Step1: Manually download all the Primary Census Abstract tables from census website

Step2: Generate lat-long data for every location and store it in a CSV file

To get the lat-long data I used the geolocation api from google. I created a CSV file like this:

 Name, Level, Bounds and Lat-Long data of every village and city in India

Name, Level, Bounds and Lat-Long data of every village and city in India

Tricky situation: There are more than a million locations (cities/villages) in India. Google lets you query only 2500 in a day.

There are a couple of ways to extend the limit of queries a little bit but none would have suffice my situation of querying more than a million times. I had no option but to split my queries across days. Save a query program in a cloud server and run it to make 2500 queries everyday and update the database. This would also mean that my project will enjoy a gradual completion process. Everyday new data would come in and complete the picture a little further. Even though this came out of constraint but I think it is nice that the true picture of India did not reveal itself in one shot but would gradually get completed on its own over months.

Final Setup:

  • All the code is uploaded on Digital Ocean
  • DataParser is a program that will run periodically every day and add more latlong data to the database by talking to google geolocation api
  • The database is a local JSON file
  • The web app is running forever on the server revealing the map

Here is the link to the map: http://174.138.84.113:8000/

Here is the github: https://github.com/kshivanku/IndiaMap_Complete  and https://github.com/kshivanku/IndiaMap_bkend

Here is what the map looks like at the moment

 Currently drawing Andra Pradesh. Clearly villages will overshoot districts with a lot.

Currently drawing Andra Pradesh. Clearly villages will overshoot districts with a lot.

 Each village or district is represented by a block

Each village or district is represented by a block

 This is what the parsed data looks like. Would be a great resource to share once the data for all the villages and cities is added

This is what the parsed data looks like. Would be a great resource to share once the data for all the villages and cities is added