The aim of this project is to visualize all the villages and cities in India in one map. Here is the data that inspired this project:
This figure is from the latest census released by the government in 2011. The first thought that came to my mind when I got to know this figure was, how little do I know about my own country. My best guess would have been 40-60 ratio of people living in villages to people in cities. This figure has changed the entire picture of India that I had in my mind. So I decided to actually paint a new picture of India. I wanted to see what India actually looks like. Hence the project.
Btw this is an example of what you get when you search for such a picture of cities and villages in India, mapped side by side, online now:
The data that I wanted was basically labelled data of the lat-long of every village and city in the country. I could not find and readymade data source so I decided to build my own. I could get lat-long information from Google’s geolocation api if I know the name of every village and city in the country. No such unified list of names of cities and villages exists but the data from census’ website can be parsed and combined to make that list so I did just that.
Step1: Manually download all the Primary Census Abstract tables from census website
Step2: Generate lat-long data for every location and store it in a CSV file
To get the lat-long data I used the geolocation api from google. I created a CSV file like this:
Tricky situation: There are more than a million locations (cities/villages) in India. Google lets you query only 2500 in a day.
There are a couple of ways to extend the limit of queries a little bit but none would have suffice my situation of querying more than a million times. I had no option but to split my queries across days. Save a query program in a cloud server and run it to make 2500 queries everyday and update the database. This would also mean that my project will enjoy a gradual completion process. Everyday new data would come in and complete the picture a little further. Even though this came out of constraint but I think it is nice that the true picture of India did not reveal itself in one shot but would gradually get completed on its own over months.
- All the code is uploaded on Digital Ocean
- DataParser is a program that will run periodically every day and add more latlong data to the database by talking to google geolocation api
- The database is a local JSON file
- The web app is running forever on the server revealing the map
Here is the link to the map: http://220.127.116.11:8000/