Community Portal

Deciding on loading of national boundaries

Created on Friday 26 June 2020, 12:26

Back to task list
  • ID
    32040
  • Project
    Metabolism of Cities Data Hub
  • Status
    Open
  • Priority
    High
  • Type
    Data research
  • Tags
    Good entry-level task
  • Assigned to
    No one yet
  • Subscribers
    Paul Hoekman

You are not logged in

Log In Register

Please join us and let's build things, together!

Description

It would be very helpful to load one single dataset that contains boundaries for ALL countries in the world. There may be national and regional shapefiles that contain country boundaries that could be slightly more detailed or updated, but for our purposes whether or not this an outcrop or river border is part of country A or B would likely not make a difference in our use-case. Also, by having one single shapefile we limit the risk of overlapping boundaries. This way every area is part of ONE (and not two or zero) countries. And as long as we can simply refer to another source for the information we shouldn't be perceived as being partial.


My suggestion is to use Natural Earth data for this. It is an amazing initiative and they seem very much on-it when it comes to getting this kind of data.


So my suggestion would be to load all national boundaries from that source. We could to try 1:10m resolution (less than 5MB for all countries). I think that's not too big, but we may want to try a slightly less detailed one to see the difference and consider if it's worth it (after all, downloading 5MB just for the map on a website is quite a lot). I would suggest we load their Countries and Sovereign States.


What are your thoughts? Any better sources or other ideas?

General instructions

Finding new sources of data, new methods of collecting data, research on data types for further development

Discussion and updates

Paul Hoekman Paul Hoekman Johannesburg 2140 points
4 years, 6 months ago

New task was created

Carolin Bellstedt Carolin Bellstedt Metro Vancouver Regional District 1280 points
4 years, 6 months ago

I don't know if this is a better source, but I just found these



  • https://mledoze.github.io/countries/

  • https://library.duke.edu/data/files/esri/esridm/2012/world/data/admin.html

  • https://gis.stackexchange.com/questions/23028/getting-polygons-of-all-countries-and-iso-3166-2-subdivisons


However, if you are entirely convinced of Natural Earth data, feel free to go with it.


Not sure if this is relevant right away as well, but for country names and sub-divisions, there is this ISO that we then link to it, for which you said we already use the names, right? https://en.wikipedia.org/wiki/ISO_3166-1

Paul Hoekman Paul Hoekman Johannesburg 2140 points
4 years, 6 months ago

Thanks Carolin. I honestly also don't know what the best option is. So far Natural Earth is the one I'd give my vote to, but I would not say that that is enough to warrant final selection...


So here is my feedback on the other ones. And please do provide counter points and let me know where you disagree!



  • mledoze: as far as I can see this only contains latitude and longitude...?

  • Duke: under use constraints it says: The data are provided by multiple, third party data vendors under license to Esri for inclusion on Data & Maps for ArcGIS® for use with Esri® software. -- I don't think we can use that.

  • GIS Stackexchange -> highest voted answer called GAMD looks interesting but meta data is extremely scarce. Website also has an SSL error which is not too inspiring (maybe a defunct project?) No github repo. Also it says in the license specifically that "Redistribution, or commercial use is not allowed without prior permission." --> I don't know if we're allowed to include it in our open source website without seeking prior permission.


What do you think?


And yes at the moment all ISO 316-1 country names are included.

Paul Hoekman Paul Hoekman Johannesburg 2140 points
4 years, 4 months ago

Just to keep record here: we discussed yesterday over the phone that Natural Earth at this point seems to be the most useful source for this information. At least as far as we can tell. Aris, you mentioned that perhaps the UN may have this information. I have not been able to find a better source - especially seeing that Natural Earth also has other levels like Admin 1 (States, Provinces, etc), and things like roads and airports. Going with a single source as our "main global shapefile" provider has some benefits and at this point I feel it makes most sent to use Natural Earth and load as many of their Cultural Vectors as we feel are appropriate.

If anyone knows of a better source or has other suggestions or comments, please do share them soon!

Paul Hoekman Paul Hoekman Johannesburg 2140 points
4 years, 4 months ago

While looking into taking this to the next level, I came across a very interesting Python package: GeoPy. This can be used for looking up geo references. And they have a TON of different API services they can connect to. What this gives us is a very interesting list of possible services to use. This is not necessarily for the country boundaries, but it is worth having a look before proceeding. I'll investigate in more detail. FYI the options they offer are:


AlgoliaPlaces
ArcGIS
AzureMaps
Baidu
BaiduV3
BANFrance
Bing
DataBC
GeocodeEarth
GeocodeFarm
Geolake
GeoNames
GoogleV3
HERE
IGNFrance
MapBox
MapQuest
MapTiler
OpenCage
OpenMapQuest
Nominatim
Pelias
Photon
PickPoint
LiveAddress
TomTom
What3Words
Yandex

Paul Hoekman Paul Hoekman Johannesburg 2140 points
4 years, 4 months ago

And here is another useful document with data sources used by one of the service providers.


Global Sources


These are the main places we get our data for most of the world:


TIGER/Line® - U.S. Census Bureau
OpenStreetMap Planet File
GeoNames
Natural Earth
Statistics New Zealand
OpenGeoCode
Contains Ordnance Survey data © Crown copyright and database right 2014
Contains Royal Mail data © Royal Mail copyright and database right 2014
Contains National Statistics data © Crown copyright and database right 2014
Thematic Mapping
Zillow
AidData BETA
GPS Navigation Partners
Proprietary Data Partners
Our Paid Street Runners

TimeZone Data


These are the two places we get our TimeZone data from:


Blake Crosby's TZ-Map, Original Source: Eric Muller
Unicode.org, CLDR Version 29, Zone-Tzid Table

Elevation Data


These are the places we get our Elevation data from:


Jonathan de Ferranti's Digital Elevation Data
ASTER Global Digital Elevation Map