New task was created
Community Portal
Better deal with large shapefiles
Created on Tuesday 17 November 2020, 07:45
Back to task list-
ID511352
-
ProjectMetabolism of Cities Data Hub
-
StatusOpen
-
PriorityMedium
-
TypeProgramming work
-
Assigned toNo one yet
-
SubscribersPaul Hoekman
Description
We're now getting many more shapefiles loaded into the system, which is great. However, some are 10... 20.... 30... 50 MB files! This becomes crazy large to load on a single map. Also, pulling in all that geometry takes time in the back-end, so in general we must just better figure out what to do with this. Also, I've seen some crazy detailed files about very specific places (the type of paint used throughout the port of Barcelona, stuff like that). So we must also question what we need and what we don't need. The reference spaces table is growing by leaps and bounds which again is great but 100k items just to describe this single port seems excessive. So let's figure out how we best approach this.
Discussion and updates
4 years, 1 month ago
Some first changes include:
- When a user wants to process a huge shapefile (> 1000 items), a warning pops up, see attached
- When a map is opened that is already processed but that contains > 500 items, then the system will only show the first 500 items on the map
- When a map is opened that is more than 5 / 10 / 20 MB, then the system will use a simplification script that makes lines in the polygon less detailed but much smaller (the level of simplification increases depending on the file size). The user is still given an option to open the non-simplified map, but it can take ages so the user is warned.
1 year, 2 months ago
I've stepped down from my Metabolism of Cities involvement but if another volunteer wants to pick this up later and ask me any question about this, feel free to reach out and I'll try to assist where I can.
1 year, 2 months ago
Paul Hoekman is no longer responsible for this task