Understanding the complexity and sustainability of infrastructure development is crucial for reconciling economic growth, human wellbeing, and environmental conservation. However, previous studies on infrastructure’s material metabolism were mainly conducted on a global or national scale, due largely to lack of more spatially refined data, and thus could not reveal the spatial patterns and dynamics on a city scale. Here, we integrated material flow analysis (MFA) and geographical information system (GIS) for uncovering the spatiotemporal patterns of the material stocks and flows accompanying the infrastructure development at a high spatial resolution for a case of Shanghai, China. From 1980 to 2010, material stocks and waste output flows of Shanghai’s infrastructure system exhibited a significant increase from 83 million metric tons (Mt) to 561 Mt and from 2 Mt to 17 Mt, respectively. Input flows peaked in 2005 due to the economic slowdown and stepped-up policies to cool down the housing market. Spatially, the center and peri-urban areas were the largest container of material stocks and biggest generator of demolition waste, while suburban areas absorbed 58% to 76% of material inputs. Plans to make the city more compact will enhance the service capacity of stocks but may also increase the use of more energy and emissions intensive construction materials (e.g., steel). Prolonging the service lifetime of infrastructure through proper management and increasing the recycling and reuse rate of demolition waste are also identified as highly efficient strategies.