ClearTables icon indicating copy to clipboard operation
ClearTables copied to clipboard

Whole planet performance

Open pnorman opened this issue 9 years ago • 1 comments

After a full-planet import with --slim --cache 35000 --number-processes 4 --flat-nodes I have some import performance data

Node stats: total(3760704706), max(4694471408) in 17887s
Way stats: total(396150635), max(475723496) in 30638s
Relation stats: total(4804916), max(6994576) in 7245s
Pending: 389555748 ways in 14452
All indexes on airport_polygon created in 3s
All indexes on transit_polygon created in 5s
All indexes on airport_point created in 5s
All indexes on pier_area created in 8s
All indexes on landform_line created in 18s
All indexes on healthcare_polygon created in 31s
All indexes on aeroway_line created in 35s
All indexes on wikidata_point created in 41s
All indexes on recreation_point created in 42s
All indexes on healthcare_point created in 45s
All indexes on pier_line_raw created in 47s
All indexes on protected_area created in 63s
All indexes on landform_point created in 63s
All indexes on education_point created in 73s
All indexes on road_areas created in 75s
All indexes on place_polygon created in 80s
All indexes on education_polygon created in 163s
All indexes on transit_point created in 313s
All indexes on wikidata_line created in 427s
All indexes on recreation_polygon created in 832s
All indexes on rail created in 929s
All indexes on road_points created in 1001s
All indexes on wetland_areas created in 1154s
All indexes on place_point created in 1276s
All indexes on wikidata_polygon created in 1576s
All indexes on barrier_line created in 1652s
All indexes on admin_area created in 1746s
All indexes on admin_line created in 1791s
All indexes on water_areas created in 2470s
All indexes on wood_area created in 2544s
All indexes on waterways created in 2848s
All indexes on addresses_polygon created in 3486s
All indexes on addresses_point created in 3710s
All indexes on roads created in 6523s
All indexes on buildings created in 10130s
Stopped table: planet_osm_ways in 33477s
103721s overall

A slim drop import can take <1k sec for nodes and ~3k for ways. I wonder about short-circuiting the transforms in osm2pgsql. That would help hugely with nodes, not so much with ways.

pnorman avatar Mar 04 '17 04:03 pnorman

image image image image image image image image

pnorman avatar Mar 04 '17 05:03 pnorman