i've looked a bit more at the cucumber tests. the idea is to enable test based on small, well-known datasets. here's what tests look like:
@routing @basic
Feature: Basic Routing
Scenario: Smallest possible datasat
Given the nodes
| a | b |
And the ways
| nodes |
| ab |
When I route I should get
| from | to | route |
| a | b | ab |
| b | a | ba |
Scenario: Connected ways
Given the nodes
| a | | c |
| | b | |
And the ways
| nodes |
| ab |
| bc |
When I route I should get
| from | to | route |
| a | c | abc |
| c | a | cba |
| a | b | ab |
| b | a | ba |
| b | c | bc |
| c | b | cb |
Scenario: Unconnected ways
Given the nodes
| a | b |
| c | d |
And the ways
| nodes |
| ab |
| cd |
When I route I should get
| from | to | route |
| a | b | ab |
| b | a | ba |
| c | d | cd |
| d | c | dc |
| a | c | |
| c | a | |
| b | d | |
| d | c | |
| a | d | |
| d | a | |
Scenario: Pick the fastest way type
Given the nodes
| a | s |
| p | b |
And the ways
| nodes | highway |
| apb | primary |
| asb | secondary |
When I route I should get
| from | to | route |
| a | b | apb |
| b | a | bpa |
~/code/Project-OSRM$ cucumber --tags @basic
@routing @basic
Feature: Basic Routing
Scenario: Smallest possible datasat # features/basic.feature:4
Given the nodes # features/step_definitions/test_data.rb:124
| a | b |
And the ways # features/step_definitions/test_data.rb:139
| nodes |
| ab |
When I route I should get # features/step_definitions/routing.rb:167
| from | to | route |
| a | b | ab |
| b | a | ba |
| a | b | Empty route: [["Head", "", 0, 0, 0, "0", "NE", 22.5]] |
| b | a | Empty route: [["Head", "", 0, 0, 0, "0", "NE", 22.5]] |
Tables were not identical (Cucumber::Ast::Table::Different)
./features/step_definitions/routing.rb:180:in `/^I route I should get$/'
features/basic.feature:12:in `When I route I should get'
Scenario: Connected ways # features/basic.feature:17
Given the nodes # features/step_definitions/test_data.rb:124
| a | | c |
| | b | |
And the ways # features/step_definitions/test_data.rb:139
| nodes |
| ab |
| bc |
When I route I should get # features/step_definitions/routing.rb:167
| from | to | route |
| a | c | abc |
| c | a | cba |
| a | c | Empty route: [["Head", "", 0, 0, 0, "0", "NE", 22.5]] |
| a | b | ab |
| c | a | Empty route: [["Head", "", 0, 0, 0, "0", "NE", 22.5]] |
| b | a | ba |
| a | b | Empty route: [["Head", "", 0, 0, 0, "0", "NE", 22.5]] |
| b | c | bc |
| b | a | Empty route: [["Head", "", 0, 0, 0, "0", "NE", 22.5]] |
| c | b | cb |
| b | c | Empty route: [["Head", "", 0, 0, 0, "0", "NE", 22.5]] |
| c | b | Empty route: [["Head", "", 0, 0, 0, "0", "NE", 22.5]] |
Tables were not identical (Cucumber::Ast::Table::Different)
./features/step_definitions/routing.rb:180:in `/^I route I should get$/'
features/basic.feature:27:in `When I route I should get'
Scenario: Unconnected ways # features/basic.feature:36
Given the nodes # features/step_definitions/test_data.rb:124
| a | b |
| c | d |
And the ways # features/step_definitions/test_data.rb:139
| nodes |
| ab |
| cd |
When I route I should get # features/step_definitions/routing.rb:167
| from | to | route |
| a | b | ab |
| b | a | ba |
| a | b | Empty route: [["Head", "", 0, 0, 0, "0", "NE", 22.5]] |
| c | d | cd |
| b | a | Empty route: [["Head", "", 0, 0, 0, "0", "NE", 22.5]] |
| d | c | dc |
| c | d | |
| a | c | (i) "" |
| d | c | (i) nil |
| c | a | (i) "" |
| a | c | (i) nil |
| b | d | |
| c | a | Empty route: [["Head", "", 0, 0, 0, "0", "NE", 22.5]] |
| d | c | (i) "" |
| b | d | (i) nil |
| a | d | (i) "" |
| d | c | (i) nil |
| d | a | (i) "" |
| a | d | (i) nil |
| d | a | Empty route: [["Head", "", 0, 0, 0, "0", "NE", 22.5]] |
Tables were not identical (Cucumber::Ast::Table::Different)
./features/step_definitions/routing.rb:180:in `/^I route I should get$/'
features/basic.feature:46:in `When I route I should get'
Scenario: Pick the fastest way type # features/basic.feature:59
Given the nodes # features/step_definitions/test_data.rb:124
| a | s |
| p | b |
And the ways # features/step_definitions/test_data.rb:139
| nodes | highway |
| apb | primary |
| asb | secondary |
When I route I should get # features/step_definitions/routing.rb:167
| from | to | route |
| a | b | apb |
| b | a | bpa |
| a | b | Empty route: [["Head", "", 111, 0, 266, "0", "NE", 22.5]] |
| b | a | Empty route: [["Head", "", 0, 0, 0, "0", "NE", 22.5]] |
Tables were not identical (Cucumber::Ast::Table::Different)
./features/step_definitions/routing.rb:180:in `/^I route I should get$/'
features/basic.feature:69:in `When I route I should get'
Failing Scenarios:
cucumber features/basic.feature:4 # Scenario: Smallest possible datasat
cucumber features/basic.feature:17 # Scenario: Connected ways
cucumber features/basic.feature:36 # Scenario: Unconnected ways
cucumber features/basic.feature:59 # Scenario: Pick the fastest way type
4 scenarios (4 failed)
12 steps (4 failed, 8 passed)
1m56.321s
launching and quitting the binaries currently takes a while (perhaps due to large ram/disk allocations?) which makes the tests rather slow.
another idea would be to merge all three binaries into one, which could be kept running. it would then be instructed (perhaps by touching a file) to reprocess data. this would have the added benefit that request during reprocessing could be handled gracefully by sending back an appropriate status code.
i've looked a bit more at the cucumber tests. the idea is to enable test based on small, well-known datasets. here's what tests look like:
what goes on is:
here's an example run:
all tests are currently failing due to #88 (missing way names in routing instructions).
launching and quitting the binaries currently takes a while (perhaps due to large ram/disk allocations?) which makes the tests rather slow.
perhaps an option could be added to allocate small amounts when testing?
another idea would be to merge all three binaries into one, which could be kept running. it would then be instructed (perhaps by touching a file) to reprocess data. this would have the added benefit that request during reprocessing could be handled gracefully by sending back an appropriate status code.