Test listed as passed if the test execution encounters a script error #210
Description
Given the following test case:
func test_updown_signal_includes_column_index():
var cut = component_template.instance()
add_child_autofree(cut)
watch_signals(cut)
cut.setSize(3, 3)
cut.set_column_index(2)
var up = cut.find_node("Up")
var down = cut.find_node("Down")
util.do_a_left_click(util.get_control_center(down))
assert_signal_emitted_with_parameters(cut, "pressed", [2, 1])
util.do_a_left_click(util.get_control_center(up))
assert_signal_emitted_with_parameters(cut, "pressed", [2, 0])I see this output:
res://test/integration/inventory/test_four_ways_component.gd
[...]
* test_updown_signal_includes_column_index
SCRIPT ERROR: test_updown_signal_includes_column_index: Invalid call. Nonexistent function 'set_column_index' in base 'MarginContainer (dd_single_4-ways_component.gdns)'.
At: res://test/integration/inventory/test_four_ways_component.gd:64.
* test_mouse_click_on_leftright_raises_signal_with_coordinates
* test_leftright_signal_includes_row_index
SCRIPT ERROR: test_leftright_signal_includes_row_index: Invalid call. Nonexistent function 'set_row_index' in base 'MarginContainer (dd_single_4-ways_component.gdns)'.
At: res://test/integration/inventory/test_four_ways_component.gd:104.
* test_leftright_is_clamped_to_size_and_zero
SCRIPT ERROR: test_leftright_is_clamped_to_size_and_zero: Invalid call. Nonexistent function 'get_column' in base 'MarginContainer (dd_single_4-ways_component.gdns)'.
At: res://test/integration/inventory/test_four_ways_component.gd:124.
11/11 passed.
cut is a script implemented with GDNative, which does not have a method named set_column_index. This means, the assertion would fail, if it were executed.
It looks like the script error breaks out of the test function, and as there were not failed assertions, the test "passed".
If I just remove the offending method call you can see the difference:
func test_updown_signal_includes_column_index():
var cut = component_template.instance()
add_child_autofree(cut)
watch_signals(cut)
cut.setSize(3, 3)
#cut.set_column_index(2)
var up = cut.find_node("Up")
var down = cut.find_node("Down")
util.do_a_left_click(util.get_control_center(down))
assert_signal_emitted_with_parameters(cut, "pressed", [2, 1])
util.do_a_left_click(util.get_control_center(up))
assert_signal_emitted_with_parameters(cut, "pressed", [2, 0])res://test/integration/inventory/test_four_ways_component.gd
[...]
* test_updown_signal_includes_column_index
[Failed]: Expected object [MarginContainer:1738](res://bin/dd_single_4-ways_component.gdns) to emit signal [pressed] with parameters [2, 1], got [0, 1]
at line -1
[Failed]: Expected object [MarginContainer:1738](res://bin/dd_single_4-ways_component.gdns) to emit signal [pressed] with parameters [2, 0], got [0, 0]
at line -1
* test_mouse_click_on_leftright_raises_signal_with_coordinates
* test_leftright_signal_includes_row_index
SCRIPT ERROR: test_leftright_signal_includes_row_index: Invalid call. Nonexistent function 'set_row_index' in base 'MarginContainer (dd_single_4-ways_component.gdns)'.
At: res://test/integration/inventory/test_four_ways_component.gd:104.
* test_leftright_is_clamped_to_size_and_zero
SCRIPT ERROR: test_leftright_is_clamped_to_size_and_zero: Invalid call. Nonexistent function 'get_column' in base 'MarginContainer (dd_single_4-ways_component.gdns)'.
At: res://test/integration/inventory/test_four_ways_component.gd:124.
11/13 passed.
I would expect the script error to cause the same result, i.e. count the aborted test as failed.
As it is right now, this causes false-positives unless you carefully read the log.