Let some future us add this to some test. 😉

I actually wanted to extend tests/shell. Turns out RIOT_VERSION is not exported to the test runner and I did not want to mess around with that atm ^^".

Turns out RIOT_VERSION is not exported to the test runner and I did not want to mess around with that atm

Adding export RIOT_VERSION after include $(RIOTBASE)/Makefile.include in the application Makefile should be enough to be able to retrieve it from the python script.

Adding export RIOT_VERSION after include $(RIOTBASE)/Makefile.include in the application Makefile should be enough to be able to retrieve it from the python script.

That test should only fail if the git commit is not dirty. Otherwise the (soon to be available) make test-murdock fails unless the local changes are all commited.

Sooo should I add a test or not?

Sooo should I add a test or not?

Feel free. I ACK this without.

(unfortunately this PR seems to push some boards into insufficient memory...)

There is also the issue, that the tests/periph_wdt test now does not fit our atmega328p platforms anymore. This test previously had no blacklisting. Since it is the RAM that is running out a simple solution could be to set RIOT_VERSION="" for this particular test.

On the other hand, I'm wondering what the shell_commands module is needed for in this PR.

The RAM problem is related to some work @fjmolinas made for AVR. Related issues/PR: #13016, #12940

In this case however the app developer did not seem to have been as frugal as they could have been. I propose to remove the USEMODULE += shell_commands line from that application in a separate PR.

In this case however the app developer did not seem to have been as frugal as they could have been. I propose to remove the USEMODULE += shell_commands line from that application in a separate PR.

Mh, when I remove that module, the test fails on the second iteration. I don't really understand why

BOARD=samr21-xpro make -C tests/periph_wdt/ test
make: Entering directory '/home/mlenders/Repositories/RIOT-OS/RIOT/tests/periph_wdt'
PASSED target time: 128[ms], actual_time: 127.648[ms]
FAILED target time: 512[ms], actual_time: 9986.882[ms]

make: *** [/home/mlenders/Repositories/RIOT-OS/RIOT/tests/periph_wdt/../../Makefile.include:735: test] Error 1
make: Leaving directory '/home/mlenders/Repositories/RIOT-OS/RIOT/tests/periph_wdt'

@fjmolinas any idea?

I'm suspecting there is something hidden happening around the test_utils_interactive_sync module which I don't expect, but I tried to add that function to main() and it still did not work :-/.

Ping @fjmolinas??!?

Ping @fjmolinas??!?

Not sure why its failing, I'll take a look. I have an idea on how to remove the need for interactive_sync when using the shell, I'll test it out.

I had added shell_commands to the test in 416c048, the commit describes why.

@miri64 I propose you just add a Makefile.ci in this PR, and I'll remove it in #13613, #13195 is running into the same issue, and #13613 needs more extensive testing and its going to take me a little bit.

If you promise to remove the file in Makefile.ci in #13613 again, I'll do it ;-).

Done.

just FYI the app_metadata has this information among other things. Though I am fine with this as well if you don't want the other noise.

make all -C test/shell

> app_metadata
{"cmd": "app_metadata_print_json()"}
{"data": {"APP_NAME": "tests_shell"}}
{"data": {"BOARD": "native"}}
{"data": {"CPU": "native"}}
{"data": {"APP_SHELL_FMT": "NONE"}}
{"data": {"MCU": "native"}}
{"data": {"OS_VERSION": "2020.04-devel-1105-g1b5f2"}}
{"result": "SUCCESS"}

I don't want to rely on an optional module, but rather have it be always compiled in most of the time automatically ;-).