Skip to content

Conversation

@douzzer
Copy link
Collaborator

@douzzer douzzer commented Oct 20, 2023

Makefile:

  • set NM to a usable value.
  • provide for invoker-supplied TEST_JSON_CFLAGS.
  • add feature flags NO_ADDR_BITMASK_MATCHING and NO_IPV6.
  • rename feature flag NO_STDIO to NO_STDIO_STREAMS.
  • allow override for TEST_JSON_CFLAGS.
  • in doc recipes, use -L flag to cp to force early failure for missing files.

Makefile.analyzers:

  • rename no-stdio-build-test to no-stdio-streams-build-test.
  • in minimal-threaded-build-test recipe, change -DWOLFSENTRY_USE_NONPOSIX_SEMAPHORES to -DWOLFSENTRY_NO_SEM_BUILTIN.
  • add targets library-dependency-singlethreaded-build-test, library-dependency-multithreaded-build-test, no-addr-bitmask-matching-test, no-ipv6-test, linux-lwip-test-no-ipv6.
  • raise WOLFSENTRY_MAX_CYCLES_PER_CALL_JSON_LOADED in benchmark-test and benchmark-singlethreaded-test recipes, to accommodate additional overhead from bitmask matching logic.
  • fix default FREERTOS_TOP_FOR_TEST to have $(SRC_TOP)/ prefix.
  • refactor freertos-arm32-build-test to include final link of test_lwip kernel, and check on size of kernel.
  • split LWIP_SRC_LIST into LWIP_SRC_LIST_BASE, LWIP_SRC_LIST_UNIX, and LWIP_SRC_LIST_FREERTOS, add FREERTOS_SRC_LIST, and add recipes for compiling FreeRTOS sources (needed by refactored freertos-arm32-build-test).

README.md: rename NO_STDIO to NO_STDIO_STREAMS, and add docs for WOLFSENTRY_NO_STDIO_H, NO_ADDR_BITMASK_MATCHING/WOLFSENTRY_NO_ADDR_BITMASK_MATCHING, NO_IPV6/WOLFSENTRY_NO_IPV6, and WOLFSENTRY_NO_SEM_BUILTIN.

doc/json_configuration.md: add documentation and ABNF grammar for "bitmask" node in route endpoints.

ports/FreeRTOS-lwIP/include/arch/cc.h: copy inet_ntop and inet_pton definitions from ports/Linux-lwIP/include/arch/cc.h.

ports/FreeRTOS-lwIP/include/arch/sys_arch.h: add file.

ports/FreeRTOS-lwIP/include/lwipopts.h: synchronize with ports/Linux-lwIP/include/lwipopts.h.

ports/Linux-lwIP/include/lwipopts.h: allow override definition of LWIP_IPV6, and set LWIP_ICMP6 to LWIP_IPV6, to allow disabling IPv6.

wolfsentry/wolfsentry.h:

  • bump version to 1.6.0.
  • add WOLFSENTRY_ACTION_RES_USER0-WOLFSENTRY_ACTION_RES_USER6 to wolfsentry_action_res_t enum, add WOLFSENTRY_ACTION_RES_USER7 macro, and refactor WOLFSENTRY_ACTION_RES_USER_BASE as a macro aliased to WOLFSENTRY_ACTION_RES_USER0.
  • add WOLFSENTRY_ROUTE_FLAG_REMOTE_ADDR_BITMASK and WOLFSENTRY_ROUTE_FLAG_LOCAL_ADDR_BITMASK to wolfsentry_route_flags_t.
  • remove !WOLFSENTRY_NO_STDIO gate around wolfsentry_kv_render_value().

wolfsentry/wolfsentry_errcodes.h: add WOLFSENTRY_ERROR_ID_WRONG_ATTRIBUTES.

src/wolfsentry_internal.h:

  • add wolfsentry_table_reset_fn_t and wolfsentry_coupled_ent_fn_t, and add reset_fn and coupled_ent_fn to struct wolfsentry_table_header.
  • add last_af_wildcard_route to struct wolfsentry_route_table.
  • add n_bitmask_matching_afs and bitmask_matching_afs[] to struct wolfsentry_route_table, gated on WOLFSENTRY_ADDR_BITMASK_MATCHING.
  • add wolfsentry_table_cursor_set() inline implementation.

wolfsentry/wolfsentry_settings.h:

  • rename WOLFSENTRY_NO_STDIO to WOLFSENTRY_NO_STDIO_STREAMS.
  • rename WOLFSENTRY_HAVE_NONGNU_ATOMICS to WOLFSENTRY_NO_GNU_ATOMICS.
  • add handling for WOLFSENTRY_NO_SEM_BUILTIN, WOLFSENTRY_NO_ADDR_BITMASK_MATCHING, and WOLFSENTRY_NO_IPV6.
  • gate inclusion of stdio.h on !WOLFSENTRY_NO_STDIO_H, formerly !WOLFSENTRY_NO_STDIO.
  • eliminate inclusion of errno.h -- now included only in source files that need it.
  • fix handling for WOLFSENTRY_SEMAPHORE_INCLUDE to give it effect in all code paths (previously ignored in POSIX and FreeRTOS paths).
  • add WOLFSENTRY_CONFIG_FLAG_ADDR_BITMASKS, and rename WOLFSENTRY_CONFIG_FLAG_NO_STDIO to WOLFSENTRY_CONFIG_FLAG_NO_STDIO_STREAMS.

src/addr_families.c and wolfsentry/wolfsentry_af.h:

  • split WOLFSENTRY_AF_LINK into WOLFSENTRY_AF_LINK48 and WOLFSENTRY_AF_LINK64, with WOLFSENTRY_AF_LINK aliased to WOLFSENTRY_AF_LINK48.
  • handle WOLFSENTRY_AF_CAN in wolfsentry_addr_family_max_addr_bits_1().
  • add wolfsentry_addr_family_get_coupled_ent().

src/kv.c: remove !WOLFSENTRY_NO_STDIO gate around wolfsentry_kv_render_value().

src/routes.c:

  • rename cmp_addrs() to cmp_addrs_prefixful().
  • add is_bitmask_matching(), wolfsentry_bitmask_matching_upref(), wolfsentry_bitmask_matching_downref(), cmp_addrs_bitmaskful(), and addr_bitmask_match_size(), and integrate them into wolfsentry_route_key_cmp_1() and compare_match_exactness(), pivoting on WOLFSENTRY_ROUTE_FLAG_REMOTE_ADDR_BITMASK and WOLFSENTRY_ROUTE_FLAG_LOCAL_ADDR_BITMASK.
  • add bitmask handling and .last_af_wildcard_route accounting to wolfsentry_route_init(), wolfsentry_route_init_by_exports(), wolfsentry_route_new(), wolfsentry_route_new_by_exports(), wolfsentry_route_insert_1(), and wolfsentry_route_delete_0().
  • add bitmask handling and AF-mismatch optimization to wolfsentry_route_lookup_0().
  • in wolfsentry_route_event_dispatch_0(), move update of meta.purge_after inside the mutex.
  • in wolfsentry_route_get_metadata(), conditionalize use of 64 bit WOLFSENTRY_ATOMIC_LOAD() on pointer size, to avoid dependency on library implementation of __atomic_load_8().
  • add WOLFSENTRY_AF_CAN support to wolfsentry_route_format_address() and wolfsentry_route_render_address().
  • add bitmask support to wolfsentry_route_format_json(), wolfsentry_route_render_flags(), and wolfsentry_route_render_endpoint().
  • add wolfsentry_route_table_reset().

src/wolfsentry_internal.c:

  • add .last_af_wildcard_route accounting to wolfsentry_table_clone().
  • fix use-after-free bug in wolfsentry_table_free_ents(), using new table->coupled_ent_fn mechanism.

src/wolfsentry_util.c: implement support for WOLFSENTRY_SEM_BUILTINS gate.

tests/test-config*.json: add several bitmask-matched routes, and add no-bitmasks and no-ipv6 variants.

tests/unittests.c:

  • additional tweaks for portability to 32 bit FreeRTOS
  • add FreeRTOS-specific implementations of test_lwip() and main().
  • in test_json(), add wolfsentry_addr_family_handler_install(...,"my_AF2",...).
  • in test_json(), add bitmask tests.
  • add stub implementations for various FreeRTOS/newlib dependencies.

src/json/load_config.c:

  • gate on !WOLFSENTRY_NO_ERRNO_H uses of errno to check for errors from strtol() and friends (fall back to testing for incomplete parsing of data).
  • in convert_sockaddr_address()
    • add separate handling for WOLFSENTRY_AF_LINK48 and WOLFSENTRY_AF_LINK64.
    • add handling for WOLFSENTRY_AF_CAN.
    • add JSON type checking -- WOLFSENTRY_AF_CAN addresses can be native JSON numbers.
    • add a separate sa_addr arg, allowing the caller to designate an offset address for output (used to build bitmask addresses).
    • handle sa->addr_len consistently -- don't overwrite nonzero values.
  • in handle_route_endpoint_clause() add handling for bitmasks.

src/json/{centijson_dom.c,centijson_sax.c,centijson_value.c}: eliminate direct calls to heap allocator functions in WOLFSENTRY code paths, i.e. use only wolfsentry_allocator.

src/json/centijson_value.c: fix uninited-variable defect on cmp in json_value_dict_get_or_add_().

* set NM to a usable value.
* provide for invoker-supplied TEST_JSON_CFLAGS.
* add feature flags `NO_ADDR_BITMASK_MATCHING` and `NO_IPV6`.
* rename feature flag `NO_STDIO` to `NO_STDIO_STREAMS`.
* allow override for TEST_JSON_CFLAGS.
* in doc recipes, use -L flag to cp to force early failure for missing files.

Makefile.analyzers:
* rename no-stdio-build-test to no-stdio-streams-build-test.
* in minimal-threaded-build-test recipe, change -DWOLFSENTRY_USE_NONPOSIX_SEMAPHORES to -DWOLFSENTRY_NO_SEM_BUILTIN.
* add targets library-dependency-singlethreaded-build-test, library-dependency-multithreaded-build-test, no-addr-bitmask-matching-test, no-ipv6-test, linux-lwip-test-no-ipv6.
* raise WOLFSENTRY_MAX_CYCLES_PER_CALL_JSON_LOADED in benchmark-test and benchmark-singlethreaded-test recipes, to accommodate additional overhead from bitmask matching logic.
* fix default FREERTOS_TOP_FOR_TEST to have $(SRC_TOP)/ prefix.
* refactor freertos-arm32-build-test to include final link of test_lwip kernel, and check on size of kernel.
* split LWIP_SRC_LIST into LWIP_SRC_LIST_BASE, LWIP_SRC_LIST_UNIX, and LWIP_SRC_LIST_FREERTOS, add FREERTOS_SRC_LIST, and add recipes for compiling FreeRTOS sources (needed by refactored freertos-arm32-build-test).

README.md: rename NO_STDIO to NO_STDIO_STREAMS, and add docs for WOLFSENTRY_NO_STDIO_H, NO_ADDR_BITMASK_MATCHING/WOLFSENTRY_NO_ADDR_BITMASK_MATCHING, NO_IPV6/WOLFSENTRY_NO_IPV6, and WOLFSENTRY_NO_SEM_BUILTIN.

doc/json_configuration.md: add documentation and ABNF grammar for "bitmask" node in route endpoints.

ports/FreeRTOS-lwIP/include/arch/cc.h: copy inet_ntop and inet_pton definitions from ports/Linux-lwIP/include/arch/cc.h.

ports/FreeRTOS-lwIP/include/arch/sys_arch.h: add file.

ports/FreeRTOS-lwIP/include/lwipopts.h: synchronize with ports/Linux-lwIP/include/lwipopts.h.

ports/Linux-lwIP/include/lwipopts.h: allow override definition of LWIP_IPV6, and set LWIP_ICMP6 to LWIP_IPV6, to allow disabling IPv6.

wolfsentry/wolfsentry.h:
* bump version to 1.6.0.
* add WOLFSENTRY_ACTION_RES_USER0-WOLFSENTRY_ACTION_RES_USER6 to wolfsentry_action_res_t enum, add WOLFSENTRY_ACTION_RES_USER7 macro, and refactor WOLFSENTRY_ACTION_RES_USER_BASE as a macro aliased to WOLFSENTRY_ACTION_RES_USER0.
* add WOLFSENTRY_ROUTE_FLAG_REMOTE_ADDR_BITMASK and WOLFSENTRY_ROUTE_FLAG_LOCAL_ADDR_BITMASK to wolfsentry_route_flags_t.
* remove !WOLFSENTRY_NO_STDIO gate around wolfsentry_kv_render_value().

wolfsentry/wolfsentry_errcodes.h: add WOLFSENTRY_ERROR_ID_WRONG_ATTRIBUTES.

src/wolfsentry_internal.h:
* add wolfsentry_table_reset_fn_t and wolfsentry_coupled_ent_fn_t, and add reset_fn and coupled_ent_fn to struct wolfsentry_table_header.
* add last_af_wildcard_route to struct wolfsentry_route_table.
* add n_bitmask_matching_afs and bitmask_matching_afs[] to struct wolfsentry_route_table, gated on WOLFSENTRY_ADDR_BITMASK_MATCHING.
* add wolfsentry_table_cursor_set() inline implementation.

wolfsentry/wolfsentry_settings.h:
* rename WOLFSENTRY_NO_STDIO to WOLFSENTRY_NO_STDIO_STREAMS.
* rename WOLFSENTRY_HAVE_NONGNU_ATOMICS to WOLFSENTRY_NO_GNU_ATOMICS.
* add handling for WOLFSENTRY_NO_SEM_BUILTIN, WOLFSENTRY_NO_ADDR_BITMASK_MATCHING, and WOLFSENTRY_NO_IPV6.
* gate inclusion of stdio.h on !WOLFSENTRY_NO_STDIO_H, formerly !WOLFSENTRY_NO_STDIO.
* eliminate inclusion of errno.h -- now included only in source files that need it.
* fix handling for WOLFSENTRY_SEMAPHORE_INCLUDE to give it effect in all code paths (previously ignored in POSIX and FreeRTOS paths).
* add WOLFSENTRY_CONFIG_FLAG_ADDR_BITMASKS, and rename WOLFSENTRY_CONFIG_FLAG_NO_STDIO to WOLFSENTRY_CONFIG_FLAG_NO_STDIO_STREAMS.

src/addr_families.c and wolfsentry/wolfsentry_af.h:
* split WOLFSENTRY_AF_LINK into WOLFSENTRY_AF_LINK48 and WOLFSENTRY_AF_LINK64, with WOLFSENTRY_AF_LINK aliased to WOLFSENTRY_AF_LINK48.
* handle WOLFSENTRY_AF_CAN in wolfsentry_addr_family_max_addr_bits_1().
* add wolfsentry_addr_family_get_coupled_ent()

src/kv.c: remove !WOLFSENTRY_NO_STDIO gate around wolfsentry_kv_render_value().

src/routes.c:
* rename cmp_addrs() to cmp_addrs_prefixful().
* add is_bitmask_matching(), wolfsentry_bitmask_matching_upref(), wolfsentry_bitmask_matching_downref(), cmp_addrs_bitmaskful(), and addr_bitmask_match_size(), and integrate them into wolfsentry_route_key_cmp_1() and compare_match_exactness(), pivoting on WOLFSENTRY_ROUTE_FLAG_REMOTE_ADDR_BITMASK and WOLFSENTRY_ROUTE_FLAG_LOCAL_ADDR_BITMASK.
* add bitmask handling and .last_af_wildcard_route accounting to wolfsentry_route_init(), wolfsentry_route_init_by_exports(), wolfsentry_route_new(), wolfsentry_route_new_by_exports(), wolfsentry_route_insert_1(), and wolfsentry_route_delete_0().
* add bitmask handling and AF-mismatch optimization to wolfsentry_route_lookup_0().
* in wolfsentry_route_event_dispatch_0(), move update of meta.purge_after inside the mutex.
* in wolfsentry_route_get_metadata(), conditionalize use of 64 bit WOLFSENTRY_ATOMIC_LOAD() on pointer size, to avoid dependency on library implementation of __atomic_load_8().
* add WOLFSENTRY_AF_CAN support to wolfsentry_route_format_address() and wolfsentry_route_render_address().
* add bitmask support to wolfsentry_route_format_json(), wolfsentry_route_render_flags(), and wolfsentry_route_render_endpoint().
* add wolfsentry_route_table_reset().

src/wolfsentry_internal.c:
* add .last_af_wildcard_route accounting to wolfsentry_table_clone().
* fix use-after-free bug in wolfsentry_table_free_ents(), using new table->coupled_ent_fn mechanism.

src/wolfsentry_util.c: implement support for WOLFSENTRY_SEM_BUILTINS gate.

tests/test-config*.json: add several bitmask-matched routes, and add no-bitmasks and no-ipv6 variants.

tests/unittests.c:
* additional tweaks for portability to 32 bit FreeRTOS
* add FreeRTOS-specific implementations of test_lwip() and main().
* in test_json(), add wolfsentry_addr_family_handler_install(...,"my_AF2",...).
* in test_json(), add bitmask tests.
* add stub implementations for various FreeRTOS/newlib dependencies.

src/json/load_config.c:
* gate on !WOLFSENTRY_NO_ERRNO_H uses of errno to check for errors from strtol() and friends (fall back to testing for incomplete parsing of data).
* in convert_sockaddr_address()
  * add separate handling for WOLFSENTRY_AF_LINK48 and WOLFSENTRY_AF_LINK64.
  * add handling for WOLFSENTRY_AF_CAN.
  * add JSON type checking -- WOLFSENTRY_AF_CAN addresses can be native JSON numbers.
  * add a separate sa_addr arg, allowing the caller to designate an offset address for output (used to build bitmask addresses).
  * handle sa->addr_len consistently -- don't overwrite nonzero values.
* in handle_route_endpoint_clause() add handling for bitmasks.

src/json/{centijson_dom.c,centijson_sax.c,centijson_value.c}: eliminate direct calls to heap allocator functions in WOLFSENTRY code paths, i.e. use only wolfsentry_allocator.

src/json/centijson_value.c: fix uninited-variable defect on cmp in json_value_dict_get_or_add_().
… of prefix-matched CAN addresses, fix CAN mixing of bitmask and prefix matching, and add unit test coverage.
* add handling of VERY_QUIET flag to Makefile refman.pdf recipe;
* fix PRINT_VERSION_RECIPE path quoting flub;
* refactor doc-check to build target "doc" rather than just "doc-html";
* remove \usepackage{listings} from doc/doxy-formats/pdf/wolfsentry_header.tex as pdflatex now rejects it with "Undefined control sequence./l.1 \indexentry".
…ddr_len to consider bitmask length doubling, and change check on addr to restrict to 32 bits, rather than 29 bits.

tests/test-config.json and tests/test-config-numeric.json: swap order of "address" and "bitmask" in CAN set-user-3 and set-user-4 routes.
Copy link
Member

@cconlon cconlon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In addition to the copyright comment, I did try compiling on Mac OSX and saw this show up. But, all tests passed successfully.

wolfsentry/src/json/centijson_sax.c:224:25: error: implicit conversion loses integer precision: 'unsigned int' to 'enum (unnamed enum at /Users/chrisc/Desktop/douzzer/wolfsentry/wolfsentry/centijson_sax.h:206:5)' [-Werror,-Wimplicit-int-conversion]
    parser->automaton = automaton;
                      ~ ^~~~~~~~~
1 error generated.

gcc (clang) version on OSX:

gcc --version
Apple clang version 14.0.3 (clang-1403.0.22.14.1)
Target: x86_64-apple-darwin22.4.0

@cconlon cconlon assigned douzzer and unassigned wolfSSL-Bot Oct 24, 2023
…NTRY_ROUTE_FLAG_{REMOTE,LOCAL}_ADDR_BITMASK.

src/json/centijson_sax.c: give the automaton enum a name, and change json_switch_automaton() to take an enum centijson_automaton to fix a conversion error on MacOSX clang-14.
@douzzer douzzer force-pushed the 20230929-address-bitmasks branch from b1f0204 to 8b1f72d Compare October 24, 2023 21:06
@douzzer douzzer requested a review from cconlon October 24, 2023 21:06
@cconlon cconlon merged commit 8e698e0 into wolfSSL:master Oct 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants