-
Notifications
You must be signed in to change notification settings - Fork 20
20230929-address-bitmasks #57
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* set NM to a usable value.
* provide for invoker-supplied TEST_JSON_CFLAGS.
* add feature flags `NO_ADDR_BITMASK_MATCHING` and `NO_IPV6`.
* rename feature flag `NO_STDIO` to `NO_STDIO_STREAMS`.
* allow override for TEST_JSON_CFLAGS.
* in doc recipes, use -L flag to cp to force early failure for missing files.
Makefile.analyzers:
* rename no-stdio-build-test to no-stdio-streams-build-test.
* in minimal-threaded-build-test recipe, change -DWOLFSENTRY_USE_NONPOSIX_SEMAPHORES to -DWOLFSENTRY_NO_SEM_BUILTIN.
* add targets library-dependency-singlethreaded-build-test, library-dependency-multithreaded-build-test, no-addr-bitmask-matching-test, no-ipv6-test, linux-lwip-test-no-ipv6.
* raise WOLFSENTRY_MAX_CYCLES_PER_CALL_JSON_LOADED in benchmark-test and benchmark-singlethreaded-test recipes, to accommodate additional overhead from bitmask matching logic.
* fix default FREERTOS_TOP_FOR_TEST to have $(SRC_TOP)/ prefix.
* refactor freertos-arm32-build-test to include final link of test_lwip kernel, and check on size of kernel.
* split LWIP_SRC_LIST into LWIP_SRC_LIST_BASE, LWIP_SRC_LIST_UNIX, and LWIP_SRC_LIST_FREERTOS, add FREERTOS_SRC_LIST, and add recipes for compiling FreeRTOS sources (needed by refactored freertos-arm32-build-test).
README.md: rename NO_STDIO to NO_STDIO_STREAMS, and add docs for WOLFSENTRY_NO_STDIO_H, NO_ADDR_BITMASK_MATCHING/WOLFSENTRY_NO_ADDR_BITMASK_MATCHING, NO_IPV6/WOLFSENTRY_NO_IPV6, and WOLFSENTRY_NO_SEM_BUILTIN.
doc/json_configuration.md: add documentation and ABNF grammar for "bitmask" node in route endpoints.
ports/FreeRTOS-lwIP/include/arch/cc.h: copy inet_ntop and inet_pton definitions from ports/Linux-lwIP/include/arch/cc.h.
ports/FreeRTOS-lwIP/include/arch/sys_arch.h: add file.
ports/FreeRTOS-lwIP/include/lwipopts.h: synchronize with ports/Linux-lwIP/include/lwipopts.h.
ports/Linux-lwIP/include/lwipopts.h: allow override definition of LWIP_IPV6, and set LWIP_ICMP6 to LWIP_IPV6, to allow disabling IPv6.
wolfsentry/wolfsentry.h:
* bump version to 1.6.0.
* add WOLFSENTRY_ACTION_RES_USER0-WOLFSENTRY_ACTION_RES_USER6 to wolfsentry_action_res_t enum, add WOLFSENTRY_ACTION_RES_USER7 macro, and refactor WOLFSENTRY_ACTION_RES_USER_BASE as a macro aliased to WOLFSENTRY_ACTION_RES_USER0.
* add WOLFSENTRY_ROUTE_FLAG_REMOTE_ADDR_BITMASK and WOLFSENTRY_ROUTE_FLAG_LOCAL_ADDR_BITMASK to wolfsentry_route_flags_t.
* remove !WOLFSENTRY_NO_STDIO gate around wolfsentry_kv_render_value().
wolfsentry/wolfsentry_errcodes.h: add WOLFSENTRY_ERROR_ID_WRONG_ATTRIBUTES.
src/wolfsentry_internal.h:
* add wolfsentry_table_reset_fn_t and wolfsentry_coupled_ent_fn_t, and add reset_fn and coupled_ent_fn to struct wolfsentry_table_header.
* add last_af_wildcard_route to struct wolfsentry_route_table.
* add n_bitmask_matching_afs and bitmask_matching_afs[] to struct wolfsentry_route_table, gated on WOLFSENTRY_ADDR_BITMASK_MATCHING.
* add wolfsentry_table_cursor_set() inline implementation.
wolfsentry/wolfsentry_settings.h:
* rename WOLFSENTRY_NO_STDIO to WOLFSENTRY_NO_STDIO_STREAMS.
* rename WOLFSENTRY_HAVE_NONGNU_ATOMICS to WOLFSENTRY_NO_GNU_ATOMICS.
* add handling for WOLFSENTRY_NO_SEM_BUILTIN, WOLFSENTRY_NO_ADDR_BITMASK_MATCHING, and WOLFSENTRY_NO_IPV6.
* gate inclusion of stdio.h on !WOLFSENTRY_NO_STDIO_H, formerly !WOLFSENTRY_NO_STDIO.
* eliminate inclusion of errno.h -- now included only in source files that need it.
* fix handling for WOLFSENTRY_SEMAPHORE_INCLUDE to give it effect in all code paths (previously ignored in POSIX and FreeRTOS paths).
* add WOLFSENTRY_CONFIG_FLAG_ADDR_BITMASKS, and rename WOLFSENTRY_CONFIG_FLAG_NO_STDIO to WOLFSENTRY_CONFIG_FLAG_NO_STDIO_STREAMS.
src/addr_families.c and wolfsentry/wolfsentry_af.h:
* split WOLFSENTRY_AF_LINK into WOLFSENTRY_AF_LINK48 and WOLFSENTRY_AF_LINK64, with WOLFSENTRY_AF_LINK aliased to WOLFSENTRY_AF_LINK48.
* handle WOLFSENTRY_AF_CAN in wolfsentry_addr_family_max_addr_bits_1().
* add wolfsentry_addr_family_get_coupled_ent()
src/kv.c: remove !WOLFSENTRY_NO_STDIO gate around wolfsentry_kv_render_value().
src/routes.c:
* rename cmp_addrs() to cmp_addrs_prefixful().
* add is_bitmask_matching(), wolfsentry_bitmask_matching_upref(), wolfsentry_bitmask_matching_downref(), cmp_addrs_bitmaskful(), and addr_bitmask_match_size(), and integrate them into wolfsentry_route_key_cmp_1() and compare_match_exactness(), pivoting on WOLFSENTRY_ROUTE_FLAG_REMOTE_ADDR_BITMASK and WOLFSENTRY_ROUTE_FLAG_LOCAL_ADDR_BITMASK.
* add bitmask handling and .last_af_wildcard_route accounting to wolfsentry_route_init(), wolfsentry_route_init_by_exports(), wolfsentry_route_new(), wolfsentry_route_new_by_exports(), wolfsentry_route_insert_1(), and wolfsentry_route_delete_0().
* add bitmask handling and AF-mismatch optimization to wolfsentry_route_lookup_0().
* in wolfsentry_route_event_dispatch_0(), move update of meta.purge_after inside the mutex.
* in wolfsentry_route_get_metadata(), conditionalize use of 64 bit WOLFSENTRY_ATOMIC_LOAD() on pointer size, to avoid dependency on library implementation of __atomic_load_8().
* add WOLFSENTRY_AF_CAN support to wolfsentry_route_format_address() and wolfsentry_route_render_address().
* add bitmask support to wolfsentry_route_format_json(), wolfsentry_route_render_flags(), and wolfsentry_route_render_endpoint().
* add wolfsentry_route_table_reset().
src/wolfsentry_internal.c:
* add .last_af_wildcard_route accounting to wolfsentry_table_clone().
* fix use-after-free bug in wolfsentry_table_free_ents(), using new table->coupled_ent_fn mechanism.
src/wolfsentry_util.c: implement support for WOLFSENTRY_SEM_BUILTINS gate.
tests/test-config*.json: add several bitmask-matched routes, and add no-bitmasks and no-ipv6 variants.
tests/unittests.c:
* additional tweaks for portability to 32 bit FreeRTOS
* add FreeRTOS-specific implementations of test_lwip() and main().
* in test_json(), add wolfsentry_addr_family_handler_install(...,"my_AF2",...).
* in test_json(), add bitmask tests.
* add stub implementations for various FreeRTOS/newlib dependencies.
src/json/load_config.c:
* gate on !WOLFSENTRY_NO_ERRNO_H uses of errno to check for errors from strtol() and friends (fall back to testing for incomplete parsing of data).
* in convert_sockaddr_address()
* add separate handling for WOLFSENTRY_AF_LINK48 and WOLFSENTRY_AF_LINK64.
* add handling for WOLFSENTRY_AF_CAN.
* add JSON type checking -- WOLFSENTRY_AF_CAN addresses can be native JSON numbers.
* add a separate sa_addr arg, allowing the caller to designate an offset address for output (used to build bitmask addresses).
* handle sa->addr_len consistently -- don't overwrite nonzero values.
* in handle_route_endpoint_clause() add handling for bitmasks.
src/json/{centijson_dom.c,centijson_sax.c,centijson_value.c}: eliminate direct calls to heap allocator functions in WOLFSENTRY code paths, i.e. use only wolfsentry_allocator.
src/json/centijson_value.c: fix uninited-variable defect on cmp in json_value_dict_get_or_add_().
… of prefix-matched CAN addresses, fix CAN mixing of bitmask and prefix matching, and add unit test coverage.
* add handling of VERY_QUIET flag to Makefile refman.pdf recipe;
* fix PRINT_VERSION_RECIPE path quoting flub;
* refactor doc-check to build target "doc" rather than just "doc-html";
* remove \usepackage{listings} from doc/doxy-formats/pdf/wolfsentry_header.tex as pdflatex now rejects it with "Undefined control sequence./l.1 \indexentry".
…ddr_len to consider bitmask length doubling, and change check on addr to restrict to 32 bits, rather than 29 bits. tests/test-config.json and tests/test-config-numeric.json: swap order of "address" and "bitmask" in CAN set-user-3 and set-user-4 routes.
cconlon
requested changes
Oct 24, 2023
Member
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In addition to the copyright comment, I did try compiling on Mac OSX and saw this show up. But, all tests passed successfully.
wolfsentry/src/json/centijson_sax.c:224:25: error: implicit conversion loses integer precision: 'unsigned int' to 'enum (unnamed enum at /Users/chrisc/Desktop/douzzer/wolfsentry/wolfsentry/centijson_sax.h:206:5)' [-Werror,-Wimplicit-int-conversion]
parser->automaton = automaton;
~ ^~~~~~~~~
1 error generated.
gcc (clang) version on OSX:
gcc --version
Apple clang version 14.0.3 (clang-1403.0.22.14.1)
Target: x86_64-apple-darwin22.4.0
…NTRY_ROUTE_FLAG_{REMOTE,LOCAL}_ADDR_BITMASK.
src/json/centijson_sax.c: give the automaton enum a name, and change json_switch_automaton() to take an enum centijson_automaton to fix a conversion error on MacOSX clang-14.
b1f0204 to
8b1f72d
Compare
cconlon
approved these changes
Oct 24, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Makefile:NMto a usable value.TEST_JSON_CFLAGS.NO_ADDR_BITMASK_MATCHINGandNO_IPV6.NO_STDIOtoNO_STDIO_STREAMS.TEST_JSON_CFLAGS.-Lflag tocpto force early failure for missing files.Makefile.analyzers:no-stdio-build-testtono-stdio-streams-build-test.minimal-threaded-build-test recipe, change-DWOLFSENTRY_USE_NONPOSIX_SEMAPHORESto-DWOLFSENTRY_NO_SEM_BUILTIN.library-dependency-singlethreaded-build-test,library-dependency-multithreaded-build-test,no-addr-bitmask-matching-test,no-ipv6-test,linux-lwip-test-no-ipv6.WOLFSENTRY_MAX_CYCLES_PER_CALL_JSON_LOADEDinbenchmark-testandbenchmark-singlethreaded-testrecipes, to accommodate additional overhead from bitmask matching logic.FREERTOS_TOP_FOR_TESTto have$(SRC_TOP)/prefix.freertos-arm32-build-testto include final link oftest_lwip kernel, and check on size of kernel.LWIP_SRC_LISTintoLWIP_SRC_LIST_BASE,LWIP_SRC_LIST_UNIX, andLWIP_SRC_LIST_FREERTOS, addFREERTOS_SRC_LIST, and add recipes for compiling FreeRTOS sources (needed by refactoredfreertos-arm32-build-test).README.md: renameNO_STDIOtoNO_STDIO_STREAMS, and add docs forWOLFSENTRY_NO_STDIO_H,NO_ADDR_BITMASK_MATCHING/WOLFSENTRY_NO_ADDR_BITMASK_MATCHING,NO_IPV6/WOLFSENTRY_NO_IPV6, andWOLFSENTRY_NO_SEM_BUILTIN.doc/json_configuration.md: add documentation and ABNF grammar for"bitmask"node in route endpoints.ports/FreeRTOS-lwIP/include/arch/cc.h: copyinet_ntopandinet_ptondefinitions fromports/Linux-lwIP/include/arch/cc.h.ports/FreeRTOS-lwIP/include/arch/sys_arch.h: add file.ports/FreeRTOS-lwIP/include/lwipopts.h: synchronize withports/Linux-lwIP/include/lwipopts.h.ports/Linux-lwIP/include/lwipopts.h: allow override definition ofLWIP_IPV6, and setLWIP_ICMP6toLWIP_IPV6, to allow disabling IPv6.wolfsentry/wolfsentry.h:WOLFSENTRY_ACTION_RES_USER0-WOLFSENTRY_ACTION_RES_USER6towolfsentry_action_res_tenum, addWOLFSENTRY_ACTION_RES_USER7macro, and refactorWOLFSENTRY_ACTION_RES_USER_BASEas a macro aliased toWOLFSENTRY_ACTION_RES_USER0.WOLFSENTRY_ROUTE_FLAG_REMOTE_ADDR_BITMASKandWOLFSENTRY_ROUTE_FLAG_LOCAL_ADDR_BITMASKtowolfsentry_route_flags_t.WOLFSENTRY_NO_STDIOgate aroundwolfsentry_kv_render_value().wolfsentry/wolfsentry_errcodes.h: addWOLFSENTRY_ERROR_ID_WRONG_ATTRIBUTES.src/wolfsentry_internal.h:wolfsentry_table_reset_fn_tandwolfsentry_coupled_ent_fn_t, and addreset_fnandcoupled_ent_fntostruct wolfsentry_table_header.last_af_wildcard_routetostruct wolfsentry_route_table.n_bitmask_matching_afsandbitmask_matching_afs[]to structwolfsentry_route_table, gated onWOLFSENTRY_ADDR_BITMASK_MATCHING.wolfsentry_table_cursor_set()inline implementation.wolfsentry/wolfsentry_settings.h:WOLFSENTRY_NO_STDIOtoWOLFSENTRY_NO_STDIO_STREAMS.WOLFSENTRY_HAVE_NONGNU_ATOMICStoWOLFSENTRY_NO_GNU_ATOMICS.WOLFSENTRY_NO_SEM_BUILTIN,WOLFSENTRY_NO_ADDR_BITMASK_MATCHING, andWOLFSENTRY_NO_IPV6.stdio.hon !WOLFSENTRY_NO_STDIO_H, formerly !WOLFSENTRY_NO_STDIO.errno.h-- now included only in source files that need it.WOLFSENTRY_SEMAPHORE_INCLUDEto give it effect in all code paths (previously ignored in POSIX and FreeRTOS paths).WOLFSENTRY_CONFIG_FLAG_ADDR_BITMASKS, and renameWOLFSENTRY_CONFIG_FLAG_NO_STDIOtoWOLFSENTRY_CONFIG_FLAG_NO_STDIO_STREAMS.src/addr_families.candwolfsentry/wolfsentry_af.h:WOLFSENTRY_AF_LINKintoWOLFSENTRY_AF_LINK48andWOLFSENTRY_AF_LINK64, withWOLFSENTRY_AF_LINKaliased toWOLFSENTRY_AF_LINK48.WOLFSENTRY_AF_CANinwolfsentry_addr_family_max_addr_bits_1().wolfsentry_addr_family_get_coupled_ent().src/kv.c: remove !WOLFSENTRY_NO_STDIOgate aroundwolfsentry_kv_render_value().src/routes.c:cmp_addrs()tocmp_addrs_prefixful().is_bitmask_matching(),wolfsentry_bitmask_matching_upref(),wolfsentry_bitmask_matching_downref(),cmp_addrs_bitmaskful(), andaddr_bitmask_match_size(), and integrate them intowolfsentry_route_key_cmp_1()andcompare_match_exactness(), pivoting onWOLFSENTRY_ROUTE_FLAG_REMOTE_ADDR_BITMASKandWOLFSENTRY_ROUTE_FLAG_LOCAL_ADDR_BITMASK..last_af_wildcard_routeaccounting towolfsentry_route_init(),wolfsentry_route_init_by_exports(),wolfsentry_route_new(),wolfsentry_route_new_by_exports(),wolfsentry_route_insert_1(), andwolfsentry_route_delete_0().wolfsentry_route_lookup_0().wolfsentry_route_event_dispatch_0(), move update ofmeta.purge_afterinside the mutex.wolfsentry_route_get_metadata(), conditionalize use of 64 bitWOLFSENTRY_ATOMIC_LOAD()on pointer size, to avoid dependency on library implementation of__atomic_load_8().WOLFSENTRY_AF_CANsupport towolfsentry_route_format_address()andwolfsentry_route_render_address().wolfsentry_route_format_json(),wolfsentry_route_render_flags(), andwolfsentry_route_render_endpoint().wolfsentry_route_table_reset().src/wolfsentry_internal.c:.last_af_wildcard_routeaccounting towolfsentry_table_clone().wolfsentry_table_free_ents(), using newtable->coupled_ent_fnmechanism.src/wolfsentry_util.c: implement support forWOLFSENTRY_SEM_BUILTINSgate.tests/test-config*.json: add several bitmask-matched routes, and add no-bitmasks and no-ipv6 variants.tests/unittests.c:test_lwip()andmain().test_json(), addwolfsentry_addr_family_handler_install(...,"my_AF2",...).test_json(), add bitmask tests.src/json/load_config.c:WOLFSENTRY_NO_ERRNO_Huses oferrnoto check for errors fromstrtol()and friends (fall back to testing for incomplete parsing of data).convert_sockaddr_address()WOLFSENTRY_AF_LINK48andWOLFSENTRY_AF_LINK64.WOLFSENTRY_AF_CAN.WOLFSENTRY_AF_CANaddresses can be native JSON numbers.sa_addrarg, allowing the caller to designate an offset address for output (used to build bitmask addresses).sa->addr_lenconsistently -- don't overwrite nonzero values.handle_route_endpoint_clause()add handling for bitmasks.src/json/{centijson_dom.c,centijson_sax.c,centijson_value.c}: eliminate direct calls to heap allocator functions inWOLFSENTRYcode paths, i.e. use onlywolfsentry_allocator.src/json/centijson_value.c: fix uninited-variable defect oncmpinjson_value_dict_get_or_add_().