Skip to content

Conversation

@jepler
Copy link
Collaborator

@jepler jepler commented Apr 29, 2021

FLAG_ASYNC must be within FLAG_ALL_SIG, so it does need to be renumbered down to the 0x10 position, moving the other flags up one bit. This has to be done once in C and once in Python.

FLAG_VIPERRET_POS must be shifted up 1, since (1<<7) is the next bit after FLAG_HASCONSTS at 0x40.

This affects the encoding of scope_flags, causing them to spread over an additional byte in the flexible-length integer encoding. Everything except the hard-coded mpy data in mpy_native.py accounts for this automatically.

The fancy prefix-encoding of the function signature handles this, though as each extension byte only carries a single flag bit, it takes 5 more bytes to specify an async function than a bog-standard one. Oh well.

tannewt and others added 3 commits April 28, 2021 15:00
FLAG_ASYNC must be within FLAG_ALL_SIG, so it _does_ need to be
renumbered down to the 0x10 position, moving the other flags up
one bit.

FLAG_VIPERRET_POS must be shifted up 1, since (1<<7) is the next bit
after FLAG_HASCONSTS at 0x40.  This has to be done once in C and once
in Python.

This affects the encoding of scope_flags, causing them to spread over
an additional byte in the flexible-length integer encoding.  Everything
except the hard-coded mpy data in mpy_native.py accounts for this
automatically.

The fancy prefix-encoding of the function signature handles this,
though as each extension _byte_ only carries a single _flag bit_,
it takes 5 more bytes to specify an async function than a bog-standard
one.  Oh well.
@tannewt tannewt force-pushed the merge_1.12 branch 6 times, most recently from 3944177 to b35fa44 Compare May 3, 2021 21:01
@jepler jepler closed this May 5, 2021
tannewt pushed a commit that referenced this pull request Jun 24, 2021
asan considers that memcmp(p, q, N) is permitted to access N bytes at each
of p and q, even for values of p and q that have a difference earlier.
Accessing additional values is frequently done in practice, reading 4 or
more bytes from each input at a time for efficiency, so when completing
"non_exist<TAB>" in the repl, this causes a diagnostic:

    ==16938==ERROR: AddressSanitizer: global-buffer-overflow on
    address 0x555555cd8dc8 at pc 0x7ffff726457b bp 0x7fffffffda20 sp 0x7fff
    READ of size 9 at 0x555555cd8dc8 thread T0
        #0 0x7ffff726457a  (/usr/lib/x86_64-linux-gnu/libasan.so.5+0xb857a)
        #1 0x555555b0e82a in mp_repl_autocomplete ../../py/repl.c:301
        #2 0x555555c89585 in readline_process_char ../../lib/mp-readline/re
        #3 0x555555c8ac6e in readline ../../lib/mp-readline/readline.c:513
        #4 0x555555b8dcbd in do_repl /home/jepler/src/micropython/ports/uni
        #5 0x555555b90859 in main_ /home/jepler/src/micropython/ports/unix/
        #6 0x555555b90a3a in main /home/jepler/src/micropython/ports/unix/m
        #7 0x7ffff619a09a in __libc_start_main ../csu/libc-start.c:308
        #8 0x55555595fd69 in _start (/home/jepler/src/micropython/ports/uni

    0x555555cd8dc8 is located 0 bytes to the right of global variable
    'import_str' defined in '../../py/repl.c:285:23' (0x555555cd8dc0) of
    size 8
      'import_str' is ascii string 'import '

Signed-off-by: Jeff Epler <jepler@gmail.com>
tannewt pushed a commit that referenced this pull request Sep 16, 2024
Although the original motivation given for the workaround[1] is correct,
nlr.o and nlrthumb.o are linked with a small enough distance that the
problem does not occur, and the workaround isn't necessary. The distance
between the b instruction and its target (nlr_push_tail) is just 64
bytes[2], well within the ±2046 byte range addressable by an
unconditional branch instruction in Thumb mode.

The workaround induces a relocation in the text section (textrel), which
isn't supported everywhere, notably not on musl-libc[3], where it causes
a crash on start-up. With the workaround removed, micropython works on an
ARMv5T Linux system built with musl-libc.

This commit changes nlrthumb.c to use a direct jump by default, but
leaves the long jump workaround as an option for those cases where it's
actually needed.

[1]: commit dd376a2

Author: Damien George <damien.p.george@gmail.com>
Date:   Fri Sep 1 15:25:29 2017 +1000

    py/nlrthumb: Get working again on standard Thumb arch (ie not Thumb2).

    "b" on Thumb might not be long enough for the jump to nlr_push_tail so
    it must be done indirectly.

[2]: Excerpt from objdump -d micropython:

000095c4 <nlr_push_tail>:
    95c4:       b510            push    {r4, lr}
    95c6:       0004            movs    r4, r0
    95c8:       f02d fd42       bl      37050 <mp_thread_get_state>
    95cc:       6943            ldr     r3, [r0, #20]
    95ce:       6023            str     r3, [r4, #0]
    95d0:       6144            str     r4, [r0, #20]
    95d2:       2000            movs    r0, #0
    95d4:       bd10            pop     {r4, pc}

000095d6 <nlr_pop>:
    95d6:       b510            push    {r4, lr}
    95d8:       f02d fd3a       bl      37050 <mp_thread_get_state>
    95dc:       6943            ldr     r3, [r0, #20]
    95de:       681b            ldr     r3, [r3, #0]
    95e0:       6143            str     r3, [r0, #20]
    95e2:       bd10            pop     {r4, pc}

000095e4 <nlr_push>:
    95e4:       60c4            str     r4, [r0, #12]
    95e6:       6105            str     r5, [r0, #16]
    95e8:       6146            str     r6, [r0, #20]
    95ea:       6187            str     r7, [r0, micropython#24]
    95ec:       4641            mov     r1, r8
    95ee:       61c1            str     r1, [r0, micropython#28]
    95f0:       4649            mov     r1, r9
    95f2:       6201            str     r1, [r0, micropython#32]
    95f4:       4651            mov     r1, sl
    95f6:       6241            str     r1, [r0, micropython#36]   @ 0x24
    95f8:       4659            mov     r1, fp
    95fa:       6281            str     r1, [r0, micropython#40]   @ 0x28
    95fc:       4669            mov     r1, sp
    95fe:       62c1            str     r1, [r0, micropython#44]   @ 0x2c
    9600:       4671            mov     r1, lr
    9602:       6081            str     r1, [r0, #8]
    9604:       e7de            b.n     95c4 <nlr_push_tail>

[3]: https://www.openwall.com/lists/musl/2020/09/25/4

Signed-off-by: J. Neuschäfer <j.ne@posteo.net>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants