Skip to content

Issue in stack.c with AIX xlc compiler with optimization level 2 (-O2) (branch 1.1.1) #10624

@LukasKust

Description

@LukasKust

I'm building openssl on AIX using xlc and found an issue in the function OPENSSL_sk_dup() in stack.c when compiling with optimization level 2 or higher (-O2).
The issue is that the direct struct assignment gets removed by the compiler (probably a compiler bug)

OPENSSL_STACK *OPENSSL_sk_dup(const OPENSSL_STACK *sk)
{
    OPENSSL_STACK *ret;

    if ((ret = OPENSSL_malloc(sizeof(*ret))) == NULL) {
        CRYPTOerr(CRYPTO_F_OPENSSL_SK_DUP, ERR_R_MALLOC_FAILURE);
        return NULL;
    }

    /* direct structure assignment */
    *ret = *sk;      /* <---- This line gets removed with -O2 */

    if (sk->num == 0) {
        /* postpone |ret->data| allocation */
        ret->data = NULL;
        ret->num_alloc = 0;
        return ret;
    }
    /* duplicate |sk->data| content */
    if ((ret->data = OPENSSL_malloc(sizeof(*ret->data) * sk->num_alloc)) == NULL)
        goto err;
    memcpy(ret->data, sk->data, sizeof(void *) * sk->num);
    return ret;
 err:
    OPENSSL_sk_free(ret);
    return NULL;
}

I did some testing and created a simple test program. When the program is compiled on AIX with xlc -O2, the program output will be "Copy failed!!".
The program can be fixed by moving the function wrapper_get_num() below the definition of "struct number" or by replacing the structure assignment by a memcopy or building with -O0.

The function "OPENSSL_sk_dup" in stack.c suffer from the same issue as the function "dup" in this test program.

Is this issue known by openssl and how would you suggest to fix it?
For now I replaced the assignment with memcpy, but there might be better options. Also I don't like to build openssl with custom patches if it can be avoided.

simple test program:

#include <stdlib.h>
#include <stdio.h>

int get_num(struct number * st) ;

int wrapper_get_num(void  * st) { 
    return get_num(st); 
}

struct number
{
    int num;
};

/* If the function wrapper_get_num(void *st) is moved here, then the issue disappears */

struct number *dup(const struct number *a)
{

    struct number *b;

    b = malloc(sizeof(*b));

    /* Assignment gets removed by xlc compiler with -O2 */
    *b = *a;

    return b;
}

int get_num(struct number * st){
    return st->num;
}

int main (void) {

    struct number *a;
    struct number *b;

    a = malloc(sizeof(*a));
    a->num = 1;

    b = dup(a);
    if (a->num != b->num) {
        printf("Copy failed!!\n");
        return 1;
    }
    printf("Copy worked!!\n");
    return 0;

}

Compile with:

/usr/vac/bin/xlc_r  -qpic -q64 -qmaxmem=16384 -qro -qroconst -qthreaded -O2 -o test.out test.c

xlc version I used:

>/usr/vac/bin/xlc_r -qVersion
IBM XL C/C++ for AIX, V10.1
Version: 10.01.0000.0000

Metadata

Metadata

Assignees

No one assigned

    Labels

    resolved: wont fixThe issue has been confirmed but won't be fixedtriaged: bugThe issue/pr is/fixes a bug

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions