Skip to content

update rcutils_get_env to always use getenv#237

Merged
clalancette merged 3 commits intoros2:masterfrom
suyashb95:threadsafe-get-env
Apr 27, 2020
Merged

update rcutils_get_env to always use getenv#237
clalancette merged 3 commits intoros2:masterfrom
suyashb95:threadsafe-get-env

Conversation

@suyashb95
Copy link
Copy Markdown
Contributor

@suyashb95 suyashb95 commented Apr 14, 2020

fixes #30
Signed-off-by: Suyash458 suyash.behera458@gmail.com

Signed-off-by: Suyash458 <suyash.behera458@gmail.com>
src/get_env.c Outdated
*env_value = NULL;
#ifdef _WIN32
size_t required_size;
char * __env_buffer = (char*) malloc(WINDOWS_ENV_BUFFER_SIZE * sizeof(char));
Copy link
Copy Markdown
Contributor

@clalancette clalancette Apr 14, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I think this is the wrong way to go. The problem is that on Linux/macOS, this would return a string that the caller does not free, and on Windows it would return a string that the caller does free. That's much too difficult to get right, so we should make it consistent.

But let's back up a second. There are three levels of thread-safety that are currently being violated by this current version of rcutils_get_env. The first level is that, on Windows, we are returning a pointer to a global array, so if two threads try to get an environment variable at the same time, they'll stomp all over each other. The second level is that, on all platforms, if a thread gets an environment variable, and then somewhere down the line something else calls setenv, that can cause the pointer to be invalidated. The third level of thread safety being violated is that if something calls {un}setenv at the same time that something else is calling getenv, a race and/or crash can occur.

With that being said, I propose that we split this into three different patches:

  1. We remove the global Windows buffer completely, and switch from getenv_s to getenv on Windows. That will make the behavior consistent between Linux, macOS, and Windows, and won't cause us to change this function's signature. This doesn't cause us to be any less thread-safe then we are on Linux and macOS now, and improves thread safety for Windows. (I'll suggest we just modify this simple PR to accomplish this, but we can open a new one as well).
  2. Later do a follow-up where we getenv and copy the string for all platforms. This will solve the second thread-safety issue, since a later setenv won't affect outstanding pointers. That will cause us to change the signature and the contract of rcutils_get_env so we can have an allocator, and so downstream users free the buffer we give them.
  3. Even later, do an audit of any places that call rcutils_get_env and setenv, and add locking as appropriate.

Doing it this way addresses our most pressing thread-safety issue (number 1), while leaving the less likely ones to follow-up later.

Does this make sense? @hidmic @ivanpauno @wjwwood thoughts?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That sounds right to me.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@clalancette just wanted to point out that getenv is deprecated so, should we just go ahead with 2 and 3 instead?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@clalancette just wanted to point out that getenv is deprecated so, should we just go ahead with 2 and 3 instead?

Personally, I'd still go with using the deprecated getenv for Windows (with appropriate changes to silence the compiler warnings). If you want to go straight for 2, go for it, but I warn you that it will probably be complicated and span several repositories.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good to me, and I agree with @clalancette that option 1 is the first thing to do.
The deprecated warning can be silenced using pragma warning appropriately.

Signed-off-by: Suyash458 <suyash.behera458@gmail.com>
Copy link
Copy Markdown
Contributor

@clalancette clalancette left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks reasonable to me. We'll also want to open a new issue to track making this even more thread-safe; i.e. the contents of #237 (comment) . @suyash458 can you do that? I'll run CI on it, and assuming that comes back clean on all platforms, I'll approve.

src/get_env.c Outdated
}
#else
*env_value = getenv(env_name);
// Note: getenv is deprecated; consider using getenv_s instead
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This note should be above the line it talks about, and should be something more like:

Suggested change
// Note: getenv is deprecated; consider using getenv_s instead
// TODO(Suyash458): getenv is deprecated on Windows; consider using getenv_s instead

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, will update this and create a new issue

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@clalancette do we want to close this and create a new issue or just update it with these discussions?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's close #30 (which will happen just by merging this PR) and open a new one.

@clalancette
Copy link
Copy Markdown
Contributor

CI:

  • Linux Build Status
  • Linux-aarch64 Build Status
  • macOS Build Status
  • Windows Build Status

static char __env_buffer[WINDOWS_ENV_BUFFER_SIZE];
#endif // _WIN32

#pragma warning(disable : 4996)
Copy link
Copy Markdown
Contributor

@clalancette clalancette Apr 15, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CI is showing unknown pragmas from GCC and clang. You'll need to wrap this in an #ifdef _WIN32.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've updated this bit and the comment

Signed-off-by: Suyash458 <suyash.behera458@gmail.com>
Copy link
Copy Markdown
Contributor

@clalancette clalancette left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks reasonable to me now. I'll run CI on it again to see what it looks like.

@clalancette
Copy link
Copy Markdown
Contributor

CI:

  • Linux Build Status
  • Linux-aarch64 Build Status
  • macOS Build Status
  • Windows Build Status

@dirk-thomas
Copy link
Copy Markdown
Member

This looks like a step back to me. There is a reason why this function is deprecated and why we moved away from it in the past.

@clalancette
Copy link
Copy Markdown
Contributor

This looks like a step back to me. There is a reason why this function is deprecated and why we moved away from it in the past.

I disagree. #237 (comment) lays it out in a lot of detail; using getenv_s as we are currently using it is worse than using getenv. Change 2 above will be even better, but this change removes the worst offender of thread unsafety we have.

@ivanpauno
Copy link
Copy Markdown
Member

This looks like a step back to me. There is a reason why this function is deprecated and why we moved away from it in the past.

getenv on Windows isn't actually more "unsafe" than on Linux.
Windows marks getenv as deprecated not because is going to be removed, but because they think that getenv_s signature allows to catch more errors (see https://docs.microsoft.com/en-us/cpp/c-runtime-library/security-features-in-the-crt?view=vs-2019).
The same logic also applies on Linux, but all *_s functions weren't implemented there (which is an optional C11 extension AFAIK).

IMO, this solves the problem of calling rcutils_getenv concurrently (it doesn't solve the problem of also modifying the environment concurrently with putenv or unsetenv, which is a different one).

We can also add another version of rcutils_getenv mimicking getenv_s (which on Linux/osX will just use getenv), and deprecate the old one if we don't want to ignore windows CRT warnings (which IMO, they can safely be ignored).

@clalancette
Copy link
Copy Markdown
Contributor

Bah, my github gist is now out-of-date. Updating it and kicking off another round of CI.

@clalancette
Copy link
Copy Markdown
Contributor

clalancette commented Apr 22, 2020

CI:

  • Linux Build Status
  • Linux-aarch64 Build Status
  • macOS Build Status
  • Windows Build Status

@ivanpauno ivanpauno changed the title update rcutils_get_env to return a malloc'ed string for Windows update rcutils_get_env to always use getenv Apr 22, 2020
@ivanpauno ivanpauno changed the title update rcutils_get_env to always use getenv update rcutils_get_env to always use getenv Apr 22, 2020
}
#else

// TODO(Suyash458): getenv is deprecated on Windows; consider using getenv_s instead
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@suyash458 aren't we consciously ignoring that deprecation? Why keeping a TODO?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I asked for that. It's mostly so that we can make the even better (but bigger) change in step 2 in #237 (comment) . At that point, we most likely will switch back to getenv_s.

@clalancette
Copy link
Copy Markdown
Contributor

I'm going to go ahead and merge this as we have green CI. I'll also open a follow-up issue so that we can do more thread-safetyness of this function in the future.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement a thread safe version of rcutils_get_env

6 participants