Conversation
When RawTag.UnmarshalCBOR() is called by codec (normal case), the codec will first check if data is well-formed before calling RawTag.UnmarshalCBOR(data). However, it can also be called by user app (not intended use) and user apps might not check if data is well-formed. In such cases, this function can panic if given malformed data. This commit updates RawTag.UnmarshalCBOR() to check for well-formedness inside the function, so it behaves the same whether it is called by codec internally or by user app. Unfortunately, this approach means the same data is checked twice for the normal case of the codec using Unmarshal(data, *RawTag). This can be revisited and maybe optimized in the future.
When SimpleValue.UnmarshalCBOR() is called by codec (normal case), the codec will first check if data is well-formed before calling SimpleValue.UnmarshalCBOR(data). However, it can also be called by user app (not intended use) and user apps might not check if data is well-formed. In such cases, this function can panic if given malformed data. This commit updates SimpleValue.UnmarshalCBOR() to check for well-formedness inside the function, so it behaves the same whether it is called by codec internally or by user app. Unfortunately, this approach means the same data is checked twice for the normal case of the codec using Unmarshal(data, *SimpleValue). This can be revisited and maybe optimized in the future.
When ByteString.UnmarshalCBOR() is called by codec (normal case), the codec will first check if data is well-formed before calling ByteString.UnmarshalCBOR(data). However, it can also be called by user app (not intended use) and user apps might not check if data is well-formed. In such cases, this function can panic if given malformed data. This commit updates ByteString.UnmarshalCBOR() to check for well-formedness inside the function, so it behaves the same whether it is called by codec internally or by user app. Unfortunately, this approach means the same data is checked twice for the normal case of the codec using Unmarshal(data, *ByteString). This can be revisited and maybe optimized in the future.
Owner
Author
|
@x448 PTAL 🙏 |
x448
approved these changes
Mar 27, 2025
This was referenced Mar 27, 2025
fxamacker
added a commit
that referenced
this pull request
Mar 28, 2025
Currently, unreleased changes in PR #636 and #645 cause the input data to be checked twice when UnmarshalCBOR() is called internally by Unmarshal() for: - ByteString - RawTag - SimpleValue UnmarshalCBOR() checks input data because it can be called by user apps providing bad data. However, the codec already checks input data before internally calling UnmarshalCBOR() so the 2nd check is redundant. This commit avoids redundant check on the input data by having Unmarshal() call the private unmarshalCBOR() if implemented by ByteString, RawTag, SimpleValue, etc.: - Internally, the codec calls the private unmarshalCBOR() to avoid the redundant check on input data. - Externally, MarshalCBOR() is available as a wrapper that checks input data before calling the private unmarshalCBOR(). MarshalCBOR() for ByteString, RawTag, and SimpleValue are marked as deprecated and Unmarshal() should be used instead.
fxamacker
added a commit
that referenced
this pull request
Mar 28, 2025
Currently, unreleased changes in PR #636 and #645 cause the input data to be checked twice when UnmarshalCBOR() is called internally by Unmarshal() for: - ByteString - RawTag - SimpleValue UnmarshalCBOR() checks input data because it can be called by user apps providing bad data. However, the codec already checks input data before internally calling UnmarshalCBOR() so the 2nd check is redundant. This commit avoids redundant check on the input data by having Unmarshal() call the private unmarshalCBOR() if implemented by ByteString, RawTag, SimpleValue, etc.: - Internally, the codec calls the private unmarshalCBOR() to avoid the redundant check on input data. - Externally, UnmarshalCBOR() is available as a wrapper that checks input data before calling the private unmarshalCBOR(). UnmarshalCBOR() for ByteString, RawTag, and SimpleValue are marked as deprecated and Unmarshal() should be used instead.
fxamacker
added a commit
that referenced
this pull request
Mar 28, 2025
Currently, unreleased changes in PR #636 and #645 cause the input data to be checked twice when UnmarshalCBOR() is called internally by Unmarshal() for: - ByteString - RawTag - SimpleValue UnmarshalCBOR() checks input data because it can be called by user apps providing bad data. However, the codec already checks input data before internally calling UnmarshalCBOR() so the 2nd check is redundant. This commit avoids redundant check on the input data by having Unmarshal() call the private unmarshalCBOR() if implemented by ByteString, RawTag, SimpleValue, etc.: - Internally, the codec calls the private unmarshalCBOR() to avoid the redundant check on input data. - Externally, UnmarshalCBOR() is available as a wrapper that checks input data before calling the private unmarshalCBOR(). UnmarshalCBOR() for ByteString, RawTag, and SimpleValue are marked as deprecated and Unmarshal() should be used instead.
fxamacker
added a commit
that referenced
this pull request
Mar 28, 2025
Currently, unreleased changes in PR #636 and #645 cause the input data to be checked twice when UnmarshalCBOR() is called internally by Unmarshal() for: - ByteString - RawTag - SimpleValue UnmarshalCBOR() checks input data because it can be called by user apps providing bad data. However, the codec already checks input data before internally calling UnmarshalCBOR() so the 2nd check is redundant. This commit avoids redundant check on the input data by having Unmarshal() call the private unmarshalCBOR() if implemented by ByteString, RawTag, SimpleValue, etc.: - Internally, the codec calls the private unmarshalCBOR() to avoid the redundant check on input data. - Externally, UnmarshalCBOR() is available as a wrapper that checks input data before calling the private unmarshalCBOR(). UnmarshalCBOR() for ByteString, RawTag, and SimpleValue are marked as deprecated and Unmarshal() should be used instead.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This ports PR #636 (in release-2.7 branch) to main branch so it can be included in v2.8.0.
RawTag.UnmarshalCBOR()is intended to be called by the codec internally and the codec checks for malformed data before calling it. However, it is possible for user apps to directly call it, so user apps might provide malformed data which can cause panic.This PR updates these 3 functions to use same error handling as
cbor.Unmarshal():ByteString.UnmarshalCBOR(data)RawTag.UnmarshalCBOR(data)SimpleValue.UnmarshalCBOR(data)Basically, this adds the same well-formedness checks on input data already done by
cbor.Unmarshal(), soUnmarshalCBOR()will return same error if input data is malformed (not panic).Caveats
Unfortunately, this approach means the same data is checked twice for the intended case of the codec calling
UnmarshalCBOR()internally. This can be revisited and maybe optimized in the future.PR #636 passed very brief fuzzing on Sunday, March 16, 2025. However, the fuzzing needs to be run for longer duration before tagging new release.