In other words I think we can reuse logic that already exists in NSJSONSerialization class for handling unpaired surrogates instead of implementing from scratch, so it's easier to maintain:

my example Swift playground does this and it seems to work fine:

import UIKit

var data = "hello".data(using: .utf16)!
data.append(Data([0x00, 0xDB]))
let string = NSString(data: data, encoding: NSUTF16StringEncoding)!
let rawJSON = "[\"\(string)\"]"
  
let json = try? JSONSerialization.jsonObject(with: rawJSON.data(using: .utf16)!) 

In other words I think we can reuse logic that already exists in NSJSONSerialization class for handling unpaired surrogates instead of implementing from scratch, so it's easier to maintain:

my example Swift playground does this and it seems to work fine:

import UIKit

var data = "hello".data(using: .utf16)!
data.append(Data([0x00, 0xDB]))
let string = NSString(data: data, encoding: NSUTF16StringEncoding)!
let rawJSON = "[\"\(string)\"]"
  
let json = try? JSONSerialization.jsonObject(with: rawJSON.data(using: .utf16)!) 

Thanks for the suggestion! I tried implementing the UTF-16 approach, but unfortunately it didn't work

When I changed the code to:

1. Convert the sanitized string (the output of the PASS 1 in FLTSanitizeUTF8ForJSON) to UTF-16 data
2. Pass the UTF-16 data to NSJSONSerialization

OR in FlutterJSONMessageCodec::decode

NSString* string = [[NSString alloc] initWithData:message encoding:NSUTF8StringEncoding];
NSData* utf16Message = [string dataUsingEncoding:NSUTF16StringEncoding];

The app crashes because NSJSONSerialization JSONObjectWithData:options:error: returns nil for the UTF-16 encoded data, even with valid JSON content. It seems NSJSONSerialization doesn't reliably parse UTF-16 encoded JSON in this context

When trying with Swift's JSONSerialization it works, but not with NSJSONSerialization, I might be missing something.

I looked into what did the string sanitization in the swift case, it turns out to be one of the NSString initializer: https://developer.apple.com/documentation/foundation/nsstring/init(bytes:length:encoding:). And interestingly the same test implementation in objective-c doesn't replace unpaired surrogates with U+FFFD. I'll re-implement the helper in swift.

I looked into what did the string sanitization in the swift case, it turns out to be one of the NSString initializer: https://developer.apple.com/documentation/foundation/nsstring/init(bytes:length:encoding:). And interestingly the same test implementation in objective-c doesn't replace unpaired surrogates with U+FFFD. I'll re-implement the helper in swift.

Alright, then I'm going to close this PR?

I looked into what did the string sanitization in the swift case, it turns out to be one of the NSString initializer: developer.apple.com/documentation/foundation/nsstring/init(bytes:length:encoding:). And interestingly the same test implementation in objective-c doesn't replace unpaired surrogates with U+FFFD. I'll re-implement the helper in swift.

Alright, then I'm going to close this PR?

I made #181921 but ran into issues with swift name mangling. I'm going to turn this into a draft since #181921 should fix the crash when merged.

Hey @mahmuttaskiran, checking in here during triage. Is this superseded by #181921?

Hey @mahmuttaskiran, checking in here during triage. Is this superseded by #181921?

Hey, it's yes. Closing it now. Sorry for late