We could optimize this to not need this try/finally:

IntPtr __retval;
int hresult = (*(delegate* unmanaged[Stdcall]<IntPtr, IntPtr*, int>**)ThisPtr)[3](ThisPtr, &__retval);
dataPtr = __retval;

Marshal.Release(ThisPtr);

global::WinRT.ExceptionHelpers.ThrowExceptionForHR(hresult);

return true;

But also, why are we throwing in the first place here, given the method is named "Try"?

Try doesn't mean we never throw. Try here means we are not throwing if the type doesn't implement IBufferByteAccess. If it does implement it but the implementation is bad we still throw.

I'm not entirely sure that's intuitive, but I suppose it's fine. Can we drop the try/finally though?

I don't think it's an optimization though.
try/finally doesn't have performance impact for fast path (no exception being thrown) and it can make sure the finally block being executed once there's an exception. You can check the codegen, unless you don't have catch blocks in a try, it shouldn't make any meaningful impact on the performance.