Use SECPKG_ATTR_REMOTE_CERT_CHAIN to retrieve server cert during handshake #65134

rzikm · 2022-02-10T12:23:07Z

SECPKG_ATTR_REMOTE_CERT_CHAIN can be used before the handshake completes (contrary to SECPKG_ATTR_REMOTE_CERT_CHAIN_CONTEXT used until now which led to SEC_E_INVALID_HANDLE if called before handshake completion).

Tests were updated to assert that the remoteCertificate parameter in LocalCertificateSelectionCallback is not null.

ghost · 2022-02-10T12:23:14Z

Tagging subscribers to this area: @dotnet/ncl, @vcsjones
See info in area-owners.md if you want to be subscribed.

Issue Details

Fixes #63321

SECPKG_ATTR_REMOTE_CERT_CHAIN can be used even before the handshake completes (contrary to _CONTEXT used until now which led to SEC_E_INVALID_HANDLE if called before handshake completion).

Tests were updated to assert that the remoteCertificate parameter in LocalCertificateSelectionCallback is not null.

Author:	rzikm
Assignees:	-
Labels:	`area-System.Net.Security`
Milestone:	-

rzikm · 2022-02-10T12:41:16Z

On second thought, I am not 100% sure how the new code behaves if the cert chain is longer than 1 certificate, @wfurt thoughts?

rzikm · 2022-02-11T11:29:04Z

/azp run runtime-libraries-coreclr outerloop

azure-pipelines · 2022-02-11T11:29:20Z

Azure Pipelines successfully started running 1 pipeline(s).

rzikm · 2022-02-11T11:41:02Z

I have run our handshake benchmarks to see if it matters from perf perspective if we check SECPKG_ATTR_REMOTE_CERT_CHAIN first or SECPKG_ATTR_REMOTE_CERT_CHAIN_CONTEXT (with fallback to the other). I changed the benchmarks locally a bit to always request the client cert, and either return it during first or second call to LocalClientCertificateSelectionCallback (delayCert), similarly like we do in functional tests. It seems to me that there is no significant perf difference between the two.

Benchmark output

BenchmarkDotNet=v0.13.1.1694-nightly, OS=Windows 11 (10.0.22000.493/21H2)                                                                 
Intel Core i9-10900K CPU 3.70GHz, 1 CPU, 20 logical and 10 physical cores                                                                 
.NET SDK=7.0.100-preview.2.22110.13                                                                                                       
  [Host]     : .NET 7.0.0 (7.0.22.10302), X64 RyuJIT                                                                                      
  Job-PUVUEU : .NET 7.0.0 (42.42.42.42424), X64 RyuJIT                                                                                    
  Job-ODDRLR : .NET 7.0.0 (42.42.42.42424), X64 RyuJIT                                                                                    
  Job-KNOFCC : .NET 7.0.0 (42.42.42.42424), X64 RyuJIT                                                                                    
                                                                                                                                          
PowerPlanMode=00000000-0000-0000-0000-000000000000  Arguments=/p:DebugType=portable,-bl:benchmarkdotnet.binlog  IterationTime=250.0000 ms 
MaxIterationCount=20  MinIterationCount=15  WarmupCount=1                                                                                 

|                     Method |            Toolchain | protocol | delayCert | Ratio | RatioSD |      Mean |     Error |    StdDev |   Median |      Min |      Max |Allocated | Alloc Ratio |
|--------------------------- |--------------------- |--------- |---------- |------:|--------:|----------:|----------:|----------:|---------:|---------:|---------:|---------:|------------:|
|      HandshakeContosoAsync | \63321-CONTEXT_FIRST |    Tls12 |     False |  1.00 |    0.01 |  9.450 ms | 0.1061 ms | 0.0993 ms | 9.440 ms | 9.330 ms | 9.661 ms | 24.09 KB |        1.00 |
|      HandshakeContosoAsync |            \63321-pr |    Tls12 |     False |  1.02 |    0.01 |  9.572 ms | 0.0921 ms | 0.0862 ms | 9.567 ms | 9.417 ms | 9.702 ms | 24.15 KB |        1.00 |
|      HandshakeContosoAsync |                \main |    Tls12 |     False |  1.00 |    0.00 |  9.420 ms | 0.0871 ms | 0.0815 ms | 9.397 ms | 9.307 ms | 9.579 ms | 24.15 KB |        1.00 |
|                            |                      |          |           |       |         |           |           |           |          |          |          |          |             |
| HandshakeECDSA256CertAsync | \63321-CONTEXT_FIRST |    Tls12 |     False |  0.99 |    0.01 |  4.184 ms | 0.0340 ms | 0.0318 ms | 4.192 ms | 4.125 ms | 4.240 ms | 19.47 KB |        1.00 |
| HandshakeECDSA256CertAsync |            \63321-pr |    Tls12 |     False |  1.01 |    0.01 |  4.254 ms | 0.0401 ms | 0.0375 ms | 4.244 ms | 4.188 ms | 4.305 ms | 19.46 KB |        1.00 |
| HandshakeECDSA256CertAsync |                \main |    Tls12 |     False |  1.00 |    0.00 |  4.209 ms | 0.0295 ms | 0.0276 ms | 4.208 ms | 4.169 ms | 4.269 ms | 19.46 KB |        1.00 |
|                            |                      |          |           |       |         |           |           |           |          |          |          |          |             |
|  HandshakeRSA1024CertAsync | \63321-CONTEXT_FIRST |    Tls12 |     False |  0.99 |    0.01 |  3.024 ms | 0.0339 ms | 0.0301 ms | 3.028 ms | 2.974 ms | 3.076 ms | 19.83 KB |        1.00 |
|  HandshakeRSA1024CertAsync |            \63321-pr |    Tls12 |     False |  1.02 |    0.01 |  3.097 ms | 0.0298 ms | 0.0249 ms | 3.095 ms | 3.052 ms | 3.140 ms | 19.84 KB |        1.00 |
|  HandshakeRSA1024CertAsync |                \main |    Tls12 |     False |  1.00 |    0.00 |  3.042 ms | 0.0307 ms | 0.0287 ms | 3.040 ms | 2.989 ms | 3.095 ms | 19.83 KB |        1.00 |
|                            |                      |          |           |       |         |           |           |           |          |          |          |          |             |
|  HandshakeRSA2048CertAsync | \63321-CONTEXT_FIRST |    Tls12 |     False |  0.99 |    0.01 |  3.853 ms | 0.0272 ms | 0.0241 ms | 3.853 ms | 3.825 ms | 3.908 ms |  20.6 KB |        1.00 |
|  HandshakeRSA2048CertAsync |            \63321-pr |    Tls12 |     False |  1.02 |    0.01 |  3.952 ms | 0.0459 ms | 0.0429 ms | 3.937 ms | 3.884 ms | 4.035 ms |  20.6 KB |        1.00 |
|  HandshakeRSA2048CertAsync |                \main |    Tls12 |     False |  1.00 |    0.00 |  3.881 ms | 0.0375 ms | 0.0351 ms | 3.879 ms | 3.828 ms | 3.959 ms |  20.6 KB |        1.00 |
|                            |                      |          |           |       |         |           |           |           |          |          |          |          |             |
|  HandshakeRSA4096CertAsync | \63321-CONTEXT_FIRST |    Tls12 |     False |  0.99 |    0.01 |  9.466 ms | 0.0835 ms | 0.0781 ms | 9.484 ms | 9.343 ms | 9.573 ms | 22.16 KB |        1.00 |
|  HandshakeRSA4096CertAsync |            \63321-pr |    Tls12 |     False |  1.00 |    0.01 |  9.536 ms | 0.0951 ms | 0.0890 ms | 9.518 ms | 9.392 ms | 9.703 ms | 22.13 KB |        1.00 |
|  HandshakeRSA4096CertAsync |                \main |    Tls12 |     False |  1.00 |    0.00 |  9.571 ms | 0.1133 ms | 0.1060 ms | 9.579 ms | 9.418 ms | 9.805 ms | 22.16 KB |        1.00 |
|                            |                      |          |           |       |         |           |           |           |          |          |          |          |             |
|      HandshakeContosoAsync | \63321-CONTEXT_FIRST |    Tls12 |      True |  1.00 |    0.02 |  9.493 ms | 0.1095 ms | 0.1024 ms | 9.512 ms | 9.321 ms | 9.680 ms | 24.08 KB |        1.00 |
|      HandshakeContosoAsync |            \63321-pr |    Tls12 |      True |  1.00 |    0.01 |  9.494 ms | 0.1144 ms | 0.1014 ms | 9.503 ms | 9.337 ms | 9.645 ms | 24.08 KB |        1.00 |
|      HandshakeContosoAsync |                \main |    Tls12 |      True |  1.00 |    0.00 |  9.481 ms | 0.1336 ms | 0.1250 ms | 9.441 ms | 9.319 ms | 9.701 ms | 24.08 KB |        1.00 |
|                            |                      |          |           |       |         |           |           |           |          |          |          |          |             |
| HandshakeECDSA256CertAsync | \63321-CONTEXT_FIRST |    Tls12 |      True |  1.01 |    0.01 |  4.221 ms | 0.0234 ms | 0.0195 ms | 4.224 ms | 4.176 ms | 4.250 ms | 19.47 KB |        1.00 |
| HandshakeECDSA256CertAsync |            \63321-pr |    Tls12 |      True |  1.02 |    0.01 |  4.275 ms | 0.0289 ms | 0.0271 ms | 4.275 ms | 4.234 ms | 4.319 ms | 19.47 KB |        1.00 |
| HandshakeECDSA256CertAsync |                \main |    Tls12 |      True |  1.00 |    0.00 |  4.189 ms | 0.0264 ms | 0.0234 ms | 4.193 ms | 4.145 ms | 4.234 ms | 19.46 KB |        1.00 |
|                            |                      |          |           |       |         |           |           |           |          |          |          |          |             |
|  HandshakeRSA1024CertAsync | \63321-CONTEXT_FIRST |    Tls12 |      True |  1.01 |    0.02 |  3.065 ms | 0.0431 ms | 0.0382 ms | 3.060 ms | 3.017 ms | 3.160 ms | 19.83 KB |        1.00 |
|  HandshakeRSA1024CertAsync |            \63321-pr |    Tls12 |      True |  1.03 |    0.01 |  3.116 ms | 0.0265 ms | 0.0235 ms | 3.116 ms | 3.063 ms | 3.144 ms | 19.83 KB |        1.00 |
|  HandshakeRSA1024CertAsync |                \main |    Tls12 |      True |  1.00 |    0.00 |  3.039 ms | 0.0271 ms | 0.0253 ms | 3.038 ms | 2.982 ms | 3.081 ms | 19.83 KB |        1.00 |
|                            |                      |          |           |       |         |           |           |           |          |          |          |          |             |
|  HandshakeRSA2048CertAsync | \63321-CONTEXT_FIRST |    Tls12 |      True |  0.99 |    0.01 |  3.871 ms | 0.0301 ms | 0.0251 ms | 3.864 ms | 3.839 ms | 3.928 ms | 20.61 KB |        1.00 |
|  HandshakeRSA2048CertAsync |            \63321-pr |    Tls12 |      True |  1.01 |    0.02 |  3.949 ms | 0.0421 ms | 0.0394 ms | 3.939 ms | 3.897 ms | 4.021 ms |  20.6 KB |        1.00 |
|  HandshakeRSA2048CertAsync |                \main |    Tls12 |      True |  1.00 |    0.00 |  3.915 ms | 0.0506 ms | 0.0474 ms | 3.896 ms | 3.870 ms | 4.017 ms | 20.61 KB |        1.00 |
|                            |                      |          |           |       |         |           |           |           |          |          |          |          |             |
|  HandshakeRSA4096CertAsync | \63321-CONTEXT_FIRST |    Tls12 |      True |  0.99 |    0.01 |  9.527 ms | 0.0958 ms | 0.0896 ms | 9.509 ms | 9.414 ms | 9.694 ms | 22.13 KB |        1.00 |
|  HandshakeRSA4096CertAsync |            \63321-pr |    Tls12 |      True |  1.00 |    0.01 |  9.556 ms | 0.0683 ms | 0.0606 ms | 9.554 ms | 9.468 ms | 9.659 ms | 22.13 KB |        1.00 |
|  HandshakeRSA4096CertAsync |                \main |    Tls12 |      True |  1.00 |    0.00 |  9.596 ms | 0.0640 ms | 0.0599 ms | 9.586 ms | 9.519 ms | 9.728 ms | 22.13 KB |        1.00 |
|                            |                      |          |           |       |         |           |           |           |          |          |          |          |             |
|      HandshakeContosoAsync | \63321-CONTEXT_FIRST |    Tls13 |     False |  1.00 |    0.02 |  8.699 ms | 0.0866 ms | 0.0768 ms | 8.668 ms | 8.630 ms | 8.869 ms |    24 KB |        1.00 |
|      HandshakeContosoAsync |            \63321-pr |    Tls13 |     False |  1.00 |    0.01 |  8.686 ms | 0.0675 ms | 0.0599 ms | 8.681 ms | 8.595 ms | 8.809 ms | 24.03 KB |        1.00 |
|      HandshakeContosoAsync |                \main |    Tls13 |     False |  1.00 |    0.00 |  8.707 ms | 0.1266 ms | 0.1184 ms | 8.707 ms | 8.541 ms | 8.905 ms |    24 KB |        1.00 |
|                            |                      |          |           |       |         |           |           |           |          |          |          |          |             |
| HandshakeECDSA256CertAsync | \63321-CONTEXT_FIRST |    Tls13 |     False |  0.99 |    0.01 |  3.296 ms | 0.0234 ms | 0.0207 ms | 3.297 ms | 3.245 ms | 3.327 ms | 19.38 KB |        1.00 |
| HandshakeECDSA256CertAsync |            \63321-pr |    Tls13 |     False |  1.01 |    0.01 |  3.365 ms | 0.0280 ms | 0.0248 ms | 3.369 ms | 3.314 ms | 3.403 ms | 19.38 KB |        1.00 |
| HandshakeECDSA256CertAsync |                \main |    Tls13 |     False |  1.00 |    0.00 |  3.323 ms | 0.0402 ms | 0.0356 ms | 3.321 ms | 3.257 ms | 3.373 ms | 19.38 KB |        1.00 |
|                            |                      |          |           |       |         |           |           |           |          |          |          |          |             |
|  HandshakeRSA1024CertAsync | \63321-CONTEXT_FIRST |    Tls13 |     False |  0.99 |    0.01 |  2.122 ms | 0.0155 ms | 0.0137 ms | 2.122 ms | 2.104 ms | 2.143 ms | 19.75 KB |        1.00 |
|  HandshakeRSA1024CertAsync |            \63321-pr |    Tls13 |     False |  1.01 |    0.01 |  2.150 ms | 0.0187 ms | 0.0175 ms | 2.154 ms | 2.117 ms | 2.175 ms | 19.75 KB |        1.00 |
|  HandshakeRSA1024CertAsync |                \main |    Tls13 |     False |  1.00 |    0.00 |  2.134 ms | 0.0285 ms | 0.0253 ms | 2.141 ms | 2.080 ms | 2.174 ms | 19.76 KB |        1.00 |
|                            |                      |          |           |       |         |           |           |           |          |          |          |          |             |
|  HandshakeRSA2048CertAsync | \63321-CONTEXT_FIRST |    Tls13 |     False |  0.98 |    0.02 |  2.996 ms | 0.0524 ms | 0.0490 ms | 2.989 ms | 2.944 ms | 3.100 ms | 20.51 KB |        1.00 |
|  HandshakeRSA2048CertAsync |            \63321-pr |    Tls13 |     False |  0.99 |    0.01 |  3.015 ms | 0.0280 ms | 0.0248 ms | 3.014 ms | 2.974 ms | 3.063 ms | 20.52 KB |        1.00 |
|  HandshakeRSA2048CertAsync |                \main |    Tls13 |     False |  1.00 |    0.00 |  3.052 ms | 0.0458 ms | 0.0406 ms | 3.050 ms | 2.989 ms | 3.134 ms | 20.51 KB |        1.00 |
|                            |                      |          |           |       |         |           |           |           |          |          |          |          |             |
|  HandshakeRSA4096CertAsync | \63321-CONTEXT_FIRST |    Tls13 |     False |  0.98 |    0.01 |  8.683 ms | 0.0819 ms | 0.0726 ms | 8.684 ms | 8.491 ms | 8.795 ms | 22.07 KB |        1.00 |
|  HandshakeRSA4096CertAsync |            \63321-pr |    Tls13 |     False |  0.99 |    0.01 |  8.731 ms | 0.1057 ms | 0.0989 ms | 8.729 ms | 8.613 ms | 8.931 ms | 22.04 KB |        1.00 |
|  HandshakeRSA4096CertAsync |                \main |    Tls13 |     False |  1.00 |    0.00 |  8.827 ms | 0.1373 ms | 0.1284 ms | 8.808 ms | 8.669 ms | 9.123 ms | 22.04 KB |        1.00 |
|                            |                      |          |           |       |         |           |           |           |          |          |          |          |             |
|      HandshakeContosoAsync | \63321-CONTEXT_FIRST |    Tls13 |      True |  1.00 |    0.01 |  8.744 ms | 0.0855 ms | 0.0758 ms | 8.742 ms | 8.644 ms | 8.877 ms |    24 KB |        1.00 |
|      HandshakeContosoAsync |            \63321-pr |    Tls13 |      True |  1.02 |    0.01 |  8.831 ms | 0.0772 ms | 0.0722 ms | 8.840 ms | 8.720 ms | 8.985 ms |    24 KB |        1.00 |
|      HandshakeContosoAsync |                \main |    Tls13 |      True |  1.00 |    0.00 |  8.708 ms | 0.0681 ms | 0.0604 ms | 8.707 ms | 8.603 ms | 8.786 ms |    24 KB |        1.00 |
|                            |                      |          |           |       |         |           |           |           |          |          |          |          |             |
| HandshakeECDSA256CertAsync | \63321-CONTEXT_FIRST |    Tls13 |      True |  0.99 |    0.01 |  3.298 ms | 0.0291 ms | 0.0243 ms | 3.306 ms | 3.254 ms | 3.333 ms | 19.39 KB |        1.00 |
| HandshakeECDSA256CertAsync |            \63321-pr |    Tls13 |      True |  1.01 |    0.01 |  3.349 ms | 0.0368 ms | 0.0344 ms | 3.343 ms | 3.301 ms | 3.418 ms | 19.38 KB |        1.00 |
| HandshakeECDSA256CertAsync |                \main |    Tls13 |      True |  1.00 |    0.00 |  3.314 ms | 0.0268 ms | 0.0237 ms | 3.315 ms | 3.268 ms | 3.355 ms | 19.38 KB |        1.00 |
|                            |                      |          |           |       |         |           |           |           |          |          |          |          |             |
|  HandshakeRSA1024CertAsync | \63321-CONTEXT_FIRST |    Tls13 |      True |  1.02 |    0.01 |  2.139 ms | 0.0187 ms | 0.0175 ms | 2.141 ms | 2.105 ms | 2.164 ms | 19.76 KB |        1.00 |
|  HandshakeRSA1024CertAsync |            \63321-pr |    Tls13 |      True |  1.01 |    0.02 |  2.130 ms | 0.0295 ms | 0.0276 ms | 2.131 ms | 2.084 ms | 2.183 ms | 19.75 KB |        1.00 |
|  HandshakeRSA1024CertAsync |                \main |    Tls13 |      True |  1.00 |    0.00 |  2.104 ms | 0.0277 ms | 0.0259 ms | 2.095 ms | 2.069 ms | 2.150 ms | 19.75 KB |        1.00 |
|                            |                      |          |           |       |         |           |           |           |          |          |          |          |             |
|  HandshakeRSA2048CertAsync | \63321-CONTEXT_FIRST |    Tls13 |      True |  1.01 |    0.01 |  3.022 ms | 0.0326 ms | 0.0305 ms | 3.019 ms | 2.953 ms | 3.068 ms | 20.52 KB |        1.00 |
|  HandshakeRSA2048CertAsync |            \63321-pr |    Tls13 |      True |  1.02 |    0.01 |  3.060 ms | 0.0233 ms | 0.0218 ms | 3.069 ms | 3.022 ms | 3.089 ms | 20.51 KB |        1.00 |
|  HandshakeRSA2048CertAsync |                \main |    Tls13 |      True |  1.00 |    0.00 |  2.991 ms | 0.0364 ms | 0.0341 ms | 2.986 ms | 2.939 ms | 3.051 ms | 20.51 KB |        1.00 |
|                            |                      |          |           |       |         |           |           |           |          |          |          |          |             |
|  HandshakeRSA4096CertAsync | \63321-CONTEXT_FIRST |    Tls13 |      True |  1.00 |    0.01 |  8.640 ms | 0.0672 ms | 0.0596 ms | 8.648 ms | 8.548 ms | 8.754 ms | 22.04 KB |        1.00 |
|  HandshakeRSA4096CertAsync |            \63321-pr |    Tls13 |      True |  1.01 |    0.01 |  8.684 ms | 0.0614 ms | 0.0544 ms | 8.682 ms | 8.608 ms | 8.802 ms | 22.04 KB |        1.00 |
|  HandshakeRSA4096CertAsync |                \main |    Tls13 |      True |  1.00 |    0.00 |  8.627 ms | 0.0732 ms | 0.0684 ms | 8.644 ms | 8.481 ms | 8.727 ms | 22.04 KB |        1.00 |

wfurt · 2022-02-12T02:32:41Z

src/libraries/System.Net.Security/src/System/Net/CertificateValidationPal.Windows.cs

Is the ever called on server? It would be pretty normal that the remote certificate is null but it would not make sense to try again IMHO. We can probably create static property to show if the call is available - similar to what were do for TLS 1.3.

wfurt · 2022-02-12T02:36:53Z

nice. Thanks for the benchmark.

rzikm · 2022-02-15T14:28:02Z

CI failures are transient and unrelated.

…shake Fixes dotnet#63321

rzikm · 2022-02-23T09:10:19Z

@wfurt As per our offline discussion, I checked the behavior when no peer cert is available. In such cases, the QueryContextAttributes returns 0x809030E (NoCredentials), I made the code consider this as a success so that the query is not repeated using the other attribute.

wfurt

LGTM.
Assuming the disabled tests would pass.

wfurt · 2022-02-23T18:05:36Z

src/libraries/System.Net.Security/tests/FunctionalTests/CertificateValidationClientServer.cs

-        [InlineData(false, true)]
-        [InlineData(true, false)]
+        // [InlineData(true, true)]
+        // [InlineData(false, true)]


Debug leftover???

good catch :/, yes, they pass

rzikm · 2022-02-24T12:28:25Z

Test failures are:

ghost added the area-System.Net.Security label Feb 10, 2022

ghost assigned rzikm Feb 10, 2022

rzikm requested a review from wfurt February 10, 2022 12:23

runfoapp bot mentioned this pull request Feb 10, 2022

TLS1.3 networking tests are flaky on Windows 11 #58927

Closed

runfoapp bot mentioned this pull request Feb 11, 2022

System.Net.Security.Tests CertificateValidationRemoteServer_EndToEnd_Ok failing on Windows 7 #65168

Closed

wfurt reviewed Feb 12, 2022

View reviewed changes

rzikm requested a review from wfurt February 15, 2022 14:27

rzikm added 5 commits February 23, 2022 09:38

Use SECPKG_ATTR_REMOTE_CERT_CHAIN to retrieve server cert during hand…

de5f958

…shake Fixes dotnet#63321

Fix tests on Windows 7

6fe01f2

Minor formatting change

e29b9d3

Code review feedback

8a4acdc

Don't check again if no peer cert was provided

9900540

rzikm force-pushed the 63321-remoteCertificate-passed-to-LocalCertificateSelectionCallback-is-always-null-on-Windows branch from 8e4367b to 9900540 Compare February 23, 2022 09:08

wfurt approved these changes Feb 23, 2022

View reviewed changes

Revert unwanted change

dd3a405

rzikm merged commit 8a60ecc into dotnet:main Feb 24, 2022

rzikm removed their assignment Feb 24, 2022

kunalspathak mentioned this pull request Mar 1, 2022

Regressions in System.Net.Security.Tests.SslStreamTests #66012

Closed

rzikm mentioned this pull request Mar 2, 2022

Fix performance regression in SSL handshake #66077

Merged

ghost locked as resolved and limited conversation to collaborators Mar 26, 2022

karelz added this to the 7.0.0 milestone Apr 8, 2022

Use SECPKG_ATTR_REMOTE_CERT_CHAIN to retrieve server cert during handshake #65134

Use SECPKG_ATTR_REMOTE_CERT_CHAIN to retrieve server cert during handshake #65134

Uh oh!

Conversation

rzikm commented Feb 10, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ghost commented Feb 10, 2022

Uh oh!

rzikm commented Feb 10, 2022

Uh oh!

rzikm commented Feb 11, 2022

Uh oh!

azure-pipelines bot commented Feb 11, 2022

Uh oh!

rzikm commented Feb 11, 2022

Uh oh!

wfurt Feb 12, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

wfurt commented Feb 12, 2022

Uh oh!

rzikm commented Feb 15, 2022

Uh oh!

rzikm commented Feb 23, 2022

Uh oh!

wfurt left a comment

Choose a reason for hiding this comment

Uh oh!

wfurt Feb 23, 2022

Choose a reason for hiding this comment

Uh oh!

rzikm Feb 23, 2022

Choose a reason for hiding this comment

Uh oh!

rzikm commented Feb 24, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

rzikm commented Feb 10, 2022 •

edited

Loading

wfurt Feb 12, 2022 •

edited

Loading