I see no way currently to figure out when a swapchain image is actually visible on the screen.

Imagine an application which needs 4ms to execute a draw call and is running on a 16ms vsync display. Here's what a timeline could look like: (correct me if I'm wrong), supposing that we start the application immediately after a vsync already happened.

1. t=0ms: Swapchain is created and all command buffers drawing to its images are recorded. Images are guarded by a semaphore (respectively).
2. t=0ms: The application acquires the next image for use, which will signal the semaphore and (optionally) a fence
3. t=0ms: The application submits a draw command which will wait for the signal, remove it, and resignal it once done
4. t=0ms: The application queues up the image for presentation, which will wait for signal and remove it again

After this batch of setup, the following things happen:
3. t=0ms: the semaphore is signaled right away, and (optionally) the fence is triggered indicating that the image acquired in step 2 is available for use. The semaphore being signalled allows the draw command to start
4. t=4ms: The draw command finishes, and signals the semaphore again. This allows the GPU to start using the image for presentation (removing the signal). But it is not visible yet, because the next page flip has not yet occurred
5. t=16ms: The GPU flips pages and actually starts displaying the screen.
... at this point it is assume that the application also does whatever is necessary for drawing the next frame
6. t=32ms: The GPU flips pages again and stops using the surface (signalling the semaphore). Assume it takes 1ms for the image to get freed up and be reusable again
7. t=33ms: The application would be able to acquire the image again (i.e. triggering the fence)

To summarize it, on the CPU side of things I can get accurate information about the following points in time:

1. The image is ready for use (t=0ms, t=33ms)
2. The draw command completes, but the image is not yet visible (e.g. by triggering an event from the end of the command buffer)

But I can't seem to get any reliable information about t=16ms, i.e. when the frame I just submitted is actually visible. This is important to me because I need to measure display latency and effective refresh rate accurately.

The problem gets worse if I use a large swapchain. For example, suppose my swapchain is size 4.

In the first world, i.e. where I wait on the fence indicating that the image is ready for use again, I would measure differences in frame times something like this:

1. t=0ms -> delta = 0ms
2. t=0ms -> delta = 0ms
3. t=0ms -> delta = 0ms
4. t=0ms -> delta = 0ms
5. t=17ms -> delta = 17ms
6. t=33ms -> delta = 16ms
7. t=49ms -> delta = 16ms
   ...

In the second world, i.e. where I trigger an event once I've finished rendering and wait on that to complete, I would measure frame times like this:

1. t=4ms -> delta = 4ms
2. t=8ms -> delta = 4ms
3. t=12ms -> delta = 4ms
4. t=16ms -> delta = 4ms
5. t=20ms -> delta = 4ms
6. t=36ms -> delta = 16ms
7. t=52ms -> delta = 16ms
   ...

Basically, they all converge to the true vsync timing (16ms) in the limit, but the measurements at the start will always be off since the GPU can already acquire the next image and/or render to it well in advance of when it will actually be used.

How do you advise accomplishing what I want? (Measuring the real delay between submitting a frame and it being visible on screen)