Deprecate Media Overlays embedded media by marisademeglio · Pull Request #2402 · w3c/epub-specs

marisademeglio · 2022-08-16T20:54:54Z

In the core spec:

Removed 9.3.2.4 entirely, added note to MO text element about how referring to embedded timed media with the MO text element is possible but unspecified.

In the RS spec:

Removed 9.3.2 and added a note about deprecation to the top of 9.3.

See:

For EPUB 3.3 Reading Systems:
- Preview
- Diff

Preview | Diff

In the core spec: Removed 9.3.2.4 entirely, added note to MO text element about how referring to embedded timed media with the MO text element is possible but unspecified. In the RS spec: Removed 9.3.2 and added a note about deprecation to the top of 9.3.

mattgarrish · 2022-08-16T21:44:48Z

I wonder if we should give this the TTS treatment and instead of making it like it never existed, move the authoring and reading system requirements to a note? (Maybe including the TTS section.)

It might never get implemented, but publishing as a note leaves it as an option.

marisademeglio · 2022-08-16T21:50:34Z

I wonder if we should give this the TTS treatment and instead of making it like it never existed, move the authoring and reading system requirements to a note? (Maybe including the TTS section.)

It might never get implemented, but publishing as a note leaves it as an option.

I honestly don't think it's worth the effort. I don't think it will ever get implemented and to specify it properly, even as a note, would be a not-insignificant time investment.

iherman

@marisademeglio (also answering your question) the entries in the change sections are added by hand. Copy-paste-modify is your friend...

epub33/rs/index.html

iherman · 2022-08-17T04:21:58Z

I wonder if we should give this the TTS treatment and instead of making it like it never existed, move the authoring and reading system requirements to a note? (Maybe including the TTS section.)
It might never get implemented, but publishing as a note leaves it as an option.

I honestly don't think it's worth the effort. I don't think it will ever get implemented and to specify it properly, even as a note, would be a not-insignificant time investment.

I agree. The TTS seems to be on much more stable foundations, using external and, mainly, more widely known standards. Although MO is based on SMIL, the latter is, alas!, a completely unknown standard and probably the only reason it has not been deprecated is EPUB's usage of (a small subset of) SMIL. A WG note, to be made worthy of publication, would require quite some work.

mattgarrish · 2022-08-17T16:54:01Z

The TTS seems to be on much more stable foundations

Look out for multiple renditions, in that case...

At any rate, if we don't, then this introduces ambiguity onto the authoring side, as it doesn't say what's supposed to happen now if you do reference an audio or video element. It might be helpful to add a note where media elements are mentioned that previous version of the specification defined auto-playback of these elements, but it's no longer the case (if audio is not provided, then TTS kicks in). Authors should consider alerting users that these elements exist and they need to go back and find them after listening to the work.

mattgarrish · 2022-08-17T17:07:06Z

epub33/rs/index.html

+				<p class="note">Earlier versions of this specification included some information about embedded audio and 
+					video [epubmediaoverlays-32]. This feature has been deprecated.</p>


We can't deprecate in a note, otherwise the deprecation doesn't have normative standing. I'd recreate 9.3.2 rather than completely deleting the section so it matches other deprecated features. I can't suggest a fix through the github interface, but put back something like:

<section id="sec-embedded-media"> <h4>Embedded audio and video (deprecated)</h4> <p>Automatic playback of embedded audio and video is now <a data-cite="epub-33#deprecated">deprecated</a>.</p> <p>Refer to its definition in [[epubmediaoverlays-32]] for more information.</p> </section>

This change should also be integrated before merging.

Done in 9d5bb34

mattgarrish · 2022-08-17T17:12:31Z

mentioned that previous version of the specification defined auto-playback of these elements

Actually, re-reading the note that's there I can probably live with it as-is. I was trying to read the line in the diff and it looked like it was only referring to video.

Co-authored-by: Ivan Herman <ivan@ivan-herman.net>

iherman · 2022-08-18T06:30:08Z

The TTS seems to be on much more stable foundations

Look out for multiple renditions, in that case...

At any rate, if we don't, then this introduces ambiguity onto the authoring side, as it doesn't say what's supposed to happen now if you do reference an audio or video element. It might be helpful to add a note where media elements are mentioned that previous version of the specification defined auto-playback of these elements, but it's no longer the case (if audio is not provided, then TTS kicks in). Authors should consider alerting users that these elements exist and they need to go back and find them after listening to the work.

There is such a mention in the RS spec. Do you mean to add something similar in the core spec as well?

epub33/core/index.html

iherman · 2022-08-26T15:00:04Z

The issue was discussed in a meeting on 2022-08-26

List of resolutions:

Resolution No. 1: Once the edits are finalized, merge PR 2402 and close issue 2397.

View the transcript

1. Deprecate embedded audio or video as targets for Media Overlay `<text>` references.

See github issue epub-specs#2397.

See github pull request epub-specs#2402.

Ivan Herman: See Description of the issue background.

Ivan Herman: on a high level, what you can do with MO is associate a piece of time-stamped audio with a piece of text.
… it is used in a bunch of a11y situations.
… it is pretty well specified.
… based on the tests I have made so far it is well implemented too.
… however the specifications also try to allow association of an audio clip with a video (or several videos).
… now you have two time related media side-by-side.
… for that to work, the spec needs to define what is happening, considering that videos themselves often have UI controls themselves.
… the spec tries to make some sense of this use-case, but this is really underspecified.
… also, even for the cases which are relatively clear, we haven't found any implementations.
… so we have a complicated case, underspecified in spec, and not implemented.
… in side conversation with Avneesh, we did not find any important proper use case in a11y area.
… the tests I created were trying to take a video and add a piece of background audio. But this is an artificial use case, as adding background audio to a video can be done much more easily in editing software rather than via MO.
… so this lead to suggestion that we should just deprecate this type of video/audio MO from spec.
… Marisa, mgarrish, and I opened #2397 to discuss.
… based on this Marisa created PR #2402.
… but we wanted to discuss with WG first.

Dave Cramer: the fact that we don't have implementations of this weighs heavily to me.
… reluctant to keep this in the spec for that reason.

John Foliot: is anyone aware of popcorn.js?.

Ivan Herman: is there any use-cases for this in the a11y area which requires this?.
… MO has been included in the spec primarily for a11y reasons, although it is useful for other things.

George Kerscher: i agree generally. Audio descriptions for the blind is implemented in TV as a second audio channel that can be turned on.
… don't know the technology behind it (mixer behind the scenes that combines audio channels?).
… don't know that we can break new ground here by adding that feature, but its something people would want.
… what happens with descriptive audio is that the on-screen action is described for the blind person.

John Foliot: http://ncamftp.wgbh.org/rd/hgp/hgp.html.

John Foliot: watch the example at: http://ncamftp.wgbh.org/rd/hgp/hgp.html.

John Foliot: there is a package called popcorn.js. Work done a while back by WGBH, where they were using this js library to use time-stamp markers to activate functions.
… this is a demonstrated use-case, but i'm not aware of anyone who has taken this and run with it.
… not sure that we need this in our spec, as long as we're not frustrating the use of it in epub.

Avneesh Singh: this is removal of 9.3.2 from the RS spec. No one is blocking the creation of such MO files.
… say you have a video going on, and you need description for it, MO doesn't seem like a good solution for this.

Ivan Herman: the largest part of the MO concentrates on associating audio track with text.
… this is not being touched.
… the section that is affected is "embedded-media overlays", i.e. where HTML includes a video track.
… to what John said, if there are js libraries that can do things in a website, they can do it in an epub as well. No problem..
… what you described is a nice tool that can be used to get additional feature while a video is running. This can work in an epub as well.

Wendy Reid: agreed. I think we're just removing something that is creating confusion/opening ourselves up to adding another unimplemented feature.
… i haven't seen this in an epub or in any RS.
… description of video of visually impaired is not done via MO.
… RS would have to implement accessible media-player or epub would have to come with one, larger problem.

John Foliot: More info about accessible media requirements: https://www.w3.org/TR/media-accessibility-reqs/.

Ivan Herman: can we pass a resolution that once editors are ready, that we can merge PR #2402?.

Proposed resolution: Once the edits are finalized, merge PR 2402 and close issue 2397. (Wendy Reid)

Matthew Chan: +1.

Wendy Reid: +1.

Dave Cramer: +1.

GeorgeK: +1.

Ivan Herman: +1.

Brady Duga: +1.

Masakazu Kitahara: +1.

Avneesh Singh: +1.

John Foliot: +1.

Charles LaPierre: +1.

Resolution #1: Once the edits are finalized, merge PR 2402 and close issue 2397.

iherman · 2022-08-26T15:05:06Z

@marisademeglio @mattgarrish we have now a WG resolution to merge this after we all agree on the textual details. From my point of view, it is a go; I do not have any strong feeling about #2402 (comment)

marisademeglio · 2022-08-26T18:27:24Z

@marisademeglio @mattgarrish we have now a WG resolution to merge this after we all agree on the textual details. From my point of view, it is a go; I do not have any strong feeling about #2402 (comment)

Great, no more changes needed then ?

iherman · 2022-08-27T03:25:28Z

@marisademeglio @mattgarrish we have now a WG resolution to merge this after we all agree on the textual details. From my point of view, it is a go; I do not have any strong feeling about #2402 (comment)

Great, no more changes needed then ?

Well, you should wait for @mattgarrish : he seems to have proposed some changes and those seem to be open still. Once that is settled it is a go...

Co-authored-by: Matt Garrish <mattgarrish@users.noreply.github.com>

mattgarrish · 2022-08-30T13:16:51Z

This looks fine to me now but looks like we need to add a biblio entry for MO 3.2 to the RS file. I'm going to merge and manually correct that now.

marisademeglio changed the title ~~Deprecated Media Overlays embedded media~~ Deprecate Media Overlays embedded media Aug 16, 2022

iherman requested review from iherman and mattgarrish August 17, 2022 04:03

iherman requested changes Aug 17, 2022

View reviewed changes

epub33/rs/index.html Outdated Show resolved Hide resolved

iherman mentioned this pull request Aug 17, 2022

Removing embedded video tests w3c/epub-tests#200

Merged

mattgarrish reviewed Aug 17, 2022

View reviewed changes

Update epub33/rs/index.html

d88fc70

Co-authored-by: Ivan Herman <ivan@ivan-herman.net>

mattgarrish reviewed Aug 18, 2022

View reviewed changes

epub33/core/index.html Outdated Show resolved Hide resolved

iherman mentioned this pull request Aug 26, 2022

Deprecate embedded audio or video as targets for Media Overlay <text> reference #2397

Closed

marisademeglio and others added 5 commits August 28, 2022 13:57

Re-add 9.3.2 to MO and formally deprecate it

9d5bb34

Add MO embedded media deprecation to change log

012cf63

Add MO text element embedded media usage note to change log

1b0c5fd

Minor edit to core spec change log entry

b9b06df

Update epub33/core/index.html

76eb127

Co-authored-by: Matt Garrish <mattgarrish@users.noreply.github.com>

mattgarrish merged commit a3cd243 into w3c:main Aug 30, 2022

		<p class="note">Earlier versions of this specification included some information about embedded audio and
		video [epubmediaoverlays-32]. This feature has been deprecated.</p>

Conversation

marisademeglio commented Aug 16, 2022 • edited by pr-preview bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mattgarrish commented Aug 16, 2022

Uh oh!

marisademeglio commented Aug 16, 2022

Uh oh!

iherman left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

iherman commented Aug 17, 2022

Uh oh!

mattgarrish commented Aug 17, 2022

Uh oh!

mattgarrish Aug 17, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mattgarrish Aug 28, 2022

Choose a reason for hiding this comment

Uh oh!

marisademeglio Aug 28, 2022

Choose a reason for hiding this comment

Uh oh!

mattgarrish commented Aug 17, 2022

Uh oh!

iherman commented Aug 18, 2022

Uh oh!

Uh oh!

iherman commented Aug 26, 2022

1. Deprecate embedded audio or video as targets for Media Overlay <text> references.

Uh oh!

iherman commented Aug 26, 2022

Uh oh!

marisademeglio commented Aug 26, 2022

Uh oh!

iherman commented Aug 27, 2022

Uh oh!

mattgarrish commented Aug 30, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

marisademeglio commented Aug 16, 2022 •

edited by pr-preview bot

Loading

mattgarrish Aug 17, 2022 •

edited

Loading

1. Deprecate embedded audio or video as targets for Media Overlay `<text>` references.