You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
EPUB 3.x uses a subset of SMIL 3.0 for Media Overlays.
A number of specialized libraries have adopted EPUB with Media Overlays as a replacement for Daisy with synchronized text and audio. In Northern Europe, some of these libraries have produced a number of accessible comics over the years using images and pre-recorded audio.
Due to the lack of support for <img> in SMIL for EPUB, most of these libraries are using proprietary formats right now and haven't been able to industrialize their processes.
Describe the fix or new feature you propose
In addition to <text> and <audio>, the EPUB 3.4 specification should also reference <img>.
These references to <img> would use a spatial fragment (#xywh) in their src rather than the region attribute available in SMIL.
Can we reasonably assume that we will get two implementations for CR? "These libraries" may not bother changing and/or at least submitting an implementation report... But that is just my Staff Contact self talking :-)
Adoption can only come from a good partnership between a significant set of accessible content producers, and innovative reading system developers.
Today, large producers are Daisy members, boosted by automatic alignment AI-tools. As for innovative reading system developers, we have names.
I agree with Hadrien that using spatial fragments rather than SMIL region (an id/idref indirection) is much simpler re. technical implementation.
If not, this could be "incubated" as long as we don't outright reject EPUB files using <img> in SMIL.
Getting the schema epubcheck uses updated to allow img probably wouldn't be that hard. We've allowed metadata in the past prior to it being standardized in the specifications.
It'd be easy to put together a CG note making the case for img, plus what restrictions there might be on its use, and then see how it goes first before committing this to the spec.
I've reached out to the maintainers of an automated alignment tool (Storyteller: https://storyteller-platform.gitlab.io/storyteller/) that also provides mobile (Android and iOS) and Web reading apps and they're willing to support this feature.
If we also count Thorium Mobile, we could have two implementations available for the usual rec track process.
@Hadrien: SMIL allows references to text, audio, and video … currently in epub we only use text. It references IDs. We also use the audio element without a time fragment
<ivan> strictly speaking, 'img' and the others are defined as "alias" of 'ref'
@Hadrien: a number of specialized libraries have experimented with narrated comics using pre recorded audio … the current spec doesn't provide a good way to do this, so the current experiments are in proprietary formats … I have been investigating how much work it would be to support this in EPUB … we would need a way to specify a fragment of an image … we can either use regions or spacial fragments, which I think is much easier … we already use this in a few other pages … there is interest for this, there is no format to do this either in DAISY or EPUB … it is a pretty small gap to allow this feature in EPUB
@ivan: for the record, the SMIL specification predates the specification for media fragments … I agree that in 2025 we should use media fragments … are there any prospects of real epub readers that would implement this … we need two independent implementations for a new feature
duga: I think there were RS that were offering implementations
Hadrien: there is support in readium to support this, once it is in the Readium toolkit it is pretty easy to support … Thorium will support this … I have asked the maintainer of StoryTeller to support this also … it is mostly used by people who do this on their own, who offer this to their library patrons … I cannot commit on the timeline for Storyteller but Thorium will support it this year
@ivan: Thorium is clear, Storyteller, from the official point of view, the AC would ask for an EPUB Reader like Thorium … to really implement and use this … Colibrio would be fine, it doesn't need to a big player
Hadrien: I expect specialized libraries to use this
@ivan: if I have an EPUB that uses this, I should be able to read it in another system
duga: what are the implications for existing implementations for Read Aloud? … could a publisher just reference portions of an image and not have the audio overlay? The image would then be the synced media? … is that a potential use here? are we worried about it
Hadrien: I don't think it would be a good thing to overload text … wrapping an image in text is a bad thing for accessibility … the benefit of having image, is opening the door to image based content … it also opens the door to something highly visual in nature, plus audio, and a textural element … it opens the door to more things without negatively impacting the current system … media support in EPUB remains more limited than we wish … even with the current take on media overlays things could be better with RS support
GeorgeK: …read aloud means to me, taking the text and playing it into the system, different than audio overlays … if we have a fixed image, and there are four audio clips in parallel with that image … the media fragment shifts focus on the image, would you have something like … two people in an image, a focus on one person, then a shift to the other person like a conversation
Hadrien: that is one way you could implement it … currently they can zoom into a panel and play the audio from that panel
CharlesL: about implementation concerns, I don't think we need explicitly two reading system implementations … just two implementations of the technology so a tool would be fine … if you had a screen reader enabled, and you have the textural part overlayed on audio, I'm trying to understand
GeorgeK: I think the SMIL implementation would play and the screen reader would be silent … if you pause the screen reader
duga: when I said Read Aloud I meant reading the included audio … to Hadrien 's point, we have a flow for children's books … the publisher adds audio to existing images/media … the reading system then takes that and plays it … is it expected now that Reading Systems must synchronize the audio? … are we breaking children's books by enabling this?
Hadrien: I don't think this is very likely. Producing an audio layer is much more work than including text … the kind of org that will produce this media overlay is one that cares deeply about accessibility … the cost is much higher to produce them
duga: these are not text to speech books, but with media overlays
DaleRogers: I wonder if there would be a smooth transition from one image fragment to another?
Hadrien: this is not about controlling visual presentation. A lot of movement can be hard for people with certain sensitivities
<Zakim> wendyreid, you wanted to react to DaleRogers
wendreid: for example, a lot of RS have settings for page turn behavior. I think this would fall in that same class … especially on mobile, where you can have multiple page turns actions
@ivan: we may be getting into areas that are not covered by this proposal … the original SMIL spec refers only to the ref element … the spec doesn't say anything about transitions … it is always a trick discussion about the AC and how they accept implementation reports … I think beyond the legal terminology for the process, they ask … if I create a book with this feature, am I stuck with only one reading system? … if we decide to put this in the spec, we mark it as an at risk feature … so if we don't get enough implementation we can easily remove it at the end without triggering reviews
mgarrish: you want to see two similar implementations, like two Reading Systems and two authoring systems … is this incubation material? Where are we going with this? … is it at the point we want to implement this? … what are the knock on implications? I'd like to be sure of this before we add this to the implementation at the end of the process … maybe the at risk implementation or a cg note would be the way to do this
GeorgeK: what is the relationship of this technique to other ways to make comics? … it seems like an alternative to other developments for comics
Hadrien: this is directly a response to a number of specialized libraries have implemented especially in Northern Europe
<gman> Hadrien the proprietary use cases in Scandinavia are comics only?
Hadrien: they have produced files that do this, and everyone does it slightly differently … I look at what people are using, and what people need … this is quite specialized, mostly about adaptive content, but it is missing … this is different than adding something no one asked for
GeorgeK: I have seen the US Library of Congress dabbling with this too
Hadrien: this is one way we can support this
@ivan: I don't see this as an incubation matter because we don't have to create new vocabulary … we are just allowing a tiny bit of what is already in SMIL in EPUB … it is defined already, we just allow it here
CharlesL: I get your point about two implementations on one side or another, and the … Readium implementation could go out to multiple readers … in a comic, could you have it read each panel in the correct order … if you click a Read button in the RS, would it do that? Is this a potential use case
Hadrien: yes, you could have a panel with a textural and/or an audio equivalent
gman: another potential use case is language acquisition/application. Having two audio setups in a Manga
gman: you could switch between the translations … this kind of use case could be studied a bit further
Hadrien: I think this is different. You are talking about having two images in two languages?
gman: SMIL doesn't support multiple track?
Hadrien: not as we use it, no
wendyreid: that gets into something we are also discussing, parallelized content … we may have an open Issue about that where we do want to talk about language acquisition
<gautierchomel> Parallels content is a discussion at #2829
wendyreid: this is where text to speech might be better than SMIL so they can choose their narration voice … SMIL is fixed, referencing specific content … it sounds to me like we want to keep exploring this … it might not be a huge change to the spec … the major question is if we have the right amount of implementations … I'm concerned about industry uptake, if it will be supported by most major reading systems … we need to make sure that there is more than one reading platform available … just for the sake of user choice
duga: I'm not convinced, I'm still worried about breaking existing workflows. … there is nothing tying this to specific content, and I would need more reassurance that people … won't start making children's books that would break on current reading systems
wendreid: we have a greenlight to keep studying this. Can we get sample files to study?
Hadrien: Nota has public examples. We might get the files in JSON
@ivan: I'd like to explore what it means editiorialy, can Hadrien do a draft PR for the author and RS specs?
Hadrien: I can do this, but not immediately, maybe in February
<gman> Hadrien on one of these sleepless nights
wendyreid: we will carry over our last topic, selectors, next week
It is certainly different, and it is remarkably unlikely that it works out of the box today, so it is requiring a change to existing reading system implementations. There is the added question of how styles are applied to the area of the image that is synced. For text we use CSS classes and allow the publisher to style it, would there be something similar for synced images?
In any case, I would want explicit buy in from existing reading systems that they either don't care that they may receive incompatible MO books, or that they are willing to implement image sync. Unless there is a plan to restrict this feature to some new, experimental content.
This is a more general issue, not related exclusively to <img>: what exactly is our expectation when we introduce a new feature that require change/addition to the RS implementation. The same issue can be raised with regard to roll layout.
For roll, a somewhat larger RS community is (we hope) interested in it (e.g., the Japanese community), with lots of interested parties. I cannot be sure at the moment that the same is true for <img>; hence my proposal to add it as an 'at risk' feature, which will make it in the final spec provided we get some significant RS interest in implementing it, and that would be explicitly highlighted in the spec. (I.e., if, when moving the document to Rec (i.e., at the last moment) we feel we do not have enough implementations, we can just simply remove it from the document without going through yet another formal CR snapshot publication.)
The same issue can be raised with regard to roll layout.
But we came into the revision with this as an objective, so anyone looking at the charter scope knew this was in play.
Changing SMIL to add spatial focus on images synchronized with text and audio wasn't on our list of things to do and is popping up at the same time that we're starting to turn our focus to wrapping up and getting to CR. We're compressing the timeline to ensure this will work without causing unexpected issues and that's always a problematic way to do things.
The same issue can be raised with regard to roll layout.
But we came into the revision with this as an objective, so anyone looking at the charter scope knew this was in play.
Changing SMIL to add spatial focus on images synchronized with text and audio wasn't on our list of things to do and is popping up at the same time that we're starting to turn our focus to wrapping up and getting to CR. We're compressing the timeline to ensure this will work without causing unexpected issues and that's always a problematic way to do things.
The same issue can be raised with regard to roll layout.
And we worked closely with the existing stake holders, both implementors and content creators, to make sure we didn't break them, either retroactively (existing content) or going forward (they could support the new approach easily).
But this is also a different case. I may be simply misunderstanding the proposal (and honestly I don't know much about SMIL itself), but it looks like this would offer a new way to implement a feature that already exists in epub, using the same general technologies. So a RS that supports children's books with media overlays using the epub subset of SMIL, would suddenly no longer fully support children's books created using the epub subset of SMIL. There are no reading systems that are implementing webcomics using epub, since there is no existing capability to do that using (valid) epub, so there is no way to really break that (and even then we made sure to maintain compatibility with the epub-ish format that was being used).
Step 2 is a nightmare. A publisher might put a full paragraph of invisible text over the existing image, or they might place it visibly and delete the text in the page. It might be a full paragraph, it might be sentence by sentence, or it might be word for word. It's really hard to get it right, since the layout will break across reading systems. For instance, a common issue is to implement this for Apple Books and use Times New Roman as the font, which is fine on an iPad. Now you take that same book to an Android device and suddenly you get random line wrapping so the words don't properly align with the text. There are workarounds for that (for instance, there is an entire font with the same metrics as TNR), and over the years these have been adopted by reading systems to display media overlay books.
This has very little to do with the discussion here IMO.
It's true that without font or glyph embedding, there's a risk that a FXL will render very poorly, but this is tied to the nature of pre-paginated content where everything is placed with absolute positioning in HTML or SVG.
It's also worth pointing out that this effort doesn't really result in an accessible publication either. In many cases, these files are produced with words or letters positioned individually from one another and not necessarily in the right reading order. At best, you get group of words but I've seen very little FXL files so far where the DOM is truly useful for a screen reader or read aloud feature (using TTS) without additional work.
The concern is, since step 2 is hard, with images publishers could just stop doing it. This is fine, and might even be an improvement, since the text layer is rarely suitable for use in accessibility outside of aligned audio playback. However, it breaks almost all implementations of MO in existing reading systems. The worry is that 1 reading system ends up supporting this, then publishers start providing this (now perfectly legal) content, and it breaks all the other implementations, which is definitely a thing that happens (see previous Times New Roman issue).
If you want text highlighting, you'll definitely need to include real text in your file and author styles for this highlight.
What you're describing as a risk is already possible today without the addition of <img>. An author could easily add empty elements, reference them with an id and sync an audio fragment with that empty element.
On the other hand, support for <img> opens the door to new types of publications that we cannot handle today in EPUB such as:
highly illustrated content with synchronized audio
highly illustrated content with synchronized text (where the text is not present in the image or hand-written)
highly illustrated content with both synchronized text and audio
With EAA, specialized libraries are re-positioning themselves to mostly focus on publications that won't become natively accessible (because they fall under specific exemptions such as disproportionate burden or fundamental alteration). With this addition, we would give them the mean to create accessible versions of such publications.
I reached out to Nota and asked them for a publicly available example since the one I had from them is no longer available.
Here's a video from L3I, a research lab based in France focusing on the automated creations of accessible comics that illustrates what the experience could sound like: https://www.youtube.com/watch?v=W8jWBGkZvKM
As mentioned in the call, several of our members at EDRLab would like to produce such content, but they're missing a format capable of handling such features.
I know that several specialized libraries would like to focus on this topic for future Daisy board meetings later this year (June in Oslo, November in Tokyo).
This is based on EPUB with Media Overlays, where the regions are drawn using absolute-positioned <div> and referenced using their ids in SMIL through a <text> element. There's no text per se, since it's all hand drawn and included in the image (it's a good illustration of what I answered to @bduga in #2883 (comment)).
In Apple Books, you can listen continuously, spread-by-spread or page-by-page to the audio narration. In Thorium Desktop, you can listen continuously or utterance-by-utterance, with a yellow overlay displayed over the region being read.
This narration is limited to text available on the page (the previous example from L3I includes description as well).
Because <text> is already used to point to an image fragment, it can't be used for a textual alternative right now, whereas <img> would be a much better fit for what is done here.
From a reading system perspective, it's also much harder to extract the position and size of a region drawn like this in HTML than the equivalent using a media fragment.
@HadrienGardeur I am a bit worried that we are having parallel lots of discussions on the thread, which leads nowhere close to a decision...
I do not think anybody needs convincing that this is a useful feature, and we also know now that there is a real interest for it coming from different places. It is also clear that adding it to the SMIL format is, specification-wise, trivial, because it is just another "alias" to SMIL's <ref> element, just like <text> is. We also agree, I believe, that it is implementable by reading systems: Readium has proven it can be done, and we can also trust that the engineers at Apple or Google or Kobo are in position to do it if they decide to do so. Finally, we can also agree that <img> is not the panacea, there are a number of issues in potential applications of SMIL which remain open, which are hard (e.g., in the accessibility area) which are not solved with its introduction; there is no reasons to discuss them here.
I believe the only issue that stands in the way of adding this feature to EPUB 3.4 is how to tackle the effects on the ecosystem at large, where major RS developers may be lagging in their implementations (for whatever reasons), creating distortions on the market. @bduga may formulate it a bit more clearly, but I believe that is the essence of the problem. And the fact that Readium implements it and, as a consequence, there are a number of reading systems out there that will offer the feature to their clients is not the complete answer (with all the due respect to Readium!). We would need the reactions of some of the other, major RS-s from the US or Japan (all represented on the WG) on whether they would ignore <img> or not. I believe that is what we have to concentrate on.
@iherman in essence, this is a chicken and egg issue.
The following questions come to mind:
Could we allow specialized libraries and reading systems who would like to move forward with this feature to do so without being incompatible with the specification?
If so, how could this be handled? As a note that could become part of a revision later on (like we did with EPUB FXL) ?
How could we handle epubcheck in particular (I know @mattgarrish has some ideas on this one) ?
I'm a little doubtful that we'll gain the kind of support that you're looking for in the next 11 months (major RS rarely give us this type of commitment on any feature, plus Media Overlays are not implemented in some RS). But I also don't think that we can leave this one opened without a path forward, otherwise this type of innovation will need to be handled in a different format than EPUB.
SueNeu: Hadrien, do you want to tell us what is happening in this issue so far?
Hadrien: Just a headsup that I opened another SMIL-related issue, not directly related but of interest … SMIL for images is something we've discussed, I proposed this as I think it's a missing feature … we lack the ability to reference an image or fragment of an image and link that to text or audio for that fragment … and role, so we can identify the fragment … I've shared examples, one from Bibliotheque Simon Roland(?), human recorded audio for the captions and speech bubbles of a comic … they had to use the text element, wasn't ideal … because they used text, they couldn't also provide the textual alternative … audio equivalent of handwritten text … it demonstrates what Brady might be concerned about, but if its possible today, it may not change things … if they're doing it already today, it's as a way to make this work where there's a gap … there is interest from other groups as well … there is a paper published by a French university, where they generated a script for comics using computer vision, genAI, they can generate a description and extract the text … create fragments, they generate text, then audio based on the text … shows what more modern tools can do … not everyone, there's huge interest from DAISY, and there's a technical conference after the DAISY meeting in June … there's interest, but it's currently missing … pointed out one way to do it, adding an existing SMIL feature to our subset … discussion with Ivan, do we put it in the spec and we're not sure if it gets adopted … MO is not well supported across the board, but it could also be a missed opportunity … where there are gaps, people will invent their own formats … we could consider writing a note or just allowing EPUBCheck to not flag it
Hadrien: there are many ways to do something about it, doing nothing would be a waste
ivan: I will also repeat a bit of what was discussed on the issue, I don't think it is worth spending our time on the technical value and usefulness … as far as I'm concerned, this is a useful feature … that's not a question, in my memory, Brady raised issues that are not necessarily technical problems, but what do we do with a feature that wasn't in the charter, which came in and we have no idea whether big publishers will be interested … how will it distort the market?
ivan: This is what we need to discuss … it's not about the technical value, it's an easy extension … there's no disagreement there, the bigger question is the "market"
duga: To elaborate more on my concern, we have an existing ecosystem of children's books using SMIL, the way that wordflow works is taking an image-only book, add a text layer, get word by work highlighting
duga: If we add this, it's a new flow for generating children's books … it's true you can make bad content, you can avoid putting text in the divs you're highlighting, there's a lot of features built into the text layer, it works. … we're saying that by making this change you can just do images with attached audio … most of the children's books aren't good for accessibility for other reasons … my worry is we're saying you can do either audio or text, and someone comes to the spec and sees two options … an author can come along and see "oh I can do this with images only", but platforms like Play Books won't support that … we should at least be upfront about the new way of doing the books, is that something you want to see in this version … I'm not disagreeing that this is a good feature, but we have a method, and an ecosystem built around that method, there could be an impact
<SueNeu> ack: Hadrien
Hadrien: Going back to Ivan's question and Brady, we focus way too much on a few players, and the ecosystem is much bigger than just a few companies … we have orgs moving off DAISY and going to EPUB with MO … reflow files with MO, which isn't even supported in Apple Books. … focusing on just a few players that don't do these things, and missing that there are players that are doing this work, and making these advances … we are seeing passionate users building tools and platforms that support these new things … instead of focusing on big players, should we also support the more specialized libraries, specialized use cases
wendyreid: We need to start having some discussions about who are we concerned with? … there are more niche or smaller players in the EPub market that want advancement of the spec … using web technologies. They do not have the same use cases … that we have been focusing on. Because we are primarily focused on larger vendors … most of the innovation is coming from smaller platforms and specialized libraries … including regional use cases, like web comics … we need to look more closely at these smaller players … even if we don't expect the bigger players to adopt it quickly … if there is a reading system, say in Sweden, that would benefit from this we should do it
<ivan> s/ack wendyreid//
wendyreid: to @duga 's point, we do need to reach out and create more best practices around it
ack: mgarrish
mgarrish: It's important to take everyone's concerns, my concern has been expressed by Ivan and Brady, we might be squeezing this in right before CR … yes it's part of SMIL, there's a number of details to work out, it's not that this doesn't have value, but is this the right time … or is this a case where we do this through EPUBCheck and notes, so bigger players have a chance to voice this … this wasn't in the charter, there was no "notice" of this change, everyone agrees this is good, are we doing it the right way
Hadrien: I'd have to read the charter again, but I think we had accessibility and comics, so this might fit, the work on annotations is bigger than this … I'm not advocating to push this in the spec, we can do a note, a future revision, as long as it doesn't get flagged in EPUBCheck … it raises questions about how we handle revisions, we don't know if there will be a charter after this one … the window for doing anything feels narrow, sometimes we know things in advance, and other times we talk to people and learn something bring those things back to the spec
<ivan> quoting from the charter: "Finally, the Working Group will also
incubate issues without necessarily aiming at the creation of final Recommendations during the lifetime of this
charter. "
DaleRogers: I'm listening to this and it makes me wonder, what is the purpose of a spec? I guess it goes back to the charter … I take it as we said we'd do this, and we're telling people "this is how you should do it" not that "you must do this"
mgarrish: Don't want to veer too far off course, when we do revisions, we need to be sure up front what we're doing. We should talk to people, but we need to do it more in advance, we need some guardrails … we're already looking at doing horizontal reviews, but now we're looking at a significant change, we need a plan for these things … it's not a critique of bringing this up, just that this always seems to happen and we try to accommodate where we want, but we need some organization in the future
ivan: Putting on my staff contact hat, I was looking at the charter, we do these things, but we also incubate things without necessarily committing them to the final version of the rec … we gave ourselves an escape hatch … I would personally, staff contact hat off, I'd be happy to add it with a warning, I agree it's not a major work, not like the layout issue … we can put it in as at risk, it doesn't mean it has to be implemented, we use the remaining year to reach out to learn about the community's reaction to it … if we get positive feedback, we leave it in, if we don't, we can remove it. … that's do-able, we have the ability to roll back
SueNeu: People see the utility of this, concerns about process, whether this is in scope, the current proposal are to add this to the spec with a warning, removeable if needed, or just add a note and change EPUBCheck to allow this.
ivan: What I think is to produce a separate note describing this feature
Hadrien: Correct, a separate document
ivan: If the feedback from the community comes out negative, we can remove from the spec and create a note, so it doesn't disappear
Hadrien: This sounds like a good process to me
Proposed: We will add SMIL in images to EPUB 3,4 spec as at risk, and then move it to a note if it comes out of the final spec?
<wendyreid> +1
<SueNeu> +1
<Hadrien> +1
<gman> +1
<toshiakikoike> +1
<gautierchomel_> +1
<DaleRogers> +1
<shiestyle> 0
<duga> 0
<MasakazuKitahara> +1
<ivan> +
<ajellinek> +1
<ivan> +1
RESOLUTION: We will add SMIL in images to EPUB 3,4 spec as at risk, and then move it to a note if it comes out of the final spec
ivan: No good thing goes unpunished, it would be good if you produced the PR to add this into the document
Hadrien: Each element has its own subsection, so one for images?
ivan: Yes, and we'll support editorially
duga: And the RS spec for behaviour
Hadrien: I'll look it up and figure out where things need to go
@reidmore-online I see that this is marked as "Deferred" but we did had a vote on it:
RESOLUTION: We will add SMIL in images to EPUB 3,4 spec as at risk, and then move it to a note if it comes out of the final spec
After this vote, I started looking into the Media Overlay sections of our specs and that's how I ended up opening a lot of different issues. For the sake of making things easier for 3.4, I'm fine pushing this to 3.5 but let's not start entirely from scratch since we already had a vote on it.
I'm fine pushing this to 3.5 but let's not start entirely from scratch since we already had a vote on it.
The issue is not closed. Deferred simply means that it is not the subject of 3.4, ie, this issue is not relevant to the CR transition request on the way.
Activity
sueneu commentedon Jan 13, 2026
This would be great for making comics accessible. I would love to see it.
Is there enough adoption of SMIL throughout reading systems to support making this change? I thought it is unevenly implemented at this time.
iherman commentedon Jan 13, 2026
Can we reasonably assume that we will get two implementations for CR? "These libraries" may not bother changing and/or at least submitting an implementation report... But that is just my Staff Contact self talking :-)
HadrienGardeur commentedon Jan 13, 2026
If not, this could be "incubated" as long as we don't outright reject EPUB files using
<img>in SMIL.In Readium we'll implement support for this in at least two of our codebases in Q1 (Swift and Go, they don't share a single line of code).
sueneu commentedon Jan 13, 2026
Reaching out to a friend who might know of another implementation. I’ll post it here if he does.
llemeurfr commentedon Jan 13, 2026
Adoption can only come from a good partnership between a significant set of accessible content producers, and innovative reading system developers.
Today, large producers are Daisy members, boosted by automatic alignment AI-tools. As for innovative reading system developers, we have names.
I agree with Hadrien that using spatial fragments rather than SMIL
region(an id/idref indirection) is much simpler re. technical implementation.HadrienGardeur commentedon Jan 13, 2026
Here's an extended example based on the work that I've done for Readium and more specifically the second panel of the first page in this example:
Where
page1.html#panel2-descriptionwould be:mattgarrish commentedon Jan 14, 2026
Getting the schema epubcheck uses updated to allow img probably wouldn't be that hard. We've allowed metadata in the past prior to it being standardized in the specifications.
It'd be easy to put together a CG note making the case for img, plus what restrictions there might be on its use, and then see how it goes first before committing this to the spec.
HadrienGardeur commentedon Jan 14, 2026
I've reached out to the maintainers of an automated alignment tool (Storyteller: https://storyteller-platform.gitlab.io/storyteller/) that also provides mobile (Android and iOS) and Web reading apps and they're willing to support this feature.
If we also count Thorium Mobile, we could have two implementations available for the usual rec track process.
mattgarrish commentedon Jan 14, 2026
Maybe, but that's aiming for a low bar. We have a history of rushing changes that never pan out.
HadrienGardeur commentedon Jan 14, 2026
Fair point. Let's discuss this in a future call (tomorrow?) but I'm not against the idea of a CG note if we can avoid the epubcheck trap.
iherman commentedon Jan 15, 2026
This was discussed during the pmwg meeting on 15 January 2026.
View the transcript
<img> in SMIL - w3c/epub-specs#2883
@Hadrien: SMIL allows references to text, audio, and video… currently in epub we only use text. It references IDs. We also use the audio element without a time fragment
<ivan> strictly speaking, 'img' and the others are defined as "alias" of 'ref'
@Hadrien: a number of specialized libraries have experimented with narrated comics using pre recorded audio… the current spec doesn't provide a good way to do this, so the current experiments are in proprietary formats
… I have been investigating how much work it would be to support this in EPUB
… we would need a way to specify a fragment of an image
… we can either use regions or spacial fragments, which I think is much easier
… we already use this in a few other pages
… there is interest for this, there is no format to do this either in DAISY or EPUB
… it is a pretty small gap to allow this feature in EPUB
@ivan: for the record, the SMIL specification predates the specification for media fragments… I agree that in 2025 we should use media fragments
… are there any prospects of real epub readers that would implement this
… we need two independent implementations for a new feature
duga: I think there were RS that were offering implementations
Hadrien: there is support in readium to support this, once it is in the Readium toolkit it is pretty easy to support
… Thorium will support this
… I have asked the maintainer of StoryTeller to support this also
… it is mostly used by people who do this on their own, who offer this to their library patrons
… I cannot commit on the timeline for Storyteller but Thorium will support it this year
@ivan: Thorium is clear, Storyteller, from the official point of view, the AC would ask for an EPUB Reader like Thorium… to really implement and use this
… Colibrio would be fine, it doesn't need to a big player
Hadrien: I expect specialized libraries to use this
@ivan: if I have an EPUB that uses this, I should be able to read it in another systemduga: what are the implications for existing implementations for Read Aloud?
… could a publisher just reference portions of an image and not have the audio overlay? The image would then be the synced media?
… is that a potential use here? are we worried about it
Hadrien: I don't think it would be a good thing to overload text
… wrapping an image in text is a bad thing for accessibility
… the benefit of having image, is opening the door to image based content
… it also opens the door to something highly visual in nature, plus audio, and a textural element
… it opens the door to more things without negatively impacting the current system
… media support in EPUB remains more limited than we wish
… even with the current take on media overlays things could be better with RS support
GeorgeK: …read aloud means to me, taking the text and playing it into the system, different than audio overlays
… if we have a fixed image, and there are four audio clips in parallel with that image
… the media fragment shifts focus on the image, would you have something like
… two people in an image, a focus on one person, then a shift to the other person like a conversation
Hadrien: that is one way you could implement it
… currently they can zoom into a panel and play the audio from that panel
CharlesL: about implementation concerns, I don't think we need explicitly two reading system implementations
… just two implementations of the technology so a tool would be fine
… if you had a screen reader enabled, and you have the textural part overlayed on audio, I'm trying to understand
GeorgeK: I think the SMIL implementation would play and the screen reader would be silent
… if you pause the screen reader
duga: when I said Read Aloud I meant reading the included audio
… to Hadrien 's point, we have a flow for children's books
… the publisher adds audio to existing images/media
… the reading system then takes that and plays it
… is it expected now that Reading Systems must synchronize the audio?
… are we breaking children's books by enabling this?
Hadrien: I don't think this is very likely. Producing an audio layer is much more work than including text
… the kind of org that will produce this media overlay is one that cares deeply about accessibility
… the cost is much higher to produce them
duga: these are not text to speech books, but with media overlays
DaleRogers: I wonder if there would be a smooth transition from one image fragment to another?
Hadrien: this is not about controlling visual presentation. A lot of movement can be hard for people with certain sensitivities
<Zakim> wendyreid, you wanted to react to DaleRogers
wendreid: for example, a lot of RS have settings for page turn behavior. I think this would fall in that same class
… especially on mobile, where you can have multiple page turns actions
@ivan: we may be getting into areas that are not covered by this proposal… the original SMIL spec refers only to the ref element
… the spec doesn't say anything about transitions
… it is always a trick discussion about the AC and how they accept implementation reports
… I think beyond the legal terminology for the process, they ask
… if I create a book with this feature, am I stuck with only one reading system?
… if we decide to put this in the spec, we mark it as an at risk feature
… so if we don't get enough implementation we can easily remove it at the end without triggering reviews
mgarrish: you want to see two similar implementations, like two Reading Systems and two authoring systems
… is this incubation material? Where are we going with this?
… is it at the point we want to implement this?
… what are the knock on implications? I'd like to be sure of this before we add this to the implementation at the end of the process
… maybe the at risk implementation or a cg note would be the way to do this
GeorgeK: what is the relationship of this technique to other ways to make comics?
… it seems like an alternative to other developments for comics
Hadrien: this is directly a response to a number of specialized libraries have implemented especially in Northern Europe
<gman> Hadrien the proprietary use cases in Scandinavia are comics only?
Hadrien: they have produced files that do this, and everyone does it slightly differently
… I look at what people are using, and what people need
… this is quite specialized, mostly about adaptive content, but it is missing
… this is different than adding something no one asked for
GeorgeK: I have seen the US Library of Congress dabbling with this too
Hadrien: this is one way we can support this
@ivan: I don't see this as an incubation matter because we don't have to create new vocabulary… we are just allowing a tiny bit of what is already in SMIL in EPUB
… it is defined already, we just allow it here
CharlesL: I get your point about two implementations on one side or another, and the
… Readium implementation could go out to multiple readers
… in a comic, could you have it read each panel in the correct order
… if you click a Read button in the RS, would it do that? Is this a potential use case
Hadrien: yes, you could have a panel with a textural and/or an audio equivalent
gman: another potential use case is language acquisition/application. Having two audio setups in a Manga
gman: you could switch between the translations
… this kind of use case could be studied a bit further
Hadrien: I think this is different. You are talking about having two images in two languages?
gman: SMIL doesn't support multiple track?
Hadrien: not as we use it, no
wendyreid: that gets into something we are also discussing, parallelized content
… we may have an open Issue about that where we do want to talk about language acquisition
<gautierchomel> Parallels content is a discussion at #2829
wendyreid: this is where text to speech might be better than SMIL so they can choose their narration voice
… SMIL is fixed, referencing specific content
… it sounds to me like we want to keep exploring this
… it might not be a huge change to the spec
… the major question is if we have the right amount of implementations
… I'm concerned about industry uptake, if it will be supported by most major reading systems
… we need to make sure that there is more than one reading platform available
… just for the sake of user choice
duga: I'm not convinced, I'm still worried about breaking existing workflows.
… there is nothing tying this to specific content, and I would need more reassurance that people
… won't start making children's books that would break on current reading systems
wendreid: we have a greenlight to keep studying this. Can we get sample files to study?
Hadrien: Nota has public examples. We might get the files in JSON
@ivan: I'd like to explore what it means editiorialy, can Hadrien do a draft PR for the author and RS specs?Hadrien: I can do this, but not immediately, maybe in February
<gman> Hadrien on one of these sleepless nights
wendyreid: we will carry over our last topic, selectors, next week
4 remaining items
iherman commentedon Jan 16, 2026
This is a more general issue, not related exclusively to
<img>: what exactly is our expectation when we introduce a new feature that require change/addition to the RS implementation. The same issue can be raised with regard to roll layout.For roll, a somewhat larger RS community is (we hope) interested in it (e.g., the Japanese community), with lots of interested parties. I cannot be sure at the moment that the same is true for
<img>; hence my proposal to add it as an 'at risk' feature, which will make it in the final spec provided we get some significant RS interest in implementing it, and that would be explicitly highlighted in the spec. (I.e., if, when moving the document to Rec (i.e., at the last moment) we feel we do not have enough implementations, we can just simply remove it from the document without going through yet another formal CR snapshot publication.)mattgarrish commentedon Jan 16, 2026
But we came into the revision with this as an objective, so anyone looking at the charter scope knew this was in play.
Changing SMIL to add spatial focus on images synchronized with text and audio wasn't on our list of things to do and is popping up at the same time that we're starting to turn our focus to wrapping up and getting to CR. We're compressing the timeline to ensure this will work without causing unexpected issues and that's always a problematic way to do things.
iherman commentedon Jan 16, 2026
Good point.
bduga commentedon Jan 16, 2026
And we worked closely with the existing stake holders, both implementors and content creators, to make sure we didn't break them, either retroactively (existing content) or going forward (they could support the new approach easily).
But this is also a different case. I may be simply misunderstanding the proposal (and honestly I don't know much about SMIL itself), but it looks like this would offer a new way to implement a feature that already exists in epub, using the same general technologies. So a RS that supports children's books with media overlays using the epub subset of SMIL, would suddenly no longer fully support children's books created using the epub subset of SMIL. There are no reading systems that are implementing webcomics using epub, since there is no existing capability to do that using (valid) epub, so there is no way to really break that (and even then we made sure to maintain compatibility with the epub-ish format that was being used).
HadrienGardeur commentedon Jan 17, 2026
This has very little to do with the discussion here IMO.
It's true that without font or glyph embedding, there's a risk that a FXL will render very poorly, but this is tied to the nature of pre-paginated content where everything is placed with absolute positioning in HTML or SVG.
It's also worth pointing out that this effort doesn't really result in an accessible publication either. In many cases, these files are produced with words or letters positioned individually from one another and not necessarily in the right reading order. At best, you get group of words but I've seen very little FXL files so far where the DOM is truly useful for a screen reader or read aloud feature (using TTS) without additional work.
If you want text highlighting, you'll definitely need to include real text in your file and author styles for this highlight.
What you're describing as a risk is already possible today without the addition of
<img>. An author could easily add empty elements, reference them with anidand sync an audio fragment with that empty element.On the other hand, support for
<img>opens the door to new types of publications that we cannot handle today in EPUB such as:With EAA, specialized libraries are re-positioning themselves to mostly focus on publications that won't become natively accessible (because they fall under specific exemptions such as disproportionate burden or fundamental alteration). With this addition, we would give them the mean to create accessible versions of such publications.
HadrienGardeur commentedon Jan 17, 2026
I reached out to Nota and asked them for a publicly available example since the one I had from them is no longer available.
Here's a video from L3I, a research lab based in France focusing on the automated creations of accessible comics that illustrates what the experience could sound like: https://www.youtube.com/watch?v=W8jWBGkZvKM
Additional resources are available as well:
As mentioned in the call, several of our members at EDRLab would like to produce such content, but they're missing a format capable of handling such features.
I know that several specialized libraries would like to focus on this topic for future Daisy board meetings later this year (June in Oslo, November in Tokyo).
HadrienGardeur commentedon Feb 2, 2026
Here's another example that Bibliothèque Sonore Romande (cc @sjabsr) presented last year:
This is based on EPUB with Media Overlays, where the regions are drawn using absolute-positioned
<div>and referenced using their ids in SMIL through a<text>element. There's no text per se, since it's all hand drawn and included in the image (it's a good illustration of what I answered to @bduga in #2883 (comment)).In Apple Books, you can listen continuously, spread-by-spread or page-by-page to the audio narration. In Thorium Desktop, you can listen continuously or utterance-by-utterance, with a yellow overlay displayed over the region being read.
This narration is limited to text available on the page (the previous example from L3I includes description as well).
Because
<text>is already used to point to an image fragment, it can't be used for a textual alternative right now, whereas<img>would be a much better fit for what is done here.From a reading system perspective, it's also much harder to extract the position and size of a region drawn like this in HTML than the equivalent using a media fragment.
iherman commentedon Feb 2, 2026
@HadrienGardeur I am a bit worried that we are having parallel lots of discussions on the thread, which leads nowhere close to a decision...
I do not think anybody needs convincing that this is a useful feature, and we also know now that there is a real interest for it coming from different places. It is also clear that adding it to the SMIL format is, specification-wise, trivial, because it is just another "alias" to SMIL's
<ref>element, just like<text>is. We also agree, I believe, that it is implementable by reading systems: Readium has proven it can be done, and we can also trust that the engineers at Apple or Google or Kobo are in position to do it if they decide to do so. Finally, we can also agree that<img>is not the panacea, there are a number of issues in potential applications of SMIL which remain open, which are hard (e.g., in the accessibility area) which are not solved with its introduction; there is no reasons to discuss them here.I believe the only issue that stands in the way of adding this feature to EPUB 3.4 is how to tackle the effects on the ecosystem at large, where major RS developers may be lagging in their implementations (for whatever reasons), creating distortions on the market. @bduga may formulate it a bit more clearly, but I believe that is the essence of the problem. And the fact that Readium implements it and, as a consequence, there are a number of reading systems out there that will offer the feature to their clients is not the complete answer (with all the due respect to Readium!). We would need the reactions of some of the other, major RS-s from the US or Japan (all represented on the WG) on whether they would ignore
<img>or not. I believe that is what we have to concentrate on.HadrienGardeur commentedon Feb 2, 2026
@iherman in essence, this is a chicken and egg issue.
The following questions come to mind:
I'm a little doubtful that we'll gain the kind of support that you're looking for in the next 11 months (major RS rarely give us this type of commitment on any feature, plus Media Overlays are not implemented in some RS). But I also don't think that we can leave this one opened without a path forward, otherwise this type of innovation will need to be handled in a different format than EPUB.
iherman commentedon Feb 5, 2026
This was discussed during the pmwg meeting on 05 February 2026.
View the transcript
SMIL images
w3c/epub-specs#2883
SueNeu: Hadrien, do you want to tell us what is happening in this issue so far?
Hadrien: Just a headsup that I opened another SMIL-related issue, not directly related but of interest
… SMIL for images is something we've discussed, I proposed this as I think it's a missing feature
… we lack the ability to reference an image or fragment of an image and link that to text or audio for that fragment
… and role, so we can identify the fragment
… I've shared examples, one from Bibliotheque Simon Roland(?), human recorded audio for the captions and speech bubbles of a comic
… they had to use the text element, wasn't ideal
… because they used text, they couldn't also provide the textual alternative
… audio equivalent of handwritten text
… it demonstrates what Brady might be concerned about, but if its possible today, it may not change things
… if they're doing it already today, it's as a way to make this work where there's a gap
… there is interest from other groups as well
… there is a paper published by a French university, where they generated a script for comics using computer vision, genAI, they can generate a description and extract the text
… create fragments, they generate text, then audio based on the text
… shows what more modern tools can do
… not everyone, there's huge interest from DAISY, and there's a technical conference after the DAISY meeting in June
… there's interest, but it's currently missing
… pointed out one way to do it, adding an existing SMIL feature to our subset
… discussion with Ivan, do we put it in the spec and we're not sure if it gets adopted
… MO is not well supported across the board, but it could also be a missed opportunity
… where there are gaps, people will invent their own formats
… we could consider writing a note or just allowing EPUBCheck to not flag it
Hadrien: there are many ways to do something about it, doing nothing would be a waste
ivan: I will also repeat a bit of what was discussed on the issue, I don't think it is worth spending our time on the technical value and usefulness
… as far as I'm concerned, this is a useful feature
… that's not a question, in my memory, Brady raised issues that are not necessarily technical problems, but what do we do with a feature that wasn't in the charter, which came in and we have no idea whether big publishers will be interested
… how will it distort the market?
ivan: This is what we need to discuss
… it's not about the technical value, it's an easy extension
… there's no disagreement there, the bigger question is the "market"
duga: To elaborate more on my concern, we have an existing ecosystem of children's books using SMIL, the way that wordflow works is taking an image-only book, add a text layer, get word by work highlighting
duga: If we add this, it's a new flow for generating children's books
… it's true you can make bad content, you can avoid putting text in the divs you're highlighting, there's a lot of features built into the text layer, it works.
… we're saying that by making this change you can just do images with attached audio
… most of the children's books aren't good for accessibility for other reasons
… my worry is we're saying you can do either audio or text, and someone comes to the spec and sees two options
… an author can come along and see "oh I can do this with images only", but platforms like Play Books won't support that
… we should at least be upfront about the new way of doing the books, is that something you want to see in this version
… I'm not disagreeing that this is a good feature, but we have a method, and an ecosystem built around that method, there could be an impact
<SueNeu> ack: Hadrien
Hadrien: Going back to Ivan's question and Brady, we focus way too much on a few players, and the ecosystem is much bigger than just a few companies
… we have orgs moving off DAISY and going to EPUB with MO
… reflow files with MO, which isn't even supported in Apple Books.
… focusing on just a few players that don't do these things, and missing that there are players that are doing this work, and making these advances
… we are seeing passionate users building tools and platforms that support these new things
… instead of focusing on big players, should we also support the more specialized libraries, specialized use cases
wendyreid: We need to start having some discussions about who are we concerned with?
… there are more niche or smaller players in the EPub market that want advancement of the spec
… using web technologies. They do not have the same use cases
… that we have been focusing on. Because we are primarily focused on larger vendors
… most of the innovation is coming from smaller platforms and specialized libraries
… including regional use cases, like web comics
… we need to look more closely at these smaller players
… even if we don't expect the bigger players to adopt it quickly
… if there is a reading system, say in Sweden, that would benefit from this we should do it
<ivan> s/ack wendyreid//
wendyreid: to
@duga's point, we do need to reach out and create more best practices around itack: mgarrish
mgarrish: It's important to take everyone's concerns, my concern has been expressed by Ivan and Brady, we might be squeezing this in right before CR
… yes it's part of SMIL, there's a number of details to work out, it's not that this doesn't have value, but is this the right time
… or is this a case where we do this through EPUBCheck and notes, so bigger players have a chance to voice this
… this wasn't in the charter, there was no "notice" of this change, everyone agrees this is good, are we doing it the right way
Hadrien: I'd have to read the charter again, but I think we had accessibility and comics, so this might fit, the work on annotations is bigger than this
… I'm not advocating to push this in the spec, we can do a note, a future revision, as long as it doesn't get flagged in EPUBCheck
… it raises questions about how we handle revisions, we don't know if there will be a charter after this one
… the window for doing anything feels narrow, sometimes we know things in advance, and other times we talk to people and learn something bring those things back to the spec
<ivan> quoting from the charter: "Finally, the Working Group will also incubate issues without necessarily aiming at the creation of final Recommendations during the lifetime of this charter. "
DaleRogers: I'm listening to this and it makes me wonder, what is the purpose of a spec? I guess it goes back to the charter
… I take it as we said we'd do this, and we're telling people "this is how you should do it" not that "you must do this"
mgarrish: Don't want to veer too far off course, when we do revisions, we need to be sure up front what we're doing. We should talk to people, but we need to do it more in advance, we need some guardrails
… we're already looking at doing horizontal reviews, but now we're looking at a significant change, we need a plan for these things
… it's not a critique of bringing this up, just that this always seems to happen and we try to accommodate where we want, but we need some organization in the future
ivan: Putting on my staff contact hat, I was looking at the charter, we do these things, but we also incubate things without necessarily committing them to the final version of the rec
… we gave ourselves an escape hatch
… I would personally, staff contact hat off, I'd be happy to add it with a warning, I agree it's not a major work, not like the layout issue
… we can put it in as at risk, it doesn't mean it has to be implemented, we use the remaining year to reach out to learn about the community's reaction to it
… if we get positive feedback, we leave it in, if we don't, we can remove it.
… that's do-able, we have the ability to roll back
SueNeu: People see the utility of this, concerns about process, whether this is in scope, the current proposal are to add this to the spec with a warning, removeable if needed, or just add a note and change EPUBCheck to allow this.
ivan: What I think is to produce a separate note describing this feature
Hadrien: Correct, a separate document
ivan: If the feedback from the community comes out negative, we can remove from the spec and create a note, so it doesn't disappear
Hadrien: This sounds like a good process to me
Proposed: We will add SMIL in images to EPUB 3,4 spec as at risk, and then move it to a note if it comes out of the final spec?
<wendyreid> +1
<SueNeu> +1
<Hadrien> +1
<gman> +1
<toshiakikoike> +1
<gautierchomel_> +1
<DaleRogers> +1
<shiestyle> 0
<duga> 0
<MasakazuKitahara> +1
<ivan> +
<ajellinek> +1
<ivan> +1
RESOLUTION: We will add SMIL in images to EPUB 3,4 spec as at risk, and then move it to a note if it comes out of the final spec
ivan: No good thing goes unpunished, it would be good if you produced the PR to add this into the document
Hadrien: Each element has its own subsection, so one for images?
ivan: Yes, and we'll support editorially
duga: And the RS spec for behaviour
Hadrien: I'll look it up and figure out where things need to go
HadrienGardeur commentedon Feb 11, 2026
Here's an example from Nota with image + audio: https://lbs.nota.dk/embedded/50272?guest=true
HadrienGardeur commentedon Jul 2, 2026
@reidmore-online I see that this is marked as "Deferred" but we did had a vote on it:
After this vote, I started looking into the Media Overlay sections of our specs and that's how I ended up opening a lot of different issues. For the sake of making things easier for 3.4, I'm fine pushing this to 3.5 but let's not start entirely from scratch since we already had a vote on it.
iherman commentedon Jul 3, 2026
The issue is not closed. Deferred simply means that it is not the subject of 3.4, ie, this issue is not relevant to the CR transition request on the way.