Merge pull request #507 from IIIF/0489-multimedia-canvas

Recipe #489 Multimedia Canvas

Author: glenrobson

_includes/links.md

@@ -77,6 +77,7 @@
 [0485]: {{ site.cookbook_url | absolute_url }}/recipe/0485-contentstate-canvas-region/ "Open a specific region of a canvas in a viewer"
 
 [0434]: {{ site.cookbook_url | absolute_url }}/recipe/0434-choice-av/ "Multiple Choice of Audio Formats in a Single View (Canvas)"
+[0489]: {{ site.cookbook_url | absolute_url }}/recipe/0489-multimedia-canvas/ "Rendering Multiple Media Types on a Time-Based Canvas"
 
 [cookbook-process]: {{site.cookbook_url | absolute_url }}/recipe
 

index.md

@@ -114,6 +114,7 @@ _(leading on to segmentation examples later)_
 * [Multiple choice of images in a single view][0033] (29)
 * [Foldouts, Flaps, and Maps][0035]
 * [Composition from Multiple Images][0036] (30,31)
+* [Rendering Multiple Media Types on a Time-Based Canvas][0489]
 * Multiple images and multiple choices (32,33,34)
 * [Annotating part of an image to a Canvas][recipe-segment-image-part] (e.g., crop out scanner) (35,36,37,38)
 * [Image Rotation Two Ways][0040]

recipe/0489-multimedia-canvas/index.md

@@ -0,0 +1,52 @@
+---
+title: Rendering Multiple Media Types on a Time-Based Canvas
+id: 499
+layout: recipe
+tags: Complex Object
+summary: "Paint a still image, a video with sound, and text onto a single Canvas"
+viewers:
+ - Theseus
+topic: 
+ - annotation
+---
+
+## Use Case
+
+You want to create a digital assignment by providing students with a IIIF image resource to analyze in a fixed amount of time, a timer to show students how long they have, and text instructions for how to complete the assignment. You want to keep everything in one IIIF viewer.
+
+## Implementation Notes
+
+This recipe pulls together techniques and structures described in other recipes, and adds new elements. In other recipes, this cookbook shows [how to place multiple resources on a single Canvas][0036] and how to work with [a resource with different dimensions than its Canvas][0004]. Here, the aim is to mix AV and image IIIF resources, restrict Annotations' visibility by time ranges, and resize and position an AV resource on a Canvas.
+
+Likewise, while other recipes have discussed [simple textual annotations][0266], this recipe uses a `painting` motivation for its textual annotations (rather than `commenting` as the linked recipe does) to include the text directly and visibly on the Canvas.
+
+A Canvas containing a `duration` property has a notional timeline that can be used to target resources to a time point or range of the Canvas. If a resource's `target` value includes a `t` parameter, the resource can be expected to be shown during only and all of that span of time on the timeline. The order in which resources appear on the Canvas is governed by the `t` parameter of the `target` when specified; they can be entered in the Manifest in any order. (See "Restrictions" below for caveats and see [the Presentation specification][prezi3] for more detail about time and IIIF resources, including [the `duration` property][https://iiif.io/api/presentation/3.0/#duration].)
+
+The `t` parameter of the `target` value can also serve to trim an AV resource to be displayed for something other than its whole duration. An AV resource can thus have a `target` that governs its placement on the Canvas, its dimensions relative to the Canvas, its appearance duration, and the amount of its total running time that is used. An AV resource can also take [a `timeMode` property](https://iiif.io/api/presentation/3.0/#timemode) that governs its rendering in conjunction with the `t` parameter of its `target` and the `duration` property of a Canvas.
+
+## Restrictions
+
+When using timing for showing and hiding resources on a Canvas, a high degree of precision is not likely to be achieved for all people viewing the Canvas. Resources may load slowly for many reasons, including image server issues, network traffic, or browser and client customizations. Except in a very predictable environment, timing should be considered approximate. As well, people interacting with a multimedia Canvas may have greater or lesser control over the Canvas timeline. Consequently, creators cannot assume rigid implementation of timeline manipulation restrictions, even implied ones as in this recipe.
+
+## Example
+
+In this example, a still image, a video, and plaintext annotations are combined on a single Canvas to simulate a classroom assignment designed to introduce students to image study and notetaking in a compressed period of time. Note that this example contains material pedagogical design flaws and is not to be used as is for a classroom assignment.
+
+The simultaneously visible resources are listed in the Manifest from the foreground to the background. The timer video comes first in the Manifest, as the only currently capable viewer places resources on the Canvas so that the first resource is the most foregrounded.
+
+A person using a viewer that supports this recipe is presented first with the instruction to "Press Play", added using a time-bound plaintext annotation. After pressing play, additional instructions appear, also created using a time-bound plaintext annotation. When the time elapses on this annotation, the person is presented with a clock video overlaid on top of a still image. In the hypothetical assignment, a student would follow the instructions and take notes on the still image while the clock counts up the time allotted. After 30 seconds passes, the still image and clock video disappear, replaced by text instructions to close the browser. These final instructions stay visible for approximately 2-1/2 minutes to avoid an immediate return to the initial instructions.
+
+{% include manifest_links.html manifest="manifest.json" %}
+
+{% include jsonviewer.html src="manifest.json" config='data-line="16,40,49-51,54,59,66,71,78,83,90"' %}
+
+## Related Recipes
+
+* [Image and Canvas with Differing Dimensions][0004] for relative dimensioning of a resource and a Canvas
+* [Table of Contents for Multiple A/V Files on a Single Canvas][0064] for time ranges in `target`s
+* [Simplest Annotation][0266] for a basic textual annotation
+* [Composition from Multiple Images][0036] for using multiple IIIF resources on a single Canvas
+
+{% include acronyms.md %}
+{% include links.md %}
+

recipe/0489-multimedia-canvas/manifest.json

@@ -0,0 +1,93 @@
+{
+    "@context": "http://iiif.io/api/presentation/3/context.json",
+    "id": "{{ id.url }}",
+    "type": "Manifest",
+    "label": { "en": [ "Multimedia Canvas" ] },
+    "items": [
+      {
+        "id": "{{ id.path }}/canvas",
+        "type": "Canvas",
+        "height": 31722,
+        "width": 70399,
+        "duration": 180,
+        "items": [
+          {
+            "id": "{{ id.path }}/page/p1/1",
+            "type": "AnnotationPage",
+            "items": [
+              {
+                "id": "{{ id.path }}/annotation/p0001-image",
+              "type": "Annotation",
+              "motivation": "painting",
+              "body": {
+                "id": "https://iiif.io/api/image/3.0/example/reference/36ca0a3370db128ec984b33d71a1543d-100320001004/full/full/0/default.jpg",
+                "type": "Image",
+                "format": "image/jpeg",
+                "height": 31722,
+                "width": 70399,
+                "service": [
+                  {
+                    "id": "https://iiif.io/api/image/3.0/example/reference/36ca0a3370db128ec984b33d71a1543d-100320001004",
+                    "profile": "level1",
+                    "type": "ImageService3"
+                  }
+                ]
+              },
+                "target": "{{ id.path }}/canvas#t=11,42"
+              },
+              {
+                "id": "{{ id.path }}/annotation/p0002-video",
+                "type": "Annotation",
+                "motivation": "painting",
+                "body": {
+				  "id": "https://fixtures.iiif.io/video/indiana/30-minute-clock/medium/30-minute-clock.mp4",
+				  "type": "Video",
+				  "height": 360,
+				  "width": 640,
+				  "duration": 1801.055,
+				  "format": "video/mp4"
+                },
+                "target": "{{ id.path }}/canvas#xywh=1000,500,5000,6000&t=11,42"
+              },
+              {
+                "id": "{{ id.path }}/annotation/p0004-text",
+                "type": "Annotation",
+                "motivation": "painting",
+                "body": {
+				  "type": "TextualBody",
+				  "format": "text/html",
+				  "value": "<p style='font-size:2000px'>Press Play</p>",
+				  "language": "en"
+                },
+                "target": "{{ id.path }}/canvas#xywh=30200,10200,15000,5000&t=0,1"
+              },
+              {
+                "id": "{{ id.path }}/annotation/p0005-text",
+                "type": "Annotation",
+                "motivation": "painting",
+                "body": {
+				  "type": "TextualBody",
+				  "format": "text/html",
+				  "value": "<p style='font-size:1500px'>In 10 seconds, this text will be replaced by a clock and an image. You will have 30 seconds (shown on the clock) in which to take notes on the image you see. After 30 seconds, the image will be replaced by the start screen. You will not be responsible for the part of the image covered by the clock.</p>",
+				  "language": "en"
+                },
+                "target": "{{ id.path }}/canvas#xywh=20220,5000,30000,5000&t=1,11"
+              },
+              {
+                "id": "{{ id.path }}/annotation/p0006-text",
+                "type": "Annotation",
+                "motivation": "painting",
+                "body": {
+				  "type": "TextualBody",
+				  "format": "text/html",
+				  "value": "<p style='font-size:2000px'>Close your browser</p>",
+				  "language": "en"
+                },
+                "target": "{{ id.path }}/canvas#xywh=27000,10200,25000,5000&t=42,180"
+              }
+            ]
+          }
+        ]
+      }    
+    ]
+  }