Skip to content

Commit de60fcf

Browse files
committed
Add hybrid video decoder for frame-accurate stepping
- Implement VideoDecoder abstraction with HTML5 and WebCodecs backends - Add HybridVideoDecoder to orchestrate automatic decoder switching: * HTML5 decoder for playback (color accuracy, audio support) * Mediabunny/WebCodecs decoder for frame-accurate stepping * Automatic fallback to HTML5-only if WebCodecs unavailable - Fix frame stepping accuracy issues with keyframe-based HTML5 seeking - Support VFR videos via sample.duration in WebCodecs decoder - Add proper VRAM cleanup (sample.close()) and serialized seek queue - Extract timecode conversion utilities to separate module - Fix race conditions in decoder initialization - Update documentation with decoder architecture details - Replaces previous VideoFrame.js implementation
1 parent 5ce6010 commit de60fcf

18 files changed

+1741
-946
lines changed

client/package-lock.json

Lines changed: 146 additions & 113 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

client/package.json

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -39,13 +39,14 @@
3939
"tailwindcss": "^4.1.10",
4040
"tslib": "^2.8.1",
4141
"typescript": "^5.7.2",
42-
"vite": "^7.2.4",
42+
"vite": "^7.3.0",
4343
"vite-plugin-checker": "^0.8.0",
4444
"vitest": "^4.0.8"
4545
},
4646
"dependencies": {
4747
"flowbite": "^4.0.0",
4848
"flowbite-svelte": "^1.28.0",
49-
"flowbite-svelte-icons": "^3.0.0"
49+
"flowbite-svelte-icons": "^3.0.0",
50+
"mediabunny": "^1.27.0"
5051
}
5152
}

client/public/clapshot_client.conf.json

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,5 +10,7 @@
1010
"app_title": "Clapshot",
1111

1212
"default_locale": "en",
13-
"supported_locales": ["en", "zh"]
13+
"supported_locales": ["en", "zh"],
14+
15+
"enable_mediabunny": true
1416
}

client/src/__tests__/VideoPlayer.test.ts

Lines changed: 13 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -312,16 +312,9 @@ describe('VideoPlayer.svelte - Elementary Tests', () => {
312312

313313
it('should return number for getCurFrame', () => {
314314
const { component } = render(VideoPlayer, { props: { src: 'test-video.mp4' } });
315-
316-
// getCurFrame might fail if VideoFrame is not initialized properly
317-
// This is expected in the test environment
318-
try {
319-
const frame = component.getCurFrame();
320-
expect(typeof frame).toBe('number');
321-
} catch (error) {
322-
// Expected: VideoFrame not initialized properly in test environment
323-
expect(error.message).toContain('fps');
324-
}
315+
316+
// getCurFrame returns 0 if videoDecoder not initialized (graceful fallback)
317+
expect(component.getCurFrame()).toBe(0);
325318
});
326319
});
327320

@@ -1277,16 +1270,14 @@ describe('VideoPlayer.svelte - Elementary Tests', () => {
12771270
}
12781271
});
12791272

1280-
it('should handle missing VideoFrame for frame calculations', () => {
1273+
it('should return fallback when videoDecoder is missing for frame calculations', () => {
12811274
const { component } = render(VideoPlayer, { props: { src: 'test-video.mp4' } });
1282-
1283-
// Clear the VideoFrame instance
1284-
(component as any).vframeCalc = null;
1285-
1286-
expect(() => {
1287-
const frame = component.getCurFrame();
1288-
// Should handle missing VideoFrame gracefully
1289-
}).toThrow(); // Expected to throw with fps error
1275+
1276+
// Clear the videoDecoder instance
1277+
(component as any).videoDecoder = null;
1278+
1279+
// Should return 0 as graceful fallback when videoDecoder is missing
1280+
expect(component.getCurFrame()).toBe(0);
12901281
});
12911282

12921283
it('should handle volume control edge cases', () => {
@@ -1543,15 +1534,9 @@ describe('VideoPlayer.svelte - Elementary Tests', () => {
15431534
// Note: HappyDOM video element properties have limited functionality
15441535
}
15451536
}).not.toThrow();
1546-
1547-
// Test getCurFrame separately as it may throw due to VideoFrame not being initialized
1548-
try {
1549-
const currentFrame = component.getCurFrame();
1550-
expect(typeof currentFrame).toBe('number');
1551-
} catch (error) {
1552-
// Expected: VideoFrame not initialized in test environment
1553-
expect(error.message).toMatch(/fps/);
1554-
}
1537+
1538+
// getCurFrame returns 0 as fallback since videoDecoder is not initialized in test environment
1539+
expect(component.getCurFrame()).toBe(0);
15551540

15561541
// CLEANUP VERIFICATION: Component should handle cleanup properly
15571542
expect(() => {

0 commit comments

Comments
 (0)