webcodecs-node

WebCodecs API implementation for Node.js using node-av.

This package provides a Node.js-compatible implementation of the WebCodecs API, enabling video and audio encoding/decoding in server-side JavaScript applications.

Developed by Aptum - High-performance webcodecs implementation for Node.js.

Features

VideoEncoder / VideoDecoder - H.264, HEVC, VP8, VP9, AV1
AudioEncoder / AudioDecoder - AAC, Opus, MP3, FLAC, Vorbis
ImageDecoder - PNG, JPEG, WebP, GIF, AVIF, BMP, TIFF (including animated with frame timing)
ImageEncoder - Encode VideoFrames to PNG, JPEG, WebP
VideoFrame / AudioData - Frame-level data manipulation
MediaCapabilities - Query codec support, smooth playback, and power efficiency
Hardware Acceleration - VAAPI, NVENC, QSV support
Streaming Support - Real-time frame-by-frame encoding/decoding
Latency Modes - Configure for real-time streaming vs maximum compression
Bitrate Modes - Constant, variable, and quantizer (CRF) encoding modes
Alpha Channel - Preserve transparency with VP9 codec (software encoding only)
10-bit & HDR - I420P10, P010 formats with HDR10 metadata support
Container Support - MP4, WebM demuxing/muxing utilities

Documentation

API Reference - Detailed API documentation for all classes
Codec Support - Supported video, audio, and image codecs
Configuration Guide - bitrateMode, alpha, latencyMode, and more
Examples - Practical usage examples

Requirements

Node.js 18+
The node-av package (automatically installed as a dependency)

# node-av provides native FFmpeg bindings - no separate FFmpeg installation required
npm install webcodecs-node

Installation

npm install webcodecs-node

Quick Start

Using the Polyfill

Install the WebCodecs API globally to make it available as browser-compatible globals:

import { installWebCodecsPolyfill } from 'webcodecs-node';

// Install globally
installWebCodecsPolyfill();

// Now use standard WebCodecs API
const encoder = new VideoEncoder({
  output: (chunk, metadata) => {
    console.log('Encoded chunk:', chunk.byteLength, 'bytes');
  },
  error: (e) => console.error(e),
});

encoder.configure({
  codec: 'avc1.42001E', // H.264 Baseline
  width: 1280,
  height: 720,
  bitrate: 2_000_000,
});

Direct Import

import {
  VideoEncoder,
  VideoDecoder,
  VideoFrame,
  AudioEncoder,
  AudioDecoder,
  AudioData,
  ImageDecoder,
  mediaCapabilities,
} from 'webcodecs-node';

API Reference

VideoEncoder

Encodes raw video frames to compressed video.

const encoder = new VideoEncoder({
  output: (chunk, metadata) => {
    // chunk is EncodedVideoChunk
    // metadata contains decoder config info
  },
  error: (e) => console.error(e),
});

encoder.configure({
  codec: 'avc1.42001E',  // H.264
  width: 1920,
  height: 1080,
  bitrate: 5_000_000,
  framerate: 30,
  bitrateMode: 'variable',                 // Optional: 'constant', 'variable', or 'quantizer'
  latencyMode: 'realtime',                 // Optional: 'realtime' for streaming, 'quality' for best compression
  hardwareAcceleration: 'prefer-hardware', // Optional: use GPU encoding
});

// Create a frame from raw RGBA data
const frame = new VideoFrame(rgbaBuffer, {
  format: 'RGBA',
  codedWidth: 1920,
  codedHeight: 1080,
  timestamp: 0,
});

encoder.encode(frame);
frame.close();

await encoder.flush();
encoder.close();

Supported codecs:

avc1.* - H.264/AVC
hev1.*, hvc1.* - H.265/HEVC
vp8 - VP8
vp09.* - VP9
av01.* - AV1

VideoDecoder

Decodes compressed video to raw frames.

const decoder = new VideoDecoder({
  output: (frame) => {
    // frame is VideoFrame with raw pixel data
    console.log(`Frame: ${frame.codedWidth}x${frame.codedHeight}`);
    frame.close();
  },
  error: (e) => console.error(e),
});

decoder.configure({
  codec: 'avc1.42001E',
  codedWidth: 1920,
  codedHeight: 1080,
});

// Decode an encoded chunk
decoder.decode(encodedVideoChunk);
await decoder.flush();
decoder.close();

AudioEncoder

Encodes raw audio samples to compressed audio.

const encoder = new AudioEncoder({
  output: (chunk, metadata) => {
    console.log('Encoded audio:', chunk.byteLength, 'bytes');
  },
  error: (e) => console.error(e),
});

encoder.configure({
  codec: 'opus',
  sampleRate: 48000,
  numberOfChannels: 2,
  bitrate: 128000,
});

// Create audio data from raw samples
const audioData = new AudioData({
  format: 'f32',
  sampleRate: 48000,
  numberOfChannels: 2,
  numberOfFrames: 1024,
  timestamp: 0,
  data: float32Samples,
});

encoder.encode(audioData);
audioData.close();

await encoder.flush();
encoder.close();

Supported codecs:

opus - Opus
mp4a.40.2 - AAC-LC
mp3 - MP3
flac - FLAC
vorbis - Vorbis

ImageDecoder

Decodes images (including animated) to VideoFrames. Fully compliant with the WebCodecs ImageDecoder API.

import { readFileSync } from 'fs';

const imageData = readFileSync('animation.gif');

const decoder = new ImageDecoder({
  type: 'image/gif',
  data: imageData,
});

// Wait for parsing to complete
await decoder.completed;

// Access track information
const track = decoder.tracks.selectedTrack;
console.log(`Type: ${decoder.type}`);
console.log(`Frames: ${track?.frameCount}`);
console.log(`Animated: ${track?.animated}`);
console.log(`Loop count: ${track?.repetitionCount}`); // Infinity = loop forever

// Decode each frame with timing info
for (let i = 0; i < track.frameCount; i++) {
  const { image, complete } = await decoder.decode({ frameIndex: i });
  console.log(`Frame ${i}: ${image.codedWidth}x${image.codedHeight}`);
  console.log(`  Timestamp: ${image.timestamp / 1000}ms`);
  console.log(`  Duration: ${image.duration / 1000}ms`);
  image.close();
}

decoder.close();

Supported formats:

image/png, image/apng
image/jpeg
image/webp
image/gif
image/avif
image/bmp
image/tiff

ImageEncoder

Encodes VideoFrames to image formats (PNG, JPEG, WebP). This is a utility class that mirrors ImageDecoder.

import { ImageEncoder, VideoFrame } from 'webcodecs-node';

// Check format support
ImageEncoder.isTypeSupported('image/webp'); // true

// Encode a frame to JPEG
const result = await ImageEncoder.encode(frame, {
  type: 'image/jpeg',
  quality: 0.85,
});

fs.writeFileSync('output.jpg', Buffer.from(result.data));

// Synchronous encoding
const pngResult = ImageEncoder.encodeSync(frame, { type: 'image/png' });

// Batch encode multiple frames
const results = await ImageEncoder.encodeBatch(frames, { type: 'image/webp' });

Supported output formats:

image/png - Lossless, supports transparency
image/jpeg - Lossy, quality 0-1 (default: 0.92)
image/webp - Lossy/lossless, quality 0-1 (default: 0.8)

MediaCapabilities API

Query codec capabilities before encoding/decoding. Implements the standard MediaCapabilities API.

import { mediaCapabilities } from 'webcodecs-node';

// Query decoding capabilities
const decodeInfo = await mediaCapabilities.decodingInfo({
  type: 'file',
  video: {
    contentType: 'video/mp4; codecs="avc1.42E01E"',
    width: 1920,
    height: 1080,
    bitrate: 5_000_000,
    framerate: 30,
  },
  audio: {
    contentType: 'audio/mp4; codecs="mp4a.40.2"',
    channels: 2,
    bitrate: 128000,
    samplerate: 44100,
  },
});

console.log('Supported:', decodeInfo.supported);
console.log('Smooth playback:', decodeInfo.smooth);
console.log('Power efficient:', decodeInfo.powerEfficient);

// Query encoding capabilities
const encodeInfo = await mediaCapabilities.encodingInfo({
  type: 'record',
  video: {
    contentType: 'video/webm; codecs="vp9"',
    width: 1280,
    height: 720,
    bitrate: 2_000_000,
    framerate: 30,
  },
});

if (encodeInfo.supported && encodeInfo.powerEfficient) {
  console.log('Hardware-accelerated encoding available!');
}

Hardware Acceleration

Detect and use hardware encoding/decoding:

import {
  detectHardwareAcceleration,
  getHardwareAccelerationSummary,
  getBestEncoder,
} from 'webcodecs-node';

// Get a summary of available hardware acceleration
const summary = await getHardwareAccelerationSummary();
console.log(summary);

// Detect capabilities
const capabilities = await detectHardwareAcceleration();
console.log('Available methods:', capabilities.methods);
console.log('Hardware encoders:', capabilities.encoders);
console.log('Hardware decoders:', capabilities.decoders);

// Get best encoder for a codec
const best = await getBestEncoder('h264', 'prefer-hardware');
console.log(`Using: ${best.encoder} (hardware: ${best.isHardware})`);

// Use in VideoEncoder config
encoder.configure({
  codec: 'avc1.42001E',
  width: 1920,
  height: 1080,
  bitrate: 5_000_000,
  hardwareAcceleration: 'prefer-hardware',
});

Supported acceleration methods:

VAAPI - Intel/AMD on Linux
NVENC/NVDEC - NVIDIA GPUs
QSV - Intel Quick Sync Video
VideoToolbox - macOS

Container Utilities

Import container demuxing/muxing utilities for working with MP4, WebM, and MKV files:

import { Demuxer, Muxer, muxChunks, extractVideoFrames } from 'webcodecs-node/containers';

// Demux a video file
const demuxer = new Demuxer({ path: 'input.mp4' });
await demuxer.open();

console.log('Video:', demuxer.videoConfig);
console.log('Audio:', demuxer.audioConfig);

for await (const chunk of demuxer.videoChunks()) {
  // chunk is EncodedVideoChunk ready for VideoDecoder
}
await demuxer.close();

// Mux encoded chunks to a file
const muxer = new Muxer({ path: 'output.mp4' });
await muxer.open();
await muxer.addVideoTrack({
  codec: 'avc1.42001E',
  codedWidth: 1920,
  codedHeight: 1080,
  description: spsNaluBuffer, // Optional: H.264 SPS/PPS
});

for (const chunk of encodedChunks) {
  await muxer.writeVideoChunk(chunk);
}

const result = await muxer.closeWithResult();
console.log(`Muxed with ${result.backend} in ${result.durationMs}ms`);

// Or use the convenience function
const result = await muxChunks({
  path: 'output.mp4',
  video: { config: videoTrackConfig, chunks: videoChunks },
  audio: { config: audioTrackConfig, chunks: audioChunks },
});

// Extract decoded frames directly
for await (const frame of extractVideoFrames('input.mp4')) {
  console.log(`Frame: ${frame.timestamp}us`);
  frame.close();
}

Muxer Fallback Architecture:

The Muxer class uses a two-tier approach for reliability:

Primary: node-av (~5ms) - Fast native muxing using node-av's FormatContext API
Fallback: FFmpeg subprocess (~130ms) - Spawns FFmpeg process if node-av fails

const muxer = new Muxer({
  path: 'output.mp4',
  onFallback: (err) => console.warn('Using FFmpeg fallback:', err.message),
  forceBackend: 'node-av', // Optional: 'node-av' or 'ffmpeg-spawn'
});

You can also use the backend-specific classes directly:

import { NodeAvMuxer, FFmpegMuxer } from 'webcodecs-node/containers';

// Fast path only
const fastMuxer = new NodeAvMuxer({ path: 'output.mp4' });

// FFmpeg subprocess only
const ffmpegMuxer = new FFmpegMuxer({ path: 'output.mp4' });

Streaming & Latency Modes

For real-time streaming applications, use latencyMode: 'realtime' to minimize encoding latency:

encoder.configure({
  codec: 'avc1.42001E',
  width: 1280,
  height: 720,
  bitrate: 2_000_000,
  framerate: 30,
  latencyMode: 'realtime', // Prioritize low latency
});

Latency mode options:

'quality' (default) - Best compression, higher latency (uses B-frames, lookahead)
'realtime' - Minimum latency for live streaming (no B-frames, zero-delay)

Bitrate Modes

Control how bitrate is managed during encoding:

encoder.configure({
  codec: 'avc1.42001E',
  width: 1920,
  height: 1080,
  bitrate: 5_000_000,
  bitrateMode: 'constant', // CBR for streaming
});

Mode	Description	Use Case
`'variable'`	VBR - varies bitrate for quality (default)	General purpose
`'constant'`	CBR - fixed bitrate throughout	Streaming, broadcast
`'quantizer'`	CRF/CQ - fixed quality level	Archival, quality-first

Alpha Channel (Transparency)

Preserve transparency when encoding with VP9 or AV1:

encoder.configure({
  codec: 'vp9',
  width: 1920,
  height: 1080,
  alpha: 'keep', // Preserve transparency
});

// Create RGBA frame with transparency
const frame = new VideoFrame(rgbaWithAlpha, {
  format: 'RGBA',
  codedWidth: 1920,
  codedHeight: 1080,
  timestamp: 0,
});

encoder.encode(frame);

10-bit Pixel Formats & HDR

Support for high bit-depth content and HDR metadata:

import {
  VideoFrame,
  VideoColorSpace,
  createHdr10MasteringMetadata,
  createContentLightLevel,
  is10BitFormat,
  getBitDepth,
} from 'webcodecs-node';

// Create a 10-bit frame
const frame = new VideoFrame(yuv10bitData, {
  format: 'I420P10',  // 10-bit YUV 4:2:0
  codedWidth: 3840,
  codedHeight: 2160,
  timestamp: 0,
  colorSpace: new VideoColorSpace({
    primaries: 'bt2020',
    transfer: 'pq',        // HDR10 PQ transfer
    matrix: 'bt2020-ncl',
  }),
});

// Check format properties
console.log(is10BitFormat('I420P10'));  // true
console.log(getBitDepth('I420P10'));    // 10

// HDR metadata for mastering display
const hdrMetadata = {
  smpteSt2086: createHdr10MasteringMetadata(1000, 0.0001), // max/min luminance
  contentLightLevel: createContentLightLevel(800, 400),    // MaxCLL, MaxFALL
};

const colorSpace = new VideoColorSpace({
  primaries: 'bt2020',
  transfer: 'pq',
  hdrMetadata,
});

console.log(colorSpace.isHdr);          // true
console.log(colorSpace.hasHdrMetadata); // true

10-bit pixel formats:

I420P10 - YUV 4:2:0 planar, 10-bit
I422P10 - YUV 4:2:2 planar, 10-bit (downconverted to 4:2:0 during encoding)
I444P10 - YUV 4:4:4 planar, 10-bit (downconverted to 4:2:0 during encoding)
P010 - YUV 4:2:0 semi-planar, 10-bit

Note: 10-bit encoding is supported for HEVC, VP9, and AV1 codecs only. I422P10 and I444P10 formats preserve chroma resolution for decoding but are currently downconverted to I420P10 during encoding (chroma subsampling reduction). H.264 does not support 10-bit encoding and will downconvert to 8-bit.

Pixel format utilities:

is10BitFormat(format) - Check if format is 10-bit
getBitDepth(format) - Get bit depth (8 or 10)
get8BitEquivalent(format) - Get 8-bit version of a 10-bit format
get10BitEquivalent(format) - Get 10-bit version of an 8-bit format

Canvas Rendering (skia-canvas)

GPU-accelerated 2D canvas rendering with automatic hardware detection:

import {
  createCanvas,
  createFrameLoop,
  detectGpuAcceleration,
  isGpuAvailable,
  getGpuApi,
  ensureEvenDimensions,
  VideoEncoder,
} from 'webcodecs-node';

// Check GPU availability
const gpuInfo = detectGpuAcceleration();
console.log(`Renderer: ${gpuInfo.renderer}`); // 'GPU' or 'CPU'
console.log(`API: ${getGpuApi()}`);           // 'Metal', 'Vulkan', 'D3D', or null

// Create GPU-accelerated canvas
const canvas = createCanvas({
  width: 1920,
  height: 1080,
  gpu: true, // or omit for auto-detection
});

const ctx = canvas.getContext('2d');
ctx.fillStyle = 'red';
ctx.fillRect(0, 0, 1920, 1080);

// Create VideoFrame directly from canvas
const frame = new VideoFrame(canvas, { timestamp: 0 });

FrameLoop helper for animation with backpressure:

const loop = createFrameLoop({
  width: 1920,
  height: 1080,
  frameRate: 30,
  maxQueueSize: 8, // Backpressure limit
  onFrame: (ctx, timing) => {
    // Draw each frame
    ctx.fillStyle = `hsl(${timing.frameIndex % 360}, 100%, 50%)`;
    ctx.fillRect(0, 0, 1920, 1080);
  },
});

loop.start(300); // Generate 300 frames

while (loop.getState() !== 'stopped' || loop.getQueueSize() > 0) {
  const frame = loop.takeFrame();
  if (frame) {
    encoder.encode(frame);
    frame.close(); // Always close frames!
  }
}

OffscreenCanvas polyfill for browser-compatible code:

import { installOffscreenCanvasPolyfill } from 'webcodecs-node';

installOffscreenCanvasPolyfill();

// Now use standard OffscreenCanvas API
const canvas = new OffscreenCanvas(1920, 1080);
const ctx = canvas.getContext('2d');
const blob = await canvas.convertToBlob({ type: 'image/png' });

Performance Tuning

Memory Management

Always close VideoFrames and AudioData when done:

const frame = new VideoFrame(buffer, { ... });
try {
  encoder.encode(frame);
} finally {
  frame.close(); // Prevent memory leaks
}

Even Dimensions

Video codecs require even dimensions for YUV420 chroma subsampling:

import { ensureEvenDimensions, validateEvenDimensions } from 'webcodecs-node';

// Auto-fix odd dimensions (rounds up)
const { width, height } = ensureEvenDimensions(1279, 719);
// Returns { width: 1280, height: 720 }

// Strict validation (throws if odd)
validateEvenDimensions(1280, 720); // OK
validateEvenDimensions(1279, 720); // Throws TypeError

Backpressure Handling

Monitor encoder queue to prevent memory exhaustion:

encoder.addEventListener('dequeue', () => {
  // Queue size decreased, safe to encode more
  if (encoder.encodeQueueSize < 10) {
    encodeNextFrame();
  }
});

Raw Buffer Export

For maximum performance, use raw RGBA buffers instead of PNG/JPEG:

import { getRawPixels } from 'webcodecs-node';

// Fast: raw RGBA buffer (no compression)
const pixels = getRawPixels(canvas); // Returns Buffer

// Slow: PNG encoding (avoid in hot paths)
const png = await canvas.toBuffer('png');

GPU vs CPU Tradeoffs

Scenario	Recommendation
HD/4K encoding	`hardwareAcceleration: 'prefer-hardware'`
Real-time streaming	Hardware + `latencyMode: 'realtime'`
Maximum quality	Software + `bitrateMode: 'quantizer'`
Batch processing	Hardware for throughput
Low-end systems	Software (more compatible)

Debugging

Enable debug logging to troubleshoot encoding/decoding issues:

# Enable debug output
WEBCODECS_DEBUG=1 node your-script.js

# Or set programmatically
import { setDebugMode } from 'webcodecs-node';
setDebugMode(true);

Debug mode outputs detailed information about:

Hardware acceleration detection and selection
Encoder/decoder initialization
Muxer backend selection and fallback events
Filter chain configuration
Error details with context

Example debug output:

[webcodecs:Transcode] Using hardware acceleration: vaapi
[webcodecs:Transcode] Using hardware decoder for h264
[webcodecs:Transcode] Using hardware encoder for h264
[webcodecs:Transcode] Using filter chain: scale_vaapi=format=nv12
[webcodecs:NodeAvMuxer] writeTrailer returned error code -22

Demos

Run the included demos to test functionality:

npm run build

# Basic demo
npm run demo

# WebCodecs API demo
npm run demo:webcodecs

# Image decoding demo (animated GIF/PNG/WebP with frame timing)
npm run demo:image

# Hardware acceleration detection
npm run demo:hwaccel

# Streaming demo (real-time encoding)
npm run demo:streaming

# Sample-based encoding demo
npm run demo:samples

# Container demuxing/muxing demo
npm run demo:containers

# Video quadrant compositor demo (four-up render)
npm run demo:fourcorners

# 1080p transcoding demo
npm run demo:1080p

# DVD bouncing logo animation
npm run demo:dvd

# Audio visualizer with waveform and spectrum
npm run demo:visualizer

Benchmarking

Compare software vs hardware encoding performance:

# Quick benchmark (30 frames, 360p)
npm run bench:quick

# Default benchmark (120 frames, 720p)
npm run bench

# Full benchmark (300 frames, 1080p)
npm run bench:full

# Custom options
node scripts/encoding-benchmark.mjs --frames 100 --resolution 1080p --codecs h264,hevc

Options:

--frames <n> - Number of frames to encode (default: 120)
--resolution <res> - 360p, 480p, 720p, 1080p, 4k (default: 720p)
--bitrate <bps> - Target bitrate in bps
--framerate <fps> - Target framerate (default: 30)
--codecs <list> - Comma-separated: h264,hevc,vp9,av1
--skip-software - Only test hardware encoding
--verbose - Show detailed output

Example output:

════════════════════════════════════════════════════════════════════════════════
ENCODING BENCHMARK RESULTS (720p)
════════════════════════════════════════════════════════════════════════════════
Codec       Mode           FPS      Time   Latency        Size       Bitrate
────────────────────────────────────────────────────────────────────────────────
H.264/AVC   SW           213.6     562ms     391ms     2.00 MB     4.20 Mbps
H.264/AVC   HW           370.4     324ms     187ms     2.11 MB     4.43 Mbps
H.265/HEVC  SW           141.4     848ms     106ms     1.94 MB     4.06 Mbps
H.265/HEVC  HW           589.0     204ms      61ms     2.16 MB     4.54 Mbps

API Compatibility

This implementation follows the WebCodecs specification with some Node.js-specific adaptations:

Feature	Browser	webcodecs-node
VideoEncoder	✓	✓
VideoDecoder	✓	✓
AudioEncoder	✓	✓
AudioDecoder	✓	✓
ImageDecoder	✓	✓
VideoFrame	✓	✓
AudioData	✓	✓
EncodedVideoChunk	✓	✓
EncodedAudioChunk	✓	✓
ImageTrack/ImageTrackList	✓	✓
MediaCapabilities	✓	✓
Hardware Acceleration	Auto	Opt-in
latencyMode	✓	✓
bitrateMode	✓	✓
alpha (transparency)	✓	✓ (VP9 only)
isConfigSupported()	✓	✓

Architecture

This library uses node-av as its backend, which provides native bindings to FFmpeg's libav* libraries. This approach offers:

Native performance - Direct library calls instead of subprocess spawning
Lower latency - No IPC overhead between Node.js and FFmpeg
Better resource management - Native memory handling and cleanup
Simplified deployment - No need for separate FFmpeg installation

License

webcodecs-node is distributed under the GNU Affero General Public License v3.0. See LICENSE for full terms.

Name		Name	Last commit message	Last commit date
Latest commit History 72 Commits
docs		docs
examples		examples
media		media
scripts		scripts
src		src
webcodecs-tests		webcodecs-tests
.gitignore		.gitignore
LICENSE		LICENSE
README.md		README.md
ffmpeg-quality.js		ffmpeg-quality.js
jest.config.js		jest.config.js
package-lock.json		package-lock.json
package.json		package.json
tsconfig.json		tsconfig.json
webcodecs-config.example.js		webcodecs-config.example.js

License

aptumfr/webcodecs-node

Folders and files

Latest commit

History

Repository files navigation

webcodecs-node

Features

Documentation

Requirements

Installation

Quick Start

Using the Polyfill

Direct Import

API Reference

VideoEncoder

VideoDecoder

AudioEncoder

ImageDecoder

ImageEncoder

MediaCapabilities API

Hardware Acceleration

Container Utilities

Streaming & Latency Modes

Bitrate Modes

Alpha Channel (Transparency)

10-bit Pixel Formats & HDR

Canvas Rendering (skia-canvas)

Performance Tuning

Memory Management

Even Dimensions

Backpressure Handling

Raw Buffer Export

GPU vs CPU Tradeoffs

Debugging

Demos

Benchmarking

API Compatibility

Architecture

License

About

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Languages

Packages