Skip to content

srad/MediaSink

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

202 Commits
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

MediaSink

License Go Version Build Status Build

MediaSink is a powerful web-based video management, editing and streaming server written in Go. It provides automated stream recording capabilities and a versioned REST API at /api/v2, making it an ideal solution for media-heavy applications. The Vue 3 frontend is bundled directly into the Go binary, and the repository also includes a Flutter mobile client under mobile/ plus a standalone Rust terminal client under cli/ for terminal-first access to the same MediaSink server.

Features

  • Media Management: Scans all media and generate previews and organizes them. Allows bookmarking folders, channel, media items, and tagging the media.
  • Automated Stream Recording: Capture and store video streams automatically.
  • REST API for Video Editing: Perform video editing tasks programmatically.
  • Video Analysis (ONNX + sqlite-vec): Detect scenes and highlights from preview frames using ONNX feature extraction and sqlite-vec similarity queries.
  • Integrated Web UI: Vue 3 frontend embedded directly in the binary β€” served from the same port as the API, no nginx or separate deployment needed.
  • Integrated Mobile UI: Flutter app under mobile/ with server setup, login, stream/channel/video browsing, history, and mobile video playback.
  • Integrated Terminal UI: Separate Rust CLI under cli/ with login, live workspace views, WebSocket updates, themes, forms, and popup video playback.
  • Scalable & Lightweight: Optimized for performance with a minimal resource footprint.
  • Easy Integration: RESTful API for seamless integration with other applications.
  • Disaster Recovery: If the system crashes during recordings or while processing background jobs, it will recover on the next restart and check the media files for integrity.

Installation

This is mainly for development purposes. In production you'd use the Docker image.

Prerequisites

  • Go 1.x or later
  • Node.js 22+ and npm (for building the frontend)
  • Rust + Cargo (only if you want to build or run cli/)
  • FFmpeg (for video processing)
  • yt-dlp
  • FFprobe
  • SQLite 3
  • ONNX Runtime shared library (for ONNX-based video analysis)

If you run the application outside of Docker, you must manually install the above dependencies.

JavaScript tooling in this repository is npm-only. Do not use pnpm for frontend or CLI wrapper tasks.

Debian setup:

sudo apt update && sudo apt install -y wget ffmpeg sqlite3
# Install yt-dlp
curl -SL https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp_linux -o /usr/local/bin/yt-dlp && chmod +x /usr/local/bin/yt-dlp

Go setup (replace with latest version):

sudo apt update && sudo apt install -y wget
wget https://golang.org/dl/go1.23.linux-amd64.tar.gz
sudo tar -C /usr/local -xvzf go1.23.linux-amd64.tar.gz
echo "export PATH=$PATH:/usr/local/go/bin" >> ~/.bashrc
source ~/.bashrc

Clone the Repository

git clone https://github.com/srad/MediaSink.git
cd MediaSink

Build

Builds the frontend, mirrors the built web assets into server/frontend/dist, regenerates the server Swagger spec under server/docs/, and then builds the self-contained Go binary:

./build.sh

All configuration is provided via environment variables (see the Run section below). There is no config file.

For ONNX-based analysis outside Docker, set ONNXRUNTIME_LIB if the runtime library is not on your default linker path.

Run

./run.sh

Builds the frontend if needed, regenerates the server Swagger spec under server/docs/, refreshes the frontend v2 API client from that local spec, then builds and starts the server on http://0.0.0.0:3000. The web UI is available at the same address.

run.sh exports sensible local-development defaults for all required variables. The only hard requirement is SECRET (a JWT secret); everything else has a working default.

Frontend development (hot-reload)

cd frontend && npm run dev

Runs the Vite dev server (typically http://localhost:5173) with hot module replacement. Copy frontend/public/env.js.default to frontend/public/env.js if you haven't already, and make sure the Go server is running on :3000.

Frontend layout note for contributors:

  • Bootstrap is customized in frontend/src/assets/custom-bootstrap.scss.
  • The app uses a 24-column grid, not Bootstrap's default 12-column grid. Use col-24-based math when building or fixing layouts.
  • .card-body padding is globally reset to 0, so cards need explicit padding classes where spacing is required.

CLI

The terminal client lives in cli/ and is built separately from the Go server and Vue frontend.

Run it with npm:

cd cli && npm start

or directly with Cargo:

cd cli && cargo run

CLI build/test:

cd cli && cargo build --locked
cd cli && cargo test --locked

CLI notes:

  • Rust-first project layout under cli/src
  • Minimal npm wrapper only for packaging/distribution
  • Full-screen ratatui TUI with login/registration, themes, live views, confirm dialogs, mouse support, and popup video playback
  • Reads runtime settings from the target server's /env.js and /build.js
  • Rejects incompatible MediaSink servers when the exposed APP_API_VERSION is missing or does not match
  • See cli/README.md for CLI-specific shortcuts, features, and packaging details

Mobile

The Flutter mobile client lives in mobile/ and talks to the same /api/v2 backend as the web UI and CLI.

Typical development flow:

cd mobile
apiclient.bat
flutter analyze --no-pub
flutter test --no-pub
flutter run

Mobile notes:

  • Regenerate server/docs/swagger.json from the repo root before running mobile/apiclient.bat; the script copies it to mobile/schema/swagger.json, regenerates the mobile API models/client, runs build_runner, and refreshes l10n output
  • On first launch the app asks for the MediaSink server origin, derives API/WebSocket URLs from it, and validates APP_API_VERSION from /build.js before login
  • Main mobile navigation currently includes Streams, Channels, Videos, History, and Jobs; Settings is opened from the top-right gear icon instead of the bottom navigation
  • The Channels page supports grid/list layout, search, favorites-only filtering, persisted sorting, and JSON import/export
  • Local play history stores the last 100 played videos per server after 5 seconds of playback, with per-item removal and clear-all support
  • mobile/test-all.ps1 runs the Flutter unit/widget suite and then integration tests when a supported Android device is attached; otherwise it skips the integration step cleanly
  • See mobile/README.md for mobile-specific runtime and development details

Run Tests

cd server && go test ./...

CLI tests:

cd cli && cargo test --locked

Mobile tests:

cd mobile
./test-all.ps1

Usage

Storage device file system

You might want to spend some time looking at a reasonable choice for your file system because it might have a significant effect on the lifespan of your storage device, especially with write-heavy large files workloads.

These are the most common file systems and their characteristics in this context:

File System Performance Data Integrity Tuning Complexity Best Use Case
XFS πŸš€ Very High ❌ Basic only πŸ”§ Minimal Streaming large files
EXT4 ⚑ Good ❌ Basic only πŸ”§ Minimal General-purpose, legacy support
ZFS βš–οΈ Medium βœ… Excellent πŸ”§πŸ”§πŸ”§ High When data integrity > raw speed
Btrfs ⚑ Okay βœ… Good πŸ”§ Medium Light snapshots, lower overhead than ZFS

If you do not require the highest amout of data integritity checking and snapshots, at the cost of your device's lifespan, then it is highly recommended to format your storage device with the XFS filesystem, since it is optimized large write file write heavy workloads.

You can do that from the shell:

mkfs.xfs -f /dev/sdX
mount -o noatime /dev/sdX /mnt/video

API Endpoints

MediaSink provides a REST API to manage video recording and editing. Below are some key endpoints: For a complete API reference, check the API Documentation.

The current public API base path is /api/v2.

Visual similarity endpoints:

  • POST /api/v2/analysis/search/image (multipart): upload an image (file) and search similar videos. Supports similarity slider values in 0..1 or 0..100.
  • POST /api/v2/analysis/group (json): group similar videos by similarity threshold. Supports optional recordingIds, pairLimit, and includeSingletons.

Docker

The Docker image bundles the Go server, Vue frontend, FFmpeg, yt-dlp, and SQLite into a single container. No separate nginx, client container, or external web server is needed.

Docker Compose

Minimal setup β€” only SECRET and a volume for persistent storage are required:

services:
  mediasink:
    image: sedrad/mediasink
    environment:
      - SECRET=change-me
      - LOG_LEVEL=info
      - STREAM_DEBUG_LEVEL=error
    volumes:
      - /path/to/recordings:/recordings
    ports:
      - "3000:3000"

Full setup with persistent storage and timezone:

services:
  mediasink:
    image: sedrad/mediasink
    environment:
      - TZ=${TIMEZONE}
      - SECRET=${SECRET}
      - LOG_LEVEL=${LOG_LEVEL:-info}
      - STREAM_DEBUG_LEVEL=${STREAM_DEBUG_LEVEL:-error}
    volumes:
      - ${DATA_PATH}:/recordings
      - ${DISK}:/disk
      - /etc/localtime:/etc/localtime:ro
      - /etc/timezone:/etc/timezone:ro
    ports:
      - "3000:3000"

compose variables (host-side only, not passed into the container):

Variable Example Description
TIMEZONE Europe/Berlin Sets TZ inside the container
SECRET change-me JWT signing secret β€” required
DATA_PATH /path/to/your/recordings Host path mounted as /recordings
DISK /mnt/disk1 Host path mounted as /disk
LOG_LEVEL info Application log verbosity passed through to the container
STREAM_DEBUG_LEVEL error Verbosity for yt-dlp and ffmpeg stream diagnostics

Application environment variables (passed into the container, all optional):

Variable Required Default Description
SECRET yes β€” JWT signing secret β€” use a long random string
TZ no Europe/Berlin Container timezone
DB_FILENAME no /recordings/mediasink.sqlite3 SQLite database file path
REC_PATH no /recordings Recordings directory inside the container
DATA_DIR no .previews Preview/thumbnail cache directory
DATA_DISK no /disk Disk mount path used for storage status queries
NET_ADAPTER no eth0 Network interface for bandwidth monitoring
LOG_LEVEL no info Application logrus level. Supports panic, fatal, error, warn, info, debug, trace
STREAM_DEBUG_LEVEL no error Stream downloader/capture verbosity. Supports quiet, error, warning, info, debug, trace
DB_ADAPTER no sqlite Relational adapter setting. The shipped v2 server currently boots a SQLite/sqlite-vec-backed vector pipeline and should be treated as SQLite-first.
DB_HOST, DB_PORT, DB_NAME, DB_USER, DB_PASSWORD no β€” Relevant only for non-SQLite adapters in the lower DB layer; they are not part of the primary v2 runtime path.

To debug stream failures, start with:

docker run --rm -p 3000:3000 \
  -e SECRET=change-me \
  -e LOG_LEVEL=debug \
  -e STREAM_DEBUG_LEVEL=debug \
  -v /path/to/recordings:/recordings \
  -v /path/to/disk:/disk \
  sedrad/mediasink

Recommended values:

  • normal runtime: LOG_LEVEL=info and STREAM_DEBUG_LEVEL=error
  • stream debugging: LOG_LEVEL=debug and STREAM_DEBUG_LEVEL=debug
  • maximum command verbosity: LOG_LEVEL=debug and STREAM_DEBUG_LEVEL=trace

The web UI is available at http://<host>:3000 and the API at http://<host>:3000/api/v2.

Deploy

docker-compose --env-file .env up -d

Contributing

We welcome contributions! To get started:

  1. Fork the repository.
  2. Create a new branch.
  3. Make your changes and commit them.
  4. Submit a pull request.

License

MediaSink is dual-licensed under the GNU Affero General Public License (AGPL) and a commercial license.

  • Open-Source Use (AGPL License): MediaSink is free to use, modify, and distribute under the terms of the GNU AGPL v3. Any modifications and derivative works must also be open-sourced under the same license.
  • Commercial Use: Companies that wish to use MediaSink without AGPL restrictions must obtain a commercial license. Contact the project maintainer for licensing details. MediaSink is available for free for non-profit and educational institutions. However, a commercial license is required for companies.

Contact

For issues and feature requests, please use the GitHub Issues section.

Notes & Limitations

  1. All streaming services allow only a limited number of request made by each client. If this limit is exceeded the client will be temporarily or permanently blocked. In order to circumvent this issue, the application does strictly control the timing between each request. However, this might cause that the recording will only start recording after a few minutes and not instantly.
  2. The system has disaster recovery which means that if the system crashes during recordings, it will try to recover all recordings on the next launch. However, due to the nature of streaming videos and the crashing behavior, the video files might get corrupted. In this case they will be automatically delete from the system, after they have been checked for integrity. Otherwise, they are added to the library.

Star the repo if you find it useful! ⭐

About

Video management with automated stream recording capability and REST API for video editing and AI chapter and highlits generation and vector similarily search. Contains server, web frontend, terminal cli, and android app

Resources

Stars

Watchers

Forks

Contributors