adaptive-batching

Here are 4 public repositories matching this topic...

Real-time ML inference engine with adaptive batching, model hot-swap, and circuit breaker patterns

python machine-learning streaming real-time deep-learning inference pytorch model-serving mlops adaptive-batching

Local inference serving with adaptive batching, benchmark sweeps, and regression gates for batching tradeoffs.

Stream live data via WebSocket and perform real-time ML inference with adaptive batching, backpressure control, and zero-downtime model updates.

python machine-learning streaming real-time deep-learning inference pytorch model-serving mlops adaptive-batching

Deliver real-time streaming ML inference with adaptive batching, backpressure control, and zero-downtime model hot-swapping.

python machine-learning streaming real-time deep-learning inference pytorch model-serving mlops adaptive-batching

Add a description, image, and links to the adaptive-batching topic page so that developers can more easily learn about it.

To associate your repository with the adaptive-batching topic, visit your repo's landing page and select "manage topics."