FlashInfer: Kernel Library for LLM Serving
-
Updated
Dec 10, 2025 - C++
#
distributed-inference
Here are 11 public repositories matching this topic...
GPU cluster manager for optimized AI model deployment
cuda inference openai llama maas rocm ascend llm llm-serving vllm genai llm-inference qwen deepseek sglang distributed-inference high-performance-inference mindie
-
Updated
Dec 10, 2025 - Python
一個基於 llama.cpp 的分佈式 LLM 推理程式，讓您能夠利用區域網路內的多台電腦協同進行大型語言模型的分佈式推理，使用 Electron 的製作跨平台桌面應用程式操作 UI。
-
Updated
Aug 24, 2025 - JavaScript
Analyze and generate unstructured data using LLMs, from quick experiments to billion token jobs.
csv s3 pandas data-engineering parquet data-processing data-pipelines observability synthetic-data unstructured-data mlops batch-inference polars llm-inference evals distributed-inference
-
Updated
Nov 14, 2025 - Python
Source code of the paper "Private Collaborative Edge Inference via Over-the-Air Computation".
machine-learning ensemble-learning wireless-communications differential-privacy multi-view-learning edge-inference over-the-air-computation collaborative-inference distributed-inference
-
Updated
Jan 14, 2025 - Python
Super Ollama Load Balancer - Performance-aware routing for distributed Ollama deployments with Ray, Dask, and adaptive metrics
python caching machine-learning streaming performance ai multi-node http2 load-balancer rpc asyncio ray model-serving gpu-monitoring inference-optimization llm llama-cpp ollama distributed-inference hybrid-routing
-
Updated
Nov 14, 2025 - Python
Official impl. of ACM MM paper "Identity-Aware Attribute Recognition via Real-Time Distributed Inference in Mobile Edge Clouds". A distributed inference model for pedestrian attribute recognition with re-ID in an MEC-enabled camera monitoring system. Jointly training of pedestrian attribute recognition and Re-ID.
mec re-identification edge-computing re-id attribute-recognition mobile-edge-computing distributed-inference
-
Updated
Apr 26, 2020 - Python
A comprehensive framework for multi-node, multi-GPU scalable LLM inference on HPC systems using vLLM and Ollama. Includes distributed deployment templates, benchmarking workflows, and chatbot/RAG pipelines for high-throughput, production-grade AI services
hpc rag ai-inference vllm llm-inference ollama distributed-inference vllm-serve ai-inference-server multinode-training
-
Updated
Dec 10, 2025 - Python
Encrypted Decentralized Inference and Learning (E.D.I.L.)
deep-neural-networks deep-learning he federated-learning decentralized-training distributed-inference ratio1
-
Updated
Dec 3, 2025 - Python
neurogrid-inference delivers decentralized AI inference for NeuroGrid, enabling secure, on-chain-verified model outputs. Supports biomedical AI agents, permissioned data retrieval, encrypted model calls, and verifiable inference proofs. Built for BNB Chain and fully open-source.
hackathon web3 ai-agents data-governance desci ai-inference bnb-chain biomedical-ai opbnb model-execution distributed-inference verifiable-ai onchain-inference secure-compute decentralized-compute
-
Updated
Dec 10, 2025
Improve this page
Add a description, image, and links to the distributed-inference topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the distributed-inference topic, visit your repo's landing page and select "manage topics."