This repository presents a decentralized extension of Retrieval-Augmented Generation (RAG), addressing privacy, scalability, and security challenges in traditional RAG systems using IPFS, MQTT, and blockchain technologies.
This work is grounded in our research on decentralized learning systems, particularly Decentralized Retrieval-Augmented Generation (DRAG), as presented in [3].
DRAG enables users to interact with local knowledge bases while contributing to a global shared database—promoting knowledge democratization, trustless collaboration, and incentivized participation.
This repository is based on our published research work:
Continuous Learning in Decentralized Retrieval-Augmented Generation (DRAG) and Data Management F. A. Khan, C. Peiper, A. Jaberzadeh, M. A. Shaikh, et al. Proceedings of the 4th Blockchain and Cryptocurrency Conference (B2C'25), 2025, pp. 45–48
DRAG enhances traditional RAG by decentralizing storage, communication, and computation layers, ensuring:
- Privacy → Secure, decentralized data storage using IPFS
- Scalability → Distributed knowledge contribution without central bottlenecks
- Security → Blockchain-backed transparency and tamper-proof records
- Incentivization → Reward mechanisms for contributors
- Collaborative Learning → Continuous improvement from distributed nodes
- IPFS → Decentralized storage layer
- MQTT → Lightweight, low-latency communication protocol
- Blockchain → Trustless validation and reward system
- Qdrant → High-performance vector database for semantic retrieval
The core difference lies in centralization vs decentralization.
Centralized architecture with a single knowledge base
Decentralized architecture with multiple nodes contributing to a global knowledge base
The system consists of two primary node types:
- Provide domain-specific knowledge
- Generate embeddings and contribute to the global vector database
- Validate incoming contributions
- Maintain data quality and integrity
- Records contributions transparently
- Rewards high-quality data providers
- Penalizes malicious or low-quality inputs
This mechanism ensures:
- Higher data reliability
- Sustainable ecosystem growth
- Trustless collaboration
git clone https://github.com/bayesianinstitute/Decentralized-RAG
cd Decentralized-RAGpython setup.py sdist bdist_wheel
pip install .Start all services:
docker compose up -dDownload model and run:
bash run.shdocker pull qdrant/qdrantdocker run -d -p 6333:6333 -p 6334:6334 \
-v ./qdrant_data:/qdrant/storage \
qdrant/qdrantdocker run -d --name qdrant_container -p 6333:6333 -p 6334:6334 \
-v C:/path/to/qdrant_data:/qdrant/storage \
qdrant/qdrant:latestFollow: https://ollama.ai/
ollama pull llama3:8bollama pull nomic-embed-text:latestEdit main.py:
admin→ Global coordinator nodedata→ Knowledge contributor node
python main.py --data-dir data --nodetype adminFollow official docs: https://docs.ipfs.tech/install/
wget https://dist.ipfs.tech/kubo/v0.23.0/kubo_v0.23.0_windows-amd64.zip -Outfile kubo_v0.23.0.zip
Expand-Archive -Path kubo_v0.23.0.zip -DestinationPath .\kubo
cd .\kubo
.\install.batDRAG demonstrates how decentralized architectures can significantly enhance Retrieval-Augmented Generation systems by:
- Reducing hallucination and retrieval errors
- Preserving user privacy
- Enabling continuous decentralized learning
- Incentivizing high-quality knowledge contributions
By integrating blockchain, distributed storage, and vector search, DRAG provides a scalable and secure foundation for next-generation AI systems.
GitHub: https://github.com/bayesianinstitute/Decentralized-RAG
[1] Ollama Docker Hub: https://hub.docker.com/r/ollama/ollama [2] IPFS Documentation: https://docs.ipfs.tech
[3] F. A. Khan, C. Peiper, A. Jaberzadeh, M. A. Shaikh, et al., "Continuous learning in decentralized retrieval-augmented generation (DRAG) and data management," in Proceedings of the 4th Blockchain and Cryptocurrency Conference (B2C'25), 2025, pp. 45–48. Available: https://www.researchgate.net/profile/Sergey-Yurish/publication/398276090_Blockchain_and_Cryptocurrency_B2C'_2025_Edited_by_Sergey_Y_Yurish/links/6930219f0e91876082c0d022/Blockchain-and-Cryptocurrency-B2C-2025-Edited-by-Sergey-Y-Yurish.pdf#page=46