Pulse: Fine-Grained Hierarchical Hashing Index for Disaggregated Memory

Pulse is a hierarchical hashing index designed for disaggregated memory (DM); it leverages one-sided RDMA verbs to achieve efficient indexing operations. Pulse introduces three core techniques to overcome traffic amplification and high insertion latency in DM:

• Multi-level index structure for fine-grained directory synchronization, reducing traffic during concurrent insertions.

• Predictor-assisted resizing that maintains partial hashed keys in memory pools to guide item relocation without extra RTTs.

• RTT-optimized batching using doorbell batching with selective signaling to accelerate insertions and updates.

For more details, please refer to our paper:

[HPCA'26] Pulse: Fine-Grained Hierarchical Hashing Index for Disaggregated Memory. Guangyang Deng, Zixiang Yu, Zhirong Shen, Qiangsheng Su, Zhinan Cheng and Jiwu Shu.

Project Structure

Pulse follows a disaggregated memory architecture with clear separation between storage and computation:

pulse/
├── memory_pool/         # Memory Pool
│
├── compute_pool/        # Compute Pool
│
├── core/                # Core Implementation
│   └── pulse/           # Pulse hierarchical hashing index
│
├── workload/            # Workload Implementations
│   ├── pulse/           # Pulse workload definition
│   └── config/          # Shared workload configuration types
│
├── scripts/             # Management Scripts
│
└── config/              # Configuration Files

Getting Started

Prerequisites

• A cluster environment with RDMA-capable network interfaces
• Memory node: The node that hosts the memory pool server
• Compute nodes: The nodes that execute workload operations

Step 1: Clone the Repository

Clone the Pulse repository and navigate to the project directory:

git clone https://github.com/syslucas/pulse
cd pulse

Step 2: Download and Setup YCSB

Download and extract YCSB (Yahoo! Cloud Serving Benchmark) for workload generation:

curl -O --location https://github.com/brianfrankcooper/YCSB/releases/download/0.17.0/ycsb-0.17.0.tar.gz
tar xfvz ycsb-0.17.0.tar.gz

This will create the ycsb-0.17.0/ directory in the project root.

Step 3: Configure Settings

Edit scripts/settings.sh to configure your cluster environment. All script operations should be performed on the memory node.

Key Configuration Items:

1. YCSB Data Path (DATA_PATH_BASE):

   • Automatically set to ${BASE_DIR_PREFIX}/ycsb_output
   • This is where generated workload data files will be stored
   • You can modify the base path if needed

2. Log Directory (LOG_DIR):

   • Automatically set to ${BASE_DIR_PREFIX}/logs
   • Execution logs from compute nodes will be collected here

3. Compute Nodes (COMPUTE_NODES):

   • Configure the list of compute nodes in the format username@ip_address
   • Example: COMPUTE_NODES=("user@192.168.1.2" "user@192.168.1.3")

4. Memory Node (MEMORY_NODE_IP, MEMORY_NODE_PORT, MEMORY_NODE_META_PORT):

   • Set the memory node IP address
   • Configure RDMA port and metadata port numbers

5. SSH Port (SSH_PORT):

   • Configure the SSH port for remote operations (default: 2222)

Important Note: All scripts in the scripts/ directory are designed to run on the memory node only. The scripts handle cross-machine communication and remote execution automatically, so you don't need to manually run anything on compute nodes.

Step 4: Generate Workloads and Run Experiments

All following steps should be executed on the memory node. Replace $MACHINE_NUM with the actual number of compute nodes you want to use (e.g., 1, 2, or 4).

4.1 Generate YCSB Workloads

Generate YCSB workload data files and perform preprocessing:

./scripts/generate_ycsb_workloads.sh

This script will:

• Generate workload files for different request distributions (zipfian, uniform)
• Create load and run workloads with various operation counts
• Post-process and format the generated data files
• Split data for multi-node configurations (2-node, 4-node)
• Output files will be stored in ${DATA_PATH_BASE}/1-node/<distribution>/ and split into ${DATA_PATH_BASE}/<n>-node/<distribution>/ directories (where <n> is the number of nodes)

Note: This step can be time-consuming depending on the workload size. Ensure you have sufficient disk space before running.

4.2 Distribute YCSB Data to Compute Nodes

Distribute the generated YCSB workload data to compute nodes:

./scripts/cluster_communication.sh ycsb_broadcast $MACHINE_NUM

Or specify a specific dataset path:

./scripts/cluster_communication.sh ycsb_broadcast $MACHINE_NUM 4-node/zipfian

This will copy the YCSB data directory to all compute nodes specified in settings.sh. The data will be placed in the same path on each compute node as configured in DATA_PATH_BASE.

4.3 Distribute Source Code and Build

Distribute the source code to compute nodes and build the project on each node:

./scripts/cluster_communication.sh source_broadcast $MACHINE_NUM

This script will:

• Copy the entire project directory to each compute node
• Build the project using ./scripts/debug.sh on each compute node
• Ensure all compute nodes have the latest code compiled and ready

4.4 Run Experiments

Execute the Pulse experiments:

./scripts/ex.sh pulse

This will:

• Start the memory pool server on the memory node
• Launch compute nodes to execute the Pulse workload
• Wait for all compute nodes to finish execution

The experiment configuration (thread numbers, coroutine counts, etc.) can be modified in scripts/ex.sh before running.

Note: Make sure huge pages are enabled on the memory node before running experiments. You can use ./scripts/hugepage.sh if needed.

4.5 Collect Logs

After experiments complete, collect logs from all compute nodes:

./scripts/cluster_communication.sh log_gather $MACHINE_NUM /path/to/your/logs_collect

Replace /path/to/your/logs_collect with your desired local target directory. This will:

• Collect logs from each compute node's LOG_DIR (configured in settings.sh)
• Store logs in separate subdirectories for each node in the target directory
• Organize logs by node for easy analysis

The log directory structure will be:

/path/to/your/logs_collect
├── user@192.168.6.2/
│   └── [log files]
├── user@192.168.6.3/
│   └── [log files]
└── ...

Acknowledgments

This project builds on the RDMA framework from FORD [FAST'22], and we thank the authors for open-sourcing it.