CSV Data Specification for PQC Hyperledger Fabric Benchmark

# CSV Data Specification for PQC Hyperledger Fabric Benchmark

## Overview

This document defines the data structure, expected ranges, and correlations for the Enhanced CSV Structure used in Post-Quantum Cryptography (PQC) Hyperledger Fabric benchmarking.

---

## Q1: Timestamp Format

### Recommendation: Unix Epoch Time (seconds with decimals)

**Type:** Monotonically increasing float
**Format:** `1735920000.000` (Unix epoch) or ISO 8601 string
**Resolution:** 1 second (or sub-second if possible)
**Behavior:** Does NOT reset during a run, continuously increments

### Options

| Option | Format | Example | Use Case |
|--------|--------|---------|----------|
| **Unix Epoch** | Float | `1735920123.456` | Analysis, plotting |
| **ISO 8601** | String | `2024-12-04T10:30:45.123` | Human-readable logs |
| **Relative** | Integer | `0, 1, 2, 3, ...` | Workshop demos |

### Example
```csv
timestamp,crypto_mode,load_profile,run_id,tx_rate,...
1735920000.000,ECDSA,LOWLOAD,RUN1,120,...
1735920001.000,ECDSA,LOWLOAD,RUN1,118,...
1735920002.000,ECDSA,LOWLOAD,RUN1,122,...
```

### Important
- **Sequential**: Increments by ~1 second (or sampling interval)
- **Independent**: Not correlated with latency or block times
- **Continuous**: No reset between rows in the same run

---

## Q2: Latency Metrics - Ranges & Correlations

### Expected Ranges (milliseconds)

| Crypto Mode | latency_avg | latency_p95 | latency_p99 |
|-------------|-------------|-------------|-------------|
| **ECDSA** | 50-200 ms | 150-400 ms | 200-600 ms |
| **DILITHIUM3** | 80-350 ms | 200-700 ms | 300-1000 ms |
| **HYBRID** | 65-280 ms | 180-550 ms | 250-800 ms |

### Correlations

```
latency ∝ 1/tx_rate          → Higher load = Higher latency
latency ∝ crypto_complexity   → DILITHIUM3 > HYBRID > ECDSA
latency ∝ block_size          → Larger blocks = Higher latency
latency ∝ node_count          → More nodes = Slower consensus
latency ∝ network_latency     → Network delay impact
```

### Load Profile Impact

**LOWLOAD (50-150 TPS):**
```csv
timestamp,crypto_mode,load_profile,tx_rate,latency_avg,latency_p95
0,ECDSA,LOWLOAD,95,75,150
1,ECDSA,LOWLOAD,98,72,145
2,ECDSA,LOWLOAD,92,78,155
```

**HIGHLOAD (500-800 TPS):**
```csv
timestamp,crypto_mode,load_profile,tx_rate,latency_avg,latency_p95
0,ECDSA,HIGHLOAD,520,280,550
1,ECDSA,HIGHLOAD,495,310,620
2,ECDSA,HIGHLOAD,540,265,520
```

### Relationship
- **LOWLOAD**: Lower tx_rate → Lower latency (less congestion)
- **HIGHLOAD**: Higher tx_rate → Higher latency (more congestion)

### Validation Rules
```python
assert latency_p95 > latency_avg
assert latency_p99 > latency_p95
assert latency_p99 < latency_p95 * 2  # Reasonable upper bound
```

---

## Q3: Cryptographic Timing Metrics

### Recommendation: Per-Transaction Averages

**Unit:** Microseconds (μs)
**Calculation:** Mean of all transactions in that sampling interval (e.g., 1 second)

### Expected Ranges (microseconds)

| Crypto Mode | sig_gen_time (μs) | sig_verify_time (μs) | Notes |
|-------------|-------------------|----------------------|-------|
| **ECDSA (P-256)** | 50-150 | 100-250 | Baseline |
| **DILITHIUM3** | 200-500 | 800-1500 | **Verify is slow!** |
| **HYBRID** | 250-650 | 900-1750 | Sum of both |

### Strong Dependencies on Crypto Mode

```python
# DILITHIUM3 is ~3-6x slower than ECDSA
sig_verify_time_DILITHIUM3 ≈ 5 × sig_verify_time_ECDSA
sig_gen_time_DILITHIUM3 ≈ 3 × sig_gen_time_ECDSA

# HYBRID (ECDSA + DILITHIUM3)
sig_gen_time_HYBRID ≈ sig_gen_time_ECDSA + sig_gen_time_DILITHIUM3
sig_verify_time_HYBRID ≈ sig_verify_time_ECDSA + sig_verify_time_DILITHIUM3
```

### Crypto Tax Calculation

```python
# Percentage of time spent on crypto operations
crypto_tax = (sig_gen_time + sig_verify_time) / latency_avg × 100

# Expected values:
# ECDSA:      5-15%  (crypto is fast)
# DILITHIUM3: 20-40% (crypto is bottleneck!)
# HYBRID:     25-45% (double signature overhead)
```

### Example CSV
```csv
timestamp,crypto_mode,sig_gen_time,sig_verify_time,tx_rate
0,ECDSA,85,180,100        # Average of 100 tx in 1 second
1,DILITHIUM3,320,1100,75  # Average of 75 tx in 1 second
2,HYBRID,410,1300,82      # Average of 82 tx in 1 second
```

### Validation Rules
```python
assert sig_verify_time_DILITHIUM3 > sig_verify_time_ECDSA * 3
assert sig_gen_time_DILITHIUM3 > sig_gen_time_ECDSA * 2
assert sig_gen_time > 0 and sig_verify_time > 0
```

---

## Q4: Load Profile Definitions

### Recommended Load Profiles

| Load Profile | Target TPS | Pattern | Duration | Description |
|--------------|-----------|---------|----------|-------------|
| **LOWLOAD** | 50-150 | Steady | 5-10 min | Baseline performance |
| **MEDIUMLOAD** | 200-400 | Steady | 5-10 min | Normal operation |
| **HIGHLOAD** | 500-800 | Steady | 5-10 min | Peak capacity |
| **SUSTAINED** | 300-500 | Constant | 30-60 min | Long-term stability test |

### Alternative Patterns (Optional)

| Load Profile | Pattern | Visual | Use Case |
|--------------|---------|--------|----------|
| **BURST** | Spike + settle | `___/‾‾‾‾\___` | Flash sale scenario |
| **RAMP** | Gradual increase | `___/‾‾‾‾‾` | User onboarding |
| **SPIKE** | Sharp peaks | `_/\_/\_/\_` | Volatile workload |
| **WAVE** | Sinusoidal | `∿∿∿∿` | Daily usage cycles |

### Impact on Metrics

**LOWLOAD → Stable metrics:**
```python
tx_rate: 95, 98, 92, 96, 94      # Low variance
latency_avg: 75, 72, 78, 74      # Predictable
cpu_util: 25, 26, 24, 25         # Steady
```

**HIGHLOAD → Variable metrics:**
```python
tx_rate: 520, 495, 540, 480      # Higher variance
latency_avg: 280, 310, 265, 340  # More jitter
cpu_util: 85, 92, 88, 95         # Near saturation
```

**SUSTAINED → Drift over time:**
```python
# First 5 min:  latency_avg = 150ms, cpu = 60%
# After 30 min: latency_avg = 180ms, cpu = 65%  # Resource exhaustion
```

### Python Implementation Example
```python
load_profiles = {
    "LOWLOAD": {
        "target_tps": 100,
        "variance": 0.1,  # ±10%
        "pattern": "steady"
    },
    "MEDIUMLOAD": {
        "target_tps": 300,
        "variance": 0.15,
        "pattern": "steady"
    },
    "HIGHLOAD": {
        "target_tps": 600,
        "variance": 0.20,
        "pattern": "steady"
    },
    "SUSTAINED": {
        "target_tps": 400,
        "variance": 0.15,
        "pattern": "constant",
        "duration": 1800  # 30 minutes
    }
}
```

---

## Complete CSV Example

```csv
timestamp,crypto_mode,load_profile,run_id,tx_rate,latency_avg,latency_p95,cpu_util,mem_util,block_size,block_commit_time,sig_gen_time,sig_verify_time
1735920000,ECDSA,LOWLOAD,RUN1,95,75,150,28,45,1024,45,85,180
1735920001,ECDSA,LOWLOAD,RUN1,98,72,145,29,46,1056,43,82,175
1735920002,ECDSA,LOWLOAD,RUN1,92,78,155,27,44,998,47,88,185
1735920003,DILITHIUM3,LOWLOAD,RUN1,68,145,310,52,58,1024,85,320,1100
1735920004,DILITHIUM3,LOWLOAD,RUN1,71,138,295,54,59,1089,82,310,1080
1735920005,HYBRID,LOWLOAD,RUN1,82,105,240,38,51,1045,62,410,1300
1735920100,ECDSA,HIGHLOAD,RUN1,520,280,550,85,72,2048,120,90,190
1735920101,DILITHIUM3,HIGHLOAD,RUN1,380,450,880,92,78,2048,200,340,1150
1735920102,HYBRID,HIGHLOAD,RUN1,440,360,720,88,75,2048,165,420,1320
```

---

## Metrics Summary Table

| Metric | Type | Unit | Expected Range | Key Dependencies |
|--------|------|------|----------------|------------------|
| `timestamp` | Float | seconds | Unix epoch | Monotonically increasing |
| `crypto_mode` | String | - | ECDSA, DILITHIUM3, HYBRID | - |
| `load_profile` | String | - | LOWLOAD, MEDIUMLOAD, HIGHLOAD, SUSTAINED | - |
| `run_id` | String | - | RUN1, RUN2, ..., RUN5 | - |
| `tx_rate` | Float | TPS | 50-800 | load_profile |
| `latency_avg` | Float | ms | 50-350 | crypto_mode, load_profile |
| `latency_p95` | Float | ms | 150-700 | 1.5-2x latency_avg |
| `latency_p99` | Float | ms | 200-1000 | 1.2-1.5x latency_p95 |
| `cpu_util` | Float | % | 20-95 | crypto_mode, tx_rate |
| `mem_util` | Float | % | 30-80 | tx_rate, node_count |
| `block_size` | Integer | bytes | 500-2500 | tx_rate, batching policy |
| `block_commit_time` | Float | ms | 30-200 | block_size, consensus |
| `sig_gen_time` | Float | μs | 50-500 | **crypto_mode** (3x for PQC) |
| `sig_verify_time` | Float | μs | 100-1500 | **crypto_mode** (5x for PQC) |

---

## Validation Rules

### Data Consistency Checks

```python
# Latency percentiles must be ordered
assert latency_p95 > latency_avg
assert latency_p99 > latency_p95

# PQC must be slower than ECDSA
assert sig_verify_time_DILITHIUM3 > sig_verify_time_ECDSA * 3
assert sig_gen_time_DILITHIUM3 > sig_gen_time_ECDSA * 2

# Resource utilization must be valid percentages
assert 0 <= cpu_util <= 100
assert 0 <= mem_util <= 100

# Load profile correlations
assert latency_avg_HIGHLOAD > latency_avg_LOWLOAD
assert cpu_util_HIGHLOAD > cpu_util_LOWLOAD

# SUSTAINED runs must be longer
assert duration_SUSTAINED > duration_LOWLOAD * 3

# Timestamp must be monotonic
assert all(timestamps[i] < timestamps[i+1] for i in range(len(timestamps)-1))

# Crypto tax sanity check
crypto_tax = (sig_gen_time + sig_verify_time) / (latency_avg * 1000) * 100
assert 1 <= crypto_tax <= 50  # Between 1% and 50%
```

### Expected Relationships

```python
# Throughput vs Latency (inverse relationship under load)
correlation(tx_rate, latency_avg) < 0  # Negative correlation at high load

# Crypto mode impact on performance
throughput_ECDSA > throughput_DILITHIUM3 > throughput_HYBRID
latency_ECDSA < latency_HYBRID < latency_DILITHIUM3

# Resource utilization vs throughput
correlation(tx_rate, cpu_util) > 0.7  # Strong positive correlation

# Block commit time vs block size
correlation(block_size, block_commit_time) > 0.5  # Moderate positive correlation
```

---

## Data Generation Guidelines

### For Workshop (Quick Start)
- **3 crypto modes** × **2 load profiles** (LOW, HIGH) × **3 runs** = 18 CSV files
- **Duration:** 5 minutes per run
- **Sampling:** 1 row per second (300 rows per file)
- **Focus metrics:** tx_rate, latency_avg, latency_p95, cpu_util, sig_gen_time, sig_verify_time

### For Q1 Journal (Complete)
- **3 crypto modes** × **4 load profiles** × **5 runs** = 60 CSV files
- **Duration:** 10 minutes (LOWLOAD, MEDIUMLOAD, HIGHLOAD), 30 minutes (SUSTAINED)
- **Sampling:** 1 row per second
- **All metrics:** Complete schema with all columns
- **Additional:** Scalability tests with varying node_count (4, 8, 12, 16, 20 nodes)

---

## File Naming Convention

```
<CRYPTO_MODE>_<LOAD_PROFILE>_RUN<N>.csv

Examples:
ECDSA_LOWLOAD_RUN1.csv
ECDSA_LOWLOAD_RUN2.csv
DILITHIUM3_MEDIUMLOAD_RUN1.csv
HYBRID_SUSTAINED_RUN3.csv
```

---

## Key Takeaways

1. **Timestamp**: Unix epoch, monotonically increasing, 1-second intervals
2. **Latency**: DILITHIUM3 is 1.5-2.5x slower than ECDSA, strongly correlated with crypto_mode and load_profile
3. **Crypto Timing**: Per-transaction averages, DILITHIUM3 verify is ~5x slower than ECDSA
4. **Load Profiles**: Define clear TPS targets (LOWLOAD=100, MEDIUMLOAD=300, HIGHLOAD=600, SUSTAINED=400)
5. **Dependencies**: latency ∝ crypto_complexity, tx_rate ∝ load_profile, cpu_util ∝ tx_rate
6. **Validation**: Always check percentile ordering, crypto mode ratios, and resource bounds

---

## References

- NIST PQC Standardization: CRYSTALS-Dilithium
- Hyperledger Fabric Performance Benchmarking
- Post-Quantum Cryptography in Blockchain: Survey and Analysis

Option	Format	Example	Use Case
Unix Epoch	Float	`1735920123.456`	Analysis, plotting
ISO 8601	String	`2024-12-04T10:30:45.123`	Human-readable logs
Relative	Integer	`0, 1, 2, 3, ...`	Workshop demos

Crypto Mode	latency_avg	latency_p95	latency_p99
ECDSA	50-200 ms	150-400 ms	200-600 ms
DILITHIUM3	80-350 ms	200-700 ms	300-1000 ms
HYBRID	65-280 ms	180-550 ms	250-800 ms

Crypto Mode	sig_gen_time (μs)	sig_verify_time (μs)	Notes
ECDSA (P-256)	50-150	100-250	Baseline
DILITHIUM3	200-500	800-1500	Verify is slow!
HYBRID	250-650	900-1750	Sum of both

Load Profile	Target TPS	Pattern	Duration	Description
LOWLOAD	50-150	Steady	5-10 min	Baseline performance
MEDIUMLOAD	200-400	Steady	5-10 min	Normal operation
HIGHLOAD	500-800	Steady	5-10 min	Peak capacity
SUSTAINED	300-500	Constant	30-60 min	Long-term stability test

Load Profile	Pattern	Visual	Use Case
BURST	Spike + settle	`___/‾‾‾‾\___`	Flash sale scenario
RAMP	Gradual increase	`___/‾‾‾‾‾`	User onboarding
SPIKE	Sharp peaks	`_/\_/\_/\_`	Volatile workload
WAVE	Sinusoidal	`∿∿∿∿`	Daily usage cycles

Metric	Type	Unit	Expected Range	Key Dependencies
`timestamp`	Float	seconds	Unix epoch	Monotonically increasing
`crypto_mode`	String	-	ECDSA, DILITHIUM3, HYBRID	-
`load_profile`	String	-	LOWLOAD, MEDIUMLOAD, HIGHLOAD, SUSTAINED	-
`run_id`	String	-	RUN1, RUN2, ..., RUN5	-
`tx_rate`	Float	TPS	50-800	load_profile
`latency_avg`	Float	ms	50-350	crypto_mode, load_profile
`latency_p95`	Float	ms	150-700	1.5-2x latency_avg
`latency_p99`	Float	ms	200-1000	1.2-1.5x latency_p95
`cpu_util`	Float	%	20-95	crypto_mode, tx_rate
`mem_util`	Float	%	30-80	tx_rate, node_count
`block_size`	Integer	bytes	500-2500	tx_rate, batching policy
`block_commit_time`	Float	ms	30-200	block_size, consensus
`sig_gen_time`	Float	μs	50-500	crypto_mode (3x for PQC)
`sig_verify_time`	Float	μs	100-1500	crypto_mode (5x for PQC)

CSV Data Specification for PQC Hyperledger Fabric Benchmark #8

Description

CSV Data Specification for PQC Hyperledger Fabric Benchmark

Overview

Q1: Timestamp Format

Recommendation: Unix Epoch Time (seconds with decimals)

Options

Example

Important

Q2: Latency Metrics - Ranges & Correlations

Expected Ranges (milliseconds)

Correlations

Load Profile Impact

Relationship

Validation Rules

Q3: Cryptographic Timing Metrics

Recommendation: Per-Transaction Averages

Expected Ranges (microseconds)

Strong Dependencies on Crypto Mode

Crypto Tax Calculation

Example CSV

Validation Rules

Q4: Load Profile Definitions

Recommended Load Profiles

Alternative Patterns (Optional)

Impact on Metrics

Python Implementation Example

Complete CSV Example

Metrics Summary Table

Validation Rules

Data Consistency Checks

Expected Relationships

Data Generation Guidelines

For Workshop (Quick Start)

For Q1 Journal (Complete)

File Naming Convention

Key Takeaways

References

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions