Multi-Threaded Log Processing System (Core Java)

A Java CLI tool that processes large log files and produces severity counts (INFO/WARN/ERROR).
Includes a single-thread baseline and a multi-thread implementation using ExecutorService, Callable, and Future, with correctness verification across thread counts.

Features

• Single-thread log processing (baseline)
• Multi-thread processing with fixed thread pool
• Chunk-based parallel processing
• Timing breakdown: read time, processing time, total time
• Correctness verification (multi-thread results must match baseline)
• CLI arguments: --file and --threads
• Benchmark runner for 1/2/4/8 threads

Project Structure

• processor/SingleThreadProcessor – baseline (streaming read + count)
• processor/MultiThreadProcessor – thread pool processing + merge
• model/ProcessingResult – holds counts and timing stats
• LogFileGenerator – generates large test log files
• BenchmarkRunner – runs benchmarks across thread counts

How to Run

Generate a large log file

Run:

• LogFileGenerator

It creates: logs/big.log (default 500,000 lines)

Run the main program (CLI)

Run:

• Main

Program arguments example:

--file=logs/big.log --threads=8

Run benchmark suite

Run:

• BenchmarkRunner

Sample Benchmark Results (500,000 lines)

Baseline (single-thread): 87 ms

Threads	Read (ms)	Process (ms)	Total (ms)	Match Baseline
1	54	37	92	✅
2	38	32	71	✅
4	82	5	87	✅
8	54	4	58	✅

Note: Total time varies due to disk I/O and OS scheduling. Processing time scales with threads, while read time may fluctuate.