HEX File Analysis Toolkit

Overview

This project provides a set of Python scripts to analyze Intel HEX files, with a focus on extracting and parsing data from an example HEX file (example.hex). The file is believed to be a runtime log or performance record generated by a system with 8 chips, containing configuration and test data. The toolkit includes scripts to extract ASCII strings, non-FF bytes, and XML data blocks to aid in understanding the chip behavior.

Background

The original HEX file (119324.hex) was converted to a binary format and analyzed to reveal:

• Device identifiers (e.g., "E8030-66423")
• 16 XML-formatted data blocks, likely representing 8 chips (each with a calibration and test result block)
• Timestamps (1628168128 and 1628169865, corresponding to 2021-08-05 12:15:28 UTC and 12:44:25 UTC)
• Parameters such as 1024, 1028, and large numbers (e.g., 9956106017), possibly indicating memory sizes, calibration values, or performance counters

The file is renamed to example.hex as a generic repsresentation for analysis purposes.

Files

• ascii_from_hex.py: Extracts continuous ASCII printable strings from the HEX file and outputs their offsets and contents.
• hex_non_ff_extractor.py: Extracts and prints the positions and values of non-FF bytes from the HEX file, including statistics (total bytes, non-FF count, percentage).
• hex_xml_parser.py: Parses and extracts XML data blocks from the HEX file, focusing on <i64> tags, and outputs the parsed numbers.
• example.hex: An example Intel HEX file containing runtime log data for 8 chips. This is a renamed version of the original 119324.hex for demonstration purposes.
• output/: Directory containing the output files generated by the scripts:
  • ascii_strings.txt: Extracted ASCII strings with their offsets.
  • non_ff_bytes.txt: List of non-FF bytes with their offsets and values.
  • xml_blocks.txt: Parsed XML blocks with their <i64> contents.

Requirements

• Python 3.x
• intelhex library (pip install intelhex)
• re (regular expression) module (included in Python standard library)

Installation

1. Install Python 3.x from python.org.
2. Install the required intelhex library:

   pip install intelhex
   

Usage

Each script can be run independently with the path to <example.hex> (or any other HEX file) as an argument.

1. Extract ASCII Strings

Run ascii_from_hex.py to extract continuous ASCII strings:

python ascii_from_hex.py path to <example.hex>

• Output: Prints offsets and ASCII strings to the console and saves them to output/ascii_strings.txt.
• Example Output:

Offset: 0x00000000, String: E8030-66423
Offset: 0x0000000d, String: 2,=4=
Offset: 0x0000001a, String: E8030-66620 012 ZZ 130 026674
...
Extraction results have been saved to output/ascii_strings.txt

2. Extract Non-FF Bytes

Run hex_non_ff_extractor.py to extract non-FF bytes and calculate statistics:

python hex_non_ff_extractor.py path to <example.hex>

• Output: Prints offsets and values of non-FF bytes to the console, displays statistics, and saves results to output/non_ff_bytes.txt.
• Example Output:

Positions and values of non-FF bytes:
Offset: 0x00000000, Value: 0x45 (69)
Offset: 0x00000001, Value: 0x38 (56)
...
Total bytes: 65536
Non-FF byte count: 5878
Non-FF byte percentage: 8.97%
Extraction results have been saved to output/non_ff_bytes.txt

3. Parse XML Blocks

Run hex_xml_parser.py to extract and parse XML data blocks (focusing on tags):

python hex_xml_parser.py path to <example.hex>

• Output: Prints the offset and parsed block contents to the console and saves them to output/xml_blocks.txt.
• Example Output:

Parsed XML blocks:

Offset: 0x000009bb
i64 Block 1: ['1', '1628168128', '10000', '8', '8', '0']
i64 Block 2: ['1028', '1028', '1028', '1028', '1028', '1028']
i64 Block 3: ['1024', '1024', '1024', '1024', '1024', '1024']
i64 Block 4: ['1041', '1041', '1038', '1038', '1038', '1042']
i64 Block 5: ['1026', '1026', '1026', '1026', '1025', '1025']
...

Parsing results have been saved to output/xml_blocks.txt

Results

• example.hex is likely a runtime log or performance record file generated by a system with 8 chips.
• Calibration data (timestamp 1628168128) includes parameters such as memory block sizes (1024), baseline values (1028), and calibration values (1041, 1038, etc.), recorded at 2021-08-05 12:15:28 UTC.
• Test results (timestamp 1628169865) include large numbers (e.g., 9956106017), possibly performance counters or accumulated metrics, recorded at 2021-08-05 12:44:25 UTC.

Limitations

• Lack of specific chip model or hardware details limited precise interpretation of the parameters.
• Unable to perform hardware testing to validate the data's purpose or accuracy.