Skip to content

SMB/NTLM capture server rewrite#38

Open
StrongWind1 wants to merge 8 commits intoMatrixEditor:masterfrom
StrongWind1:update/smb-ntlm-pr
Open

SMB/NTLM capture server rewrite#38
StrongWind1 wants to merge 8 commits intoMatrixEditor:masterfrom
StrongWind1:update/smb-ntlm-pr

Conversation

@StrongWind1
Copy link
Copy Markdown
Contributor

This also closes #24

What?

This rewrites smb.py and ntlm.py to capture NTLM hashes from every Windows version (XP RTM through Win 11 25H2, Server 2003 through Server 2025) across all SMB dialects (NT LM 0.12 through SMB 3.1.1). It also centralizes NTLM configuration to a single [NTLM] section, updates all protocol handlers to use the new API, adds 514 unit tests, and brings all documentation in line with the code.

26 files changed, +8435/-2165 lines across 8 commits.

Why?

The previous SMB server only negotiated SMB 2.1 (hardcoded), had no SMB 3.1.1 negotiate context support, no tree connect or path capture, and no SMB1 basic-security handling. NTLM hash extraction had no ESS detection, no LMv2 companion capture, no dummy LM filtering, and no anonymous detection. NTLM config was scattered across every protocol's _fields_ list with per-protocol overrides that made behavior hard to predict.

How?

ntlm.py

Full NTLMSSP implementation: NTLM_build_challenge_message constructs the CHALLENGE from client NEGOTIATE flags with configurable AV_PAIRs, ESS/NTLMv2 control, and VERSION. NTLM_handle_authenticate_message classifies hashes by byte structure (ESS detection from LM[8:24]==Z(16) is authoritative over flags), formats hashcat lines (-m 5500 and -m 5600), filters dummy LM responses, and detects anonymous sessions. NTLM_handle_legacy_raw_auth handles SMB1 basic-security (non-NTLMSSP) separately. MsvAvTimestamp is intentionally omitted to maximize LMv2 companion capture. All 10 NTLM config attributes use section_local=False so they're only read from [NTLM].

smb.py

Three authentication paths: SMB2/3 (dialect 2.002-3.1.1 with negotiate contexts for preauth, encryption, signing), SMB1 Extended (SPNEGO/NTLMSSP, WordCount=12), and SMB1 Basic (raw challenge/response, WordCount=13, cleartext). Post-auth stub handlers for create, read, write, close, query info, query directory, IOCTL (validate negotiate + DFS), flush, lock, set info. Tree connect with UNC path capture for both SMB1 and SMB2. Filename capture from CREATE. CapturesPerConnection for multi-credential SSPI retry. Per-connection client info summary logging.

Protocol handlers

Removed NTLM_split_fqdn and per-protocol ATTR_NTLM_* from http, smtp, imap, pop3, ldap, mssql, and rpc. All now read NTLM config from SessionConfig (populated by [NTLM] apply_config). Added NTLM_handle_negotiate_message calls, keyword args for authenticate, and per-connection logger on challenge messages. Fixed HTTP to use HTTP/1.1 for connection-based NTLM auth. Fixed SMTP to return explicit AuthResult from downgrade path.

Config

Rewrote [SMB] and [NTLM] sections in Dementor.toml. Every option documented with default and description. Challenge default changed from fixed to random (commented out). Per-protocol NTLM overrides removed from all protocol sections.

Documentation

Rewrote smb.rst and ntlm.rst to match the code. Removed per-protocol NTLM config sections from dcerpc, http, imap, pop3, mssql docs. Updated compat.rst with current capabilities and Responder bugs (hardcoded dialect, threshold inconsistency, AV_PAIR swap, missing flags). Fixed stale ESS attribute names in example docs.

Testing?

514 unit tests covering every public function in ntlm.py and smb.py. Tests include real pcap vectors from 14 Windows versions (TCP-flow-matched challenges from a filtered capture), hash classification, hashcat format validation, ESS detection, anonymous probe detection, and negotiate flag coverage. SMB tests use object.__new__ mock handlers with manually set state.

End-to-end tested with:

  • curl: 38 tests across HTTP, WinRM, SMB, SMTP, IMAP, POP3, FTP, LDAP
  • smbclient: SMB 2.002, SMB 2.1, SMB 3.1.1, SMB1 NT1, port 139
  • Win XP RTM through Win 11 25H2
  • Server 2003 through Server 2025
  • Win NT 4.0
  • hashcat: all captured hashes crack (-m 5500 and -m 5600)
uv run --group dev ruff check .    # passes
uv run --group dev pytest -q       # 514 passed, 1 skipped

Commits

7570635 feat: comprehensive NTLM protocol implementation
76069e0 feat: comprehensive SMB server implementation
29894b0 refactor: update all protocols for centralized NTLM config
3dafd86 config: rewrite SMB and NTLM sections in Dementor.toml
eb8a125 docs: rewrite smb.rst and ntlm.rst
5a7a1bb docs: remove per-protocol NTLM config from all protocol docs
7e072fe docs: update Responder vs Dementor compatibility matrix
bbafdf1 tests: comprehensive unit tests for SMB and NTLM modules

Rewrite ntlm.py with full NTLMSSP handling for all hash types:
NetNTLMv1, NetNTLMv1-ESS, NetNTLMv2, and NetLMv2.

NTLM handshake:
- NTLM_build_challenge_message: construct CHALLENGE from NEGOTIATE
  flags with configurable AV_PAIRs, ESS/NTLMv2 control, VERSION
- NTLM_handle_negotiate_message: parse and log NEGOTIATE fields
- NTLM_handle_authenticate_message: extract identity, classify hash
  type, format hashcat lines, detect anonymous/ESS/NTLMv2
- NTLM_handle_legacy_raw_auth: SMB1 basic-security (non-NTLMSSP)
  hash extraction with OS/domain metadata in host_info
- NTLM_to_hashcat: hashcat -m 5500 (v1/ESS) and -m 5600 (v2/LMv2)
- ESS detection: LM[8:24]==Z(16) is authoritative over flags
- MsvAvTimestamp intentionally omitted to maximize LMv2 capture
- Per-connection protocol logger context on CHALLENGE log line
- Type-safe default_val assignments in apply_config()

Config changes:
- All NTLM config centralized to [NTLM] section (section_local=False)
- No per-protocol NTLM overrides
- 10 ATTR_NTLM_* attributes with defaults matching Dementor.toml

Supporting files:
- spnego.py: NegTokenResp construction updates
- session.py: type hints for 7 new NTLM attributes
- toml.py: config resolution updates
- log/__init__.py: protocol logger support
Rewrite SMB handler with full dialect support from NT LM 0.12
through SMB 3.1.1.

SMB2/3 (Path A):
- Multi-dialect negotiate with negotiate contexts (preauth integrity,
  encryption capabilities, signing capabilities for SMB 3.1.1)
- SESSION_SETUP with SPNEGO/NTLMSSP auth flow
- TREE_CONNECT with path capture from wire bytes (PathOffset/
  PathLength, bypasses impacket AlignPad off-by-one bug)
- CREATE, CLOSE, READ, WRITE, FLUSH, LOCK, SET_INFO, QUERY_INFO,
  QUERY_DIRECTORY, IOCTL (VALIDATE_NEGOTIATE_INFO + DFS) handlers
- Server-allocated random SessionID
- CapturesPerConnection for multi-credential SSPI retry on SMB1+SMB2

SMB1 Extended (Path B):
- SESSION_SETUP WordCount=12 with NTLMSSP/SPNEGO
- Shared handle_ntlmssp code path with SMB2

SMB1 Basic (Path C):
- SESSION_SETUP WordCount=13 with raw LM/NT response extraction
- Cleartext password capture from non-encrypted sessions
- Manual string parsing with alignment pad calculation (bypasses
  impacket SMBTreeConnectAndX_Data missing pad field)

SMB1 shared:
- NT_CREATE_ANDX, TRANS2 (15 info levels from pcap), READ, CLOSE
- TREE_CONNECT with path capture
- SMB1-to-SMB2 protocol upgrade

Post-auth:
- ConnectionResetError/BrokenPipeError handling at dispatch level
- OS/LanMan metadata passed through extras for host_info display
- Per-connection client info summary (os, lanman, path, dialect, files)
Update HTTP, SMTP, IMAP, POP3, LDAP, MSSQL, and RPC handlers:

- Remove NTLM_split_fqdn usage; use config ntlm_nb_computer/nb_domain
- Remove per-protocol ATTR_NTLM_* field overrides from _fields_
- Add NTLM_handle_negotiate_message call with negotiate_fields passing
- Use keyword args for NTLM_handle_authenticate_message
- Pass per-connection logger via log= to NTLM_build_challenge_message
- HTTP: set protocol_version to HTTP/1.1, add Content-Length:0 on
  401 CHALLENGE response, store negotiate_fields on self
- SMTP: return AuthResult(success=False, handled=True) instead of None
  from downgrade path; set logger host from session peer; fix typo
Rewrite [SMB] and [NTLM] config sections to match current code:

[NTLM] section:
- Challenge commented out (code default is random, not fixed)
- Every option marked optional with description and default
- Accepted challenge formats documented (hex:, ascii:, auto-detect)
- AV_PAIR fields documented with spec IDs (0x0001-0x0005)
- Required vs optional AV_PAIRs noted

[SMB] section:
- Identity options moved below post-auth behaviour
- ErrorCode accepts integer or string name (via getattr)
- All defaults match SMBServerConfig._fields_

Other protocol sections:
- Remove per-protocol NTLM overrides (Challenge, ESS, NTLMv2)
  from SMTP, HTTP, LDAP, RPC, MSSQL, POP3, IMAP
- Replace with one-line note pointing to [NTLM]
- Update resolution order header to reflect global-only NTLM
smb.rst:
- Rewrite intro to list all implemented handlers (create, read,
  write, close, query info, IOCTL, etc.) and captured data
- Reorder auth paths: SMB2/3 (Path A), SMB1 Extended (B), Basic (C)
- Add tested client list: XP RTM through Win 11 25H2, Server 2003
  through Server 2025, NT 4.0, smbclient, curl smb://
- Replace client info table with note linking to ntlm.rst
- Add Logging section with info-level summary format and debug
  per-command examples
- Fix CapturesPerConnection: CPC=0 returns STATUS_SUCCESS for
  tree connect, not the configured ErrorCode
- Remove IS_GUEST section (internal detail)

ntlm.rst:
- Clarify ESS as server-controlled: server echoes flag back,
  client falls back to plain NTLMv1 if server strips it
- Fix LmCompatibilityLevel table: all levels 0-2 produce ESS
  when server echoes it, not just level 1
- Fix interaction table: Level 2 shows NTLMv1-ESS with defaults
- Remove Extended Protection and Channel Binding Tokens section
- Simplify LMv2 suppression to two reasons (MsvAvTimestamp, Win 7+)
- Simplify observed behavior table to two rows
- Add Linked-to cross-refs for 7 identity options
- Remove phantom LMv2 Z(16) condition
- Remove per-protocol NTLM override references throughout
Remove stale per-protocol NTLM attribute sections (ESS, Challenge,
resolution precedence lists) from dcerpc.rst, http.rst, imap.rst,
pop3.rst, and mssql.rst. Replace with note pointing to [NTLM].

Update examples:
- multicast.rst: fix attribute name to DisableExtendedSessionSecurity,
  use NTLM. prefix instead of RPC. prefix in CLI example
- smtp_downgrade.rst: fix attribute name to
  DisableExtendedSessionSecurity
SMB Features table:
- Tree connect: Responder SMB1 only, Dementor SMB1+SMB2
- Cleartext capture: both supported (was marked X for Dementor)
- Multi-credential: Responder SMB1 only, Dementor SMB1+SMB2
- Add: filename capture, client info extraction, SMB 3.1.1
  negotiate contexts, configurable dialect range, per-listener
  config, SMB2 SessionID allocation
- Surface Responder bugs: hardcoded SMB 2.1 dialect, echoed
  SessionID

NTLM Specifics table:
- Add: LMv2 companion capture, client OS/version extraction
- Fix threshold label: > 24 bytes (not >= 48 B)
- Surface Responder bugs: AV_PAIR 0x0003/0x0004 values swapped,
  NTLMv2 threshold inconsistency (>60 in SMB, >25 in HTTP),
  HTTP NTLM flags missing SIGN/SEAL/KEY_EXCH/ALWAYS_SIGN,
  SMB NTLM flags missing SEAL

Protocol table:
- SMB 1.0 Raw: Dementor now supported (was X)
- Dementor SMB overall: check-check (was partial)
- NetNTLMv1-ESS Responder: partial + mislabeled (was broken)
- Fix stray double </tr> tag

Remove footnote [1] about NTLMv1-SSP labeling and all references.
Fix heading typo: Spcifics -> Specifics.
514 tests covering every public function in ntlm.py and smb.py.

test_ntlm.py (~275 tests):
- Pure functions: NTLM_timestamp, _config_version_to_bytes,
  NTLM_decode_string, NTLM_encode_string, _classify_hash_type,
  _compute_dummy_lm_responses, NTLM_to_hashcat
- Mock-dependent: NTLM_build_challenge_message, negotiate/authenticate
  handlers, anonymous detection, NTLMv2 blob parsing
- Real pcap vectors from 14 Windows versions (XP RTM through
  Server 2022), TCP-flow-matched challenges
- Hash classification, hashcat format, ESS detection, full auth
  pipeline, anonymous probe detection, negotiate flag coverage

test_smb.py (~101 tests):
- Pure functions: _split_smb_strings, parse_dialect, get_command_name,
  set_smb_error_code, _smb3_neg_context_pad, _build_trans2_file_info
- Response builders: SMB2 negotiate, query_info, create, tree_connect
- Handler tests: SMB2 negotiate, IOCTL, all simple handlers, SMB1
  handlers, dispatch routing
- Mock handler via object.__new__ with manually set state
@MatrixEditor MatrixEditor self-requested a review March 26, 2026 06:45
@MatrixEditor MatrixEditor added Type - Enhancement In Review Protocol: SMB Errors/Features related to the SMB server labels Mar 26, 2026
if self.client_files:
parts.append(f"files:{','.join(sorted(self.client_files))}")
if parts:
self.logger.info("SMB: %s", " | ".join(parts))
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be a debug message.

Suggested change
self.logger.info("SMB: %s", " | ".join(parts))
self.logger.debug("SMB: %s", " | ".join(parts))

is_client=True,
)
if path:
self.client_info["smb_path"] = path
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When a path gets captured, it should be logged immediately (and should get highlighted)

Suggested change
self.client_info["smb_path"] = path
self.client_info["smb_path"] = path
if not path.endswith("IPC$"):
self.logger.success(f"Captured CIFS path from {self.client_host}")
self.logger.highlight(f"Path: [b]{escape(path)}[/]")

else:
# Non-IPC$ disk share — capture the path for intelligence
if path:
self.client_info["smb_path"] = path
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as below; the captured path should get highlighted

Suggested change
self.client_info["smb_path"] = path
self.client_info["smb_path"] = path
self.logger.success(f"Captured CIFS path from {self.client_host}")
self.logger.highlight(f"Path: [b]{escape(path)}[/]")

"SMB_COM_NT_CREATE_ANDX Name=%s", name or "(empty)", is_client=True
)
if name:
self.client_files.add(name)
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The target file name should get logged to the TUI.

Suggested change
self.client_files.add(name)
self.client_files.add(name)
is_ipc = (
"IPC$ "
if self.client_info.get("smb_path", "").endswith("IPC$")
else ""
)
self.logger.success(
f"Captured requested CIFS file name from {self.client_host}"
)
self.logger.highlight(f"FileName: {is_ipc}[b]{escape(name)}[/]")

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There must be at least a ..versionchanged/..versionremoved to denote the attribute removal

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here: There must be at least a ..versionchanged/..versionremoved to denote the attribute removal

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here: There must be at least a ..versionchanged/..versionremoved to denote the attribute removal

lm_response: bytes = token["lanman"] or b""

# [MS-NLMP] §3.2.5.1.2: structural anonymous detection
is_anon = (
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sometimes, the username is empty but represented with a space (' ') character. This should be treated as anonymous too.

if ntlm_fields.get(k)
]
if parts:
log.info("NTLM: %s", " | ".join(parts))
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be a debug nessage.

Suggested change
log.info("NTLM: %s", " | ".join(parts))
log.debug("NTLM: %s", " | ".join(parts))

targeting known legacy NTLMv1-only environments.


Server Identity
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This section will most likely be moved into a separate file. See #35 for more context. This comment can be marked as resolved

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

In Review Protocol: SMB Errors/Features related to the SMB server Type - Enhancement

Projects

None yet

Development

Successfully merging this pull request may close these issues.

smb.py does not support SMB_COM_TREE_CONNECT

2 participants