SMB/NTLM capture server rewrite#38
Conversation
Rewrite ntlm.py with full NTLMSSP handling for all hash types: NetNTLMv1, NetNTLMv1-ESS, NetNTLMv2, and NetLMv2. NTLM handshake: - NTLM_build_challenge_message: construct CHALLENGE from NEGOTIATE flags with configurable AV_PAIRs, ESS/NTLMv2 control, VERSION - NTLM_handle_negotiate_message: parse and log NEGOTIATE fields - NTLM_handle_authenticate_message: extract identity, classify hash type, format hashcat lines, detect anonymous/ESS/NTLMv2 - NTLM_handle_legacy_raw_auth: SMB1 basic-security (non-NTLMSSP) hash extraction with OS/domain metadata in host_info - NTLM_to_hashcat: hashcat -m 5500 (v1/ESS) and -m 5600 (v2/LMv2) - ESS detection: LM[8:24]==Z(16) is authoritative over flags - MsvAvTimestamp intentionally omitted to maximize LMv2 capture - Per-connection protocol logger context on CHALLENGE log line - Type-safe default_val assignments in apply_config() Config changes: - All NTLM config centralized to [NTLM] section (section_local=False) - No per-protocol NTLM overrides - 10 ATTR_NTLM_* attributes with defaults matching Dementor.toml Supporting files: - spnego.py: NegTokenResp construction updates - session.py: type hints for 7 new NTLM attributes - toml.py: config resolution updates - log/__init__.py: protocol logger support
Rewrite SMB handler with full dialect support from NT LM 0.12 through SMB 3.1.1. SMB2/3 (Path A): - Multi-dialect negotiate with negotiate contexts (preauth integrity, encryption capabilities, signing capabilities for SMB 3.1.1) - SESSION_SETUP with SPNEGO/NTLMSSP auth flow - TREE_CONNECT with path capture from wire bytes (PathOffset/ PathLength, bypasses impacket AlignPad off-by-one bug) - CREATE, CLOSE, READ, WRITE, FLUSH, LOCK, SET_INFO, QUERY_INFO, QUERY_DIRECTORY, IOCTL (VALIDATE_NEGOTIATE_INFO + DFS) handlers - Server-allocated random SessionID - CapturesPerConnection for multi-credential SSPI retry on SMB1+SMB2 SMB1 Extended (Path B): - SESSION_SETUP WordCount=12 with NTLMSSP/SPNEGO - Shared handle_ntlmssp code path with SMB2 SMB1 Basic (Path C): - SESSION_SETUP WordCount=13 with raw LM/NT response extraction - Cleartext password capture from non-encrypted sessions - Manual string parsing with alignment pad calculation (bypasses impacket SMBTreeConnectAndX_Data missing pad field) SMB1 shared: - NT_CREATE_ANDX, TRANS2 (15 info levels from pcap), READ, CLOSE - TREE_CONNECT with path capture - SMB1-to-SMB2 protocol upgrade Post-auth: - ConnectionResetError/BrokenPipeError handling at dispatch level - OS/LanMan metadata passed through extras for host_info display - Per-connection client info summary (os, lanman, path, dialect, files)
Update HTTP, SMTP, IMAP, POP3, LDAP, MSSQL, and RPC handlers: - Remove NTLM_split_fqdn usage; use config ntlm_nb_computer/nb_domain - Remove per-protocol ATTR_NTLM_* field overrides from _fields_ - Add NTLM_handle_negotiate_message call with negotiate_fields passing - Use keyword args for NTLM_handle_authenticate_message - Pass per-connection logger via log= to NTLM_build_challenge_message - HTTP: set protocol_version to HTTP/1.1, add Content-Length:0 on 401 CHALLENGE response, store negotiate_fields on self - SMTP: return AuthResult(success=False, handled=True) instead of None from downgrade path; set logger host from session peer; fix typo
Rewrite [SMB] and [NTLM] config sections to match current code: [NTLM] section: - Challenge commented out (code default is random, not fixed) - Every option marked optional with description and default - Accepted challenge formats documented (hex:, ascii:, auto-detect) - AV_PAIR fields documented with spec IDs (0x0001-0x0005) - Required vs optional AV_PAIRs noted [SMB] section: - Identity options moved below post-auth behaviour - ErrorCode accepts integer or string name (via getattr) - All defaults match SMBServerConfig._fields_ Other protocol sections: - Remove per-protocol NTLM overrides (Challenge, ESS, NTLMv2) from SMTP, HTTP, LDAP, RPC, MSSQL, POP3, IMAP - Replace with one-line note pointing to [NTLM] - Update resolution order header to reflect global-only NTLM
smb.rst: - Rewrite intro to list all implemented handlers (create, read, write, close, query info, IOCTL, etc.) and captured data - Reorder auth paths: SMB2/3 (Path A), SMB1 Extended (B), Basic (C) - Add tested client list: XP RTM through Win 11 25H2, Server 2003 through Server 2025, NT 4.0, smbclient, curl smb:// - Replace client info table with note linking to ntlm.rst - Add Logging section with info-level summary format and debug per-command examples - Fix CapturesPerConnection: CPC=0 returns STATUS_SUCCESS for tree connect, not the configured ErrorCode - Remove IS_GUEST section (internal detail) ntlm.rst: - Clarify ESS as server-controlled: server echoes flag back, client falls back to plain NTLMv1 if server strips it - Fix LmCompatibilityLevel table: all levels 0-2 produce ESS when server echoes it, not just level 1 - Fix interaction table: Level 2 shows NTLMv1-ESS with defaults - Remove Extended Protection and Channel Binding Tokens section - Simplify LMv2 suppression to two reasons (MsvAvTimestamp, Win 7+) - Simplify observed behavior table to two rows - Add Linked-to cross-refs for 7 identity options - Remove phantom LMv2 Z(16) condition - Remove per-protocol NTLM override references throughout
Remove stale per-protocol NTLM attribute sections (ESS, Challenge, resolution precedence lists) from dcerpc.rst, http.rst, imap.rst, pop3.rst, and mssql.rst. Replace with note pointing to [NTLM]. Update examples: - multicast.rst: fix attribute name to DisableExtendedSessionSecurity, use NTLM. prefix instead of RPC. prefix in CLI example - smtp_downgrade.rst: fix attribute name to DisableExtendedSessionSecurity
SMB Features table: - Tree connect: Responder SMB1 only, Dementor SMB1+SMB2 - Cleartext capture: both supported (was marked X for Dementor) - Multi-credential: Responder SMB1 only, Dementor SMB1+SMB2 - Add: filename capture, client info extraction, SMB 3.1.1 negotiate contexts, configurable dialect range, per-listener config, SMB2 SessionID allocation - Surface Responder bugs: hardcoded SMB 2.1 dialect, echoed SessionID NTLM Specifics table: - Add: LMv2 companion capture, client OS/version extraction - Fix threshold label: > 24 bytes (not >= 48 B) - Surface Responder bugs: AV_PAIR 0x0003/0x0004 values swapped, NTLMv2 threshold inconsistency (>60 in SMB, >25 in HTTP), HTTP NTLM flags missing SIGN/SEAL/KEY_EXCH/ALWAYS_SIGN, SMB NTLM flags missing SEAL Protocol table: - SMB 1.0 Raw: Dementor now supported (was X) - Dementor SMB overall: check-check (was partial) - NetNTLMv1-ESS Responder: partial + mislabeled (was broken) - Fix stray double </tr> tag Remove footnote [1] about NTLMv1-SSP labeling and all references. Fix heading typo: Spcifics -> Specifics.
514 tests covering every public function in ntlm.py and smb.py. test_ntlm.py (~275 tests): - Pure functions: NTLM_timestamp, _config_version_to_bytes, NTLM_decode_string, NTLM_encode_string, _classify_hash_type, _compute_dummy_lm_responses, NTLM_to_hashcat - Mock-dependent: NTLM_build_challenge_message, negotiate/authenticate handlers, anonymous detection, NTLMv2 blob parsing - Real pcap vectors from 14 Windows versions (XP RTM through Server 2022), TCP-flow-matched challenges - Hash classification, hashcat format, ESS detection, full auth pipeline, anonymous probe detection, negotiate flag coverage test_smb.py (~101 tests): - Pure functions: _split_smb_strings, parse_dialect, get_command_name, set_smb_error_code, _smb3_neg_context_pad, _build_trans2_file_info - Response builders: SMB2 negotiate, query_info, create, tree_connect - Handler tests: SMB2 negotiate, IOCTL, all simple handlers, SMB1 handlers, dispatch routing - Mock handler via object.__new__ with manually set state
| if self.client_files: | ||
| parts.append(f"files:{','.join(sorted(self.client_files))}") | ||
| if parts: | ||
| self.logger.info("SMB: %s", " | ".join(parts)) |
There was a problem hiding this comment.
This should be a debug message.
| self.logger.info("SMB: %s", " | ".join(parts)) | |
| self.logger.debug("SMB: %s", " | ".join(parts)) |
| is_client=True, | ||
| ) | ||
| if path: | ||
| self.client_info["smb_path"] = path |
There was a problem hiding this comment.
When a path gets captured, it should be logged immediately (and should get highlighted)
| self.client_info["smb_path"] = path | |
| self.client_info["smb_path"] = path | |
| if not path.endswith("IPC$"): | |
| self.logger.success(f"Captured CIFS path from {self.client_host}") | |
| self.logger.highlight(f"Path: [b]{escape(path)}[/]") |
| else: | ||
| # Non-IPC$ disk share — capture the path for intelligence | ||
| if path: | ||
| self.client_info["smb_path"] = path |
There was a problem hiding this comment.
Same as below; the captured path should get highlighted
| self.client_info["smb_path"] = path | |
| self.client_info["smb_path"] = path | |
| self.logger.success(f"Captured CIFS path from {self.client_host}") | |
| self.logger.highlight(f"Path: [b]{escape(path)}[/]") |
| "SMB_COM_NT_CREATE_ANDX Name=%s", name or "(empty)", is_client=True | ||
| ) | ||
| if name: | ||
| self.client_files.add(name) |
There was a problem hiding this comment.
The target file name should get logged to the TUI.
| self.client_files.add(name) | |
| self.client_files.add(name) | |
| is_ipc = ( | |
| "IPC$ " | |
| if self.client_info.get("smb_path", "").endswith("IPC$") | |
| else "" | |
| ) | |
| self.logger.success( | |
| f"Captured requested CIFS file name from {self.client_host}" | |
| ) | |
| self.logger.highlight(f"FileName: {is_ipc}[b]{escape(name)}[/]") |
There was a problem hiding this comment.
There must be at least a ..versionchanged/..versionremoved to denote the attribute removal
There was a problem hiding this comment.
Same here: There must be at least a ..versionchanged/..versionremoved to denote the attribute removal
There was a problem hiding this comment.
Same here: There must be at least a ..versionchanged/..versionremoved to denote the attribute removal
| lm_response: bytes = token["lanman"] or b"" | ||
|
|
||
| # [MS-NLMP] §3.2.5.1.2: structural anonymous detection | ||
| is_anon = ( |
There was a problem hiding this comment.
Sometimes, the username is empty but represented with a space (' ') character. This should be treated as anonymous too.
| if ntlm_fields.get(k) | ||
| ] | ||
| if parts: | ||
| log.info("NTLM: %s", " | ".join(parts)) |
There was a problem hiding this comment.
This should be a debug nessage.
| log.info("NTLM: %s", " | ".join(parts)) | |
| log.debug("NTLM: %s", " | ".join(parts)) |
| targeting known legacy NTLMv1-only environments. | ||
|
|
||
|
|
||
| Server Identity |
There was a problem hiding this comment.
This section will most likely be moved into a separate file. See #35 for more context. This comment can be marked as resolved
This also closes #24
What?
This rewrites
smb.pyandntlm.pyto capture NTLM hashes from every Windows version (XP RTM through Win 11 25H2, Server 2003 through Server 2025) across all SMB dialects (NT LM 0.12 through SMB 3.1.1). It also centralizes NTLM configuration to a single[NTLM]section, updates all protocol handlers to use the new API, adds 514 unit tests, and brings all documentation in line with the code.26 files changed, +8435/-2165 lines across 8 commits.
Why?
The previous SMB server only negotiated SMB 2.1 (hardcoded), had no SMB 3.1.1 negotiate context support, no tree connect or path capture, and no SMB1 basic-security handling. NTLM hash extraction had no ESS detection, no LMv2 companion capture, no dummy LM filtering, and no anonymous detection. NTLM config was scattered across every protocol's
_fields_list with per-protocol overrides that made behavior hard to predict.How?
ntlm.py
Full NTLMSSP implementation:
NTLM_build_challenge_messageconstructs the CHALLENGE from client NEGOTIATE flags with configurable AV_PAIRs, ESS/NTLMv2 control, and VERSION.NTLM_handle_authenticate_messageclassifies hashes by byte structure (ESS detection fromLM[8:24]==Z(16)is authoritative over flags), formats hashcat lines (-m 5500 and -m 5600), filters dummy LM responses, and detects anonymous sessions.NTLM_handle_legacy_raw_authhandles SMB1 basic-security (non-NTLMSSP) separately. MsvAvTimestamp is intentionally omitted to maximize LMv2 companion capture. All 10 NTLM config attributes usesection_local=Falseso they're only read from[NTLM].smb.py
Three authentication paths: SMB2/3 (dialect 2.002-3.1.1 with negotiate contexts for preauth, encryption, signing), SMB1 Extended (SPNEGO/NTLMSSP, WordCount=12), and SMB1 Basic (raw challenge/response, WordCount=13, cleartext). Post-auth stub handlers for create, read, write, close, query info, query directory, IOCTL (validate negotiate + DFS), flush, lock, set info. Tree connect with UNC path capture for both SMB1 and SMB2. Filename capture from CREATE. CapturesPerConnection for multi-credential SSPI retry. Per-connection client info summary logging.
Protocol handlers
Removed
NTLM_split_fqdnand per-protocolATTR_NTLM_*from http, smtp, imap, pop3, ldap, mssql, and rpc. All now read NTLM config fromSessionConfig(populated by[NTLM]apply_config). AddedNTLM_handle_negotiate_messagecalls, keyword args for authenticate, and per-connection logger on challenge messages. Fixed HTTP to use HTTP/1.1 for connection-based NTLM auth. Fixed SMTP to return explicitAuthResultfrom downgrade path.Config
Rewrote
[SMB]and[NTLM]sections in Dementor.toml. Every option documented with default and description. Challenge default changed from fixed to random (commented out). Per-protocol NTLM overrides removed from all protocol sections.Documentation
Rewrote smb.rst and ntlm.rst to match the code. Removed per-protocol NTLM config sections from dcerpc, http, imap, pop3, mssql docs. Updated compat.rst with current capabilities and Responder bugs (hardcoded dialect, threshold inconsistency, AV_PAIR swap, missing flags). Fixed stale ESS attribute names in example docs.
Testing?
514 unit tests covering every public function in ntlm.py and smb.py. Tests include real pcap vectors from 14 Windows versions (TCP-flow-matched challenges from a filtered capture), hash classification, hashcat format validation, ESS detection, anonymous probe detection, and negotiate flag coverage. SMB tests use
object.__new__mock handlers with manually set state.End-to-end tested with:
curl: 38 tests across HTTP, WinRM, SMB, SMTP, IMAP, POP3, FTP, LDAPsmbclient: SMB 2.002, SMB 2.1, SMB 3.1.1, SMB1 NT1, port 139Commits