SYNC plugin improvements - skipping non-reachable nodes, SYNC_BEHAVIOR

docs/ADVISORY_MULTI_NETWORK.md

@@ -8,7 +8,7 @@ Effective multi-network monitoring starts with understanding how NetAlertX "sees
 * **B. Plan Subnet & Scan Interfaces:** Explicitly configure each accessible segment in `SCAN_SUBNETS` with the corresponding interfaces.
 * **C. Remote & Inaccessible Networks:** For networks unreachable via ARP, use these strategies:
 * **Alternate Plugins:** Supplement discovery with [SNMPDSC](SNMPDSC) or [DHCP lease imports](https://docs.netalertx.com/PLUGINS/?h=DHCPLSS#available-plugins).
-* **Centralized Multi-Tenant Management using Sync Nodes:** Run secondary NetAlertX instances on isolated networks and aggregate data using the **SYNC plugin**.
+* **Centralized Multi-Tenant Management using Sync Nodes:** Run secondary NetAlertX instances on isolated networks and aggregate data using the **SYNC plugin**. Use the [`SYNC_BEHAVIOR`](../front/plugins/sync/README.md#hub-device-write-behavior-sync_behavior) setting on the hub to control whether the hub inherits device config from nodes or manages it independently.
 * **Manual Entry:** For static assets where only ICMP (ping) status is needed.
 
 > [!TIP]

docs/API_SYNC.md

@@ -1,4 +1,4 @@
-# Sync API Endpoint 
+# Sync API Endpoint
 
 ---
 
@@ -35,7 +35,7 @@ curl 'http://<server>:<GRAPHQL_PORT>/sync' \
 
 ---
 
-#### 9.2 POST `/sync` 
+#### 9.2 POST `/sync`
 
 The **POST** endpoint is used by nodes to **send data to the hub**. The hub expects the data as **form-encoded fields** (application/x-www-form-urlencoded or multipart/form-data). The hub then stores the data in the plugin log folder for processing.
 
@@ -91,7 +91,7 @@ curl -X POST 'http://<hub>:<PORT>/sync' \
 
 * The `data` field contains JSON with a **`data` array**, where each element is a **device object** or **plugin data object**.
 * The `plugin` and `node_name` fields allow the hub to **organize and store the file correctly**.
-* The data is only processed if the relevant plugins are enabled and run on the target server. 
+* The data is only processed if the relevant plugins are enabled and run on the target server.
 
 ---
 
@@ -112,14 +112,28 @@ last_result.<plugin>.encoded.<node_name>.<sequence>.log
 
 * Both encoded and decoded files are tracked, and new submissions increment the sequence number.
 * If storing fails, the API returns HTTP 500 with an error message.
-* The data is only processed if the relevant plugins are enabled and run on the target server. 
+* The data is only processed if the relevant plugins are enabled and run on the target server.
 
 ---
 
 #### 9.3 Notes and Best Practices
 
 * **Authorization Required** – Both GET and POST require a valid API token.
 * **Data Integrity** – Ensure that `node_name` and `plugin` are consistent to avoid overwriting files.
-* **Monitoring** – Notifications are generated whenever data is sent or received (`write_notification`), which can be used for alerting or auditing.
+* **Monitoring** – An in-app log entry is written via `write_notification` whenever data is sent or received, which can be used for auditing.
 * **Use Case** – Typically used in multi-node deployments to consolidate device and event data on a central hub.
 
+---
+
+#### 9.4 Hub Device-Write Behavior (`SYNC_BEHAVIOR`)
+
+The `SYNC_BEHAVIOR` setting controls how the hub writes devices received from nodes (Mode 3 — RECEIVE). It only affects the hub.
+
+| Value | Default? | Writes to Devices |
+|---|---|---|
+| `copy-new` | ✅ | New MACs only (INSERT OR IGNORE) |
+| `carbon-copy` | | All MACs every sync (UPSERT) |
+| `hub-defaults` | | None — hub pipeline handles it |
+
+For full details and per-mode behaviour, see [SYNC plugin README — Hub Device-Write Behavior](../front/plugins/sync/README.md#hub-device-write-behavior-sync_behavior).
+

docs/NOTIFICATIONS.md

@@ -63,5 +63,3 @@ You can completely ignore detected devices globally. This could be because your
 
 1. Ignored MACs (`NEWDEV_ignored_MACs`) - List of MACs to ignore.
 2. Ignored IPs (`NEWDEV_ignored_IPs`) - List of IPs to ignore.
-
-

docs/REMOTE_NETWORKS.md

@@ -45,6 +45,9 @@ You can use supplementary plugins that employ alternate methods. Protocols used
 
 If you have servers in different networks, you can set up separate NetAlertX instances on those subnets and synchronize the results into one instance using the [`SYNC` plugin](https://github.com/netalertx/NetAlertX/tree/main/front/plugins/sync).
 
+> [!TIP]
+> The [`SYNC_BEHAVIOR`](../front/plugins/sync/README.md#hub-device-write-behavior-sync_behavior) setting controls how the hub handles newly discovered devices from nodes - whether it inherits node config, overwrites on every sync, or applies its own NEWDEV defaults.
+
 ### Manual Entry
 
 If you don't need to discover new devices and only need to report on their status (`online`, `offline`, `down`), you can manually enter devices and check their status using the [`ICMP` plugin](https://github.com/netalertx/NetAlertX/blob/main/front/plugins/icmp_scan/), which uses the `ping` command internally.

front/plugins/sync/README.md

@@ -53,6 +53,7 @@ The plugin operates in three different modes based on the configuration settings
 - **Schedule** `[n,h]`: `SYNC_RUN_SCHD`
 - **Encryption Key** `[n,h]`: `SYNC_encryption_key`
 - **Nodes to Pull From** `[h]`: `SYNC_nodes` + `GRAPHQL_PORT` of the source nodes
+- **Hub Behavior** `[h]`: `SYNC_BEHAVIOR` — controls how the hub writes devices received from nodes (see [below](#hub-device-write-behavior-sync_behavior))
 
 ### Usage
 
@@ -63,11 +64,59 @@ The plugin operates in three different modes based on the configuration settings
 
 ### Notes
 
-- Existing devices on the hub will not be updated by the data received from this SYNC plugin if their MAC addresses are already present.
+- How existing and new devices are handled on the hub depends on the `SYNC_BEHAVIOR` setting (see below).
 - It is recommended to use Device synchronization primarily. Plugin data synchronization is more suitable for specific use cases.
 
 ![Sync Hub Setup Diagram](/front/plugins/sync/sync_hub.png)
 
+---
+
+### Hub Device-Write Behavior (`SYNC_BEHAVIOR`)
+
+The `SYNC_BEHAVIOR` setting — configured on the **hub only** — controls how the hub writes devices received from nodes.
+
+| Value | Default? | Devices written | Source of truth | Recommended when |
+|---|---|---|---|---|
+| `copy-new` | ✅ | New devices only (INSERT OR IGNORE) | Node (first sync), then Hub | You want the hub to start with the node's existing config and manage devices from there. |
+| `carbon-copy` | | All devices on every sync (UPSERT) | Node | The node owns device config end-to-end. **All** hub fields are overwritten on every sync, including LOCKED. Do not customize devices on the hub. |
+| `hub-defaults` | | None — hub pipeline handles insertion | Hub | Nodes provide presence data only; all device config is set and maintained on the hub. |
+
+#### `copy-new` (default)
+
+New devices are inserted using all available column values from the node's existing record (name, alert settings, vendor, etc.). If the device already exists on the hub, the INSERT is silently skipped.
+
+Subsequent syncs update only empty/unknown fields on the hub (e.g., if the hub's `devName` is `(unknown)` and the node now has a resolved name, it propagates). Fields customized by a user on the hub (fields with source set to `USER` or `LOCKED`) are never overwritten.
+
+```
+First sync:  INSERT with node's full config
+Next syncs:  empty fields updated only (name, vendor) via scan pipeline
+User edits:  protected — never overwritten
+```
+
+#### `carbon-copy`
+
+All received devices are upserted on every sync. The node is treated as fully authoritative: its values overwrite **all** hub fields on every sync cycle, including fields with `USER` or `LOCKED` source.
+
+> ⚠️ Do not customize devices on the hub when using `carbon-copy`. Any hub-side changes will be overwritten on the next sync.
+
+```
+First sync:  UPSERT with node config
+Next syncs:  UPSERT — node values win (all fields, no exceptions)
+User edits:  overwritten on next sync
+```
+
+#### `hub-defaults`
+
+**The hub is the source of truth.** Nodes contribute only presence data (MAC, IP, vendor from scans). All device configuration — name, alerts, notes, group — should be set on the hub. Node-side values for those fields are ignored.
+
+Use this mode when you want the hub to behave as a fully independent instance — it receives presence data from nodes but manages its own device configuration.
+
+```
+First sync:  NEWDEV defaults applied
+Next syncs:  empty fields updated only via scan pipeline
+User edits:  set and maintained on the hub — never overwritten
+```
+
 ### Example use case: Network Setup with Multiple VLANs and VM Scanning
 
 > Thank you to [@richtj999](https://github.com/richtj999) for the use case 🙏
@@ -76,7 +125,7 @@ I have 6 VLANs, all isolated by a firewall, except for one VLAN that has access
 
 Initially, I had one virtual machine (VM) with 6 network cards, one for each VLAN. While this setup worked, it introduced delays due to other concurrent scans. To optimize this, I switched to a multi-VM setup:
 
-- I created 6 VMs, each attached to a single VLAN. 
+- I created 6 VMs, each attached to a single VLAN.
 - One VM acts as the "server," and the other 5 as "clients."
 - The server has access to all VLANs (via firewall rules) and collects data from the client VMs, which each scan their own VLAN.
 
@@ -87,22 +136,22 @@ Initially, I had one virtual machine (VM) with 6 network cards, one for each VLA
 
 #### Example Setup
 
-- **VM1 ("Server")**: Network 1 (can access all networks) - IP: `10.10.10.106`  
+- **VM1 ("Server")**: Network 1 (can access all networks) - IP: `10.10.10.106`
   Receives data from all NetAlertX clients and scans network 1.
 
-- **VM2 ("Client")**: Network 2 (can access only network 2) - IP: `192.168.x.x`  
+- **VM2 ("Client")**: Network 2 (can access only network 2) - IP: `192.168.x.x`
   Scans network 2; VM1 retrieves this data.
 
-- **VM3 ("Client")**: Network 3 (can access only network 3) - IP: `192.168.x.x`  
+- **VM3 ("Client")**: Network 3 (can access only network 3) - IP: `192.168.x.x`
   Scans network 3; VM1 retrieves this data.
 
-- **VM4 ("Client")**: Network 4 (can access only network 4) - IP: `192.168.x.x`  
+- **VM4 ("Client")**: Network 4 (can access only network 4) - IP: `192.168.x.x`
   Scans network 4; VM1 retrieves this data.
 
-- **VM5 ("Client")**: Network 5 (can access only network 5) - IP: `192.168.x.x`  
+- **VM5 ("Client")**: Network 5 (can access only network 5) - IP: `192.168.x.x`
   Scans network 5; VM1 retrieves this data.
 
-- **VM6 ("Client")**: Network 6 (can access only network 6) - IP: `192.168.x.x`  
+- **VM6 ("Client")**: Network 6 (can access only network 6) - IP: `192.168.x.x`
   Scans network 6; VM1 retrieves this data.
 
 ---

front/plugins/sync/config.json

@@ -597,6 +597,38 @@
         }
       ]
     },
+    {
+      "function": "BEHAVIOR",
+      "type": {
+        "dataType": "string",
+        "elements": [
+          {
+            "elementType": "select",
+            "elementOptions": [],
+            "transformers": []
+          }
+        ]
+      },
+      "default_value": "copy-new",
+      "options": [
+        "copy-new",
+        "carbon-copy",
+        "hub-defaults"
+      ],
+      "localized": ["name", "description"],
+      "name": [
+        {
+          "language_code": "en_us",
+          "string": "Hub behavior [h]"
+        }
+      ],
+      "description": [
+        {
+          "language_code": "en_us",
+          "string": "Controls how the hub handles devices received from nodes.<br><br><b>copy-new</b> (default): New devices are inserted using the node's existing configuration (name, alert settings, etc.). Subsequent node-side changes only update empty fields on the hub. Recommended: customize devices on the hub after first discovery.<br><br><b>carbon-copy</b>: All devices are upserted on every sync — the node is fully authoritative and its values overwrite <i>all</i> hub fields on every sync, including fields with <code>USER</code> or <code>LOCKED</code> source. Do not customize devices on the hub.<br><br><b>hub-defaults</b>: The hub skips the direct INSERT and creates new devices through its own pipeline, applying NEWDEV defaults. The hub is the source of truth; nodes contribute presence data only. Recommended: manage all device settings on the hub."
+        }
+      ]
+    },
     {
       "function": "SET_ALWAYS",
       "type": {

front/plugins/sync/sync.py

@@ -5,6 +5,7 @@
 import requests
 import json
 import base64
+import binascii
 
 
 # Define the installation path and extend the system path for plugin imports
@@ -130,12 +131,20 @@ def main():
         for node_url in pull_nodes:
             response_json = get_data(api_token, node_url)
 
+            if not isinstance(response_json, dict):
+                mylog('none', [f'[{pluginName}] Skipping node "{node_url}" due to failed or invalid response'])
+                continue
+
             # Extract node_name and base64 data
             node_name = response_json.get('node_name', 'unknown_node')
             data_base64 = response_json.get('data_base64', '')
 
             # Decode base64 data
-            decoded_data = base64.b64decode(data_base64)
+            try:
+                decoded_data = base64.b64decode(data_base64)
+            except (binascii.Error, ValueError, TypeError) as e:
+                mylog('none', [f'[{pluginName}] Skipping node "{node_name}": base64 decode failed for data_base64="{data_base64}": {e}'])
+                continue
 
             # Create log file name using node name
             log_file_name = f'{file_prefix}.{node_name}.log'
@@ -257,7 +266,7 @@ def main():
             cursor.execute("PRAGMA table_info(Devices)")
             db_columns = {row[1] for row in cursor.fetchall()}
 
-            # Filter out existing devices
+            # Filter new devices (MACs not yet known on hub).
             new_devices = [
                 device for device in device_data
                 if device['devMac'].lower() not in existing_mac_addresses
@@ -266,29 +275,83 @@ def main():
             mylog('verbose', [f'[{pluginName}] All devices: "{len(device_data)}"'])
             mylog('verbose', [f'[{pluginName}] New devices: "{len(new_devices)}"'])
 
-            # Prepare the insert statement
-            if new_devices:
-
-                # Only keep keys that are real columns in the target DB; computed
-                # or unknown fields are silently dropped regardless of source schema.
-                insert_cols = [k for k in new_devices[0].keys() if k in db_columns]
-                columns = ', '.join(insert_cols)
-                placeholders = ', '.join('?' for _ in insert_cols)
-                sql = f'INSERT INTO Devices ({columns}) VALUES ({placeholders})'
-
-                # Extract only the whitelisted column values for each device
-                values = [tuple(device.get(col) for col in insert_cols) for device in new_devices]
+            # Determine which devices to write and how, based on SYNC_BEHAVIOR.
+            #
+            #   copy-new     (default) — INSERT new devices only, using node config.
+            #                            Subsequent node changes only update empty hub fields.
+            #
+            #   carbon-copy            — UPSERT all devices every sync.
+            #                            Node is fully authoritative; raw SQL bypasses
+            #                            can_overwrite_field(), so ALL hub fields are
+            #                            overwritten on every sync, including USER/LOCKED-
+            #                            sourced fields. (update_devices_data_from_scan
+            #                            respects field locks but is not invoked here;
+            #                            see README "carbon-copy" for the contract.)
+            #
+            #   hub-defaults           — Skip direct INSERT entirely.
+            #                            Hub creates new devices via create_new_devices()
+            #                            with its own NEWDEV defaults.
+            #
+            # For copy-new/carbon-copy we insert them here (before the Devices INSERT
+            # would pre-seed the table and block create_new_devices()).
+            # For hub-defaults, create_new_devices() handles it naturally.
+
+            sync_behavior = get_setting_value('SYNC_BEHAVIOR') or 'copy-new'
+            mylog('verbose', [f'[{pluginName}] SYNC_BEHAVIOR: "{sync_behavior}"'])
+
+            if sync_behavior == 'hub-defaults':
+                mylog('verbose', [f'[{pluginName}] hub-defaults: skipping direct Devices write; hub pipeline handles new devices and events'])
+
+            else:
+                # Fire "New Device" events for genuinely new MACs before the Devices
+                # INSERT pre-seeds the table (which would block create_new_devices()).
+                if new_devices:
+                    now = timeNowUTC()
+                    cursor.executemany(
+                        """INSERT OR IGNORE INTO Events
+                           (eveMac, eveIp, eveDateTime, eveEventType, eveAdditionalInfo, evePendingAlertEmail)
+                           VALUES (?, ?, ?, 'New Device', ?, 1)""",
+                        [(d['devMac'], d.get('devLastIP', ''), now, d.get('devVendor', ''))
+                         for d in new_devices]
+                    )
+                    mylog('verbose', [f'[{pluginName}] Queued "New Device" events for {len(new_devices)} device(s)'])
+
+                devices_to_write = new_devices if sync_behavior == 'copy-new' else device_data
+
+                if devices_to_write:
+                    # Only keep keys that are real DB columns; computed or unknown
+                    # fields are silently dropped regardless of the source schema.
+                    insert_cols = [k for k in devices_to_write[0].keys() if k in db_columns]
+                    columns     = ', '.join(insert_cols)
+                    placeholders = ', '.join('?' for _ in insert_cols)
+
+                    if sync_behavior == 'carbon-copy':
+                        # UPSERT: on MAC conflict update all columns except devMac.
+                        # devMac is the PRIMARY KEY so it is excluded from the SET clause.
+                        # NOTE: this raw SQL bypasses can_overwrite_field() — ALL fields
+                        # including USER/LOCKED-sourced ones are overwritten. Node is fully
+                        # authoritative in this mode.
+                        update_cols   = [col for col in insert_cols if col != 'devMac']
+                        update_clause = ', '.join(f'{col}=excluded.{col}' for col in update_cols)
+                        sql = (
+                            f'INSERT INTO Devices ({columns}) VALUES ({placeholders}) '
+                            f'ON CONFLICT(devMac) DO UPDATE SET {update_clause}'
+                        )
+                    else:
+                        # copy-new: skip silently if MAC already exists (race-condition safety).
+                        sql = f'INSERT OR IGNORE INTO Devices ({columns}) VALUES ({placeholders})'
 
-                mylog('verbose', [f'[{pluginName}] Inserting Devices SQL   : "{sql}"'])
-                mylog('verbose', [f'[{pluginName}] Inserting Devices VALUES: "{values}"'])
+                    values = [tuple(device.get(col) for col in insert_cols) for device in devices_to_write]
 
-                # Use executemany for batch insertion
-                cursor.executemany(sql, values)
+                    mylog('verbose', [f'[{pluginName}] Devices SQL   : "{sql}"'])
+                    mylog('verbose', [f'[{pluginName}] Devices VALUES: "{values}"'])
 
-                message = f'[{pluginName}] Inserted "{len(new_devices)}" new devices'
+                    cursor.executemany(sql, values)
 
-                mylog('verbose', [message])
-                write_notification(message, 'info', timeNowUTC())
+                    write_count = len(new_devices) if sync_behavior == 'copy-new' else len(devices_to_write)
+                    message = f'[{pluginName}] {sync_behavior}: wrote "{write_count}" device(s) to Devices'
+                    mylog('verbose', [message])
+                    write_notification(message, 'info', timeNowUTC())
 
         # Commit and close the connection
         conn.commit()