Consolidated hardening, bug fixes, and CI improvements#290
Consolidated hardening, bug fixes, and CI improvements#290somethingwithproof wants to merge 10 commits intoCacti:developfrom
Conversation
TheWitness
left a comment
There was a problem hiding this comment.
Some comments and some additional questions. The json_encode on the title I'd like more background on before merging this one. Also, the change of the config.php in the plugins directory was done for a reason as there is a mandatory config.php for both syslog and flowview now.
b163448 to
da87864
Compare
- Remove redundant locales/index.php and locales/LC_MESSAGES/index.php - Revert database.php optional null param (single connection per script) - Replace json_encode with __esc() for JS string output in setup.php Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
TheWitness
left a comment
There was a problem hiding this comment.
I'm saying request changes here, but just validate. If we are doing a process interlock using the register_process function, we can get rid of all these locks. Also, I had asked Sean that the reorganization of the partition should happen an hour or so before the time of the rotation to avoid overloading the max date partition that I have seen happen in my own instance. I'm not sure if that happened or not. Lost track.
|
Some fixes here @somethingwithproof , but now a merge conflict 🥇 |
- Remove redundant locales/index.php and locales/LC_MESSAGES/index.php - Revert database.php optional null param (single connection per script) - Replace json_encode with __esc() for JS string output in setup.php Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
8b392fe to
9ba8a60
Compare
- LICENSE: remove stale conflict markers
- database.php: restore backward-compat optional param on syslog_db_fetch_insert_id
- js/functions.js: fix missing var declaration, #test->#term, prop('disabled'), remove empty .each()
- CI workflow: authenticate MySQL healthcheck, remove password echo
- populate script: INSERT -> REPLACE INTO for idempotent reruns
- README.md: fix malformed code fence
- locales: add exit after redirect, fix sed -i for macOS compat
- es-ES.po: fix 'Cactus' -> 'Cacti', 'Depuraración' -> 'Depuración'
Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
- Remove $uniqueID filter from syslog_remove query (incorrectly filtered removal rules by random batch marker) - Reorder CI workflow: lint/PHPStan before integration tests - Switch echo to print in syslog_batch_transfer.php Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
- Remove redundant locales/index.php and locales/LC_MESSAGES/index.php - Revert database.php optional null param (single connection per script) - Replace json_encode with __esc() for JS string output in setup.php Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
9ba8a60 to
694fea1
Compare
|
Re: json_encode on title and config.php: json_encode: The setup.php dialog titles were using __esc() inside a JavaScript string literal. __esc() is an HTML-context escaper, so a translated string containing an apostrophe (e.g. French "l'alerte") would break the JS. json_encode() produces correctly quoted/escaped values for JS contexts. Same pattern used in cacti core's auth_changepassword.php after the JS hardening PR. config.php: Understood, will revert that removal. I wasn't aware it was mandatory for syslog and flowview. It stays. |
|
Re: register_process interlock and partition timing: Addressed inline on the lock comment. Short version: keeping the lock for now as a safety net, will remove it once a proper pcntl_signal handler is added to clear registration on forced shutdown. On the partition rotation timing: I'll check whether Sean implemented the early rotation (rotating an hour before the actual rollover). If not, that's a follow-up item. |
|
Merge conflict resolved. Rebased onto develop. |
- Fix broken SQL in syslog_get_alert_sql: remove $uniqueID concatenation after ? placeholder (was producing ?42 instead of using bind param) - Fix XSS in setup.php: use json_encode with JSON_HEX_* flags for JS string context instead of __esc (HTML escaper) - Fix duplicate global $syslogdb_default in syslog_process_alerts - Parameterize retention DELETE queries in syslog_delete_records - Parameterize count query in syslog_remove_items Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
|
@somethingwithproof, is this still in process, or are we going to wait? |
|
All review feedback has been addressed. The lock timeout and signal handler items are tracked in #298 as follow-up. This PR is ready for merge. |
|
You still have it marked Draft. I'll attempt to take out of Draft and review. |
|
@somethingwithproof, please address the failed check. Thanks! |
There was a problem hiding this comment.
Pull request overview
Backport focused on tightening security/escaping in the Syslog plugin UI, hardening certain unsafe behaviors (JS callback execution, CSRF flow), and updating CI/regression tests to catch these regressions earlier.
Changes:
- Adds/standardizes multiple regression tests and a new Pest unit test to enforce security hardening (XSS escaping, CSRF, removal of
eval, parameterized queries). - Hardens UI output and utilities actions (HTML escaping in various outputs; POST+CSRF requirements for purge action).
- Updates CI workflow execution order/steps and adjusts some scripts/docs/locale strings.
Reviewed changes
Copilot reviewed 22 out of 22 changed files in this pull request and generated 15 comments.
Show a summary per file
| File | Description |
|---|---|
tests/regression/issue261_domain_strip_parameterized_test.php |
New regression test asserting parameterized domain-strip update query. |
tests/regression/issue260_remove_eval_callback_test.php |
New regression test asserting eval() removal for autocomplete callbacks. |
tests/regression/issue259_csrf_purge_test.php |
New regression test asserting purge action is POST+CSRF protected and JS strings are safely encoded. |
tests/regression/issue252_xss_output_test.php |
New regression test asserting HTML escaping and DOM-safe option rendering. |
tests/Unit/SyslogImportPayloadTest.php |
New Pest test targeting XML import payload helper. |
tests/Integration/SyslogPartitioningIntegrationTest.php |
Integration tests around partition allowlist/locking behavior. |
tests/Integration/SyslogDomainIntegrationTest.php |
Integration test asserting parameter binding for domain-strip update. |
tests/Helpers/GlobalStubs.php |
Adds DB/config stubs to enable integration testing of functions.php logic. |
syslog_reports.php |
Escapes report list content to prevent XSS. |
syslog_removal.php |
Escapes removal list content to prevent XSS. |
syslog_batch_transfer.php |
Replaces echo with print in CLI output. |
syslog.php |
Escapes multiple output paths (stats table + select options) to prevent XSS. |
setup.php |
Adds POST+CSRF enforcement for purge action and replaces GET link with a dialog-driven POST flow. |
locales/po/es-ES.po |
Fixes Spanish translation typos/wording. |
locales/build_gettext.sh |
Modifies sed -i usage for POT header cleanup. |
js/functions.js |
Removes eval() callback execution; switches to DOM-safe option construction; fixes minor JS issues. |
functions.php |
Large refactor touching partitioning, removal rules, alert command execution, and incoming-record preprocessing semantics. |
README.md |
Fixes markdown formatting. |
LICENSE |
Removes merge-conflict artifacts around end of license text. |
CHANGELOG.md |
Adjusts changelog entries for backported changes. |
.github/workflows/populate_syslog_incoming.sh |
Uses REPLACE INTO for test data seeding. |
.github/workflows/plugin-ci-workflow.yml |
Reorders/changes CI steps; adds plugin lint + PHPStan steps before running regression + poller. |
functions.php
Outdated
| $command = read_config_option('syslog_ticket_command'); | ||
|
|
||
| if ($command != '') { | ||
| $command = trim($command); | ||
| } | ||
|
|
||
| if ($alert['open_ticket'] == 'on' && $command != '') { | ||
| if (is_executable($command)) { | ||
| $command = $command . | ||
| ' --alert-name=' . cacti_escapeshellarg(clean_up_name($alert['name'])) . | ||
| ' --severity=' . cacti_escapeshellarg($alert['severity']) . | ||
| ' --hostlist=' . cacti_escapeshellarg(implode(',',$hostlist)) . |
There was a problem hiding this comment.
This block uses is_executable($command) on the full command string. If the configured command includes arguments or a quoted path, is_executable() will always fail and the ticket/alert command won’t run. The repo’s regression tests expect extracting the executable (with surrounding-quote stripping) and checking is_executable($executable) instead.
| $command = read_config_option('syslog_ticket_command'); | |
| if ($command != '') { | |
| $command = trim($command); | |
| } | |
| if ($alert['open_ticket'] == 'on' && $command != '') { | |
| if (is_executable($command)) { | |
| $command = $command . | |
| ' --alert-name=' . cacti_escapeshellarg(clean_up_name($alert['name'])) . | |
| ' --severity=' . cacti_escapeshellarg($alert['severity']) . | |
| ' --hostlist=' . cacti_escapeshellarg(implode(',',$hostlist)) . | |
| $command = read_config_option('syslog_ticket_command'); | |
| if ($command != '') { | |
| $command = trim($command); | |
| } | |
| if ($alert['open_ticket'] == 'on' && $command != '') { | |
| /* | |
| * Extract the executable portion from the configured command. | |
| * This allows for quoted paths and additional arguments in the | |
| * configuration while still validating the executable itself. | |
| */ | |
| $executable = $command; | |
| $firstChar = substr($executable, 0, 1); | |
| if ($firstChar === '"' || $firstChar === "'") { | |
| $quoteChar = $firstChar; | |
| $closing = strpos($executable, $quoteChar, 1); | |
| if ($closing !== false) { | |
| $executable = substr($executable, 1, $closing - 1); | |
| } else { | |
| // Unbalanced quotes; fall back to trimming quotes/whitespace. | |
| $executable = trim($executable, " \t\n\r\0\x0B\"'"); | |
| } | |
| } else { | |
| $parts = preg_split('/\s+/', $executable); | |
| if (is_array($parts) && isset($parts[0])) { | |
| $executable = $parts[0]; | |
| } | |
| $executable = trim($executable, " \t\n\r\0\x0B\"'"); | |
| } | |
| if ($executable !== '' && is_executable($executable)) { | |
| $command = $command . | |
| ' --alert-name=' . cacti_escapeshellarg(clean_up_name($alert['name'])) . | |
| ' --severity=' . cacti_escapeshellarg($alert['severity']) . | |
| ' --hostlist=' . cacti_escapeshellarg(implode(',', $hostlist)) . |
| $lock_name = hash('sha256', $syslogdb_default . 'syslog_partition_create.' . $table); | ||
|
|
||
| if ((int)$locked !== 1) { | ||
| cacti_log("SYSLOG: Unable to acquire partition create lock for '$table'", false, 'SYSTEM'); | ||
| return false; | ||
| } | ||
| try { | ||
| syslog_db_fetch_cell_prepared('SELECT GET_LOCK(?, 10)', array($lock_name)); | ||
|
|
||
| try { | ||
| /* determine the format of the table name */ | ||
| $time = time(); | ||
| $cformat = 'd' . date('Ymd', $time); | ||
| $lnow = date('Y-m-d', $time+86400); | ||
|
|
||
| $exists = syslog_db_fetch_row_prepared("SELECT * | ||
| FROM `information_schema`.`partitions` | ||
| WHERE table_schema = ? | ||
| AND partition_name = ? | ||
| AND table_name = ? | ||
| ORDER BY partition_ordinal_position", | ||
| array($syslogdb_default, $cformat, $table)); | ||
|
|
||
| if (!cacti_sizeof($exists)) { | ||
| cacti_log("SYSLOG: Creating new partition '$cformat'", false, 'SYSTEM'); | ||
|
|
||
| syslog_debug("Creating new partition '$cformat'"); | ||
|
|
||
| /* | ||
| * MySQL does not support parameter binding for DDL identifiers | ||
| * or partition definitions. $table is safe because it passed | ||
| * syslog_partition_table_allowed() (two-value allowlist plus | ||
| * regex guard). $cformat and $lnow derive from date() and | ||
| * contain only digits, hyphens, and the letter 'd'. | ||
| */ | ||
| syslog_db_execute("ALTER TABLE `" . $syslogdb_default . "`.`$table` REORGANIZE PARTITION dMaxValue INTO ( | ||
| cacti_log("SYSLOG: Creating new partition '$cformat' for table '$table'", false, 'SYSTEM'); | ||
|
|
||
| syslog_debug("Creating new partition '$cformat' for table '$table'"); | ||
|
|
||
| /* MySQL does not support parameter binding for DDL statements */ | ||
| syslog_db_execute_prepared("ALTER TABLE `" . $syslogdb_default . "`.`$table` REORGANIZE PARTITION dMaxValue INTO ( |
There was a problem hiding this comment.
syslog_partition_create() acquires a named lock via GET_LOCK(...) but ignores the return value. If the lock can’t be acquired (contention/error), the code will still run ALTER TABLE, which defeats the concurrency control. Please check the GET_LOCK result (1/0/NULL) and return early (and log) when the lock is not acquired.
| exit(1); | ||
| } | ||
|
|
||
| echo "issue252_xss_output_test passed\n"; |
There was a problem hiding this comment.
These regression tests print their pass message using echo, but the PR description mentions standardizing CLI scripts to use print (php-cs-fixer rule). Please switch this to print for consistency with the rest of the regression suite.
| echo "issue252_xss_output_test passed\n"; | |
| print "issue252_xss_output_test passed\n"; |
| - name: Create PHPStan config | ||
| run: | | ||
| cd ${{ github.workspace }}/cacti/plugins/syslog | ||
| cat > phpstan.neon << 'EOF' | ||
| parameters: | ||
| level: 5 | ||
| paths: | ||
| - . | ||
| excludePaths: | ||
| - vendor/ | ||
| - locales/ | ||
| ignoreErrors: | ||
| - '#has invalid return type the\.#' | ||
| bootstrapFiles: | ||
| - ../../include/global.php | ||
| EOF | ||
|
|
||
| - name: Install PHPStan | ||
| run: | | ||
| cd ${{ github.workspace }}/cacti/plugins/syslog | ||
| composer require --dev phpstan/phpstan --with-all-dependencies || composer global require phpstan/phpstan | ||
|
|
||
| - name: Run PHPStan Analysis | ||
| run: | | ||
| cd ${{ github.workspace }}/cacti/plugins/syslog | ||
| if [ -f vendor/bin/phpstan ]; then | ||
| vendor/bin/phpstan analyse --no-progress --error-format=github || true | ||
| else | ||
| phpstan analyse --no-progress --error-format=github || true | ||
| fi | ||
| continue-on-error: true |
There was a problem hiding this comment.
The PR description says CI should run lint/PHPStan/CS-Fixer before integration tests. In this workflow, PHPStan is marked continue-on-error: true (so it won’t fail the job) and there is no CS-Fixer step. Either update the workflow to match the stated enforcement/order, or adjust the PR description to reflect the actual behavior.
There was a problem hiding this comment.
Removed redundant || true; continue-on-error handles non-blocking.
functions.php
Outdated
| WHERE `status` = 1 | ||
| AND `seq` <= ?', | ||
| array($max_seq)); | ||
| syslog_db_execute_prepared('DELETE FROM `' . $syslogdb_default . '`.`syslog_incoming` WHERE status=' . $uniqueID); |
There was a problem hiding this comment.
This DELETE concatenates $uniqueID directly into the SQL while calling syslog_db_execute_prepared(). For consistency (and to avoid edge cases if the value isn’t an int), use a placeholder and pass $uniqueID via the params array.
| syslog_db_execute_prepared('DELETE FROM `' . $syslogdb_default . '`.`syslog_incoming` WHERE status=' . $uniqueID); | |
| syslog_db_execute_prepared( | |
| 'DELETE FROM `' . $syslogdb_default . '`.`syslog_incoming` WHERE status = ?', | |
| [$uniqueID] | |
| ); |
functions.php
Outdated
| $command = read_config_option('syslog_ticket_command'); | ||
|
|
||
| if ($command != '') { | ||
| $command = trim($command); | ||
| } | ||
|
|
||
| if ($alert['open_ticket'] == 'on' && $command != '') { | ||
| if (is_executable($command)) { | ||
| $command = $command . | ||
| ' --alert-name=' . cacti_escapeshellarg(clean_up_name($alert['name'])) . | ||
| ' --severity=' . cacti_escapeshellarg($alert['severity']) . | ||
| ' --hostlist=' . cacti_escapeshellarg(implode(',',$hostlist)) . | ||
| ' --message=' . cacti_escapeshellarg($alert['message']); | ||
|
|
||
| $output = array(); | ||
| $return = 0; | ||
|
|
||
| exec($command, $output, $return); | ||
|
|
||
| if ($return != 0) { | ||
| cacti_log(sprintf('ERROR: Ticket Command Failed. Alert:%s, Exit:%s, Output:%s', $alert['name'], $return, implode(', ', $output)), false, 'SYSLOG'); | ||
| } | ||
| } | ||
| } | ||
|
|
||
| if (trim($alert['command']) != '' && !$found) { | ||
| syslog_execute_alert_command($alert, $results, $hostname); | ||
| $command = alert_replace_variables($alert, $results, $hostname); | ||
|
|
||
| $logMessage = "SYSLOG NOTICE: Executing '$command'"; | ||
|
|
||
| $cparts = explode(' ', $command); | ||
|
|
||
| if (is_executable($cparts[0])) { | ||
| exec($command, $output, $returnCode); | ||
| } else { | ||
| exec('/bin/sh ' . $command, $output, $returnCode); | ||
| } |
There was a problem hiding this comment.
The shared helpers syslog_execute_ticket_command() / syslog_execute_alert_command() appear to have been removed, and the logic is now duplicated inline. This reintroduces behaviors the repo’s regression tests explicitly forbid (e.g., /bin/sh fallback execution and naive tokenization), and will fail tests/regression/issue278_command_execution_refactor_test.php. Please restore the shared helpers and call them from here, keeping the hardened executable parsing/tokenization and without any shell-fallback path.
functions.php
Outdated
| $command = alert_replace_variables($alert, $results, $hostname); | ||
|
|
||
| $logMessage = "SYSLOG NOTICE: Executing '$command'"; | ||
|
|
||
| $cparts = explode(' ', $command); | ||
|
|
||
| if (is_executable($cparts[0])) { | ||
| exec($command, $output, $returnCode); | ||
| } else { | ||
| exec('/bin/sh ' . $command, $output, $returnCode); | ||
| } | ||
|
|
||
| // Append the return code to the log message without the dot | ||
| $logMessage .= " Command return code: $returnCode"; | ||
|
|
||
| // Log the combined message | ||
| cacti_log($logMessage, true, 'SYSTEM'); |
There was a problem hiding this comment.
$output and $returnCode are used in the command execution path without being initialized in this scope, which can emit notices and also risks leaking prior values from earlier exec calls. Initialize/reset these variables before calling exec(), and prefer the shared helper that encapsulates this safely.
| exit(1); | ||
| } | ||
|
|
||
| echo "issue259_csrf_purge_test passed\n"; |
There was a problem hiding this comment.
These regression tests print their pass message using echo, but the PR description mentions standardizing CLI scripts to use print (php-cs-fixer rule). Please switch this to print for consistency with the rest of the regression suite.
| echo "issue259_csrf_purge_test passed\n"; | |
| print "issue259_csrf_purge_test passed\n"; |
| function syslog_preprocess_incoming_records() { | ||
| global $syslogdb_default; | ||
|
|
||
| $max_seq = syslog_db_fetch_cell('SELECT MAX(seq) FROM `' . $syslogdb_default . '`.`syslog_incoming` WHERE status = 0'); | ||
| while (1) { | ||
| $uniqueID = rand(1, 127); | ||
|
|
||
| if ($max_seq > 0) { | ||
| /* flag all records with the status = 1 prior to moving */ | ||
| syslog_db_execute_prepared('UPDATE `' . $syslogdb_default . '`.`syslog_incoming` | ||
| SET `status` = 1 | ||
| WHERE `status` = 0 | ||
| AND `seq` <= ?', | ||
| array($max_seq)); | ||
| $count = syslog_db_fetch_cell_prepared('SELECT COUNT(*) | ||
| FROM `' . $syslogdb_default . '`.`syslog_incoming` | ||
| WHERE `status` = ?', | ||
| array($uniqueID)); | ||
|
|
||
| syslog_debug('Max Sequence ID = ' . $max_seq); | ||
| syslog_debug('-------------------------------------------------------------------------------------'); | ||
| if ($count == 0) { | ||
| break; | ||
| } | ||
| } | ||
|
|
||
| $syslog_incoming = syslog_db_fetch_cell_prepared('SELECT COUNT(seq) | ||
| FROM `' . $syslogdb_default . '`.`syslog_incoming` | ||
| WHERE `status` = 1 | ||
| AND `seq` <= ?', | ||
| array($max_seq)); | ||
| /* flag all records with the uniqueID prior to moving */ | ||
| syslog_db_execute_prepared('UPDATE `' . $syslogdb_default . '`.`syslog_incoming` | ||
| SET `status` = ? | ||
| WHERE `status` = 0', | ||
| array($uniqueID)); | ||
|
|
||
| syslog_debug(sprintf('Found %5s - New Message(s) to process', $syslog_incoming)); | ||
| syslog_debug('Unique ID = ' . $uniqueID); | ||
| syslog_debug('-------------------------------------------------------------------------------------'); | ||
|
|
||
| /* strip domains if we have requested to do so */ | ||
| syslog_strip_incoming_domains($max_seq); | ||
| $syslog_incoming = syslog_db_fetch_cell_prepared('SELECT COUNT(seq) | ||
| FROM `' . $syslogdb_default . '`.`syslog_incoming` | ||
| WHERE `status` = ?', | ||
| array($uniqueID)); | ||
|
|
||
| api_plugin_hook('plugin_syslog_before_processing'); | ||
| syslog_debug(sprintf('Found %5s - New Message(s) to process', $syslog_incoming)); | ||
|
|
||
| return array('max_seq' => $max_seq, 'incoming' => $syslog_incoming); | ||
| } | ||
| /* strip domains if we have requested to do so */ | ||
| syslog_strip_incoming_domains($uniqueID); | ||
|
|
||
| return array('max_seq' => 0, 'incoming' => 0); | ||
| api_plugin_hook('plugin_syslog_before_processing'); | ||
|
|
||
| return array('uniqueID' => $uniqueID, 'incoming' => $syslog_incoming); |
There was a problem hiding this comment.
syslog_preprocess_incoming_records() now returns ['uniqueID' => ...] and no longer returns max_seq. Existing callers (e.g. syslog_process.php) still expect max_seq, which will cause undefined-index errors and break the poller. Either keep returning max_seq for backward compatibility or update all call sites in this backport to use the new key consistently.
| /* flag all records with the uniqueID prior to moving */ | ||
| syslog_db_execute_prepared('UPDATE `' . $syslogdb_default . '`.`syslog_incoming` | ||
| SET `status` = ? | ||
| WHERE `status` = 0', | ||
| array($uniqueID)); |
There was a problem hiding this comment.
The new uniqueID-based status marking (UPDATE ... SET status = ? WHERE status = 0) can strand records if the poller crashes after marking but before moving/deleting: those rows will never be picked up again (since subsequent runs only process status=0) and may be silently purged by the 1-hour cleanup. The previous max_seq approach was crash-recoverable. Consider either reverting to the max_seq+seq<=max_seq scheme, or adding recovery logic to reprocess non-zero statuses that were not completed (e.g., by age / last-run marker).
- Extract executable from ticket command before is_executable() check; the full command string with arguments always fails validation - Check GET_LOCK() return value before ALTER TABLE; log and return early when the partition lock cannot be acquired - Use prepared statement placeholder for $uniqueID in DELETE query - Remove redundant || true from PHPStan commands; continue-on-error already handles non-blocking analysis - Standardize regression test output from echo to print Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
- Apply db_qstr() to all raw string concatenation in syslog_manage_items() for facility, host, messageb, messagec, and messagee removal types, matching the pattern already used in syslog_remove_items() - Add array_map intval cast to $hostarray before IN() clause to prevent injection via unvalidated host_id elements - Add XML validation for import payloads - Add intval cast to exporter session serialization Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
Backport of fixes and improvements from develop to main.
Bug fixes
Hardening
Refactoring
CI and testing
Locales