v2: Allow partial processing of XML, JSON, and multipart request body #3483

hnakamur · 2025-12-31T13:56:37Z

what

This pull request prevents a parse error that occurs when SecRequestBodyLimitAction is set to ProcessPartial and the length of an XML or JSON request body exceeds the configured limit.

why

When SecRequestBodyLimitAction is ProcessPartial, the expected behavior is to process the partial XML, JSON, or multipart body up to the defined limit.
The request should then be rejected if any invalid values are found within that processed part, or passed through if the processed part is valid.
The previous implementation failed to handle the partial body correctly, leading to a parse error when the limit was exceeded.

references

The same modification for v3 is at: #3476

…ssing" Due to the previous change, the "no final boundary missing" error never occurs when SecRequestBodyLimitAction is ProcessPartial.

Adjust the expected error message to match the message changes introdueced in: owasp-modsecurity@dfbde55

But reject incomplete epilogue when body fits in limit.

airween · 2025-12-31T15:25:17Z

Hi @hnakamur,

thanks for this PR too.

When SecRequestBodyLimitAction is ProcessPartial, the expected behavior is to process the partial XML, JSON, or multipart body up to the defined limit.

I think SecRequestBodyLimitAction regards not only for XML, JSON or multipart bodies - but also to www-url-encoded body too. Consider this setup:

SecRequestBodyLimitAction ProcessPartial
SecRequestBodyNofilesLimit 10

and the payload:

aaa=123&bbb=123

then the aaa variable must have seen in ARGS with value 123.

@dune73 what do you think?

The request should then be rejected if any invalid values are found within that processed part, or passed through if the processed part is valid.

How can we be sure that the payload is valid, if the error is in the chunked part? Eg. consider the config above and this JSON:

{"a":1,"b":2,[1]]}

(the second ] makes the JSON non well-formed, but the key a must have seen in ARGS).

You helped me in #3456, which isn't merged yet (I'm waiting for a review and an approve). But could you add a new test case to the v2's source tree?

hnakamur · 2026-01-01T11:17:32Z

Hello @airween. Thanks for your comment.

I added tests for url-encoded, JSON, and XML request bodies. It appears that the URL-encoded parser already works as expected with ProcessPartial.

By the way, I realized that it would be better to add more tests for multipart requests. I think it will take me a day or two to do so.

This check is added to satisfy SonarQube Cloud Quality Gate on CI. owasp-modsecurity#3483 (comment)

To satisfy Quality Gate in CI. owasp-modsecurity#3483 (comment)

hnakamur · 2026-01-06T02:42:41Z

It appears that the URL-encoded parser already works as expected with ProcessPartial.

I was mistaken about that. I modified the URL-encoded parser to include names and values up to the delimiter before the limit when SecRequestBodyLimitAction is set to ProcessPartial and the request body exceeds SecRequestBodyLimit.

I also modified the multipart parser to include parts whose boundary appears before the limit under the same conditions.

In addition, I added tests to cover these changes.

Copilot

Pull request overview

This pull request fixes a bug where SecRequestBodyLimitAction ProcessPartial would cause parse errors when the request body exceeded the configured limit for XML, JSON, and multipart content types. The implementation enables parsers to gracefully handle truncated bodies by allowing partial/ill-formed data when the limit is exceeded.

Changes:

Added modsecurity_request_body_enable_partial_processing() function to enable partial processing for different body processors
Modified XML, JSON, multipart, and URL-encoded parsers to handle partial data when the limit is exceeded
Added comprehensive test coverage for partial processing across all supported content types
Changed request body limit comparison from >= to > to correctly allow bodies exactly at the limit

Reviewed changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated 13 comments.

Show a summary per file

File	Description
apache2/apache2_io.c	Modified to truncate buffers at limit and enable partial processing mode
apache2/msc_reqbody.c	Added function to enable partial processing per processor type
apache2/modsecurity.h	Added field and function declaration for partial processing
apache2/msc_json.c	Fixed indentation and enabled yajl partial value parsing
apache2/msc_json.h	Added function declaration for JSON partial processing
apache2/msc_xml.c	Modified to skip well-formed checks when partial processing enabled
apache2/msc_xml.h	Added flag to allow ill-formed XML in partial mode
apache2/msc_multipart.c	Enhanced boundary handling for partial multipart bodies
apache2/msc_multipart.h	Added flag for partial multipart processing
apache2/msc_parsers.c	Modified URL-encoded parser to skip final parameter in partial mode
tests/regression/rule/15-json.t	Added 6 test cases for JSON partial processing
tests/regression/rule/10-xml.t	Added 3 test cases for XML partial processing
tests/regression/config/10-request-directives.t	Added extensive multipart, JSON, XML, and URL-encoded partial processing tests

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-01-22T14:56:03Z

apache2/apache2_io.c

+    if (msr->txcfg->if_limit_action == REQUEST_BODY_LIMIT_ACTION_REJECT) {
+        if (rcbe == -5) {
+            return HTTP_REQUEST_ENTITY_TOO_LARGE;
+        }
+        if (rcbe < 0) {
+            return HTTP_INTERNAL_SERVER_ERROR;
+        }
    }


The error handling logic has been modified to only return error codes when if_limit_action is REQUEST_BODY_LIMIT_ACTION_REJECT. However, this means that when if_limit_action is REQUEST_BODY_LIMIT_ACTION_PARTIAL, errors from modsecurity_request_body_end (rcbe) are silently ignored and APR_SUCCESS is returned. This could mask legitimate parsing errors that occur during partial processing. Consider returning errors for non -5 error codes even in ProcessPartial mode.

Suggested change

if (msr->txcfg->if_limit_action == REQUEST_BODY_LIMIT_ACTION_REJECT) {

if (rcbe == -5) {

return HTTP_REQUEST_ENTITY_TOO_LARGE;

}

if (rcbe < 0) {

return HTTP_INTERNAL_SERVER_ERROR;

}

}

/* Handle request body processing result:

* - rcbe == -5: body too large -> 413 only in REJECT mode, ignore otherwise.

* - other rcbe < 0: treat as internal error in all modes.

*/

if (rcbe == -5) {

if (msr->txcfg->if_limit_action == REQUEST_BODY_LIMIT_ACTION_REJECT) {

return HTTP_REQUEST_ENTITY_TOO_LARGE;

}

} else if (rcbe < 0) {

return HTTP_INTERNAL_SERVER_ERROR;

}

This change causes a parser error returns a response with status code 500, which is not wanted since this case is not an internal server error.
In case of a parse error, REQBODY_ERROR is set to 1, which can be used by a rule to return a 400 Bad Request response, for example:

SecRule REQBODY_ERROR "!@eq 0" "id:'200001', phase:2,t:none,log,deny,status:400,msg:'Failed to parse request body.',logdata:'%{reqbody_error_msg}',severity:2"

Side note: However, I found that no rule in OWASP CRS 4.22.0 checks REQBODY_ERROR.

This is wanted behavior for both of REQUEST_BODY_LIMIT_ACTION_REJECT and REQUEST_BODY_LIMIT_ACTION_PARTIAL.

Therefore I think the right modification here is:

if (rcbe == -5 && msr->txcfg->if_limit_action == REQUEST_BODY_LIMIT_ACTION_REJECT) { return HTTP_REQUEST_ENTITY_TOO_LARGE; }

This change causes a parser error returns a response with status code 500, which is not wanted since this case is not an internal server error.

okay, then we can make this as resolved. Thanks.

Side note: However, I found that no rule in OWASP CRS 4.22.0 checks REQBODY_ERROR.

This is wanted behavior for both of REQUEST_BODY_LIMIT_ACTION_REJECT and REQUEST_BODY_LIMIT_ACTION_PARTIAL.

Good advice - would you mind to open a PR or just an issue under coreruleset?

Therefore I think the right modification here is:

if (rcbe == -5 && msr->txcfg->if_limit_action == REQUEST_BODY_LIMIT_ACTION_REJECT) { return HTTP_REQUEST_ENTITY_TOO_LARGE; }

Do you want to add/change this one too? I'm waiting for your response, until that I don't resolve this comment.

tests/regression/rule/15-json.t

apache2/msc_reqbody.c

apache2/msc_parsers.c

tests/regression/rule/15-json.t

tests/regression/rule/10-xml.t

apache2/msc_parsers.c

tests/regression/config/10-request-directives.t

airween · 2026-01-22T17:43:45Z

tests/regression/config/10-request-directives.t

+		SecDebugLogLevel 9
+		SecRequestBodyAccess On
+		SecRequestBodyLimitAction ProcessPartial
+		SecRequestBodyLimit 59


To be clear, please add here and all other tests where you check the limit(s) the SecRequestBodyNoFilesLimit.

I think that would be useful to avoid the unwanted side effects (eg. not the relevant settings applied and a negative test is successed).

Before adding tests for SecRequestBodyNoFilesLimit, let me understand it for ProcessPartial.

Though the document for SecRequestBodyNoFilesLimit does not mention for ProcessPartial case, I believe we should not reject a request whose length exceeds the SecRequestBodyNoFilesLimit for ProcessPartial case. Is this correct?

And I found the current implemention is handling the request whose length exceeds SecRequestBodyNoFilesLimit inconsistently between modsecurity_request_body_store and modsecurity_request_body_end.

I fixed it at hnakamur@96a8326.
Could you take a look?

Should I include this commit to this pull request or should I create another pull request for SecRequestBodyNoFilesLimit?

I'll add tests for SecRequestBodyNoFilesLimit after the above fix is settled.

Before adding tests for SecRequestBodyNoFilesLimit, let me understand it for ProcessPartial.

Right, thanks.

Though the document for SecRequestBodyNoFilesLimit does not mention for ProcessPartial case, I believe we should not reject a request whose length exceeds the SecRequestBodyNoFilesLimit for ProcessPartial case. Is this correct?

Well, that's a good question.

@dune73 what do you think?

And I found the current implemention is handling the request whose length exceeds SecRequestBodyNoFilesLimit inconsistently between modsecurity_request_body_store and modsecurity_request_body_end.

I fixed it at hnakamur@96a8326. Could you take a look?

Let me take a look at this soon.

Should I include this commit to this pull request or should I create another pull request for SecRequestBodyNoFilesLimit?

It would be great if you merge the current v2/master branch into your working branch. Then after each commit the whole regression test will run, then we can see the tests are passed.

I'll add tests for SecRequestBodyNoFilesLimit after the above fix is settled.

Okay. I'll let you know if I'm done. Thanks.

airween · 2026-01-22T17:44:19Z

tests/regression/config/10-request-directives.t

+		SecDebugLogLevel 9
+		SecRequestBodyAccess On
+		SecRequestBodyLimitAction ProcessPartial
+		SecRequestBodyLimit 58


Please add the SecRequestBodyNoFilesLimit too with an exact value.

airween · 2026-01-22T17:48:16Z

tests/regression/config/10-request-directives.t

+		SecDebugLogLevel 9
+		SecRequestBodyAccess On
+		SecRequestBodyLimitAction ProcessPartial
+		SecRequestBodyLimit 8


Please add the SecRequestBodyNoFilesLimit with an exact value.

In these cases (where you check non-multipart requests) could you add more tests with higher SecRequestBodyLimit values?

airween · 2026-01-22T17:49:50Z

Hi @hnakamur,

thank you again for your work, I think this is a very important fix.

Please take a look at GH Copilot review, there are a logic change and a typo in the code.

I also added a few notices to tests.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

sonarqubecloud · 2026-01-23T05:04:08Z

Quality Gate passed

Issues
0 New issues
6 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

hnakamur added 7 commits December 31, 2025 06:22

Allow partial processing of multipart, JSON, and XML request body

a682f13

Adjust test since ProcessPartial no more causes "no final boundary mi…

55270e5

…ssing" Due to the previous change, the "no final boundary missing" error never occurs when SecRequestBodyLimitAction is ProcessPartial.

Fix json test match_log

586e1ef

Fix expected error message in regression test

e9dca3c

Adjust the expected error message to match the message changes introdueced in: owasp-modsecurity@dfbde55

Accept partial epilogue in body larger than limit for ProcessPartial

0e4728f

But reject incomplete epilogue when body fits in limit.

Fix indent in apache2/msc_multipart.c

24508d6

Fix indent in apache2/msc_json.c

bea943b

hnakamur mentioned this pull request Dec 31, 2025

Allow partial processing of XML, JSON, and multipart request body #3476

Open

hnakamur added 2 commits January 1, 2026 15:41

Add tests for url-encoded, JSON, and XML with ProcessPartial

9b2a5fe

Refine tests for multipart with ProcessPartial

0bfb828

hnakamur added a commit to hnakamur/ModSecurity that referenced this pull request Jan 1, 2026

Add NULL check for multipart boundary in multipart_complete

a96b557

This check is added to satisfy SonarQube Cloud Quality Gate on CI. owasp-modsecurity#3483 (comment)

hnakamur added a commit to hnakamur/ModSecurity that referenced this pull request Jan 1, 2026

Add assertion for multipart boundary in multipart_complete

f685253

To satisfy Quality Gate in CI. owasp-modsecurity#3483 (comment)

hnakamur added a commit to hnakamur/ModSecurity that referenced this pull request Jan 1, 2026

Add assertion for multipart boundary in multipart_complete

5b38984

To satisfy Quality Gate in CI. owasp-modsecurity#3483 (comment)

hnakamur force-pushed the v2/stop_processing_after_reqbody_limit_for_process_partial branch 2 times, most recently from 5b38984 to 0bfb828 Compare January 1, 2026 14:39

hnakamur added 8 commits January 3, 2026 08:24

Modify url-encoded reqbody tests for ProcesPartial

43f95fd

Support partial processing of url-encoded reqbody

3d73c02

Add tests for MULTIPART_PART_HEADERS with ProcessPartial

d914f23

Reject invalid final boundary for multipart with ProcessPartial

55a1828

Fix indent in apache2/msc_multipart.c

0b599ad

Modify tests for multipart with ProcessPartial

cb95a24

Modify multipart_complete for incomplete final boundary

61d4e45

Process an incomplete boundary as a non-final boundary

be5feee

airween requested a review from Copilot January 22, 2026 14:44

Copilot started reviewing on behalf of airween January 22, 2026 14:44 View session

Copilot AI reviewed Jan 22, 2026

View reviewed changes

airween requested changes Jan 22, 2026

View reviewed changes

hnakamur and others added 4 commits January 23, 2026 11:31

Update tests/regression/rule/15-json.t

50de8bb

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update tests/regression/rule/15-json.t

a64b544

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Fix spelling of reqbody_partial_processing_enabled

a83c26c

Fix indentations in test files

da8b174

hnakamur force-pushed the v2/stop_processing_after_reqbody_limit_for_process_partial branch from 1ea8ee9 to da8b174 Compare January 23, 2026 02:45

hnakamur added 2 commits January 23, 2026 13:24

Add rules to check multipart error in tests

f679e11

Fix indentations in tests/regression/rule/10-xml.t

9277589

v2: Allow partial processing of XML, JSON, and multipart request body #3483

Are you sure you want to change the base?

v2: Allow partial processing of XML, JSON, and multipart request body #3483

Conversation

hnakamur commented Dec 31, 2025

what

why

references

Uh oh!

airween commented Dec 31, 2025

Uh oh!

hnakamur commented Jan 1, 2026

Uh oh!

hnakamur commented Jan 6, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Jan 22, 2026

Choose a reason for hiding this comment

Uh oh!

hnakamur Jan 23, 2026

Choose a reason for hiding this comment

Uh oh!

airween Jan 23, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

airween Jan 22, 2026

Choose a reason for hiding this comment

Uh oh!

hnakamur Jan 23, 2026

Choose a reason for hiding this comment

Uh oh!

airween Jan 23, 2026

Choose a reason for hiding this comment

Uh oh!

airween Jan 22, 2026

Choose a reason for hiding this comment

Uh oh!

airween Jan 22, 2026

Choose a reason for hiding this comment

Uh oh!

airween commented Jan 22, 2026

Uh oh!

sonarqubecloud bot commented Jan 23, 2026

Quality Gate passed

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants