[appsec] Large uploads (>134MB) fail with "body size above the configured limit"

[Tech-no-1]

What happened?

Whenever I take a video which is more than a few seconds long (in this case 180 MB and 33 s), the (auto) upload to Immich fails. I'm using Crowdsec with Appsec enabled and NPMplus, all running in a docker compose.

In the NPMplus logs I get this:

[alert] 8001#8001: *78964 [lua] crowdsec.lua:642: Allow(): [Crowdsec] denied 'my_public_ip' with 'ban' (by appsec) while sending to client, client: 'my_public_ip' , server: immich.domain.com, request: "POST /api/assets HTTP/1.1", host: "immich.domain.com"

Looks like my public ip address got banned, so I've tried "cscli decisions delete -i my_public_ip" and got "INFO 0 decision(s) deleted". No ban, so what's going on? After that I looked at the crowdsec logs:

level=warning msg="Disrupting transaction with body size above the configured limit (Action Reject)" band=inband chain_rule_id=1208191248 name=appsec runner_uuid=dc6feccb-160b-44ca-b0a7-be8fc875df0d tx_id=864ebda2-42e9-4692-baf0-cf89849623b3 type=appsec
level=warning msg="Disrupting transaction with body size above the configured limit (Action Reject)" band=outband chain_rule_id=4217647230 name=appsec runner_uuid=dc6feccb-160b-44ca-b0a7-be8fc875df0d tx_id=864ebda2-42e9-4692-baf0-cf89849623b3 type=appsec

I've tried several things:

1. I've added a custom parser whitelist with an expression pattern that (hopefully) whitelisted the HTTP request/path and I've also whitelisted my dynamic public ip address with the "LookupHost" function (in postoverflows). Both did nothing because they don't affect appsec.
2. I've made a custom appsec config in order to increase the "request_body_in_memory_limit", but I was only able to increase it to 100 MB and it didn't help. I've also tried an appsec config with a hook "on_match" allowing the URI, but apparently appsec disrupts the process before that.
3. I've set "APPSEC_FAILURE_ACTION=passthrough" in the crowdsec.conf of NPMplus, but that didn't work either.
4. I've allowlisted my public ip, which worked and I was able to upload the video, but as my ip is dynamic (like for many others) I always have to manually allowlist the current ip whenever someone wants to upload a larger file. That's not really a great/viable solution.
5. I can disable the Appsec body inspection altogether, which of course works too, but I don't know if that's such a good solution?

Possible solution (I see):
I've came across the Traefik bouncer config (bouncer-middleware.yaml) which has this option "crowdsecAppsecBodyLimit: 10485760". The crowdsec.conf of NPMplus doesn't provide a similar config option to modify the body limit, something like "APPSEC_BODY_LIMIT" for example. Would probably need to be integrated into Crowdsec and Nginx/NPMplus.

#3656 and fosrl/pangolin#436 (comment)

What did you expect to happen?

Big file/video upload get's processed by appsec and ultimately uploaded to Immich successfully.

How can we reproduce it (as minimally and precisely as possible)?

Take a longer video and try uploading it to Immich (probably any large file upload that exceeds the body limit). Crowdsec, Appsec enabled and NPMplus proxy.

Crowdsec version

Details

$ cscli version
version: v1.6.11-d64ee2ae
Codename: alphaga
BuildDate: 2025-07-23_13:32:19
GoVersion: 1.24.5
Platform: docker
libre2: C++
User-Agent: crowdsec/v1.6.11-d64ee2ae-docker
Constraint_parser: >= 1.0, <= 3.0
Constraint_scenario: >= 1.0, <= 3.0
Constraint_api: v1
Constraint_acquis: >= 1.0, < 2.0
Built-in optional components: cscli_setup, datasource_appsec, datasource_cloudwatch, datasource_docker, datasource_file, datasource_http, datasource_journalctl, datasource_k8s-audit, datasource_kafka, datasource_kinesis, datasource_loki, datasource_s3, datasource_syslog, datasource_victorialogs, datasource_wineventlog

Acquisition config

Details

appsec_configs:
  - crowdsecurity/appsec-default
name: appsec
source: appsec
labels:
  type: appsec

[github-actions]

@Tech-no-1: Thanks for opening an issue, it is currently awaiting triage.

In the meantime, you can:

1. Check Crowdsec Documentation to see if your issue can be self resolved.
2. You can also join our Discord.
3. Check Releases to make sure your agent is on the latest version.

Details

I am a bot created to help the crowdsecurity developers manage community feedback and contributions. You can check out my manifest file to understand my behavior and what I can do. If you want to use this for your project, you can check out the BirthdayResearch/oss-governance-bot repository.

[OfficialMuffin]

I have a similar issue here. I get blocked by appsec despite my ip being in a whitelist. After some digging around and asking NPMPlus devs for help. Its pointing to a appsec issue.

ZoeyVid/NPMplus#2050

[LaurenceJJones]

Could you provide which appsec configurations you are using?

as we have configuration options to make the max body limit higher https://docs.crowdsec.net/docs/next/appsec/configuration#performance-and-processing-options

plus we agree there needs to be a max body size on the nginx side as if you upload a 3gb file you dont want to send it all across.

[Tech-no-1]

I'm just using the crowdsecurity/appsec-default configuration.

Yes, I've tried that already. Has been one of the first things I've tried, but I can't increase it to more than 100 MB. If I increase it further, say to 300 MB, the crowdsec container won't start.

name: custom-appsec
inband_options:
  request_body_in_memory_limit: 300000000     # Max body size in memory (bytes)

outofband_options:
  request_body_in_memory_limit: 300000000

level=fatal msg="crowdsec init: while loading acquisition config: while configuring datasource of type appsec from /etc/crowdsec/acquis.d/npmplus.yaml (position 4): unable to initialize runner: unable to initialize inband engine : request body limit should be at least the memory limit"

I don't really understand what that means. Why "at least"? For me "at most" would make more sense and what memory limit? I've tried the container and system memory limits (capacity) and even more, same result. The docker container only starts with <= 100 MB request_body_in_memory_limit.

Well, would be nice to have a max body size above which appsec skips the body inspection (and nginx doesn't need to send it) and/or it should be possible to exceed the 100 MB (maybe I'm doing something wrong, I don't know) , because now appsec disrupts the process with "reject" and the upload fails.

[muffn]

I face the same problem using pangolin with crowdsec and tried the same custom appsec to no avail. As soon as I upload files to immich over pangolin with crowdsec my VPS burns through RAM and disc space and becomes unresponsive till restarted.

If anyone has some workaround I'd be happy to hear, as it stands right now, the only way to "fix" it for me has been disabling crowdsec altogether.

[ManuVice]

same here with Immich and Nextcloud WebDAV uploads. Nextcloud uploads in my browser are working fine.

[devpikachu]

It seems like whilst we can set a higher body size "in memory" limit, it doesn't also set Coraza's body size limit.

crowdsec/pkg/acquisition/modules/appsec/appsec_runner.go

Line 85 in c572b53

r.AppsecInbandEngine, err = coraza.NewWAF(inbandCfg)

https://github.com/corazawaf/coraza/blob/961f8599c0d16254b8ef31cb2067662e36ae1f73/config.go#L34

[jesper1994]

I have the same problem

[Tech-no-1]

It seems like whilst we can set a higher body size "in memory" limit, it doesn't also set Coraza's body size limit.

crowdsec/pkg/acquisition/modules/appsec/appsec_runner.go

Line 85 in c572b53
r.AppsecInbandEngine, err = coraza.NewWAF(inbandCfg)

https://github.com/corazawaf/coraza/blob/961f8599c0d16254b8ef31cb2067662e36ae1f73/config.go#L34

You're right, good finding. I did a little research myself.

So, we can set these two values in https://github.com/crowdsecurity/crowdsec/blob/master/pkg/appsec/appsec.go:

type AppsecSubEngineOpts struct {
	DisableBodyInspection    bool `yaml:"disable_body_inspection"`
	RequestBodyInMemoryLimit *int `yaml:"request_body_in_memory_limit"`
}

But there's no RequestBodyLimit (full, not only in memory).

Coraza uses two body limits: The RequestBodyLimit (for the max size in memory + on disk) and RequestBodyInMemoryLimit (for memory only). Coraza's default RequestBodyLimit = 134217728 Byte = 134.22 MB. Since we can't set our own RequestBodyLimit, we use this default value. This means we can't upload files larger than this value. By the way, Corazas hard limit is 1 GiB.

https://github.com/corazawaf/coraza/blob/38f45710447a1599f9adcfdd2d07529dd8ab080d/internal/corazawaf/waf.go#L300
RequestBodyLimit: 134217728, // Hard limit equal to _1gib

Now for the strange error message: For example, if we increase the RequestBodyInMemoryLimit to 300000000 (bytes), this value is larger than the (full) RequestBodyLimit of 134 MB, which explains why this value cannot be exceeded and why Coraza throws an error.

https://github.com/corazawaf/coraza/blob/main/waf_test.go

		"memory limit bigger than limit": {
			limit:         5,
			inMemoryLimit: 9,
			expectedErr:   errors.New("request body limit should be at least the memory limit"),

In summary, we need a RequestBodyLimit config option to increase Coraza's default RequestBodyLimit. Then we're still limited to 1 GiB, which is better than 134 MB though.

[germanyague]

Is there any solution for this issue? It's been almost two months without any answers and my domains still doesn't let me upload files larger than 134Mb without disabling appsec entirely.
Thanks.

[blotus]

Hello,

Sorry for not answering sooner.

The quick fix if you are using nginx/openresty: you can disable the WAF at runtime by setting a nginx variable (see here).
The config would look like this, assuming the upload is performed on the /upload endpoint:

server {
  location /upload {
   set $crowdsec_disable_appsec 1;
   #proxy_pass and friends go here
 }
}

This will entirely disable the WAF in the scope of the variable, so be careful where you put it (ie, don't set it directly in a server block, as it will disable the WAF for the entire vhost).

Now, for the actual answer:

As a general rule, you do not want to process a giant file with a WAF, it will consume tons of resources for pretty much no gains.
Just as a point of reference, some WAFs are very strict about the maximum body size for analysis (although they are in a different situation as they need to handle an enormous amount of requests, Cloudflare has a 1MB limit and AWS an 8kB limit).

That being said, we do not currently offer a satisfactory way of handling this for every reverse proxy.

What I have in mind at the moment is to implement something a lot more granular than a simple drop/pass configuration wise:

• Drop the request (current behaviour, arguably the safest option, but bad UX in some cases)
• Only analyze headers + query string
• Only analyze the part of the body that is under the configured limit
• Combine all the above with pre_eval hooks to allow changing the behaviour depending on the endpoint, eg:
  • Drop upload bigger than 10MB on /customer/update
  • Only analyze headers on /video/upload
• If you have other ideas, I'm all ears :)

In any case, I'm not willing to override the built-in limit of coraza of 1GiB (which is already very generous).

[germanyague]

Thank you, this throws some clarity for me. In my opinion it would be great to not drop anything. I have a few wordpress sites and I cannot upload a bigger than 134MB video or document, having to disable appsec entirely to do so. Any option that allows me to upload any file without a headache would be great.
Thanks.