chore: update Windows build to use PCRE2 as default#3491
chore: update Windows build to use PCRE2 as default#3491fzipi wants to merge 27 commits intoowasp-modsecurity:v2/masterfrom
Conversation
Signed-off-by: Felipe Zipitria <felipe.zipitria@owasp.org>
There was a problem hiding this comment.
Pull request overview
Updates the Windows/IIS build tooling and documentation to prefer PCRE2 (and newer dependency versions), aligning with PCRE deprecation and addressing stability concerns like the reported pcre.dll stack overflow.
Changes:
- Bump Windows dependency versions (CMake, PCRE2, zlib, libxml2, Lua, cURL, Apache, ssdeep) and update download URLs.
- Switch Windows build/release scripts and NMake linkage from
pcre.dll/pcre.libtopcre2-8.dll/pcre2-8.lib, enabling PCRE JIT flags in NMake builds. - Refresh
README_WINDOWS.mdbuild instructions to reference PCRE2 and newer dependency versions; CI attempts to enable PCRE JIT.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| iis/download_files.bat | Updates dependency versions and download URLs (including PCRE2). |
| iis/dependencies/build_pcre.bat | Switches dependency build from PCRE to PCRE2 and copies PCRE2 artifacts. |
| iis/build_release.bat | Packages pcre2-8.dll into release artifacts instead of pcre.dll. |
| iis/build_dependencies.bat | Updates dependency version variables used by the batch build pipeline. |
| iis/Makefile.win | Links against pcre2-8.lib and defines WITH_PCRE_JIT. |
| README_WINDOWS.md | Updates Windows build documentation to use PCRE2 and newer dependency versions. |
| .github/workflows/test-ci-windows.yml | Adds -DWITH_PCRE_JIT=ON to the IIS CMake configure step. |
Comments suppressed due to low confidence (2)
README_WINDOWS.md:57
- The updated doc switches the Apache install dir to
C:\Apache2466, but the example still references Apache/httpd 2.4.27 (httpd-2.4.27andHTTPD_BUILD=C:\work\httpd-2.4.27). This is inconsistent and will mislead users following the updated instructions; update these example paths/versions to match the new Apache 2.4.66 guidance.
The directory where you build software from source ( ``C:\work`` in this exmaple)
must contain the Apache source you used to build the Apache web serverand the mod_security source
Apache source is in C:\work\httpd-2.4.27 in this example.
Apache has been installed to C:\Apache2466 in this example.
Mod_security source is in C:\work\mod_security in this example.
## Download and untar the prerequisite library sources:
Download pcre2-10.47.zip from https://github.com/PCRE2Project/pcre2/releases/download/pcre2-10.47/
unzip it into C:\work\ creating C:\work\pcre2-10.47
Download libxml2-2.15.1.tar.gz from https://download.gnome.org/sources/libxml2/2.15/
untar it into C:\work\ creating C:\work\libxml2-2.15.1
Download lua-5.4.8.tar.gz from http://www.lua.org/ftp/
untar it into C:\work\ creating C:\work\lua-5.4.8
Download curl-8.18.0.zip from https://curl.se/download/
unzip it into C:\work\ creating C:\work\curl-8.18.0
Download zlib-1.3.1.tar.gz from https://github.com/madler/zlib/releases/download/v1.3.1/
untar it into C:\work\ creating C:\work\zlib-1.3.1
## Setup your build environment:
1. The ``PATH`` environment variable must include the Visual Studio variables as set by ``vsvars32.bat``
2. The ``PATH`` environment variable must also include the CMAKE ``bin\`` directory
3. Set an environment variable to the Apache source code directory:
SET HTTPD_BUILD=C:\work\httpd-2.4.27
README_WINDOWS.md:204
- The mlogc configuration examples still reference
Apache2427paths (e.g.,CollectorRootandSecAuditLog |C:/Apache2427/...) even though the PR updates the guide toApache2466. These should be updated for consistency, otherwise users will end up with incorrect paths in their configs.
CollectorRoot "C:/Apache2427/logs"
ConsoleURI "https://localhost:8888/rpc/auditLogReceiver"
SensorUsername "test"
SensorPassword "testtest"
LogStorageDir "data"
TransactionLog "mlogc-transaction.log"
QueuePath "mlogc-queue.log"
ErrorLog "mlogc-error.log"
LockFile "mlogc.lck"
KeepEntries 0
ErrorLogLevel 2
MaxConnections 10
MaxWorkerRequests 1000
TransactionDelay 50
StartupDelay 5000
CheckpointInterval 15
ServerErrorTimeout 60
Change the SecAuditLog directive in ``conf\httpd.conf`` to pipe the log data to mlogc instead of writing them to a file:
SecAuditLog |C:/Apache2427/bin/mlogc.bat
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Signed-off-by: Felipe Zipitria <felipe.zipitria@owasp.org>
Signed-off-by: Felipe Zipitria <felipe.zipitria@owasp.org>
Signed-off-by: Felipe Zipitria <felipe.zipitria@owasp.org>
Signed-off-by: Felipe Zipitria <felipe.zipitria@owasp.org>
Signed-off-by: Felipe Zipitria <felipe.zipitria@owasp.org>
Signed-off-by: Felipe Zipitria <felipe.zipitria@owasp.org>
Signed-off-by: Felipe Zipitria <felipe.zipitria@owasp.org>
Signed-off-by: Felipe Zipitria <felipe.zipitria@owasp.org>
Signed-off-by: Felipe Zipitria <felipe.zipitria@owasp.org>
Signed-off-by: Felipe Zipitria <felipe.zipitria@owasp.org>
Signed-off-by: Felipe Zipitria <felipe.zipitria@owasp.org>
Signed-off-by: Felipe Zipitria <felipe.zipitria@owasp.org>
f4fcb69 to
6334d28
Compare
Signed-off-by: Felipe Zipitria <felipe.zipitria@owasp.org>
Signed-off-by: Felipe Zipitria <felipe.zipitria@owasp.org>
Signed-off-by: Felipe Zipitria <felipe.zipitria@owasp.org>
Signed-off-by: Felipe Zipitria <felipe.zipitria@owasp.org>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 16 out of 20 changed files in this pull request and generated 10 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
|
@copilot open a new pull request to apply changes based on the open comments in this thread |
Signed-off-by: Felipe Zipitria <felipe.zipitria@owasp.org>
Signed-off-by: Felipe Zipitria <felipe.zipitria@owasp.org>
|
Documentation on Windows should be updated also on the wiki, if this one gets merged (follow up). |
Signed-off-by: Felipe Zipitria <felipe.zipitria@owasp.org>
Signed-off-by: Felipe Zipitria <felipe.zipitria@owasp.org>
Signed-off-by: Felipe Zipitria <felipe.zipitria@owasp.org>
Signed-off-by: Felipe Zipitria <felipe.zipitria@owasp.org>
Signed-off-by: Felipe Zipitria <felipe.zipitria@owasp.org>
Signed-off-by: Felipe Zipitria <felipe.zipitria@owasp.org>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 17 out of 21 changed files in this pull request and generated 6 comments.
Comments suppressed due to low confidence (1)
iis/wix/list_dependencies.bat:83
- Several writes to the log file and the final
explorerinvocation use%log_file%without quotes (e.g.,echo. >> %log_file%,>> %log_file%,explorer %log_file%). Since%TEMP%commonly contains spaces (user profile paths), these commands can break or write to the wrong file. Quote%log_file%consistently everywhere it’s used for redirection or as a command argument.
echo. >> %log_file%
REM Updated paths to include Visual Studio 2019 and 2022
set POSSIBLE_PATHS_X86="C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\*\bin\Hostx86\x86\dumpbin.exe" "C:\Program Files\Microsoft Visual Studio\2022\Professional\VC\Tools\MSVC\*\bin\Hostx86\x86\dumpbin.exe" "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\*\bin\Hostx86\x86\dumpbin.exe" "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\*\bin\Hostx86\x86\dumpbin.exe" "C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\VC\Tools\MSVC\*\bin\Hostx86\x86\dumpbin.exe" "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\*\bin\Hostx86\x86\dumpbin.exe" "C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\bin\dumpbin.exe" "C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\dumpbin.exe"
set POSSIBLE_PATHS_X64="C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\*\bin\Hostx64\x64\dumpbin.exe" "C:\Program Files\Microsoft Visual Studio\2022\Professional\VC\Tools\MSVC\*\bin\Hostx64\x64\dumpbin.exe" "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\*\bin\Hostx64\x64\dumpbin.exe" "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\*\bin\Hostx64\x64\dumpbin.exe" "C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\VC\Tools\MSVC\*\bin\Hostx64\x64\dumpbin.exe" "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\*\bin\Hostx64\x64\dumpbin.exe" "C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\x86_amd64\dumpbin.exe"
for %%i in (%POSSIBLE_PATHS_X86%) do (
echo Checking for dumpbin x86... %%i
echo Checking for dumpbin x86... %%i >> %log_file%
if exist %%i (
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Fix "installion" typo in Makefile.win usage comments - Fix "mlocg.exe" typo in README_WINDOWS.md - Fix SecAuditLogRelevantStatus regex to use negative lookahead - Check PCRE2 archive existence in build_pcre.bat instead of Apache - Wire up WITH_PCRE_JIT option in iis/CMakeLists.txt - Quote %log_file% paths in list_dependencies.bat for spaces - Use explicit PowerShell path via [SystemFolder] in installer.wxs - Change CreateModSecurityDirs to Return="check" to surface errors - Restore Action="createAndRemoveOnUninstall" on RegistryKey - Only print verbose MSI log on installation failure in CI
|
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 18 out of 22 changed files in this pull request and generated 4 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| C:\work\lua-5.4.8\src\lua54.dll C:\Apache2466\bin\ | ||
| C:\work\libxml2-2.15.1\win32\bin.msvc\libxml2.dll C:\Apache2466\bin\ | ||
| C:\work\curl-8.18.0\builds\libcurl-vc-x64-release-dll-zlib-dll-ipv6-sspi-schannel-obj-lib\libcurl.dll C:\Apache2466\bin\ | ||
| C:\work\mod_security\apache2\mlogc-src\mlogc.exe C:\Apache2466\bin\ |
There was a problem hiding this comment.
The documentation references C:\work\mod_security\apache2\mlogc-src\mlogc.exe, but this repository’s mlogc project lives under the top-level mlogc/ directory (and there is no apache2\mlogc-src folder). Please update this path to the actual built mlogc.exe location so Windows users can follow the instructions successfully.
| C:\work\mod_security\apache2\mlogc-src\mlogc.exe C:\Apache2466\bin\ | |
| C:\work\mod_security\mlogc\mlogc.exe C:\Apache2466\bin\ |
| /* These functions are safe in ModSecurity's usage and required for portability */ | ||
| #pragma warning(disable: 4996) |
There was a problem hiding this comment.
#pragma warning(disable: 4996) in a public header affects every translation unit that includes it, which can unintentionally suppress CRT warnings outside ModSecurity code. Prefer limiting the scope with #pragma warning(push)/pop, or moving this suppression into the Windows build flags (e.g., /wd4996) for the specific targets that need it.
| /* These functions are safe in ModSecurity's usage and required for portability */ | |
| #pragma warning(disable: 4996) | |
| /* These functions are safe in ModSecurity's usage and required for portability */ | |
| #pragma warning(push) | |
| #pragma warning(disable: 4996) | |
| #pragma warning(pop) |
| The latest versions of ModSecurity dependencies known to work well are the following: | ||
|
|
||
| cmake-3.8.2-win32-x86 | ||
| pcre-8.40 (patch required and included on file "patch-pcre-8.40.vbs") | ||
| zlib-1.2.11 | ||
| libxml2-2.9.4 | ||
| lua-5.3.4 | ||
| curl-7.54.1 | ||
| httpd-2.4.27 (bin-VC11) | ||
| cmake-3.8.2-win32-x86 (or later) | ||
| pcre2-10.x (PCRE2 is now the default, PCRE 8.x is deprecated) | ||
| - Note: PCRE2 provides better performance and is actively maintained | ||
| zlib-1.2.11 (or later) | ||
| libxml2-2.9.4 (or later) | ||
| lua-5.3.4 (or later) | ||
| curl-7.54.1 (or later) | ||
| httpd-2.4.27 (bin-VC15 or later for VS2019/2022) |
There was a problem hiding this comment.
This “latest versions … known to work well” list still calls out very old versions (e.g., zlib 1.2.11, libxml2 2.9.4, lua 5.3.4, curl 7.54.1, httpd 2.4.27) even though the build scripts in this PR now default to much newer versions. Updating this list to match the versions pinned in iis/build_dependencies.bat/iis/download_files.bat will avoid confusing builders about which versions are actually expected/tested.
|
|
||
| :: BITSAdmin refuses to download YAJL from GitHub URL | ||
| :: @set YAJL_URL=https://github.com/lloyd/yajl/archive/%YAJL:~-9% | ||
| @set YAJL_URL=http://http.debian.net/debian/pool/main/y/yajl/yajl_2.1.0.orig.tar.gz |
There was a problem hiding this comment.
The YAJL_URL download still uses plain HTTP, which allows a network attacker to tamper with the yajl_2.1.0.orig.tar.gz archive in transit and inject malicious code into the libraries you later build and package. Because this script is intended to fetch build-time dependencies for ModSecurity, a compromised YAJL tarball could result in a backdoored module being compiled and shipped. Switch this URL to an authenticated HTTPS source (or add checksum/signature verification) so the YAJL source integrity is cryptographically verified before use.



what
ssdeep 2.14.1)
why
references