v3.1.0: Security hardening, robustness fixes, BSON chunking, and OPA compatibility#741
Merged
v3.1.0: Security hardening, robustness fixes, BSON chunking, and OPA compatibility#741
Conversation
- Fix 38 critical/high severity issues across 50+ source files - Replace eval/exec/yaml.load with safe alternatives (ast.literal_eval, importlib, yaml.safe_load) - Eliminate command injection via os.system and shell=True Popen - Add HTTP request timeouts, fix bare excepts, replace deprecated datetime.utcnow - Add thread-safe MongoDB connections with query sanitization - Fix MongoDB BSON 16MB limit crash by splitting/merging large snapshot documents - Replace hardcoded /tmp paths with tempfile, remove credentials from config/logs - Add 988+ unit tests preserving contracts for JSON schemas, snapshots, and validation - Bump version to 3.1.0 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
farchide
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
eval()/exec()with safe alternatives, eliminated command injection viaos.system()andshell=True, replaced unsafeyaml.load(), removed hardcoded credentials from config/logs, added Kubernetes SSL verification, replacedrandomwithsecretsfor ID generationexcept:clauses with proper exception handling and logging, replaced deprecateddatetime.utcnow()withdatetime.now(timezone.utc), fixed mutable default arguments, fixed file handle leaks, added thread-safe MongoDB connections with query sanitization--v0-compatibleflag to OPA eval subprocess calls so Rego v0 syntax compliance rules work correctly with OPA 1.8+update_crawler_run_statusfrom MongoDB calls when running withdatabase=NONEFiles Changed
64 files changed across source code (52 files), tests (11 new + 1 updated), and audit documentation (1 new).
See
CRITICAL_ISSUES_AUDIT.mdfor the full issue inventory and remediation details.Test plan
python setup.py sdist bdist_wheel)--v0-compatibleflag correctly evaluates all Rego v0 rules🤖 Generated with Claude Code