Open
Conversation
Users can store environment variables as a single JSON secret in Secrets Manager, loaded in 1 API call per cold start instead of ceil(N/10) SSM batch calls. This reduces cold start latency and provides 10x TPS headroom (10,000 vs SSM's 1,000). Two modes: BREF_SECRETS_MANAGER env var for global JSON import, and bref-secret: prefix for individual secrets with optional JSON key extraction. Smart batching deduplicates API calls across both modes. Also fixes SSM error handling: HTTP 400 catch-all no longer mislabels throttling errors as permission issues.
…omparison Keep the original SSM explanation and examples largely intact. Add Secrets Manager as an additional option rather than replacing SSM content. Replace the generic comparison table with a 'when to use' section explaining the 40 TPS threshold, per-parameter billing, and a cost breakdown at different cold start volumes.
Secrets Manager ARNs include a random 6-character suffix, so IAM resource policies need a -* wildcard or requests are denied. Also document that bref/bref currently only triggers the secrets loader when bref-ssm: env vars are present — users need at least one SSM var until LazySecretsLoader is updated upstream.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
At scale (hundreds of concurrent Lambda cold starts during queue fanout), SSM Parameter Store becomes a bottleneck. The default 40 TPS limit is shared across all
GetParameter*APIs, andGetParametersbatches at most 10 parameters per call. With 50 parameters, each cold start consumes 5 TPS — so only 8 simultaneous cold starts saturate the limit. Beyond that, requests get throttled.Enabling higher throughput removes the TPS cap but introduces per-interaction billing, where each individual parameter returned counts as a separate billable interaction (not each API call). This gets expensive quickly.
AWS Secrets Manager avoids both problems: store all env vars as a single JSON secret, fetch it in 1 API call per cold start, with a 10,000 TPS limit.
What this adds
BREF_SECRETS_MANAGERenv var: points to a secret name (e.g.,api/production). All JSON keys in that secret become env vars with a single API call.bref-secret:prefix: per-variable Secrets Manager references, with optional JSON key extraction (bref-secret:name:key)./tmp/bref-secrets-manager.php) so SSM and Secrets Manager caching don't interfere with each other.Bug fix (SSM)
The existing SSM error handling catches all HTTP 400 errors and labels them as "permissions issues." This is incorrect for throttling errors (
ThrottlingException), which are also HTTP 400. This PR fixes the catch to check the error message before assuming it's a permissions issue.Backward compatibility
Fully backward compatible. The
loadSecretEnvironmentVariables()method signature adds an optional?SecretsManagerClient $smClient = nullsecond parameter. Existing users who only usebref-ssm:are unaffected.Known issue:
bref/brefLazySecretsLoaderThe
LazySecretsLoaderinbref/brefcurrently only checks forbref-ssm:prefixed env vars to decide whether to trigger the secrets loader. If a user only hasBREF_SECRETS_MANAGERorbref-secret:vars (no SSM), the loader won't run.Current workaround: keep at least one
bref-ssm:variable in the configuration alongside Secrets Manager vars.If this PR is approved, updates should be made to
bref/brefto updateLazySecretsLoader::areThereSecretsToLoad()to also detectBREF_SECRETS_MANAGERandbref-secret:prefixes, to avoid this semi-dirty workaround.@mnapoli Do you see any issues with this PR or have any recommendations?