fix: collect metrics to help analyze performance problems by rix0rrr · Pull Request #1124 · aws/aws-cdk-cli

rix0rrr · 2026-02-09T09:56:01Z

This PR changes telemetry to help diagnose and address performance problems.

The effective changes are as follows:

We are seeing slow synth times.
- Log the total number of assemblies and stacks to correlate and segment synth times and stack counts (is the relationship linear/nonlinear, are slow synth times due to huge applications, ...)
- Log the (guessed) implementation language, to see if synth times behave differently between languages, if startup times make a lot of difference, etc.
We are concerned about the time needed for agents to produce a successful CDK application.
- Try to determine whether we are currently being run by an agent so we can segment on that.
- Log the counts of warnings and errors, to see if deployment failure is or can be preceded/prevented by better error reporting at synth time.
The library version we were collecting was a rough guess, do a better one once we have accurate information.
Count the number of resources in a stack to correlate and segment deployment times by resource count.

For consistency, the same counters changes are being made to toolkit-lib, even though those counters aren't being consumed by anything.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

In multiple places the `SYNTH` span doesn't measure the time to synthesize the assembly, but rather the time to select stacks from an already-synthesized assembly. Presumably this was done in order to avoid duplicating this line: ```ts const selectStacks = options.stacks ?? ALL_STACKS; ``` Instead, introduce a helper function that gets called from all places, and centralize the `selectStacks` logic so we don't fear duplication anymore.

…e-more-perf

…-perf

Changes are proposed in aws/aws-cdk-cli#1124. We could have sent this data as simple "counters" and not have had to change this page, but instead we are more obvious about it and send them as independent fields that we document. cr: https://code.amazon.com/reviews/CR-252851509

…-perf

…li into huijbers/measure-more-perf

…-perf

packages/aws-cdk/lib/cli/util/guess-agent.ts

kaizencc · 2026-03-17T13:26:42Z

packages/aws-cdk/lib/cli/telemetry/session.ts

+
+function mutable<A extends object>(x: A): { -readonly [k in keyof A]: A[k] } {
+  return x;
+}


are we gasp playing god here and turning immutable objects mutable?

Yes. There are good reasons for it.

https://rust-unofficial.github.io/patterns/idioms/temporary-mutability.html

https://clojure.org/reference/transients

I've only ever seen this as re-defining the entire variable

packages/aws-cdk/lib/cli/telemetry/session.ts

kaizencc · 2026-03-17T13:33:37Z

packages/@aws-cdk/toolkit-lib/lib/api/io/private/span.ts

    await this.notify(this.definition.end.msg(
      endMsg, {
        duration: duration.asMs,
+        ...(Object.keys(this.counters).length > 0 ? { counters: this.counters } : {}),


nit, doesn't matter: is there a reason you didn't want to send counters: {}?

Yeah, I like sending as little as possible.

kaizencc · 2026-03-17T13:38:26Z

packages/@aws-cdk/toolkit-lib/lib/api/io/private/span.ts

+ */
+type SpanEndArguments<T> = keyof T extends keyof SpanEnd
+  ? (Pick<Partial<SpanEnd>, keyof T & keyof SpanEnd> | void)
+  : Optional<T, keyof T & keyof SpanEnd>;


i read your comments but this was still greek to me. This is what Kiro said, if true, it was a lot simpler to understand:

In short: if the span type already covers everything SpanEnd needs, ending is no-arg. If the span type has extra fields beyond SpanEnd, you're forced to supply them when calling end().

Kiro is not entirely correct. You can still, but don't have to override the implicit values with optionals.

packages/aws-cdk/test/cli/telemetry/session.test.ts

kaizencc · 2026-03-17T13:45:28Z

packages/aws-cdk/lib/cli/util/guess-agent.ts

+ * It's hard for us to say `false` for sure, so we only respond
+ * with `yes` or `don't know`.


It's hard for us to say 'yes' for sure too, since anyone can add an env variable like this.

True. That's why the name is guess

rix0rrr added 4 commits February 5, 2026 14:58

fix: not enough information to root cause performance problems

2e96368

Merge branch 'huijbers/not-measuring-synth-time' into huijbers/measur…

0ffad66

…e-more-perf

WIP

b1fc30f

rix0rrr had a problem deploying to automation February 9, 2026 09:56 — with GitHub Actions Failure

rix0rrr had a problem deploying to integ-approval February 9, 2026 09:56 — with GitHub Actions Failure

rix0rrr temporarily deployed to automation February 9, 2026 09:56 — with GitHub Actions Inactive

rix0rrr force-pushed the huijbers/measure-more-perf branch from d141182 to 9fa8629 Compare February 9, 2026 09:57

rix0rrr had a problem deploying to integ-approval February 9, 2026 09:57 — with GitHub Actions Failure

github-actions bot added the p2 label Feb 9, 2026

aws-cdk-automation requested a review from a team February 9, 2026 09:57

rix0rrr changed the title ~~fix: collect metrics to help analyze performance impact~~ fix: collect metrics to help analyze performance problems Feb 9, 2026

rix0rrr force-pushed the huijbers/measure-more-perf branch from 9fa8629 to 671ef9f Compare February 9, 2026 09:58

rix0rrr had a problem deploying to integ-approval February 9, 2026 09:58 — with GitHub Actions Failure

Guess agent

063862f

rix0rrr force-pushed the huijbers/measure-more-perf branch from 671ef9f to 063862f Compare February 9, 2026 09:59

rix0rrr had a problem deploying to integ-approval February 9, 2026 10:00 — with GitHub Actions Failure

Version, resource count

3304597

rix0rrr had a problem deploying to integ-approval February 9, 2026 10:49 — with GitHub Actions Failure

rix0rrr added 2 commits February 9, 2026 13:35

Merge remote-tracking branch 'origin/main' into huijbers/measure-more…

8d37850

…-perf

Update toolkit metrics

0e32997

rix0rrr had a problem deploying to integ-approval February 9, 2026 12:50 — with GitHub Actions Failure

Boop

01acd0d

rix0rrr had a problem deploying to integ-approval February 9, 2026 13:04 — with GitHub Actions Failure

These tests don't make sense

be68914

rix0rrr had a problem deploying to integ-approval February 9, 2026 13:15 — with GitHub Actions Failure

Some tests

91e34b2

rix0rrr had a problem deploying to integ-approval February 9, 2026 13:32 — with GitHub Actions Failure

Tests

2203e37

rix0rrr had a problem deploying to integ-approval February 9, 2026 14:12 — with GitHub Actions Failure

aws-cdk-automation temporarily deployed to run-tests March 5, 2026 08:29 — with GitHub Actions Inactive

rix0rrr added 2 commits March 9, 2026 09:46

Merge remote-tracking branch 'origin/main' into huijbers/measure-more…

9a5b355

…-perf

Merge branch 'huijbers/measure-more-perf' of github.com:aws/aws-cdk-c…

f3ee487

…li into huijbers/measure-more-perf

rix0rrr had a problem deploying to integ-approval March 9, 2026 09:05 — with GitHub Actions Failure

Fix imports

245c919

rix0rrr temporarily deployed to integ-approval March 9, 2026 14:19 — with GitHub Actions Inactive

rix0rrr temporarily deployed to run-tests March 9, 2026 15:58 — with GitHub Actions Inactive

rix0rrr had a problem deploying to run-tests March 9, 2026 15:58 — with GitHub Actions Failure

rix0rrr temporarily deployed to run-tests March 9, 2026 15:58 — with GitHub Actions Inactive

rix0rrr added 2 commits March 16, 2026 11:56

Merge remote-tracking branch 'origin/main' into huijbers/measure-more…

878eb70

…-perf

Fix integ tests

e5d0a92

kaizencc reviewed Mar 17, 2026

View reviewed changes

rix0rrr added 2 commits March 17, 2026 16:16

Review comments

cceb32c

Trigger build

3841c14

kaizencc approved these changes Mar 17, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: collect metrics to help analyze performance problems#1124

fix: collect metrics to help analyze performance problems#1124
rix0rrr merged 24 commits intomainfrom
huijbers/measure-more-perf

rix0rrr commented Feb 9, 2026 •

edited

Loading

Uh oh!

Uh oh!

kaizencc Mar 17, 2026

Uh oh!

rix0rrr Mar 17, 2026

Uh oh!

kaizencc Mar 17, 2026

Uh oh!

Uh oh!

kaizencc Mar 17, 2026

Uh oh!

rix0rrr Mar 17, 2026

Uh oh!

kaizencc Mar 17, 2026

Uh oh!

rix0rrr Mar 17, 2026

Uh oh!

Uh oh!

kaizencc Mar 17, 2026

Uh oh!

rix0rrr Mar 17, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

		* It's hard for us to say `false` for sure, so we only respond
		* with `yes` or `don't know`.

Conversation

rix0rrr commented Feb 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rix0rrr Mar 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

rix0rrr commented Feb 9, 2026 •

edited

Loading

rix0rrr Mar 17, 2026 •

edited

Loading