Add org-quota-usage utility

[mogul]

Changes proposed in this pull request:

• add org-quota-usage script

security considerations

The utility doesn't do its own auth, instead relying on the cf CLI utility to already be logged in. This means it will act with the currently-authenticated user's permissions, though the inquiries are constrained to the specified organization.

---

but why?

To track org quota usage, the cloud.gov docs recommend using the report-usage plugin. I tried it out and discovered it has aged poorly since the last time I used it.

• It gives confusing output about "billable AIs", a concept which may vary across CF vendors
• It reports on SCS, a concept of billable services that's only relevant on Pivotal's CF deployments
• It doesn't indicate what apps contribute to the memory usage of individual spaces (without using jq on its suspect JSON output)
• It appears to use the older CF API for gathering data, and I couldn't get it to give me numbers that were consistent with what I was seeing in reality
  • It showed me that we were using ~64% of our org quota despite the fact that we were consistently running into "no more memory in quota" messages during deployments.
  • I tried to reconcile the numbers using totals in a spreadsheet after importing the CSV output, and got ~64% calculating one way and ~87% calculating another way

In summary, I didn't find the report-usage plugin useful or trustworthy, particularly when trying to figure out what I should be estimating for a contract mod to select an appropriate tier in cloud.gov's new credit-based pricing system to raise our ceiling, at a time when the current ceiling was stopping us cold and that tool was telling us we had headroom.

So with the help of Copilot, I wrote the included org-quota-usage.py utility:

• It uses the V3 CF API
• It understands processes vs apps, and in particular process types beyond web and worker, which are now possible
• It gives consistent and useful results

Output samples

Here are the bake-off results.

Old busted report-usage output

% cf report-usage -o gsa-10x-prototyping --format string 
org gsa-10x-prototyping is consuming 9216 MB of 12288 MB
        space usda_fns_staging is consuming 0 MB memory of org quota
                AIs billable: 0
                AIs canonical: 1 (0 running, 1 stopped)
                SCS instances: 0
        space 10x-nad-dev is consuming 0 MB memory of org quota
                AIs billable: 0
                AIs canonical: 0 (0 running, 0 stopped)
                SCS instances: 0
        space 10x-dependency-upgrades is consuming 0 MB memory of org quota
                AIs billable: 0
                AIs canonical: 0 (0 running, 0 stopped)
                SCS instances: 0
        space sandbox is consuming 512 MB memory of org quota
                AIs billable: 1
                AIs canonical: 6 (1 running, 5 stopped)
                SCS instances: 0
        space archive is consuming 0 MB memory of org quota
                AIs billable: 0
                AIs canonical: 0 (0 running, 0 stopped)
                SCS instances: 0
        space pra-navigator is consuming 256 MB memory of org quota
                AIs billable: 1
                AIs canonical: 1 (1 running, 0 stopped)
                SCS instances: 0
        space daniel.ivovich is consuming 0 MB memory of org quota
                AIs billable: 0
                AIs canonical: 0 (0 running, 0 stopped)
                SCS instances: 0
        space tts-10x-forms-dev is consuming 2048 MB memory of org quota
                AIs billable: 2
                AIs canonical: 2 (2 running, 0 stopped)
                SCS instances: 0
        space tts-10x-forms-demo is consuming 512 MB memory of org quota
                AIs billable: 1
                AIs canonical: 1 (1 running, 0 stopped)
                SCS instances: 0
        space tts-pic-dev is consuming 3584 MB memory of org quota
                AIs billable: 5
                AIs canonical: 5 (5 running, 0 stopped)
                SCS instances: 0
        space tts-pic-prod-mgmt is consuming 0 MB memory of org quota
                AIs billable: 0
                AIs canonical: 1 (0 running, 1 stopped)
                SCS instances: 0
        space tts-pic-demo is consuming 768 MB memory of org quota
                AIs billable: 2
                AIs canonical: 2 (2 running, 0 stopped)
                SCS instances: 0
across 1 org(s), you have 12 billable AIs, 19 are canonical AIs (12 running, 7 stopped), 0 are SCS instances

New hotness org-quota-usage.py output

% scripts/org-quota-usage.py gsa-10x-prototyping    
Org 'gsa-10x-prototyping'
  Quota: 12288 MB
  Used: 9216 MB (75.0%)
  Space 'archive':                                 0 MB (0.0%) 
  Space 'usda_fns_staging':                        0 MB (0.0%) 
      App 'usda-fns-ingestor':                     0 MB (0.0%)  [ 512 MB x 0/1 running]
  Space 'daniel.ivovich':                          0 MB (0.0%) 
  Space '10x-dependency-upgrades':                 0 MB (0.0%) 
  Space 'sandbox':                               512 MB (4.2%) 
      App '10x-sandbox-11ty-staging':              0 MB (0.0%)  [1024 MB x 0/1 running]
      App '10x-sandbox-api-staging':               0 MB (0.0%)  [1024 MB x 0/0 running]
      App '10x-atj-doc-automation-staging':      512 MB (4.2%)  [ 512 MB x 1/1 running]
      App 'gsa-ai-sandbox':                        0 MB (0.0%)  [1024 MB x 0/0 running]
      App 'gsai-gsa-gsai-dev':                     0 MB (0.0%)  [1024 MB x 0/2 running]
      App 'gsai-gsa-gsai-gsa-dev-ai-sandbox':      0 MB (0.0%)  [1024 MB x 0/2 running]
  Space '10x-nad-dev':                             0 MB (0.0%) 
      App '10x-nad-st-dev':                        0 MB (0.0%)  [ 512 MB x 0/0 running]
  Space 'tts-10x-forms-dev':                    2048 MB (16.7%)
      App 'tts-10x-forms-dev-server-doj-app':   1024 MB (8.3%)  [1024 MB x 1/1 running]
      App 'tts-10x-forms-dev-sandbox-app':      1024 MB (8.3%)  [1024 MB x 1/1 running]
  Space 'tts-10x-forms-demo':                    512 MB (4.2%) 
      App 'tts-10x-forms-demo-server-doj-app':   512 MB (4.2%)  [ 512 MB x 1/1 running]
  Space 'pra-navigator':                         256 MB (2.1%) 
      App 'pra-app':                             256 MB (2.1%)  [ 256 MB x 1/1 running]
  Space 'tts-pic-prod-mgmt':                       0 MB (0.0%) 
      App 'pic-demo-workflow-connector':           0 MB (0.0%)  [ 256 MB x 0/1 running]
  Space 'tts-pic-demo':                          768 MB (6.2%) 
      App 'pic-demo-workflow-connector':           0 MB (0.0%)  [ 512 MB x 0/0 running]
      App 'pic-demo-workflow-frontend':            0 MB (0.0%)  [ 256 MB x 0/0 running]
      App 'pic-demo-workflow-backend':             0 MB (0.0%) 
          Process 'web':                                        [1024 MB x 0/0 running]
          Process 'scheduler':                                  [ 512 MB x 0/0 running]
          Process 'worker':                                     [1024 MB x 0/0 running]
      App 'sample-gis-nepa-screening':           256 MB (2.1%)  [ 256 MB x 1/1 running]
      App 'rjsf-helper-2':                       512 MB (4.2%)  [ 512 MB x 1/1 running]
      App 'pic-demo-frontend':                     0 MB (0.0%)  [ 256 MB x 0/0 running]
      App 'pic-demo-auth':                         0 MB (0.0%)  [1536 MB x 0/0 running]
  Space 'tts-pic-dev':                          5120 MB (41.7%)
      App 'pic-dev-workflow-connector':          512 MB (4.2%)  [ 512 MB x 1/1 running]
      App 'pic-dev-workflow-frontend':           256 MB (2.1%)  [ 256 MB x 1/1 running]
      App 'pic-dev-frontend':                    256 MB (2.1%)  [ 256 MB x 1/1 running]
      App 'pic-dev-auth':                       1536 MB (12.5%) [1536 MB x 1/1 running]
      App 'pic-dev-workflow-backend':           2560 MB (20.8%)
          Process 'web':                                        [1024 MB x 1/1 running]
          Process 'scheduler':                                  [ 512 MB x 1/1 running]
          Process 'worker':                                     [1024 MB x 1/1 running]

Here's a screenshot of the CSV report after import into Google Docs, with an added summary row:

I hope this will be useful for support purposes, and I hope you will update the docs to point to this script instead of the plugin.

[mogul]

PS: I wanted to post about this over in the Discussions area, but there's no appropriate category for me to post in.

[pburkholder]

Thanks for submitting this. The report-usage plugin is dependent on v2, and this could be a useful stopgap before we rewrite it (if ever).
Can you move this into the `/cloudfoundry' directory?

[pburkholder]

FWIW, for one org this too 3m49s while the plugin took 14s. They have up to 50 apps in a space, so they are definitely an outlier.

[mogul]

FWIW, for one org this too 3m49s while the plugin took 14s. They have up to 50 apps in a space, so they are definitely an outlier.

Give this new version a try. My local test case went from 66s to 12s, and I suspect it'll be a bigger improvement for you.