Add pool stats to /metrics admin endpoint

[steve-chavez]

As mentioned on #2042 (comment), this adds a /metrics endpoint on the admin server.

For now, it contains the database pool stats:

GET /metrics

{"dbPoolStats":{"currentUsage":20,"creates":43,"highwaterUsage":20,"createFailures":0,"takes":102097}}

New metrics would be namespaced on their own json field.

With the addition of avanov/pool#3 we'd also get the waiters in the queue (discussed on #2042 (comment))

{ 
  "dbPoolStats": {
    "currentUsage":10,"creates":421,
    "highwaterUsage":10,"createFailures":0,
    "totalWaiters":483,"takes":190445
  }
}

[wolfgangwalther]

Before inventing our own metrics output format, I guess it would be better to use something that is already in-use. Note: I have no idea about those formats - so you might already be doing so?

I'm sure you have #1526 on your radar, i.e. Prometheus-style metrics.

Are there any other "standard" formats for metrics out there?

If not, we might as well do Prometheus-style if no other suggestion comes up.

And when we do that - I wonder whether https://github.com/fimad/prometheus-haskell can be of any help?

[wolfgangwalther]

So it looks like prometheus uses a text-based format:
https://prometheus.io/docs/instrumenting/exposition_formats/

There's also OpenMetrics which builds on the prometheus format and tries to standardize it.

I briefly thought about whether we could provide different output depending on an Accept header - but I don't really know which header prometheus will send. It doesn't seem to be configurable: https://prometheus.io/docs/prometheus/latest/configuration/configuration/#scrape_config

I'd assume it sends a Accept: text/plain header or something similar - at least almost certainly not a Accept: application/json. So we could - in theory - implement a json metrics endpoint only when this is sent.

But I guess it would make more sense to evaluate the prometheus-haskell way.

[steve-chavez]

I recall trying to use prometheus-haskell but I had issues because https://github.com/avanov/pool (and other resource-pool variants) maintained its own counters and it was not possible to sync them. The newest hasql-pool version doesn't use resource-pool(we're migrating to that one on #2391), so this PR is no longer relevant.