Skip to content

Comments

Fix #1159: try Granian #1162

Open
leplatrem wants to merge 7 commits intomainfrom
replace-uwsgi
Open

Fix #1159: try Granian #1162
leplatrem wants to merge 7 commits intomainfrom
replace-uwsgi

Conversation

@leplatrem
Copy link
Contributor

@leplatrem leplatrem commented Feb 5, 2026

Fix #1159

Notes for deployment:

  • GRANIAN_TRUSTED_HOSTS would have to be set for X-Forwarded-Host to be read (used in redirections)
  • Prometheus metrics scraped on localhost:9090

@leplatrem leplatrem force-pushed the replace-uwsgi branch 2 times, most recently from fa15480 to d2b769e Compare February 9, 2026 12:25
@leplatrem leplatrem marked this pull request as ready for review February 9, 2026 12:25
@leplatrem leplatrem requested a review from a team as a code owner February 9, 2026 12:25
@alexcottner
Copy link
Contributor

It looks like the web server is hanging and not returning responses with granian much of the time. I think we have a compatibility issue between kinto and granian's wsgi implementation.
Example request:

curl --request GET \
  --url 'http://localhost:8888/v1/buckets?_sort=-last_modified' \
  --header 'Authorization: Basic dXNlcjpwYXNz'

This just hangs forever for me. The root v1 response seems to work most of the time, but not every time.

I don't have more time to dig into it at the moment. I tried updating uvicorn just in case that was causing issues too, but no change.

Copy link
Contributor

@alexcottner alexcottner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't make this work consistently locally. I'm not sure why, but it's very flaky with responses. The logs indicate it's doing the work, but then it doesn't return a response.

@leplatrem
Copy link
Contributor Author

@alexcottner removing the wrapper seems to fix the issue.
We would have to be careful when rolling this out though, because fo forwared-hosts and redirections

@leplatrem
Copy link
Contributor Author

Spoiler: it doens't

dependabot bot and others added 2 commits February 24, 2026 08:48
The previous config tried to run a single memcached process on two ports
(`-p 11211 -p 11212`), which is invalid — memcached only accepts one `-p`
flag. This left one port unbound and the other broken under qemu emulation
(`platform: linux/amd64` on Apple Silicon), causing python-memcache to
block for its 3-second socket timeout on every authenticated request,
freezing Granian's blocking thread.

Replace with two separate memcached services (one per port), using the
multi-arch image without a platform constraint so each host pulls the
native variant automatically.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@leplatrem
Copy link
Contributor Author

  Root Cause

  The browser tests were hanging because of a postgres deadlock specific to Granian's multi-threaded
  model:

  1. kinto defaults transaction_per_request=True, meaning postgres writes are managed by Pyramid's
  transaction manager (pyramid_tm) and committed at the end of each HTTP request.
  2. auto_create_resources fires on the ApplicationCreated startup event — outside of any HTTP request
  context, so pyramid_tm never commits those postgres writes. The INSERTs for main-workspace,
  main-preview, etc. were left in an idle in transaction state indefinitely.
  3. When the test's create_bucket(main-workspace) request arrived, a second postgres connection tried
  to INSERT the same row and blocked waiting for the open transaction to commit or rollback — deadlock.
  4. Why not uWSGI? uWSGI used multiple processes, each with their own postgres connections. With
  Granian's single-process multi-threaded model, _CLIENTS (kinto's module-level dict) is shared, so both
   the startup event and request threads share the same connection pool.

  Fix

  In auto_create_resources, wrap the resource creation loop in with transaction.manager: to ensure the
  postgres writes are explicitly committed when running outside of a Pyramid request context.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Replace uwsgi with another WSGI server

2 participants