Issues with concurrency in resource-constrained environments like CI

[anuraaga]

I am wiring up granian to the test suite for connect-python, an RPC framework. Unfortunately I have been having issues getting stable results from it on CI and is the only server exhibiting this out of gunicorn, hypercorn, uvicorn. I wonder if I may not be understanding the threading knobs well.

The test suite consists of a test runner in Go that makes concurrent requests to a server. Other servers work well with the default concurrency of up to 40, but I have to limit granian.

https://github.com/connectrpc/connect-python/pull/33/files#diff-63c2ddded4f2738d6dcf063607d7f8a02b8b4cd45b44dc9ff9237c7256006940R56

I start granian with subprocess here

https://github.com/connectrpc/connect-python/pull/33/files#diff-1770620794e995e16b29e3662d0c056cd03eb04afd7bb89712ed50189c23f910R434

Before that, even with concurrency of 1 I couldn't get it to reliably pass without adding --workers 8. I didn't expect to need multiple workers without any concurrency. I have tried runtime threads instead and didn't get improvement out of it.

The errors are connection-related, for example connection reset by peer, connection refused, broken pipe, or just timeout from hanging. I have seen similar errors both for ASGI and WSGI.

Just a couple example logs

https://github.com/connectrpc/connect-python/actions/runs/18550272910/job/52876338071
https://github.com/connectrpc/connect-python/actions/runs/18549058777/job/52872718245

Incidentally I see many ASGI Transport Error SendError log messages, but this may be expected since we have tests of client closing connection before server completes.

One other observation is even with parallelism 1, I couldn't get reliable results on a mac with the default ulimit and had to raise it - other servers had issue with default ulimit with concurrency but not without. It feels like that could only happen if sockets are getting leaked somehow.

I wanted to share this since I'd like to remove that parallelism limit and have stable performance with granian - connect uses HTTP/2 for some features and it would be great to be able to recommend granian as a default choice with it. But until understanding this performance issue, I have trouble doing that.

As I am running a complicated test suite, it may be hard to debug this but first I wanted to try to see if you have any intuitive thoughts from this description, perhaps misconfiguration by me or potential socket-related handling in the code. Thanks.

[anuraaga]

In the end I had to give granian a pass on CI

https://github.com/connectrpc/connect-python/blob/main/conformance%2Ftest%2Ftest_server.py#L81

Unfortunately I won't be able to mention it in any docs while that is in place.

[gi0baro]

I'm not aware of any specific Granian issues on resource-constrained environments.

Given what you described, my suggestions are:

• configure blocking threads and/or backpressure on WSGI
• if you want to limit the amount of requests in queue, lower down the backlog value (default is 1024)

In general, running in a CI with more than 1 worker doesn't sound like a great idea.

[anuraaga]

Sorry I forgot to mention while I thought things were fine with low concurrency, it turns out even with just 1 it wasn't stable. Hence the pass.

Currently I plan to document support for hypercorn but not granian for http/2 support since that is what our tests show. If you're interested in users welcome support in understanding this better so we can mention granian too, more options are always better but we need them to be stable with low concurrency. It's up to the author though to commit to getting there.