Generation of a tar.gz for a big artifact should be interrupted if the client disconnected
We know that the generation of .tar.gz can be very expensive when you try to download the result of a linux build for instance. So expensive in fact that haproxy cuts you the connection after a minute without having received anything from the server.
When we are in this situation, we should immediately stop the process of generating the archive as the work is wasted and if we don't stop quickly, daphne gets unhappy and kills the application instance entirely:
Aug 25 01:24:17 poseidon /usr/bin/daphne[955174]: Application instance <Task pending name='Task-4117726' coro=<ProtocolTypeRouter.__call__() running at /usr/lib/python3/dist-packages/channels/routing.py:62> wait_for=<Future pending cb=[_chain_future.<locals>._call_check_cancel() at /usr/lib/python3.11/asyncio/futures.py:387, Task.task_wakeup()] created at /usr/lib/python3.11/asyncio/base_events.py:427> created at /usr/lib/python3.11/asyncio/tasks.py:670> for connection <WebRequest at 0x7f537e702690 method=GET uri=/debian/developers/artifact/2289505/download/ clientproto=HTTP/1.0> took too long to shut down and was killed.
It's not clear how we can fix this.
https://fastapiexpert.com/blog/2024/06/06/understanding-client-disconnection-in-fastapi/#check-client-disconnection has some example for FastAPI
https://forum.djangoproject.com/t/how-do-i-detect-a-client-disconnect-from-an-async-streaminghttpresponse/22323 says that we might need Django 5.2 ...
Maybe a simple hack could be to start the StreamingResponse earlier, before we finished to even generate the data? And rely on its feature to detect when we have been disconnected?