HTTP stream buffering in static site rewrite?

Hello, I am using a Render Static Site with rewrites in front of two 2 Web Services. One service provides some HTTP streaming responses for an LLM chat application.

When I request the streaming endpoint against the API directly and compare that to hitting the same streaming endpoint through the Render static site rewrite, the response through the Render static site rewrite appears to have some buffering characteristics. That is, the response is delivered as a smaller number of larger chunks.

Here is a comparison of the per-line timings from hitting the API directly vs through the static site: 0-measurement-curl.sh · GitHub

With the static site rewrite, lines 0-20 are all received at 1569ms, then lines 21-43 at 2206ms, etc.

Is it possible to control or disable this buffering in some way, either through HTTP response headers from the API backend, or configurations, or something else? If not, do you have a recommended reverse proxy to run on Render, eg nginx? (I have had difficulty configuring Caddy).

Here are HTTP response headers from hitting the API directly on Render:

HTTP/2 200
date: Fri, 31 May 2024 21:12:32 GMT
content-type: text/event-stream; charset=utf-8
cf-ray: 88c9f672fe711736-SJC
cf-cache-status: DYNAMIC
access-control-allow-credentials: true
rndr-id: b4efbbc1-9dd8-40fa
x-render-origin-server: uvicorn
server: cloudflare
alt-svc: h3=":443"; ma=86400

And the HTTP response headers when requesting through the Static Site Rewrite:

HTTP/2 200
date: Fri, 31 May 2024 21:13:06 GMT
content-type: text/event-stream; charset=utf-8
cf-ray: 88c9f73c8d7e1739-SJC
cf-cache-status: DYNAMIC
cache-control: max-age=0
vary: Accept-Encoding
access-control-allow-credentials: true
alt-svc: h3=":443"; ma=86400
rndr-id: ae173047-7a5b-4f20
x-render-origin-server: uvicorn
server: cloudflare

Thank you for any suggestions!

Also, requesting to the API server directly (uvicorn FastAPI) it includes the transfer-encoding: chunked response header which is not present in the cloudflare responses above, in case that’s relelvant/helpful.

HTTP/1.1 200 OK
date: Fri, 31 May 2024 21:29:11 GMT
server: uvicorn
content-type: text/event-stream; charset=utf-8
access-control-allow-credentials: true
transfer-encoding: chunked

Hi there,

The web server that serves our static site content is not configurable by a customer. If the buffering of this content is causing issues for your front end, then you will likely need to call your web service directly.

Regards,

Keith
Render Support, UTC+10 :australia:

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.