Websocket new connections entering in a loop

Hi Render team and community!

I am using render hobby to host a Node JS websocket server (using ws lib), that is used for some ESP8266 clients to connect. I found an issue that I can’t figure out what is going on, because there’s no error on the server side, just a behavior occurring only when the app is deployed on render, not reproducible locally. The issue is that my clients, sometimes, enters in a loop when connecting to my server. I have an authentication function and the client reaches the point it is authenticated with success but something happens on the way, that is causing an error to establish the connection. I can see a protocols error on client side (ESP8266) and this keeps trying to connect until it gets success (and I dont understand why sometimes they can connect with success…). No errors from Node JS side, I can only follow what is happening by the logs and it seems to be fine… The client is authenticated but the connection crashes without any reason from Node side. This behavior is even worse if I add more than one async function inside my upgrade handler and if I have multiple clients connecting to it…

Technical description:

1 - On websocket server I do a simple authentication based on a header received from the ESP8266 client. I am using the server.on('upgrade') handler to do that:

The minimal node JS code is:

const app = express()
app.use(express.json())
app.use(bodyParser.urlencoded({ extended: false }))
app.use(cors())

const server = http.createServer(app)

const wss = new WebSocketServer({
	noServer: true
})
server.on('upgrade', function upgrade(request, socket, head) {
  socket.on('error', (err) => console.error(err))

  authenticateWsClient(request).then((data) => {
		const { err, req } = data
		if (err) {
			socket.write('HTTP/1.1 401 Unauthorized\r\n\r\n')
			socket.destroy()
			return
		}
	
		socket.removeListener('error', (err) => console.error(err))
	
		wss.handleUpgrade(request, socket, head, function done(ws) {
			wss.emit('connection', ws, request)
		})
	})
})

This async authenticateClients do a simple query on MongoDB database to check if the incoming id received from header exists

async function authenticateWsClient(req) {
  const url = req.url
  const splittedUrl = url.split('/')

  const deviceId = splittedUrl[1]
  console.log('------verify connected client------')

  if (!deviceId ) {
    return { err: 403, status: 403, message: 'Invalid token'}
  }

  const isDeviceValid= await Device.findOne({ deviceId })

  if (!isDeviceValid) {
    return { err: 403, status: 403, message: 'Not authorized'}
  }

  console.log(`Serial key ${deviceSerialKey} authorized. Connection Opened.`)
  return { req, message: 'Authorized' }
}

I tried to open spcecific topics on GitHub for each library I am using, and it has some additional details in my previous investigations:

Arduino Websockets: Cannot add async functions to server.upgrade. Causes client connection close during the authentication. · Issue #2235 · websockets/ws · GitHub
node js ws: Getting my connection closed when on node js wss server · Issue #163 · gilmaimon/ArduinoWebsockets · GitHub

Is there a way to track from render side what is happening?

There are two possible problems here I can think of up front, and unfortunately it’s not certain which is more likely. The only error we have to go off of is protocols error.

Is that a TLS protocol error? Or an HTTP protocol error? I lean towards the latter because you state that the client does authenticate which means it’s at least getting through TLS negotiation once. Does the ESP8266 support websockets? Is there detail that you can extract from the client on how much it does before it fails?

Embedded clients are stereotypically picky, and I would sooner start collecting as much diagnostic information from it before anything to do with the Render platform.

Hey Jason, thanks a lot for your answer. I am a little bit late because I was configuring an environment to reproduce the error only, without the other services running, to do a clear debug.

For some questions:
The code 1002 - protocols error coming from my ESP client: The library itself not gives information, but researching on internet, I found that 1002 means the HTTP response was not a valid WebSocket upgrade. So I think it is a HTTP error. (Sorry, on original post I did not send the code).

Does the ESP8266 support websockets?: Yes, websockets works fine with the library ArduinoWebsockets. GitHub - gilmaimon/ArduinoWebsockets: A library for writing modern websockets applications with Arduino (ESP8266 and ESP32)

Is there detail that you can extract from the client on how much it does before it fails? - Unfortunately, not. It seems to be an abrupt interruption, and the only thing that triggers is the onClose event on client side just after a successfull connection.

Another (maybe) useful information:

  • I am using https url to connect to my render app, like: “https://my-websocket-server.onrender.com/
  • The error ONLY occurs when I use “upgrade” functions, like the onUpgrade on the snippet I sent in the original post or the old approach of verifyClient from Node JS ws.
  • I am able to workaround the problem not using the upgrade functions. And to authenticate I use the onConnection instead, but I think this is a workaround to authenticate possible connections.

I don’t see any reference to onUpgrade functions ( ArduinoWebsockets/README.md at master · gilmaimon/ArduinoWebsockets · GitHub ). It seems like this is explicitly a WebSocket interaction library, and needs to connect directly to the WS Socket and immediately be speaking WebSocket requests.

Hey Jason, thanks again for your answer… In my understanding the upgrade function should be only server side. This function is used by the server to deny underised
incoming websocket requests. The websocket client tries to connect and at the beginning it is a Http request, then the server allows (or not) the client to be connected to wss connection. I think anything should be handled by the client at this point, this is why we should not find references to upgrade on Arduino websockets.

From my tests I have some points that validates the issue is not the Arduino websockets handling:

1) I do have a react app as a client too. The same issue occurs using this client to connect to the server. Enters in an eternal loop… then somehow it connects successfull.

2) I can connect normally both clients (esp8266 and React app) running my server from my machine using the snippet I sent. The issue only occurs when I use the deployed server on a render app. Do you know what changes can happen when we upload the server on the app?

I just remembered another curious thing… I was using the upgrade approach before without any issues, I had 2 fixed clients and sometimes the UI clients connecting to it. Once I added some new fixed clients to use my server (3 new ESP8266 clients), it became impossible to use due the loops on connection.

To keep the things working I am using the approach you said. Completely removed the upgrade function on the server, and I am handling the clients directly on the connection event. No issues from render/server side.

It’s not necessary to use HTTP → WS upgrade as the only means of allowing clients through. WS sessions can (and should) similarly be evaluated and shut down if undesired.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.