TCP segments of an HTTP Request in wrong order

My web-services server sometimes does not receive correct HTTP requests and returns “500 – Internal Server Error”. Using tcpdump and Wireshark on the server, I found out that HTTP requests are splitted into 2 TCP packets, and that sometimes, the server tries to process the request before the second packet could arrive.

This wireshark capture has been taken on the server side.

So what I see is that :

• The first fragment of the HTTP request is received at 54.659
• It is received again at 71.168
• The second (and last) fragment of the request is received at 99.869 (that is 45 seconds after the first one)
• Four millisecond before, at 99.865, the server kind of timed-out and tried to process an incomplete request (which gives a 500 Error)

I don’t know where to look now. I would say that it is a network issue but I have several TCP streams where the server tries to process the request several milliseconds before it is completely received. On the other hand, TCP paquets that takes 45+ seconds to arrive means that the network is really bad.

Do you have any pointers on how to investigate more ?

I don’t know where to look now

Nowhere. Seriously.

Happens.

hat is 45 seconds after the first one

That is HUGH. Seriously. Internet latency Europe to USA is around 150ms. You are 30 times that much – it is a drop without resend. Sadly, unless you control BOTH SIDES (!) you can not control the behavior of the client. Stuff like that happens.

If that is your LAN – it is seriously crappy. It that is internet, that is just how it is. The main question is how bad it is – if it is “some connections of thousands” it could be a serious network problem on the other end. If that happens at times for nearly everyone, it is closer to your side (connection, data center etc.). Yesterday, for example, we had a situation like that here – some stupid * DDOS’s one of my links. SERIOUS congestion could lead to packets getting nearly impossible to get through. But if that is not you – there is nothing you can do.

This is like someone coming too late to a meeting and telling you he had a serious traffic jam. unless the Jam is in your street you can not know. The Internet can be at times in places quite crappy.

Check more discussion of this question.

Related posts:

1. http request to a web server timeout on firts request
2. How to debug “HTTP request sent, awaiting response”?
3. Tool to catch http request, modify it and send again
4. What is a CCM_POST HTTP Request?
5. Does Apache spawn piped logger on each HTTP request?