TCP ARP resolution issue

[aersek]

Hello

We were testing our uTasker telnet implementation over internet. As internet gateway is used GPRS modem. After reset of module, on received SYNC from telnet client uTasker had sent ARP resolution to find MAC of gateway. On reception of ARP resolution from gateway uTasker didn't send SYC ACK frame. To fix this situation I have changed tcp.c code:

extern void fnTaskTCP(TTASKTABLE *ptrTaskTable)
{
    QUEUE_HANDLE PortIDInternal = ptrTaskTable->TaskID;                  // queue ID for task input
    unsigned char ucInputMessage[MEDIUM_MESSAGE];                        // reserve space for receiving messages

    while ( fnRead( PortIDInternal, ucInputMessage, HEADER_LENGTH )) {   // check input queue
        switch ( ucInputMessage[ MSG_SOURCE_TASK ] ) {
            case TIMER_EVENT:
                fnPollTCP();                                             // do TCP management on a periodic basis
                uTaskerMonoTimer( OWN_TASK, T_TCP_PERIOD, E_POLL_TCP );  // restart the timer
                break;

            case TASK_ARP:
            {
                TCP_CONTROL *ptr_TCP = tTCP;
                fnRead( PortIDInternal, ucInputMessage, ucInputMessage[MSG_CONTENT_LENGTH]);// read the contents
                switch (ucInputMessage[ 0 ]) {                           // ARP sends us either ARP resolution success or failed
                case ARP_RESOLUTION_SUCCESS:                             // a TCP socket is owner of resolved address
                    ptr_TCP += ucInputMessage[ 1 ];                      // extract the socket reference
                    if (ptr_TCP->ucTCP_state == TCP_STATE_SYN_SENT) {    // ARP resolve on connection establishment    
                        #ifdef ANNOUNCE_MAX_SEGMENT_SIZE                 // TCP socket waiting for connection
                        fnSendSyn(ptr_TCP, TCP_FLAG_SYN);                // send SYN with MSS announcement
                        #else
                        fnSendTCPControl(ptr_TCP);                       // resent because IP address has just been resolved
                        #endif
                    }
                 else if (ptr_TCP->ucTCP_state == TCP_STATE_SYN_RCVD) { // next state is SYN_RCVD where an ACK will establish the connection
                  #ifdef ANNOUNCE_MAX_SEGMENT_SIZE
                        fnSendSyn(ptr_TCP, (TCP_FLAG_SYN | TCP_FLAG_ACK));     // send SYN, ACK with MSS announcement
                  #else
                        ptr_TCP->ucSendFlags = (TCP_FLAG_SYN | TCP_FLAG_ACK);  // prepare flags to be transmitted
                        fnSendTCPControl(ptr_TCP);                             // send SYN, ACK
                  #endif  

                    }
                   else {
.
.
.


Is this correction OK? With this adaptation everything works fine.

[mark]

Hi

I understand the fix but I am not sure about the problem at the moment.

1) If there has been a SYN reception at the TELNET listerer the ARP address will normally be known (the reception will have added it to the ARP cache) and so there is normally never the need to send an ARP resolution and thus be able to handle the ARP resolution event in the TCP_STATE_SYN_RCVD state.

2) You are however using a GPRS modem and this may make a difference. Could you explain how the modem works? Is the traffic passing through the Ethernet driver or Ethernet task (i.e. the modem is like a PHY to the Ethernet module)? Or is it communicating via a different interface (like SPI or UART?). Note that, it may be that you are missing updating the ARP cache on IP reception if the path is a bit different and it may be better to solve it there(?)

Thanks, regards

Mark

[aersek]

GPRS modem is connected to board using etherent cable, it is independent device which acts similar to ADSL modem router, just using GPRS. Modem and board are powered up, for first time board is accessed over internet, so modem and board have updated their ARP-s. After that happen reset on board, modem still has board MAC in ARP table, but board has empty ARP table. In that situation I access board again over internet, modem directly forward SYN to board without ARP, but board before returning SYN ACK need to resolve default gateway (modem) MAC. In this situation happen issue from first post.

Best regards

Andrija

[aersek]

Another one problem which I have for some time and can't fix it:
Using telnet board based on ColdFire processor sends large amount of data. It sends it fast and PC TCP window become small. On this wireshark log PC is 192.168.1.100 and board 192.168.1.10.
Board instead of waiting to window become larger sends to much of data. It seems that board process frame 2329 (which inform board that window is 879) after sending frame 2330 (size of frame is 971, same to window size from frame 2326). In processing of frame 2329 board updates peers window size, bit peers window size is actually smaller for size of frame 2330. Because of late processing of ACK frame it sends next frame 2331 which is 879 bytes long and overfills peers input buffer.


To avoid this situation I have changed in function fnHandleTCP next line
#ifdef SUPPORT_PEER_WINDOW
//      ptr_TCP->usTxWindowSize = rx_tcp_packet.usWindowSize;             // save the present windows size advertised by the peer
   ptr_TCP->usTxWindowSize = rx_tcp_packet.usWindowSize-(ptr_TCP->ulNextTransmissionNumber-rx_tcp_packet.ulAckNo);
#endif

With this change I don't get buffer overfill any more, but now have error in processing silly window situation.



In fnSendBufTCP routine Silly window situation is detected and then is send empty TCP frame 1473 with flags PSH and ACK. Then on retransmition call of function fnSendBufTCP(Socket, 0, 0, TCP_BUF_REP) returns 0 after which connection is closed.

[mark]

Hi Andrija

Sorry about taking so long to get back to you.

I have analysed the first point about the address resolution and now agree that the improvement here is correct. What I didn't realise before was that a remote connection request which arrives via the gateway doesn't cause the gateway address to be entered into the local ARP table (I think that this needs to remain this way too since there could be multiple gateways or other traffic outside the local subnet and so it can not be assumed that any reception from outside the local sub-net is arriving from the default gateway).

This means that it is also normal that the SYN + ACK can not be sent until the default gateway's MAC address (or GPRS modem in your case) has been resolved.

As you noticed, when responding with SYN + ACK the ARP resolution event was being ignored. This would have led to a delay because the remote client will then need to resend the original SYN, which would then get answered since the gateway's address has been resolved in the meantime.

I have therefore accepted your code modification.

Concerning the peer buffer overrun / Silly-Window problem that you have had I will look into that next. I have used TELNET in situations where flow control is handled by the TCP window and it is also quite easy for the embeded web server to close the clients window when working with Windows 98 PCs (presumably slower due to their age) but didn't have such a difficulty. I'll let you know once I have information.

Regards

Mark

P.S. If possible, please also send the Wireshark recordings via Email since they are then easier to analyse in detail.

[mark]

Andrija

I have started looking into the case where you could overrun the peers input buffer.

1) I believe that it is true that the length of transmitted data which hasn't been acknowledged is not being considered when the peer's window size if updated. This happens only when windowing is in operation and the transmitter is sending multiple frame before waiting for an ACK, "and" when the peer acknowledges only a part of the outstanding data.

2) If I am not mistaken, the correction can therefore also be restricted to the case when we receive ACKs to parts of outstanding data (reference TCP_EVENT_PARTIAL_ACK). Therefore I propose to leave it generally as
ptr_TCP->usTxWindowSize = rx_tcp_packet.usWindowSize;                // save the present windows size advertised by the peer

Since the correction will usually result in no change it is also more efficient to not adjust the value when not needed.

3) I propose that the adjustment be made only when handling partial ACKs. That is, when we are expecting an ACK to outstanding data [ptr_TCP->ulSendUnackedNumber != ptr_TCP->ulNextTransmissionNumber] but the peer is not acknowledging "all" outstanding data [rx_tcp_packet.ulAckNo != ptr_TCP->ulNextTransmissionNumber].

4) This is handled as follows at the moment:
                    if ((ptr_TCP->ulNextTransmissionNumber != rx_tcp_packet.ulAckNo) && (APP_REQUEST_CLOSE & (iAppTCP |= ptr_TCP->event_listener(ptr_TCP->MySocketNumber, TCP_EVENT_PARTIAL_ACK, ptr_TCP->ucRemoteIP, (unsigned short)(ptr_TCP->ulNextTransmissionNumber - rx_tcp_packet.ulAckNo))))) {
                        if (rx_tcp_packet.usHeaderLengthAndFlags & TCP_FLAG_FIN) { // the application has commanded the close of the connection
                            // we have just received FIN + ACK, meaning that the other side has also initiated a close
                            fnNewTCPState(ptr_TCP, TCP_STATE_CLOSING);   // this a simultaneous close, so we go to state closing and wait for the last ack
                        }
                        return;
                    }
                    iAppTCP |= HANDLING_PARTICAL_ACK;                    // mark that we are dealing with a partial ACK

5)
I propose changing this as follows:

                    unsigned short usPartialDataLength = (unsigned short)(ptr_TCP->ulNextTransmissionNumber - rx_tcp_packet.ulAckNo); // the length of the data being acknowledged
                    if (usPartialDataLength != 0) {                     
                        ptr_TCP->usTxWindowSize -= usPartialDataLength; // since there is still data underway to the peer respect this in the window size
                        if (APP_REQUEST_CLOSE & (iAppTCP |= ptr_TCP->event_listener(ptr_TCP->MySocketNumber, TCP_EVENT_PARTIAL_ACK, ptr_TCP->ucRemoteIP, usPartialDataLength))) {
                            if (rx_tcp_packet.usHeaderLengthAndFlags & TCP_FLAG_FIN) { // the application has commanded the close of the connection
                                // We have just received FIN + ACK, meaning that the other side has also initiated a close
                                //
                                fnNewTCPState(ptr_TCP, TCP_STATE_CLOSING);   // this a simultaneous close, so we go to state closing and wait for the last ack
                            }
                            return;
                        }
                    }
                    iAppTCP |= HANDLING_PARTICAL_ACK;                    // mark that we are dealing with a partial ACK

This restricts the adjustment (correction) to the case where it is relevant (efficiency).

Regards

Mark