µTasker Forum > µTasker general

Using TCP in the uTasker project

(1/10) > >>

mark:
Hi All

The uTasker demo project uses TCP for several services. These include HTTP, SMTP, FTP, TIME SERVER and TELNET, to name the ones which initially spring to mind.

The demo project enables these services to be easily studied and own services to be written (usually termed clients and servers) based on these example. There is, at the time of writing, no 'official' document which contains a definitive guide.

This is logically the reason why the use of TCP is a frequent question and frequent discussion topic in private mails. It is also a reason why this document, which is long overdue, should soon be made available.

Therefore this thread is designed to be an introduction to the subject to answer the first burning questions and to give first practical examples, as well as more detailed insight into how it works, how it can be used and why it all works like that. If all goes well, the thread comntent can be moved to a more structured document at a later point in time.

TCP is in fact a very simple protocol - in its simpest form. But in order to allow it to perform with the speed and reliability which has enabled it to become originally the standard in the Internet and eventually the standard in almost all networks it does need a few additional frills which can then make it quite complicated indeed.

If you would like to discuss standard TCP details please feel free to create your own topic to get a discussion going. I have always found it very difficult to find any really meaningful discussions about the practical aspects in the Internet so it would be nice to be able to find some good ones here!


A simple TCP server example. (The HTTP service is a good source for reference).

First we get a socket from the TCP socket pool. The size of the socket pool is defined in config.h, where the define USER_TCP_SOCKETS can be incremented for every additional socket which user code (as opposed to standard services like HTTP) will need.
The following configures a socket with the defined characteristic (minimum delay as opposed to maximum throughput or maximum reliability - these are router parameters), a timeout of 10 seconds (after this length of time with no activity it will close the connection. A value 0xffff means no timeout!), and a callback routine called fnTestListener() which will be doing the work later.


--- Code: ---static USOCKET Test_socket = -1;                                          // declare a static socket variable
...
    Test_socket = fnGetTCP_Socket(TOS_MINIMISE_DELAY, (unsigned short)10, fnTestListener); // from task or sub-routine code
--- End code ---

The socket is not in the listening state (which is needed for it to act as a server) so we start it in this state by calling

--- Code: ---static unsigned short usTestPort = 0x9999;                               // declare a static socket variable (the value could also be fixed)
....
    fnTCP_Listen(test_socket, usTestPort, 0);                             // after getting the socket, bind it to a listening port
--- End code ---

Now the socket is listening on port 0x9999, which means if you have a test client (eg. Telnet on this port) send a connection request to it it will start a connection process.
Note that the parameter 0 when setting the listening state is the maximum windows size which the connection can receive - a zero defaults to the standard LAN TCP pay load size of 1460 bytes. For some applications with special buffer restrictions this can be used to improve flow control in the rx sense (CONTROL_WINDOW_SIZE needs to be enabled in config.h) but this won't be described here just yet.

The simple server listener looks something like this.


--- Code: ---#define TEST_BUFFER_LENGTH 10
typedef struct stTCP_MESSAGE
{
    TCP_HEADER     tTCP_Header;     // reserve header space
    unsigned char   ucTCP_Message[TEST_BUFFER_LENGTH];
} TCP_MESSAGE;

static int fnTestListener(USOCKET Socket, unsigned char ucEvent, unsigned char *ucIp_Data, unsigned short usPortLen) {
   TCP_MESSAGE test_message;

    switch (ucEvent) {
    case TCP_EVENT_CONREQ:                                             
    case TCP_EVENT_CONNECTED:
    case TCP_EVENT_CLOSE:
    case TCP_EVENT_ACK:
    case TCP_EVENT_ARP_RESOLUTION_FAILED:
    case TCP_EVENT_PARTIAL_ACK:
        break;
    case TCP_EVENT_REGENERATE:
    case TCP_EVENT_DATA:
        if ((ucEvent == TCP_EVENT_REGENERATE) || (!uMemcmp((CHAR*)ucIp_Data, "TEST" , 4))) {
            uStrcpy((CHAR*)test_message.ucTCP_Message, "Hi!!");
            if (fnSendTCP(Socket, (unsigned char *)&test_message.tTCP_Header, 4, TCP_FLAG_PUSH) > 0) {
                return APP_SENT_DATA;
            }
        }
        break;
    case TCP_EVENT_ABORT:
    case TCP_EVENT_CLOSED:
        fnTCP_Listen(Socket, ++usTestPort, 0);                    // go back to listening state on next port number
        break;
    }
    return APP_ACCEPT;
}
--- End code ---

As you can see it receives events from the TCP layer, some with other details such as a pointer to data.

This is a list of the events which can arrive:


* TCP_EVENT_ABORT               TCP connection was closed (due to error or a reset from peer)
* TCP_EVENT_CONNECTED        TCP connection has been successfully established
* TCP_EVENT_ACK                   Data which we sent have been successfully acknowledged (for all sent data, with outstanding acks)
* TCP_EVENT_DATA                 Data frame received with data content for us to treat
* TCP_EVENT_CLOSE                Peer is starting a close of the present TCP connection
* TCP_EVENT_CLOSED              The TCP connection has been fully closed
* TCP_EVENT_REGENERATE       The last data frame was lost and must be resent!!!
* TCP_EVENT_CONREQ              A peer wants to establish a TCP connection (SYN received)
* TCP_EVENT_ARP_RESOLUTION_FAILED  Generally only when working as client and sending a connection request or data after a longer period of no activity. It means that the the SYN or the data could not be sent by the IP layer since the address of the destination could not be resolved (destination powered down or network problem, or just doesn’t exist). In this cause the application is informed that the connection is either not possible of the connection has broken down due to this.
* TCP_EVENT_PARTIAL_ACK       This is only used when Windowing is in operation. Presently only TELNET uses it. It means that an ACK has been received for a part of the data which has been sent (rather that all data that has been sent). This is used to acknowledge that only a part of the data has been successfully delivered rather than all outstanding data. This has to be treated differently to a normal ACK event since the ack number has to be used to calculate which outstanding data acks are still open and expect their own ack later.
Now if you are new to TCP, enter in the above test program - best in the simulator - and you can see your first simple TCP server in operation. It won't do a lot but it will at least do the following.

1. When the client requests a connection to port 0x9999 the event TCP_EVENT_CONREQ will be received. Here it would be possible to deny the connection to a black-listed IP for example but our server always allows connections.
2. The TCP connection will be established and the event TCP_EVENT_CONNECTED is received.
3. If the client sends any data it will arrive with the event TCP_EVENT_DATA. The server will only respond to the message "TEST" in which case it will send the string "Hi!!" as answer.
4. After 10s with no activity, the server will close the connection (it is actually TCP that does this after the idle timer fires but the server receives the event TCP_EVENT_CLOSED when the connection has successfully closed).
5. The lister socket is not active after the connection has been closed so the code sets it up as listener again, this time with port number 0x999a.
6. If the connection is repeated, the port number is always incremented each time and the connection always times out after 10s.

Note that when the server sends data it returns APP_SENT_DATA. This informs TCP that the application has just successfully sent something and TCP doesn't have to acknowledge the received data - the ack has already been piggy-backed with the transmitted data.

If the transmission were to fail and need repeating, the server has to handle the event TCP_EVENT_REGENERATE. Since we only ever send one message it is obviouse that this has failed and so it is simply resent. The event TCP_EVENT_ACK will be received when the transmitted message actually has arrived safely.
It is in fact a great shame that messages can and do get lost (but it is not a perfect world and so we have to be ready for it) because regeneration of previously sent messages is probably the most complicated and/or RAM consuming parts of TCP implementations. This example has it real easy, but this is also a real simple example!

Here is a list of all possible return codes from the listener.


* APP_ACCEPT             This is normal return when no data has been sent. The TCP level will then acknowledge the received frame and the connection will continue
* APP_SENT_DATA       The listener is informing TCP that it has sent data (be sure that data was really sent when returning this, it should not be returned if a transmission attempt failed)
The TCP level will therefore not send an ACK of its own since the data will have had a piggy-back ack of its own. If there is a mistake and data is sent and APP_ACCEPT is returned it is not serious – there will simply be an unnecessary ACK sent which doesn’t actually hurt anything (just not so efficient)
* APP_REJECT             This is used to reject a connection (for example if the IP address is on a black list…) It causes TCP level to return a RESET rather than perform the TCP connection sequence
* APP_REQUEST_CLOSE    The application informs TCP that the application has just actively started the close of the present TCP connection.
It is returned after the application calls fnTCP_close(socket); 
* APP_REJECT_DATA       This is a special case where the application doesn’t want to accept data at the moment. It doesn’t cause the connection to be broken down but TCP will not ACK the data. The peer will then be forced to repeat the data – resulting in a delay (using the peer as a queue, possibly until data can be received).
* APP_WAIT                Same as previous but delay on TCP connection establishment.
Our example has made use of very few features up to now and there is much more to be explained. However this should allow first steps to be taken and also the servers in the project (HTTP, FTP etc.) to already make a lot more sense.

I will continue the topic later with a first simple client example. If you already have confusions or questions, or corrections etc. please feel free to comment.

Regards

Mark

mark:
Hi

Here is some general information about TCP use in the uTasker project. It is an overview rather than a tutorial and is possibly a little vague in some respects. I hope that it doesn't confuse more that it helps, but it may well contain something of interest so here it is:


Here are the main user calls for working with TCP (see tcpip.h). You will find that the uTasker project is not based on Berkley conventions – it was designed as an alternative solution in small footprint projects where a mix between performance, reliability, simplicity and user comfort was defined, based on typical embedded project use. Therefore it doesn’t not try to copy any existing solution but goes its own way.


extern USOCKET fnTCP_Listen(USOCKET TCP_socket, unsigned short usPort, unsigned short usMaxWindow);

extern unsigned short fnGetFreeTCP_Port(void);

extern USOCKET fnGetTCP_Socket(unsigned char ucTos, unsigned short usIdleTimeout, int (*listener)(USOCKET, unsigned char, unsigned char *, unsigned short) );

extern USOCKET fnReleaseTCP_Socket(USOCKET TCPSocket);

extern USOCKET fnTCP_Connect(USOCKET TCP_socket, unsigned char *RemoteIP, unsigned short usRemotePort, unsigned short usOurPort, unsigned short usMaxWindow );

extern USOCKET fnTCP_close(USOCKET TCP_Socket);

extern signed short fnSendTCP(USOCKET TCP_socket, unsigned char *ptrBuf, unsigned short usDataLen, unsigned char ucTempFlags);

extern QUEUE_TRANSFER fnSendBufTCP(USOCKET TCP_socket, unsigned char *ptrBuf, unsigned short usDataLen, unsigned char ucCommand);

extern void fnModifyTCPWindow(USOCKET Socket, unsigned short usSafetyWindow);

extern void fnReportWindow(USOCKET Socket, unsigned short usBufferSpace);

 

The TCP task handles ARP so ARP is fully transparent to the user. However there are three types of TCP sockets which can be used.

The first is a RAW socket where the user supplied call back handles all events (see for example the http call back) which means that also repetitions must be handled there.

The second is a buffered socket. TELNET is an example of a protocol which makes use of this. The TCP socket has an inbuild data buffer for backing up data. It can send data of 'random' nature  rather than 'regeneratable' data at higher rates because it uses windowing and the buffered for retransmissions if necessary, as well as grouping single data transissions into larger transmission packets.

The third is a TELNET like socket where a TELNET socket is used instead of a TCP buffered socket – the TELNET socket can be in RAW mode so that it is otherwise transparent and doesn’t do negotiation or byte stuffing etc. The advantage of the TELNET like socket is that the TELNET layer provides an intermediate handler which handles repetitions and also TCP windowing so that high through put can be easily achieved. The down side is that it requires some buffer space (it works with fnSendBufTCP() rather than fnSendTCP()) to allow back ups of transmitted frames where the user simply writes to the socket as if it were any data port like a serial interface – the TELNET layer gets rid of the buffered data in the most efficient manor possible and ensures reliable delivery. There is an example of this in the demo project in debug.c (Telnet_socket and fnTELNETListener). The advantage to a application using a TELNET like socket rather than directly a buffered TCP socket is that the TELNET layer handles all detail - the application interface profits from the intermediate TELNET layer to do the "dirty work".

 

These are the TELNET specific routines:

extern USOCKET fnStartTelnet(unsigned short usTelnetPortNumber, unsigned short usIdleTimeout, unsigned short usMaxWindow, UTASK_TASK wakeOnAck, int (*listener)(USOCKET, unsigned char, unsigned char *, unsigned short) );

extern void fnStopTelnet(USOCKET TelnetSocket);

extern int  fnTelnet(USOCKET Telnet_socket, int iCommand);

extern int  fnCheckTelnetBinaryTx(USOCKET Socket);



The sockets in the uTasker project are always 'non-blocking' sockets (it is not possible to really compare them with sockets used with Windows etc.). Reception frames result in call back events and transmitted frames are either successful (an ACK call back event is received at some time in the future) or fail (a REGENERATE call back event is received at some time in the future – note that the REGENERATE event also results when a frame couldn’t be sent until ARP was resolved). These events will be handled by a TELNET layer if a TELNET like socket is used or else it is up to the application behind the TCP socket to handle them. Again the way this is performed is easy to see in the various protocols. If something is not clear, simply run the demo project in the simulator and set a break point in any code which is of interest – the code will break when it is used and the rest is usually self-explanatory.




I expect that various points here will be elaborated in further posts!

Regards

Mark

neil:
Hi Mark,
  In the above call back function 'static int fnTestListener(...)' , would it be best to leave as it is in most applications ? Simply call a task when data comes in, and leave all the rest as it is?

 Also..
 'fnGetTCP_Socket(TOS_MINIMISE_DELAY, (unsigned short)10, fnTestListener); '

Our application will have a permanent TCP connection to our server, and sends a byte of information to the server every 2 minutes, so is it feesable to have a timeout of 130 seconds?

I also assume that for creating a client connection application, it will be the same as your server description, but instead of listning, simply make a connection, then all else is the same?

Many Thanks
Neil

mark:
Hi Neil

The framework of the call back is basically the same in every use. In some cases unused cases can be grouped to a default handling, although the compiler probably optimises to this any way. Data can be either handled in the TCP_EVENT_DATA case directly or else call further subroutines for readability.
Since the data is passed via pointer to the callback (it is still in the LAN input buffer) it should either be fully processed or copied to a permenent buffer if the processing were to be delayed for later. As soon as the call back returns the LAN input buffer will be unblocked so that the LAN controller can re-use it.
Note also that handling the data in another task has no real benefits since it is already being handled in the Ethernet task and so is not blocking interrupts. The LAN Rx interrupt simply schedules the LAN task which is doing the work 'off-line'. It has checked IP and determined that it is a valid TCP frame and the TCP module is a subroutine to the Ethernet task. TCP does have its own task but this is only used for monitoring connection timers and retransmissions in case of timeouts or ARP resolution. The TCP task doesn't handle rx data.


If you set fnGetTCP_Socket(TOS_MINIMISE_DELAY, 130, fnTestListener) the socket will have an idle time out of 130s and so will do what you require. As soon as your 'Keep-Alive' message stops retriggering the connection it will timeout and close.
The range of the idle timer is 1...0xfffe (1s to 65534s). 0 should not be used and 0xffff means NO IDLE TIMEOUT or infinite.

A client doesn't need to listen and so after actively starting a connection the rest is basically the same, so your assumption is correct. Also a listener can establish a client type connection by actively establishing it itself (when no connection already exists on the socket), so the only real difference is that it is not possible to establish a TCP connection with a client socket from a remote peer since there is no listener on the port.

Note that several listeners (servers) are possible on a single port (each needs a seperate socket) but only one client is possible. A client will generally get a "random" free port number for each connection - the function which does this is called:
extern unsigned short fnGetFreeTCP_Port(void);
and it is handled by TCP itself when a connection is to be established.

If you have multiple servers on the port they can share the call back by adding a loops as follows. This example if from the HTTP code where each HTTP socket has a structure which saves the socket index of each connection. Since the socket belonging to the TCP event is passed as a parameter it is then quite easy to search which session the connection belongs to.


--- Code: ---static int fnHTTPListener(USOCKET Socket, unsigned char ucEvent, unsigned char *ucIp_Data, unsigned short usPortLen)
    HTTP *HTTP_session = ptrHTTP;
    int iSessionNumber;

    for (iSessionNumber = 0; iSessionNumber < NO_OF_HTTP_SESSIONS; iSessionNumber++) {
        if (HTTP_session->OwnerTCPSocket == Socket) {                    // search for the session to handle this event
            switch (ucEvent) {
            case TCP_EVENT_CONREQ:                                       // session requested
                return APP_ACCEPT;                                            // accept connection request
...etc.
            }
        }
        HTTP_session++;
    }
    return APP_REJECT;                                                        // owner socket not found!
}
--- End code ---


Regards

Mark

neil:
Hi Mark,
In the previous reply you mentioned:

>>Note that several listeners (servers) are possible on a single port (each needs a seperate socket) >>but only one client is possible.

Does this mean its not possible to have 2 or more clients connecting to a server?

Neil

Navigation

[0] Message Index

[#] Next page