µTasker Forum

µTasker Forum => µTasker general => Topic started by: mark on July 20, 2007, 10:18:51 PM

Title: Using TCP in the uTasker project
Post by: mark on July 20, 2007, 10:18:51 PM
Hi All

The uTasker demo project uses TCP for several services. These include HTTP, SMTP, FTP, TIME SERVER and TELNET, to name the ones which initially spring to mind.

The demo project enables these services to be easily studied and own services to be written (usually termed clients and servers) based on these example. There is, at the time of writing, no 'official' document which contains a definitive guide.

This is logically the reason why the use of TCP is a frequent question and frequent discussion topic in private mails. It is also a reason why this document, which is long overdue, should soon be made available.

Therefore this thread is designed to be an introduction to the subject to answer the first burning questions and to give first practical examples, as well as more detailed insight into how it works, how it can be used and why it all works like that. If all goes well, the thread comntent can be moved to a more structured document at a later point in time.

TCP is in fact a very simple protocol - in its simpest form. But in order to allow it to perform with the speed and reliability which has enabled it to become originally the standard in the Internet and eventually the standard in almost all networks it does need a few additional frills which can then make it quite complicated indeed.

If you would like to discuss standard TCP details please feel free to create your own topic to get a discussion going. I have always found it very difficult to find any really meaningful discussions about the practical aspects in the Internet so it would be nice to be able to find some good ones here!


A simple TCP server example. (The HTTP service is a good source for reference).

First we get a socket from the TCP socket pool. The size of the socket pool is defined in config.h, where the define USER_TCP_SOCKETS can be incremented for every additional socket which user code (as opposed to standard services like HTTP) will need.
The following configures a socket with the defined characteristic (minimum delay as opposed to maximum throughput or maximum reliability - these are router parameters), a timeout of 10 seconds (after this length of time with no activity it will close the connection. A value 0xffff means no timeout!), and a callback routine called fnTestListener() which will be doing the work later.

Code: [Select]
static USOCKET Test_socket = -1;                                          // declare a static socket variable
...
    Test_socket = fnGetTCP_Socket(TOS_MINIMISE_DELAY, (unsigned short)10, fnTestListener); // from task or sub-routine code

The socket is not in the listening state (which is needed for it to act as a server) so we start it in this state by calling
Code: [Select]
static unsigned short usTestPort = 0x9999;                               // declare a static socket variable (the value could also be fixed)
....
    fnTCP_Listen(test_socket, usTestPort, 0);                             // after getting the socket, bind it to a listening port

Now the socket is listening on port 0x9999, which means if you have a test client (eg. Telnet on this port) send a connection request to it it will start a connection process.
Note that the parameter 0 when setting the listening state is the maximum windows size which the connection can receive - a zero defaults to the standard LAN TCP pay load size of 1460 bytes. For some applications with special buffer restrictions this can be used to improve flow control in the rx sense (CONTROL_WINDOW_SIZE needs to be enabled in config.h) but this won't be described here just yet.

The simple server listener looks something like this.

Code: [Select]
#define TEST_BUFFER_LENGTH 10
typedef struct stTCP_MESSAGE
{
    TCP_HEADER     tTCP_Header;     // reserve header space
    unsigned char   ucTCP_Message[TEST_BUFFER_LENGTH];
} TCP_MESSAGE;

static int fnTestListener(USOCKET Socket, unsigned char ucEvent, unsigned char *ucIp_Data, unsigned short usPortLen) {
   TCP_MESSAGE test_message;

    switch (ucEvent) {
    case TCP_EVENT_CONREQ:                                             
    case TCP_EVENT_CONNECTED:
    case TCP_EVENT_CLOSE:
    case TCP_EVENT_ACK:
    case TCP_EVENT_ARP_RESOLUTION_FAILED:
    case TCP_EVENT_PARTIAL_ACK:
        break;
    case TCP_EVENT_REGENERATE:
    case TCP_EVENT_DATA:
        if ((ucEvent == TCP_EVENT_REGENERATE) || (!uMemcmp((CHAR*)ucIp_Data, "TEST" , 4))) {
            uStrcpy((CHAR*)test_message.ucTCP_Message, "Hi!!");
            if (fnSendTCP(Socket, (unsigned char *)&test_message.tTCP_Header, 4, TCP_FLAG_PUSH) > 0) {
                return APP_SENT_DATA;
            }
        }
        break;
    case TCP_EVENT_ABORT:
    case TCP_EVENT_CLOSED:
        fnTCP_Listen(Socket, ++usTestPort, 0);                    // go back to listening state on next port number
        break;
    }
    return APP_ACCEPT;
}

As you can see it receives events from the TCP layer, some with other details such as a pointer to data.

This is a list of the events which can arrive:


Now if you are new to TCP, enter in the above test program - best in the simulator - and you can see your first simple TCP server in operation. It won't do a lot but it will at least do the following.

1. When the client requests a connection to port 0x9999 the event TCP_EVENT_CONREQ will be received. Here it would be possible to deny the connection to a black-listed IP for example but our server always allows connections.
2. The TCP connection will be established and the event TCP_EVENT_CONNECTED is received.
3. If the client sends any data it will arrive with the event TCP_EVENT_DATA. The server will only respond to the message "TEST" in which case it will send the string "Hi!!" as answer.
4. After 10s with no activity, the server will close the connection (it is actually TCP that does this after the idle timer fires but the server receives the event TCP_EVENT_CLOSED when the connection has successfully closed).
5. The lister socket is not active after the connection has been closed so the code sets it up as listener again, this time with port number 0x999a.
6. If the connection is repeated, the port number is always incremented each time and the connection always times out after 10s.

Note that when the server sends data it returns APP_SENT_DATA. This informs TCP that the application has just successfully sent something and TCP doesn't have to acknowledge the received data - the ack has already been piggy-backed with the transmitted data.

If the transmission were to fail and need repeating, the server has to handle the event TCP_EVENT_REGENERATE. Since we only ever send one message it is obviouse that this has failed and so it is simply resent. The event TCP_EVENT_ACK will be received when the transmitted message actually has arrived safely.
It is in fact a great shame that messages can and do get lost (but it is not a perfect world and so we have to be ready for it) because regeneration of previously sent messages is probably the most complicated and/or RAM consuming parts of TCP implementations. This example has it real easy, but this is also a real simple example!

Here is a list of all possible return codes from the listener.


Our example has made use of very few features up to now and there is much more to be explained. However this should allow first steps to be taken and also the servers in the project (HTTP, FTP etc.) to already make a lot more sense.

I will continue the topic later with a first simple client example. If you already have confusions or questions, or corrections etc. please feel free to comment.

Regards

Mark
Title: Re: Using TCP in the uTasker project
Post by: mark on July 20, 2007, 11:22:34 PM
Hi

Here is some general information about TCP use in the uTasker project. It is an overview rather than a tutorial and is possibly a little vague in some respects. I hope that it doesn't confuse more that it helps, but it may well contain something of interest so here it is:


Here are the main user calls for working with TCP (see tcpip.h). You will find that the uTasker project is not based on Berkley conventions – it was designed as an alternative solution in small footprint projects where a mix between performance, reliability, simplicity and user comfort was defined, based on typical embedded project use. Therefore it doesn’t not try to copy any existing solution but goes its own way.


extern USOCKET fnTCP_Listen(USOCKET TCP_socket, unsigned short usPort, unsigned short usMaxWindow);

extern unsigned short fnGetFreeTCP_Port(void);

extern USOCKET fnGetTCP_Socket(unsigned char ucTos, unsigned short usIdleTimeout, int (*listener)(USOCKET, unsigned char, unsigned char *, unsigned short) );

extern USOCKET fnReleaseTCP_Socket(USOCKET TCPSocket);

extern USOCKET fnTCP_Connect(USOCKET TCP_socket, unsigned char *RemoteIP, unsigned short usRemotePort, unsigned short usOurPort, unsigned short usMaxWindow );

extern USOCKET fnTCP_close(USOCKET TCP_Socket);

extern signed short fnSendTCP(USOCKET TCP_socket, unsigned char *ptrBuf, unsigned short usDataLen, unsigned char ucTempFlags);

extern QUEUE_TRANSFER fnSendBufTCP(USOCKET TCP_socket, unsigned char *ptrBuf, unsigned short usDataLen, unsigned char ucCommand);

extern void fnModifyTCPWindow(USOCKET Socket, unsigned short usSafetyWindow);

extern void fnReportWindow(USOCKET Socket, unsigned short usBufferSpace);

 

The TCP task handles ARP so ARP is fully transparent to the user. However there are three types of TCP sockets which can be used.

The first is a RAW socket where the user supplied call back handles all events (see for example the http call back) which means that also repetitions must be handled there.

The second is a buffered socket. TELNET is an example of a protocol which makes use of this. The TCP socket has an inbuild data buffer for backing up data. It can send data of 'random' nature  rather than 'regeneratable' data at higher rates because it uses windowing and the buffered for retransmissions if necessary, as well as grouping single data transissions into larger transmission packets.

The third is a TELNET like socket where a TELNET socket is used instead of a TCP buffered socket – the TELNET socket can be in RAW mode so that it is otherwise transparent and doesn’t do negotiation or byte stuffing etc. The advantage of the TELNET like socket is that the TELNET layer provides an intermediate handler which handles repetitions and also TCP windowing so that high through put can be easily achieved. The down side is that it requires some buffer space (it works with fnSendBufTCP() rather than fnSendTCP()) to allow back ups of transmitted frames where the user simply writes to the socket as if it were any data port like a serial interface – the TELNET layer gets rid of the buffered data in the most efficient manor possible and ensures reliable delivery. There is an example of this in the demo project in debug.c (Telnet_socket and fnTELNETListener). The advantage to a application using a TELNET like socket rather than directly a buffered TCP socket is that the TELNET layer handles all detail - the application interface profits from the intermediate TELNET layer to do the "dirty work".

 

These are the TELNET specific routines:

extern USOCKET fnStartTelnet(unsigned short usTelnetPortNumber, unsigned short usIdleTimeout, unsigned short usMaxWindow, UTASK_TASK wakeOnAck, int (*listener)(USOCKET, unsigned char, unsigned char *, unsigned short) );

extern void fnStopTelnet(USOCKET TelnetSocket);

extern int  fnTelnet(USOCKET Telnet_socket, int iCommand);

extern int  fnCheckTelnetBinaryTx(USOCKET Socket);



The sockets in the uTasker project are always 'non-blocking' sockets (it is not possible to really compare them with sockets used with Windows etc.). Reception frames result in call back events and transmitted frames are either successful (an ACK call back event is received at some time in the future) or fail (a REGENERATE call back event is received at some time in the future – note that the REGENERATE event also results when a frame couldn’t be sent until ARP was resolved). These events will be handled by a TELNET layer if a TELNET like socket is used or else it is up to the application behind the TCP socket to handle them. Again the way this is performed is easy to see in the various protocols. If something is not clear, simply run the demo project in the simulator and set a break point in any code which is of interest – the code will break when it is used and the rest is usually self-explanatory.




I expect that various points here will be elaborated in further posts!

Regards

Mark

Title: Re: Using TCP in the uTasker project
Post by: neil on July 21, 2007, 09:21:42 AM
Hi Mark,
  In the above call back function 'static int fnTestListener(...)' , would it be best to leave as it is in most applications ? Simply call a task when data comes in, and leave all the rest as it is?

 Also..
 'fnGetTCP_Socket(TOS_MINIMISE_DELAY, (unsigned short)10, fnTestListener); '

Our application will have a permanent TCP connection to our server, and sends a byte of information to the server every 2 minutes, so is it feesable to have a timeout of 130 seconds?

I also assume that for creating a client connection application, it will be the same as your server description, but instead of listning, simply make a connection, then all else is the same?

Many Thanks
Neil
Title: Re: Using TCP in the uTasker project
Post by: mark on July 21, 2007, 01:21:44 PM
Hi Neil

The framework of the call back is basically the same in every use. In some cases unused cases can be grouped to a default handling, although the compiler probably optimises to this any way. Data can be either handled in the TCP_EVENT_DATA case directly or else call further subroutines for readability.
Since the data is passed via pointer to the callback (it is still in the LAN input buffer) it should either be fully processed or copied to a permenent buffer if the processing were to be delayed for later. As soon as the call back returns the LAN input buffer will be unblocked so that the LAN controller can re-use it.
Note also that handling the data in another task has no real benefits since it is already being handled in the Ethernet task and so is not blocking interrupts. The LAN Rx interrupt simply schedules the LAN task which is doing the work 'off-line'. It has checked IP and determined that it is a valid TCP frame and the TCP module is a subroutine to the Ethernet task. TCP does have its own task but this is only used for monitoring connection timers and retransmissions in case of timeouts or ARP resolution. The TCP task doesn't handle rx data.


If you set fnGetTCP_Socket(TOS_MINIMISE_DELAY, 130, fnTestListener) the socket will have an idle time out of 130s and so will do what you require. As soon as your 'Keep-Alive' message stops retriggering the connection it will timeout and close.
The range of the idle timer is 1...0xfffe (1s to 65534s). 0 should not be used and 0xffff means NO IDLE TIMEOUT or infinite.

A client doesn't need to listen and so after actively starting a connection the rest is basically the same, so your assumption is correct. Also a listener can establish a client type connection by actively establishing it itself (when no connection already exists on the socket), so the only real difference is that it is not possible to establish a TCP connection with a client socket from a remote peer since there is no listener on the port.

Note that several listeners (servers) are possible on a single port (each needs a seperate socket) but only one client is possible. A client will generally get a "random" free port number for each connection - the function which does this is called:
extern unsigned short fnGetFreeTCP_Port(void);
and it is handled by TCP itself when a connection is to be established.

If you have multiple servers on the port they can share the call back by adding a loops as follows. This example if from the HTTP code where each HTTP socket has a structure which saves the socket index of each connection. Since the socket belonging to the TCP event is passed as a parameter it is then quite easy to search which session the connection belongs to.

Code: [Select]
static int fnHTTPListener(USOCKET Socket, unsigned char ucEvent, unsigned char *ucIp_Data, unsigned short usPortLen)
    HTTP *HTTP_session = ptrHTTP;
    int iSessionNumber;

    for (iSessionNumber = 0; iSessionNumber < NO_OF_HTTP_SESSIONS; iSessionNumber++) {
        if (HTTP_session->OwnerTCPSocket == Socket) {                    // search for the session to handle this event
            switch (ucEvent) {
            case TCP_EVENT_CONREQ:                                       // session requested
                return APP_ACCEPT;                                            // accept connection request
...etc.
            }
        }
        HTTP_session++;
    }
    return APP_REJECT;                                                        // owner socket not found!
}


Regards

Mark
Title: Re: Using TCP in the uTasker project
Post by: neil on July 21, 2007, 02:19:40 PM
Hi Mark,
In the previous reply you mentioned:

>>Note that several listeners (servers) are possible on a single port (each needs a seperate socket) >>but only one client is possible.

Does this mean its not possible to have 2 or more clients connecting to a server?

Neil
Title: Re: Using TCP in the uTasker project
Post by: neil on July 21, 2007, 02:49:44 PM
Hi Mark,
 I quickly tried to add a client, and have it connecting to my server application, but without any luck.  Could you let me know what I am doing wrong? Here is the small code.. (It just simply connects, no sends/receives)

USOCKET Test_socket;

static int fnTestSocket(USOCKET Socket, unsigned char ucEvent, unsigned char *ucIp_Data, unsigned short usPortLen)
{

    switch (ucEvent)
   {
      case TCP_EVENT_CONREQ:                                             
      case TCP_EVENT_CONNECTED:
      case TCP_EVENT_CLOSE:
      case TCP_EVENT_ACK:
      case TCP_EVENT_ARP_RESOLUTION_FAILED:
      case TCP_EVENT_PARTIAL_ACK:
         break;
      case TCP_EVENT_REGENERATE:
      case TCP_EVENT_DATA:
         if ((ucEvent == TCP_EVENT_REGENERATE) || (!uMemcmp((CHAR*)ucIp_Data, "TEST" , 4)))
         {
         }            }
         break;
      case TCP_EVENT_ABORT:
      case TCP_EVENT_CLOSED:
         break;
   }
    return APP_ACCEPT;
}


Within the 'fnApplication(..)', making it the last bit of code within 'if (!iAppState)':

Test_socket = fnGetTCP_Socket(TOS_MINIMISE_DELAY, (unsigned short)20, fnTestSocket);
fnTCP_Connect(Test_socket, "192.168.0.184", 10000, 0, 0); //my server address and port...

The connection is never made (all done within simulator), and the call back function always gets called with switch value of '9'.

I have my server application running in debug, so I can tell when a connection is made.
Regards
Neil
Title: Re: Using TCP in the uTasker project
Post by: mark on July 21, 2007, 04:31:35 PM
Neil

Quote
Does this mean its not possible to have 2 or more clients connecting to a server?

What I meant was that it is no two clients from the same IP address may have the same port number when connected to the same server port number. This is also the same restriction for the clients at the other end of the connection when we are acting as server.
So there is no problem with having multiple clients connected to a server.

I should be more precise. This is not an implementation restriction but a simple TCP connection rule:
A TCP connection is determined by two parameters; IP addresses at the two ends and Port numbers at the two ends.
For example Device A is at 192.168.0.3 and device B is at 192.168.0.4 in a local network
Device A and B have servers running on Port 80 (HTTP port).

Now Device A and Device B both have clients which connect to the server.
Let's say there are 2 connections from each. The clients will use a 'random' port number when connecting for the first (the uTasker uses a dynamic port range between 49152 and 65535) and each subsequent connection then uses the next port number.

Therefore the following 4 connection may exist, which are defined by the unique connection pairs (IP and Port):
Connection 1 (Client at A connected to server at B)
    192.168.0.3:50672 <-> 192.168.0.4:80
Connection 2 (Client at A connected to server at B)
    192.168.0.3:50673 <-> 192.168.0.4:80
Connection 3 (Client at B connected to server at A)
    192.168.0.3:80 <-> 192.168.0.4:52819
Connection 4 (Client at B connected to server at A)
    192.168.0.3:80 <-> 192.168.0.4:52820

As you can see. The server uses the same port but there is always a different client port number. Therefore each TCP connection is unique since the port pairs are never the same.

Regards

Mark

Title: Re: Using TCP in the uTasker project
Post by: mark on July 21, 2007, 04:36:56 PM
Neil

Code: [Select]
fnTCP_Connect(Test_socket, "192.168.0.184", 10000, 0, 0); //my server address and port...
The first problem that I see is in the use of the IP address.
The uTasker project doesn't store IP addresses as strings but rather as arrays.

Therefore you need to spefify your IP address like this :

Code: [Select]
unsigned char ucMyIp[IPV4_LENGTH] = {192,168,0,198};
fnTCP_Connect(Test_socket, ucMyIp, 10000, 0, 0); //my server address and port...

This may then work.
What the previous version probably did was send the connection request to 31.39.32.31 which would then be directed to your default gateway and sent out onto the internet.
If you are receiving TCP_EVENT_ARP_RESOLUTION_FAILED (this will occur after about 4..5 seconds) it would imply that you don't have a local gateway.

Note that you can also activate "USE_TIME_SERVER" in config.h to see how it program operates. It is a client example and there are a number of timer servers which it attempts to contact until one if successful. The time server will send one message (with the time - surprise surprise) and close the connection.

On thing to note is a difference here between when working with the simulator and working on the target. When the client tries to connect in the initialisation of application.c the simulator can send immediately. On the target it generally takes a few seconds for the Ethernet link to become ready and so it takes a little longer before it all works.

Another point to note is that a client connection attempt often causes an ARP resolve to be initialted. This is because the address of the destination is not yet in the local ARP table. A server will invariably have the IP address already in the table since it was refreshed there as it received the connection attempt (SYN) from the remote client. For the user this is in fact totally transparent since TCP handles it itself when connecting. It is however possible that during an active connection the ARP entry times out (due to non activity or a network with high activity causing it to be flushed). In this case the ARP resolve will be automatically started but the listener will need to be able to resend the last message via the TCP_EVENT_REGENERATE.

fnSendTCP() will return the number of bytes transmitted. If it has to start an ARP resolution it will however return NO_ARP_ENTRY. In this case the user can also prepare for the regeneration event which is expected. Should instead TCP_EVENT_ARP_RESOLUTION_FAILED be received, it means that the connection partner is no longer reachable - it has maybe turned off in the meantime and so the connection could be terminated.

Regards

Mark




Title: Re: Using TCP in the uTasker project
Post by: neil on July 21, 2007, 05:09:12 PM
Hi Mark,
  The above worked fine, thanks, I can now connect to the server application.  I will soon start looking at sending/receiving side.

Many Thanks
Neil
Title: Re: Using TCP in the uTasker project
Post by: neil on July 22, 2007, 01:08:13 PM
Hi mark,
  After succesfully connecting to my server application (Windows based), I thought I would send some data. I want to send the following block of data, which my server application understands as a command:

After a connection is made, within the call back function I insert the code to send to my server, as follows:

unsigned char Buff[10];
//call back function
static int fnTestSocket(USOCKET Socket, unsigned char ucEvent, unsigned char *ucIp_Data, unsigned short usPortLen)
{
   TCP_MESSAGE test_message;

    switch (ucEvent)
   {
         case TCP_EVENT_CONNECTED:
                Buff[0]=0;
                Buff[1]=5;
                Buff[2]=233;
                 Buff[3]=5;
                Buff[4]=32;
                          if (fnSendTCP(Socket, &Buff[0], 5, TCP_FLAG_PUSH) > 0)
                   return APP_SENT_DATA;
                         break;

                    ....
             }


The above gets sent, but when I debug my server application, the above values are not received, there are 5 characters sent, but not the above.

Can you point me to where I am going wrong?

Regards
Neil
Title: Re: Using TCP in the uTasker project
Post by: mark on July 23, 2007, 02:17:13 PM
Hi Neil

Take a look at the first example. It uses:

if (fnSendTCP(Socket, (unsigned char *)&test_message->tTCP_Header, 4, TCP_FLAG_PUSH) > 0)

when sending.

Note that the buffer should also supply space for the TCP layer to add its header. This is so that TCP doesnt have to copy data contents to merge it with its header and is a rather specific solution - however once you have the buffer set up accordingly it should be easy to use.

Your copy is effectively giving  a pointer to your test message which TCP will overwrite with the header and it will then send 5 extra bytes beyond the header length, which will be a bit random since this is beyond the end of your buffer.

If you use a structure as in the example and build your test message up in the data part of the buffer it should then send the content that you are expecting.

Good luck

Regards

Mark
Title: Re: Using TCP in the uTasker project
Post by: neil on July 23, 2007, 03:06:16 PM
Hi Mark,
  Thanks for the reply.
It worked fine as you mentioned, I just never realised that the header had to be included, just thought need to provide a buffer,but worked fine.

It strange, as I tried it , and all worked fine, I then closed my server and simulator. Then re-ran them the same way. The simulator tried to connect, but a connections didnt make a connection. Ive tried it several times, but no luck. Ive managed 2 connections from about 10 attempts. If I run my windows client application we use, we can do the exactly the same thing I am trying with the simulator, and everytime it connects tot he server.

Regards
Neil
Title: Re: Using TCP in the uTasker project
Post by: neil on July 23, 2007, 04:08:16 PM
Hi mark,
  Regarding the previous problem of not being able to connect.

With the problem above, not connecting, I was trying it on the sumilator. I now copied the code into the 52233 flash. And it worked fine, without modifying any code.

Within the call back routine above 'static int fnTestListener(USOCKET Socket, unsigned char ucEvent, unsigned char *ucIp_Data, unsigned short usPortLen)' is 'ucIp_Data' a pointer to the data read in, and 'usPortLen', The number of characters?

Regards
Neil
Title: Re: Using TCP in the uTasker project
Post by: mark on July 24, 2007, 09:46:39 PM
Neil

I cant explain why it is not working every time with the simulator. If you send me an Ethereal recording I may be able to see what is happening.

ucIp_Data is a pointer to either the data in the input buffer or the IP address of the connection partner (in the same format as all local IP addresses). usPortLen is either a port number or a length, according to the event type.

Here is a complete list:

TCP_EVENT_ARP_RESOLUTION_FAILED. ucIp_Data is a pointer to the remote IP address. usPortLen is the port number.
TCP_EVENT_CONREQ. ucIp_Data is a pointer to the remote IP address. usPortLen is the port number.
TCP_EVENT_CONNECTED. ucIp_Data is a pointer to the remote IP address. usPortLen is the port number.
TCP_EVENT_ACK. ucIp_Data is a pointer to the remote IP address. usPortLen is the port number.
TCP_EVENT_CLOSE. ucIp_Data is a pointer to the remote IP address. usPortLen is the port number.
TCP_EVENT_ABORT. ucIp_Data is a pointer to the remote IP address. usPortLen is the port number.
TCP_EVENT_CLOSED. ucIp_Data is a pointer to the remote IP address. usPortLen is the port number. Exception: if REUSE_TIME_WAIT_SOCKETS option is enabled it is possible that a connection in the TCP_STATE_FIN_WAIT_2 state will be prematurely terminated rather than rejecting a connection due to no sockets available in the TCP socket pool. In this case the port number is 0.

TCP_EVENT_REGENERATE. ucIp_Data is not used and is 0. usPortLen is a length - how many bytes need regenerating.

TCP_EVENT_PARTIAL_ACK. ucIp_Data is a pointer to the remote IP address. usPortLen is a length - the number of bytes not yet acked.

TCP_EVENT_DATA. ucIp_Data is a pointer to the received data content. usPortLen is a length - the number of bytes of data.

Regards

Mark
Title: Re: Using TCP in the uTasker project
Post by: mark on July 24, 2007, 09:47:11 PM
Here is a simple TCP client example:

Earlier I gave a simple example of creating a TCP server socket which could be connected to from a remote client. In that case it was necessary to set the socket to listening state so that it could respond to connection attempts.

In the case of the client it is not necessary to set the socket to any special state, however it is also possible to set the socket to listening state but subsequently use it to connect as a client to a remote server. This means that any socket can be used as a client socket - but it can only establish a client connection if it has not already a connection from a remote client. (Put another way, a socket can only handle one connection; either the connection is established as a server from the listening state or as a client from any state).

The following code wil thus create a socket for client use and then attempt to establish a connection to a remote server.

USOCKET test_client_socket = fnGetTCP_Socket(TOS_MINIMISE_DELAY, TCP_DEFAULT_TIMEOUT, fnTestListener);

As in the case of the server, there must be a free socket available and the user must ensure that there are enough sockets for a particular application in the socket pool. USER_TCP_SOCKETS defines the number of sockets reserved for use by the application.

If no socket is available for use, the routine fnGetTCP_Socket() will return a negative error code.

static unsigned char ucRemoteIP[IPV4_LENGTH] = {192,168,0,101};
#define TEST_TCP_PORT 0x1234
if (test_client_socket >= 0) {
    fnTCP_Connect(test_client_socket, ucRemoteIP, TEST_TCP_PORT, 0, 0);
}

This call only attempts to establish a connection when the socket is valid. It causes TCP to send a connection request (SYN) to the IP address 192.168.0.101 on port 0x1234. Note that all IP addresses in the uTasker project are defined as arrays and not as strings which is another common method ("192.168.0.101" is therefore not a valid IP address definition).
In the example, the last 2 parameters of the fnTCP_Connect() call are 0. The first can be used to define the local port number but typically a client uses a more or less random port number when connecting and so a 0 informs TCP that it should define the port using its standard algorithm. The last parameter allows a maximum TCP window size to be specified. This will not be discussed further here since it will be used in a later topic. A zero informs TCP to use the standard rx window size, which is equivalent to the maximum TCP payload of an Ethernet frame (1460 bytes).


The client connection attempt may cause the connection frame to be sent out onto a local LAN (if the IP address belongs to the local subnet) or else to the default gateway address if it is outside of the local subnet. In either case, if the destination IP address does not yet exist in the ARP table this will first be resolved by the ARP protocol.

The way that ARP operates is transparent to the user when connecting. Either the connection is sent (either immediately or after ARP has resolved the IP address) or it can not be sent due to the fact that ARP - after several attempts, and a delay of about 5s - still can not resolve. The the callback is un this case called with event TCP_EVENT_ARP_RESOLUTION_FAILED.

Assuming that the connection is successful, TCP will call the listener with the event TCP_EVENT_CONNECTED.

Once connected, the rules for the connection are the same. Whether client-server or server-client the connection is effectively peer-peer.
As with the listener case, either party can close the TCP connection.


Note that the Time Server demo in the uTasker demo project shows how a client connects to a remote server, which automatically sends back the time before immediately closing the TCP connection. To activate this, set the define USE_TIME_SERVER in config.h.




Regards

Mark
Title: Re: Using TCP in the uTasker project
Post by: mark on July 24, 2007, 09:47:36 PM
Here is a discussion about sending TCP data using the simplest method. Later I will discuss using buffered TCP to achieve greater bulk data transfer rates as well as TELNET sockets to do the same without need for the application to handle the details.


The simplest method of sending data is by using the following code (based on fnSendTCP())

#define MAX_DATA_LEN 100
static unsigned char usTestData[MIN_TCP_HLEN + MAX_DATA_LEN];
usData[MIN_TCP_HLEN + 0] = 0x01;
usData[MIN_TCP_HLEN + 1] = 0x02;
usData[MIN_TCP_HLEN + 2] = 0x03;
usData[MIN_TCP_HLEN + 3] = 0x04;
fnSendTCP(Test_TCP_socket, ucTestData, 4, TCP_FLAG_PUSH);

Note here that the data buffer passed to TCP must contain a header space which TCP will then fill for TCP header information. In this example 4 user data bytes are added at the correct location in the buffer and then sent with the TCP_FLAG_PUSH set. This flag indicates to the destination that all TCP data has been sent and it should process it now. If the data is sent as several TCP frames this fag is usually only set in the last one - however practically there is no great diffence in most applications.

Another method of achieving the same thing is with the following code which is equivalent but uses a structure - this is often preferrable since it may be a bit easier to read.

#define MAX_DATA_LEN 100
typedef struct stTCP_MESSAGE
{
    TCP_HEADER     tTCP_Header;                    // reserve header space
    unsigned char  ucTCP_Message[MAX_DATA_LEN];    // data payload space
} TCP_MESSAGE;

static TCP_MESSAGE TestData;
TestData.ucTCP_Message[0] = 0x01;
TestData.ucTCP_Message[1] = 0x02;
TestData.ucTCP_Message[2] = 0x03;
TestData.ucTCP_Message[3] = 0x04;
fnSendTCP(Test_TCP_socket, (unsigned char *)&TestData, 4, TCP_FLAG_PUSH);

To check whether the data could be sent, the following check can be added.

if (fnSendTCP(Test_TCP_socket, (unsigned char *)&TestData, 4, TCP_FLAG_PUSH) > 0) {
    // successful
}

The call will usually only fail when the destination IP address (which is not given here since it is a parameter of the connected socket) is not resolved and so ARP has first to be started. This can happen when the TCP connection has been idle for some time and the IP entry flushed from the ARP table. It is therefore not an error but a natural occurance.

A successfully sent message can also fail to arrive at the destination due to a transmission error for example - it may arrive too but the ack could get lost instead. In this case the transmission should be retried after TCP has detected the timeout - or after ARP has resolved the destination IP address. Both result in the callback event TCP_EVENT_REGENERATE.

This means that the previous message much be resent. In the example case it can be rebuilt using the same subroutine and once TCP has determined that it has successfully reached its destination, the callback event TCP_EVENT_ACK will be received.

If however the data can not be simply regenerated as in our example - for example it arrived from a serial interface - a copy must be kept until the TCP_EVENT_ACK has finally arrived. The code must be prepared to repeat the message (by copying it from a backup to the TCPbuffer) as many times as necessary before it can destroy the backup (probably by overwriting it with the next data). The need for keeping a backup and repeating data is usually the most complicated part of using TCP...

One last but important point about using fnSendTCP() is its use from within the callback function. When data has been successfully sent, it should be signalled by returning APP_SENT_DATA. This informs TCP that data has been sent and it then does not have to generate an ACK to any received frame. The ACK will already have been piggy-backed with the data. It is not a serious error if this is not respected, instead it simply means that TCP will generate unnecessary ACK frames which cause a little more data on the LAN than really necessary.

This technique is simple, reliable and adequate for many TCP connections. However if you require large amounts of bulk data to be transferred you will probably have a surprise since the throughput is not as high as initially expected. This is due to the fact that the next block of data can only be sent when there has been an acknowledgement to the previous. In a LAN the round trip time is very short but in the Internet this may rise to hundreds of ms or even several seconds. This will clearly reduce throughput since there will be dead times.
However in a LAN you will often find that the receiver will be using a technique called "delayed acks" where rather than immediately sending an ACK to received TCP frames there is a delay of 150ms..200ms inserted. This is in the hope that a receiving application will be sending a frame back and the TCP layer can piggy back its ACK on it - and also save unnecessary short ACK frames. The result is that usually only about 5 TCP frames can be transferred in a second. This gives a data throughput (using 1460 byte payloads) of only 7300 bytes a second (or 58400 bits a second). This is no faster than most serial interfaces.

However the simple technique results also in absolute minimum complication and code size so there is no reason why it is not used if the bulk speed is not the issue. Many TCP applications also allow delayed acks to be disabled, which then compensates most speed loss in a LAN.

In the next part in the series about using TCP in the uTasker project I will introduce buffered TCP to show how TCP windowing can be activated to greatly improve bulk data throughtput, although often at the cost of more RAM requirement...


Regards

Mark

Title: Re: Using TCP in the uTasker project
Post by: neil on July 25, 2007, 02:28:37 PM
Hi Mark,
  Nice article above.

Regarding 'TCP_EVENT_REGENERATE'. You mention that the previous message must be resent. Is there a chance that there can be an endless event, where the call back keeps getting 'TCP_EVENT_REGENERATE' everytime the message is resent (for some weird reason)? If so I assume that its best to retry 'x' amount of times before closing the connection?

I have written a lot in sockets on Windows applications, and never had to worry about the lo-level 'ACKs' etc as these are all hidden. So please excuse my basic  questions  :)

I understand that if I simply receive a 'TCP_EVENT_REGENERATE', I do a resend. And if I get a 'APP_SENT_DATA', the data was transmitted/received okay.
I thought the only messages I would really have to manage are 'TCP_EVENT_REGENERATE' , 'APP_SENT_DATA', 'TCP_EVENT_CONNECTED'. But after I resend if a  'TCP_EVENT_REGENERATE'  is returned, I will get a ''TCP_EVENT_ACK' instead of a  'APP_SENT_DATA' . Is this correct?

But if the data cannot be resent, if I receive a  'TCP_EVENT_REGENERATE', I simply wait for a ''TCP_EVENT_ACK' before doing anything else with the socket (apart from closing it)?

Is that the basics, or am I missing something else  ???
Thanks
Neil
 

Title: Re: Using TCP in the uTasker project
Post by: mark on July 25, 2007, 08:56:16 PM
The following details how to use buffered TCP to transmit TCP frames.


Buffered TCP support can be activated by setting the define USE_BUFFERED_TCP in config.h.
This then allows the use of the following routine to transmit frames:

QUEUE_TRANSFER fnSendBufTCP(USOCKET TCP_socket, unsigned char *ptrBuf, unsigned short usDataLen, unsigned char ucCommand)

This technique is used for example by the TELNET module, which serves as a good example of its use.

The noticable difference to the the fnSendTCP() call is the ucCommand parameter instead of the Flag parameter. The following show how it can be used.

CHAR test_message[] = "Hello World ";
QUEUE_TRANSFER nr_of_bytes = (sizeof(test_message) - 1);
fnSendBufTCP(Test_socket, (unsigned char *)test_message, nr_of_bytes, (TCP_BUF_SEND | TCP_BUF_SEND_REPORT_COPY));
fnSendBufTCP(Test_socket, (unsigned char *)test_message, nr_of_bytes, (TCP_BUF_SEND | TCP_BUF_SEND_REPORT_COPY));
fnSendBufTCP(Test_socket, (unsigned char *)test_message, nr_of_bytes, (TCP_BUF_SEND | TCP_BUF_SEND_REPORT_COPY));

The next noticable difference is the fact that 3 test messages are sent immediately after another, whereas previous fnSendTCP() examples have always sent only one message and waited until it had been acked. The command flag TCP_BUF_SEND informs the buffered TCP socket that the data should be transmitted as soon as possible. The flag TCP_BUF_SEND_REPORT_COPY causes the function to return the number of bytes of the test message which were actually accepted for transmission.

The first time that a buffered socket is used, a structure TCP_TX_BUFFER is automatically created which contains various control variables and also a data buffer of size TCP_BUFFER.
The size of this buffer can be defined in app_hw_xxxx.h as in the following:
#define TCP_BUFFER            2800
This allocates buffer space of 2800 bytes. When fnSendBufTCP() is called, the data to transmit is first copied into this buffer. As many bytes are copied as possible and if the buffer becomes full, the trailing ones will be ignored.
If the user needs to ensure than no data can be lost due to such a buffer overflow, the following will first verify whether there is enough buffer space to accept the message:

UTASK_TASK OurTask = OWN_TASK;
if (fnSendBufTCP(Test_socket, (unsigned char *)&OurTask, nr_of_bytes, TCP_BUF_CHECK) > 0) {
    fnSendBufTCP(Test_socket, (unsigned char *)test_message, nr_of_bytes, TCP_BUF_SEND);
}
The value returned will be equal to the remaining buffer space BEFORE adding the data. If the available space is less that the message length it will return 0. In addition to returning 0, the owner task will be entered to be woken by an TX_FREE interrupt event when the buffer becomes free again. This allows tasks to be suspended on waiting for buffer space to become freed.

Note that the define WAKE_BLOCKED_TCP_BUF should be set in config.h to enable this option!

As soon as the first TCP frame has been copied to the transmitter buffer a TCP frame is also sent. Generally there will be no ACK received from this frame for at least 150ms (assuming active delayed ack operation at the destination). The second call to fnSendBufTCP(), which follows immediately in the example, will cause the second part of the data to be added to the buffer. As long as the congestion windows (this will be explained later) allows it, a second TCP frame will immediately be sent, although there has not yet been an acknowledge to the first one. The same is also true for the third call of fnSendBufTCP() although the congestion window value will generally not allow it to actually send the associated data just yet.

At this point in time two TCP frames are under way to the destination and three blocks of data are in the tcpbuffer. The fact that the data is available as a backup in the tcp buffer means that the application itself doesn't need to also have a copy of the data, or be able to regenerate it if a frame needs to be repeated. In the case of a repetition - signalled by TCP_EVENT_REGENERATE, the socket's callback routine can simply call
fnSendBufTCP(Test_socket, 0, 0, TCP_BUF_REP);
which will start the repetition of lost data.
NOTE: See the further example at the end of this note for the recommended method of repetating to ensure that windowing operation remains efficient.

One important fact to be noted about buffered TCP is that the data copied to the output buffer is treated as a stream of data and not as individual frames of the same lengths as entered. This means that it is very suited to debug interfaces and the likes since the user can send the data in small packets (such as "text + value + text", as often used when constructing an output) and the data will tend to be sent as frames of maximum size rather than small TCP frames.
This behaviour would be observed if the example frames were not to be acked:
1. Frame 1 "Hello World "
2. Frame 2 "Hello World "
3. No ack to either of the 2 frames so TCP_EVENT_REGENERATE received by the callback routine.
4. On use of fnSendBufTCP(Test_socket, 0, 0, TCP_BUF_REP); the next TCP frame will have the content "Hello World Hello World Hello World " since the frame is constructed from as many waiting bytes as possible (the first 2 non-acked frame contents plus the third waiting one).

The maximum size of individual framnes can also be defined in app_hw_xxxx.h
    #define TCP_BUFFER_FRAME      1400

When using buffered TCP, the fact that more than one tx frame is sent before receiving an ACK means that TCP windowing is in operation. One great advantage of TCP windowing is the fact that bulk data throughput is much greater than when using the simple TCP transmission method described earlier. Not only do round trip delays impact less the transmission efficiency but the fact that the destination immedately receives two TCP frames effectively stops any outstanding Delayed ACK timer it has in operation and thus causes an ACK to be immediately returned to the second TCP frame. By using just 2 outspanding frames in a LAN can already increase data throughput greatly.

The defined SUPPORT_PEER_WINDOW should also be used when operating with buffered TCP because this support the processing of partial ACKs by TCP. Partical ACKs are are acknowledgements to TCP data which do not include all outstanding frames and these must be handled in the callback in the case for TCP_EVENT_PARTIAL_ACK.

This handling is in fact very simple since it only has to do the following:
fnSendBufTCP(Socket, 0, usPortLen, TCP_BUF_NEXT);

In the case of TCP_EVENT_ACK (which means that all outstanding data have been acked), the corresponding call is fnSendBufTCP(Socket, 0, 0, TCP_BUF_NEXT);

In both cases further waiting data (again as much as possible) will be sent in following TCP frames. The variable usPortLen, rather than 0, informs the buffered TCP routine exactly how much of the outstanding TCP data has successfully been acked. The TCP buffer is effectively freed of acked data so that the buffer has more space to accept further data from the user.

Buffered TCP operation allows improved data throughput at the expense of RAM buffer space. The routines suppport a congestion window which allows a number of TCP frames to be sent before their ACKs have been received. Initially a connection allows 2 frames to be outstanding but this value is allowed to increase by one between each ack up to the maximum defined by WINDOWING_BUFFERS in config.h. This ensures that the number of outstanding TCP acks builds up to mathh the return trip time of a connection and so achieves optimum bulk data throughput.

A difficults is generally in deciding the amount of SRAM to allocate to the TCP buffer since in small footprint projects such RAM represently quite a luxury. In any case there is also the question as to what happens if the user data rate is still faster than the TCP bulk data throughput rate, in which case even the largest buffer will probably not stop it ever filling up completely. The use of the call fnSendBufTCP() with flag TCP_BUF_CHECK has already been introduced and this can be used to stop temporarily the source of data until the TCP buffer size has increased again. The uTasker demo project uses quite a small TELNET buffer (which is also using buffered TCP) to illustrate how, for example, large menus are displayed with limited buffer space. If you study this example code, where the copying of the source text to the TCPbuffer is suspended until space becomes available again, you will probably get enough ideas to solve your own such data bottle-necks.

One popular use of such a TCP connection is for the transfer of data from a UART to a remote PC via TCP. flow control is required throughout the link and a typical technique would be to issue an XOFF or remove CTS (depending on UART flow control in operation) when the TCP buffer is getting critically full) and removing it when the TX_FREE event is received.
Here is a full list of the flags available when using fnSendBufTCP(). Some of the flags can be used together - their combined use should be logical.

    #define TCP_BUF_SEND               put the attached data into the buffer and send asap
    #define TCP_BUF_NEXT               an ack has been received so send as much more waiting data as possible from the TCP buffer.
    #define TCP_BUF_REP                data has been lost. Repeat as much more waiting data as possible from the TCP buffer.
    #define TCP_BUF_CHECK              check whether the specified amount of data will fit into the TCP buffer. If it won't fit, send a TX_FREE event when the buffer is free.
    #define TCP_BUF_SEND_REPORT_COPY   report the amound of data copied to teh TCP buffer (used together with TCP_BUF_SEND).
    #define TCP_CONTENT_NEGOTIATION    used by TELNET together with TCP_BUF_SEND to flag that data is a negotiation frame and so must not be combined with other data
    #define TCP_REPEAT_WINDOW          used only internally to TCP to handle specific repetitions
    #define TCP_BUF_KICK_NEXT          This can be used kick in windowing operation after data loss. The next example illustrates the recommended use of it in response to the TCP_EVENT_REGENERATE event.

case TCP_EVENT_REGENERATE:
    if (fnSendBufTCP(Socket, 0, 0, TCP_BUF_REP) != 0) {  // repeat send buffered
        fnSendBufTCP(Socket, 0, 0, (TCP_BUF_NEXT | TCP_BUF_KICK_NEXT)); // kick off any following data as long as windowing allows it
            return APP_SENT_DATA;
        }
        break;


The buffered TCP support allows quite simple use of the technique to achieve high speed transmission in most applications. However the operation of TCP is still rather more complicated than has been described here and the description of TCP itself is far from complete. Flow control from one end of a connection (possibly including more than the TCP stage in practical systems) is not as simple as one initially believes. But the uTasker should support all that is necessary to achieve respectable results even in small footprint solution (a uTasker powered Serial<->TCP converter based on the ATSAM7X128 - 48MHz ARM7 with 32k SRAM - has in fact outperformed an ARM9 solution with 16Meg SDRAM - based on my own measurements...). But this description should serve as a good starting point and the forum can be used to elaborate on the things which I have unintentionally or intentionally left out for the moment.

Comments are very welcome.

The final part of this series will discuss TELNET, which is based on the buffered TCP technique. It will be shown how a TELNET-like socket can be used rather than a direct buffered TCP socket to acheive the same improvements but with a more simple interface - using the existing TELNET handling routines as a interface engine.

Before closing it is worth mentioning one more point. It is also possible, in some situations, to still achieve high data throughput based on TCP windowing without using buffered TCP. This is feasible when the data is not of random nature (like data which is being transferred from a UART) but can be regenerated on demand. A web server is a good example since the files which it is serving (also when partly dynamically generated) can usually be regenerated without having to keep a large data store of previous transmissions.

A new version of HTTP server for the uTasker project is being developed (tests have already proved very successful with page serving rates of larger http files or graphics in a LAN of up to 25x the rate when using simple TCP transmission) which should be available with the next service pack!!

Regards

Mark
Title: Re: Using TCP in the uTasker project
Post by: mark on July 25, 2007, 09:39:09 PM
Hi Neil

TCP_EVENT_REGENERATE is sent when ARP has resolved an IP address. This will only ever happen once.
It is also called when a TCP message transmission times out without an ACK. The socket counts the timeouts and automatically limits the repeats to TCP_DEF_RETRIES (this is fixed at 7 in tcpip.h). If maximum repeats still fails it is obvious that the connection has failed and so it will be closed by using a reset.

Yes, when using Windows sockets there are a lot of details which are hidden. However there is also a lot of software involved and this obviously has to be handled somewhere. I have now also added a description of using buffered TCP where ACKs do still have to be handled. The last in the series will detail TELNET and the use of a TELNET-like socket. In this case it inserts another layer between the application and the buffered TCP socket which does in fact offload the 'dirty-work' from the user layer. Then the user never has anything to do with the regeneration event. It makes it easier to use and, if you do already have Telnet active in your device, the software is all there so it doesn't add to code size. Most events are handled by the TELNET layer but some are also passed on via a second 'user' callback so that some details can still be controlled there as well if needed.

Regards

Mark
Title: Re: Using TCP in the uTasker project
Post by: mark on July 26, 2007, 08:38:48 PM
The following details how Telnet works and how to use a TELNET-like socket for general TCP socket implementation.

When TELNET is activated - by setting the define USE_TELNET in config.h - buffered TCP is also enabled since the TELNET implemmentation is based on this method. The uTasker demo project controls the TELNET socket in the file debug.c where is is used to allow a menu-driven command interface to be controlled via a TELNET connection. When no Telnet connection is active the same menu-driven interface is available via a serial interface (activated by the define SERIAL_INTERFACE in config.h). This shows that the actual use of the TELNET connection is very similar to a typical serial terminal connection and it is thus very useful for configuration and general debugging purposes.

Before beginning with the details of Telnet it should be mentioned that the uTasker project includes several optimised routines which help display data, some of which work with a serial interface. The most common is the following function:

extern QUEUE_TRANSFER fnDebugMsg (CHAR *ucToSend); // send string to debug interface

Using this call, the standard "Hello World!" welcome can be send:

    fnDebugMsg("Hello World!");

The question is to which interface this message is actually sent? The answer is that it depends on the setting of a global variable:
    QUEUE_HANDLE DebugHandle = NETWORK_HANDLE;

This can be set by the user to any serial interface handle - for example the uTasker demo project sets it to a serial port if a serial port is enabled in the project:
    DebugHandle = SerialPortID;

If nothing is changed, it is destined to NETWORK_HANDLE which means that the message will in fact be passed on to a routine called fnNetworkTx():

extern QUEUE_TRANSFER fnNetworkTx(unsigned char *output_buffer, QUEUE_TRANSFER nr_of_bytes);

This routine must be defined somewhere in the project and it can also be a dummy function if it is not actually used. In the uTasker demo project it can be found in debug.c where it transfers data to the Telnet socket, if this is enabled:

return (fnSendBufTCP(Telnet_socket, output_buffer, nr_of_bytes, (TCP_BUF_SEND | TCP_BUF_SEND_REPORT_COPY)));

Here we see that it is using the buffered TCP interface and the socket is defined as the Telnet_socket, which we will discuss below.
Note however that DebugHandle can be changed by any code to switch between serial interfaces or a network interface, it doesn't have to be left at the default NETWORK_HANDLE value. The uTasker demo project does just this and so enables messages to be sent over the serial interface or over an active Telnet interface, depending on which is desired.

To open a Telnet socket use the function:
   Telnet_socket = fnStartTelnet(23, (5*60), 0, 0, fnTELNETListener); 

This is the prototype of the call:
extern USOCKET fnStartTelnet(unsigned short usTelnetPortNumber,
      unsigned short usIdleTimeout,
      unsigned short usMaxWindow,
      UTASK_TASK wakeOnAck,
      int (*listener)(USOCKET, unsigned char, unsigned char *, unsigned short) )

As can be seen, a Telnet socket is started (it will be set to listener state) on the standard Telnet port 23. It will have an inactivity timeout period of 5 minutes and uses the callback function fnTELNETListener(). For the moment we will leave the other parameters at 0, which means that they will take default values.

Note that the Telnet socket can later be stopped by simply calling fnStopTelnet(Telnet_socket);

The Telnet callback function (in its simplest form) is in fact quite similar to a standard TCP callback function:

static int fnTELNETListener(USOCKET Socket, unsigned char ucEvent, unsigned char *ucIp_Data, unsigned short usPortLen)
{
    switch (ucEvent) {
    case TCP_EVENT_CONREQ:                                     // session request
    case TCP_EVENT_ABORT:
    case TCP_EVENT_CLOSE:
    case TCP_EVENT_CLOSED:
    case TCP_EVENT_ACK:
        break;

    case TCP_EVENT_CONNECTED:                                  // TCP connection
        return fnTelnet(Telnet_socket, GOTO_ECHO_MODE);        // Set echo mode

    case TCP_EVENT_DATA:
        if (fnCommandInput(ucIp_Data, usPortLen, SOURCE_NETWORK)) { // collect data input and try to interpret if a LF is found
            return APP_SENT_DATA;                // we will always have sent an answer
        }
        break;
    }
    return APP_ACCEPT;                           // default is to accept a connection
}

Here it has been simplified in comparison to the use in debug.c since it uses the events to switch debug output between the serial port and Telnet and also supports a user name / password login sequence.

When data is received, it is passed on to the menu handling routine, which will not be described here but quite simply matches a list of available command to the received data and performs the command if found. When a TCP connection has been established - event TCP_EVENT_CONNECTED is received - a further Telnet command is used to switch the Telnet layer to echo mode. In this mode each received frame or, in the case of a remote user typing data in, each received character will be echoed back.

It can thus be seen that by opening a Telnet socket we have in fact opened a buffered TCP socket which has a few more capabilities that a standard buffered TCP socket. There is no handling of TCP_EVENT_REGENERATE since the TELNET layer is doing most of the dirty work for us, which also makes its use very simple. It is however a Telnet socket, which means that it has some Telnet specific function. It is best to look a little closer at the function fnTelnet() which can be used together with a Telnet socket.

extern int fnTelnet(USOCKET Telnet_socket, int iCommand);

The following command are possible:
GOTO_ECHO_MODE                 switches echo on
LEAVE_ECHO_MODE                switches echo off
PASSWORD_ENTRY                 flags that text entry is a password and so * is echoed back
CLEAR_TEXT_ENTRY               leave password entry mode
TELNET_ASCII_MODE              sets ASCII mode (this is the default mode of the Telnet socket)
TELNET_RAW_MODE                sets RAW mode - this means that the Telnet socket will not perform any Telnet-specific things
TELNET_RAW_RX_IAC_ON           enables IAC searching although in RAW mode
TELNET_RAW_RX_IAC_OFF          disables IAC searching in RAW mode
TELNET_RAW_TX_IAC_ON           enables IAC byte stuffing although in RAW mode
#define TELNET_RAW_TX_IAC_OFF  disables IAC byte stuffing in RAW mode
#define TELNET_RESET_MODE      reset mode to default (ASCII mode)

The first thing to note is that by calling fnTelnet(Telnet_socket, TELNET_RAW_MODE); you have now a Telnet-like socket which is not doing any Telnet specific stuff. This is then a buffered TCP socket which is doing the dirty work without trying to interpret any Telnet negotiations. This is thus a handy method of working with one or more buffered TCP sockets with minimum effort at the application layer!

If a Telnet-like socket is required to connect as a client to a remote server, it could use the following sequence, based on the standard fnTCP_Connect() command:

Telnet_socket = fnStartTelnet(usPortNumber, (5*60), 0, 0, fnTELNETListener);
if (Telnet_socket >= 0) {
    fnTelnet(Telnet_socket, TELNET_RAW_MODE);               // set RAW mode
    fnTCP_Connect(Telnet_socket, ucIP, usPortNumber, 0, 0); // connect to remote server
}



It has to be said that Telnet is rather complicated due to the various things that it can do. The uTasker Telnet implementation just does some basic but useful things to enable a reliable Telnet connection in ASCII or Binary mode (or Telnet-like connection in RAW mode). Since it supports TCP Windowing it is fast, but don't forget that the price penalty is in the RAM required for the TCP buffer(s).

Since a typical application for Telnet is to allow simple remote debugging, it supports negotiation of an ECHO mode. In this mode all received characters are echoes back to the remote client.

The default Telnet mode is ASCII mode. In this mode all received data is being scanned for negotiation sequences. These negotiation commands begin always with the IAC character (0xff) which doesn't occur in the ASCII data content.
However when transmitting binary data over a Telnet connection (with possibility of 0xff appearing in the data stream) byte stuffing is used. When 0xff is detecting in the tx data stream it is sent as a sequence of 2 x 0xff. The receiver performs the inverse so that it is still possible to detect real command sequences within binary data transfers.
This allows switching between ASCII and Binary modes, as well as other Telnet commands, during data transfers.


Depending on which programs are being used to transfer data, different types of operation are often observed. By setting up the various combinations of modes using fnTelnet() it is always possible to define the specific combination required to successfully transfer any type of data.

Some special notes on the operation of buffered TCP relevant to Telnet:

1. When negotiation sequences are sent, they are marked as such so that they will always be sent (including regeneration) as individual commands. This ensures that they will never be converted to binary data due to byte stuffing, nor be grouped together with other commands or data when being repeated. The buffered TCP transmission call uses the flag TCP_CONTENT_NEGOTIATION to acheive this.

2. When sending data which needs to be byte-stuffed there is a risk that the data content can, in the most extreme case, double. i.e. when 100 x 0xff are to be sent, they will be sent as 200 x 0xff because each byte will be stuffed to produce 2!
This means that the TCP buffer would have to be dimensioned to twice the normal required size to ensure that all data could be backed up in the most extreme case. The solution in the uTasker project is however not to back up the stuffed data (which would infact be the simplest method, but with the disadvantage of doubled buffer sizes) but instead to back up only the raw data and mark it appropriately.
In the case of TCP regeneration (hidded in the Telnet part) retransmitted frames are always regenerated from a backup of the raw data. This means a little more work (and it is rather more complicated...) when regenerating, but optimum buffer space efficiency as a result. Since the efficiency of regenerating lost messages is unimportant (they have already suffered a long timeout delay) the space saving has much higher importance.


I hope that the practical use of Telnet and Telnet-like sockets will now be very simple. They are easy to use and enable high speed data transfer to be acheived. The configuration possibilites enable fast data transfer between any combination of programs.

Not all details of TCP operation have been mentioned here but the descripton should be adequate for first implementations. There are some additional complications concerning TCP flow control across connections which may or may not be noticable in specific projects. The uTasker implementation does however allow all known difficulties to be solved and I look forward to comments and feedback so that additional points can be elaborated in real-world contexts.

Best regards

Mark
Title: Re: Using TCP in the uTasker project
Post by: eduardovra on January 24, 2008, 07:31:31 PM
Hi Mark!

I working on a project that uTasker will have to work as a protocol converter. A PC will connect (TCP connection) to the board running uTasker and sends some datagrams, then the microcontroler converts them into ethernet raw packets and send to an ASIC chipset that we have. The problem is that some times the TCP datagram sent by the PC application is bigger than the MSS of TCP protocol. In this case, the datagram is divided and sent into two separate packages.
How can I know if the datadram came fragmented and how can I remount them ?

Thanks in advance !
Regards

Eduardo Vieira
Title: Re: Using TCP in the uTasker project
Post by: mark on January 24, 2008, 08:53:03 PM
Hi Eduardo

TCP data can arrive at the TCP receiver call-back function as a sequence of several frames. The order in which the frames arrive is always the same as they were sent, but there is no information as to whether the content is a single data packet or a fragment of a larger data packet.

The application which controls the data content can add its own extra information (such as a small header which contains the total length of data which will arrive belonging to a single unity). In this case the receiver call-back handler can know that the data packet has not yet completed (if the present frame length < reported total length). It also knows how much more data is expected before it is complete. I this case it will need to save the data to a local buffer until it is complete - warning: once the call back routine is quit the input data, which is still in the Ethernet reception buffer, will be destroyed (overwritten by subsequent Ethernet receptions). Once the complete data has been received (the buffer used to save it must be adequately large for the largest expected datagram) it can be passed for handling.

Another possible solution (without having to add a header) is if the TCP stack at the PC is using the TCP PUSH flag correctly. The push flag is used to indicate that the present frame is the last in a sequence - it is used to push data to the application, whereas frames received without the PUSH flag set indicate that they first have to be buffered until complete. On the PC the PUSH flag can possibly be controlled by the program sending the data (but you will need to consult the details of the stack used there). It may work automatically but it may not...

The details of the present TCP header is presently not available in the TCP call back routine but this could be easily changed to make the PUSH flag visible (or to pass a different event depending on PUSH flag active or not) if necessary.

Personally I would add a small header to the data content since this will be fully controlable - the PUSH flag operation may change between PC stack implementations and versions so I wouldn't like to have to rely on it.

Regards

Mark





Title: Re: Using TCP in the uTasker project
Post by: seulater on March 29, 2008, 02:47:29 PM
can you expound on the static int fnTestListener.

it it set up as a task of its own ? i would imagine that in your example for this it is set up as a task since it is not in a while loop.




Title: Re: Using TCP in the uTasker project
Post by: mark on March 29, 2008, 03:09:23 PM
Hi

fnTestListener() is a call-back function and so doen't have any association with tasks in itself.

A call-back function is a routine defined when the socket was opened, which will be called by TCP whenever it has received data or has to report on events (such as connection open or close). It is called directly from the TCP code with a pointer to any data and length of the data.

This means that the call-back is not being scheduled to start as a result of TCP, but instead being called as a sub-routine from TCP. The Ethernet input buffer is thus guarantied to be stable during its processing by the call-back routine. The data is (depending on processor and Ethernet controller details) directly in the Ethernet input buffer ("zero copy" technique as sometimes known as).

Regards

Mark
Title: Re: Using TCP in the uTasker project
Post by: seulater on March 29, 2008, 03:18:21 PM
so if things are slowly sinking in, i could create to listening sockets by doing...


USOCKET Test_socket_1 = fnGetTCP_Socket(TOS_MINIMISE_DELAY, (unsigned short)10, fnTestListener_1);

USOCKET Test_socket_2 = fnGetTCP_Socket(TOS_MINIMISE_DELAY, (unsigned short)10, fnTestListener_2);


static unsigned short usTestPort_1 = 10000;
fnTCP_Listen(test_socket_1, usTestPort_1, 0);

static unsigned short usTestPort_2 = 20000;
fnTCP_Listen(test_socket_2, usTestPort_2, 0);





Title: Re: Using TCP in the uTasker project
Post by: mark on March 29, 2008, 04:55:31 PM
Yes, this is correct.

Three notes:

1. Make sure that you have enough sockets in the TCP socket pool to allow as many as you would like to use.
USER_TCP_SOCKETS in config.h specifies how many (non-standard) user TCP sockets should be reserved. The "standard" ones, like HTTP etc., are reserved automatically.

2. Look at the HTTP listerer to see how to handle multiple users on a single port.

3. Try using the uTasker simulator when experimenting with TCP stuff because it is much faster and easier to practice and learn. Using the Ethereal/Wireshark playback support helps playing with TCP frames because connections will not break down when debugging. [All is described, with example, in the tutorial].

Regards

Mark
Title: Re: Using TCP in the uTasker project
Post by: seulater on March 29, 2008, 11:41:36 PM
Thanks, i will try it on the real thing, debugging is out of the question for me. i dont trust them
Title: Re: Using TCP in the uTasker project
Post by: mark on March 30, 2008, 12:09:20 AM
I would still give it one try for TCP/Network work - I assure you, you can save well over 50% development time. It is very accurate and I know commercial projects which have used only the simulator for all web interface development and testing. On the target there was no difference in behaviour (sometimes after several days of new development additions) but the project schedule could easily be halved over traditional target level debugging.

Personally I can't live without it - I would be otherwise totally overwhelmed by the support questions. As it is I can run the project with any processor anywhere I am (as long as I have my laptop handy) and even play back complete network transitions from a remote installation (form a Wireshark recording) and usually immediately explain what the problem is or verify a solution.

The tutorial takes about 20 minutes to work through and you may find that you will in fact like what you see!!

Regards

Mark

Title: Re: Using TCP in the uTasker project
Post by: seulater on March 30, 2008, 12:13:03 AM
ok, i will give it a whirl. i dont think the Ethernet will be cause of many problems as using new external peripheral. I will take your advise and give it a try.
So the PC debugger even is able to match the boards cpu speed in simulating as well ?
Title: Re: Using TCP in the uTasker project
Post by: mark on March 30, 2008, 12:38:18 AM
Hi

The simulator will not try to run at the CPU speed but it will check the PLL settings and display the speed that it is programmed to (usefully for checking). You will find that if you play around with the PLL settings (MUL and DIV etc.) and do anything illegal it will give compiler errors (not necessarily on the Luminary because this is more or less automated).

[I wrote the next after misreading your question as to be related to the COM port speeds. I won't delete it though because it may be of interest...]
The UART simulator will automatically adjust the COM settings to match the UART settings (map the UARTs to COMs in app_hw_xxxx.h - eg: #define SERIAL_PORT_0   '3' maps UART to COM3, etc.)

The simulator will also change the COM setting 'on-the'fly' if you change the UART configuration via the serial menu (either via the UART/COM or via Telnet).

In the Coldfire project it will also accurately control the RTS/CTS lines for HW flow control work and can work with break generation and detection. Also UART DMA mode is simulated to work with the COM port. Presently the LPC/Luminary projects (and their simulators) will do neither hardware flow control nor break detection (useful for terminating frames when receiving in DMA mode) - this is not due to the lack of support in the simulator but rather in the present UART driver for these chips.

Regards

Mark

PS. The idea of the simulator is not to simulate the instruction set of the CPU but rather to allow applications to run in 'real-time' using all project code, including interrupt routines. The goal was to allow embedded projects to be developed on a PC even before chips were available and allow network debugging to be performed in 'real-time' and also in off-line mode.

Title: Re: Using TCP in the uTasker project
Post by: mhoneywill on February 24, 2009, 12:36:13 AM
Mark,

I've just used the demo code at the head of this post to test how to handle TCP and I found a number of typo's, I'm mentioning them here so you can amend the code listing so others don't have the same problems I did. The errors are as follows

1. The Code below has an extra }

    case TCP_EVENT_CLOSED:
        fnTCP_Listen(Socket, ++usTestPort, 0);                    // go back to listening state on next port number
        break;
        }
 
should be
   
    case TCP_EVENT_CLOSED:
        fnTCP_Listen(Socket, ++usTestPort, 0);                    // go back to listening state on next port number
        break;


2. In the following code test_message->tTCP_Header should be test_message.tTCP_Header
   
    case TCP_EVENT_DATA:
        if ((ucEvent == TCP_EVENT_REGENERATE) || (!uMemcmp((CHAR*)ucIp_Data, "TEST" , 4))) {
            uStrcpy((CHAR*)test_message.ucTCP_Message, "Hi!!");
            if (fnSendTCP(Socket, (unsigned char *)&test_message->tTCP_Header, 4, TCP_FLAG_PUSH) > 0) {
                return APP_SENT_DATA;
            }
        }

    should be

    case TCP_EVENT_DATA:
        if ((ucEvent == TCP_EVENT_REGENERATE) || (!uMemcmp((CHAR*)ucIp_Data, "TEST" , 4))) {
            uStrcpy((CHAR*)test_message.ucTCP_Message, "Hi!!");
            if (fnSendTCP(Socket, (unsigned char *)&test_message.tTCP_Header, 4, TCP_FLAG_PUSH) > 0) {
                return APP_SENT_DATA;
            }
        }

3. In the start-up code you suggest using
    USOCKET Test_socket = fnGetTCP_Socket(TOS_MINIMISE_DELAY, (unsigned short)10, fnTestListener);
    This should be
    USOCKET Test_socket = fnGetTCP_socket(TOS_MINIMISE_DELAY, (unsigned short)10, fnTestListener);
    Note the lowercase s in fnGetTCP_socket

Looking at tcpip.h it seems the case of the function words after a _ symbol is a bit random, for example

extern USOCKET fnGetTCP_Socket(unsigned char ucTos, unsigned short usIdleTimeout, int (*listener)(USOCKET, unsigned char, unsigned char *, unsigned short) );
extern USOCKET fnGetUDP_socket(unsigned char ucTOS, int (*fnListener)(USOCKET, unsigned char, unsigned char *, unsigned short, unsigned char *, unsigned short), unsigned char ucOpts);
extern USOCKET fnReleaseUDP_socket(USOCKET SocketHandle);
extern USOCKET fnReleaseTCP_Socket(USOCKET TCPSocket);

I hope the above helps

Cheers

Martin

Title: Re: Using TCP in the uTasker project
Post by: mark on February 24, 2009, 11:46:58 AM
Hi Martin

Thanks, I have corrected these.

I didn't make the third change since fnGetTCP_Socket() with capital matches with the header file and the routine in tcp.c.

It is a shame about the mixture between use for capital and small - I won't change anything just yet because it may lead to problems. (Perhaps a #define fnGetTCP_socket() fnGetTCP_Socket() would make it case insensitive...?)

Regards

Mark
Title: Any plans to have AUTOIP ?
Post by: syakovlev on March 26, 2009, 09:21:17 PM
So far search for AUTOIP returns 0 results.
Title: Re: Using TCP in the uTasker project
Post by: mark on March 26, 2009, 09:55:15 PM
Hi Sergei

I don't know much about AUTO-IP but I did think that the SW to do this was running on a PC in the network together with SNMP capabilities of managed switches.

As to what the embedded device needs to do, I think it just needs to request its IP address using a BOOTP reguest.

Do you have a specific requirement?

Regards

Mark
Title: Re: Using TCP in the uTasker project
Post by: syakovlev on March 26, 2009, 10:29:02 PM
Hi Mark,

A plan is to assign "Reasonable" default IP address to an instrument before shipment.
We were thinking about using private address in the range 169.254.x.x.
In this case user will be able to connect to the instrument before getting static IP address from his sysadmin.
And it looks like AUTOIP support is needed in case <"Reasonable" default IP address> will happen to be in
conflict with some other instrument which is also using address in range 169.254.x.x.
Our instrument will need new address dynamicly "self-assigned" in range 169.254.x.x.

Best regards.
Sergei.
Title: Re: Using TCP in the uTasker project
Post by: aaronlawrence on April 30, 2010, 01:03:52 AM
Mark,
can you either "pin" this topic so it stays at the top, or link to it from the documentation page?
I came back to try and find it and could not, even knowing it was there in the general forum.
Thanks.
(unfortunately the code explorer still doesn't have some of this stuff)
Title: Re: Using TCP in the uTasker project
Post by: hossain2 on May 27, 2010, 09:53:56 AM
With the problem above, not connecting, I was trying it on the simulator. I now copied the code into the 52233 flash. And it worked fine, without modifying any code.
Within the call back routine above 'static int listener(SOCKET Socket, unsigned char prevent, unsigned char *Luci_Data, unsigned short unsporting)' is 'Luci_Data' a pointer to the data read in, and 'unsporting', The number of characters?
Title: Re: Using TCP in the uTasker project
Post by: hossain2 on May 27, 2010, 09:57:44 AM
On thing to note is a difference here between when working with the simulator and working on the target. When the client tries to connect in the initialization of application.c the simulator can send immediately. On the target it generally takes a few seconds for the Ethernet link to become ready and so it takes a little longer before it all works.
Title: Re: Using TCP in the uTasker project
Post by: hossain2 on May 27, 2010, 09:59:20 AM
This is logically the reason why the use of TCP is a frequent question and frequent discussion topic in private mails. It is also a reason why this document, which is long overdue, should soon be made available.
Title: Re: Using TCP in the uTasker project
Post by: mark on May 27, 2010, 11:49:19 AM
Hi

Concerning the listener parameters, these are described in the following document:
http://www.utasker.com/docs/CodeDoc/fnGetTCP_Socket.htm

All TCP functions are described formally in the code documentation: http://www.utasker.com/docs/Code.html -> see TCP/IP -> TCP.C

Regards

Mark

Title: Re: Using TCP in the uTasker project
Post by: hossain on May 29, 2010, 10:17:12 AM
OK, i think A successfully sent message can also fail to arrive at the destination due to a transmission error for example - it may arrive too but the ack could get lost instead. In this case the transmission should be retried after TCP has detected the timeout - or after ARP has resolved the destination IP address.
thanks
Title: Re: Using TCP in the uTasker project
Post by: aloft on June 27, 2010, 06:24:45 AM
Hi Mark,

I'm trying to implement a ftp client but I'm running into some problems.  After I send the USER <username> on the control socket, I'm getting a TCP_EVENT_PARTIAL_ACK back that doesn't seem to get handled properly.  From your thread, it sounds like this shouldn't happen. 

Any ideas what I'm doing wrong.

Thanks,

TW
Title: Re: Using TCP in the uTasker project
Post by: mark on June 27, 2010, 01:42:24 PM
Hi

Generally you shouldn't get a PARTIAL_ACK event when a simple TCP socket is used unless multiple frames are sent without first waiting for an ACK. This would be an error case since a simple TCP socket must wait for an ACK before sending the next frame.

However there does seem to be a case just after a connection when there is a repetition that a PARTIAL_ACK event can be received with a length of zero. Generally when partial ACKs are ignored  this doesn't disturb - otherwise by ignoring when the length is zero is a strategy (my development version no longer calls the event when the length is zero to avoid any potential confusion).

Is this partial ACK received consistently? Please post a WireShark recording since the reason for it may be visible there.

Regards

Mark
Title: Re: Using TCP in the uTasker project
Post by: aloft on June 30, 2010, 04:36:08 AM
Hi Mark,

My mistake on this.  It was apparent when looking at this with Wireshark that my breakpoint was causing the PARTIAL_ACK event. 

I see another problem where I'm not responding with an ACK - but I think I should be sending one unless I misunderstand the process.  I'm getting an ACK in my Listener routine after I send the USER <username>.  Since the TCP_EVENT_ACK just breaks out of the switch statement, I should return the APP_ACCEPT which should generate an ACK, correct?

I sent my screenshots and code as well as a wireshark trace in a separate email.

Thanks,

TW
Title: Re: Using TCP in the uTasker project
Post by: mark on June 30, 2010, 09:01:20 AM
Hi

I was not able to answer to your email addresses - something seems to be blocking it or my sender-address and returning the mails. I did send a personal message though with the following answer:



I don't see any TCP problems (all ACKs are visible in frames and there is no ACK to an ACK). All that I see is that you are not getting an answer from the server when you send "USER lbmwelldata2" but this is probably due to the fact that "\r\n" is missing at the end of the string so that the server knows that the input is complete. The server should then return a request 331 for the password.

Regards

Mark

Title: Re: Using TCP in the uTasker project
Post by: aloft on June 30, 2010, 09:23:11 PM
Thanks Mark. 

That helped - I'm now getting further.  I didn't realize the FTP server needed the "\r\n".

Thanks again,

TW
Title: Re: Using TCP in the uTasker project
Post by: mark on April 06, 2013, 12:31:31 AM
Hi All

I would like to note that when the application receives the TCP_EVENT_REGENERATE is should repeat a lost frame and return APP_SENT_DATA.
If APP_ACCEPT is returned it will cause the TCP layer to consider this as an application error and close the connection.

Regards

Mark
Title: Re: Using TCP in the uTasker project
Post by: k64konfused on October 10, 2014, 06:28:52 AM
Hi Mark, although this is related to my "newbie" thread, I wanted to post here since it's related to TCP.  I am not an experienced network programmer by any means.  Most of the code I have written on Windows to program sockets uses really high level .NET libraries and aren't ever used in production code.

In order to get myself familiarized with sockets on uTasker, I felt that a reasonable starting point would be to leverage the simulator and modify the fnMyFirstTask() function from the getting started guide.  Here's what I ended up with:

Code: [Select]
extern void fnMyFirstTask(TTASKTABLE *ptrTaskTable)
{
static unsigned char ucCounter = 0;

/*
fnDebugMsg("Hello, World!  Test number ");
fnDebugDec(ucCounter++, 0);
fnDebugMsg("\r\n");
*/

static unsigned char ucRemoteIP[IPV4_LENGTH] = {192,168,0,112};
unsigned char data[MIN_TCP_HLEN + 4];
//memset( data, 0, 1024);

USOCKET test_client_socket = fnGetTCP_Socket(TOS_MINIMISE_DELAY, TCP_DEFAULT_TIMEOUT, fnTestTcpServerConnect);
if (test_client_socket >= 0) {
USOCKET ret;
    if( (ret = fnTCP_Connect(test_client_socket, ucRemoteIP, 50000, 0, 0)) > 0) {
fnDebugMsg( "connected\r\n");
if( fnSendTCP(test_client_socket, data, 4, TCP_FLAG_PUSH) > 0) {
fnDebugMsg( "sent successfully\r\n");
} else {
fnDebugMsg( "send failed\r\n");
}
fnReleaseTCP_Socket( test_client_socket);
} else {
if( ret == SOCKET_NOT_FOUND)
fnDebugMsg( "Socket not found\r\n");
else if( ret == NO_FREE_PORTS_AVAILABLE)
fnDebugMsg( "No free ports available\r\n");
else if( ret == SOCKET_STATE_INVALID)
fnDebugMsg( "Socket state invalid\r\n");
}
} else {
if( test_client_socket == -2) {
fnDebugMsg( "No socket available\r\n");
}
}
}

I have the task set up to start after 10s and call fnMyFirstTask() every 10s.  This gives me enough time after starting the application to select the right NIC from the uTasker dialog.

I wrote a simple listener app in C# that listens on port 50000.  I know this app works because I have some code running on my FRDM-K64F that connects to the port and streams it data.  I can see the connection occur and bytes coming in.

When I run my application, via Telnet I can see that the socket gets created and connects successfully, and supposedly data is also sent successfully.  However, my app doesn't see the connection or byte stream.  On top of that, Wireshark reports some kind of error:

105964   4274.560943000   192.168.0.88   192.168.0.112   TCP   58   [TCP Out-Of-Order] 64416?50000 [SYN, PSH] Seq=0 Win=1460 Len=0

From reading the past posts in this thread, it sounds like it's possible that data could be sent, but not actually get received due to things like ACKs not being sent?  Anyhow, I put breakpoints in fnTestTcpServerConnect() and none of them get hit.  Actually, the function seems to not ever get called.

Can you provide me with any suggestions on debugging this further?  Can you also recommend any good reads for people that don't understand the nitty-gritty details of TCP, but want to learn?

Thank you!
Title: Re: Using TCP in the uTasker project
Post by: mark on October 10, 2014, 12:49:43 PM
Hi

>>This gives me enough time after starting the application to select the right NIC from the uTasker dialog

The selected NIC is stored when the simulator is closed normally. You should only ever have to set it once and then it should stay correct.

I think that the main problem that you have is that you are writing code as if the socket operation were blocking, but it is not blocking; you need to make use of the callback function so that the operation is state-event based.

Eg. After fnTCP_Connect(() returns with success it means that it has sent a SYN to the destination (or at least started ARP resolution so that this can be performed shortly) - it doesn't mean that there is a connection established, which is reported (after a short time - but could even be several seconds later if the distance involved is long and there needs to be a repetition due to lost data) by the event TCP_EVENT_CONNECTED at the callback.

Therefore you need to place the code that sends in the callback instead.

You will also get an event confirming that your data was successfully received and then you can close the connection.
Generally there is no advantage in releasing the socket if you want to use it again later. If you initialise it to -1 and only call fnGetTCP_Socket() when it is negatiev it will save calling it multiple times.

You can find examples towards the beginning of this thread and it is useful to put to some fnDebugMsg() in the callback so that you see the events taking place and then you will quickly get the idea of the operation. Note that you hae teh choice between simple sockets, buffered sockets and TELNET-like sockets. Start with the simple socket operation since it will show the complete workings and uses least resources. Then try buffered socket (allows greater transmit speed at the cost of buffer space) and finalyl a TELNET-like socket which is the simplest to use since it encapsulates TELNET code to handle most of the nitty-gritty.

TCP illustrated is the classic book (although a bit outdated now): https://www.goodreads.com/book/show/505560.The_Protocols


Regards

Mark
Title: Re: Using TCP in the uTasker project
Post by: k64konfused on October 10, 2014, 02:53:47 PM
Hi Mark, as usual, thank you for the comprehensive (and comprehensible!) explanation.  The reason I was doing that was a) misunderstanding, and b) I was trying to do it quick-and-dirty, so closing the connection each time from a periodic task was necessary.  I haven't yet looked into how to initialize peripherals / features.  Ideally, I would open a listening socket upon startup, and then have multiple remote sockets get created only once when a connection needs to be made to its respective server/device.  Then in the periodic task I would just send ModbusTCP commands over each connection, and only close the socket upon shutdown.

I will definitely proceed in the order that you have recommended.  Hopefully, I will become more well-versed in TCP details in the coming weeks.

Thank you!