µTasker Forum > µTasker general

Using TCP in the uTasker project

<< < (4/10) > >>

mark:
Here is a discussion about sending TCP data using the simplest method. Later I will discuss using buffered TCP to achieve greater bulk data transfer rates as well as TELNET sockets to do the same without need for the application to handle the details.


The simplest method of sending data is by using the following code (based on fnSendTCP())

#define MAX_DATA_LEN 100
static unsigned char usTestData[MIN_TCP_HLEN + MAX_DATA_LEN];
usData[MIN_TCP_HLEN + 0] = 0x01;
usData[MIN_TCP_HLEN + 1] = 0x02;
usData[MIN_TCP_HLEN + 2] = 0x03;
usData[MIN_TCP_HLEN + 3] = 0x04;
fnSendTCP(Test_TCP_socket, ucTestData, 4, TCP_FLAG_PUSH);

Note here that the data buffer passed to TCP must contain a header space which TCP will then fill for TCP header information. In this example 4 user data bytes are added at the correct location in the buffer and then sent with the TCP_FLAG_PUSH set. This flag indicates to the destination that all TCP data has been sent and it should process it now. If the data is sent as several TCP frames this fag is usually only set in the last one - however practically there is no great diffence in most applications.

Another method of achieving the same thing is with the following code which is equivalent but uses a structure - this is often preferrable since it may be a bit easier to read.

#define MAX_DATA_LEN 100
typedef struct stTCP_MESSAGE
{
    TCP_HEADER     tTCP_Header;                    // reserve header space
    unsigned char  ucTCP_Message[MAX_DATA_LEN];    // data payload space
} TCP_MESSAGE;

static TCP_MESSAGE TestData;
TestData.ucTCP_Message[0] = 0x01;
TestData.ucTCP_Message[1] = 0x02;
TestData.ucTCP_Message[2] = 0x03;
TestData.ucTCP_Message[3] = 0x04;
fnSendTCP(Test_TCP_socket, (unsigned char *)&TestData, 4, TCP_FLAG_PUSH);

To check whether the data could be sent, the following check can be added.

if (fnSendTCP(Test_TCP_socket, (unsigned char *)&TestData, 4, TCP_FLAG_PUSH) > 0) {
    // successful
}

The call will usually only fail when the destination IP address (which is not given here since it is a parameter of the connected socket) is not resolved and so ARP has first to be started. This can happen when the TCP connection has been idle for some time and the IP entry flushed from the ARP table. It is therefore not an error but a natural occurance.

A successfully sent message can also fail to arrive at the destination due to a transmission error for example - it may arrive too but the ack could get lost instead. In this case the transmission should be retried after TCP has detected the timeout - or after ARP has resolved the destination IP address. Both result in the callback event TCP_EVENT_REGENERATE.

This means that the previous message much be resent. In the example case it can be rebuilt using the same subroutine and once TCP has determined that it has successfully reached its destination, the callback event TCP_EVENT_ACK will be received.

If however the data can not be simply regenerated as in our example - for example it arrived from a serial interface - a copy must be kept until the TCP_EVENT_ACK has finally arrived. The code must be prepared to repeat the message (by copying it from a backup to the TCPbuffer) as many times as necessary before it can destroy the backup (probably by overwriting it with the next data). The need for keeping a backup and repeating data is usually the most complicated part of using TCP...

One last but important point about using fnSendTCP() is its use from within the callback function. When data has been successfully sent, it should be signalled by returning APP_SENT_DATA. This informs TCP that data has been sent and it then does not have to generate an ACK to any received frame. The ACK will already have been piggy-backed with the data. It is not a serious error if this is not respected, instead it simply means that TCP will generate unnecessary ACK frames which cause a little more data on the LAN than really necessary.

This technique is simple, reliable and adequate for many TCP connections. However if you require large amounts of bulk data to be transferred you will probably have a surprise since the throughput is not as high as initially expected. This is due to the fact that the next block of data can only be sent when there has been an acknowledgement to the previous. In a LAN the round trip time is very short but in the Internet this may rise to hundreds of ms or even several seconds. This will clearly reduce throughput since there will be dead times.
However in a LAN you will often find that the receiver will be using a technique called "delayed acks" where rather than immediately sending an ACK to received TCP frames there is a delay of 150ms..200ms inserted. This is in the hope that a receiving application will be sending a frame back and the TCP layer can piggy back its ACK on it - and also save unnecessary short ACK frames. The result is that usually only about 5 TCP frames can be transferred in a second. This gives a data throughput (using 1460 byte payloads) of only 7300 bytes a second (or 58400 bits a second). This is no faster than most serial interfaces.

However the simple technique results also in absolute minimum complication and code size so there is no reason why it is not used if the bulk speed is not the issue. Many TCP applications also allow delayed acks to be disabled, which then compensates most speed loss in a LAN.

In the next part in the series about using TCP in the uTasker project I will introduce buffered TCP to show how TCP windowing can be activated to greatly improve bulk data throughtput, although often at the cost of more RAM requirement...


Regards

Mark

neil:
Hi Mark,
  Nice article above.

Regarding 'TCP_EVENT_REGENERATE'. You mention that the previous message must be resent. Is there a chance that there can be an endless event, where the call back keeps getting 'TCP_EVENT_REGENERATE' everytime the message is resent (for some weird reason)? If so I assume that its best to retry 'x' amount of times before closing the connection?

I have written a lot in sockets on Windows applications, and never had to worry about the lo-level 'ACKs' etc as these are all hidden. So please excuse my basic  questions  :)

I understand that if I simply receive a 'TCP_EVENT_REGENERATE', I do a resend. And if I get a 'APP_SENT_DATA', the data was transmitted/received okay.
I thought the only messages I would really have to manage are 'TCP_EVENT_REGENERATE' , 'APP_SENT_DATA', 'TCP_EVENT_CONNECTED'. But after I resend if a  'TCP_EVENT_REGENERATE'  is returned, I will get a ''TCP_EVENT_ACK' instead of a  'APP_SENT_DATA' . Is this correct?

But if the data cannot be resent, if I receive a  'TCP_EVENT_REGENERATE', I simply wait for a ''TCP_EVENT_ACK' before doing anything else with the socket (apart from closing it)?

Is that the basics, or am I missing something else  ???
Thanks
Neil
 

mark:
The following details how to use buffered TCP to transmit TCP frames.


Buffered TCP support can be activated by setting the define USE_BUFFERED_TCP in config.h.
This then allows the use of the following routine to transmit frames:

QUEUE_TRANSFER fnSendBufTCP(USOCKET TCP_socket, unsigned char *ptrBuf, unsigned short usDataLen, unsigned char ucCommand)

This technique is used for example by the TELNET module, which serves as a good example of its use.

The noticable difference to the the fnSendTCP() call is the ucCommand parameter instead of the Flag parameter. The following show how it can be used.

CHAR test_message[] = "Hello World ";
QUEUE_TRANSFER nr_of_bytes = (sizeof(test_message) - 1);
fnSendBufTCP(Test_socket, (unsigned char *)test_message, nr_of_bytes, (TCP_BUF_SEND | TCP_BUF_SEND_REPORT_COPY));
fnSendBufTCP(Test_socket, (unsigned char *)test_message, nr_of_bytes, (TCP_BUF_SEND | TCP_BUF_SEND_REPORT_COPY));
fnSendBufTCP(Test_socket, (unsigned char *)test_message, nr_of_bytes, (TCP_BUF_SEND | TCP_BUF_SEND_REPORT_COPY));

The next noticable difference is the fact that 3 test messages are sent immediately after another, whereas previous fnSendTCP() examples have always sent only one message and waited until it had been acked. The command flag TCP_BUF_SEND informs the buffered TCP socket that the data should be transmitted as soon as possible. The flag TCP_BUF_SEND_REPORT_COPY causes the function to return the number of bytes of the test message which were actually accepted for transmission.

The first time that a buffered socket is used, a structure TCP_TX_BUFFER is automatically created which contains various control variables and also a data buffer of size TCP_BUFFER.
The size of this buffer can be defined in app_hw_xxxx.h as in the following:
#define TCP_BUFFER            2800
This allocates buffer space of 2800 bytes. When fnSendBufTCP() is called, the data to transmit is first copied into this buffer. As many bytes are copied as possible and if the buffer becomes full, the trailing ones will be ignored.
If the user needs to ensure than no data can be lost due to such a buffer overflow, the following will first verify whether there is enough buffer space to accept the message:

UTASK_TASK OurTask = OWN_TASK;
if (fnSendBufTCP(Test_socket, (unsigned char *)&OurTask, nr_of_bytes, TCP_BUF_CHECK) > 0) {
    fnSendBufTCP(Test_socket, (unsigned char *)test_message, nr_of_bytes, TCP_BUF_SEND);
}
The value returned will be equal to the remaining buffer space BEFORE adding the data. If the available space is less that the message length it will return 0. In addition to returning 0, the owner task will be entered to be woken by an TX_FREE interrupt event when the buffer becomes free again. This allows tasks to be suspended on waiting for buffer space to become freed.

Note that the define WAKE_BLOCKED_TCP_BUF should be set in config.h to enable this option!

As soon as the first TCP frame has been copied to the transmitter buffer a TCP frame is also sent. Generally there will be no ACK received from this frame for at least 150ms (assuming active delayed ack operation at the destination). The second call to fnSendBufTCP(), which follows immediately in the example, will cause the second part of the data to be added to the buffer. As long as the congestion windows (this will be explained later) allows it, a second TCP frame will immediately be sent, although there has not yet been an acknowledge to the first one. The same is also true for the third call of fnSendBufTCP() although the congestion window value will generally not allow it to actually send the associated data just yet.

At this point in time two TCP frames are under way to the destination and three blocks of data are in the tcpbuffer. The fact that the data is available as a backup in the tcp buffer means that the application itself doesn't need to also have a copy of the data, or be able to regenerate it if a frame needs to be repeated. In the case of a repetition - signalled by TCP_EVENT_REGENERATE, the socket's callback routine can simply call
fnSendBufTCP(Test_socket, 0, 0, TCP_BUF_REP);
which will start the repetition of lost data.
NOTE: See the further example at the end of this note for the recommended method of repetating to ensure that windowing operation remains efficient.

One important fact to be noted about buffered TCP is that the data copied to the output buffer is treated as a stream of data and not as individual frames of the same lengths as entered. This means that it is very suited to debug interfaces and the likes since the user can send the data in small packets (such as "text + value + text", as often used when constructing an output) and the data will tend to be sent as frames of maximum size rather than small TCP frames.
This behaviour would be observed if the example frames were not to be acked:
1. Frame 1 "Hello World "
2. Frame 2 "Hello World "
3. No ack to either of the 2 frames so TCP_EVENT_REGENERATE received by the callback routine.
4. On use of fnSendBufTCP(Test_socket, 0, 0, TCP_BUF_REP); the next TCP frame will have the content "Hello World Hello World Hello World " since the frame is constructed from as many waiting bytes as possible (the first 2 non-acked frame contents plus the third waiting one).

The maximum size of individual framnes can also be defined in app_hw_xxxx.h
    #define TCP_BUFFER_FRAME      1400

When using buffered TCP, the fact that more than one tx frame is sent before receiving an ACK means that TCP windowing is in operation. One great advantage of TCP windowing is the fact that bulk data throughput is much greater than when using the simple TCP transmission method described earlier. Not only do round trip delays impact less the transmission efficiency but the fact that the destination immedately receives two TCP frames effectively stops any outstanding Delayed ACK timer it has in operation and thus causes an ACK to be immediately returned to the second TCP frame. By using just 2 outspanding frames in a LAN can already increase data throughput greatly.

The defined SUPPORT_PEER_WINDOW should also be used when operating with buffered TCP because this support the processing of partial ACKs by TCP. Partical ACKs are are acknowledgements to TCP data which do not include all outstanding frames and these must be handled in the callback in the case for TCP_EVENT_PARTIAL_ACK.

This handling is in fact very simple since it only has to do the following:
fnSendBufTCP(Socket, 0, usPortLen, TCP_BUF_NEXT);

In the case of TCP_EVENT_ACK (which means that all outstanding data have been acked), the corresponding call is fnSendBufTCP(Socket, 0, 0, TCP_BUF_NEXT);

In both cases further waiting data (again as much as possible) will be sent in following TCP frames. The variable usPortLen, rather than 0, informs the buffered TCP routine exactly how much of the outstanding TCP data has successfully been acked. The TCP buffer is effectively freed of acked data so that the buffer has more space to accept further data from the user.

Buffered TCP operation allows improved data throughput at the expense of RAM buffer space. The routines suppport a congestion window which allows a number of TCP frames to be sent before their ACKs have been received. Initially a connection allows 2 frames to be outstanding but this value is allowed to increase by one between each ack up to the maximum defined by WINDOWING_BUFFERS in config.h. This ensures that the number of outstanding TCP acks builds up to mathh the return trip time of a connection and so achieves optimum bulk data throughput.

A difficults is generally in deciding the amount of SRAM to allocate to the TCP buffer since in small footprint projects such RAM represently quite a luxury. In any case there is also the question as to what happens if the user data rate is still faster than the TCP bulk data throughput rate, in which case even the largest buffer will probably not stop it ever filling up completely. The use of the call fnSendBufTCP() with flag TCP_BUF_CHECK has already been introduced and this can be used to stop temporarily the source of data until the TCP buffer size has increased again. The uTasker demo project uses quite a small TELNET buffer (which is also using buffered TCP) to illustrate how, for example, large menus are displayed with limited buffer space. If you study this example code, where the copying of the source text to the TCPbuffer is suspended until space becomes available again, you will probably get enough ideas to solve your own such data bottle-necks.

One popular use of such a TCP connection is for the transfer of data from a UART to a remote PC via TCP. flow control is required throughout the link and a typical technique would be to issue an XOFF or remove CTS (depending on UART flow control in operation) when the TCP buffer is getting critically full) and removing it when the TX_FREE event is received.
Here is a full list of the flags available when using fnSendBufTCP(). Some of the flags can be used together - their combined use should be logical.

    #define TCP_BUF_SEND               put the attached data into the buffer and send asap
    #define TCP_BUF_NEXT               an ack has been received so send as much more waiting data as possible from the TCP buffer.
    #define TCP_BUF_REP                data has been lost. Repeat as much more waiting data as possible from the TCP buffer.
    #define TCP_BUF_CHECK              check whether the specified amount of data will fit into the TCP buffer. If it won't fit, send a TX_FREE event when the buffer is free.
    #define TCP_BUF_SEND_REPORT_COPY   report the amound of data copied to teh TCP buffer (used together with TCP_BUF_SEND).
    #define TCP_CONTENT_NEGOTIATION    used by TELNET together with TCP_BUF_SEND to flag that data is a negotiation frame and so must not be combined with other data
    #define TCP_REPEAT_WINDOW          used only internally to TCP to handle specific repetitions
    #define TCP_BUF_KICK_NEXT          This can be used kick in windowing operation after data loss. The next example illustrates the recommended use of it in response to the TCP_EVENT_REGENERATE event.

case TCP_EVENT_REGENERATE:
    if (fnSendBufTCP(Socket, 0, 0, TCP_BUF_REP) != 0) {  // repeat send buffered
        fnSendBufTCP(Socket, 0, 0, (TCP_BUF_NEXT | TCP_BUF_KICK_NEXT)); // kick off any following data as long as windowing allows it
            return APP_SENT_DATA;
        }
        break;


The buffered TCP support allows quite simple use of the technique to achieve high speed transmission in most applications. However the operation of TCP is still rather more complicated than has been described here and the description of TCP itself is far from complete. Flow control from one end of a connection (possibly including more than the TCP stage in practical systems) is not as simple as one initially believes. But the uTasker should support all that is necessary to achieve respectable results even in small footprint solution (a uTasker powered Serial<->TCP converter based on the ATSAM7X128 - 48MHz ARM7 with 32k SRAM - has in fact outperformed an ARM9 solution with 16Meg SDRAM - based on my own measurements...). But this description should serve as a good starting point and the forum can be used to elaborate on the things which I have unintentionally or intentionally left out for the moment.

Comments are very welcome.

The final part of this series will discuss TELNET, which is based on the buffered TCP technique. It will be shown how a TELNET-like socket can be used rather than a direct buffered TCP socket to acheive the same improvements but with a more simple interface - using the existing TELNET handling routines as a interface engine.

Before closing it is worth mentioning one more point. It is also possible, in some situations, to still achieve high data throughput based on TCP windowing without using buffered TCP. This is feasible when the data is not of random nature (like data which is being transferred from a UART) but can be regenerated on demand. A web server is a good example since the files which it is serving (also when partly dynamically generated) can usually be regenerated without having to keep a large data store of previous transmissions.

A new version of HTTP server for the uTasker project is being developed (tests have already proved very successful with page serving rates of larger http files or graphics in a LAN of up to 25x the rate when using simple TCP transmission) which should be available with the next service pack!!

Regards

Mark

mark:
Hi Neil

TCP_EVENT_REGENERATE is sent when ARP has resolved an IP address. This will only ever happen once.
It is also called when a TCP message transmission times out without an ACK. The socket counts the timeouts and automatically limits the repeats to TCP_DEF_RETRIES (this is fixed at 7 in tcpip.h). If maximum repeats still fails it is obvious that the connection has failed and so it will be closed by using a reset.

Yes, when using Windows sockets there are a lot of details which are hidden. However there is also a lot of software involved and this obviously has to be handled somewhere. I have now also added a description of using buffered TCP where ACKs do still have to be handled. The last in the series will detail TELNET and the use of a TELNET-like socket. In this case it inserts another layer between the application and the buffered TCP socket which does in fact offload the 'dirty-work' from the user layer. Then the user never has anything to do with the regeneration event. It makes it easier to use and, if you do already have Telnet active in your device, the software is all there so it doesn't add to code size. Most events are handled by the TELNET layer but some are also passed on via a second 'user' callback so that some details can still be controlled there as well if needed.

Regards

Mark

mark:
The following details how Telnet works and how to use a TELNET-like socket for general TCP socket implementation.

When TELNET is activated - by setting the define USE_TELNET in config.h - buffered TCP is also enabled since the TELNET implemmentation is based on this method. The uTasker demo project controls the TELNET socket in the file debug.c where is is used to allow a menu-driven command interface to be controlled via a TELNET connection. When no Telnet connection is active the same menu-driven interface is available via a serial interface (activated by the define SERIAL_INTERFACE in config.h). This shows that the actual use of the TELNET connection is very similar to a typical serial terminal connection and it is thus very useful for configuration and general debugging purposes.

Before beginning with the details of Telnet it should be mentioned that the uTasker project includes several optimised routines which help display data, some of which work with a serial interface. The most common is the following function:

extern QUEUE_TRANSFER fnDebugMsg (CHAR *ucToSend); // send string to debug interface

Using this call, the standard "Hello World!" welcome can be send:

    fnDebugMsg("Hello World!");

The question is to which interface this message is actually sent? The answer is that it depends on the setting of a global variable:
    QUEUE_HANDLE DebugHandle = NETWORK_HANDLE;

This can be set by the user to any serial interface handle - for example the uTasker demo project sets it to a serial port if a serial port is enabled in the project:
    DebugHandle = SerialPortID;

If nothing is changed, it is destined to NETWORK_HANDLE which means that the message will in fact be passed on to a routine called fnNetworkTx():

extern QUEUE_TRANSFER fnNetworkTx(unsigned char *output_buffer, QUEUE_TRANSFER nr_of_bytes);

This routine must be defined somewhere in the project and it can also be a dummy function if it is not actually used. In the uTasker demo project it can be found in debug.c where it transfers data to the Telnet socket, if this is enabled:

return (fnSendBufTCP(Telnet_socket, output_buffer, nr_of_bytes, (TCP_BUF_SEND | TCP_BUF_SEND_REPORT_COPY)));

Here we see that it is using the buffered TCP interface and the socket is defined as the Telnet_socket, which we will discuss below.
Note however that DebugHandle can be changed by any code to switch between serial interfaces or a network interface, it doesn't have to be left at the default NETWORK_HANDLE value. The uTasker demo project does just this and so enables messages to be sent over the serial interface or over an active Telnet interface, depending on which is desired.

To open a Telnet socket use the function:
   Telnet_socket = fnStartTelnet(23, (5*60), 0, 0, fnTELNETListener); 

This is the prototype of the call:
extern USOCKET fnStartTelnet(unsigned short usTelnetPortNumber,
      unsigned short usIdleTimeout,
      unsigned short usMaxWindow,
      UTASK_TASK wakeOnAck,
      int (*listener)(USOCKET, unsigned char, unsigned char *, unsigned short) )

As can be seen, a Telnet socket is started (it will be set to listener state) on the standard Telnet port 23. It will have an inactivity timeout period of 5 minutes and uses the callback function fnTELNETListener(). For the moment we will leave the other parameters at 0, which means that they will take default values.

Note that the Telnet socket can later be stopped by simply calling fnStopTelnet(Telnet_socket);

The Telnet callback function (in its simplest form) is in fact quite similar to a standard TCP callback function:

static int fnTELNETListener(USOCKET Socket, unsigned char ucEvent, unsigned char *ucIp_Data, unsigned short usPortLen)
{
    switch (ucEvent) {
    case TCP_EVENT_CONREQ:                                     // session request
    case TCP_EVENT_ABORT:
    case TCP_EVENT_CLOSE:
    case TCP_EVENT_CLOSED:
    case TCP_EVENT_ACK:
        break;

    case TCP_EVENT_CONNECTED:                                  // TCP connection
        return fnTelnet(Telnet_socket, GOTO_ECHO_MODE);        // Set echo mode

    case TCP_EVENT_DATA:
        if (fnCommandInput(ucIp_Data, usPortLen, SOURCE_NETWORK)) { // collect data input and try to interpret if a LF is found
            return APP_SENT_DATA;                // we will always have sent an answer
        }
        break;
    }
    return APP_ACCEPT;                           // default is to accept a connection
}

Here it has been simplified in comparison to the use in debug.c since it uses the events to switch debug output between the serial port and Telnet and also supports a user name / password login sequence.

When data is received, it is passed on to the menu handling routine, which will not be described here but quite simply matches a list of available command to the received data and performs the command if found. When a TCP connection has been established - event TCP_EVENT_CONNECTED is received - a further Telnet command is used to switch the Telnet layer to echo mode. In this mode each received frame or, in the case of a remote user typing data in, each received character will be echoed back.

It can thus be seen that by opening a Telnet socket we have in fact opened a buffered TCP socket which has a few more capabilities that a standard buffered TCP socket. There is no handling of TCP_EVENT_REGENERATE since the TELNET layer is doing most of the dirty work for us, which also makes its use very simple. It is however a Telnet socket, which means that it has some Telnet specific function. It is best to look a little closer at the function fnTelnet() which can be used together with a Telnet socket.

extern int fnTelnet(USOCKET Telnet_socket, int iCommand);

The following command are possible:
GOTO_ECHO_MODE                 switches echo on
LEAVE_ECHO_MODE                switches echo off
PASSWORD_ENTRY                 flags that text entry is a password and so * is echoed back
CLEAR_TEXT_ENTRY               leave password entry mode
TELNET_ASCII_MODE              sets ASCII mode (this is the default mode of the Telnet socket)
TELNET_RAW_MODE                sets RAW mode - this means that the Telnet socket will not perform any Telnet-specific things
TELNET_RAW_RX_IAC_ON           enables IAC searching although in RAW mode
TELNET_RAW_RX_IAC_OFF          disables IAC searching in RAW mode
TELNET_RAW_TX_IAC_ON           enables IAC byte stuffing although in RAW mode
#define TELNET_RAW_TX_IAC_OFF  disables IAC byte stuffing in RAW mode
#define TELNET_RESET_MODE      reset mode to default (ASCII mode)

The first thing to note is that by calling fnTelnet(Telnet_socket, TELNET_RAW_MODE); you have now a Telnet-like socket which is not doing any Telnet specific stuff. This is then a buffered TCP socket which is doing the dirty work without trying to interpret any Telnet negotiations. This is thus a handy method of working with one or more buffered TCP sockets with minimum effort at the application layer!

If a Telnet-like socket is required to connect as a client to a remote server, it could use the following sequence, based on the standard fnTCP_Connect() command:

Telnet_socket = fnStartTelnet(usPortNumber, (5*60), 0, 0, fnTELNETListener);
if (Telnet_socket >= 0) {
    fnTelnet(Telnet_socket, TELNET_RAW_MODE);               // set RAW mode
    fnTCP_Connect(Telnet_socket, ucIP, usPortNumber, 0, 0); // connect to remote server
}



It has to be said that Telnet is rather complicated due to the various things that it can do. The uTasker Telnet implementation just does some basic but useful things to enable a reliable Telnet connection in ASCII or Binary mode (or Telnet-like connection in RAW mode). Since it supports TCP Windowing it is fast, but don't forget that the price penalty is in the RAM required for the TCP buffer(s).

Since a typical application for Telnet is to allow simple remote debugging, it supports negotiation of an ECHO mode. In this mode all received characters are echoes back to the remote client.

The default Telnet mode is ASCII mode. In this mode all received data is being scanned for negotiation sequences. These negotiation commands begin always with the IAC character (0xff) which doesn't occur in the ASCII data content.
However when transmitting binary data over a Telnet connection (with possibility of 0xff appearing in the data stream) byte stuffing is used. When 0xff is detecting in the tx data stream it is sent as a sequence of 2 x 0xff. The receiver performs the inverse so that it is still possible to detect real command sequences within binary data transfers.
This allows switching between ASCII and Binary modes, as well as other Telnet commands, during data transfers.


Depending on which programs are being used to transfer data, different types of operation are often observed. By setting up the various combinations of modes using fnTelnet() it is always possible to define the specific combination required to successfully transfer any type of data.

Some special notes on the operation of buffered TCP relevant to Telnet:

1. When negotiation sequences are sent, they are marked as such so that they will always be sent (including regeneration) as individual commands. This ensures that they will never be converted to binary data due to byte stuffing, nor be grouped together with other commands or data when being repeated. The buffered TCP transmission call uses the flag TCP_CONTENT_NEGOTIATION to acheive this.

2. When sending data which needs to be byte-stuffed there is a risk that the data content can, in the most extreme case, double. i.e. when 100 x 0xff are to be sent, they will be sent as 200 x 0xff because each byte will be stuffed to produce 2!
This means that the TCP buffer would have to be dimensioned to twice the normal required size to ensure that all data could be backed up in the most extreme case. The solution in the uTasker project is however not to back up the stuffed data (which would infact be the simplest method, but with the disadvantage of doubled buffer sizes) but instead to back up only the raw data and mark it appropriately.
In the case of TCP regeneration (hidded in the Telnet part) retransmitted frames are always regenerated from a backup of the raw data. This means a little more work (and it is rather more complicated...) when regenerating, but optimum buffer space efficiency as a result. Since the efficiency of regenerating lost messages is unimportant (they have already suffered a long timeout delay) the space saving has much higher importance.


I hope that the practical use of Telnet and Telnet-like sockets will now be very simple. They are easy to use and enable high speed data transfer to be acheived. The configuration possibilites enable fast data transfer between any combination of programs.

Not all details of TCP operation have been mentioned here but the descripton should be adequate for first implementations. There are some additional complications concerning TCP flow control across connections which may or may not be noticable in specific projects. The uTasker implementation does however allow all known difficulties to be solved and I look forward to comments and feedback so that additional points can be elaborated in real-world contexts.

Best regards

Mark

Navigation

[0] Message Index

[#] Next page

[*] Previous page