Author Topic: Request: please complete proper TCP/IP API documents :)  (Read 13767 times)

Offline aaronlawrence

  • Jr. Member
  • **
  • Posts: 66
    • View Profile
Request: please complete proper TCP/IP API documents :)
« on: January 12, 2010, 12:02:18 AM »
Hello Mark,
Could you please finalise the TCP/IP API document?
I know about the forum thread, but it is a very clumsy way to read and not very complete.
I would really appreciate some standard API documentation describing the parameters, return codes and usage of each function.
Thanks.

Offline mark

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 3234
    • View Profile
    • uTasker
Re: Request: please complete proper TCP/IP API documents :)
« Reply #1 on: January 12, 2010, 12:44:16 AM »
Hi Aaron

The request has been noted.

Presently work is being undertaken on an IPV6 stack (to work as dual-stack alongside IPV4). A new document including both stacks may be appropriate.

regards

Mark

Offline aaronlawrence

  • Jr. Member
  • **
  • Posts: 66
    • View Profile
Re: Request: please complete proper TCP/IP API documents :)
« Reply #2 on: January 12, 2010, 02:12:57 AM »
OK, but that sounds some time away...

At least could you add to the forum thread, a description of each function and it's parameters? I know its a lot more work to do it in a pretty document...
Thanks.

Offline mark

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 3234
    • View Profile
    • uTasker
Re: Request: please complete proper TCP/IP API documents :)
« Reply #3 on: January 13, 2010, 01:47:54 AM »
Hi Aaron

I managed a couple of hours on documentation today - specifically TCP.

You can find this on the code document page: http://www.utasker.com/docs/Code.html (see TCP/IP -> TCP section)

Note that each new function description usually takes half an hour or longer to complete so it will take some time to finish the TCP list (complete functions unfortunately rather longer...). If you require specific explanations to individual functions please ask them here so that the order of work can also be prioritized.

Regards

Mark


Offline aaronlawrence

  • Jr. Member
  • **
  • Posts: 66
    • View Profile
Re: Request: please complete proper TCP/IP API documents :)
« Reply #4 on: January 13, 2010, 06:40:37 AM »
Hello Mark,

Thanks that is looking helpful. The section on fnGetTCP_Socket is good.
Observations:
- it would be better if it didn't use browser Frames because it makes it difficult to link to a page within the docs.
- seems odd to categorise it by the implementation file - as an API I wouldn't expect to have to know which source file to use :) Only a nit.
- a search or index function to look up functions by name is useful.

An observation on the API: the function naming seems a bit inconsistent.  For example some TCP functions
fnHandleTCP
fnTCP_Listen
fnGetTCP_state
sometimes the TCP is at the start with an underscore after, sometimes in the middle, sometimes at the end. Sometimes the part after the underscore is capitalised (_Listen) , sometimes not (_state).
This is only minor but makes the API a little bit harder to use because the names of the functions are harder to remember and/or determine.

« Last Edit: January 14, 2010, 06:18:21 AM by aaronlawrence »

Offline aaronlawrence

  • Jr. Member
  • **
  • Posts: 66
    • View Profile
Re: Request: please complete proper TCP/IP API documents :)
« Reply #5 on: January 14, 2010, 06:20:24 AM »
(separated this from the rest of the general comments)

fnTCP_close() : are there any conditions on this? I am wanting to use it to close a server socket after a timeout - so not inside a call from the TCP stack. I could see that if I wanted to close inside a callback, I would use APP_REQUEST_CLOSE...
I ask because I basically copied out of one of the examples, first using fnTCP_Close, then fnTCP_Listen after getting the CLOSE event; and the client doesn't seem to be able to connect again.
Code: [Select]
fnTCP_close( g_ControlSocket );
...
static int fnControlListener( USOCKET Socket,  ... ) {
    switch (ucEvent) {
...
    case TCP_EVENT_ABORT:
    case TCP_EVENT_CLOSED:
fnTCP_Listen( Socket, CONTROL_PORT, 0 );                    // go back to listening state

Edit: Now I have this working, although there is a period of a few seconds after close when new connections are refused (get a Windows socket error 10061). Is that expected? Does it mean I'm not doing a proper graceful shutdown? The Windows side is doing shutdown, waiting for FD_CLOSE, then closesocket...

Is there a general way to find out if a socket is connected? fnGetTCP_State looking for TCP_STATE_ESTABLISHED?? Seems overly reliant on internal details. I'd rather not have to have another variable of my own ...

Offline mark

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 3234
    • View Profile
    • uTasker
Re: Request: please complete proper TCP/IP API documents :)
« Reply #6 on: January 15, 2010, 01:23:13 AM »
Hi Aaron

I managed to add the fnTCP_close() description:
http://www.utasker.com/docs/CodeDoc/fnTCP_close.htm

This was a bit more complicated than expected since there are various important bits of background information involved.

I won't try changing anything in the function documentation project structure at the moment since that is rather involved and it is better to concentrate on the content. Apart from there, frames have been avoided - however by right mouse clicking on a document link and then selecting "Copy Link Location" it is at least relative easy to give links to the content (as above). This refers to Firefox since the web site is not designed for IE - in fact the new IE completely ignores scrolling bars in the forum and so is not usable. Strangely no one has complained about this, so I can only assume that everyone uses Firefox when visiting the forum (!!)

The close function discussion contains various details and also links to the TCP transition state diagram showing the states involved when closing a connection. There are some delays involved but I would not normally expect these to be noticed in practice. It may be that your Windows application is attempting a further connection before the first connection has fully closed - in this case, unless you have multiple sockets listening (like HTTP) the socket on this port will send a RST back to a second connection attempt in parallel. If you send me a Wireshark recording of the close and the next connection attempt I may be able to see something (otherwise try to find out which state the socket is in at that time).

There are two methods of monitoring a connection. First is to follow it in the call back function since it will receive an event TCP_EVENT_CONNECTED when the connection has been established and a TCP_EVENT_CLOSED (or TCP_EVENT_ABORT) when it has been closed. Between the two the connection is open - with small intervals of opening and closing - and outside the two it is closed.
The routine fnGetTCP_state() can also be used:

if (fnGetTCP_state(socket_number) & TCP_STATE_ESTABLISHED) {
    // connected
}
else {
  // not connected
}


This actually includes (TCP_STATE_SYN_SENT, TCP_STATE_FIN_WAIT_2, TCP_STATE_CLOSING, TCP_STATE_CLOSE_WAIT, TCP_STATE_LAST_ACK) so it could be connecting or closing.

if (fnGetTCP_state(socket_number) == TCP_STATE_ESTABLISHED)

would be ONLY when the connection has been established and NOT during connection or close phases.

The states have been designed to allow multiple states (usually logically grouped) to be easily recognised - this saves code doing things like

if ((state == TCP_STATE_SYN_SENT) || (state == TCP_STATE_FIN_WAIT_2) || (state == TCP_STATE_CLOSING) || ....)


and helps the TCP code efficiency.

Regards

Mark






Offline aaronlawrence

  • Jr. Member
  • **
  • Posts: 66
    • View Profile
Re: Request: please complete proper TCP/IP API documents :)
« Reply #7 on: January 15, 2010, 03:42:58 AM »
Hm OK lots of interesting notes in the TCP Close docs .. thanks, I will absorb these...

I'm still a bit confused with this section:
Quote
The event TCP_EVENT_CLOSED is sent to the application although the socket state is TIME WAIT - the socket has not fully closed, even though the TCP partner has completely acknowledged the close sequence. The TIME WAIT generally lasts for a period of 2 minutes (specified as 2MSL - twice the Maximum Seqment Lifetime).

This seems to be saying that reusing the socket by doing a fnTCP_listen is a bad idea (at least for a socket going across the internet). Or, does fnTCP_listen still respect the WAIT TIME?

Perhaps this is what I'm experiencing, although it only seems to last a few seconds?

On Windows, the FD_CLOSE is sent once the FIN is completed, it doesn't wait for 2MSL; so in this respect your event seems the same.

Where is SOCKET_STATE_INVALID defined?
Is fnGetTCP_Socket the same as fnTCP_Connect?

Offline aaronlawrence

  • Jr. Member
  • **
  • Posts: 66
    • View Profile
Re: Request: please complete proper TCP/IP API documents :)
« Reply #8 on: January 15, 2010, 04:02:12 AM »
Hello Mark, just so you know you're getting something out of this, we bought a project license :) It's under Vicki Toon, our administrator/CC holder ..

Offline mark

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 3234
    • View Profile
    • uTasker
Re: Request: please complete proper TCP/IP API documents :)
« Reply #9 on: January 15, 2010, 03:37:27 PM »
Hi Aaron

As long as you set a socket to listening state it should be prepared for any following connection request (a request will only result in a RST when there is no listening socket on the TCP port in question - meaning that the socket's state is not suitable for accepting a connection). Therefore I can't explain any delays in a following attempt. If you send me a Wireshark recording (to email address) I may be able to see something which explains why it is not immediate.

Back to the 2MSL topic: First of all it should be noted that, according to the TCP specification, a TCP server should not accept any connections for a period of 2 minutes after first being turned on. This is in fact the same as 2MSL after ending a connection since the thinking is that a server may have been rebooted during or just after a connection and so also shouldn't accept a connection on a previously used socket within the same time period. PC TCP stacks don't respect this because generally it takes more than 2 minutes to boot up anyway....but the question is whether embedded stacks, which are ready to operate as soon as the Ethernet has auto-negotiates (say 3 or 4 seconds) should stay in a "silent" mode for two minutes? I think that the question is superfluous since most complaints heard about embedded systems is why they are not ready in less that these 3 or 4 seconds!

Since the 2MSL time is protecting against something that will (especially today with fast Internet) hardly take place and also if does will result in a TCP RST being sent back it does seem justified to accept a deviation from the word of the specification in preference of better embedded operation. It can also be disabled so that no connection is possible for  minutes if this needs to be performed...

Therefore I would suggest that there is generally no problem in setting a socket which has just closed immediately back to the listening state. It allows one socket to be used for connections which close and are then immediately opened again.

It is also possible to reserve multiple sockets for use by a single TCP connection. This can be done in two ways:

- either like the HTTP which has a number of sockets listening and can handle multiple connections in parallel - these are called HTTP sessions and each session has its own resources so that the parallel operation is really possible. As long as there is a socket listening (meaning free for use) a further connection can take place

- or by having only one of the possible sockets actually listening at a time, in which case they share the call back and only one set of resources (eg. variables used during a connection) is needed. Also only one connection is possible at a time. The reason for having multiple sockets is that the 2MSL could indeed be respected by allowing the closed socket to stay in that state for a further 2 minutes and bridge this time by setting the next available socket to the listening state. It would thus be possible to have several sockets timing out in the 2MSL phase while there is always a fresh one immediately made available for use by a subsequent connection. The limit is of course the number of connections and closes possible within the 2MSL period since this would be the number of sockets that need to be reserved to handle the worst case. If the number is low it could be a valid technique, if high it may result in a waste of memory resources - again a mixture of the two would work . if the second strategy does lead to a situation where all available sockets are in the 2MSL phase the oldest could still be forced to listening state (as emergency measure).

fnTCP_Connect() is the inverse of fnTCP_close(). It actively starts a TCP connection by sending a SYN to the TCP port at the remote IP address. I will add this as next job. It is not the same as fnGetTCP_Socket() which reserves the socket for subsequent use; fnGetTCP_Socket() is usually called only once.

SOCKET_STATE_INVALID During the documentation process I realised that some return values were not giving full information (often a failure was simply -1 and in one case a 0 was returned in error case which could not be distinguished from a value USOCKET number). These error cases generally only occur when the system is not configured correctly - for example the application tries to use more sockets that reserved, or there is not enough heap for the configuration to be established and so don't occur randomly during operation. Due to this I decided that it would be best to also improve this situation by adding some more meaningful return values - in addition documenting the non-specific return values would also have been difficult to do...

This explains why do don't find this new define. The return is still there but the define has been optimised. I have sent you (email) my working versions of TCP.c and tcpip.h so that you can either use them immediately or have them as reference during this phase. They will then be included in the next release so that the doc and code fully corresponds.

Regards

Mark


- fnTCP_Connect() added at http://www.utasker.com/docs/CodeDoc/fnTCP_Connect.htm
« Last Edit: January 15, 2010, 10:29:56 PM by mark »

Offline aaronlawrence

  • Jr. Member
  • **
  • Posts: 66
    • View Profile
Re: Request: please complete proper TCP/IP API documents :)
« Reply #10 on: January 19, 2010, 12:03:33 AM »
Next question.
I presume that fnTCPSend will complain if I already sent some data that is unacked? Or, will it just allow me to send again?

Offline mark

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 3234
    • View Profile
    • uTasker
Re: Request: please complete proper TCP/IP API documents :)
« Reply #11 on: January 19, 2010, 01:26:09 AM »
Hi Aaron

If you are using a simple socket (not a buffered one) you are not generally allowed to send more data until the previous packet has been acknowledged. If this is performed it will result in sequence counter error at the receiver. When buffered TCP is used the buffered interface handles this and the user doesn't need to get involved with it.

There are ways around it in case multiple packets are sent (HTTP sends 2 packets, which is adequate to overcome delayed ack operation at a PC and thus result in fast serving in a network) but this involves checking that the remote TCP receivers input window can handle it, managing a congestion window and handling partial acknowledgements.

fnTCPSend() will not complain since it can send further frames but it is up to the application to ensure that this doesn't normally happen.

Simple sockets are easy to handle and also very reliable. If speed is not of essence I would stick with these at first - ensure that you use the TCP_EVENT_ACK event to allow following frames to be transmitted (never transmit another before this has been received). Also ensure that you handle TCP_EVENT_REGENERATE by resenting the last packet (this needs either a backup of the content until the TCP_EVENT_ACK confirms that it has arrived at the destination without error - or its content must be regenerated, as the HTTP server does).

Regards

Mark

Offline aaronlawrence

  • Jr. Member
  • **
  • Posts: 66
    • View Profile
Re: Request: please complete proper TCP/IP API documents :)
« Reply #12 on: January 20, 2010, 07:46:30 AM »
Yes, I am definitely sticking with simple sockets. Having started the design from uIP which has no buffering, its a lot easier (and anyway suits the kind of thing we are doing).