Messages

1

µTasker general / Re: Large HTTP transfers failing due to retransmission counter

« on: May 18, 2010, 03:48:09 PM »

Hello Mark,

After digging I've found my problem. Totally my fault. I made some modifications a long time ago in the HTTP module. Instead of returning something like (fnSendTCP(...) > 0) I forgot the "> 0" part and returned fnSendTCP(). Since fnSendTCP() will almost always return 1400 (0b10101111000), the fourth bit will always be set to 1, thus causing the fnHandleTCP to think that the APP_REQUEST_CLOSE flag was set. Therefore the fnHandleTCP returned without calling fnNewTCPState:

Code: [Select]

if (APP_REQUEST_CLOSE & (iAppTCP |= ptr_TCP->event_listener(ptr_TCP->MySocketNumber, TCP_EVENT_ACK, ptr_TCP->ucRemoteIP, ptr_TCP->usRemport)))
{
    /* ... */
    return;
}


Anyway, sorry for wasting your time and thanks for pointing me out in the right direction.

Phil

2

µTasker general / Re: Large HTTP transfers failing due to retransmission counter

« on: May 17, 2010, 08:38:12 PM »

Hello Mark,

The transfer happens on a local network, and the uTasker device is sending the file to a PC.

I don't understand exactly why the connection is being reset.

The connection is reset by the fnCloseTCPSession function which does the following:

Code: [Select]

    ptr_TCP->ucSendFlags = TCP_FLAG_RESET;
    fnSendTCPControl(ptr_TCP);


If there is an ACK received from any of the retransmissions it will cause the counter to be reset to its maximum value again.

Are you sure about that? I've searched the variable ucRetransmissions in source code. The only time that it is set to TCP_DEF_RETRIES is when fnNewTCPState is called. Obviously, this function is not called after each frame. To be sure, I printed the value of ucRetransmissions each time fnPollTCP is hit and it does not seem to be reset.

If my diagnosis happen to be correct then a fix could be to reset the counter upon receiving an ACK (in fnHandleTCP under the TCP_STATE_ESTABLISHED case).

Code: [Select]

                if (rx_tcp_packet.ulAckNo == ptr_TCP->ulNextTransmissionNumber ) {
                    ptr_TCP->ulSendUnackedNumber = ptr_TCP->ulNextTransmissionNumber; // No more unacked data
                    ptr_TCP->ucRetransmissions = TCP_DEF_RETRIES; // <-------------- FIX
                    // inform application that data has been successfully acked
                    if (APP_REQUEST_CLOSE & (iAppTCP |= ptr_TCP->event_listener(ptr_TCP->MySocketNumber, TCP_EVENT_ACK, ptr_TCP->ucRemoteIP, ptr_TCP->usRemport))) {
                        if (rx_tcp_packet.usHeaderLengthAndFlags & TCP_FLAG_FIN) {    // The application has commanded the close of the connection
                            // We have just received FIN + ACK, meaning that the other side has also initiated a close
                            // This a simultaneous close, so we go to state closing and wait for the last ack
                            fnNewTCPState(ptr_TCP, TCP_STATE_CLOSING);
                        }
                        return;
                    }
                }



If my diagnosis is wrong the I can send some Wireshark captures.




Phil

3

µTasker general / Large HTTP transfers failing due to retransmission counter

« on: May 17, 2010, 04:17:15 PM »

Hello,

For some time now we've been having problem transferring large files (>1MB) over HTTP. uTasker would send a RST after some time. After investigation, I was able to identify the culprit. fnPollTCP will decrement the ptr_TCP->ucRetransmissions counter each time a TCP_EVENT_REGENERATE is generated, and when the counter value is zero then the connection is closed (fnCloseTCPSession is called). The only place I see the counter being reset is in fnNewTCPState which sets the counter value to TCP_DEF_RETRIES.

Shouldn't the ucRetransmissions counter be reset after each successfully transferred packet? Where would be the most appropriate place to do it?

Thanks for your support
Phil

4

µTasker general / Re: Is it possible to update values on a Web Page without loading the entire page?

« on: April 21, 2008, 04:41:11 PM »

A few people asked me how to generate dynamically XML files, so here it goes.

DISCLAIMER: Generally you should only make changes to the uTasker codebase when necessary. For simple cases you should rely on the "official" method for generating dynamic documents which is inserting values in static HTML pages (through fnInsertValue). The method I describe below works well but has drawbacks (such as documents being limited to a single packet - 1400 bytes).

The general idea is to look for a specific string at the beginning of the GET URL. For example, is someone requests '/1.htm' then the HTTP server will behave normally. However, if someone requests '/XML?GetDummyValue' then some code will detect that the query string begins with XML? and pass on 'GetDummyValue' to a special function. From this function I just write directly to the HTTP buffer.

So, a (very) simplified version of the code will look like this:

A) This is inserted in fnDoWebPage. Can't tell you exactly where since we've made extensive changes to the original code, but it shouldn't be too hard to figure out.

Code: [Select]

if (!uMemcmp(data, "XML?", 4))
{
    int len = MakeXmlDocument(tcpBuffer, data + 4);
    session->ucState = HTTP_STATE_ACTIVE;
    session->usUnacked = len;
    return fnSendTCP(session->OwnerTCPSocket, (uchar*)&tcpBuffer->tTCP_Header, len, 0) > 0;
}

B) The MakeXmlDocument writes a XML document to tcpBuffer depending on the query and returns the number of bytes written.

Code: [Select]

int MakeXmlDocument(char* tcpBuffer, char* query)
{
     if (!uStrcmp(query, "GetDummyValue"))
     {
          // Write to tcpBuffer here, return the number of bytes written.
     }
}

There you go. This post wasn't meant to be a cut-and-paste working example, but I hope you get the idea. Please tell me if you want more implementation details.

5

µTasker general / Re: Feature suggestion: HTTP Response Header

« on: January 31, 2008, 11:21:47 PM »

Hello Mark,

The HTTP Header issue is currently listed as a future enhancement in our development tracking system.
I can provide the source code when it's completed, however I don't know when.

6

µTasker general / Re: Is it possible to update values on a Web Page without loading the entire pag

« on: January 31, 2008, 11:19:14 PM »

We use AJAX in our project to update real-time informations and it works like a charm.
One of the problems with page refresh is that you will see the page "flicker" because the whole document is reloaded.
If you refresh often then it will not look good. Also, you are forced to regenerate & resend the whole page HTML every time instead of just sending the values (possibly a few bytes) you want to update.

To use AJAX, you will need some basic knowledge of JavaScript, but that's all.

Here is a very simple example:

Suppose you have a value, let's call it Dummy, that you want to refresh on a page.

First you need to create dynamically a very simple XML document that will only contain the required value.
For example whenever you request 'http://utasker/XML?Dummy' from the uTasker HTTP server you will get the following response:

Code: [Select]

<Dummy>47</Dummy>Here, my dummy value is simply a counter that is incremented every request.

Next, you store a static web page that will look like the following (the simplest AJAX implementation I can think of), let's call it dummy.html:

Code: [Select]

<html>
    <head>
        <script language="JavaScript">
            var request;         
            function onLoad()
            {     
              sendRequest();
              setTimeout('onLoad();', 500);
            }
            function sendRequest()
            {       
              request = new XMLHttpRequest();
              request.onreadystatechange = onResponseReceived;
              request.open("GET", "XML?Dummy");    
              request.send(null);
            }
            function onResponseReceived()
            {           
            if (request.readyState == 4 && request.status == 200)
            {           
            document.getElementById("dummyval").innerHTML = request.responseXML.getElementsByTagName("Dummy")[0].firstChild.data;
            }
            }                    
        </script>
    </head>
    <body onload="onLoad();">
    Some stuff will not be updated.
    <br/>
        Some stuff will (dummy value = <span id="dummyval">.</span>).   
    </body>
</html>
From a user perspective, you only need to request 'http://utasker/dummy.html'. In this case, the JavaScript code will take care of requesting 'XML?Dummy' every 500 ms and then update the value of the 'dummyval' span element.

You will see something like this in your browser:

Code: [Select]

Some stuff will not be updated.
Some stuff will (dummy value = 2248).


Of course that's the theory. My AJAX implementation above is very simplistic (no error checking) and only works with Firefox. The hard part of using AJAX is to have a consistent cross-browser experience. Firefox and IE often doesn't use the same objects or methods, so you will have to rely on a few tricks and wrapper methods like the following:

Code: [Select]

function newXmlHttpRequest()
{
if (window.XMLHttpRequest)
{
// Mozilla
return new XMLHttpRequest();
}
else if (ActiveXObject)
{
// Internet Explorer
return new ActiveXObject("Microsoft.XMLHTTP");
}
}
Luckily, there is a lot of AJAX-related documentation and code snippets on the web.

-phil

7

µTasker general / Re: Feature suggestion: HTTP Response Header

« on: January 31, 2008, 09:39:31 PM »

I second the proposition.

The problem with having no HTTP headers is that you rely on the browser default behaviors, especially when handling file types and caching. For example when no headers are provided Firefox will generally not cache anything, which makes browsing slow because common items (gif, css, xsl) are fetched every time. IE does the opposite by caching too much: async requests made through XmlHttpRequest() are cached by default which might create problems when trying to display real-time informations. Also, when the browser does not have a Content-Type header it will try to detect content-type by file extension. This might lead to inconsistent behavior between browsers. Also, some applications or libraries might not play nice with headerless HTTP responses. I know that the HTTPClient object from the .NET Framework will throw an exception when no headers are provided.