Author Topic: uTasker Documentation and Operation  (Read 14548 times)

Offline SVC1

  • Newbie
  • *
  • Posts: 2
    • View Profile
uTasker Documentation and Operation
« on: April 23, 2010, 04:24:41 PM »
Hi,

I am trying to get acquainted with the OS and the only document I found was “µTasker operating system basics - V0.04 29.03.2008” which is the µTaskerv1.3.pdf document (the name is confusing since the project is µTaskerv1.4).

After reading this document, I have some questions:

1) I guess this OS is not a preemptive rtos. There are no task priorities as such (higher priority preempting lower task)?

2) If my comprehension of question1 is correct (not a preemptive OS), what happens if we have 2 takes scheduled so that task1 runs every 100ms and task2 runs every 150ms. But, for whatever reason, task1 takes 60ms to finish. What happens to task2? Does it run 10ms late?

3) I guess there are no semaphores and no need to protect global variables (critical section) in heap since one task is running at any given time. Right?

TIA,
S.

PS: I would like to stress test the TCP/IP stack. Is there a simple echo program (a task that returns received message to sender) available?



Offline mark

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 3236
    • View Profile
    • uTasker
Re: uTasker Documentation and Operation
« Reply #1 on: April 23, 2010, 09:48:17 PM »
Hi

The V1.3 introduction documents remains relevant for V1.4 - V1.4 has more features but no change to basic operation. When the opportunity arises all references to the version will be removed when not relevant.

The uTasker operating system, is a state-event driven scheduler. It is not pre-emptive (I think that this is often also called co-operative).

There are pros and cons of co-operative and pre-emptive as with everything else in life. There is some background to the decision to use this strategy here: http://www.utasker.com/forum/index.php?topic=413.0 (and in its links).

To your question 2 - the second task would indeed run 10ms late. Tasks in the uTasker project are designed to respond to events and should generally complete their work as fast as possible. When larger amounts of work need to be performed this can usually be achieved by running a task in polling mode for a period of time, whereby it performs chunks of work each time that it is scheduled - giving the feel of a background task (eg. the LCD task and SD card tasks make a lot of use of this technique; the LCD task will poll LCD readiness while working and limits the amount to work done in a single pass; the SD card task will detect cards during slow polling the mount them and format them in polling mode so that the task's operation is effectively time sliced.

Drivers make use of interrupts for any critical I/O and the same can be used for many time critical applications level jobs too.

3) The advantage of co-operative scheduling is indeed simplification in many respects. Memory only needs to be protected against interrupts and not (generally) other tasks. Tasks don't need their own memory and the result is very efficient use of RAM resources, often allowing more to be done with smaller foot print chips.


There is a UDP echo activated by setting the define DEMO_UDP in application.c - search for this define to see the code.
There is also a TCP reference here that responds to a certain received text - it can be modified to simply echo all data back: http://www.utasker.com/forum/index.php?topic=25.0

It may also be an idea to simply connect via TELNET to the demo project. This will echo input back which is not recognised by the command line interface. Since TELNET uses TCP windowing it will generally achieve higher data rates for bulk data too. In debug.c (this contains the TELNET socket's application call back fnTELNETListener()) the command line interface can be removed by modifying the following case:

    case TCP_EVENT_DATA:
        if (fnCommandInput(ucIp_Data, usPortLen, SOURCE_NETWORK)) {      // collect data input and try to interpret if a LF is found
            return APP_SENT_DATA;                                        // we will always have sent an answer
        }
        break;



to

    case TCP_EVENT_DATA:
        if (fnSendBufTCP(Socket, ucIp_Data, usPortLen, (TCP_BUF_SEND | TCP_BUF_SEND_REPORT_COPY)) != 0) {      // echo back
            return APP_SENT_DATA;                                        // we will always have sent an answer
        }
        break;

In fact this will send a double echo since TELNET will usually be in echo mode and the application interface will be sending a second echo too! The TELNET socket can also be set to raw data mode, which is generally the simplest method for using buffered TCP [the TELNET layer performs most of the dirty work, taking it from the application layer]. As the links explain, buffered TCP mode can also be used directly  (the application call back has to do a bit more work to handle all of the possible cases) or else simple TCP mode can be used if bulk transfer is not important, which is usually the simplest to handle. You shouldn't however notice any real difference in reaction time in any of the modes when just testing echos - TELNET/buffered TCP will just take a few us longer since it passes through an additional output buffer.

Regards

Mark


Offline aaronlawrence

  • Jr. Member
  • **
  • Posts: 66
    • View Profile
Re: uTasker Documentation and Operation
« Reply #2 on: April 30, 2010, 12:45:35 AM »
Hello SVC1,
Having just written a bunch of code that effectively stresses uTasker TCP, I can tell you that there were a number of very severe problems. Mark has fixed what has been identified so far - excellent support as usual - but does not have any automated testing of his own so you will likely find many more.

It seems to me that uTasker TCP has been mostly used in strict client-server situations, mostly with it's built in webserver and other protocols. The underlying TCP/IP protocols were originally not exposed (as far as I can tell) and have not been thoroughly tested, or even documented.

Sorry, Mark, but that's the clear situation from our perspective.

Offline oneFranck

  • Newbie
  • *
  • Posts: 9
    • View Profile
Re: uTasker Documentation and Operation
« Reply #3 on: June 01, 2010, 08:27:43 PM »
Hi there,

This Echo Server that accept 3 connections seems to work well with one clients (run for a weekend long without disconnecting),  but when there is 3 clients connected to it, it seems to stop after few hours (disconnect).  Any idea what could be wrong in my code?

The clients sends a frame (len=1400) right after receiving the echo from the server.

Code: [Select]
/**************************************************************************
*                              M A C R O S                                *
***************************************************************************/
#define TCP_ECHO_PORT_NUMBER 950
#define TEST_BUFFER_LENGTH       1400

#define MAX_TCP_SOCKETS 3  // Accept 3 sockets on port 950
/**************************************************************************
*                     L O C A L    P R O T O T Y P E S                    *
***************************************************************************/
static int fnTestListener(USOCKET Socket, unsigned char ucEvent,
  unsigned char *ucIp_Data, unsigned short usPortLen);

/**************************************************************************
*                   T Y P E     D E F I N I T I O N S                     *
***************************************************************************/
typedef struct stTCP_MESSAGE
{
    TCP_HEADER     tTCP_Header;     // reserve header space
    unsigned char   ucTCP_Message[TEST_BUFFER_LENGTH];
} TCP_MESSAGE;


/**************************************************************************
* FUNCT: fnEchoServerInit
*
* starts the echo server
***************************************************************************
*                                 C O D E
***************************************************************************/
extern void fnEchoServerInit( void )
{
static USOCKET tcp_socket1 = -1;
static USOCKET tcp_socket2 = -1;                                            // declare a static socket variable
static USOCKET tcp_socket3 = -1;
     
    static unsigned short usTestPort = TCP_ECHO_PORT_NUMBER;                  // declare a port number

// create a TCP socket with a tim out of 60 sec
        tcp_socket1 = fnGetTCP_Socket(TOS_MINIMISE_DELAY, (unsigned short)60, fnTestListener);
tcp_socket2 = fnGetTCP_Socket(TOS_MINIMISE_DELAY, (unsigned short)60, fnTestListener);
tcp_socket3 = fnGetTCP_Socket(TOS_MINIMISE_DELAY, (unsigned short)60, fnTestListener);
   
    // bind socket to listening port number
    fnTCP_Listen(tcp_socket1, usTestPort, 0);                         
    fnTCP_Listen(tcp_socket2, usTestPort, 0);     
    fnTCP_Listen(tcp_socket3, usTestPort, 0);     
}




static int fnTestListener(USOCKET Socket, unsigned char ucEvent, unsigned char *ucIp_Data, unsigned short usPortLen)
{
    TCP_MESSAGE test_message;
char dbugstr[50] = {'E', 'm', 'p', 't', 'y', '/0'};
   
    switch (ucEvent) {
    case TCP_EVENT_CONREQ:
    fnDebugMsg("TCP Connection request received\r\n");                                           
    break;
    case TCP_EVENT_CONNECTED:
  sprintf(dbugstr, "TCP Connection established on socket no. %d\r\n", Socket);
fnDebugMsg(dbugstr);   
    break;
    case TCP_EVENT_CLOSE:
    fnDebugMsg("TCP Connection stop request\r\n");
    break;   
    case TCP_EVENT_ACK:
    case TCP_EVENT_ARP_RESOLUTION_FAILED:
    case TCP_EVENT_PARTIAL_ACK:
        break;
    case TCP_EVENT_REGENERATE:
        case TCP_EVENT_DATA:
  sprintf(dbugstr, "usPortLen = %d\r\n", usPortLen);
fnDebugMsg(dbugstr);
        // send copy
        if (uMemcpy((CHAR*)test_message.ucTCP_Message, (CHAR*)ucIp_Data, usPortLen)) {
            if (fnSendTCP(Socket, (unsigned char *)&test_message.tTCP_Header, usPortLen, TCP_FLAG_PUSH) > 0) {
                return APP_SENT_DATA;
            }
        }
        else
        fnDebugMsg("uMemcpy failed\n\r");
        break;
       
       
    case TCP_EVENT_ABORT:
    case TCP_EVENT_CLOSED:
        fnTCP_Listen(Socket, TCP_ECHO_PORT_NUMBER, 0);                    // go back to listening state on next port number
        fnDebugMsg("TCP Connection fully closed\r\n");
        break;
    }
    return APP_ACCEPT;
}


What do you think of that strategy to test the TCP/IP stack?

Quote
Having just written a bunch of code that effectively stresses uTasker TCP
Can you tell here what are your strategies for testing a tcp/ip stack?

Thanks

FB




Offline mark

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 3236
    • View Profile
    • uTasker
Re: uTasker Documentation and Operation
« Reply #4 on: June 02, 2010, 02:56:43 PM »
Hi

I don't "see" any reason why a connection should disconnect in yoir test case, however a disconnect can be due to one side closing the connection or an error leading to a connection being reset. In some cases it is also caused by application layer code (not necessarily related to TCP/IP) to corrupt memory, leading to follow-on-errors. It is best to first identify which of these possible causes is valid in this case - this is best done by monitoring the network so that the close/reset is seen.

If the connection is being released by the board the question is why. There was a problem solved recently which resulted in a lost transmission never being repeated - leading to a timeout and hence close of a connection. This occurred in a case when the remote server was continuously sending data (irrespective of responses from the board) so may or may not be a possible issue in your test (?)

I have sent you the latest development version of TCP which has been verified to resolve this case - generally this situation is not typical of TCP/IP use and so has not been an issue in general use.

To test TCP/IP thoroughly it is necessary to test as many different situations as possible - this in fact means testing as many error conditions as possible. Therefore a stress-test alone is generally not that useful, especially when it is repeating the same sequences; better is when it is working more randomly (including timing, content, tx and rx loss, windowing behavior and probably much more).

The test method used in the project is to use a set of known problem situations and simulate these references in the uTasker simulator. This is very good for analysing certain situations and verifying corrections or optimisations. It is however not so good for discovering not-yet identified bugs since it would mean writing and verifying such test cases for every imaginable sequence of events - and the ones that you didn't imagine may prove to be the ones that actually cause problems, so this is still not adequate.

Therefore the strategy is to allow as many users to test the code in as many situations as possible. If a difficulty is identified a Wireshark recoding of it is used as input to a new test case. Its analysis and verification are given high priority to understand what happens and how it can be best solved and verified. The strange thing is that two years can pass with no reports of any problems and then a certain newly identified problem can be reported twice on the same day - hopefully also resolved on the same day or at least during the following ones. The result is a continuous improvement of understanding in use of TCP/IP, optimisations and stability. Although one can never say that everything is perfect and there will never be any more issues identified I do believe that the strategy has generally worked quite well and and the solution has proven to be acceptable for a high number of applications.

Regards

Mark

Offline aaronlawrence

  • Jr. Member
  • **
  • Posts: 66
    • View Profile
Re: uTasker Documentation and Operation
« Reply #5 on: June 02, 2010, 11:22:50 PM »
Quote
the strategy is to allow as many users to test the code in as many situations as possible.

Mark,
I think that you should be upfront about this in your marketing (ie. website). People buy a product like this because they want something that you have ALREADY tested thoroughly, they don't want to pay to help you test and fix bugs. I certainly expected that you would have a pretty thorough test suite and was unpleasantly surprised to find large numbers of pretty basic bugs. We have spent weeks (thousands of dollars) testing and debugging your code.

Of course, if you put on the front page that this is your strategy, you will probably lose some potential customers. But that is the price of being honest.

I know you will argue that our case was special. That's really besides the point. The point is that by simply saying you have a TCP/IP stack, with lots of features, and no caveats about level of testing, you create a false impression.


Offline mark

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 3236
    • View Profile
    • uTasker
Re: uTasker Documentation and Operation
« Reply #6 on: June 03, 2010, 02:36:29 AM »
Hi Aaron

I don't want to give the impression that testing is up to users. I have tested the TCP/IP stack progressively for 5 years now (plus the various services used with it which can of course have their own specialities and problems as well as the basic protocol). Is is however not easy to test all combinations possible within the framework of capabilities.

The fact that the project is offered free of charge (including support) to anyone interested in using it for educational, study and other non-commercial use opens up the additional dimension of potential problems being discovered in places where tests haven't been envisaged or just didn't test in exactly the way that it took place in that situation (why does Microsoft release free Beta versions...?). I consider this a positive point and not a negative point since it is not a case of doing sloppy work and expecting users to work out the problems but rather increasing the discovery potential after doing diligent work in the restraints of capabilities and budgets - but not necessarily bug-free work since bugs are an integral part of SW life, even if one never likes to admit it of their own work.

I don't expect that any company - also not the ones charging $30k for a stack - will be able to put their hand in the fire and swear that they know for 100% certainly that there is no bug in any of their code. Since these stacks are restrictive on code distribution there is probably less chances of uncovering weaknesses due to bulk use too.

Having experienced using one of the 30k stacks in a past project, the amount of debugging necessary to get the project working was indeed far from zero..... So I am still hoping that the price for commercial use of the uTasker project (which includes quite a lot more than the stack alone) proves still to make up for any deficiencies experienced in any particular case.

Regards

Mark

Offline aaronlawrence

  • Jr. Member
  • **
  • Posts: 66
    • View Profile
Re: uTasker Documentation and Operation
« Reply #7 on: June 03, 2010, 12:31:14 PM »
Yes, you may be right that uTasker is as good as it gets. It seems we have been too ambitious with the technologies we picked for our product.


Offline GT_Dev

  • Newbie
  • *
  • Posts: 4
    • View Profile
Re: uTasker Documentation and Operation
« Reply #8 on: June 04, 2010, 03:11:47 AM »
I believe the features and business model adopted by Utasker result in a milestone in the embedded technology market.

A set of identical solutions to UTasker, from any competitor can cost 20 or 30k, and in many cases, one license per product and not per platform.

The business model of UTasker is innovative.

I hope the project continues maturing and gaining more and more users around the world

I'm deciding what uC core to work, AVR32 or ARM, but I will buy the license and will have UTasker as main technological partner.

I just would like to see uTasker being developed by a team, instead of a single developer.
I think, this way, improvements and competitiveness would be even better.

Offline aaronlawrence

  • Jr. Member
  • **
  • Posts: 66
    • View Profile
Re: uTasker Documentation and Operation
« Reply #9 on: June 27, 2010, 04:50:34 AM »
Yes, but to fund a team - or even one person full-time, it's just Mark part time now - will require greatly increasing the price.
At the moment testing costs are effectively paid largely by customers. This allows the cost to be much lower, which is a reasonable business model, though some commercial customers would not be comfortable with it.