http.c. Not parsing form posts correctly

[Thomas]

Hi!

It seems the parsing in http.c (SAM7X) of post packets is missing a few things. I have the form below. You start parsing by looking for "boundary=" but this string is not existing with text/plain encoding. If I try to use multipart/form-data the parsing exits prematurely later looking for "Content-Type:" which also don't exist.

It's no problem for me to implement what I need, but just wondering if an updated http.c file exists..? Always looking to save time..!

Code: [Select]

<form action="*0Menu.htm" method="post" enctype="text/plain" >
Username:
<input type="text" name="user" size="8" maxlength="8">
<br>
Password:
<input type="password" name="password" size="8" maxlength="8">
<input type="submit" value="Log in">
</form>


[mark]

Hi Thomas

I think that the plain text posting is supported in your HTTP.c but it is probably not active.

Take a look in config.h:

            #define SUPPORT_HTTP_POST                                    // support file uploads using POST command
            #define SUPPORT_POST_BINARY
          //#define SUPPORT_POST_TEXT
            #define SUPPORT_POST_GIF

This is the set up for the demo project which accepts binary (for software upload) and GIF (for a user changeable picture on a web page).
If you comment in SUPPORT_POST_TEXT, then plain text "text/plain" will also be recognised.

The idea behind the activation of only specific types is to save unnecessary code space (expecially in projects with smaller chips) and also to block undesired types.

Regards

Mark


PS. It is easy to add other types if needed:

1. In config.h add a define like #define SUPPORT_POST_NEW_ONE

2. In http.c add the new text which must be recognised - eg
#ifdef SUPPORT_POST_NEW_ONE
    static const CHAR cContentType_x[] = "new_type/new\r\n\r\n";
#endif

3. In http.c, in the routine fnPostFrame() add in state HTTP_STATE_READY_POSTING

Code: [Select]

#ifdef SUPPORT_POST_NEW_ONE
                    if (!(uMemcmp((unsigned char *)cContentType_x, ucIp_Data, (sizeof(cContentType_x) - 1)))) {
                        ucIp_Data += (sizeof(cContentType_x) - 1);
                        usPortLen -= (sizeof(cContentType_x) - 1);
                        HTTP_session->FileLength -= ((sizeof(cContentType_x) - 1) + 2);
                        HTTP_session->FileLength -= HTTP_session->ucBoundaryLength;
                        HTTP_session->ucState = HTTP_STATE_POSTING_DATA;
                        break;
                    }
#endif

[Thomas]

Hi Mark,

I had already #defined SUPPORT_POST_TEXT and SUPPORT_HTTP_POST_TO_APPLICATION (not related really). The parser doesn't make it enough far anyway (to the HTTP_STATE_READY_POSTING state) because the parser stops prematurely because it doesn't find what it needs. I think my example form is a typical one and looking at the packets I capture with Wireshark it seems the parser will not work with forms and packets like this. I tried both IE7 and Firefox2 just to be sure and they both send packets built the same way.

[mark]

Hi Thomas

OK. That is another issue, which I haven't experienced before.
Please could you send me your WireShark recording? This will allow me to play it back through the code to see exactly what is happening.
In the meantime I will also embed your HTML form code in a web page to see whether I can get it to behave the same here.

Regards

Mark

[mark]

Hi Thomas

I checked out the operation of your form and have seen that the operation of the post is quite different from the supported operation.

The one which is included in the demo is like this:

Code: [Select]

<form action=HS.bin
enctype="multipart/form-data" method="post">
<p><input type="file" name="datafile" size="30"></p>
<p><input type="submit" value="Upload new software"></p>
</form>which supports the transfer of files. The format includes a boundary, used to identify the header from the data content.

In your case I think that you are using the post method to transfer a user name and password, which results in the post using a different format without the boundary field. This results in the post not being recognised and the HTTP connection remains open, waiting for further parts of the data to be sent by the client. The client has however already sent all data. The TCP link is OK but there is no further HTTP activity.

Of couse it should be possible to extend the handling of the post sequence to recognise also posts without the boundary field, but another simple solution is to use a standard form (like used on the LAN page for MAC and IP addresses) to transfer the text strings. The disadvantes is however that you are transfering a password (although it is actually displayed as plain text in the data and is thus not very secure) which it will be visible in the URL address.

Your set up to post to application is otherwise correct and, if it were recognise, would result in the content being passed to the application callback.

Conclusion: to support the post method as such, it will be necessary to also be able to work without the boundary field.  Now that I have a reference I will have to study a little more how this can best be performed.
A temporary workaround would be to use a standard form.

Regards

Mark

[Thomas]

Thanks for looking into this and to clarify that we see the same. I will need to send many kinds of smaller and larger configuration data and I don't whan to use the GET method since you send the parameters in the URL which then gets stored in the browser history. I'll probably make some corrections to the parsing in a week or so but I guess this is probably of general interest also, to support post methods more extensively.

So, let me know if you decide to implement something soon 

[mark]

Hi Thomas

I have implemented a version which solves the user name / password case. I will send you a new HTTP.c file (plus tcpip.h due to two new HTTP states). See the define PLAIN_TEXT_POST for the new bits which is activated locally in http.c. The define SUPPORT_POST_TEXT is not in fact used since the content type is not checked (the header has a different sequence in this case...) but I think that this should always be used together with SUPPORT_HTTP_POST_TO_APPLICATION because it probably doesn't make much sense to save to a file.

When the frame is received, the application call-back (fnHandleWeb() in webInterface.c) will receive the event type POSTING_DATA_TO_APP with a pointer to the received content *ptrData. The length is HTTP_session->FileLength.

The user name and password can therefore be handled there by interpreting this buffer.

There will be a second event INFORM_POST_SUCCESS when the end of the post has been detected (in this case it will be immediately afterwards).

Here you have the possibility to define which web side should be returned, or else the file defined in the form (0Menu.htm in your example) will be automatically displayed.

My tests on the simulator were good. There is however a restriction. The complete post must be received in one TCP frame. This will always be the case as long as the actual post data is not larger than about 1.2k in size - hopefully adequate for what you are doing (?).

It may be necessary to further optimise (I don't know what other cases there may be involved) but if the solution proves useful it can be added to the main project.

Good luck

regards

Mark

[danh]

Hi Mark,

I would also like to use HTTP POST and am having the same issue as Thomas. I have tried it using my own web page as well as Thomas's example, and basically the state never gets past HTTP_STATE_START_POST. All of the related items are defined in config.h, including SUPPORT_HTTP_POST_TO_APPLICATION and SUPPORT_POST_TEXT. My test is based on nearly unmodified uTasker 1.3 SP7.

You mentioned some changes to http.c and tcpip.h, but I don't see any notes about this in subsequent service packs (SP6 and SP7 for M5223x). Are there some simple changes I can make to support this?

Thanks,

Dan

[mark]

Hi Dan

The method was not included in SP7 (in fact I forgot to add it then) but I have sent it to a few users on request.
There is however an open problem with it. The post of parameters has proved successful but it can't be used together wit the port of data (this is no longer recognised correctly!)

Now it turns out that I worked on a project last week which used both methods and so I had a chance to study it again in more depth. I have also promised a solution for the coming week.Therefore I suggest waiting a couple of days since I should then be able to send you the final solution. It may be as SP since I am a bit behind on SP8 which will include USB support for the M5221X/2X - making full use of the double buffering capabilities for high speed streaming and also enabling firmware upgrades via USB.

Regards

Mark

[danh]

Hi Mark,

Thanks - I'll just work on some other things until you have something ready.

Regards,

Dan

[mark]

Hi Dan

Since I already have a solution I posted the code to you for tests.
I could solve the compatibility problem with the file POST method and also simplified the solution. I tested with Thomas's test code.
If you can confirm that it works well for you I will commit it for the upcoming service pack release for the M5223X (and others to follow).

regards

Mark

[danh]

Hi Mark,

For the record (even though I already privately posted you about this), this seems to work fine. I tried posting of parameters, but not file posting.

Thanks,

Dan

[Stan]

Hello,

I'm  also trying to send parameters with POST method. I read this topic and saw that you are talking about 

When the frame is received, the application call-back (fnHandleWeb() in webInterface.c) will receive the event type POSTING_DATA_TO_APP with a pointer to the received content *ptrData. The length is HTTP_session->FileLength.

What I have in my webInterface.c is:
...
    switch (ucType) {
#ifdef SUPPORT_HTTP_POST
    #ifndef _HW_NE64
        #if FILE_SYSTEM_SIZE >= (160*1024)                               // support posting new firmware only with large file system
    case CAN_POST_BEGIN:
      //return (1);                                                      // block upload!!
        return (0);                                                      // allow post at all times

    case INFORM_POST_FAILED:
        http_session->ptrFileStart = (unsigned char*)cWarningHTML;
        http_session->FileLength   = sizeof(cWarningHTML)-1;
            #ifdef SUPPORT_MIME_IDENTIFIER
        http_session->ucMimeType   = MIME_HTML;                          // force HTML type
            #endif
        return DISPLAY_INTERNAL;
        #endif

    case INFORM_POST_SUCCESS:
        #ifdef SUPPORT_MIME_IDENTIFIER
        http_session->ucMimeType = MIME_HTML;                            // force HTML type
        #endif
        if (http_session->cDisplayFile == 'S') {                         // software upload
            http_session->ptrFileStart = (unsigned char*)cSuccessSW_HTML;
            http_session->FileLength = sizeof(cSuccessSW_HTML)-1;
            uTaskerMonoTimer( TASK_APPLICATION, (DELAY_LIMIT)(1*SEC), 0 ); // wait 1 second to load the page
            fnDelayResetBoard();                                         // after 2 seconds the application will reset and then the new software will be copied
        }
        return DISPLAY_INTERNAL;
    #endif
#endif
...

So how can I handle incoming parameters from POST method?
Should I add case POSTING_DATA_TO_APP by myself? I see that in http.c I have a row:
    fnHandleWeb(POSTING_DATA_TO_APP, (CHAR *)ucIp_Data, http_session);
So I guess it passes the event correctly but there is no place where fnHandleweb() handles it.

Best Regards
Stan
 

[mark]

Hi Stan

In order to post parameters the option PLAIN_TEXT_POST is required.
This will in fact call with the event POSTING_PARAMETER_DATA_TO_APP (POSTING_DATA_TO_APP is used when a file transfer is destined to the application - you are right that there is no example of this in the demo project but I have added an example for another project at the bottom of this message).

Here is a simple example of handling the posted parameters, which is included as example in more recent service packs:

    case POSTING_PARAMETER_DATA_TO_APP:                                  // the buffer contains POST parameter data, whose length is defined by http_session->FileLength
        fnWrite(DebugHandle, (unsigned char *)ptrData, (QUEUE_TRANSFER)http_session->FileLength); // send to debug output
        break;

Regards

Mark




Example of posting an SREC file to the application via web server. The content if interpreted by the routine fnSaveSrec() [not shown here] and saved to FLASH. Once the complete file has been received (event INFORM_POST_SUCCESS) the success web page is returned and a reset is performed after 2s. In case of errors the 'failed' web side is returned.


static CHAR cSrec[100];                                                  // space for receiving SREC left overs
static unsigned short usRecordLength = 0;
static unsigned char ucCorruptedDownload = 0;

in fnHandleWeb()
    case CAN_POST_BEGIN:
        ucCorruptedDownload = 0;
        return (0);                                                      // allow post at all times

        case POSTING_DATA_TO_APP:
            while (http_session->FileLength--) {
                if (usRecordLength >= sizeof(cSrec)) {                   // protect from overrun
                    usRecordLength = 0;
                    ucCorruptedDownload = 1;
                }
                if ((cSrec[usRecordLength++] = *ptrData++) == 0x0a) {
                    if ((ucCorruptedDownload == 0) && (fnSaveSrec(cSrec, 1) < 0)) {
                        ucCorruptedDownload = 1;
                    }
                    usRecordLength = 0;
                }
            }
            break;

    case INFORM_POST_FAILED:
        http_session->ptrFileStart = (unsigned char*)cWarningHTML;
        http_session->FileLength   = sizeof(cWarningHTML)-1;
        http_session->ucMimeType   = MIME_HTML;                          // force HTML type
        return DISPLAY_INTERNAL;

    case INFORM_POST_SUCCESS:
        if (http_session->cDisplayFile == 'S') {                         // software upload
            if (ucCorruptedDownload != 0) {
                ucCorruptedDownload = 0;
                return (fnHandleWeb(INFORM_POST_FAILED, 0, http_session));
            }
            http_session->ptrFileStart = (unsigned char*)cSuccessSW_HTML;
            http_session->FileLength = sizeof(cSuccessSW_HTML)-1;
            fnDelayResetBoard();                                         // after 2 seconds the application will reset and then the new software will be copied
        }
        http_session->ucMimeType = MIME_HTML;                            // force HTML type
        return DISPLAY_INTERNAL;

[Stan]

Hi Mark,

I searched all files but I don't have anywhere PLAIN_TEXT_POST in my project. I use uTasker V1.3 with SAM7X with SP3.
Although I have these in my config.h and all are defined:

#define SUPPORT_HTTP_POST                                    // support file uploads using POST command
#define SUPPORT_HTTP_POST_TO_APPLICATION
#define SUPPORT_POST_BINARY
#define SUPPORT_POST_TEXT
#define SUPPORT_POST_GIF

I can't find POSTING_PARAMETER_DATA_TO_APP as a definition too. I'm simply trying to pass a password with the POST method but still can't figure out what I'm supposed to do.

Best Regards
Stan