HTTP2 Server and MPFS2 Filesystem

This page represents analysis of the mechanism used by the Microchip Application library (TCPIP) for providing dynamic variables.

Introduction – Non MDD web site storage and dynamic variables

In the non MDD (default) version of the Microchip TCPIP Application library V5.31 and earlier, the HTTP2 web server and MPFS2 file system and MPFS2.exe utility are very tightly coupled and interdependent.

As a file system MPFS2  provides a mechanism for source website content to be wrapped into a single container file that is compatible with the HTTP2 web server, either uploaded to an external EEPROM (per the PNP-MEMEE512) or Flash (per the PNP-MEMFL32) memory module or included in the compiled firmware.

The MPFS2 file system also allows for compressible source content (Gzipped).  When HTTP2 serves such files it adds the appropriate HTTP header so the client browser will decompress it

Refer here for the MPFS2.1 file system structure

However, the process of turning a source folder of web content into an MPFS2 file system image using the supplied MPFS2.exe utility is also what enables the mechanism Microchip have implemented to enable dynamic variables.

Dynamic variables

A user creates a web page with a dynamic variable included in the html ie ~led(0)~

When the webpage is read from the MPFS2 filesystem image, the HTTP2 web server substitutes ~led(0)~ with the output of the function HTTPPrint_led(0).  The linking of the ~led(0)~ with the HTTPPrint_led function is via the callbackid mechanism which is enabled by the additional functionality of the MPFS2.exe utility.

MPFS2.exe enables dynamic variables

The Microchip utility MPFS2.exe is executed against the users source web pages to create the MPFS2 file system formatted *.bin.  However the utility does much more than just repackage the source files.  It performs these tasks:

1. Scans all the source files for dynamic variables

  • For each dynamic variable found:
    • Compare dynamic variables found to those previously used as per HTTPPrint.idx
      • If new, allocate the next sequential callbackid and update HTTPPrint.idx so in future iterations this dynamic variable & it’s callbackid is known
      • If previously used, use the same callbackid used previously for this dynamic variable
  • Recreate HTTPPrint.h
    • Add prototypes for each needed dynamic variable label ie HTTPPrint_led(WORD)
    • Add HTTPPrint(DWORD callbackID) function
      • For each callbackid, call the relevant function per its entry in HTTPPrint.idx

2. Create the destination MPFS2 file system formatted *.bin file adding each source file

For each source file in the source folder:

  • GZIP source file if allowed and add file into MPFS2 file system image (Set MPFS2_FLAG_ISZIPPED Flag for this file)
  • If the source file has dynamic variables:
    • The source file in the MPFS2 file system image has MPFS2_FLAG_HASINDEX set
    • A callback index file is added immediately after the source file into MPFS2 file system image
      • Contains an entry for each dynamic variable in the original file
      • Entries are simply a relative byte position and the callbackid for this dynamic variable per HTTPPrint.idx & HTTPPrint.h from section 1 above

HTTPPrint.idx

~inc:header.inc
+hellomsg
+inc:footer.inc
+cookiename
+
+builddate
+led(7)
+led(6)
+led(5)
+led(4)
+led(3)
+led(2)
+led(1)
+lcdtext
+ledSelected(4,TRUE)
+ledSelected(4,FALSE)
+ledSelected(3,TRUE)
+ledSelected(3,FALSE)
+ledSelected(2,TRUE)
+ledSelected(2,FALSE)
+ledSelected(1,TRUE)
+ledSelected(1,FALSE)
+version
+led(0)
+btn(0)
+btn(1)
+btn(2)
+btn(3)
pot
+uploadedmd5
+status_ok
+ddns_status
+ddns_status_msg
+ddns_user
+ddns_pass
+ddns_host
+status_fail
+config_mac
+config_hostname
+config_dhcpchecked
+config_ip
+config_gw
+config_subnet
+config_dns1
+config_dns2
+reboot
+rebootaddr
+ddns_service(0)
+ddns_service(1)
+ddns_service(2)
|inc:header.inc|hellomsg|inc:footer.inc|cookiename||builddate|led(7)|led(6)|led(5)|led(4)|led(3)|led(2)|led(1)|lcdtext|ledSelected(4,TRUE)|ledSelected(4,FALSE)|ledSelected(3,TRUE)|ledSelected(3,FALSE)|ledSelected(2,TRUE)|ledSelected(2,FALSE)|ledSelected(1,TRUE)|ledSelected(1,FALSE)|version|led(0)|btn(0)|btn(1)|btn(2)|btn(3)|pot|uploadedmd5|status_ok|ddns_status|ddns_status_msg|ddns_service(0)|ddns_service(1)|ddns_service(2)|ddns_user|ddns_pass|ddns_host|status_fail|config_mac|config_hostname|config_dhcpchecked|config_ip|config_gw|config_subnet|config_dns1|config_dns2|reboot|rebootaddr
+inc:header.inc
config_pubcomm1
config_pvtcomm1
config_readcomm1
config_readcomm2
config_readcomm3
config_writecomm1
config_writecomm2
config_writecomm3
read_community(0)
read_community(1)
read_community(2)
write_community(0)
write_community(1)
write_community(2)
+read_comm(0)
+read_comm(1)
+read_comm(2)
+write_comm(0)
+write_comm(1)
+write_comm(2)
+smtps_en
+snmp_en
+an(0)
+an(1)
+an(2)
+an(3)
+es1(0)
+es1(1)
+es1(2)
+es1(3)
+an(4)
+an(5)
+an(6)
+an(7)

Note that dynamic variable pot is unused in this example but has been used before.  The result of this is that there is no included callback function prototype for ‘pot’ nor is there a check for its callback id in function HTTPPrint. Instead there is a gap in callbackid select statements between case 0x0000001b and 0x0000001d . :

        case 0x0000001b:
            HTTPPrint_btn(3);
            break;
        case 0x0000001d:
            HTTPPrint_uploadedmd5();
            break;

The full  HTTPPrint.h.

/**************************************************************
 * HTTPPrint.h
 * Provides callback headers and resolution for user's custom
 * HTTP Application.
 *
 * This file is automatically generated by the MPFS Utility
 * ALL MODIFICATIONS WILL BE OVERWRITTEN BY THE MPFS GENERATOR
 **************************************************************/

#ifndef __HTTPPRINT_H
#define __HTTPPRINT_H

#include "TCPIP Stack/TCPIP.h"

#if defined(STACK_USE_HTTP2_SERVER)

extern HTTP_STUB httpStubs[MAX_HTTP_CONNECTIONS];
extern BYTE curHTTPID;

void HTTPPrint(DWORD callbackID);
void HTTPPrint_hellomsg(void);
void HTTPPrint_cookiename(void);
void HTTPPrint_(void);
void HTTPPrint_builddate(void);
void HTTPPrint_led(WORD);
void HTTPPrint_lcdtext(void);
void HTTPPrint_ledSelected(WORD,WORD);
void HTTPPrint_version(void);
void HTTPPrint_btn(WORD);
void HTTPPrint_uploadedmd5(void);
void HTTPPrint_status_ok(void);
void HTTPPrint_ddns_status(void);
void HTTPPrint_ddns_status_msg(void);
void HTTPPrint_ddns_user(void);
void HTTPPrint_ddns_pass(void);
void HTTPPrint_ddns_host(void);
void HTTPPrint_status_fail(void);
void HTTPPrint_config_mac(void);
void HTTPPrint_config_hostname(void);
void HTTPPrint_config_dhcpchecked(void);
void HTTPPrint_config_ip(void);
void HTTPPrint_config_gw(void);
void HTTPPrint_config_subnet(void);
void HTTPPrint_config_dns1(void);
void HTTPPrint_config_dns2(void);
void HTTPPrint_reboot(void);
void HTTPPrint_rebootaddr(void);
void HTTPPrint_ddns_service(WORD);
void HTTPPrint_read_comm(WORD);
void HTTPPrint_write_comm(WORD);
void HTTPPrint_smtps_en(void);
void HTTPPrint_snmp_en(void);
void HTTPPrint_an(WORD);
void HTTPPrint_es1(WORD);

void HTTPPrint(DWORD callbackID)
{
	switch(callbackID)
	{
        case 0x00000001:
			HTTPPrint_hellomsg();
			break;
        case 0x00000002:
			HTTPIncFile((ROM BYTE*)"footer.inc");
			break;
        case 0x00000003:
			HTTPPrint_cookiename();
			break;
        case 0x00000004:
			HTTPPrint_();
			break;
        case 0x00000005:
			HTTPPrint_builddate();
			break;
        case 0x00000006:
			HTTPPrint_led(7);
			break;
        case 0x00000007:
			HTTPPrint_led(6);
			break;
        case 0x00000008:
			HTTPPrint_led(5);
			break;
        case 0x00000009:
			HTTPPrint_led(4);
			break;
        case 0x0000000a:
			HTTPPrint_led(3);
			break;
        case 0x0000000b:
			HTTPPrint_led(2);
			break;
        case 0x0000000c:
			HTTPPrint_led(1);
			break;
        case 0x0000000d:
			HTTPPrint_lcdtext();
			break;
        case 0x0000000e:
			HTTPPrint_ledSelected(4,TRUE);
			break;
        case 0x0000000f:
			HTTPPrint_ledSelected(4,FALSE);
			break;
        case 0x00000010:
			HTTPPrint_ledSelected(3,TRUE);
			break;
        case 0x00000011:
			HTTPPrint_ledSelected(3,FALSE);
			break;
        case 0x00000012:
			HTTPPrint_ledSelected(2,TRUE);
			break;
        case 0x00000013:
			HTTPPrint_ledSelected(2,FALSE);
			break;
        case 0x00000014:
			HTTPPrint_ledSelected(1,TRUE);
			break;
        case 0x00000015:
			HTTPPrint_ledSelected(1,FALSE);
			break;
        case 0x00000016:
			HTTPPrint_version();
			break;
        case 0x00000017:
			HTTPPrint_led(0);
			break;
        case 0x00000018:
			HTTPPrint_btn(0);
			break;
        case 0x00000019:
			HTTPPrint_btn(1);
			break;
        case 0x0000001a:
			HTTPPrint_btn(2);
			break;
        case 0x0000001b:
			HTTPPrint_btn(3);
			break;
        case 0x0000001d:
			HTTPPrint_uploadedmd5();
			break;
        case 0x0000001e:
			HTTPPrint_status_ok();
			break;
        case 0x0000001f:
			HTTPPrint_ddns_status();
			break;
        case 0x00000020:
			HTTPPrint_ddns_status_msg();
			break;
        case 0x00000021:
			HTTPPrint_ddns_user();
			break;
        case 0x00000022:
			HTTPPrint_ddns_pass();
			break;
        case 0x00000023:
			HTTPPrint_ddns_host();
			break;
        case 0x00000024:
			HTTPPrint_status_fail();
			break;
        case 0x00000025:
			HTTPPrint_config_mac();
			break;
        case 0x00000026:
			HTTPPrint_config_hostname();
			break;
        case 0x00000027:
			HTTPPrint_config_dhcpchecked();
			break;
        case 0x00000028:
			HTTPPrint_config_ip();
			break;
        case 0x00000029:
			HTTPPrint_config_gw();
			break;
        case 0x0000002a:
			HTTPPrint_config_subnet();
			break;
        case 0x0000002b:
			HTTPPrint_config_dns1();
			break;
        case 0x0000002c:
			HTTPPrint_config_dns2();
			break;
        case 0x0000002d:
			HTTPPrint_reboot();
			break;
        case 0x0000002e:
			HTTPPrint_rebootaddr();
			break;
        case 0x0000002f:
			HTTPPrint_ddns_service(0);
			break;
        case 0x00000030:
			HTTPPrint_ddns_service(1);
			break;
        case 0x00000031:
			HTTPPrint_ddns_service(2);
			break;
        case 0x00000033:
			HTTPIncFile((ROM BYTE*)"header.inc");
			break;
        case 0x00000042:
			HTTPPrint_read_comm(0);
			break;
        case 0x00000043:
			HTTPPrint_read_comm(1);
			break;
        case 0x00000044:
			HTTPPrint_read_comm(2);
			break;
        case 0x00000045:
			HTTPPrint_write_comm(0);
			break;
        case 0x00000046:
			HTTPPrint_write_comm(1);
			break;
        case 0x00000047:
			HTTPPrint_write_comm(2);
			break;
        case 0x00000048:
			HTTPPrint_smtps_en();
			break;
        case 0x00000049:
			HTTPPrint_snmp_en();
			break;
        case 0x0000004a:
			HTTPPrint_an(0);
			break;
        case 0x0000004b:
			HTTPPrint_an(1);
			break;
        case 0x0000004c:
			HTTPPrint_an(2);
			break;
        case 0x0000004d:
			HTTPPrint_an(3);
			break;
        case 0x0000004e:
			HTTPPrint_es1(0);
			break;
        case 0x0000004f:
			HTTPPrint_es1(1);
			break;
        case 0x00000050:
			HTTPPrint_es1(2);
			break;
        case 0x00000051:
			HTTPPrint_es1(3);
			break;
        case 0x00000052:
			HTTPPrint_an(4);
			break;
        case 0x00000053:
			HTTPPrint_an(5);
			break;
        case 0x00000054:
			HTTPPrint_an(6);
			break;
        case 0x00000055:
			HTTPPrint_an(7);
			break;
		default:
			// Output notification for undefined values
			TCPPutROMArray(sktHTTP, (ROM BYTE*)"!DEF", 4);
	}

	return;
}

void HTTPPrint_(void)
{
	TCPPut(sktHTTP, '~');
	return;
}

#endif

#endif
/**************************************************************
* HTTPPrint.h
* Provides callback headers and resolution for user’s custom
* HTTP Application.
*
* This file is automatically generated by the MPFS Utility
* ALL MODIFICATIONS WILL BE OVERWRITTEN BY THE MPFS GENERATOR
**************************************************************/

#ifndef __HTTPPRINT_H
#define __HTTPPRINT_H

#include “TCPIP Stack/TCPIP.h”

#if defined(STACK_USE_HTTP2_SERVER)

extern HTTP_STUB httpStubs[MAX_HTTP_CONNECTIONS];
extern BYTE curHTTPID;

void HTTPPrint(DWORD callbackID);
void HTTPPrint_hellomsg(void);
void HTTPPrint_cookiename(void);
void HTTPPrint_(void);
void HTTPPrint_builddate(void);
void HTTPPrint_led(WORD);
void HTTPPrint_lcdtext(void);
void HTTPPrint_ledSelected(WORD,WORD);
void HTTPPrint_version(void);
void HTTPPrint_btn(WORD);
void HTTPPrint_uploadedmd5(void);
void HTTPPrint_status_ok(void);
void HTTPPrint_ddns_status(void);
void HTTPPrint_ddns_status_msg(void);
void HTTPPrint_ddns_user(void);
void HTTPPrint_ddns_pass(void);
void HTTPPrint_ddns_host(void);
void HTTPPrint_status_fail(void);
void HTTPPrint_config_mac(void);
void HTTPPrint_config_hostname(void);
void HTTPPrint_config_dhcpchecked(void);
void HTTPPrint_config_ip(void);
void HTTPPrint_config_gw(void);
void HTTPPrint_config_subnet(void);
void HTTPPrint_config_dns1(void);
void HTTPPrint_config_dns2(void);
void HTTPPrint_reboot(void);
void HTTPPrint_rebootaddr(void);
void HTTPPrint_ddns_service(WORD);
void HTTPPrint_read_comm(WORD);
void HTTPPrint_write_comm(WORD);
void HTTPPrint_smtps_en(void);
void HTTPPrint_snmp_en(void);
void HTTPPrint_an(WORD);
void HTTPPrint_es1(WORD);

void HTTPPrint(DWORD callbackID)
{
switch(callbackID)
{
case 0×00000001:
HTTPPrint_hellomsg();
break;
case 0×00000002:
HTTPIncFile((ROM BYTE*)”footer.inc”);
break;
case 0×00000003:
HTTPPrint_cookiename();
break;
case 0×00000004:
HTTPPrint_();
break;
case 0×00000005:
HTTPPrint_builddate();
break;
case 0×00000006:
HTTPPrint_led(7);
break;
case 0×00000007:
HTTPPrint_led(6);
break;
case 0×00000008:
HTTPPrint_led(5);
break;
case 0×00000009:
HTTPPrint_led(4);
break;
case 0x0000000a:
HTTPPrint_led(3);
break;
case 0x0000000b:
HTTPPrint_led(2);
break;
case 0x0000000c:
HTTPPrint_led(1);
break;
case 0x0000000d:
HTTPPrint_lcdtext();
break;
case 0x0000000e:
HTTPPrint_ledSelected(4,TRUE);
break;
case 0x0000000f:
HTTPPrint_ledSelected(4,FALSE);
break;
case 0×00000010:
HTTPPrint_ledSelected(3,TRUE);
break;
case 0×00000011:
HTTPPrint_ledSelected(3,FALSE);
break;
case 0×00000012:
HTTPPrint_ledSelected(2,TRUE);
break;
case 0×00000013:
HTTPPrint_ledSelected(2,FALSE);
break;
case 0×00000014:
HTTPPrint_ledSelected(1,TRUE);
break;
case 0×00000015:
HTTPPrint_ledSelected(1,FALSE);
break;
case 0×00000016:
HTTPPrint_version();
break;
case 0×00000017:
HTTPPrint_led(0);
break;
case 0×00000018:
HTTPPrint_btn(0);
break;
case 0×00000019:
HTTPPrint_btn(1);
break;
case 0x0000001a:
HTTPPrint_btn(2);
break;
case 0x0000001b:
HTTPPrint_btn(3);
break;
case 0x0000001d:
HTTPPrint_uploadedmd5();
break;
case 0x0000001e:
HTTPPrint_status_ok();
break;
case 0x0000001f:
HTTPPrint_ddns_status();
break;
case 0×00000020:
HTTPPrint_ddns_status_msg();
break;
case 0×00000021:
HTTPPrint_ddns_user();
break;
case 0×00000022:
HTTPPrint_ddns_pass();
break;
case 0×00000023:
HTTPPrint_ddns_host();
break;
case 0×00000024:
HTTPPrint_status_fail();
break;
case 0×00000025:
HTTPPrint_config_mac();
break;
case 0×00000026:
HTTPPrint_config_hostname();
break;
case 0×00000027:
HTTPPrint_config_dhcpchecked();
break;
case 0×00000028:
HTTPPrint_config_ip();
break;
case 0×00000029:
HTTPPrint_config_gw();
break;
case 0x0000002a:
HTTPPrint_config_subnet();
break;
case 0x0000002b:
HTTPPrint_config_dns1();
break;
case 0x0000002c:
HTTPPrint_config_dns2();
break;
case 0x0000002d:
HTTPPrint_reboot();
break;
case 0x0000002e:
HTTPPrint_rebootaddr();
break;
case 0x0000002f:
HTTPPrint_ddns_service(0);
break;
case 0×00000030:
HTTPPrint_ddns_service(1);
break;
case 0×00000031:
HTTPPrint_ddns_service(2);
break;
case 0×00000033:
HTTPIncFile((ROM BYTE*)”header.inc”);
break;
case 0×00000042:
HTTPPrint_read_comm(0);
break;
case 0×00000043:
HTTPPrint_read_comm(1);
break;
case 0×00000044:
HTTPPrint_read_comm(2);
break;
case 0×00000045:
HTTPPrint_write_comm(0);
break;
case 0×00000046:
HTTPPrint_write_comm(1);
break;
case 0×00000047:
HTTPPrint_write_comm(2);
break;
case 0×00000048:
HTTPPrint_smtps_en();
break;
case 0×00000049:
HTTPPrint_snmp_en();
break;
case 0x0000004a:
HTTPPrint_an(0);
break;
case 0x0000004b:
HTTPPrint_an(1);
break;
case 0x0000004c:
HTTPPrint_an(2);
break;
case 0x0000004d:
HTTPPrint_an(3);
break;
case 0x0000004e:
HTTPPrint_es1(0);
break;
case 0x0000004f:
HTTPPrint_es1(1);
break;
case 0×00000050:
HTTPPrint_es1(2);
break;
case 0×00000051:
HTTPPrint_es1(3);
break;
case 0×00000052:
HTTPPrint_an(4);
break;
case 0×00000053:
HTTPPrint_an(5);
break;
case 0×00000054:
HTTPPrint_an(6);
break;
case 0×00000055:
HTTPPrint_an(7);
break;
default:
// Output notification for undefined values
TCPPutROMArray(sktHTTP, (ROM BYTE*)”!DEF”, 4);
}

return;
}

void HTTPPrint_(void)
{
TCPPut(sktHTTP, ‘~’);
return;
}

#endif

#endif

Updated 20101225