Better RAM usage

[rkompass]

Hello @wdyichen and other w60x experts,

we made good progress on the w600 Micropython port again, recently.

I would like to ask you for further information and help:

Is it possible to get the following:

• The sources of the secboot code,
• the sources of the ROM,
• the sources of the WLAN library?

Currently we have the compiled wlan archive only.

With the Micropython heap now at >= 100K it should still be possible to extend it more.
I'm looking for ways to free and use / add a RAM section within the upper 128 K RAM area.
My preliminary tests showed that the first 64 K of that area are sometimes changed, but it is not clear, how / by what activity. Changing the RAM contents there seemed to not hurt the system so far.
Is this space used by the FreeRTOS? Or is it used by the hardware crypto mechanism?

From wm_ram_config.h:

#define MASTER_SPI_DMA_ADDR     0x20038000UL
#define SLAVE_HSPI_SDIO_ADDR    0x2003A000UL  /*High speed SPI or SDIO buffer to exchange data*/
#define WIFI_MEM_START_ADDR	0x2003C000UL  /*Wi-Fi use buffer to exchange data*/

The start of the thus declared areas are at 64K in the upper 128K area. What is below that (the above mentioned 64 K)?

Is the high speed spi slave module used currently? It seems it is only used in connection with wlan.
Is it used for the flash function with the 2MB flash variant?
Can it be deactivated and the RAM space be free'd perhaps?

Thank you very much.

[wdyichen]

For some reason, the code of secboot can be published, rom and wlan cannot. What do you need the secboot code for?

The w60x has 160+128kb of ram. The next 128kb is DRAM (doesn't support instruction fetching, only data), so it can't be used for the freertos task stack.

WIFI_MEM_START_ADDR A segment of memory that is used by the wifi driver in the wlan lib.
If HSPI is enabled, the previous memory is used; otherwise, it is idle. The code for this memory used by hspi is open source and available in the sdk.

[rkompass]

I'm just now trying to deactivate the HSPI.
Apart from the deactivation with
#define TLS_CONFIG_HS_SPI CFG_OFF /*High Speed SPI*/
in wm_config.h that involves adapting machine/machine_spi.c. Which I will try soon.

I understand that this would free the 8KB from 0x2003A000 to 0x2003C000.

If HSPI is enabled, the previous memory is used; otherwise, it is idle.

Do you mean by this that also the 8KB MASTER_SPI_DMA_ADDR space from 0x20038000 to 0x2003A000 is free then?
Or are there other procedures in the SDK that use DMA and this space?
And do you mean that also the 64KB from 0x20030000 to 0x20038000 are unused?
Or is there a known use of that space?

My/our interest in ROM and WLAN sources is basically for answering these questions.

If we could have free DRAM areas there it would allow for adding space to the Micropython heap.
Instruction fetching would not be necessary as MP is a bytecode interpreter. The instructions would be fetched from the flashed MP image.
We could build in a control that no asm-thumb/native/viper code is placed in the DRAM area. This should be sufficient.
On the other hand it would potentially allow Micropython to extend its available heap considerably, which is important for running larger programs / storing more data.

[rkompass]

Hi @wdyichen,

thanks for responding so rapidly last time.
Are you (still) working at Winner?
Do you know if a Micropython port of the w80x family is planned?
If yes, many details of the current port could be taken over, couldn't they?
I saw that the register organization of that chip is almost the same.
Seems like just the computing core has changed and Bluetooth added.

[wdyichen]

I'm sorry, but I've been very busy these two days.

The w600's ram consists of two blocks, which can be understood as 160K iam and 128k dram, so a total of 288kb. The first 224kb is used as the stack and heap of the system, and the last 64kb is used by the spi and wifi drivers. The next 64k can be reduced further by adding it to the heap, as follows:

1. In the WMSDK\Tools\GNU\link_w600_1m.ld file, the total memory used by the heap and stack is currently defined by RAM from 0x20000000-0x20038000, a total of 224kb.

2. In the WMSDK\Include\wm_ram_config.h file:

   • Declares the memory used by the master spi driver in the range 0x20038000-0x20039FFF (see WMSDK\Platform\Drivers\spi\wm_hostspi.c, which is open source).
   • The memory used by the slave spi driver is declared in the range 0x2003A000-0x2003BFFF (this can be seen in the WMSDK\Platform\Drivers\hspi\wm_hspi.c file, which is open source).
   • Declares the memory used by the wifi driver in the range 0x2003C000-0x20047FFF (this is used in wlan lib and is not open source).

As mentioned earlier, 0x20000000-0x2003BFFF can be used as stack and heap memory if the spi driver is not used.

Even if wifi performance is not considered, it is possible to further reduce the ram used by wifi. In WMSDK\Platform\Sys\wm_main.c, find the place to call the tls_wifi_mem_cfg function. Suppose we change the start address 0xn defined by the WIFI_MEM_START_ADDR macro and change the number of txcnt and rxcnt (for example, 1 and 3). Then, everything between 0x20000000 and 0xn can be used.
Rules to be satisfied between WIFI_MEM_START_ADDR, txcnt, and rxcnt: "WIFI_MEM_START_ADDR + 3208 + (3352 * txcnt) + (3248 * rxcnt)" is in the range 0x20028000-0x20047FFF(probably just no more than 0x20047FFF).

I saw that the register organization of that chip is almost the same.
Seems like just the computing core has changed and Bluetooth added.

You are right, the cpu of w800 series has changed and bt\ble has been added.

[wdyichen]

As mentioned before with w80x, I found an mpy of the w80x series ported by another developer on gitee.

https://gitee.com/haorongMango/micropython-w806

The port location is ports/xt804, which is a bit of a misnomer because xt804 is a cpu name, not an soc name.

[rkompass]

Thank you, very interesting.
And the heap info is very helpful.

I will now be absent for a few days then try to reduce machine spi.c to use only ordinary SPI.
Robert agrees that we do not need the HSPI. (Does HSPI denote high speed SPI or Host SPI?).
Anyway 12 MB/s of ordinary SPI are sufficient for Micropython.
I assume that means that we have no slave mode in SPI anymore. Which would be o.k., as the w600 would be a device controlling others like external LCD, ADC, not the other way round.
Then I/we will have to test that SPI.

After that we can make a new attempt at improving the RAM:
We should be able to use the 8 KB of HSPI RAM area, and perhaps some more from WIFI.
Also we should be able to abolish the FTP process and use its stack space of 8K RAM.
The reason is: Now, that the socket.poll() and socket.select() work, ftp may be used from within MP. I tested the latter.
After that we have at least >16 KB available and we should make a more detailed overview of the RAM usage.
One question then arising is how much heap is needed for the OS to work, assuming Wifi, perhaps TLS are used.
Could it be that the heap is not used at all anymore, as all special functionality has its own memory areas reserved beforehand anyway?
Your help will be needed and appreciated then, I suppose.

[wdyichen]

Does HSPI denote high speed SPI or Host SPI?

The master spi has a maximum clock of 20MHZ,
slave spi has a maximum clock of 50MHZ,
so hspi refers to slave spi.

Could it be that the heap is not used at all anymore, as all special functionality has its own memory areas reserved beforehand anyway?

Currently, heap memory is used in two ways. One is freertos heap2 configured with configTOTAL_HEAP_SIZE, which is pvPortMalloc/vPortFree. This is to meet the freertos task\sem\queue\timer dynamic memory request; The other uses the tls_mem_alloc/tls_mem_free scheme, that is malloc/free.
The reason for this distinction is that the task stack can only use iram. To prevent freertos from using malloc to request dram, iram memory configTOTAL_HEAP_SIZE is fixed for freertos kernel to use. Other applications can use malloc normally.
For example, in wlan lib, a certain amount of memory is still required to be used by malloc. I cannot evaluate the minimum amount, but I suggest that it can be gradually reduced, and it can be verified that it can normally use wifi networking communication.