VS Arduino
>> >> Program Size And Partition Table
https://www.visualmicro.com/forums/YaBB.pl?num=1623838272

Message started by The_Specialist on Jun 16th, 2021 at 10:11am

Title: Program Size And Partition Table
Post by The_Specialist on Jun 16th, 2021 at 10:11am
I have an esp32 wroom with 4mb of flash memory. I am stuck with my otta program that consumes a lot of program size. I look in the partition table and found out I have room for it. Now I changed the partition table and the board.txt but It still gives me the same size. see below


Program size: 990.454 bytes (used 76% of a 1.310.720 byte maximum) (155,30 secs)
Minimum Memory Usage: 50984 bytes (16% of a 327680 byte maximum)

# Name,   Type, SubType, Offset,  Size, Flags                  
nvs,      data, nvs,     0x9000,  0x5000,                  
otadata,  data, ota,     0xe000,  0x2000,                  
app0,     app,  ota_0,   0x10000, 0x3000000,                  
app1,     app,  ota_1,   0x150000,0x3000000,                  
spiffs,   data, spiffs,  0x3150000,0x170000,                  


https://www.visualmicro.com/forums/YaBB.pl?action=downloadfile;file=boards_008.txt ( 255 KB | 7 Downloads )

Title: Re: Program Size And Partition Table
Post by The_Specialist on Jun 16th, 2021 at 10:45am
esp32doit-devkit-v1.name=DOIT ESP32 DEVKIT V1

esp32doit-devkit-v1.upload.tool=esptool_py
esp32doit-devkit-v1.upload.maximum_size=1310720
esp32doit-devkit-v1.upload.maximum_data_size=327680
esp32doit-devkit-v1.upload.wait_for_upload_port=true

esp32doit-devkit-v1.serial.disableDTR=true
esp32doit-devkit-v1.serial.disableRTS=true

esp32doit-devkit-v1.build.mcu=esp32
esp32doit-devkit-v1.build.core=esp32
esp32doit-devkit-v1.build.variant=doitESP32devkitV1
esp32doit-devkit-v1.build.board=ESP32_DEV

esp32doit-devkit-v1.build.f_cpu=240000000L
esp32doit-devkit-v1.build.flash_mode=dio
esp32doit-devkit-v1.build.flash_size=4MB
esp32doit-devkit-v1.build.boot=dio
esp32doit-devkit-v1.build.partitions=default_Modified_V4
esp32doit-devkit-v1.build.defines=

esp32doit-devkit-v1.menu.FlashFreq.80=80MHz
esp32doit-devkit-v1.menu.FlashFreq.80.build.flash_freq=80m
esp32doit-devkit-v1.menu.FlashFreq.40=40MHz
esp32doit-devkit-v1.menu.FlashFreq.40.build.flash_freq=40m

esp32doit-devkit-v1.menu.UploadSpeed.921600=921600
esp32doit-devkit-v1.menu.UploadSpeed.921600.upload.speed=921600
esp32doit-devkit-v1.menu.UploadSpeed.115200=115200
esp32doit-devkit-v1.menu.UploadSpeed.115200.upload.speed=115200
esp32doit-devkit-v1.menu.UploadSpeed.256000.windows=256000
esp32doit-devkit-v1.menu.UploadSpeed.256000.upload.speed=256000
esp32doit-devkit-v1.menu.UploadSpeed.230400.windows.upload.speed=256000
esp32doit-devkit-v1.menu.UploadSpeed.230400=230400
esp32doit-devkit-v1.menu.UploadSpeed.230400.upload.speed=230400
esp32doit-devkit-v1.menu.UploadSpeed.460800.linux=460800
esp32doit-devkit-v1.menu.UploadSpeed.460800.macosx=460800
esp32doit-devkit-v1.menu.UploadSpeed.460800.upload.speed=460800
esp32doit-devkit-v1.menu.UploadSpeed.512000.windows=512000
esp32doit-devkit-v1.menu.UploadSpeed.512000.upload.speed=512000

esp32doit-devkit-v1.menu.DebugLevel.none=None
esp32doit-devkit-v1.menu.DebugLevel.none.build.code_debug=0
esp32doit-devkit-v1.menu.DebugLevel.error=Error
esp32doit-devkit-v1.menu.DebugLevel.error.build.code_debug=1
esp32doit-devkit-v1.menu.DebugLevel.warn=Warn
esp32doit-devkit-v1.menu.DebugLevel.warn.build.code_debug=2
esp32doit-devkit-v1.menu.DebugLevel.info=Info
esp32doit-devkit-v1.menu.DebugLevel.info.build.code_debug=3
esp32doit-devkit-v1.menu.DebugLevel.debug=Debug
esp32doit-devkit-v1.menu.DebugLevel.debug.build.code_debug=4


Title: Re: Program Size And Partition Table
Post by The_Specialist on Jun 16th, 2021 at 10:48am
Am I correct if I say I have 3,15M in use from the 4mb I have available?

Title: Re: Program Size And Partition Table
Post by Simon@Visual Micro on Jun 16th, 2021 at 11:15am
Thanks for the report.

You can change the Upload sizes which are defined for the board either in board.txt within the core, or by adding a Local Board.txt (vMicro > Add Code > Add Local Board.txt), and add the below entries and modify as needed:


Code (]
# Modify Maximum sizes to align with custom partition tables
upload.maximum_size=1310720
upload.maximum_data_size=327680
[/code):



An alternative to override the partitions.csv is to add the file to the root of your project, named "partitions.csv", and it will be used instead of the one defined in the properties when building.

Note, the partition table specified in your first post has an overlap:
[code]CSV Error*: Partitions overlap. Partition at line 5 sets offset 0x150000. Previous partition ends 0x3010000

Title: Re: Program Size And Partition Table
Post by The_Specialist on Jun 16th, 2021 at 1:49pm
I losed the original file on this location. Do you have one?

C:\Users\user\AppData\Local\Arduino15\packages\esp32\hardware\esp32\1.0.6\tools\partitions

It was not the good location. Which one do you mean?

C:\Program Files (x86)\Arduino\java\bin>if exist "E:\Documents\Visual Studio 2019\Projects\Arduino\Aquarium Lights ESP\Aquarium Lights ESP\partitions.csv" copy /y "E:\Documents\Visual Studio 2019\Projects\Arduino\Aquarium Lights ESP\Aquarium Lights ESP\partitions.csv" "E:\AppData\Local\Temp\VMBuilds\Aquarium Lights ESP\esp32_esp32doit-devkit-v1\Debug\partitions.csv"
C:\Program Files (x86)\Arduino\java\bin>if not exist "E:\AppData\Local\Temp\VMBuilds\Aquarium Lights ESP\esp32_esp32doit-devkit-v1\Debug\partitions.csv" copy "C:\Users\Dave2Moon\AppData\Local\arduino15\packages\esp32\hardware\esp32\1.0.6\tools\partitions\default.csv" "E:\AppData\Local\Temp\VMBuilds\Aquarium Lights ESP\esp32_esp32doit-devkit-v1\Debug\partitions.csv"

Title: Re: Program Size And Partition Table
Post by The_Specialist on Jun 16th, 2021 at 1:58pm
Thanks for your reply.

I have found the original exported file in the debug folder. Soo no need to send it to me. Is that the one I need too modify?

Kind regards


Title: Re: Program Size And Partition Table
Post by The_Specialist on Jun 16th, 2021 at 2:07pm
That's the good one.

I changed that file and the error tels me I am changing the right one. But I don't understand it. I don't see an overlap anymore

CSV Error*: Partitions overlap. Partition at line 5 sets offset 0x1017000. Previous partition ends 0x2016000

nvs,      data, nvs,     0x9000,  0x5000,
otadata,  data, ota,     0xe000,  0x2000,
app0,     app,  ota_0,   0x16000, 0x2000000,
app1,     app,  ota_1,   0x1017000,0x2000000,
spiffs,   data, spiffs,  0x3018000,0x140000,

Title: Re: Program Size And Partition Table
Post by Simon@Visual Micro on Jun 16th, 2021 at 2:28pm
I meant if you copy your CSV file from the core, to the location below (change its name to partitions.csv) it will use this over the one from the core, so when you update the core you wont lose these changes each time:
[code]E:\Documents\Visual Studio 2019\Projects\Arduino\Aquarium Lights ESP\Aquarium Lights ESP\partitions.csv[/code]

Reviewing the others in the core will help with this, and the message does hint towards the solution.

The below is my understanding of this and would use one of the standard partition tables if it has the features you require for simplicity/safety.

"CSV Error*: Partitions overlap. Partition at line 5 sets offset 0x1017000. Previous partition ends 0x2016000"

So the below partition ends a 0x2016000, on line 4 of the file:
app0,     app,  ota_0,   0x16000, 0x2000000,

Line 5 has its start as the same position, which needs to be amended to the offset of from the message:
app1,     app,  ota_1,   0x2016000 ,0x2000000,

Which will then overlap with the spiffs partition, so that would need amending in the same way. 

There may be other alignment issues if the partitions are not contiguous (e,g, app0 does not have offset 0x10000 so would also need changing, and then all later partitions aligning again).



Title: Re: Program Size And Partition Table
Post by The_Specialist on Jun 16th, 2021 at 4:04pm
Ok the messages of overlap are gone but now I have an alignment error.

Build log
-------------------------------------------
Compiling debug version of 'Aquarium Lights ESP' for 'DOIT ESP32 DEVKIT V1'
C:\Program Files (x86)\Arduino\java\bin>if exist "E:\Documents\Visual Studio 2019\Projects\Arduino\Aquarium Lights ESP\Aquarium Lights ESP\partitions.csv" copy /y "E:\Documents\Visual Studio 2019\Projects\Arduino\Aquarium Lights ESP\Aquarium Lights ESP\partitions.csv" "E:\AppData\Local\Temp\VMBuilds\Aquarium Lights ESP\esp32_esp32doit-devkit-v1\Debug\partitions.csv"
        1 bestand(en) gekopieerd.
C:\Program Files (x86)\Arduino\java\bin>if not exist "E:\AppData\Local\Temp\VMBuilds\Aquarium Lights ESP\esp32_esp32doit-devkit-v1\Debug\partitions.csv" copy "C:\Users\Dave2Moon\AppData\Local\arduino15\packages\esp32\hardware\esp32\1.0.6\tools\partitions\default.csv" "E:\AppData\Local\Temp\VMBuilds\Aquarium Lights ESP\esp32_esp32doit-devkit-v1\Debug\partitions.csv"

Partition app0 invalid*: Offset 0x18000 is not aligned to 0x10000
Program size: 990.454 bytes (used 76% of a 1.310.720 byte maximum) (55,12 secs)
Minimum Memory Usage: 50984 bytes (16% of a 327680 byte maximum)


Partition table
--------------------------------------------
nvs,      data, nvs,     0x9000,  0x5000,
otadata,  data, ota,     0xf000,  0x2000,
app0,     app,  ota_0,   0x18000, 0x2000000,
app1,     app,  ota_1,   0x2019000,0x2000000,
spiffs,   data, spiffs,  0x4020000,0x140000,

My end calculation
-----------------------------------------------
start 9000, size 5000, end 14000      
start 15000, size 2000, end 17000      
start 18000, size 2000000, end 2018000      
start 2019000, size 2000000, end 4019000      
start 4020000, size 140000, end 4160000

Title: Re: Program Size And Partition Table
Post by The_Specialist on Jun 16th, 2021 at 4:24pm
Now I see you did warn me for this error.


Quote:
There may be other alignment issues if the partitions are not contiguous (e,g, app0 does not have offset 0x10000 so would also need changing, and then all later partitions aligning again).


I will try it.

Title: Re: Program Size And Partition Table
Post by The_Specialist on Jun 16th, 2021 at 6:13pm
I cann't get it to work.

All the error messages are gone. But after uploading, it keeps spinning arround in the serial monitor. If I remove the partitiontable it runs normal.

I did this...
                 
                nvs      otadata      app0      app1      spiffs
start      9000      14000      10000      230000      450000
size       5000      2000      220000      220000      10000
end      14000      16000      230000      450000      460000
                             
defauld 9000      14000      10000      150000      290000
size      5000      2000      140000      140000      170000
end      14000      16000      150000      290000      460000

The end size is the same as the defauld, So the file is not to big to upload. cann't see what the issue is.

Title: Re: Program Size And Partition Table
Post by The_Specialist on Jun 16th, 2021 at 7:09pm
I have followed the manual that you have mention in an other topic and it works. It seems bigger now but the build log gives the same output size. strange?!

https://github.com/espressif/arduino-esp32/issues/703

Build log
--------------------
Program size: 990.454 bytes (used 76% of a 1.310.720 byte maximum) (59,62 secs)
Minimum Memory Usage: 50984 bytes (16% of a 327680 byte maximum)

Calculation
------------------------
                nvs      otadata      app0      app1      eprom      spiffs
start      9000      14000      10000      1130000      390000      391000
size      5000      2000      1120000      1120000      1000      615000
end      14000      16000      1130000      2250000      391000      1006000

Title: Re: Program Size And Partition Table
Post by The_Specialist on Jun 17th, 2021 at 12:08am
I did nothing else anymore and now it changed...

Program size: 992.866 bytes (used 54% of a 1.835.008 byte maximum) (59,15 secs)
Minimum Memory Usage: 51000 bytes (16% of a 327680 byte maximum)

Title: Re: Program Size And Partition Table
Post by The_Specialist on Oct 16th, 2021 at 12:06pm
Again I have problems with too small size.

if I use this table...

# Name,   Type, SubType, Offset,  Size, Flags
nvs, data, nvs, 0x9000, 0x5000,
otadata, data, ota, 0xe000, 0x2000,
app0, app, ota_0, 0x10000, 0x1E0000,
app1, app, ota_1, 0x1F0000,0x1E0000,
eeprom, data, 0x99, 0x3D0000,0x1000,
spiffs, data, spiffs, 0x3D1000,0x2F000,

What do I need to fill in here...

upload.maximum_size=?
upload.maximum_data_size=?

I did change the partition table but I still get the upload maximum size as limit.

If I look at what I did...
This is the size of app0 0x1E0000 = 1140000
is stil small than than the values in the board.txt
maximum_size=1310720 (this is the limit I get in my message)
maximum_data_size=327680 (don't know what this is)

I need this...
The size is 1338278 bytes (of a 1310720 byte maximum).

Title: Re: Program Size And Partition Table
Post by The_Specialist on Oct 16th, 2021 at 12:43pm
If I replace

maximum_size=1310720  with 1400000

I get it is ok after rebooting visual micro,

Program size: 1.338.278 bytes (used 96% of a 1.400.000 byte maximum) (19,98 secs)

But am I overwriting app1 partition by doing that?

Title: Re: Program Size And Partition Table
Post by Tim@Visual Micro on Oct 16th, 2021 at 2:10pm

Quote:
I get it is ok after rebooting visual micro,


If you are altering core files you either need to click "vMicro>Compiler>Reload toolchains" or restart the IDE


Quote:
But am I overwriting app1 partition by doing that?


That question is best answered by the author of the core. The core is designed in a way that it supports partitions and therefore it is a fair question to ask the author or esp community.

Sorry can't be more helpful.


Title: Re: Program Size And Partition Table
Post by The_Specialist on Oct 16th, 2021 at 10:24pm
Did some research and looked at ...

https://www.esp32.com/viewtopic.php?f=18&t=18575

I did some calculation on my own and come to the conclusion that I have a size with this partition table of 1966080. Now I need to addaped the maximum_size of 1400000  -> 1966080

My size is in hex 0x1E0000 for app0.
Soo I have in dec 1966080 = 1920kb

This is for the example in the topic above:
180000 ->      1572864->1536kb

Finally I understand, how to make partition tables and addapted the board.txt

Thanks for the tip tim.

This is table I use and it works

# Name,   Type, SubType, Offset,  Size, Flags
nvs, data, nvs, 0x9000, 0x5000,
otadata, data, ota, 0xe000, 0x2000,
app0, app, ota_0, 0x10000, 0x1E0000,
app1, app, ota_1, 0x1F0000,0x1E0000,
eeprom, data, 0x99, 0x3D0000,0x1000,
spiffs, data, spiffs, 0x3D1000,0x2F000,

Program size: 1.333.062 bytes (used 68% of a 1.966.080 byte maximum)

This partition table let you use evry byte that is available.

Title: Re: Program Size And Partition Table
Post by Tim@Visual Micro on Oct 16th, 2021 at 10:27pm
Well done and thanks for the info. It's useful.

VS Arduino » Powered by YaBB 2.6.12!
YaBB Forum Software © 2000-2024. All Rights Reserved.