Before logging an issue, please update to the latest release of Visual Micro from the Downloads Page.

When Logging a Support Issue in the Forum, please ensure you have also:-

  • Enabled vMicro > Compiler > Show Build Properties
  • Re-Compile your program with these settings enabled
 
Save the new Output to a Text File and....
  • Click the Reply button and attach as .txt file OR
  • Click here to Email us with the file attached, and a link to your post
Support requests without the output above may be impossible to answer, so please help us to help you
 
Page Index Toggle Pages: 1 Send TopicPrint
Hot Topic (More than 8 Replies) Program Size And Partition Table (Read 14615 times)
The_Specialist
Full Member
***
Offline


Posts: 227
Joined: Jul 1st, 2020
Program Size And Partition Table
Jun 16th, 2021 at 10:11am
Print Post  
I have an esp32 wroom with 4mb of flash memory. I am stuck with my otta program that consumes a lot of program size. I look in the partition table and found out I have room for it. Now I changed the partition table and the board.txt but It still gives me the same size. see below


Program size: 990.454 bytes (used 76% of a 1.310.720 byte maximum) (155,30 secs)
Minimum Memory Usage: 50984 bytes (16% of a 327680 byte maximum)

# Name,   Type, SubType, Offset,  Size, Flags                  
nvs,      data, nvs,     0x9000,  0x5000,                  
otadata,  data, ota,     0xe000,  0x2000,                  
app0,     app,  ota_0,   0x10000, 0x3000000,                  
app1,     app,  ota_1,   0x150000,0x3000000,                  
spiffs,   data, spiffs,  0x3150000,0x170000,                  

« Last Edit: Jun 16th, 2021 at 10:11am by The_Specialist »  

Please Register or Login to the Forum to see File Attachments
Back to top
 
IP Logged
 
The_Specialist
Full Member
***
Offline


Posts: 227
Joined: Jul 1st, 2020
Re: Program Size And Partition Table
Reply #1 - Jun 16th, 2021 at 10:45am
Print Post  
esp32doit-devkit-v1.name=DOIT ESP32 DEVKIT V1

esp32doit-devkit-v1.upload.tool=esptool_py
esp32doit-devkit-v1.upload.maximum_size=1310720
esp32doit-devkit-v1.upload.maximum_data_size=327680
esp32doit-devkit-v1.upload.wait_for_upload_port=true

esp32doit-devkit-v1.serial.disableDTR=true
esp32doit-devkit-v1.serial.disableRTS=true

esp32doit-devkit-v1.build.mcu=esp32
esp32doit-devkit-v1.build.core=esp32
esp32doit-devkit-v1.build.variant=doitESP32devkitV1
esp32doit-devkit-v1.build.board=ESP32_DEV

esp32doit-devkit-v1.build.f_cpu=240000000L
esp32doit-devkit-v1.build.flash_mode=dio
esp32doit-devkit-v1.build.flash_size=4MB
esp32doit-devkit-v1.build.boot=dio
esp32doit-devkit-v1.build.partitions=default_Modified_V4
esp32doit-devkit-v1.build.defines=

esp32doit-devkit-v1.menu.FlashFreq.80=80MHz
esp32doit-devkit-v1.menu.FlashFreq.80.build.flash_freq=80m
esp32doit-devkit-v1.menu.FlashFreq.40=40MHz
esp32doit-devkit-v1.menu.FlashFreq.40.build.flash_freq=40m

esp32doit-devkit-v1.menu.UploadSpeed.921600=921600
esp32doit-devkit-v1.menu.UploadSpeed.921600.upload.speed=921600
esp32doit-devkit-v1.menu.UploadSpeed.115200=115200
esp32doit-devkit-v1.menu.UploadSpeed.115200.upload.speed=115200
esp32doit-devkit-v1.menu.UploadSpeed.256000.windows=256000
esp32doit-devkit-v1.menu.UploadSpeed.256000.upload.speed=256000
esp32doit-devkit-v1.menu.UploadSpeed.230400.windows.upload.speed=256000
esp32doit-devkit-v1.menu.UploadSpeed.230400=230400
esp32doit-devkit-v1.menu.UploadSpeed.230400.upload.speed=230400
esp32doit-devkit-v1.menu.UploadSpeed.460800.linux=460800
esp32doit-devkit-v1.menu.UploadSpeed.460800.macosx=460800
esp32doit-devkit-v1.menu.UploadSpeed.460800.upload.speed=460800
esp32doit-devkit-v1.menu.UploadSpeed.512000.windows=512000
esp32doit-devkit-v1.menu.UploadSpeed.512000.upload.speed=512000

esp32doit-devkit-v1.menu.DebugLevel.none=None
esp32doit-devkit-v1.menu.DebugLevel.none.build.code_debug=0
esp32doit-devkit-v1.menu.DebugLevel.error=Error
esp32doit-devkit-v1.menu.DebugLevel.error.build.code_debug=1
esp32doit-devkit-v1.menu.DebugLevel.warn=Warn
esp32doit-devkit-v1.menu.DebugLevel.warn.build.code_debug=2
esp32doit-devkit-v1.menu.DebugLevel.info=Info
esp32doit-devkit-v1.menu.DebugLevel.info.build.code_debug=3
esp32doit-devkit-v1.menu.DebugLevel.debug=Debug
esp32doit-devkit-v1.menu.DebugLevel.debug.build.code_debug=4

  
Back to top
 
IP Logged
 
The_Specialist
Full Member
***
Offline


Posts: 227
Joined: Jul 1st, 2020
Re: Program Size And Partition Table
Reply #2 - Jun 16th, 2021 at 10:48am
Print Post  
Am I correct if I say I have 3,15M in use from the 4mb I have available?
  
Back to top
 
IP Logged
 
Simon@Visual Micro
Administrator
*****
Offline


Posts: 2145
Joined: Feb 13th, 2019
Re: Program Size And Partition Table
Reply #3 - Jun 16th, 2021 at 11:15am
Print Post  
Thanks for the report.

You can change the Upload sizes which are defined for the board either in board.txt within the core, or by adding a Local Board.txt (vMicro > Add Code > Add Local Board.txt), and add the below entries and modify as needed:

Code
Select All
# Modify Maximum sizes to align with custom partition tables
upload.maximum_size=1310720
upload.maximum_data_size=327680
 



An alternative to override the partitions.csv is to add the file to the root of your project, named "partitions.csv", and it will be used instead of the one defined in the properties when building.

Note, the partition table specified in your first post has an overlap:
Code
Select All
CSV Error*: Partitions overlap. Partition at line 5 sets offset 0x150000. Previous partition ends 0x3010000 

  
Back to top
 
IP Logged
 
The_Specialist
Full Member
***
Offline


Posts: 227
Joined: Jul 1st, 2020
Re: Program Size And Partition Table
Reply #4 - Jun 16th, 2021 at 1:49pm
Print Post  
I losed the original file on this location. Do you have one?

partitions

It was not the good location. Which one do you mean?

C:\Program Files (x86)\Arduino\java\bin>if exist "E:\Documents\Visual Studio 2019\Projects\Arduino\Aquarium Lights ESP\Aquarium Lights ESP\partitions.csv" copy /y "E:\Documents\Visual Studio 2019\Projects\Arduino\Aquarium Lights ESP\Aquarium Lights ESP\partitions.csv" "E:\AppData\Local\Temp\VMBuilds\Aquarium Lights ESP\esp32_esp32doit-devkit-v1\Debug\partitions.csv" 
C:\Program Files (x86)\Arduino\java\bin>if not exist "E:\AppData\Local\Temp\VMBuilds\Aquarium Lights ESP\esp32_esp32doit-devkit-v1\Debug\partitions.csv" copy tools\partitions\default.csv" "E:\AppData\Local\Temp\VMBuilds\Aquarium Lights ESP\esp32_esp32doit-devkit-v1\Debug\partitions.csv" 
  
Back to top
 
IP Logged
 
The_Specialist
Full Member
***
Offline


Posts: 227
Joined: Jul 1st, 2020
Re: Program Size And Partition Table
Reply #5 - Jun 16th, 2021 at 1:58pm
Print Post  
Thanks for your reply.

I have found the original exported file in the debug folder. Soo no need to send it to me. Is that the one I need too modify?

Kind regards

  
Back to top
 
IP Logged
 
The_Specialist
Full Member
***
Offline


Posts: 227
Joined: Jul 1st, 2020
Re: Program Size And Partition Table
Reply #6 - Jun 16th, 2021 at 2:07pm
Print Post  
That's the good one.

I changed that file and the error tels me I am changing the right one. But I don't understand it. I don't see an overlap anymore

CSV Error*: Partitions overlap. Partition at line 5 sets offset 0x1017000. Previous partition ends 0x2016000

nvs,      data, nvs,     0x9000,  0x5000,
otadata,  data, ota,     0xe000,  0x2000,
app0,     app,  ota_0,   0x16000, 0x2000000,
app1,     app,  ota_1,   0x1017000,0x2000000,
spiffs,   data, spiffs,  0x3018000,0x140000,
  
Back to top
 
IP Logged
 
Simon@Visual Micro
Administrator
*****
Offline


Posts: 2145
Joined: Feb 13th, 2019
Re: Program Size And Partition Table
Reply #7 - Jun 16th, 2021 at 2:28pm
Print Post  
I meant if you copy your CSV file from the core, to the location below (change its name to partitions.csv) it will use this over the one from the core, so when you update the core you wont lose these changes each time:
Code
Select All
E:\Documents\Visual Studio 2019\Projects\Arduino\Aquarium Lights ESP\Aquarium Lights ESP\partitions.csv 



Reviewing the others in the core will help with this, and the message does hint towards the solution.

The below is my understanding of this and would use one of the standard partition tables if it has the features you require for simplicity/safety.

"CSV Error*: Partitions overlap. Partition at line 5 sets offset 0x1017000. Previous partition ends 0x2016000"

So the below partition ends a 0x2016000, on line 4 of the file:
app0,     app,  ota_0,   0x16000, 0x2000000,

Line 5 has its start as the same position, which needs to be amended to the offset of from the message:
app1,     app,  ota_1,   0x2016000 ,0x2000000,

Which will then overlap with the spiffs partition, so that would need amending in the same way.   

There may be other alignment issues if the partitions are not contiguous (e,g, app0 does not have offset 0x10000 so would also need changing, and then all later partitions aligning again).


  
Back to top
 
IP Logged
 
The_Specialist
Full Member
***
Offline


Posts: 227
Joined: Jul 1st, 2020
Re: Program Size And Partition Table
Reply #8 - Jun 16th, 2021 at 4:04pm
Print Post  
Ok the messages of overlap are gone but now I have an alignment error.

Build log
-------------------------------------------
Compiling debug version of 'Aquarium Lights ESP' for 'DOIT ESP32 DEVKIT V1'
C:\Program Files (x86)\Arduino\java\bin>if exist "E:\Documents\Visual Studio 2019\Projects\Arduino\Aquarium Lights ESP\Aquarium Lights ESP\partitions.csv" copy /y "E:\Documents\Visual Studio 2019\Projects\Arduino\Aquarium Lights ESP\Aquarium Lights ESP\partitions.csv" "E:\AppData\Local\Temp\VMBuilds\Aquarium Lights ESP\esp32_esp32doit-devkit-v1\Debug\partitions.csv" 
        1 bestand(en) gekopieerd.
C:\Program Files (x86)\Arduino\java\bin>if not exist "E:\AppData\Local\Temp\VMBuilds\Aquarium Lights ESP\esp32_esp32doit-devkit-v1\Debug\partitions.csv" copy tools\partitions\default.csv" "E:\AppData\Local\Temp\VMBuilds\Aquarium Lights ESP\esp32_esp32doit-devkit-v1\Debug\partitions.csv" 
 
Partition app0 invalid*: Offset 0x18000 is not aligned to 0x10000
Program size: 990.454 bytes (used 76% of a 1.310.720 byte maximum) (55,12 secs)
Minimum Memory Usage: 50984 bytes (16% of a 327680 byte maximum)


Partition table
--------------------------------------------
nvs,      data, nvs,     0x9000,  0x5000,
otadata,  data, ota,     0xf000,  0x2000,
app0,     app,  ota_0,   0x18000, 0x2000000,
app1,     app,  ota_1,   0x2019000,0x2000000,
spiffs,   data, spiffs,  0x4020000,0x140000,

My end calculation
-----------------------------------------------
start 9000, size 5000, end 14000      
start 15000, size 2000, end 17000      
start 18000, size 2000000, end 2018000      
start 2019000, size 2000000, end 4019000      
start 4020000, size 140000, end 4160000
« Last Edit: Jun 17th, 2021 at 12:10am by The_Specialist »  
Back to top
 
IP Logged
 
The_Specialist
Full Member
***
Offline


Posts: 227
Joined: Jul 1st, 2020
Re: Program Size And Partition Table
Reply #9 - Jun 16th, 2021 at 4:24pm
Print Post  
Now I see you did warn me for this error.

Quote:
There may be other alignment issues if the partitions are not contiguous (e,g, app0 does not have offset 0x10000 so would also need changing, and then all later partitions aligning again).


I will try it.
« Last Edit: Jun 16th, 2021 at 4:24pm by The_Specialist »  
Back to top
 
IP Logged
 
The_Specialist
Full Member
***
Offline


Posts: 227
Joined: Jul 1st, 2020
Re: Program Size And Partition Table
Reply #10 - Jun 16th, 2021 at 6:13pm
Print Post  
I cann't get it to work.

All the error messages are gone. But after uploading, it keeps spinning arround in the serial monitor. If I remove the partitiontable it runs normal.

I did this...
                 
                nvs      otadata      app0      app1      spiffs
start      9000      14000      10000      230000      450000
size       5000      2000      220000      220000      10000
end      14000      16000      230000      450000      460000
                             
defauld 9000      14000      10000      150000      290000
size      5000      2000      140000      140000      170000
end      14000      16000      150000      290000      460000

The end size is the same as the defauld, So the file is not to big to upload. cann't see what the issue is.
  
Back to top
 
IP Logged
 
The_Specialist
Full Member
***
Offline


Posts: 227
Joined: Jul 1st, 2020
Re: Program Size And Partition Table
Reply #11 - Jun 16th, 2021 at 7:09pm
Print Post  
I have followed the manual that you have mention in an other topic and it works. It seems bigger now but the build log gives the same output size. strange?!

https://github.com/espressif/arduino-esp32/issues/703

Build log
--------------------
Program size: 990.454 bytes (used 76% of a 1.310.720 byte maximum) (59,62 secs)
Minimum Memory Usage: 50984 bytes (16% of a 327680 byte maximum)

Calculation
------------------------
                nvs      otadata      app0      app1      eprom      spiffs
start      9000      14000      10000      1130000      390000      391000
size      5000      2000      1120000      1120000      1000      615000
end      14000      16000      1130000      2250000      391000      1006000
  
Back to top
 
IP Logged
 
The_Specialist
Full Member
***
Offline


Posts: 227
Joined: Jul 1st, 2020
Re: Program Size And Partition Table
Reply #12 - Jun 17th, 2021 at 12:08am
Print Post  
I did nothing else anymore and now it changed...

Program size: 992.866 bytes (used 54% of a 1.835.008 byte maximum) (59,15 secs)
Minimum Memory Usage: 51000 bytes (16% of a 327680 byte maximum)
  
Back to top
 
IP Logged
 
The_Specialist
Full Member
***
Offline


Posts: 227
Joined: Jul 1st, 2020
Re: Program Size And Partition Table
Reply #13 - Oct 16th, 2021 at 12:06pm
Print Post  
Again I have problems with too small size.

if I use this table...

# Name,   Type, SubType, Offset,  Size, Flags
nvs, data, nvs, 0x9000, 0x5000,
otadata, data, ota, 0xe000, 0x2000,
app0, app, ota_0, 0x10000, 0x1E0000,
app1, app, ota_1, 0x1F0000,0x1E0000,
eeprom, data, 0x99, 0x3D0000,0x1000,
spiffs, data, spiffs, 0x3D1000,0x2F000,

What do I need to fill in here...

upload.maximum_size=?
upload.maximum_data_size=?

I did change the partition table but I still get the upload maximum size as limit.

If I look at what I did...
This is the size of app0 0x1E0000 = 1140000
is stil small than than the values in the board.txt
maximum_size=1310720 (this is the limit I get in my message)
maximum_data_size=327680 (don't know what this is)

I need this...
The size is 1338278 bytes (of a 1310720 byte maximum).
« Last Edit: Oct 16th, 2021 at 12:35pm by The_Specialist »  
Back to top
 
IP Logged
 
The_Specialist
Full Member
***
Offline


Posts: 227
Joined: Jul 1st, 2020
Re: Program Size And Partition Table
Reply #14 - Oct 16th, 2021 at 12:43pm
Print Post  
If I replace

maximum_size=1310720  with 1400000

I get it is ok after rebooting visual micro,

Program size: 1.338.278 bytes (used 96% of a 1.400.000 byte maximum) (19,98 secs)

But am I overwriting app1 partition by doing that?
  
Back to top
 
IP Logged
 
Tim@Visual Micro
Administrator
*****
Offline


Posts: 12071
Location: United Kingdom
Joined: Apr 10th, 2010
Re: Program Size And Partition Table
Reply #15 - Oct 16th, 2021 at 2:10pm
Print Post  
Quote:
I get it is ok after rebooting visual micro,


If you are altering core files you either need to click "vMicro>Compiler>Reload toolchains" or restart the IDE

Quote:
But am I overwriting app1 partition by doing that?


That question is best answered by the author of the core. The core is designed in a way that it supports partitions and therefore it is a fair question to ask the author or esp community.

Sorry can't be more helpful.

  
Back to top
WWW  
IP Logged
 
The_Specialist
Full Member
***
Offline


Posts: 227
Joined: Jul 1st, 2020
Re: Program Size And Partition Table
Reply #16 - Oct 16th, 2021 at 10:24pm
Print Post  
Did some research and looked at ...

https://www.esp32.com/viewtopic.php?f=18&t=18575 ;

I did some calculation on my own and come to the conclusion that I have a size with this partition table of 1966080. Now I need to addaped the maximum_size of 1400000  -> 1966080

My size is in hex 0x1E0000 for app0.
Soo I have in dec 1966080 = 1920kb 

This is for the example in the topic above: 
180000 ->      1572864->1536kb

Finally I understand, how to make partition tables and addapted the board.txt 

Thanks for the tip tim.

This is table I use and it works

# Name,   Type, SubType, Offset,  Size, Flags
nvs, data, nvs, 0x9000, 0x5000,
otadata, data, ota, 0xe000, 0x2000,
app0, app, ota_0, 0x10000, 0x1E0000,
app1, app, ota_1, 0x1F0000,0x1E0000,
eeprom, data, 0x99, 0x3D0000,0x1000,
spiffs, data, spiffs, 0x3D1000,0x2F000,

Program size: 1.333.062 bytes (used 68% of a 1.966.080 byte maximum)

This partition table let you use evry byte that is available.
  
Back to top
 
IP Logged
 
Tim@Visual Micro
Administrator
*****
Offline


Posts: 12071
Location: United Kingdom
Joined: Apr 10th, 2010
Re: Program Size And Partition Table
Reply #17 - Oct 16th, 2021 at 10:27pm
Print Post  
Well done and thanks for the info. It's useful.
  
Back to top
WWW  
IP Logged
 
Page Index Toggle Pages: 1
Send TopicPrint