Welcome, Guest. Please Login or Register
Arduino IDE for Visual Studio
 
 
When Logging a Support Issue in the Forum, please ensure you have also:-
  • Enabled vMicro > Compiler > Show Build Properties and Verbose
  • Re-Compile your program with these settings enabled
 
Save the new Output to a Text File and....
Support requests without the output above may be impossible to answer, so please help us to help you
 
  HomeSearch Tutorials & Wiki visualmicro.com LoginHelpRegister  
 
Page Index Toggle Pages: 1
Send Topic Print
ESP8266 Arduino rebuilds loads of files that weren't changed (Read 709 times)
dok
Junior Member
**
Offline



Posts: 34
ESP8266 Arduino rebuilds loads of files that weren't changed
Aug 8th, 2019 at 3:37pm
 
Each time, without any code changes whatsover, "Build project" (control-shift-B).
Every other build rebuilds loads of unchanged sources - wastes a lot of developer time, 29s in this case.
Back to top
 

Documents.zip (35 KB | 10 )
 
IP Logged
 
Visual Micro
Administrator
*****
Offline



Posts: 10623
United Kingdom
Re: ESP8266 Arduino rebuilds loads of files that weren't changed
Reply #1 - Aug 8th, 2019 at 3:48pm
 
I can't reproduce the problem. Ae you making any changes at all between builds?
Back to top
 
WWW  
IP Logged
 
Visual Micro
Administrator
*****
Offline



Posts: 10623
United Kingdom
Re: ESP8266 Arduino rebuilds loads of files that weren't changed
Reply #2 - Aug 8th, 2019 at 4:04pm
 
Please close the ide and open regedit and rename hkey_current_user\software\visual micro.

Then restart the ide add the arduino 1.6/1.8 program location to IDE Locations. Then open your project and try this again.

This will ensure you are using the visual micro default cache settings.

Thanks
Back to top
 
WWW  
IP Logged
 
dok
Junior Member
**
Offline



Posts: 34
Re: ESP8266 Arduino rebuilds loads of files that weren't changed
Reply #3 - Aug 8th, 2019 at 4:13pm
 
I hope this doesn't lose my license?
Back to top
 
 
IP Logged
 
dok
Junior Member
**
Offline



Posts: 34
Re: ESP8266 Arduino rebuilds loads of files that weren't changed
Reply #4 - Aug 8th, 2019 at 4:20pm
 
First time, "Rebuild Solution":
Code:
Compiling 'blinkbuttonandweb' for 'LOLIN(WEMOS) D1 R2 & mini'
Creating BIN file "C:\Users\dok\AppData\Local\Temp\VMBuilds\blinkbuttonandweb\esp8266com_d1_mini\Release/blinkbuttonandweb.ino.bin" using "C:\Users\dok\AppData\Local\Temp\VMBuilds\blinkbuttonandweb\esp8266com_d1_mini\Release/blinkbuttonandweb.ino.elf"
Program size: 320,324 bytes (used 31% of a 1,044,464 byte maximum) (56.89 secs)
Minimum Memory Usage: 28568 bytes (35% of a 81920 byte maximum)
 



Second time, no changes, control-shift-B:
Code:
Compiling 'blinkbuttonandweb' for 'LOLIN(WEMOS) D1 R2 & mini'
Creating BIN file "C:\Users\dok\AppData\Local\Temp\VMBuilds\blinkbuttonandweb\esp8266com_d1_mini\Release/blinkbuttonandweb.ino.bin" using "C:\Users\dok\AppData\Local\Temp\VMBuilds\blinkbuttonandweb\esp8266com_d1_mini\Release/blinkbuttonandweb.ino.elf"
Program size: 320,324 bytes (used 31% of a 1,044,464 byte maximum) (2.51 secs)
Minimum Memory Usage: 28568 bytes (35% of a 81920 byte maximum)
 



Third time, no changes, control-shift-B:
Code:
Compiling 'blinkbuttonandweb' for 'LOLIN(WEMOS) D1 R2 & mini'
Creating BIN file "C:\Users\dok\AppData\Local\Temp\VMBuilds\blinkbuttonandweb\esp8266com_d1_mini\Release/blinkbuttonandweb.ino.bin" using "C:\Users\dok\AppData\Local\Temp\VMBuilds\blinkbuttonandweb\esp8266com_d1_mini\Release/blinkbuttonandweb.ino.elf"
Program size: 320,324 bytes (used 31% of a 1,044,464 byte maximum) (33.24 secs)
Minimum Memory Usage: 28568 bytes (35% of a 81920 byte maximum)
 



Fourth time, no changes..., control-shift-B …
Code:
Compiling 'blinkbuttonandweb' for 'LOLIN(WEMOS) D1 R2 & mini'
Creating BIN file "C:\Users\dok\AppData\Local\Temp\VMBuilds\blinkbuttonandweb\esp8266com_d1_mini\Release/blinkbuttonandweb.ino.bin" using "C:\Users\dok\AppData\Local\Temp\VMBuilds\blinkbuttonandweb\esp8266com_d1_mini\Release/blinkbuttonandweb.ino.elf"
Program size: 320,324 bytes (used 31% of a 1,044,464 byte maximum) (4.87 secs)
Minimum Memory Usage: 28568 bytes (35% of a 81920 byte maximum)
 


Fifth time, same procedure:
Code:
Compiling 'blinkbuttonandweb' for 'LOLIN(WEMOS) D1 R2 & mini'
Creating BIN file "C:\Users\dok\AppData\Local\Temp\VMBuilds\blinkbuttonandweb\esp8266com_d1_mini\Release/blinkbuttonandweb.ino.bin" using "C:\Users\dok\AppData\Local\Temp\VMBuilds\blinkbuttonandweb\esp8266com_d1_mini\Release/blinkbuttonandweb.ino.elf"
Program size: 320,324 bytes (used 31% of a 1,044,464 byte maximum) (33.66 secs)
Minimum Memory Usage: 28568 bytes (35% of a 81920 byte maximum)
 



Obviously not expected predictable behavior of a properly functioning make.
Back to top
« Last Edit: Aug 8th, 2019 at 4:21pm by dok »  
 
IP Logged
 
Visual Micro
Administrator
*****
Offline



Posts: 10623
United Kingdom
Re: ESP8266 Arduino rebuilds loads of files that weren't changed
Reply #5 - Aug 8th, 2019 at 4:20pm
 
If it did then I could reset it but if you rename you can delete the new "visual micro" that will be created and rename the old one back after the test.

Back to top
 
WWW  
IP Logged
 
dok
Junior Member
**
Offline



Posts: 34
Re: ESP8266 Arduino rebuilds loads of files that weren't changed
Reply #6 - Aug 8th, 2019 at 4:22pm
 
About the license - thanks, I'm reverting the registry accordingly, as the suggested workaround didn't have any effect.
Back to top
 
 
IP Logged
 
dok
Junior Member
**
Offline



Posts: 34
Re: ESP8266 Arduino rebuilds loads of files that weren't changed
Reply #7 - Aug 8th, 2019 at 4:41pm
 
Just to be sure, testing other targets, and cleaning up. But:

Arduino Mini Pro - unaffected.
ESP32 - unaffected.
ESP8266 master branch in git client, cleaned up, toolchain updated (cd tools; py get.py) - as observed and reported above.
Back to top
« Last Edit: Aug 8th, 2019 at 4:41pm by dok »  
 
IP Logged
 
Visual Micro
Administrator
*****
Offline



Posts: 10623
United Kingdom
Re: ESP8266 Arduino rebuilds loads of files that weren't changed
Reply #8 - Aug 8th, 2019 at 4:43pm
 
Thanks, so to be clear you are saying the problem happens with you custom tool chain in documents\arduino\hardware (sketchbook\hardware) but does not happen with packages installed using board manager or built in arduino ide (avr)?
Back to top
« Last Edit: Aug 8th, 2019 at 4:57pm by Visual Micro »  
WWW  
IP Logged
 
dok
Junior Member
**
Offline



Posts: 34
Re: ESP8266 Arduino rebuilds loads of files that weren't changed
Reply #9 - Aug 8th, 2019 at 5:43pm
 
I didn't exactly say that.
There's no custom toolchain involved, this is HEAD of the master branch of the ESP8266 Arduino project.
To make sure it's not just about the latest version - checking out release 2.5.2 in git.

Using Arduino\hardware\esp8266com\esp8266\libraries\ESP8266WebServer\examples\HelloSer
ver\HelloServer.ino - it's the ESP8266 after all, which is all about the WiFi and web server capability.

Rebuild - 63s.
Build - 3s.
Build - 35s.
Build - 4s.

So it's not about the version.

Moving away my git clone in hardware, and d/l-ing via the board manager (really useless for me, but for testing sake).

Opening a copy of the HelloServer example (Micro Explorer/Examples).

So, rescan toolchain, rebuild (65s), build (3s), build (3s), build (3s).

Fantastic. NOT helpful.

Leaving the board manager BSP package in place, but restoring the git master to …/hardware.

Rebuild (63s), build (4s), build (38s), build (5s).

I am attaching the logs of building with the board manager supplied 2.5.2, and those of building with the git version 2.5.2.
Back to top
 

Documents_001.zip (35 KB | 5 )
 
IP Logged
 
Visual Micro
Administrator
*****
Offline



Posts: 10623
United Kingdom
Re: ESP8266 Arduino rebuilds loads of files that weren't changed
Reply #10 - Aug 8th, 2019 at 5:52pm
 
Ah okay, so to be clear. If you have a package installed into board manager and another copy of the same (or similar) core from the package installed into sketchbook\hardware then we see the problem?

edit: I think I have anough to experiment. This seems to be an issue with split or partl duplicate toolchain/cores maybe.
Back to top
« Last Edit: Aug 8th, 2019 at 5:55pm by Visual Micro »  
WWW  
IP Logged
 
dok
Junior Member
**
Offline



Posts: 34
Re: ESP8266 Arduino rebuilds loads of files that weren't changed
Reply #11 - Aug 8th, 2019 at 6:00pm
 
Not precisely.
The problem does not occur when there is no BSP distribution for the ESP8266 in the sketchbook Arduino/hardware, but one is installed via the board manager (sic).
The problem always occurs when there is an ESP8266 BSP distribution in the sketchbook, independent of the presence of one that's installed via the board manager. Haven't tried non-matching versions Wink
Back to top
 
 
IP Logged
 
dok
Junior Member
**
Offline



Posts: 34
Re: ESP8266 Arduino rebuilds loads of files that weren't changed
Reply #12 - Aug 8th, 2019 at 6:03pm
 
I hope you'll be able to figure this out without external help from the FOSS project - it would be so much nicer not having to spend 30s on each compile in vain. Thanks!!!!!!!!
Back to top
 
 
IP Logged
 
Visual Micro
Administrator
*****
Offline



Posts: 10623
United Kingdom
Re: ESP8266 Arduino rebuilds loads of files that weren't changed
Reply #13 - Aug 8th, 2019 at 6:11pm
 
Thanks, will take a look over coming few days. It will build a bit faster of you switch on "vMicro>Compiler>Parallel Builds". example: All the wifi library files in a single folder will be compiled at same time (errors can be a little more difficult to read).
Back to top
« Last Edit: Aug 8th, 2019 at 6:12pm by Visual Micro »  
WWW  
IP Logged
 
Visual Micro
Administrator
*****
Offline



Posts: 10623
United Kingdom
Re: Rebuild libraries due to custom hook event generating a new dependent(s)
Reply #14 - Aug 10th, 2019 at 1:19pm
 
It looks like Visual Micro is basically doing the right thing. The confusion appears to be caused by your prebuild event hooks where you geneterate a new version number .h file.

The version number changes during each build and you have linked the version.h into the libraries (maybe via the core).

The libraries use more than simple date time of each .cpp to determine when rebuild it need. It uses the .d depency file generated by previous build to discover all depencies of each source code file, so that it can compare the dates and times of dependants with the current source file (cpp).

Therefore because your version number changes for each build then we would expect the libraries and project code to be rebuilt each time the build happens.

In the next release of Visual Micro there is a new build event hook that fires only when the date time or a core file has changed called "core.prebuild_if_changed". If you move your core.prebuild event hook then you get a new version.h only when you have changed a core source core.

The fact that you see alternate build is probably due to when the hook is run and the fact that you create it in the core temp build folder itself. Maybe it should have rebuilt every single time to cause less confusion.

Does this make sense?

These are the hooks in your custom core.

Code:
recipe.hooks.sketch.prebuild.pattern="{runtime.tools.python.path}/python" "{runtime.tools.signing}" --mode header --publickey "{build.source.path}/public.key" --out "{build.path}/core/Updater_Signing.h"
>>> recipe.hooks.core.prebuild.pattern="{runtime.tools.python.path}/python" "{runtime.tools.makecorever}" --build_path "{build.path}" --platform_path "{runtime.platform.path}" --version "unix-{version}" 




Back to top
« Last Edit: Aug 10th, 2019 at 1:49pm by Visual Micro »  
WWW  
IP Logged
 
dok
Junior Member
**
Offline



Posts: 34
Re: ESP8266 Arduino rebuilds loads of files that weren't changed
Reply #15 - Aug 12th, 2019 at 7:05am
 
Well, if you replace in your message "your" by "the community ESP8266 Arduino core's", then I'm fine Smiley
I hope you understand that your statement

"your custom core"

sounds like assigning blame to me in my ears.

[edit: removed part talking about recent change, I forgot that I was testing 2.5.2 before already].

I don't know what the difference between "packaged release" and Git at the the same version commit is - if any.
Back to top
« Last Edit: Aug 12th, 2019 at 7:19am by dok »  
 
IP Logged
 
dok
Junior Member
**
Offline



Posts: 34
Re: ESP8266 Arduino rebuilds loads of files that weren't changed
Reply #16 - Aug 12th, 2019 at 7:47am
 
So, it's all about this one, right:

...\AppData\Local\Temp\VMBuilds\HelloServer\esp8266com_d1_mini\Release\core\core
_version.h ?

It seems in esp8266/tools/makecorever.py, in Git builds - as opposed to releases where core_version.h get placed once and for all into cores/esp8266 - the python script generates the same file contents for core_version.h over and over, but obviously updates the file time stamp. And that's consistent with what you found.

So all it might take is some hacking of the python script... yes, confirmed, if I comment out the writing of core_version.h, the build time without changes becomes a consistent 4.5s.

Thanks for looking into this, I guess I'll pick up from here.
Back to top
 
 
IP Logged
 
dok
Junior Member
**
Offline



Posts: 34
Re: ESP8266 Arduino rebuilds loads of files that weren't changed
Reply #17 - Aug 12th, 2019 at 8:18am
 
Back to top
 
 
IP Logged
 
Page Index Toggle Pages: 1
Send Topic Print