Before logging an issue, please update to the latest release of Visual Micro from the Downloads Page.

When Logging a Support Issue in the Forum, please ensure you have also:-

  • Enabled vMicro > Compiler > Show Build Properties
  • Re-Compile your program with these settings enabled
 
Save the new Output to a Text File and....
  • Click the Reply button and attach as .txt file OR
  • Click here to Email us with the file attached, and a link to your post
Support requests without the output above may be impossible to answer, so please help us to help you
 
Page Index Toggle Pages: 1 Send TopicPrint
Hot Topic (More than 8 Replies) ESP8266 Arduino rebuilds loads of files that weren't changed (Read 6561 times)
dok
Junior Member
**
Offline


Posts: 47
Joined: Apr 11th, 2019
ESP8266 Arduino rebuilds loads of files that weren't changed
Aug 8th, 2019 at 2:37pm
Print Post  
Each time, without any code changes whatsover, "Build project" (control-shift-B).
Every other build rebuilds loads of unchanged sources - wastes a lot of developer time, 29s in this case.
  

Please Register or Login to the Forum to see File Attachments
Back to top
 
IP Logged
 
Tim@Visual Micro
Administrator
*****
Offline


Posts: 12071
Location: United Kingdom
Joined: Apr 10th, 2010
Re: ESP8266 Arduino rebuilds loads of files that weren't changed
Reply #1 - Aug 8th, 2019 at 2:48pm
Print Post  
I can't reproduce the problem. Ae you making any changes at all between builds?
  
Back to top
WWW  
IP Logged
 
Tim@Visual Micro
Administrator
*****
Offline


Posts: 12071
Location: United Kingdom
Joined: Apr 10th, 2010
Re: ESP8266 Arduino rebuilds loads of files that weren't changed
Reply #2 - Aug 8th, 2019 at 3:04pm
Print Post  
Please close the ide and open regedit and rename hkey_current_user\software\visual micro.

Then restart the ide add the arduino 1.6/1.8 program location to IDE Locations. Then open your project and try this again.

This will ensure you are using the visual micro default cache settings.

Thanks
  
Back to top
WWW  
IP Logged
 
dok
Junior Member
**
Offline


Posts: 47
Joined: Apr 11th, 2019
Re: ESP8266 Arduino rebuilds loads of files that weren't changed
Reply #3 - Aug 8th, 2019 at 3:13pm
Print Post  
I hope this doesn't lose my license?
  
Back to top
 
IP Logged
 
dok
Junior Member
**
Offline


Posts: 47
Joined: Apr 11th, 2019
Re: ESP8266 Arduino rebuilds loads of files that weren't changed
Reply #4 - Aug 8th, 2019 at 3:20pm
Print Post  
First time, "Rebuild Solution":
Code
Select All
Compiling 'blinkbuttonandweb' for 'LOLIN(WEMOS) D1 R2 & mini'
Creating BIN file elease/blinkbuttonandweb.ino.bin" using elease/blinkbuttonandweb.ino.elf"
Program size: 320,324 bytes (used 31% of a 1,044,464 byte maximum) (56.89 secs)
Minimum Memory Usage: 28568 bytes (35% of a 81920 byte maximum)
 

 

Second time, no changes, control-shift-B:
Code
Select All
Compiling 'blinkbuttonandweb' for 'LOLIN(WEMOS) D1 R2 & mini'
Creating BIN file elease/blinkbuttonandweb.ino.bin" using elease/blinkbuttonandweb.ino.elf"
Program size: 320,324 bytes (used 31% of a 1,044,464 byte maximum) (2.51 secs)
Minimum Memory Usage: 28568 bytes (35% of a 81920 byte maximum)
 

 

Third time, no changes, control-shift-B:
Code
Select All
Compiling 'blinkbuttonandweb' for 'LOLIN(WEMOS) D1 R2 & mini'
Creating BIN file elease/blinkbuttonandweb.ino.bin" using elease/blinkbuttonandweb.ino.elf"
Program size: 320,324 bytes (used 31% of a 1,044,464 byte maximum) (33.24 secs)
Minimum Memory Usage: 28568 bytes (35% of a 81920 byte maximum)
 

 

Fourth time, no changes..., control-shift-B …
Code
Select All
Compiling 'blinkbuttonandweb' for 'LOLIN(WEMOS) D1 R2 & mini'
Creating BIN file elease/blinkbuttonandweb.ino.bin" using elease/blinkbuttonandweb.ino.elf"
Program size: 320,324 bytes (used 31% of a 1,044,464 byte maximum) (4.87 secs)
Minimum Memory Usage: 28568 bytes (35% of a 81920 byte maximum)
 

 
Fifth time, same procedure:
Code
Select All
Compiling 'blinkbuttonandweb' for 'LOLIN(WEMOS) D1 R2 & mini'
Creating BIN file elease/blinkbuttonandweb.ino.bin" using elease/blinkbuttonandweb.ino.elf"
Program size: 320,324 bytes (used 31% of a 1,044,464 byte maximum) (33.66 secs)
Minimum Memory Usage: 28568 bytes (35% of a 81920 byte maximum)
 

 

Obviously not expected predictable behavior of a properly functioning make.
« Last Edit: Aug 8th, 2019 at 3:21pm by dok »  
Back to top
 
IP Logged
 
Tim@Visual Micro
Administrator
*****
Offline


Posts: 12071
Location: United Kingdom
Joined: Apr 10th, 2010
Re: ESP8266 Arduino rebuilds loads of files that weren't changed
Reply #5 - Aug 8th, 2019 at 3:20pm
Print Post  
If it did then I could reset it but if you rename you can delete the new "visual micro" that will be created and rename the old one back after the test.

  
Back to top
WWW  
IP Logged
 
dok
Junior Member
**
Offline


Posts: 47
Joined: Apr 11th, 2019
Re: ESP8266 Arduino rebuilds loads of files that weren't changed
Reply #6 - Aug 8th, 2019 at 3:22pm
Print Post  
About the license - thanks, I'm reverting the registry accordingly, as the suggested workaround didn't have any effect.
  
Back to top
 
IP Logged
 
dok
Junior Member
**
Offline


Posts: 47
Joined: Apr 11th, 2019
Re: ESP8266 Arduino rebuilds loads of files that weren't changed
Reply #7 - Aug 8th, 2019 at 3:41pm
Print Post  
Just to be sure, testing other targets, and cleaning up. But:

Arduino Mini Pro - unaffected.
ESP32 - unaffected.
ESP8266 master branch in git client, cleaned up, toolchain updated (cd tools; py get.py) - as observed and reported above.
« Last Edit: Aug 8th, 2019 at 3:41pm by dok »  
Back to top
 
IP Logged
 
Tim@Visual Micro
Administrator
*****
Offline


Posts: 12071
Location: United Kingdom
Joined: Apr 10th, 2010
Re: ESP8266 Arduino rebuilds loads of files that weren't changed
Reply #8 - Aug 8th, 2019 at 3:43pm
Print Post  
Thanks, so to be clear you are saying the problem happens with you custom tool chain in documents\arduino\hardware (sketchbook\hardware) but does not happen with packages installed using board manager or built in arduino ide (avr)?
« Last Edit: Aug 8th, 2019 at 3:57pm by Tim@Visual Micro »  
Back to top
WWW  
IP Logged
 
dok
Junior Member
**
Offline


Posts: 47
Joined: Apr 11th, 2019
Re: ESP8266 Arduino rebuilds loads of files that weren't changed
Reply #9 - Aug 8th, 2019 at 4:43pm
Print Post  
I didn't exactly say that.
There's no custom toolchain involved, this is HEAD of the master branch of the ESP8266 Arduino project.
To make sure it's not just about the latest version - checking out release 2.5.2 in git.

Using ver\HelloServer.ino - it's the ESP8266 after all, which is all about the WiFi and web server capability.

Rebuild - 63s.
Build - 3s.
Build - 35s.
Build - 4s.

So it's not about the version.

Moving away my git clone in hardware, and d/l-ing via the board manager (really useless for me, but for testing sake).

Opening a copy of the HelloServer example (Micro Explorer/Examples).

So, rescan toolchain, rebuild (65s), build (3s), build (3s), build (3s).

Fantastic. NOT helpful.

Leaving the board manager BSP package in place, but restoring the git master to …/hardware.

Rebuild (63s), build (4s), build (38s), build (5s).

I am attaching the logs of building with the board manager supplied 2.5.2, and those of building with the git version 2.5.2.
  

Please Register or Login to the Forum to see File Attachments
Back to top
 
IP Logged
 
Tim@Visual Micro
Administrator
*****
Offline


Posts: 12071
Location: United Kingdom
Joined: Apr 10th, 2010
Re: ESP8266 Arduino rebuilds loads of files that weren't changed
Reply #10 - Aug 8th, 2019 at 4:52pm
Print Post  
Ah okay, so to be clear. If you have a package installed into board manager and another copy of the same (or similar) core from the package installed into sketchbook\hardware then we see the problem?

edit: I think I have anough to experiment. This seems to be an issue with split or partl duplicate toolchain/cores maybe.
« Last Edit: Aug 8th, 2019 at 4:55pm by Tim@Visual Micro »  
Back to top
WWW  
IP Logged
 
dok
Junior Member
**
Offline


Posts: 47
Joined: Apr 11th, 2019
Re: ESP8266 Arduino rebuilds loads of files that weren't changed
Reply #11 - Aug 8th, 2019 at 5:00pm
Print Post  
Not precisely.
The problem does not occur when there is no BSP distribution for the ESP8266 in the sketchbook Arduino/hardware, but one is installed via the board manager (sic).
The problem always occurs when there is an ESP8266 BSP distribution in the sketchbook, independent of the presence of one that's installed via the board manager. Haven't tried non-matching versions Wink
  
Back to top
 
IP Logged
 
dok
Junior Member
**
Offline


Posts: 47
Joined: Apr 11th, 2019
Re: ESP8266 Arduino rebuilds loads of files that weren't changed
Reply #12 - Aug 8th, 2019 at 5:03pm
Print Post  
I hope you'll be able to figure this out without external help from the FOSS project - it would be so much nicer not having to spend 30s on each compile in vain. Thanks!!!!!!!!
  
Back to top
 
IP Logged
 
Tim@Visual Micro
Administrator
*****
Offline


Posts: 12071
Location: United Kingdom
Joined: Apr 10th, 2010
Re: ESP8266 Arduino rebuilds loads of files that weren't changed
Reply #13 - Aug 8th, 2019 at 5:11pm
Print Post  
Thanks, will take a look over coming few days. It will build a bit faster of you switch on "vMicro>Compiler>Parallel Builds". example: All the wifi library files in a single folder will be compiled at same time (errors can be a little more difficult to read).
« Last Edit: Aug 8th, 2019 at 5:12pm by Tim@Visual Micro »  
Back to top
WWW  
IP Logged
 
Tim@Visual Micro
Administrator
*****
Offline


Posts: 12071
Location: United Kingdom
Joined: Apr 10th, 2010
Re: Rebuild libraries due to custom hook event generating a new dependent(s)
Reply #14 - Aug 10th, 2019 at 12:19pm
Print Post  
It looks like Visual Micro is basically doing the right thing. The confusion appears to be caused by your prebuild event hooks where you geneterate a new version number .h file.

The version number changes during each build and you have linked the version.h into the libraries (maybe via the core). 

The libraries use more than simple date time of each .cpp to determine when rebuild it need. It uses the .d depency file generated by previous build to discover all depencies of each source code file, so that it can compare the dates and times of dependants with the current source file (cpp). 

Therefore because your version number changes for each build then we would expect the libraries and project code to be rebuilt each time the build happens.

In the next release of Visual Micro there is a new build event hook that fires only when the date time or a core file has changed called "core.prebuild_if_changed". If you move your core.prebuild event hook then you get a new version.h only when you have changed a core source core.

The fact that you see alternate build is probably due to when the hook is run and the fact that you create it in the core temp build folder itself. Maybe it should have rebuilt every single time to cause less confusion.

Does this make sense? 

These are the hooks in your custom core.

Code
Select All
recipe.hooks.sketch.prebuild.pattern="{runtime.tools.python.path}/python" "{runtime.tools.signing}" --mode header --publickey "{build.source.path}/public.key" --out "{build.path}/core/Updater_Signing.h"
>>> recipe.hooks.core.prebuild.pattern="{runtime.tools.python.path}/python" "{runtime.tools.makecorever}" --build_path "{build.path}" --platform_path "{runtime.platform.path}" --version "unix-{version}" 




« Last Edit: Aug 10th, 2019 at 12:49pm by Tim@Visual Micro »  
Back to top
WWW  
IP Logged
 
dok
Junior Member
**
Offline


Posts: 47
Joined: Apr 11th, 2019
Re: ESP8266 Arduino rebuilds loads of files that weren't changed
Reply #15 - Aug 12th, 2019 at 6:05am
Print Post  
Well, if you replace in your message "your" by "the community ESP8266 Arduino core's", then I'm fine Smiley 
I hope you understand that your statement

"your custom core"

sounds like assigning blame to me in my ears.

[edit: removed part talking about recent change, I forgot that I was testing 2.5.2 before already].

I don't know what the difference between "packaged release" and Git at the the same version commit is - if any.
« Last Edit: Aug 12th, 2019 at 6:19am by dok »  
Back to top
 
IP Logged
 
dok
Junior Member
**
Offline


Posts: 47
Joined: Apr 11th, 2019
Re: ESP8266 Arduino rebuilds loads of files that weren't changed
Reply #16 - Aug 12th, 2019 at 6:47am
Print Post  
So, it's all about this one, right:

_version.h ?

It seems in esp8266/tools/makecorever.py, in Git builds - as opposed to releases where core_version.h get placed once and for all into cores/esp8266 - the python script generates the same file contents for core_version.h over and over, but obviously updates the file time stamp. And that's consistent with what you found.

So all it might take is some hacking of the python script... yes, confirmed, if I comment out the writing of core_version.h, the build time without changes becomes a consistent 4.5s.

Thanks for looking into this, I guess I'll pick up from here.
  
Back to top
 
IP Logged
 
dok
Junior Member
**
Offline


Posts: 47
Joined: Apr 11th, 2019
Re: ESP8266 Arduino rebuilds loads of files that weren't changed
Reply #17 - Aug 12th, 2019 at 7:18am
Print Post  
  
Back to top
 
IP Logged
 
Page Index Toggle Pages: 1
Send TopicPrint