Hi.  I'm adding a new webpage to an esp32 project.  The html is to be stored in flash as a compressed char array.  I have a python script that does the conversion job but I'm having much trouble getting html source code changes integrated into the project build.  Google AI has had me going round the world all day and getting nowhere.

I've had enough of that.  It occurs to me that not only is this a common task, but also that the only thing an IDE really does is handle source code changes, so there must be a standard approach to this problem.  Is there??

Many thanks!

Tim,
Thanks for the reply.

Most of my app puts the HTML in files in the LittleFS, which is quite convenient, until you run out of space.  However, I'm replacing AsyncElegantOTA to try and improve robustness.  One factor is the use of LittleFS, or rather it's tendency to corruption.  For OTA, reliability seems best served by putting the html (minified and compressed) into a char array, requiring no extra data transfers.  However, it then gets complicated to develop with multiple steps to convert an HTML file to a C++ header file. Basically, you edit the html, run it through a python script to minify, compress and output a .h file, then build.

My initial approach was to put a prebuild hook in board.txt to call the python script.  This is still the best so far, but it seems to take two builds for an html edit to become effective.  Google AI gave several different reasons for this, to do with the relative timing of the build process, but nothing worked.  Their latest suggestion is  that it can't be done in Visual Micro and suggested basically running a python script in the background looking for changes to the specific file...

So I am asking the experts:  how can I automate the incorporation of html file edits in to the final build as char arrays?

I admit to being pretty ignorant of the recipe system.  From what I can gather, and this is what AI (Google's) was telling me, was that the list of files to be compiled (and this list is just C++ files) is determined before any of the recipes are invoked.  I guess from dependencies, relative timestamps and such.  What this implies to me is that there is no way a recipe can add or change a C++ file in regard to its presence in the list to be compiled.  This would explain the need to build twice:  first time, the recipe python script creates the .h file but if it is not already in the list for a prior change, it will not change the dependencies that have been determined already; the second time, it will be seen as changed before any recipe are run so will be incorporated.

Consider another point of view:  suppose the only change you make is to the html file.  When checking the dependencies, none will be found, since only C++ files are considered.   The build log will show only  "Using previously compiled file", but the hook will be executed and the .h file will be generated.  The linker, uploader,... will also run.  On the next build, .cpp files dependent on the .h file will be compiled.  This is exactly what my build logs showed.

Anyhow, this is my board.txt line:

recipe.hooks.prebuild.0.pattern=python3 "{build.project_path}\gzip_html.py" "{build.project_path}" 

and it exhibits the double-build features I described.  I've had a look at the link you sent me, and I can't see any means for doing something before dependencies are determined.  Please correct me if I am wrong.  I am still hoping someone has a solution for this:  I would have guessed it's a pretty common thing.