Before logging an issue, please update to the latest release of Visual Micro from the Downloads Page.

When Logging a Support Issue in the Forum, please ensure you have also:-

  • Enabled vMicro > Compiler > Show Build Properties
  • Re-Compile your program with these settings enabled
 
Save the new Output to a Text File and....
  • Click the Reply button and attach as .txt file OR
  • Click here to Email us with the file attached, and a link to your post
Support requests without the output above may be impossible to answer, so please help us to help you
 
Page Index Toggle Pages: 1 Send TopicPrint
Hot Topic (More than 8 Replies) Parallel build of multple projects using the CLI can fail (call multiple VMBuild.exe at same time) (Read 3111 times)
stm
Junior Member
**
Offline


Posts: 27
Joined: Jul 5th, 2018
Parallel build of multple projects using the CLI can fail (call multiple VMBuild.exe at same time)
May 28th, 2020 at 4:31pm
Print Post  
I try to run different builds of the CLI in parallel to speed up things. It actually looks quite promising: sometimes it is working just fine. But sometimes I get the following error on some projects of the parallel build (started all at the same time):

C:\projects\BaseProject\__vm\Configuration.Release.vmps.xml
System.Exception: Visual Micro: Unable to save new program settings file
C:\projects\BaseProject\__vm\Configuration.Release.vmps.xml ---> System.IO.IOException: The process can not access the file "C:\projects\BaseProject\__vm\Configuration.Release.vmps.xml" because it is being used by another process.
   bei System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
   bei System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
   bei System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share)
   bei System.Xml.XmlDocument.Save(String filename)
   bei Visual.Micro.ProgramConfiguration.ProgramSettingsReader.Save()
   --- Ende der internen Ausnahmestapelüberwachung ---
   bei Visual.Micro.ProgramConfiguration.ProgramSettingsReader.Save()
   bei VMBuild.VMBuilder._CompileDo(VMBuilderProject oProject, Boolean isRebuild)
   bei VMBuild.VMBuilder.RunBuilder()
   bei VMBuild.VMBuilder..ctor(VMBuildArgs _vmArgs)
   bei VMBuild.Program.Main(String[] args)
Build ended

Am I being too brave or can this be possible? Thanks
  
Back to top
 
IP Logged
 
Tim@Visual Micro
Administrator
*****
Offline


Posts: 12071
Location: United Kingdom
Joined: Apr 10th, 2010
Re: Parallel build of multple projects using the CLI can fail (call multiple VMBuild.exe at same time)
Reply #1 - May 28th, 2020 at 5:24pm
Print Post  
Thanks for the report. It should work. 

The error is attempting to save the project settings and you are building the same project but with different toolchains. 

The saved settings are only used for upload/debug if overrides are not available. 

We will test and add some protection/retries or CLIE save  project settings to the MCU specific build folder or not write them at all via CLI. 

We will update this thread during the coming few days.
  
Back to top
WWW  
IP Logged
 
Tim@Visual Micro
Administrator
*****
Offline


Posts: 12071
Location: United Kingdom
Joined: Apr 10th, 2010
Re: Parallel build of multple projects using the CLI can fail (call multiple VMBuild.exe at same time)
Reply #2 - May 30th, 2020 at 1:50pm
Print Post  
Please try this version of the CLI which should resolve the issue.

Download Visual Micro 20.05.18.5 CLI



  
Back to top
WWW  
IP Logged
 
stm
Junior Member
**
Offline


Posts: 27
Joined: Jul 5th, 2018
Re: Parallel build of multple projects using the CLI can fail (call multiple VMBuild.exe at same time)
Reply #3 - Jun 5th, 2020 at 3:28pm
Print Post  
Thanks for your reply. I'm now using this CLI version and still see errors of the following two kinds in approx. 50% of the cases. Both appear almost instantly. If the first few seconds are without error, everything works smooth from then on.


1) I think the one from last time:

The operation failed with exception: Visual Micro: Unable to save new program settings file
C:\Users\...\__vm\Configuration.Release.vmps.xml
System.Exception: Visual Micro: Unable to save new program settings file
C:\Users\...\__vm\Configuration.Release.vmps.xml ---> System.IO.IOException: used by another process.
   bei System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
   bei System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
   bei System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share)
   bei System.Xml.XmlDocument.Save(String filename)
   bei Visual.Micro.ProgramConfiguration.ProgramSettingsReader.Save()
   bei VMBuild.VMBuilder._CompileDo(VMBuilderProject oProject, Boolean isRebuild)
   bei VMBuild.VMBuilder.RunBuilder()
   bei VMBuild.VMBuilder..ctor(VMBuildArgs _vmArgs)
   bei VMBuild.Program.Main(String[] args)


2) New one. Probably due to copying shared files from different projects:

The operation failed with exception: Problem moving OtaUpdater.cpp to the build folder
System.Exception: Problem moving OtaUpdater.cpp to the build folder ---> System.IO.IOException: file "C:\Users\...\espressif_pico32\Release\OtaUpdater.cpp" is used by another process.
   bei System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
   bei System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
   bei System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
   bei System.IO.StreamWriter.CreateFile(String path, Boolean append, Boolean checkHost)
   bei System.IO.StreamWriter..ctor(String path, Boolean append, Encoding encoding, Int32 bufferSize, Boolean checkHost)
   bei System.IO.File.InternalWriteAllText(String path, String contents, Encoding encoding, Boolean checkHost)
   bei System.IO.File.WriteAllText(String path, String contents)
   bei Visual.Micro.MiroAppAPI.SketchBuilder.preprocess(String buildPath, PdePreprocessor preprocessor, Boolean PreProcessOnly)
   bei Visual.Micro.MiroAppAPI.SketchBuilder.preprocess(String buildPath, Boolean PreProcessOnly)
   bei Visual.Micro.MiroAppAPI.SketchBuilder._build(String lbuildPath, Boolean verbose, Boolean PreProcessOnly)
   bei Visual.Micro.MiroAppAPI.SketchBuilder._build(Boolean verbose)
   bei Visual.Micro.MiroAppAPI.SketchBuilder.build(Boolean verbose)
   bei VMBuild.VMBuilder._CompileDo(VMBuilderProject oProject, Boolean isRebuild)
   bei VMBuild.VMBuilder.RunBuilder()
   bei VMBuild.VMBuilder..ctor(VMBuildArgs _vmArgs)
   bei VMBuild.Program.Main(String[] args)



A hack on my side would be to wait for a few seconds before starting a new parallel build but this is certainly not a nice approach if there are better solutions.
  
Back to top
 
IP Logged
 
Tim@Visual Micro
Administrator
*****
Offline


Posts: 12071
Location: United Kingdom
Joined: Apr 10th, 2010
Re: Parallel build of multple projects using the CLI can fail (call multiple VMBuild.exe at same time)
Reply #4 - Jun 5th, 2020 at 4:59pm
Print Post  
Okay will take a look thanks. You should not be seeing 1) and 2) can be resolved by giving you facility to pass the build folder in your cli command. 

2) might already be possible, if so we will document it.

More over the next week ...
  
Back to top
WWW  
IP Logged
 
stm
Junior Member
**
Offline


Posts: 27
Joined: Jul 5th, 2018
Re: Parallel build of multple projects using the CLI can fail (call multiple VMBuild.exe at same time)
Reply #5 - Jun 23rd, 2020 at 6:31am
Print Post  
Hello again. I take it you have not found the time to have another look at this yet. In any case please let me know if you need more info from my side.
  
Back to top
 
IP Logged
 
Tim@Visual Micro
Administrator
*****
Offline


Posts: 12071
Location: United Kingdom
Joined: Apr 10th, 2010
Re: Parallel build of multple projects using the CLI can fail (call multiple VMBuild.exe at same time)
Reply #6 - Jun 25th, 2020 at 6:39pm
Print Post  
Sorry for the delay. I guess you are building the same project for the same mcu but with different #defnes? That would cause a clash in the temp build folder. 

The latest CLI has a new property you can pass allowing the build folder to be set to something unique. The update also prevents an exception relating to the settings files being locked by another CLI.

You will read that the new property is:- -builder.build_temp_path=

https://www.visualmicro.com/forums/YaBB.pl?board=VS_ARDUINO_EXT_RELEASES

I ddn't have time to test fully yet but hopefully does the job.
  
Back to top
WWW  
IP Logged
 
stm
Junior Member
**
Offline


Posts: 27
Joined: Jul 5th, 2018
Re: Parallel build of multple projects using the CLI can fail (call multiple VMBuild.exe at same time)
Reply #7 - Jun 26th, 2020 at 12:07pm
Print Post  
Thanks for the reply! No problem, I will invest the time to check in a free minute and get back to you. Certainly I could also live without parallel builds. But making this work could even prevent other bugs down the line by separating things clearly.
  
Back to top
 
IP Logged
 
Tim@Visual Micro
Administrator
*****
Offline


Posts: 12071
Location: United Kingdom
Joined: Apr 10th, 2010
Re: Parallel build of multple projects using the CLI can fail (call multiple VMBuild.exe at same time)
Reply #8 - Jun 26th, 2020 at 2:35pm
Print Post  
Thanks, yes it is important to provide all options and useful to have someone pushing things for us.
  
Back to top
WWW  
IP Logged
 
stm
Junior Member
**
Offline


Posts: 27
Joined: Jul 5th, 2018
Re: Parallel build of multple projects using the CLI can fail (call multiple VMBuild.exe at same time)
Reply #9 - Jul 3rd, 2020 at 7:49am
Print Post  
Hello just tried it and still getting the exception about saving program settings file:

The operation failed with exception: Visual Micro: Unable to save new program settings file
C:\Users\Public\arduino\Project\__vm\Configuration.Release.vmps.xml
System.Exception: Visual Micro: Unable to save new program settings file
C:\Users\Public\arduino\Project\__vm\Configuration.Release.vmps.xml ---> System.IO.IOException: Process can not access file "C:\Users\Public\arduino\Project\__vm\Configuration.Release.vmps.xml" because it is used by another process.
   bei System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
   bei System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, security attributes secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
   bei System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share)
   bei System.Xml.XmlDocument.Save(String filename)
   bei Visual.Micro.ProgramConfiguration.ProgramSettingsReader.Save()
   --- Ende der internen Ausnahmestapelüberwachung ---
   bei Visual.Micro.ProgramConfiguration.ProgramSettingsReader.Save()
   bei VMBuild.VMBuilder._CompileDo(VMBuilderProject oProject, Boolean isRebuild)
   bei VMBuild.VMBuilder.RunBuilder()
   bei VMBuild.VMBuilder..ctor(VMBuildArgs _vmArgs)
   bei VMBuild.Program.Main(String[] args)
  
Back to top
 
IP Logged
 
Tim@Visual Micro
Administrator
*****
Offline


Posts: 12071
Location: United Kingdom
Joined: Apr 10th, 2010
Re: Parallel build of multple projects using the CLI can fail (call multiple VMBuild.exe at same time)
Reply #10 - Jul 3rd, 2020 at 10:59am
Print Post  
Okay can i see the two diffent cli command you are using please.
  
Back to top
WWW  
IP Logged
 
stm
Junior Member
**
Offline


Posts: 27
Joined: Jul 5th, 2018
Re: Parallel build of multple projects using the CLI can fail (call multiple VMBuild.exe at same time)
Reply #11 - Jul 3rd, 2020 at 12:02pm
Print Post  
Executable: C:\Users\Public\arduino\VMBuilder\VMBuild.exe, Arguments: -builder.project_path="C:/Users/Public/arduino/Project1/Project1.vcxproj" -builder.output_directory="C:/Users/Public/arduino/.build/publish/project1" -builder.build_temp_path="C:/Users/Public/arduino/.build/publish/project1/temp" -builder.clean=true -builder.verbose=true -builder.verbose_warnings=true -builder.build=true -builder.rebuild=false -builder.upload=false -builder.parallel_build=false -builder.dump_boards=false -builder.dump_props=false -builder.dump_source_list=true -builder.burn_bootloaderx=false -builder-close=true -builder-closeonerror=true -builder.ide_path="C:\Users\Public\arduino\ide" -builder.board_id="ESP32 Pico Kit" -builder.application_id="arduino16x" -builder-configuration="Release"

and

Executable: C:\Users\Public\arduino\VMBuilder\VMBuild.exe, Arguments: -builder.project_path="C:/Users/Public/arduino/Project2/Project2.vcxproj" -builder.output_directory="C:/Users/Public/arduino/.build/publish/project2" -builder.build_temp_path="C:/Users/Public/arduino/.build/publish/project2/temp" -builder.clean=true -builder.verbose=true -builder.verbose_warnings=true -builder.build=true -builder.rebuild=false -builder.upload=false -builder.parallel_build=false -builder.dump_boards=false -builder.dump_props=false -builder.dump_source_list=true -builder.burn_bootloaderx=false -builder-close=true -builder-closeonerror=true -builder.ide_path="C:\Users\Public\arduino\ide" -builder.board_id="ESP32 Pico Kit" -builder.application_id="arduino16x" -builder-configuration="Release"
  
Back to top
 
IP Logged
 
Tim@Visual Micro
Administrator
*****
Offline


Posts: 12071
Location: United Kingdom
Joined: Apr 10th, 2010
Re: Parallel build of multple projects using the CLI can fail (call multiple VMBuild.exe at same time)
Reply #12 - Jul 3rd, 2020 at 12:14pm
Print Post  
Oh. I thought you were trying to build the same project and the same configuration but with different settings. There should be no coflict at all with different projects.

Something confusing:- Your build commands show "Project1" and "Project2". Yet the error is for "Project". You can see this in the error message "C:\Users\Public\arduino\Project\__vm\Configuration.Release.vmps.xml"

How are you running the two commands and do you have a project called "Project"?
  
Back to top
WWW  
IP Logged
 
stm
Junior Member
**
Offline


Posts: 27
Joined: Jul 5th, 2018
Re: Parallel build of multple projects using the CLI can fail (call multiple VMBuild.exe at same time)
Reply #13 - Jul 3rd, 2020 at 12:23pm
Print Post  
Project1 and Project2 are placeholders for actual project names. Also the exception "Project" is a placeholder (so imagine Project1 in the exception)
  
Back to top
 
IP Logged
 
Tim@Visual Micro
Administrator
*****
Offline


Posts: 12071
Location: United Kingdom
Joined: Apr 10th, 2010
Re: Parallel build of multple projects using the CLI can fail (call multiple VMBuild.exe at same time)
Reply #14 - Jul 3rd, 2020 at 12:31pm
Print Post  
Can you switch on the trace of whichever build system you are using. It is useful to know which build is producing what error? Building two different projects with two different Builder.exe instances should not conflict.

I also suspect you can shorten the command to this:-

Code
Select All
"C:\Users\Public\arduino\VMBuilder\VMBuild.exe" builder.project_path="C:/Users/Public/arduino/Project1/Project1.vcxproj"

"C:\Users\Public\arduino\VMBuilder\VMBuild.exe" builder.project_path="C:/Users/Public/arduino/Project1/Project2.vcxproj" 








  
Back to top
WWW  
IP Logged
 
stm
Junior Member
**
Offline


Posts: 27
Joined: Jul 5th, 2018
Re: Parallel build of multple projects using the CLI can fail (call multiple VMBuild.exe at same time)
Reply #15 - Jul 3rd, 2020 at 2:34pm
Print Post  
Unfortunately, I currently do not have the time to dig deeper. I found a workaround though: If I wait for a second or two before starting the parallel build of the next project (currently trying 4 different projects in parallel), I do not get the exception (at least it looks like this). So it must really be accessing some file at the very same time if parallel builds of multiple different projects are initiated at the very same time.
  
Back to top
 
IP Logged
 
Tim@Visual Micro
Administrator
*****
Offline


Posts: 12071
Location: United Kingdom
Joined: Apr 10th, 2010
Re: Parallel build of multple projects using the CLI can fail (call multiple VMBuild.exe at same time)
Reply #16 - Jul 3rd, 2020 at 4:21pm
Print Post  
are you running with tfs automated build?
  
Back to top
WWW  
IP Logged
 
stm
Junior Member
**
Offline


Posts: 27
Joined: Jul 5th, 2018
Re: Parallel build of multple projects using the CLI can fail (call multiple VMBuild.exe at same time)
Reply #17 - Jul 3rd, 2020 at 5:56pm
Print Post  
No but with a local custom script that starts all builds on git checkin
  
Back to top
 
IP Logged
 
Tim@Visual Micro
Administrator
*****
Offline


Posts: 12071
Location: United Kingdom
Joined: Apr 10th, 2010
Re: Parallel build of multple projects using the CLI can fail (call multiple VMBuild.exe at same time)
Reply #18 - Jul 3rd, 2020 at 6:04pm
Print Post  
Maybe it's obvious but it would help to see the script so that we can replicate. It seems that somehow the vmbuilder.exe is running in a single instance but we would like to test.
  
Back to top
WWW  
IP Logged
 
Page Index Toggle Pages: 1
Send TopicPrint