Visual Studio Ide for Arduino
https://www.visualmicro.com/forums/YaBB.pl
Visual Micro >> Installation & Troubleshooting >> Visual Micro Arduino Command Line Interface failes to build Project from command line
https://www.visualmicro.com/forums/YaBB.pl?num=1567426178

Message started by stm on Sep 2nd, 2019 at 1:09pm

Title: Visual Micro Arduino Command Line Interface failes to build Project from command line
Post by stm on Sep 2nd, 2019 at 1:09pm
I am trying to automate a build of a project using your Command Line Interface tool (https://www.visualmicro.com/page/Visual-Micro-Command-Line-Interface.aspx).

The project builds from within Visual Studio but the build fails from the command line. Already the first header file that is used from one of our own libraries is not found and the build fails.

To me it looks like the "AdditionalIncludeDirectories" property of the project file is not respected when using the Command Line Interface. Also the additional defines don't seem to be used for the CLI build.

The AdditionalIncludeDirectories property from the project .vcxproj file looks as follows:

[code] <PropertyGroup />
  <ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">
    <ClCompile>
      <WarningLevel>Level3</WarningLevel>
      <Optimization>Disabled</Optimization>
      <SDLCheck>true</SDLCheck>
      <AdditionalIncludeDirectories>$(ProjectDir)..\RunningTargetMotor;$(ProjectDir)..\libraries\MinionSanityCheck\src; ...
...[/code]

Attachment 1: Failing build log of CLI
Attachment 2: Working build log of Visual Studio

Let me know if you need more details of anything. Thank you.
https://www.visualmicro.com/forums/YaBB.pl?action=downloadfile;file=VmCliBuildLog_20190902_1400.txt (12 KB | 15 )
https://www.visualmicro.com/forums/YaBB.pl?action=downloadfile;file=VmVsBuildLog_20190902_1400.txt (688 KB | 10 )

Title: Re: Visual Micro Arduino Command Line Interface failes to build Project from command line
Post by Visual Micro on Oct 12th, 2019 at 2:37pm
Sorry this took so long. The command line tool has been updated.

Visual Micro Command Line Tool

Title: Re: Visual Micro Arduino Command Line Interface failes to build Project from command line
Post by stm on Oct 14th, 2019 at 3:11pm
Hello Tim
Just tried the new version and unfortunately the issue has not been resolved from what I see. All entries in AdditionalIncludeDirectories are still not considered. The build log shows many -IC entries but none of them coming from the .vcxproj project file. the build log looks the same as above and shows the same error messages so I pass on attaching them again.

Title: Re: Visual Micro Arduino Command Line Interface failes to build Project from command line
Post by Visual Micro on Oct 14th, 2019 at 3:37pm
Hi

Sorry I didn't fully read your intiial post. There was an issue with the command line tool which is now fixed however your report about include directories is unrelated.

The additiona include directories in the project settings should be considered read-only for intellisense only.

You can specify additional compiler includes using the extra flags F4 project properties. Use the -IdirName syntax.

Switching on the build properties on the vMicro compiler menu will also allow you to see all the build variables which can also be used when specifying include paths. For example, if there is a project_path variable it can be used like so:-

-I{project_path}\..\myHeaders -I{project_path}\..\myHeaders2



Title: Re: Visual Micro Arduino Command Line Interface failes to build Project from command line
Post by Visual Micro on Oct 19th, 2019 at 3:35pm
Once again appologies for the delay and thanks for providing such useful info in the outputs.

The issue is some confusion that we will clarify in the documentation and also provide better feed back in the CLI.

As stated previously the include paths are not stored in VS, in fact they are dynamically recreated each time a project changes or builds. That is for Vs intellisense.

In Arduino paths are generally worked out dynamically and in this case what is missing is the Arduino Sketch Book folder location. We can see the sketch book location easly in the IDE build output but it is missing from your CLI build output.

For Arduino, the user installed libraries are always assumed to be in "SketchBook\Libraries" which the CLI can't see. This is why you believed include paths are missing.

The reason the CLI can not see the sketchbook is that your CI command does not specify an IDEName (-builder.application_id=) therefore Visual Micro will use the NoIDE option (that you can see in the VsIDE).

In the VsIDE you have specified a Sketch Book folder location of "Public/Arduino/ArduinoGIT" for the "Arduino 1.6/1.8" application but have not set a Sketch Book folder location for the NoIDE application. Hence an unknown Sketch Book location will probably default to "myDocuments\Arduino" where your \libraries do not exist.

1 - NoIDE


Code (]vmbuild.exe  -builder.ide_path="D:\Arduino\arduino-1.8.1" -builder.upload=false -builder.build=true -builder.project_path="C:\Users\tl\Documents\Arduino\MyProject\MyProject.vcxproj"  -builder.verbose=true -builder.verbose_warnings=true -builder.board_id=Pico [/code):
2 - Arduino 1.6/1.8 IDE

[code]vmbuild.exe -builder.application_id=arduino16x -builder.upload=false -builder.build=true -builder.project_path="C:\Users\tl\Documents\Arduino\MyProject\MyProject.vcxproj"  -builder.verbose=true -builder.verbose_warnings=true -builder.board_id=Pico


Solution

Therefore the solution is to either add the optional Sketch Book Location to the NoIDE option in Visual Micro or use the 2nd command syntax example above.

I hope this makes sense?

Title: Re: Visual Micro Arduino Command Line Interface failes to build Project from command line
Post by stm on Nov 6th, 2019 at 12:28pm
Hello Tim

Thanks for your last message: That really helped to proceed. Two issues remain:

1) small issue: VM_ADDITIONAL_PREPROG from the .vcxproj seem to be ignored -> would be nice if this was also taken this into account

2) Now there are errors during the linking. Please see the attached log. What did we forget?
https://www.visualmicro.com/forums/YaBB.pl?action=downloadfile;file=VMBuilder_linking_error.txt (625 KB | 10 )

Title: Re: Visual Micro Arduino Command Line Interface failes to build Project from command line
Post by Visual Micro on Nov 6th, 2019 at 1:33pm
Yes we will add the additonal_preprog to the cli over the next week or so.

For the linker error we might need the source of a simple example with the problem. One thing is to make sure you do not have any unwanted source code files in the poroject folder. regardless of what you see in the solution explorer vm will compile all files in the project folder (default setting) because the arduino ide will do the same.


Title: Re: Visual Micro Arduino Command Line Interface failes to build Project from command line
Post by stm on Nov 6th, 2019 at 4:52pm
Hello again

1) Great news!

2) I attached a simple example project. The command to build it:

VMBuild.exe -builder.application_id=arduino16x -builder.upload=false -builder.build=true -builder.project_path="C:\Users\Public\arduino\ExampleProject\libraries\MinionGpio\example\MinionGpioReadWriteExample\MinionGpioReadWriteExample.vcxproj" -builder.verbose=true -builder.verbose_warnings=true -builder.board_id=Pico -builder.output_directory="C:\Users\Public\arduino\ExampleProject\libraries\MinionGpio\example\MinionGpioReadWriteExample\build"
https://www.visualmicro.com/forums/YaBB.pl?action=downloadfile;file=ExampleProject-20191106T164940Z-001.zip (5093 KB | 8 )

Title: Re: Visual Micro Arduino Command Line Interface failes to build Project from command line
Post by Visual Micro on Nov 6th, 2019 at 6:52pm
Hi,

Before using the cli we need to ensure the project builds in VS first. I guess it failed?

One issue with the linker is that the use of TwoWire is incorrect. I changed it to the following so that it links.


Code (c++):
//vm edited will not work but will compile
TwoWire  Wire2 = TwoWire(&sercom3, 20, 21);
MinionI2cBus _i2c(Wire2, _i2cBus, SDA, SCL);


There might also be some confusion when attempting to use a normal arduino project below an arduino library. The link below shows a a solution with your project outside of the libs area. We will investigate this some more but the solution above works.

I might be wrong but if I recall arduino libs with library properties have the code in a folder called \src. This is the arduino v2 lib format. You will see that I moved the lib code to src folders. v1 format is supported but it's not very expandable due to limitiations of library sub folder.

I side note that if you move libraries that are referenced as shared libraries the main project might not open. This is a VS quirk but you can edit the vcxproj file with a text editor and remove or correct the shared lib reference paths.

This project builds okay for me although you might want to change your project and make correction to TwoWire init.

https://1drv.ms/u/s!AsT00oFsGAmRoKZjdw7X78evwOMpDA?e=gtieFr

It's possible you original config would work okay if you set the arduino sketch book path as you did. However when using shared projects you can leave sketch book as default so that any downloaded libraries stay outside of your code base. It's a preference.

Title: Re: Visual Micro Arduino Command Line Interface failes to build Project from command line
Post by stm on Nov 11th, 2019 at 10:49am
Hello Tim

TwoWire was no issue for us and of course the projects were building fine in VisualStudio but not using the CLI tool. The issues in this thread could only be seen in the CLI tool. After playing around a lot, we found that the following two points really helped to make it build in the CLI tool as well:

1) We indeed had to follow the convention of v2 and put user library code in a folder named "src"
2) Also for projects created a few months ago (can't recall exactly how, maybe using VM toolset?), we had to create new projects from scratch and then copy over the code. Then the errors dissapeared and the projects could also be built using the CLI tool.

I think we made real progress towards building projects using scripts. Thanks for your help and we are looking forward to the new version of the CLI that supports additonal_preprog!

Title: Re: Visual Micro Arduino Command Line Interface failes to build Project from command line
Post by Visual Micro on Nov 11th, 2019 at 3:01pm
Thanks, yes it won't be long for the next cli release.

Title: Re: Visual Micro Arduino Command Line Interface failes to build Project from command line
Post by stm on Nov 11th, 2019 at 6:13pm
One thing that we noticed was that one has to provide an absolute output path. If a relative path is specified then the output goes to some buried folder in AppData (see example below). I don't know if this is by design but from my point of view it would be better to be able to output to a relative path from the working directory path.


Example for -builder.output_directory=".build/publish/example.application" :

End
# Copy build result to 'Project>Property Pages>Intermediate Directory'
# Destination: file:///C:/Users/stm/AppData/Local/Temp/VMBuilds/ExampleProject/espressif_pico32/Release/.build/publish/example.application

Title: Re: Visual Micro Arduino Command Line Interface failes to build Project from command line
Post by Visual Micro on Nov 12th, 2019 at 1:12pm
Yes will do thanks

Title: Re: Visual Micro Arduino Command Line Interface failes to build Project from command line
Post by stm on Nov 15th, 2019 at 7:36am
Me again.. I have another important factor that breaks the build when using the CLI instead of Visual Studio:

When adding user libraries to a solution and adding references to the project, the CLI fails to build:

<Import Project="$(VCTargetsPath)\Microsoft.Cpp.props" />
<ImportGroup Label="ExtensionSettings">
</ImportGroup>
<ImportGroup Label="Shared">
<Import Project="..\libraries\MinionLogging\MinionLogging.vcxitems" Label="Shared" />
<Import Project="..\libraries\BldcDriver\BldcDriver.vcxitems" Label="Shared" />
</ImportGroup>
...

When removing those references, the CLI builds just fine:

<Import Project="$(VCTargetsPath)\Microsoft.Cpp.props" />
<ImportGroup Label="ExtensionSettings">
</ImportGroup>
<ImportGroup Label="Shared" />
...

My motivation to add user libraries to the solution and references to the project is that one can work on the libraries better and Intellisense works fine that way.

Is this a bug in the CLI or am I doing something wrong?

Title: Re: Visual Micro Arduino Command Line Interface failes to build Project from command line
Post by Visual Micro on Nov 16th, 2019 at 11:00pm
The VMBuild.exe tool has been updated. Thanks the feedback.

Command Line Tool for Visual Studio projects

nb: Because of the error you encountered previously, you might find a "Build>Clean Solution" in Visual Studio is required before first use of the CLI for the failed project. If the last build was a VS build then that should not be required.



Title: Re: Visual Micro Arduino Command Line Interface failes to build Project from command line
Post by stm on Nov 19th, 2019 at 7:26pm
Thanks.. it looks very good for the automated build now!!

One issue is left (probably the last one as it looks): I moved the working build setup to a BuildAgent (TeamCity) and tried to automate the build from there. It fails with "An error was encountered during the 'Deep Search' library discovery process." It does not find the header file of the first included library. I know that TeamCity sets some Environment Variables like for instance the TEMP directory. Do you rely on any environment paths to be fixed (maybe hardcoded somewhere in your code)? If so it would be great to be able to override those by a parameter.

Title: Re: Visual Micro Arduino Command Line Interface failes to build Project from command line
Post by Visual Micro on Nov 19th, 2019 at 7:32pm
The temp build folders are shown in the output. We need to see the verbose error to see what is a problem.

Possibly it can't see your users\name\appdata\local\arduino15 folder in which case move the contents to a folder below the arduino ide called portable.

Title: Re: Visual Micro Arduino Command Line Interface failes to build Project from command line
Post by stm on Nov 19th, 2019 at 7:48pm
I know it's really strange as everything else works now. That's why was hoping for a simple explanation (hardcoded path or similar). Please find the two logs attached. Once directly from script and once running the same script in the BuildAgent TeamCity context (both on same hardware).

Please note the additional include of "libraries\ezTime\src" in the working case:
https://www.visualmicro.com/forums/YaBB.pl?action=downloadfile;file=failed.txt (29 KB | 11 )
https://www.visualmicro.com/forums/YaBB.pl?action=downloadfile;file=working.txt (22 KB | 14 )

Title: Re: Visual Micro Arduino Command Line Interface failes to build Project from command line
Post by Visual Micro on Nov 19th, 2019 at 7:55pm
I need to see the full build when it works because that shows what was discovered by deep search. I suspect a library issue.

This is the error you see when it fails. exTime.h is not part of the project code so we need to look at libraries.

I suggest adding "-builder.dump_props=true" to the CLI


Code (c++):
C:\Users\Public\arduino\companyarduinoGIT\RunningMotor\RunningMotor.ino:7:20: fatal error: ezTime.h: No such file or directory

Title: Re: Visual Micro Arduino Command Line Interface failes to build Project from command line
Post by stm on Nov 19th, 2019 at 11:40pm
Your hint with -builder.dump_props=true argument showed that the only difference is that in the failing case of the BuildAgent (automated build using TeamCity) the sketchbook location is not the custom defined one but the default one which is probably the issue:

location.sketchbook=C:\Users\buildadmin\Documents\Arduino

Where does VMBuilder get this path from meaning from where should the custom path be read out?

AMEND: I could just add "-location.sketchbook=<custom_sketchbook_path>" as a build argument and that was accepted and also shown correctly in the dump_props lists. But it did not change the fact that deep search failed at the very beginning.


Title: Re: Visual Micro Arduino Command Line Interface failes to build Project from command line
Post by Visual Micro on Nov 20th, 2019 at 12:22am
it should be picking it up from the arduino 1.6/1.8 Ide Location if you have set it.

Otherwise it would pick it up from users\name\appdata\local\arduino15\preferences.txt which is where the arduino ide stores file>preferences>sketchbook location or the arduinoide\portable folder if you have mpoved the contents of arduino15 to it

The users\name\appdata\local\arduino15 folder must not exist otherwise the \portable folder will not be used



Title: Re: Visual Micro Arduino Command Line Interface failes to build Project from command line
Post by Visual Micro on Nov 20th, 2019 at 10:13pm
Where have you set the sketch book folder. In the arduino ide or in visual micro?

Title: Re: Visual Micro Arduino Command Line Interface failes to build Project from command line
Post by Visual Micro on Nov 27th, 2019 at 4:04pm
Thanks for the email. Can you confirm that you have set a sketchbook folder against the arduino 1.6/1.8 application in the visual micro IDE locations dialog, so that you are not relying on the arduino ide preferences.

If you rely on the arduino ide preferences then you are back in the users\name\appData\Local\arduino15 folder where the arduino ide preferences.txt is stored.

Does this make sense?

Title: Re: Visual Micro Arduino Command Line Interface failes to build Project from command line
Post by stm on Dec 8th, 2019 at 10:05pm
I have found a workaround in the meantime. The build agent service that is set up by TeamCity and that performs the automated build runs as a windows service ("TeamCity Build Agent"). And as such it runs under the SYSTEM user by default. After changing this user under Control Panel|Administrative Tools|Services, the build finally works.

Unfortunately, I did not find out why the build did not work otherwise. I changed APPDATA and other relevant environment variables. So I guess there must be another mechanism in the background that I have not understood yet.

Anyhow, with the described workaround, the automated build now works using the CLI tool.

Title: Re: Visual Micro Arduino Command Line Interface failes to build Project from command line
Post by Visual Micro on Dec 12th, 2019 at 6:10pm
Great thanks for the update.

FYI: The problem or alternative solution is as follows:-

Moving the Arduino15 folder to the \portable folder, below below the IDE would avoid use of the AppData folder. If you do not set the sketchBook path in Visual Micro then the one from the Arduino IDE is used, that is stored in the appData\Local\Arduino15\preferences.txt. You are not using the \portable folder and you have changed the default sketchbook path in the Arduino IDE for the current user. Therefore a different system user would probably have been using the default sketchBook path which is myDocuments\Arduino. Therefore user installed libraries would be in myDocuments\Arduino\Libraries.

Visual Studio Ide for Arduino » Powered by YaBB 2.5.2!
YaBB Forum Software © 2000-2020. All Rights Reserved.