Before logging an issue, please update to the latest release of Visual Micro from the Downloads Page.

When Logging a Support Issue in the Forum, please ensure you have also:-

  • Enabled vMicro > Compiler > Show Build Properties
  • Re-Compile your program with these settings enabled
 
Save the new Output to a Text File and....
  • Click the Reply button and attach as .txt file OR
  • Click here to Email us with the file attached, and a link to your post
Support requests without the output above may be impossible to answer, so please help us to help you
 
Page Index Toggle Pages: [1] 2  Send TopicPrint
Hot Topic (More than 8 Replies) Fun With Subdirectories.... (Read 14713 times)
RayLivingston
Full Member
***
Offline


Posts: 158
Location: California
Joined: Nov 24th, 2012
Fun With Subdirectories....
Jan 1st, 2017 at 4:28pm
Print Post  
I'm using VS2015, and Arduino v1.6.5.  I wanted to consolidate all the classes used by a single app into one directory, to make it easier to track the files, and maintain consistency.  So, yesterday I copied some of the libraries from my local "libraries" folder into the project folder, and updated the properties to point to those new copies.  Unfortunately, despite having NO includes to the original libraries in the "libraries" directory, the build is still trying to reference those original libraries, rather than the copies in the project directory.  It builds correctly only if I re-name either the new libraries, or the originals

Is this expected behavior?  Am I correct in assuming this is some Arduino stupidity?  Has it perhaps been fixed in later versions of Arduino?  I've been reluctant to update, as v1.6.5 has been working well for me for some time, and there have been MANY problems with later versions, though I believe 1.6.10 and beyond appear to be ok.

Happy New Year!

Regards,
Ray L.
  
Back to top
 
IP Logged
 
Tim@Visual Micro
Administrator
*****
Offline


Posts: 12187
Location: United Kingdom
Joined: Apr 10th, 2010
Re: Fun With Subdirectories....
Reply #1 - Jan 1st, 2017 at 5:10pm
Print Post  
Hi,

Not sure I have the full picture it would be easier if you post the .vcxproj file after closing and saving the project. Then I can see what is included in the local project.

It might be that when local project headers match a library header name then the library is taking priority. We don't use the build system of the arduino ide so that could be looked at and rectified.

However if you want to work with lib style includes that don't have paths then maybe consider local project libraries. These are standard arduino libraries below the project folder in a known special folder structure so can be included in the same way as normal libs and are not part of the project code. Use the "Add Code>Create Library" to see the spcial folder structure.

Maybe of interest.. the next version allows you to use C++ shared projects which allow you to have code in multiple projects and shared as if each is directly part of the mater project.

https://blogs.msdn.microsoft.com/vcblog/2015/07/30/cross-platform-code-sharing-w...
« Last Edit: Jan 1st, 2017 at 5:12pm by Tim@Visual Micro »  
Back to top
IP Logged
 
RayLivingston
Full Member
***
Offline


Posts: 158
Location: California
Joined: Nov 24th, 2012
Re: Fun With Subdirectories....
Reply #2 - Jan 1st, 2017 at 6:00pm
Print Post  
Tim,

vxproj file attached.  I think you've pointed me to the problem, but I'm not sure how to fix it.  When I look at the vxproj file, there is a field "AdditionalIncludeDirectories" which contains paths to about a bazillion individual library files in the libraries folder.  Those entries should, I think, be unnecessary, since the libraries directory will be searched.  How did those get there, and how do I get rid of them, other than by hand-editing the vxproj file (which seems like a bad idea...).

vxproj file attached.

Regards,
Ray L.

  

Please Register or Login to the Forum to see File Attachments
Back to top
 
IP Logged
 
RayLivingston
Full Member
***
Offline


Posts: 158
Location: California
Joined: Nov 24th, 2012
Re: Fun With Subdirectories....
Reply #3 - Jan 1st, 2017 at 6:09pm
Print Post  
This is odd....  I found the AdditionalIncludeDirectories under Project->Properties, but there are numerous references to sub-directories of libraries in the vxproj file that do NOT show up that field in the Project->Properties dialog....

Regards,
Ray L.
  
Back to top
 
IP Logged
 
Tim@Visual Micro
Administrator
*****
Offline


Posts: 12187
Location: United Kingdom
Joined: Apr 10th, 2010
Re: Fun With Subdirectories....
Reply #4 - Jan 2nd, 2017 at 12:50am
Print Post  
Hi Ray,

The AdditionalIncludeDirectories are managed by Visual Micro and are purely for intellisense. They have to match what the arduino compile should do.

I tested this in the arduino ide and it also compiles the library regardless of if the source code of a local version exists in the project.

It's important to help most users stay compatible with Arduino wherever possible. It would be confusing if by default the two IDE's handled this differently.

I know that can be restricting for more experienced users so that's where we can add a setting to optionally provide different functionality.

Having said this you might run into problems trying to move libraries to local project code. Normally they are compiled in isolation to the project code and either the lib\utility sub folder or sub folders below the lib\src folder are compiled with their own isolated "-I includes" (private lib code)

How would you like it to work?

Another option is to switch off vMicro>compiler>deep search and then #include the headers from a .h and/or .cpp not from .ino. Then the library won't be discovered.

  
Back to top
IP Logged
 
RayLivingston
Full Member
***
Offline


Posts: 158
Location: California
Joined: Nov 24th, 2012
Re: Fun With Subdirectories....
Reply #5 - Jan 2nd, 2017 at 1:32am
Print Post  
Tim,

I suppose my biggest problem is not fully understanding how it does work.  Being compatible with the Arduino IDE has not been an issue for me for years, as several of my projects are already doing things the IDE does not support, or support poorly.  The ONLY reasons I use Arduino at all are the really cheap hardware, ready availability of many useful libraries, and inertia.

It seems like every time I move a library I end up spending two hours getting the project to compile again.  It seems really odd to me that using #include with quotes would EVER find a file in the "system" library without first exhaustively searching all other locations.  But, I guess that's Arduino for you.

Truth be told, my life would be easier if I could just use a make file (I actually already HAVE a working template for a makefile), but then I'd lose the quick and easy linking to the error lines through the VS Error pane (which would be really painful) and the automatic upload of the new code to the hardware.  If there was a way to use a make file and NOT lose those capabilities, I'd probably be a happy camper.  I'd then ditch the whole setup/loop paradigm, and just write my own main(), like in the good old days.  I think the "help" the Arduino build process provides causes at least as many problems as it solves.

Quote:

Another option is to switch off vMicro>compiler>deep search and then #include the headers from a .h and/or .cpp 
not from .ino. Then the library won't be discovered.


That sounds like it might be a help.  Can you provide more information, or a link?

Regards,
Ray L.
  
Back to top
 
IP Logged
 
RayLivingston
Full Member
***
Offline


Posts: 158
Location: California
Joined: Nov 24th, 2012
Re: Fun With Subdirectories....
Reply #6 - Jan 2nd, 2017 at 1:46am
Print Post  
Tim,

OK, I searched on "Deep search include".  Oddly, searching on your site found nothing of value.  But a Google search brought up a page on your site that explained it.  I turned it off, put some duplicate libraries in my user library, and did a clean build with no problems.  So, that seems to have solved my immediate problem.

I am still curious about the rest of your comment, to make sure I understand any additional options I may have.

Regards,
Ray L.
  
Back to top
 
IP Logged
 
Tim@Visual Micro
Administrator
*****
Offline


Posts: 12187
Location: United Kingdom
Joined: Apr 10th, 2010
Re: Fun With Subdirectories....
Reply #7 - Jan 2nd, 2017 at 2:43am
Print Post  
Yes it's a difficult line to walk, keep some things simple but loose flexibility.

It would be useful to know why you want to work with local code instead of libraries. It makes sense for various reasons but it's important for me to understand your reasons before I make suggestions.

It's also an idea to look at the shared projects in the next release. They offer a way to separate the code into library like modules but the code is merge with the project code (hidden) during compilation. Microsoft have done a good job by the looks of it, intellisense in shared projects inherits from the (current) project that uses it so it's easy to develop.
  
Back to top
IP Logged
 
RayLivingston
Full Member
***
Offline


Posts: 158
Location: California
Joined: Nov 24th, 2012
Re: Fun With Subdirectories....
Reply #8 - Jan 2nd, 2017 at 2:50am
Print Post  
Tim,

My logic, if you can call it that, is pretty simple.  I use shared libraries where I can, but I sometimes start with a shared library, then end up having to make customizations that may break backwards compatibility with older projects.  So, in practice, I end up with derived libraries that are unique to a single project.  The changes sometimes can't be easily done purely through OOP, so the path of least resistance is to just copy the library into the project, and modify it there.

Regards,
Ray L.
  
Back to top
 
IP Logged
 
Tim@Visual Micro
Administrator
*****
Offline


Posts: 12187
Location: United Kingdom
Joined: Apr 10th, 2010
Re: Fun With Subdirectories....
Reply #9 - Jan 2nd, 2017 at 2:55am
Print Post  
Okay so why wasn't the local library facility I mentioned previously of use? It does exactly as you describe. You can copy a library locally and the compiler will use the local one instead of the original. 

The sources can be included in the project so easy to work with yet because they are in a special folder they remain as libraries in all other respects.
« Last Edit: Jan 2nd, 2017 at 2:55am by Tim@Visual Micro »  
Back to top
IP Logged
 
RayLivingston
Full Member
***
Offline


Posts: 158
Location: California
Joined: Nov 24th, 2012
Re: Fun With Subdirectories....
Reply #10 - Jan 2nd, 2017 at 4:42pm
Print Post  
Tim,

MAybe I'm dense, but I'm afraid I don't understand that option, and can't find any documentation for it.  When I click that option, all it seems to do is open a "libs.h" file.  I see no "directory structure", and have no idea what to do from there....

Regards,
Ray L.
« Last Edit: Jan 2nd, 2017 at 4:44pm by RayLivingston »  
Back to top
 
IP Logged
 
Tim@Visual Micro
Administrator
*****
Offline


Posts: 12187
Location: United Kingdom
Joined: Apr 10th, 2010
Re: Fun With Subdirectories....
Reply #11 - Jan 2nd, 2017 at 6:44pm
Print Post  
Okay so open a project and switch the solution explorer to "show all files"

Then right click and create a folder called "src"

Then create another folder below that called "_micro-api"

Then another folder below that called "libraries"

Then create a folder below "libraries" called MyTestLibrary

Put a MyTestLibrary.cpp and MyTestLibrary.h in the folder

Then in your normal project code #include "MyTestLibrary.h" as if it is a normal arduino library.

You will have created and used a local arduino library called MyTestLibrary

Then open windows explorer and navigate to the folder so you can see that the "show all files" view is simply showing the folder structure same as windows explorer.

The only difference between "show all files" and windows explorer is that you can right click the solution explorer and toggle sources in/out of the project. 

So you end up with a normal sketch project that also contains libraries

There is an update coming hopefully later today with a fix for the "Add Code>Create Arduino Library" which crashes in the last release so isn;t much help. If you can't see the menu item then maybe you are on an older visual micro build.

  
Back to top
IP Logged
 
RayLivingston
Full Member
***
Offline


Posts: 158
Location: California
Joined: Nov 24th, 2012
Re: Fun With Subdirectories....
Reply #12 - Jan 2nd, 2017 at 6:57pm
Print Post  
Tim,

I guess I'm having a hard time seeing the difference....  I already have dozens of libraries in sub-directories of the project directory.  I can see that this lets me group libraries under a common directory, just like the normal user libraries directory.  What's not clear is how/why this would be any different from what I already have, and how it changes the library search at build time (which was my original problem).  Does the standard libraries directory still get searched?  Is it searched AFTER this new one (which would eliminate the problem that started this thread).

Regards,
Ray L.
  
Back to top
 
IP Logged
 
Tim@Visual Micro
Administrator
*****
Offline


Posts: 12187
Location: United Kingdom
Joined: Apr 10th, 2010
Re: Fun With Subdirectories....
Reply #13 - Jan 2nd, 2017 at 7:43pm
Print Post  
Sorry I am rushing and not being clear

All of the library code you have added to sub folders is no different from normal code.

HOWEVER

Using the special folder names \src\_micro-api\libraries is where the difference is. 

This is assuming you have a licensed version of visual micro and not just the free version?

« Last Edit: Jan 2nd, 2017 at 7:43pm by Tim@Visual Micro »  
Back to top
IP Logged
 
RayLivingston
Full Member
***
Offline


Posts: 158
Location: California
Joined: Nov 24th, 2012
Re: Fun With Subdirectories....
Reply #14 - Jan 3rd, 2017 at 1:14am
Print Post  
Tim,

I have a Pro license.   

But what ARE those differences you mention, other than the location of the files?  If the build process is not somehow altered by putting the files in \src\_micro-api\libraries, what is the point?  There must be some difference, that, AFAICT, is not explained anywhere.

Regards,
Ray L.
  
Back to top
 
IP Logged
 
Tim@Visual Micro
Administrator
*****
Offline


Posts: 12187
Location: United Kingdom
Joined: Apr 10th, 2010
Re: Fun With Subdirectories....
Reply #15 - Jan 3rd, 2017 at 3:00am
Print Post  
Yep just as the process is altered to build libraries, which can exist in various known locations. The src\_micro-api\libraries folder simply becomes one of those locations.

Keep in mind that libraries and the core is pulled from various locations and the .ino code is combined into a single .cpp. To the arduino build happens in a temp folder which you will see if you switch on the vMicro>Compiler>Verbose and "Show build folder" properties.

So Visual Micro does treat that location in a special way. It allows you to create or overrride any arduino library whilst still maintaining compatibility with arduino in the way libraries are #included and compiled.
« Last Edit: Jan 3rd, 2017 at 3:00am by Tim@Visual Micro »  
Back to top
IP Logged
 
RayLivingston
Full Member
***
Offline


Posts: 158
Location: California
Joined: Nov 24th, 2012
Re: Fun With Subdirectories....
Reply #16 - Jan 3rd, 2017 at 4:29am
Print Post  
Tim,

Not trying to be difficult, but....  It's still not clear.  My whole problem (this time) started because I copied some library folders from my "user/libraries" folder, to my project folder.  They are then, like all libraries, separately compiled, in their own sub-directories off the build directory, except somehow the build was STILL (due to, I assume, a bug) building with the "old" libraries of the same name from the "user/libraries" folder, rather than using the new copies below my project folder, despite the fact that the project properties point to the new copies.  It acts exactly as though it's searching "user/libraries" first, even though I used a quoted include, rather than a bracketed include.   

So, I still don't see how putting them instead in the "\src\_micro-api\libraries" folder below my project folder, unless the build treats those differently.  I can imagine several things you might be doing that could be useful, but don't know what you actually ARE doing.  So far, simply turning off "deep search" seems to have circumvented the original problem.  What do I gain by instead moving things down into "\src\_micro-api\libraries".

Regards,
Ray L.
  
Back to top
 
IP Logged
 
Tim@Visual Micro
Administrator
*****
Offline


Posts: 12187
Location: United Kingdom
Joined: Apr 10th, 2010
Re: Fun With Subdirectories....
Reply #17 - Jan 3rd, 2017 at 1:55pm
Print Post  
Quote:
in their own sub-directories off the build directory, except somehow the build was STILL (due to, I assume, a bug) building with the "old" libraries of the same name from the "user/libraries" folder, rather than using the new copies below my project folder,


It's not  bug, it is the design of Arduino.

Quote:
I still don't see how putting them instead in the "\src\_micro-api\libraries" folder below my project folder, unless the build treats those differently


That's right, the build treats the "\src\_micro-api\libraries" sub folder differently.
« Last Edit: Jan 3rd, 2017 at 1:55pm by Tim@Visual Micro »  
Back to top
IP Logged
 
Tim@Visual Micro
Administrator
*****
Offline


Posts: 12187
Location: United Kingdom
Joined: Apr 10th, 2010
Re: Fun With Subdirectories....
Reply #18 - Jan 17th, 2017 at 11:36pm
Print Post  
Hi Ray,

Just to let you know that the latest release for vs2015 and 2017rc takes advantage of Visual Studio C++ shared projects.

Visual Micro supports both libraries and extended project sources as shared projects.

The shared libraries will make life easier for you. 

The current system of placing libraries under the _micro-api\src\libraries folder is of course still supported but with a shared library project the library can be in any location.

The code doesn't change, shared library projects are Arduino libraries but that contain a libName.vcxproj file in the root folder. This enabled the library to be opened as a normal visual studio project and can be added to multiple different solutions again avoiding the need to be working on lib sources buried the main project.

Visual Studio will automatically show intellisense inherited from the active solutions "Startup Project" so developing cross-platform code is very easy.

There are some rough draft notes here and some options to copy/clone libraries more easily will follow.


« Last Edit: Jan 17th, 2017 at 11:37pm by Tim@Visual Micro »  
Back to top
IP Logged
 
Opsimath
Newbies
*
Offline


Posts: 6
Joined: Jan 23rd, 2017
Re: Fun With Subdirectories....
Reply #19 - Jan 24th, 2017 at 5:22pm
Print Post  
Hi Tim,

Like @RayLivingston I am trying to understand header resolution rules for my project because like him I want to maintain project specific versions of my Arduino libraries so that I can modify them without affecting other unrelated projects.  This is a *very* common development practice.  I know vMicro is dependent on the Arduino IDE's design but can still share with users how things work in the build process, then we can adapt.  You have dropped most of the breadcrumbs above.  I guess my question is why would this not be documented in detail somewhere besides an obscure forum post?  vMicro is selling a professional product to professional users.  It is in their interest to document how that product works in a standard environment.  Most developers are pretty smart and will get it if the details are available.

To answer your question above as to *why* a developer would want project specific versions of a library consider the case that I want to encapsulate a fully independent version of my project to share with another dev machine, or a co-worker, who has no custom modifications to their build environment.  I want to be able to point Git at a folder, create a repository, post it for others to enlist in and collaborate.  This implies that when I make a change to the project and sync with Git those changes should not break the build on other machines nor require special instructions to manually change the build environment (e.g. copying libraries to Arduino IDE folders.

I think you have provided a solution in the src/_micro_api/libraries folder.  I have yet to try that.  If indeed that is the answer a blog post, release note or other discoverable document would be a huge advantage to other vMicro users.

FWIW, if I understand correctly, the VS shared projects solution does not address my scenario because the whole point is *not* to share the library outside my project thus adding dependencies and potential conflicts between projects.  I just want a local copy that I know will be used consistently.

I hope this explanation helps you/vMicro understand what I believe to be a common need among their users and allows them to make an even better product!
  
Back to top
 
IP Logged
 
Page Index Toggle Pages: [1] 2 
Send TopicPrint