Batch rename with MSBuild

Question

I just joined a team that has no CI process in place (not even an overnight build) and some sketchy development practices. There's desire to change that, so I've now been tasked with creating an overnight build. I've followed along with this series of articles to: create a master solution that contains all our projects (some web apps, a web service, some Windows services, and couple off tools that compile to command line executables); created an MSBuild script to automatically build, package, and deploy our products; and created a .cmd file to do it all in one click. Here's a task that I'm trying to accomplish now as part of all this:

The team currently has a practice of keeping the web.config and app.config files outside of source control, and to put into source control files called web.template.config and app.template.config. The intention is that the developer will copy the .template.config file to .config in order to get all of the standard configuration values, and then be able to edit the values in the .config file to whatever he needs for local development/testing. For obvious reasons, I would like to automate the process of renaming the .template.config file to .config. What would be the best way to do this?

Is it possible to do this in the build script itself, without having to stipulate within the script every individual file that needs to be renamed (which would require maintenance to the script any time a new project is added to the solution)? Or might I have to write some batch file that I simply run from the script?

Furthermore, is there a better development solution that I can suggest that will make this entire process unnecessary?

Answer 1

After a lot of reading about Item Groups, Targets, and the Copy task, I've figured out how to do what I need.

<ItemGroup>
    <FilesToCopy Include="..\**\app.template.config">
        <NewFilename>app.config</NewFilename>
    </FilesToCopy>
    <FilesToCopy Include="..\**\web.template.config">
        <NewFilename>web.config</NewFilename>
    </FilesToCopy>
    <FilesToCopy Include"..\Hibernate\hibernate.cfg.template.xml">
        <NewFilename>hibernate.cfg.xml</NewFilename>
    </FilesToCopy>
</ItemGroup>

<Target Name="CopyFiles"
        Inputs="@(FilesToCopy)"
        Outputs="@(FilesToCopy->'%(RootDir)%(Directory)%(NewFilename)')">
    <Message Text="Copying *.template.config files to *.config"/>
<Copy SourceFiles="@(FilesToCopy)"
      DestinationFiles="@(FilesToCopy->'%(RootDir)%(Directory)%(NewFilename)')"/>

I create an item group that contains the files that I want to copy. The ** operator tells it to recurse through the entire directory tree to find every file with the specified name. I then add a piece of metadata to each of those files called "NewFilename". This is what I will be renaming each file to.

This snippet adds every file in the directory structure named app.template.config and specifies that I will be naming the new file app.config:

<FilesToCopy Include="..\**\app.template.config">
    <NewFilename>app.config</NewFilename>
</FilesToCopy>

I then create a target to copy all of the files. This target was initially very simple, only calling the Copy task in order to always copy and overwrite the files. I pass the FilesToCopy item group as the source of the copy operation. I use transforms in order to specify the output filenames, as well as my NewFilename metadata and the well-known item metadata.

The following snippet will e.g. transform the file c:\Project\Subdir\app.template.config to c:\Project\Subdir\app.config and copy the former to the latter:

<Target Name="CopyFiles">
    <Copy SourceFiles="@(FilesToCopy)"
          DestinationFiles="@(FilesToCopy->'%(RootDir)%(Directory)%(NewFileName)')"/>
</Target>

But then I noticed that a developer might not appreciate having his customized web.config file being over-written every time the script is run. However, the developer probably should get his local file over-written if the repository's web.template.config has been modified, and now has new values in it that the code needs. I tried doing this a number of different ways--setting the Copy attribute "SkipUnchangedFiles" to true, using the "Exist()" function--to no avail.

The solution to this was building incrementally. This ensures that files will only be over-written if the app.template.config is newer. I pass the names of the files as the target input, and I specify the new file names as the target output:

<Target Name="CopyFiles"
        Input="@(FilesToCopy)"
        Output="@(FilesToCopy->'%(RootDir)%(Directory)%(NewFileName)')">
      ...
</Target>

This has the target check to see if the current output is up-to-date with respect to the input. If it isn't, i.e. the particular .template.config file has more recent changes than its corresponding .config file, then it will copy the web.template.config over the existing web.config. Otherwise, it will leave the developer's web.config file alone and unmodified. If none of the specified files needs to be copied, then the target is skipped altogether. Immediately after a clean repository clone, every file will be copied.

The above turned out be a satisfying solution, as I've only started using MSBuild and I'm surprised by its powerful capabilities. The only thing I don't like about it is that I had to repeat the exact same transform in two places. I hate duplicating any kind of code, but I couldn't figure out how to avoid this. If anyone has a tip, it'd be greatly appreciated. Also, while I think the development practice that necessitates this totally sucks, this does help in mitigating that suck factor.

Answer 2

Short answer:
Yes, you can (and should) automate this. You should be able to use MSBuild Move task to rename files.

Long answer:
It is great that there is a desire to change from a manual process to an automatic one. There are usually very few real reasons not to automate. Your build script will act as living documentation of how build and deployment actually works. In my humble opinion, a good build script is worth a lot more than static documentation (although I am not saying you should not have documentation - they are not mutually exclusive after all). Let's address your questions individually.

What would be the best way to do this?

I don't have a full understanding of what configuration you are storing in those files, but I suspect a lot of that configuration can be shared across the development team.

I would suggest raising the following questions:

• Which of the settings are developer-specific?
• Is there any way to standardise local developer machines so that settings could be shared?

Is it possible to do this in the build script itself, without having to stipulate within the script every individual file that needs to be renamed?

Yes, have a look at MSBuild Move task. You should be able to use it to rename files.

...which would require maintenance to the script any time a new project is added to the solution?

This is inevitable - your build scripts must evolve together with your solution. Accept this as a fact and include in your estimates time to make changes to your build scripts.

Furthermore, is there a better development solution that I can suggest that will make this entire process unnecessary?

I am not aware of all the requirements, so it is hard to recommend something very specific. I can say suggest this:

• Create a shared build script for your solution
• Automate manual tasks as much as possible (within reason)
• If you are struggling to automate something - it could be an indicator of an area that needs to be rethought/redesigned
• Make sure your team mates understand how the build works and are able to make changes to it themselves - don't "own" the build and become a bottleneck

Bear in mind that going from no build script to full automation is not an overnight process. Be patient and first focus on automating areas that are causing the most pain.

If I have misinterpreted any of your questions, please let me know and I will update the answer.