Build Azure Data Factory Project using TFS Build Server 2013

Question

I have Data factory project in my solution and now in the process of creating a build definition.

When the build server starts building the solution it gives an exception saying that the path C:\Windows\ServiceProfiles\NetworkService\AppData\Roaming\Microsoft\DataFactory Tools for Visual Studio\MsBuild\1.0\DataFactory.targets was not found. I have already installed the extensions on the build server but the above files go and sit in my AppData folder.

How can I build the Data factory project using TFS build definition?

PS> I am using TFS 2013 server and Visual studio ultimate 2013 is also installed on the build server.

EDIT

Even when I use msbuild from the command line the build fails with the below error

"C:\Builds\103\mysolutiion\BuildDefinition\src\DataFactory\AdfApp\AdfApp.dfproj" (default target) (1) ->
(JsonCompilerTarget target) -> 
  C:\Users\myusername\AppData\Roaming\Microsoft\DataFactory Tools for Visual Studio\MsBuild\1.0\DataFactory.targets(35,5): error MSB4018: The "ADFCompilerTask" task failed unexpectedly.\r [C:\Builds\103\mysolutiion\BuildDefinition\src\DataFactory\AdfApp\AdfApp.dfproj]
C:\Users\myusername\AppData\Roaming\Microsoft\DataFactory Tools for Visual Studio\MsBuild\1.0\DataFactory.targets(35,5): error MSB4018: System.AggregateException: One or more errors occurred. ---> System.ArgumentNullException: Value cannot be null.\r [C:\Builds\103\mysolutiion\BuildDefinition\src\DataFactory\AdfApp\AdfApp.dfproj]
C:\Users\myusername\AppData\Roaming\Microsoft\DataFactory Tools for Visual Studio\MsBuild\1.0\DataFactory.targets(35,5): error MSB4018: Parameter name: source\r [C:\Builds\103\mysolutiion\BuildDefinition\src\DataFactory\AdfApp\AdfApp.dfproj]
C:\Users\myusername\AppData\Roaming\Microsoft\DataFactory Tools for Visual Studio\MsBuild\1.0\DataFactory.targets(35,5): error MSB4018:    at System.Linq.Enumerable.FirstOrDefault[TSource](IEnumerable`1 source, Func`2 predicate)\r [C:\Builds\103\mysolutiion\BuildDefinition\src\DataFactory\AdfApp\AdfApp.dfproj]
C:\Users\myusername\AppData\Roaming\Microsoft\DataFactory Tools for Visual Studio\MsBuild\1.0\DataFactory.targets(35,5): error MSB4018:    at Microsoft.VSAuthoring.SDK.Common.VSAuthoring.VersionCenter.GetDefault()\r [C:\Builds\103\mysolutiion\BuildDefinition\src\DataFactory\AdfApp\AdfApp.dfproj]
C:\Users\myusername\AppData\Roaming\Microsoft\DataFactory Tools for Visual Studio\MsBuild\1.0\DataFactory.targets(35,5): error MSB4018:    at Microsoft.VSAuthoring.SDK.BuildSystem.JsonCompilerTask.get_ProjectVersion()\r [C:\Builds\103\mysolutiion\BuildDefinition\src\DataFactory\AdfApp\AdfApp.dfproj]
C:\Users\myusername\AppData\Roaming\Microsoft\DataFactory Tools for Visual Studio\MsBuild\1.0\DataFactory.targets(35,5): error MSB4018:    at Microsoft.VSAuthoring.SDK.BuildSystem.JsonCompilerTask.<ValidateConfigFiles>d__1b.MoveNext()\r [C:\Builds\103\mysolutiion\BuildDefinition\src\DataFactory\AdfApp\AdfApp.dfproj]
C:\Users\myusername\AppData\Roaming\Microsoft\DataFactory Tools for Visual Studio\MsBuild\1.0\DataFactory.targets(35,5): error MSB4018: --- End of stack trace from previous location where exception was thrown ---\r [C:\Builds\103\mysolutiion\BuildDefinition\src\DataFactory\AdfApp\AdfApp.dfproj]
C:\Users\myusername\AppData\Roaming\Microsoft\DataFactory Tools for Visual Studio\MsBuild\1.0\DataFactory.targets(35,5): error MSB4018:    at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)\r [C:\Builds\103\mysolutiion\BuildDefinition\src\DataFactory\AdfApp\AdfApp.dfproj]
C:\Users\myusername\AppData\Roaming\Microsoft\DataFactory Tools for Visual Studio\MsBuild\1.0\DataFactory.targets(35,5): error MSB4018:    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r [C:\Builds\103\mysolutiion\BuildDefinition\src\DataFactory\AdfApp\AdfApp.dfproj]
C:\Users\myusername\AppData\Roaming\Microsoft\DataFactory Tools for Visual Studio\MsBuild\1.0\DataFactory.targets(35,5): error MSB4018:    at Microsoft.VSAuthoring.SDK.BuildSystem.JsonCompilerTask.<ValidateJsons>d__7.MoveNext()\r [C:\Builds\103\mysolutiion\BuildDefinition\src\DataFactory\AdfApp\AdfApp.dfproj]
C:\Users\myusername\AppData\Roaming\Microsoft\DataFactory Tools for Visual Studio\MsBuild\1.0\DataFactory.targets(35,5): error MSB4018: --- End of stack trace from previous location where exception was thrown ---\r [C:\Builds\103\mysolutiion\BuildDefinition\src\DataFactory\AdfApp\AdfApp.dfproj]
C:\Users\myusername\AppData\Roaming\Microsoft\DataFactory Tools for Visual Studio\MsBuild\1.0\DataFactory.targets(35,5): error MSB4018:    at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)\r [C:\Builds\103\mysolutiion\BuildDefinition\src\DataFactory\AdfApp\AdfApp.dfproj]
C:\Users\myusername\AppData\Roaming\Microsoft\DataFactory Tools for Visual Studio\MsBuild\1.0\DataFactory.targets(35,5): error MSB4018:    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r [C:\Builds\103\mysolutiion\BuildDefinition\src\DataFactory\AdfApp\AdfApp.dfproj]
C:\Users\myusername\AppData\Roaming\Microsoft\DataFactory Tools for Visual Studio\MsBuild\1.0\DataFactory.targets(35,5): error MSB4018:    at Microsoft.WindowsAzure.DataFactoryStudio.DataFactoryProject.ProjectSystem.ADFCompilerTask.<ExecuteAsync>d__2.MoveNext()\r [C:\Builds\103\mysolutiion\BuildDefinition\src\DataFactory\AdfApp\AdfApp.dfproj]
C:\Users\myusername\AppData\Roaming\Microsoft\DataFactory Tools for Visual Studio\MsBuild\1.0\DataFactory.targets(35,5): error MSB4018:    --- End of inner exception stack trace ---\r [C:\Builds\103\mysolutiion\BuildDefinition\src\DataFactory\AdfApp\AdfApp.dfproj]
C:\Users\myusername\AppData\Roaming\Microsoft\DataFactory Tools for Visual Studio\MsBuild\1.0\DataFactory.targets(35,5): error MSB4018:    at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)\r [C:\Builds\103\mysolutiion\BuildDefinition\src\DataFactory\AdfApp\AdfApp.dfproj]
C:\Users\myusername\AppData\Roaming\Microsoft\DataFactory Tools for Visual Studio\MsBuild\1.0\DataFactory.targets(35,5): error MSB4018:    at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)\r [C:\Builds\103\mysolutiion\BuildDefinition\src\DataFactory\AdfApp\AdfApp.dfproj]
C:\Users\myusername\AppData\Roaming\Microsoft\DataFactory Tools for Visual Studio\MsBuild\1.0\DataFactory.targets(35,5): error MSB4018:    at Microsoft.VSAuthoring.SDK.BuildSystem.JsonCompilerTask.Execute()\r [C:\Builds\103\mysolutiion\BuildDefinition\src\DataFactory\AdfApp\AdfApp.dfproj]
C:\Users\myusername\AppData\Roaming\Microsoft\DataFactory Tools for Visual Studio\MsBuild\1.0\DataFactory.targets(35,5): error MSB4018:    at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute()\r [C:\Builds\103\mysolutiion\BuildDefinition\src\DataFactory\AdfApp\AdfApp.dfproj]
C:\Users\myusername\AppData\Roaming\Microsoft\DataFactory Tools for Visual Studio\MsBuild\1.0\DataFactory.targets(35,5): error MSB4018:    at Microsoft.Build.BackEnd.TaskBuilder.<ExecuteInstantiatedTask>d__20.MoveNext()\r [C:\Builds\103\mysolutiion\BuildDefinition\src\DataFactory\AdfApp\AdfApp.dfproj]
C:\Users\myusername\AppData\Roaming\Microsoft\DataFactory Tools for Visual Studio\MsBuild\1.0\DataFactory.targets(35,5): error MSB4018: ---> (Inner Exception #0) System.ArgumentNullException: Value cannot be null.\r [C:\Builds\103\mysolutiion\BuildDefinition\src\DataFactory\AdfApp\AdfApp.dfproj]
C:\Users\myusername\AppData\Roaming\Microsoft\DataFactory Tools for Visual Studio\MsBuild\1.0\DataFactory.targets(35,5): error MSB4018: Parameter name: source\r [C:\Builds\103\mysolutiion\BuildDefinition\src\DataFactory\AdfApp\AdfApp.dfproj]
C:\Users\myusername\AppData\Roaming\Microsoft\DataFactory Tools for Visual Studio\MsBuild\1.0\DataFactory.targets(35,5): error MSB4018:    at System.Linq.Enumerable.FirstOrDefault[TSource](IEnumerable`1 source, Func`2 predicate)\r [C:\Builds\103\mysolutiion\BuildDefinition\src\DataFactory\AdfApp\AdfApp.dfproj]
C:\Users\myusername\AppData\Roaming\Microsoft\DataFactory Tools for Visual Studio\MsBuild\1.0\DataFactory.targets(35,5): error MSB4018:    at Microsoft.VSAuthoring.SDK.Common.VSAuthoring.VersionCenter.GetDefault()\r [C:\Builds\103\mysolutiion\BuildDefinition\src\DataFactory\AdfApp\AdfApp.dfproj]
C:\Users\myusername\AppData\Roaming\Microsoft\DataFactory Tools for Visual Studio\MsBuild\1.0\DataFactory.targets(35,5): error MSB4018:    at Microsoft.VSAuthoring.SDK.BuildSystem.JsonCompilerTask.get_ProjectVersion()\r [C:\Builds\103\mysolutiion\BuildDefinition\src\DataFactory\AdfApp\AdfApp.dfproj]
C:\Users\myusername\AppData\Roaming\Microsoft\DataFactory Tools for Visual Studio\MsBuild\1.0\DataFactory.targets(35,5): error MSB4018:    at Microsoft.VSAuthoring.SDK.BuildSystem.JsonCompilerTask.<ValidateConfigFiles>d__1b.MoveNext()\r [C:\Builds\103\mysolutiion\BuildDefinition\src\DataFactory\AdfApp\AdfApp.dfproj]
C:\Users\myusername\AppData\Roaming\Microsoft\DataFactory Tools for Visual Studio\MsBuild\1.0\DataFactory.targets(35,5): error MSB4018: --- End of stack trace from previous location where exception was thrown ---\r [C:\Builds\103\mysolutiion\BuildDefinition\src\DataFactory\AdfApp\AdfApp.dfproj]
C:\Users\myusername\AppData\Roaming\Microsoft\DataFactory Tools for Visual Studio\MsBuild\1.0\DataFactory.targets(35,5): error MSB4018:    at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)\r [C:\Builds\103\mysolutiion\BuildDefinition\src\DataFactory\AdfApp\AdfApp.dfproj]
C:\Users\myusername\AppData\Roaming\Microsoft\DataFactory Tools for Visual Studio\MsBuild\1.0\DataFactory.targets(35,5): error MSB4018:    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r [C:\Builds\103\mysolutiion\BuildDefinition\src\DataFactory\AdfApp\AdfApp.dfproj]
C:\Users\myusername\AppData\Roaming\Microsoft\DataFactory Tools for Visual Studio\MsBuild\1.0\DataFactory.targets(35,5): error MSB4018:    at Microsoft.VSAuthoring.SDK.BuildSystem.JsonCompilerTask.<ValidateJsons>d__7.MoveNext()\r [C:\Builds\103\mysolutiion\BuildDefinition\src\DataFactory\AdfApp\AdfApp.dfproj]
C:\Users\myusername\AppData\Roaming\Microsoft\DataFactory Tools for Visual Studio\MsBuild\1.0\DataFactory.targets(35,5): error MSB4018: --- End of stack trace from previous location where exception was thrown ---\r [C:\Builds\103\mysolutiion\BuildDefinition\src\DataFactory\AdfApp\AdfApp.dfproj]
C:\Users\myusername\AppData\Roaming\Microsoft\DataFactory Tools for Visual Studio\MsBuild\1.0\DataFactory.targets(35,5): error MSB4018:    at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)\r [C:\Builds\103\mysolutiion\BuildDefinition\src\DataFactory\AdfApp\AdfApp.dfproj]
C:\Users\myusername\AppData\Roaming\Microsoft\DataFactory Tools for Visual Studio\MsBuild\1.0\DataFactory.targets(35,5): error MSB4018:    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r [C:\Builds\103\mysolutiion\BuildDefinition\src\DataFactory\AdfApp\AdfApp.dfproj]
C:\Users\myusername\AppData\Roaming\Microsoft\DataFactory Tools for Visual Studio\MsBuild\1.0\DataFactory.targets(35,5): error MSB4018:    at Microsoft.WindowsAzure.DataFactoryStudio.DataFactoryProject.ProjectSystem.ADFCompilerTask.<ExecuteAsync>d__2.MoveNext()<---\r [C:\Builds\103\mysolutiion\BuildDefinition\src\DataFactory\AdfApp\AdfApp.dfproj]
C:\Users\myusername\AppData\Roaming\Microsoft\DataFactory Tools for Visual Studio\MsBuild\1.0\DataFactory.targets(35,5): error MSB4018:  [C:\Builds\103\mysolutiion\BuildDefinition\src\DataFactory\AdfApp\AdfApp.dfproj]

Answer 1

I had the same problem, I didn't fix my particular issue, but perhaps it will fix your issue. I added the DataFactory.targets to the Dependencies folder in the solution and referenced it in the .dfproj file. Hope this helps!

Answer 2

Okay, I could reproduce your issue on my side. Just create an Empty Data Factory Projectand queue build:

The error message indicates the issue. Right click the project and Unload the project and edit the .dfproj, you'll find the such path in declaration:

In my environment, DataFactory.targets is under C:\Users\username\AppData\Roaming\Microsoft\DataFactory Tools for Visual Studio\MsBuild\1.0, NOT in C:\Windows\ServiceProfiles\NetworkService\AppData\Roaming\Microsoft\DataFactory Tools for Visual Studio\MsBuild\1.0. Try to change the path, the issue should go away.