How to cause Install-Module to also install any required modules?

Question

I am working on a PS module, which depends on SqlServer.

Here is the module manifest file:

@{
    ModuleVersion = "1.0.19103.11"
    GUID = "59bc8fa6-b480-4226-9bcc-ec243102f3cc"
    Author = "..."
    CompanyName = "..."
    Copyright = "..."
    Description = "..."
    RequiredModules = ""
    ScriptsToProcess = "vsts\config.ps1"
    NestedModules = @( ... )
    PrivateData = @{
        PSData = @{
            ExternalModuleDependencies = @(
                "SqlServer"
            )
        }
    }
}

Here are the commands I use to publish the module:

if (Get-PSRepository $PSRepositoryName -ErrorAction SilentlyContinue)
{
    Unregister-PSRepository $PSRepositoryName
}
Register-PSRepository $PSRepositoryName `
    -SourceLocation $NuGetRepoUrl `
    -PublishLocation $NuGetRepoUrl `
    -InstallationPolicy Trusted `
    -PackageManagementProvider nuget

Publish-Module -Name $ModuleManifestFile -NuGetApiKey $NuGetApiKey -Repository $PSRepositoryName -Force

The problem is that when I install this module I get the following warning:

WARNING: The externally managed, dependent module 'SqlServer' is not installed on this computer. To use the current module 'xyz.PS.Core', ensure that its dependent
module 'SqlServer' is installed.

How can I cause the SqlServer to be installed automatically when this package is installed without any warning.

I tried setting the RequiredModules property, but this fails to publish the module in the first place, if the build machine does not have the module SqlServer installed. I can install it there, but is there a better way without forcing us to install packages on the build agents?

EDIT 1

So, I installed the SqlServer module on the build agent and added the RequiredModules property. The same warning is displayed. But now I cannot even import the module:

Import-Module : The required module 'SqlServer' is not loaded. Load the module or remove the module from 'RequiredModules' in the file

EDIT 2

I am still unable to make it work. Let me show all the combinations. In all the cases, the test machine does not have the SqlServer module installed on it.

Attempt 1

RequiredModules = @(
    @{
        ModuleName = "SqlServer"
        ModuleVersion = "21.1.18068"
    }
)
PrivateData = @{
    PSData = @{
        ExternalModuleDependencies = "SqlServer"
    }
}

Yields

C:\> Publish-Module -Name $ModuleManifestFile -NuGetApiKey $NuGetApiKey -Repository $PSRepositoryName -Force
PowerShellGet cannot resolve the module dependency 'SqlServer' of the module 'xyz.PS.Core' on the repository 'xyz-QA'. Verify that the dependent module 'SqlServer' is available in the repository 'xyz-QA'. If this dependent module 'SqlServer' is managed externally, add it to the ExternalModuleDependencies entry in the PSData section of the module manifest.
At C:\Program Files\WindowsPowerShell\Modules\PowerShellGet\1.0.0.1\PSModule.psm1:1227 char:17
+                 Publish-PSArtifactUtility -PSModuleInfo $moduleInfo `
+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (:) [Publish-PSArtifactUtility], InvalidOperationException
    + FullyQualifiedErrorId : UnableToResolveModuleDependency,Publish-PSArtifactUtility
C:\>

Attempt 2

RequiredModules = @("SqlServer")
PrivateData = @{
    PSData = @{
        ExternalModuleDependencies = "SqlServer"
    }
}

Yields

C:\> Publish-Module -Name $ModuleManifestFile -NuGetApiKey $NuGetApiKey -Repository $PSRepositoryName -Force
PowerShellGet cannot resolve the module dependency 'SqlServer' of the module 'xyz.PS.Core' on the repository 'xyz-QA'. Verify that the dependent module 'SqlServer' is available in the repository 'xyz-QA'. If this dependent module 'SqlServer' is managed externally, add it to the ExternalModuleDependencies entry in the PSData section of the module manifest.
At C:\Program Files\WindowsPowerShell\Modules\PowerShellGet\1.0.0.1\PSModule.psm1:1227 char:17
+                 Publish-PSArtifactUtility -PSModuleInfo $moduleInfo `
+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (:) [Publish-PSArtifactUtility], InvalidOperationException
    + FullyQualifiedErrorId : UnableToResolveModuleDependency,Publish-PSArtifactUtility
C:\>

Attempt 3

PrivateData = @{
    PSData = @{
        ExternalModuleDependencies = "SqlServer"
    }
}

(no RequiredModules) Yields:

C:\> Publish-Module -Name $ModuleManifestFile -NuGetApiKey $NuGetApiKey -Repository $PSRepositoryName -Force
C:\> Get-Module SqlServer -ListAvailable
C:\> Install-Module xyz.PS.Core -Scope CurrentUser -Force -AllowClobber
WARNING: The externally managed, dependent module 'SqlServer' is not installed on this computer. To use the current module 'xyz.PS.Core', ensure that its dependent module 'SqlServer' is installed.
C:\> Get-Module SqlServer -ListAvailable
C:\>

As you can see, publish is successful, but installing the module outputs a warning. And no SqlServer installed.

Attempt 4

RequiredModules = @(
    @{
        ModuleName = "SqlServer"
        ModuleVersion = "21.1.18068"
    }
)

(without the ExternalModuleDependencies) Yields:

C:\> Publish-Module -Name $ModuleManifestFile -NuGetApiKey $NuGetApiKey -Repository $PSRepositoryName -Force
The specified RequiredModules entry 'SqlServer' in the module manifest 'C:\Users\mkharitonov\AppData\Local\Temp\2144858157\xyz.PS.Core\xyz.PS.Core.psd1' is invalid. Try again after updating this entry with valid values.At C:\Program Files\WindowsPowerShell\Modules\PowerShellGet\1.0.0.1\PSModule.psm1:1144 char:27+ ... oduleInfo = Microsoft.PowerShell.Core\Test-ModuleManifest -Path $mani ...
+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (C:\Users\mkhari...ce.PS.Core.psd1:String) [Test-ModuleManifest], DirectoryNotFoundException
    + FullyQualifiedErrorId : Modules_InvalidRequiredModulesinModuleManifest,Microsoft.PowerShell.Commands.TestModuleManifestCommand
C:\>

Attempt 5

RequiredModules = @("SqlServer")

(without the ExternalModuleDependencies) Yields:

C:\> Publish-Module -Name $ModuleManifestFile -NuGetApiKey $NuGetApiKey -Repository $PSRepositoryName -Force
The specified RequiredModules entry 'SqlServer' in the module manifest 'C:\Users\mkharitonov\AppData\Local\Temp\1208648280\xyz.PS.Core\xyz.PS.Core.psd1' is invalid. Try again after updating this entry with valid values.At C:\Program Files\WindowsPowerShell\Modules\PowerShellGet\1.0.0.1\PSModule.psm1:1144 char:27+ ... oduleInfo = Microsoft.PowerShell.Core\Test-ModuleManifest -Path $mani ...
+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (C:\Users\mkhari...ce.PS.Core.psd1:String) [Test-ModuleManifest], DirectoryNotFoundException
    + FullyQualifiedErrorId : Modules_InvalidRequiredModulesinModuleManifest,Microsoft.PowerShell.Commands.TestModuleManifestCommand
C:\>

So, nothing I do works. So far it seems Powershell cannot install a required module.

What am I doing wrong?

EDIT 3

The RequiredModules property does not work even if I install SqlServer on the build machine. Observe:

RequiredModules = @(
    "SqlServer"
)
PrivateData = @{
    PSData = @{
        ExternalModuleDependencies = "SqlServer"
    }
}

And the result is:

C:\> Get-Module SqlServer -ListAvailable


    Directory: C:\Users\mkharitonov\Documents\WindowsPowerShell\Modules


ModuleType Version    Name                                ExportedCommands
---------- -------    ----                                ----------------
Script     21.1.18102 SqlServer                           {Add-RoleMember, Add-SqlAvailabilityDatabase, Add-SqlAvailabilityGroupListenerStaticIp, Ad...


C:\> Publish-Module -Name $ModuleManifestFile -NuGetApiKey $NuGetApiKey -Repository $PSRepositoryName -Force
PowerShellGet cannot resolve the module dependency 'SqlServer' of the module 'xyz.PS.Core' on the repository 'xyz-QA'. Verify that the dependent module 'SqlServer' is available in the repository 'xyz-QA'. If this dependent module 'SqlServer' is managed externally, add it to the ExternalModuleDependencies entry in the PSData section of the module manifest.
At C:\Program Files\WindowsPowerShell\Modules\PowerShellGet\1.0.0.1\PSModule.psm1:1227 char:17
+                 Publish-PSArtifactUtility -PSModuleInfo $moduleInfo `
+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (:) [Publish-PSArtifactUtility], InvalidOperationException
    + FullyQualifiedErrorId : UnableToResolveModuleDependency,Publish-PSArtifactUtility
C:\>

Powershell seems to look for the SqlServer module in my repository rather than in PSGallery. I do not understand way. Here are the PS repositories registered on the machine:

C:\> Get-PSRepository

Name                      InstallationPolicy   SourceLocation
----                      ------------------   --------------
PSGallery                 Trusted              https://www.powershellgallery.com/api/v2
xyz-QA                    Trusted             http://devstatic.xyz.com/nugetserver/nuget


C:\>

So, I truly do not understand what am I supposed to do?

Answer 1

Update:

• Specifying just RequiredModules for intra-repository dependencies (not also PrivateData.PSData.ExternalModuleDependencies) may now be sufficient as of PackageManagement v1.4.8.1 / PowerShellGet v2.2.5 - see the first comment below.

• A rewritten successor to PowerShellGet that no longer depends on the PackageManagement module is now available, Microsoft.PowerShell.PSResourceGet (ships with PowerShell (Core) 7, installable in Windows PowerShell), which presumably behaves the same way.

---

Note: The following was gleaned form this discussion in the PackageManagement (OneGet) GitHub repo, notably this comment from a team member:

Install-Module works for dependences within the same repo.
For dependency modules are hosted on the repo different from that module to be installed, I recommend you to publish them to your house repo.
If you do not wish to publish them into your house repo, you may save modules to your common file share, Register-PSRepository -Name local -SourceLocation \server\commonshare, then install these modules from file share first."

As of PackageManagement module version 1.3, I conclude from your symptoms that the above still applies, even though the comment is from mid 2017 (I couldn't find any official documentation).

To summarize:

• Automatic installation of cross-repository dependencies isn't supported; the workaround is to publish the external dependencies to your in-house repo as well.

• On calling Publish-Module, seemingly, modules listed in the RequiredModules entry must be present in the same target repo.

  • That is, the presence of such modules on your machine alone is not enough. (I'm unclear on whether they also need to be present - they shouldn't have to be.)

• To support automatic installation of intra-repository dependencies, the PrivateData.PSData.ExternalModuleDependencies entry must list those dependencies too - in addition to RequiredModules - by name only - see below.

Specifying intra-repository module dependencies (unverified) for automatic dependency installation:

# Specify what other modules this module requires to work, using a FQMN,
# (a Fully Qualified Module Name), via a hashtable.
# The 'ModuleVersion' entry version numbers specifies a *minimum* required version 
# number; 'RequiredVersion' specifies an *exact* version, and  
# 'MaximumVersion' specifies a maximum version.
# IMPORTANT:
#   If you publish a module whose manifest has a 'RequiredModules'
#   entry to a repository with Publish-Module, all referenced modules
#   seemingly *must exist in that repository.*
RequiredModules = @( @{ModuleName = 'SqlServer'; ModuleVersion = '21.1.18068' } )

PrivateData = @{
  PSData = @{

    # ... 

    # [MAY NO LONGER BE REQUIRED - SEE UPDATE AT THE TOP.]
    # !! This field is *ancillary* to the more detailed 'RequiredModules' field and
    # !! must reference the *same modules*, but by *names only*,
    # !! in order to automatically install other modules
    # !! *from the same repository* that this module depends on.
    # !! To be safe, specify even a *single* name as an *array*
    # !! (While this is not a general requirement in manifests, 
    # !!  it may be necessary here due to a bug.)
    ExternalModuleDependencies = @('SqlServer')
}

As an aside: A bug in New-ModuleManifest currently prevents direct creation of the PrivateData.PSData.ExternalModuleDependencies entry:

# !! BROKEN as of Windows PowerShell v5.1 / PowerShell Core 6.2.0;
# The inner hashtable ends up represented as 'System.Collections.Hashtable'
New-ModuleManifest someManifest.psd1 -PrivateData @{ 
  PSData = @{
    ExternalModuleDependencies = @('SqlServer')
  }
}

See GitHub issue #5922.

Answer 2

In addition to the comprehensive answer by @mklement0 there is a possibility to avoid persistent publishing the external dependencies to your in-house repo, and so avoid unwanted redundancy and maintenance efforts:

Therefore you can use a repository manager like Sonatype's Nexus which provides the repository grouping feature:

They [repository groups] allow you to combine multiple repositories and other repository groups in a single repository. This in turn means that your users can rely on a single URL for their configuration needs, while the administrators can add more repositories and therefore components to the repository group.

So you have to create a nuget-group which combines the nuget formatted repositories (e.g. nuget-hosted (your PowerShell modules) and powershellgallery.com-proxy (external dependencies)) into a single repository.

With this you need to publish the external dependencies (from powershellgallery.com) to your in-house repo (nuget-hosted) for publishing of your developed PowerShell module only (Publish-Module -Repository nuget-hosted -Name ...). Afterwards you can remove the external dependency from nuget-hosted.

For Install-Module of the developed module together with the dependencies on an arbitrary PowerShell machine you have to register and use the nuget-group repository only.