PowerShell "Assembly with same name is already loaded" on different Module Versions

Question

I have a script that requires using two versions of the same module with conflicting assemblies:

#Requires -PSEdition Core
# main logic
Install-Module -Name MyModule -RequiredVersion 2 -Scope CurrentUser -Force
Install-Module -Name MyModule -RequiredVersion 3 -Scope CurrentUser -Force
& UseV2.ps1
& UseV3.ps1

~/UseV2.ps1:

#Requires -Modules @{ ModuleName = 'MyModule'; RequiredVersion = '2' }
...

~/UseV3.ps1:

#Requires -Modules @{ ModuleName = 'MyModule'; RequiredVersion = '3' }
...

The second script is never executed since I receive the following error: "Assembly with same name is already loaded".

How can I fix this and use the V3 module too?

I can see the .dll from V2 are still loaded after executing the script that uses V2 but can't remove them.

$assemblies = [AppDomain]::CurrentDomain.GetAssemblies()
$assemblyToUnload = $assemblies | Where-Object { $_.Location -like '*MyModule*`dll' }