CODEWITHSUNDEEP
windowsfilewindows-7powershell-2.0
Amazon Dies In Darkness
Amazon Dies In Darkness

Reputation: 5823

Powershell command not working: removing lines from file1 that are in file2

I am using the following code to remove lines from file1.txt that are in file2.txt.

powershell -Command "$(Get-Content file1.txt) | Where-Object {$_ -notIn $(Get-Content file2.txt)}"

But I'm getting an error regarding -notIn, looking for a value expression. But file2.txt does exist and is not null.

What is causing the error, and how to fix it?

Upvotes: 2

Views: 159

Answers (2)

mklement0
mklement0

Reputation: 438123

To complement LotPings' helpful answer:

  • For execution speed, do not execute Get-Content file2.txt in every loop iteration - cache its result beforehand.

  • For memory efficiency, do not collect all lines of file1.txt up front with $(...) before sending them through the pipeline.

Therefore (I'm omitting the powershell -Command "..." wrapper for brevity):

$f2 = Get-Content file2.txt; Get-Content file1.txt | Where-Object { $f2 -NotContains $_ }

Which $ are necessary, and why?

(...) and $(...) (the subexpression operator) and @(...) (the array subexpression operator) all collect the entire command's / commands' output as a whole, in an array ([System.Object[]]), except if the output comprises only 1 item, in which that item itself is returned.

(...), which can only enclose a single command or expression, is needed:

  • to clarify precedence in an expression,
  • to use a command (cmdlet / function / external utility call ) as part of an expression.
  • In a pipeline, you can use it to force collecting the enclosed command's entire output beforehand, but doing so negates the memory-throttling benefit of a pipeline.
    • That said, using something like (Get-Content file) | ... | Set-Content file enables updating a given file file "in-place" - but do note that this requires the entire contents of file to fit into memory.
  • Unless you have additional requirements as stated below, prefer (...) to $(...) and @(...).

$(...) is only needed: Tip of the hat to PetSerAl for his help.

  • to enclose statements such as if and foreach

  • to enclose multiple commands/expressions/statements

  • to embed the above inside "..." (a double-quoted string, whose contents are subject to interpolation (expansion)).
    Among the operators listed, $(...) is the only one that can (directly) be embedded inside a double-quoted string.

  • $(...) - unlike (...) - doesn't abort the entire command if a potentially terminating error occurs; e.g.:

    • 'hi' + $(1/0) and "hi$(1/0)" report the division-by-zero error, yet still print hi; by contrast, 'hi' + (1/0) does not - the division-by-zero error aborts the entire expression.
    • Note that an unconditionally terminating error even aborts any $(...)-containing expression; e.g., 'hi' + $(Throw 'err') and "hi$(Throw 'err')" both only print the error message, not also hi.

@(...) is only needed:

  • to ensure that a command's output is an array, even if only 1 item is returned; in other respects, it acts like $(), except that you cannot use it inside "..."; for a detailed discussion of @(), see this answer of mine.

  • In PSv3+, the unified handling of scalars and arrays typically makes @(...) unnecessary, however - see this answer of mine.

Upvotes: 2

user6811411
user6811411

Reputation:

For PowerShell v2 reverse the arguments and use -NotContains 

powershell -Command "$(Get-Content file1.txt) | Where-Object {$(Get-Content file2.txt) -NotContains $_ }

Reference

Upvotes: 1

Related Questions