Select-String not working on piped object using Out-String

Question

I am doing an API request which returns a bunch of data. In attempted to search through it with Select-String, it just spits out the entire value stored in the variable. This is an internet server which I am calling an api.

$return = Invoke-RestMethod -Method GET -Uri $uri -Headers @{"authorization" = $token} -ContentType "application/json" 
$file = $return.data
$file | Out-String -Stream | Select-String -Pattern "word"

this returns the entire value of $file. printing $file looks like same as the pipe output. Why is this not working?

$file.Gettype says it is a system.object, another answer said to use Out-String, but something is not working.

$file.Gettype
IsPublic IsSerial Name                                     BaseType                   
-------- -------- ----                                     --------                   
True     True     String                                   System.Object 

Answer 1

To complement iRon7's helpful answer with the precise logic of Out-String's -Stream switch, as of PowerShell 7.1:

Out-String, like the other Out-* cmdlets such as Out-File, uses PowerShell's rich output-formatting system to generate human-friendly representations of its input objects.

• Without -Stream, Out-String only ever produces a single, (typically) multiline string.

• With -Stream, line-by-line output behavior typically occurs - except for input objects that happen to be multiline strings, which are output as-is.

  • Because this exception is both obscure and unhelpful, GitHub proposal #14638 suggests removing it.

For so-called in-band data types, -Stream works as follows, which truly results in line-by-line output:

• Input objects are formatted by PowerShell's rich formatting system, and the lines that make up the resulting representation are then output one by one.

Out-of-band data types are individually formatted outside of the formatting system, by simply calling their .NET .ToString() method.

In short: data types that represent a single value are out-of-band, and in addition to [string] out-of-band data types also comprise [char] and the various (standard) numeric types, such as [int], [long], [double], ...

[string] is the only out-of-band type that itself can result in a multiline representation, because calling .ToString() on a string is effective no-op that returns the string itself - whether it is single- or multiline.

Therefore:

• Any string - notably also a multiline string - is output as-is, as a whole, and splitting it into individual lines requires an explicit operation; e.g. (note that regex \r?\n matches both Windows-style CRLF and Unix-style LF-only newlines):

  "line 1`nline 2`nline 3" -split '\r?\n' # -> 'line 1', 'line 2', 'line 3'
  

• If your input objects are a mix of in-band objects and (invariably out-of-band) multiline strings, you can combine Out-String -Stream with -split; e.g.:

  ((Get-Date), "line 1`nline 2`nline 3" | Out-String -Stream) -split '\r?\n' 
  

Answer 2

On closer inspection, I suspect that your issue comes from an ambiguity in the Out-String documentation:

-Stream

Indicates that the cmdlet sends a separate string for each line of an input object. By default, the strings for each object are accumulated and sent as a single string.

Where the word line should be read as item.

To split you raw string into separate lines, you will need to split your string using the following command:

$Lines = $return.data -split [Environment]::NewLine

Note that this assumes that your data uses the same characters for a new line as the system you working on. If this is not the case, you might want to split the lines using an regular expression, e.g.:

$Lines = $return.data -split "`r*`n"

So what does the-Stream parameter do?

It sends a separate string for each item of an input object.
Where in this definition, it is also a known common best PowerShell practice to use a singular name for possible plural input objectS.

Meaning if you use the above defined $Lines variable (or something like $Lines = Get-Content .\File.json), the input object "$Lines" is a collection of strings:

$Lines.GetType().Name
String[]

if you stream this to Out-String it will (by default) join all the items and return a single string:

($Lines | Out-String).GetType().Name
String

In comparison, if you use the -Stream parameter, it will pass each separated item from the $Lines collection directly to the next cmdlet:

($Lines | Out-String -Stream).GetType().Name
Object[]

I have created a document issue for this: #7133 "line" should be "item"

Note:

In general, it is a bad practice to peek and poke directly into a serialized string (including Json) using string methods and/or cmdlets (like Select-String). Instead you should use the related parser (e.g. ConvertFrom-Json) for searching and replacing which will result in an easier syntax and usually takes care of known issues and pitfalls.

Answer 3

Select-String outputs Microsoft.PowerShell.Commands.MatchInfo objects. It seems to me that the output is somehow fancified via the PS engine or something to highlight your match, but ultimately it does print the entire matched string.

You should check out the members of the object Select-String provides, like this:

$file | Out-String -Stream | Select-String -Pattern "word" | Get-Member

   TypeName: Microsoft.PowerShell.Commands.MatchInfo

Name               MemberType Definition
----               ---------- ----------
...
Matches            Property   System.Text.RegularExpressions.Match[] Matches {get;set;}
...

What you're interested in is the Matches property. It contains a bunch of information about the match. To extract exactly what you want, look at the Value property of Matches:

($file | Out-String -Stream | Select-String -Pattern "word").Matches.Value

word

Another way:

$file | Out-String -Stream | Select-String -Pattern "word" | ForEach-Object {$_.Matches} | Select-Object -Property Value

Value
-----
word

Or

$file | Out-String -Stream | Select-String -Pattern "word" | ForEach-Object {$_.Matches} | Select-Object -ExpandProperty Value

word