CODEWITHSUNDEEP
crystal-lang
Freezerburn
Freezerburn

Reputation: 1013

Piping output of shell command to grep causes "grep: write error"

I wrote a tool in Crystal that takes some command line parameters, and turns those into, basically, "find stuff | xargs grep" where xargs is instructed to use multiple processes. This is run via Process.run, and output and error are redirected into a custom IO object which filters what comes out of grep a bit, writing everything that isn't filtered into STDOUT.

When I run this normally, it mostly seems to run fine. There do seem to be some instances of output getting cut off before the search completes though, so I'm not sure I can fully trust the results. When I pipe the output from this command into grep, however, it always cuts off the search early and says "grep: write error". I have no idea why this is happening, and would love some help. Eventually I'll likely rewrite this to do everything in pure Crystal, but for now this is a quick solution to search the codebase I'm working on.

Here is the code that is getting run:

class FindFilterIO
include IO

@@generic_filter = [".diff", ".iml", "/target/"]
@@web_filter = [".css", ".js", ".jsp", ".ftl"]

def initialize(@web_search : Bool = false)
end

def read(slice : Bytes)
  raise "FindFilterIO does not support reading!"
end

def write(slice : Bytes)
  str = String.new slice
  if @@generic_filter.any? { |e| str.includes? e }
    return
  end

  if @web_search
    if !@@web_filter.any? { |e| str.includes? e }
      return
    end
  end

  STDOUT.write(slice)
end
end

  cmd = "find . -not \\( -path ./.svn  -prune \\) " \
        "-not \\( -path ./.idea -prune \\) " \
        "-type f -print0 " \
        "| xargs -0 -P 1 -n 100 grep -E -n --color=always "
  cmd += if @html_id
           "'id=['\"'\"'\"]#{@search_text}['\"'\"'\"]|\##{@search_text}'"
         elsif @html_class
           "'class=['\"'\"'\"]#{@search_text}['\"'\"'\"]|\\.#{@search_text}'"
         else
           "'#{@search_text}'"
         end
  io = FindFilterIO.new web_search: (@html_id || @html_class)
  Process.run(cmd, output: io, error: io, shell: true, chdir: File.join(@env.home_dir, @env.branch, "repodir"))

Upvotes: 1

Views: 1578

Answers (1)

Freezerburn
Freezerburn

Reputation: 1013

This seems to have been fixed now that the issue at https://github.com/crystal-lang/crystal/issues/2065 has been closed. Will need to do some more testing to make sure it's totally fixed, but using my older code seems to be working fine now.

Upvotes: 0

Related Questions