CODEWITHSUNDEEP
regexlinuxunixsedfind
David542
David542

Reputation: 110203

How to remove a block of text in multiple files

I need to delete two blocks of text from multiple files. The two blocks are:

<sales_end_date>None</sales_end_date>
 ... some text
<unavailable_for_vod_date>None</unavailable_for_vod_date>

How would I properly do the equivalent of:

find ./ -type f -name 'xml' -exec sed -i 
    '**remove <sales_end_date>None</sales_end_date>' {} \;

Upvotes: 2

Views: 1144

Answers (6)

Joel Wiklund
Joel Wiklund

Reputation: 1875

Remove a block of code from multiple files in C#

I needed to remove a block of code from multiple files. I had problems with Swedish characters in a core project, so I needed to install System.Text.CodePagesEncodingProvider nuget package and use System.Text.Encoding.GetEncoding(1252) instead of System.Text.Encoding.UTF8.

    public static void Main(string[] args)
    {
        try
        {
            var dir = @"C:\Test";
            //Get all html and htm files
            var files = DirSearch(dir);
            foreach (var file in files)
            {
                RmCode(file);
            }
        }
        catch (Exception e)
        {
            Console.WriteLine(e.Message);
            throw;
        }

    }

    private static void RmCode(string file)
    {
        string tempFile = Path.GetTempFileName();

        using (var sr = new StreamReader(file, Encoding.UTF8))
        using (var sw = new StreamWriter(new FileStream(tempFile, FileMode.Open, FileAccess.ReadWrite), Encoding.UTF8))
        {
            string line;

            var startOfBadCode = "<div>";
            var endOfBadCode = "</div>";
            var deleteLine = false;

            while ((line = sr.ReadLine()) != null)
            {
                if (line.Contains(startOfBadCode))
                {
                    deleteLine = true;
                }
                if (!deleteLine)
                {
                    sw.WriteLine(line);
                }

                if (line.Contains(endOfBadCode))
                {
                    deleteLine = false;
                }
            }
        }

        File.Delete(file);
        File.Move(tempFile, file);
    }

    private static List<String> DirSearch(string sDir)
    {
        List<String> files = new List<String>();
        try
        {
            foreach (string f in Directory.GetFiles(sDir))
            {
                files.Add(f);
            }
            foreach (string d in Directory.GetDirectories(sDir))
            {
                files.AddRange(DirSearch(d));
            }
        }
        catch (System.Exception excpt)
        {
            Console.WriteLine(excpt.Message);
        }

        return files.Where(s => s.EndsWith(".htm") || s.EndsWith(".html")).ToList();
    }

Upvotes: 0

Lenik
Lenik

Reputation: 14458

If you are luck, try this:

find -type f -name '*.xml' -exec sh -c \
    'grep -v "<sales_end_date>None</sales_end_date>" "{}" >/tmp/a; mv /tmp/a "{}"' \;

Notice: This command is only useful to remove entire lines, not some chars in a line.

You can make a little helper script process.sh to make your life easier:

#!/bin/bash
file="$1"
shift
tmp=`tempfile`
"$@" "$file" > $tmp
mv $tmp > "$file"

And then,

find -type f -name "*.xml" -exec process.sh {} grep -v "<sales_end_date>None</sales_end_date>" \;

Upvotes: 0

John Ledbetter
John Ledbetter

Reputation: 14183

Using XMLStarlet:

xml ed -d "*/sales_end_date[text()='None']" -d "*/unavailable_for_vod_date[text()='None']" your-input.xml

Example: Assuming your xml looks like this:

<here>
  <top_level>
    <something>1</something>
    <sales_end_date>None</sales_end_date>
    <unavailable_for_vod_date>None</unavailable_for_vod_date>
  </top_level>
</here>

Will output:

<here>
  <top_level>
    <something>1</something>
  </top_level>
</here>

Upvotes: 1

Ωmega
Ωmega

Reputation: 43673

If None is inside of just those tags you want to remove, then you can use:

find . -name *.xml | xargs sed -ri 's/<.*?>None<.*?>//g'

Upvotes: 0

William Pursell
William Pursell

Reputation: 212248

I think you are looking for:

find . -type f -name '*.xml' -exec sed -i \
    -e '\@^<sales_end_date>None</sales_end_date>@d' \
    -e '\@^<unavailable_for_vod_date>None</unavailable_for_vod_date>@d' {} \;

But I wouldn't call this doing it properly. The -i option to sed is arguably never proper to use at all, and parsing xml with sed is rightly considered an abomination. However, this should do the job.

Upvotes: 1

MartyE
MartyE

Reputation: 656

I can't test this right now because I don't have a bash command shell, but sed 's/before/after/g' is the foundation of what you're looking for. I've often done this with a bit of piping

#!/bin/bash
before_string1='<sales_end_date>None</sales_end_date>'
after_string1=''
before_string2='<unavailable_for_vod_date>None</unavailable_for_vod_date>'
for file in `find ./ -type f -name 'xml'`; do
    cat ${file} | sed "s/$before_string1/${after_string1}/g" > ${file}.tmp1
    cat ${file}.tmp1 | sed "s/$before_string2/${after_string2}/g" > ${file}.tmp2
    mv ${file.tmp2} ${file}
    rm -f ${file.tmp1}
done

You'll have to make sure none of your before or after strings use / or else you'll need to escape them, but I'm sure you have the bash scripting and sed skills for that.

Upvotes: 0

Related Questions