CODEWITHSUNDEEP
unicodebyte-order-mark
Macy Abbey
Macy Abbey

Reputation: 3887

Remove multiple BOMs from a file

I am using a Javascript file that is a concatenation of other JavaScript files.

Unfortunately, the person who concatenated these JavaScript files together did not use the proper encoding when reading the file, and allowed a BOM for every single JavaScript file to get written to the concatenated JavaScript file.

Does anyone know a simple way to search through the concatenated file and remove any/all BOM markers?

Using PHP or a bash script for Mac OSX would be great.

Upvotes: 12

Views: 7160

Answers (5)

Ludvig W
Ludvig W

Reputation: 824

I have written a bash script see here that works for Mac, I haven't tested on other systems but I suspect it should work there as well. The script also support files or file paths that contains spaces.

Examples

Remove BOM from all files in current directory:

rmbom .

Print all files with a BOM in the current directory

rmbom . -a

Only remove BOM from all files in current directory with extension txt or cs:

rmbom . -e txt -e cs

Print help

rmbom -h

Upvotes: 0

Macy Abbey
Macy Abbey

Reputation: 3887

I also figured out this solution which works entirely in PHP:

$packed = pack("CCC",0xef,0xbb,0xbf);
$contents = preg_replace('/'.$packed.'/','',$contents);

Upvotes: 0

user1896665
user1896665

Reputation: 11

fetch BOM files

grep -rIlo $’^\xEF\xBB\xBF’ ./

remove BOM files

grep -rIlo $’^\xEF\xBB\xBF’ . | xargs sed –in-place -e ‘s/\xef\xbb\xbf//’

exclude .svn dir

grep -rIlo –exclude-dir=”.svn” $’^\xEF\xBB\xBF’ . | xargs sed –in-place -e ‘s/\xef\xbb\xbf//’

Upvotes: 1

Pål Brattberg
Pål Brattberg

Reputation: 4698

I normally do it using vim:

vim -c "set nobomb" -c wq! myfile

Upvotes: 18

mario
mario

Reputation: 145482

See also: Using awk to remove the Byte-order mark

To remove multiple BOMs from anywhere within a text file you can try something similar. Just leave out the ^ anchor:

perl -e 's/\xef\xbb\xbf//;' -pi~ file.js

(This edits the file in-place. But creates a backup file.js~.)

Upvotes: 17

Related Questions