CODEWITHSUNDEEP
linuxscriptinghexdump
davka
davka

Reputation: 14672

Convert binary data to hexadecimal in a shell script

I want to convert binary data to hexadecimal, just that, no fancy formatting and all. hexdump seems too clever, and it "overformats" for me. I want to take x bytes from the /dev/random and pass them on as hexadecimal.

Preferably I'd like to use only standard Linux tools, so that I don't need to install it on every machine (there are many).

Upvotes: 75

Views: 135775

Answers (9)

Zibri
Zibri

Reputation: 9827

Watch out!

hexdump and xxd give the results in a different endianness!

$ echo -n $'\x12\x34' | xxd -p
1234
$ echo -n $'\x12\x34' | hexdump -e '"%02x"'
3412
$ echo -n $'\x01\x02' | hexdump -e '/1 "%02x"'
0102

Simply explained. Big-endian vs. little-endian :D

Upvotes: 46

revl
revl

Reputation: 161

Sometimes perl5 works better for portability if you target more than one platform. It comes with every Linux distribution and Unix OS. You can often find it in container images where other tools like xxd or hexdump are not available. Here's how to do the same thing in Perl:

$ head -c8 /dev/urandom | perl -0777 -ne 'print unpack "H*"'
5c9ed169dabf33ab

$ echo -n $'\x01\x23\xff' | perl -0777 -ne 'print unpack "H*"'
0123ff

$ echo abc | perl -0777 -ne 'print unpack "H*"'
6162630a

Note that this uses slurp more, which causes Perl to read the entire input into memory, which may be suboptimal when the input is large.

Upvotes: 3

marcinj
marcinj

Reputation: 49976

All the solutions seem to be hard to remember or too complex. I find using printf the shortest one:

$ printf '%x\n' 256
100

But as noted in comments, this is not what author wants, so to be fair, below is the full answer.

... to use above to output actual binary data stream:

printf '%x\n' $(cat /dev/urandom | head -c 5 | od -An -vtu1)

What it does:

  • printf '%x\n' .... - prints a sequence of integers , i.e. printf '%x,' 1 2 3, will print 1,2,3,
  • $(...) - this is a way to get output of some shell command and process it
  • cat /dev/urandom - it outputs random binary data
  • head -c 5 - limits binary data to 5 bytes
  • od -An -vtu1 - octal dump command, converts binary to decimal

As a testcase ('a' is 61 hex, 'p' is 70 hex, ...):

$ printf '%x\n' $(echo "apple" | head -c 5 | od -An -vtu1)
61
70
70
6c
65

Or to test individual binary bytes, on input let’s give 61 decimal ('=' char) to produce binary data ('\\x%x' format does it). The above command will correctly output 3d (decimal 61):

$printf '%x\n' $(echo -ne "$(printf '\\x%x' 61)" | head -c 5 | od -An -vtu1)
3d

Upvotes: 11

user2350426
user2350426

Reputation:

These three commands will print the same (0102030405060708090a0b0c):

n=12
echo "$a" | xxd -l "$n" -p
echo "$a" | od  -N "$n" -An -tx1 | tr -d " \n" ; echo
echo "$a" | hexdump -n "$n" -e '/1 "%02x"'; echo

Given that n=12 and $a is the byte values from 1 to 26:

a="$(printf '%b' "$(printf '\\0%o' {1..26})")"

That could be used to get $n random byte values in each program:

xxd -l "$n" -p                   /dev/urandom
od  -vN "$n" -An -tx1            /dev/urandom | tr -d " \n" ; echo
hexdump -vn "$n" -e '/1 "%02x"'  /dev/urandom ; echo

Upvotes: 1

Kevin Cox
Kevin Cox

Reputation: 3212

If you need a large stream (no newlines) you can use tr and xxd (part of Vim) for byte-by-byte conversion.

head -c1024 /dev/urandom | xxd -p | tr -d $'\n'

Or you can use hexdump (POSIX) for word-by-word conversion.

head -c1024 /dev/urandom | hexdump '-e"%x"'

Note that the difference is endianness.

Upvotes: 8

Håkon A. Hjortland
Håkon A. Hjortland

Reputation: 571

With od (GNU systems):

$ echo abc | od -A n -v -t x1 | tr -d ' \n'
6162630a

With hexdump (BSD systems):

$ echo abc | hexdump -ve '/1 "%02x"'
6162630a

From Hex dump, od and hexdump:

"Depending on your system type, either or both of these two utilities will be available--BSD systems deprecate od for hexdump, GNU systems the reverse."

Upvotes: 35

user405725
user405725

Reputation:

dd + hexdump will also work:

dd bs=1 count=1 if=/dev/urandom 2>/dev/null  | hexdump -e '"%x"'

Upvotes: 3

Blagovest Buyukliev
Blagovest Buyukliev

Reputation: 43498

Perhaps you could write your own small tool in C, and compile it on-the-fly:

int main (void) {
  unsigned char data[1024];
  size_t numread, i;

  while ((numread = read(0, data, 1024)) > 0) {
    for (i = 0; i < numread; i++) {
      printf("%02x ", data[i]);
    }
  }

  return 0;
}

And then feed it from the standard input:

cat /bin/ls | ./a.out

You can even embed this small C program in a shell script using the heredoc syntax.

Upvotes: 12

unutbu
unutbu

Reputation: 879073

Perhaps use xxd:

% xxd -l 16 -p /dev/random
193f6c54814f0576bc27d51ab39081dc

Upvotes: 99

Related Questions