Perl - Is this a bug with looks_like_number, or am I being a stupid person?

Question

This code:

#!/usr/bin/perl -w

use strict;
use Scalar::Util qw(looks_like_number);

sub what_the_fudge {
    my $string = "foo 123 bar";
    if ($string =~ /foo (.+) bar/) {
        if (looks_like_number($1)) {
            print "$1 looks like a number\n";
        } else {
            print "$1 doesnt look like a number\n";
        }
    }
}

&what_the_fudge;
&what_the_fudge;
&what_the_fudge;

Displays this:

123 doesnt look like a number
123 looks like a number
123 looks like a number

Why does it fail to recognize it as a number the first time? =( This baffles me.

Some information about my environment:

OS: OSX 10.6.8

perl -e 'use Scalar::Util; print "$Scalar::Util::VERSION\n"' --> 1.19

perl -v --> This is perl, v5.10.0 built for darwin-thread-multi-2level (with 2 registered patches, see perl -V for more detail)

Answer 1

It's a bug that was fixed in Scalar::Util 1.20. ("Handle overloaded and tied values" in the Changes file.)

The XS version of looks_like_number failed to handle magical arguments properly. Magic is what allows code to be called when certain operations are performed on a variable (such as fetching its value).

Solutions:

• Upgrade to Scalar::Util 1.20 or higher.
• Delete the compiled component of your Scalar::Util to force the use of the Pure Perl implementation which doesn't suffer from this problem.
• Use looks_like_number("$1") which creates a non-magical copy of $1 with the right value.

---

1.19:

int
looks_like_number(sv)
        SV *sv
PROTOTYPE: $
CODE:
#if (PERL_VERSION < 8) || (PERL_VERSION == 8 && PERL_SUBVERSION <5)
  if (SvPOK(sv) || SvPOKp(sv)) {
    RETVAL = looks_like_number(sv);
  }
  else {
    RETVAL = SvFLAGS(sv) & (SVf_NOK|SVp_NOK|SVf_IOK|SVp_IOK);
  }
#else
  RETVAL = looks_like_number(sv);
#endif
OUTPUT:
 RETVAL

1.21:

int
looks_like_number(sv)
    SV *sv
PROTOTYPE: $
CODE:
  SV *tempsv;
  if (SvAMAGIC(sv) && (tempsv = AMG_CALLun(sv, numer))) {
    sv = tempsv;
  }
  else if (SvMAGICAL(sv)) {
      SvGETMAGIC(sv);
  }
#if (PERL_VERSION < 8) || (PERL_VERSION == 8 && PERL_SUBVERSION <5)
  if (SvPOK(sv) || SvPOKp(sv)) {
    RETVAL = looks_like_number(sv);
  }
  else {
    RETVAL = SvFLAGS(sv) & (SVf_NOK|SVp_NOK|SVf_IOK|SVp_IOK);
  }
#else
  RETVAL = looks_like_number(sv);
#endif
OUTPUT:
  RETVAL

---

Try this:

what_the_fudge("foo 123 bar");
what_the_fudge("foo baz bar");
what_the_fudge("foo 123 bar");

sub what_the_fudge {
    my $string = shift;

I haven't actually tried it, but you should get

Version of Scalar::Util: 1.19
doesnt look like a number
looks like a number
doesnt look like a number

Answer 2

I made a few modifications in your program:

#! /usr/bin/env perl
#
use strict;
use warnings;
use Scalar::Util qw(looks_like_number);

print "Scalar version: $Scalar::Util::VERSION\n";
what_the_fudge();
what_the_fudge();
what_the_fudge();

sub what_the_fudge {
    my $string = "foo 123 bar";
    if ($string =~ /foo (.+) bar/) {
        if ( looks_like_number $1 ) {
            print qq("$1" looks like a number\n);
        } else {
            print qq("$1" doesn't look like a number\n);
        }
    }
}

I have a Mac running 10.8.6, and I use Perlbrew to test 5.8.9, 5.10, 5.12, and 5.18. With the first two I get this:

Scalar version: 1.19
"123" doesnt look like a number
"123" looks like a number
"123" looks like a number

With the other two versions, I get this:

Scalar version: 1.22
"123" looks like a number
"123" looks like a number
"123" looks like a number

I then made another slight change in your program. Instead of simply using $1, I set my $test = $1, then ran looks_like_number( $test );

After doing that, I got this:

Scalar version: 1.19
"123" looks like a number
"123" looks like a number
"123" looks like a number

One of the things you must understand is that $1, $_, and their ilk aren't just global variables, but always in the main namespace. This means that if you call a subroutine using $1, it is possible that the subroutine could end up munging the value.

You should always assign your own variables (and preferably my variables) instead of relying on special Perl variables. This means not using $_ in while and for loops, and not manipulating @_ directly in subroutines. And, when you use regular expression capturing, you should assign your own variables to $1, $2, etc. as soon as you can.

By the way, the most interesting difference between version 1.22 and 1.19 of Scalar::Util is that version 1.22 will use the C compiled code if it exists. If not, it loads Scalar::Util::PP which is the Perl version. However, when I used Scalar::Util::PP in Perl 5.12 and 5.18, I still get the correct output.

Curiouser and Curiouser

I tried modifying Scalar::Util to see what is going on. However, I see:

require List::Util; # List::Util loads the XS

And the Perl version of the code is encased in a big Eval statement. It is interesting that the side effect is only caught in the first one -- although the subroutine is setting the string, and running the regex. Same code, but executes two different ways.

Adding quotes around $1 makes it behave the right way:

if ( looks_like_number "$1" ) {

Also simply printing it out makes it work:

print "The value is " . $1 . "\n";
if ( looks_like_number $1 ) {

Or assigning it to another variable, but not using that variable:

my $test = $1;
if ( looks_like_number $1 ) {   works now...

Let's use the debugger and see if we can trace the problem:

$ perl -d test.pl

Loading DB routines from perl5db.pl version 1.31
Editor support available.

Enter h or `h h' for help, or `man perldebug' for more help.

main::(test.pl:7):  print "Version of Scalar::Util: $Scalar::Util::VERSION\n";
  DB<1> c
Version of Scalar::Util: 1.19
"123" looks like a number.
"123" looks like a number.
"123" looks like a number.
Debugged program terminated.  Use q to quit or R to restart,
  use o inhibit_exit to avoid stopping after program termination,
  h q, h R or h o to get additional info.

It works? Let's try the same command without the -d parameter:

$ perl test.pl
Version of Scalar::Util: 1.19
"123" doesn't look like a number.
"123" looks like a number.
"123" looks like a number.

How can you trace down this problem if you can't get see it in the debugger?

Answer 3

According to ikegami,

You're probably using the Pure Perl version of Scalar::Util, which probably uses a regex, which clobbers $1. looks_like_number("$1") will fix it.

Thanks! This fixed it.