Share lexical scope between successive eval statements in perl

Question

Can I make different snippets of evaled Perl code share the same lexical scope and yet get their return values?

Background

Perl's eval command evaluates a string as Perl code and upon success returns the value of the last statement in that code. However, lexical variables created in that code are dropped at the end of the code. This means when eval of code1 has ended and I have a second code chunk code2, which refers to the lexical variables set in code1, this will fail.

my $code1 = 'my $c = 4';
my $code2 = 'printf "%g\n", $c;';

printf 'evaluated "%s" to %s' . "\n", $code1, eval $code1;
printf 'evaluated "%s"' . "\n", $code2;

yields

evaluated "my $c = 4" to 4
evaluated "printf "%g\n", $c;"

but not a line containing just 4 as I would wish, because $code2 should use the variable $c if lexical scopes are re-used. (I generally agree with the default that lexical scopes are constrained to only one evaled code, so I expect that some conscious modification of code is required to make the above work.)

Approaches considered

I experimented with use PadWalker qw( peek_my ); to save the lexical scope at the end of each code snippet aiming to load it into the scope of the following snippet, but then I realised that this would make inaccessible the return value of the code snippet, which is needed by the calling code.

As another alternative appears to pattern-match (probably using a dedicated parser) all my-declarations in the perl code snippets and essentially translate them on the fly, but this would amount to a considerably bigger task.

Template example for discussion (see comments)

\perlExec{
    use PDL;
    my $v = vpdl [ 1, 2 ];
    my $w = vpdl [ 3, 4 ];
    sub list ($) {
            my $pdl = shift;
            return join ',', map { at( $pdl, $_, 0 ) } 0..1;
    }
}
The vector [ \perlValue{ list $v } ]
plus the vector [ \perlValue{ list $w } ]
makes [ \perlValue{ my $s = $v + $w; list $s } ].

Answer 1

As I commented, my feeling was that this was an XY Problem in that there may be other solutions to the underlying problem. As it turns out from the discussion in the comments, you seem to be implementing your own templating system, so my first suggestion would be to have a look at existing ones, such as maybe Template::Toolkit.

If you still want to stick with your current approach, then it seems that @hobbs has given an answer that seems to answer your question directly, Eval::WithLexicals (Update: since accepted). As I mentioned, I see two other possible solutions. The first, which feels the most natural to me personally, would be to not use lexicals in the fist place. When I see the code like what you showed:

\perlExec{ my $v = [ 1, 2 ]; }
The vector [ \perlValue{ $v } ]

then, simply because of the braces, I would not be surprised that each of those has their own lexical scope. If you instead were to use package variables, it would seem more natural to me. Just for example, you could use eval qq{ package $packname; no strict "vars"; $code } (of course with the caveat that strict "vars" is disabled), or your could use fully qualified variable names ($package::v) throughout.

The second thing I mentioned was to translate the entire input file into a Perl script and eval that - in other words, write your own templating system. Although I would only recommend reinventing this wheel as your last option, you did ask how to adapt this code I wrote for your purposes, so here it is. One limitation of the following is that any braces in the code blocks must be balanced (but see update below), and since this is a somewhat simplistic demonstration, there are bound to be more limitations. Use at your own risk!

use warnings;
use strict;
use feature qw/say state/;
use Data::Dumper;
use Regexp::Common qw/balanced/;
use Capture::Tiny qw/capture_stdout/;
my $DEBUG = 1;

local $/=undef;
while (my $input = <>) {
    my $code = translate($ARGV,$input);
    $DEBUG and say ">>>>> Generated Code:\n", $code, "<<<<<";
    my ($output, $rv) = capture_stdout { eval $code };
    $rv or die "eval failed: ".($@//'unknown error');
    say ">>>>> Output:\n", $output, "<<<<<";
}

sub translate {
    my ($fn,$input) = @_;
    state $packcnt = 1;
    $fn =~ tr/A-Za-z0-9/_/cs;
    my $pack = "Generated".$packcnt++."_$fn";
    my $output = "{ package $pack;\n";
    $output.= "no warnings; no strict;\n";
    $output.= "#line 1 \"$pack\"\n";
    while ( $input=~m{ \G (?<str> .*? ) \\perl(?<type> Exec|Value )
                     (?<code> $RE{balanced}{-parens=>'{}'} ) }xsgc ) {
        my ($str,$type,$code) = @+{qw/str type code/};
        $output.= "print ".perlstr($str).";\n" if length($str);
        ($code) = $code=~/\A\s*\{(.*)\}\s*\z/s or die $code;
        $code .= ";" unless $code=~/;\s*\z/;
        $code = "print do { $code };" if $type eq 'Value';
        $output.= "$code\n";
    }
    my $str = substr $input, pos($input)//0;
    $output.= "print ".perlstr($str).";\n" if length($str);
    $output.= "} # end package $pack\n1;\n";
    return $output;
}

sub perlstr { Data::Dumper->new([''.shift])
    ->Terse(1)->Indent(0)->Useqq(1)->Dump }

Input File:

\perlExec{
    use warnings; use strict;
    print "Hello, World\n";
    my $v = [ 1, 2 ];
    my $w = [ 3, 4 ];
    sub list ($) {
        my $pdl = shift;
        return join ',', @$pdl;
    }
}
The vector [ \perlValue{ list $v } ]
plus the vector [ \perlValue{ list $w } ]
makes [ \perlValue{ my $s = [@$v + @$w]; list $s } ].

Output:

>>>>> Generated Code:
{ package Generated1_input_txt;
no warnings; no strict;
#line 1 "Generated1_input_txt"

    use warnings; use strict;
    print "Hello, World\n";
    my $v = [ 1, 2 ];
    my $w = [ 3, 4 ];
    sub list ($) {
        my $pdl = shift;
        return join ',', @$pdl;
    }
;
print "\nThe vector [ ";
print do {  list $v ; };
print " ]\nplus the vector [ ";
print do {  list $w ; };
print " ]\nmakes [ ";
print do {  my $s = [@$v + @$w]; list $s ; };
print " ].\n";
} # end package Generated1_input_txt
1;
<<<<<
>>>>> Output:
Hello, World

The vector [ 1,2 ]
plus the vector [ 3,4 ]
makes [ 4 ].
<<<<<

Update: As suggested by @HåkonHægland in the comments, it's possible to use PPR to parse out the blocks. The only changes needed are to replace use Regexp::Common qw/balanced/; by use PPR; and in the regex replace (?<code> $RE{balanced}{-parens=>'{}'} ) by (?<code> (?&PerlBlock) ) $PPR::GRAMMAR - then the parser will handle a case like print "Hello, World }\n"; too!

Answer 2

Perhaps you would like to use Eval::WithLexicals? It does exactly what you are asking for. It was designed to power REPLs, and it's pure Perl. You simply make a new instance of Eval::WithLexicals, and then you call $ewl->eval($code) instead of eval $code, and the variables will persist between successive calls on the same object.

Answer 3

Here is one approach: Parse the template file twice. On first parse, write Perl statements from the template to a temp file, for example /tmp/MyTemplate.pm, add some header code to this file such as to make it a valid Perl module. Also use a sequential numbering into package variables for \perlValue statements, i.e. translate the first \perlValue{ list $v } into for example: our $perl_value1 = list $v;, the next \perlValue{ list $w } becomes our $perl_value2 = list $w; and so on..

Then require the module: require "/tmp/MyTmplate.pm"; Then parse the template a second time, extracting the correct values corresponding to Perl code in the template from the symbol table of MyTemplate. For example to get the value of \perlValue{ list $v } use $MyTemplate::perl_value1 and so on..