Unexpected syntax error in map when first element is literal #22716

happy-barney · 2024-10-31T13:59:30Z

Description
There is unexpected syntax error when using map (see example)

Steps to Reproduce
Run following code:

package Foo {
	sub list {
		+{ name => 1 },
	}
};

my $self = bless {}, Foo::;

my @list1 = map { $_->{name} => $_ } $self->list;
my @list2 = map { "$_->{name}" => $_ } $self->list;
my @list3 = map { name => $_ } $self->list;

While list1 expression is ok, list2 and list3 ends with

syntax error at ..., near "} $self"

Update: using + (eg: +name) mitigates bug

Expected behavior
both list2 and list3 expressions should be valid

Perl configuration
tested with:

perl 5.28
perl 5.38
perl 5.40
(perlbrew perls, -V for 5.40)

Summary of my perl5 (revision 5 version 40 subversion 0) configuration:
   
  Platform:
    osname=linux
    osvers=5.15.0-113-generic
    archname=x86_64-linux
    uname='linux happy-barney 5.15.0-113-generic #123-ubuntu smp mon jun 10 08:16:17 utc 2024 x86_64 x86_64 x86_64 gnulinux '
    config_args='-de -Dprefix=/home/brano/.cache/perlbrew/perls/perl-5.40 -Aeval:scriptdir=/home/brano/.cache/perlbrew/perls/perl-5.40/bin'
    hint=recommended
    useposix=true
    d_sigaction=define
    useithreads=undef
    usemultiplicity=undef
    use64bitint=define
    use64bitall=define
    uselongdouble=undef
    usemymalloc=n
    default_inc_excludes_dot=define
  Compiler:
    cc='cc'
    ccflags ='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64'
    optimize='-O2'
    cppflags='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
    ccversion=''
    gccversion='11.4.0'
    gccosandvers=''
    intsize=4
    longsize=8
    ptrsize=8
    doublesize=8
    byteorder=12345678
    doublekind=3
    d_longlong=define
    longlongsize=8
    d_longdbl=define
    longdblsize=16
    longdblkind=3
    ivtype='long'
    ivsize=8
    nvtype='double'
    nvsize=8
    Off_t='off_t'
    lseeksize=8
    alignbytes=8
    prototype=define
  Linker and Libraries:
    ld='cc'
    ldflags =' -fstack-protector-strong -L/usr/local/lib'
    libpth=/usr/local/lib /usr/lib/x86_64-linux-gnu /usr/lib /usr/lib64
    libs=-lpthread -ldb -ldl -lm -lcrypt -lutil -lc
    perllibs=-lpthread -ldl -lm -lcrypt -lutil -lc
    libc=/lib/x86_64-linux-gnu/libc.so.6
    so=so
    useshrplib=false
    libperl=libperl.a
    gnulibc_version='2.35'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs
    dlext=so
    d_dlsymun=undef
    ccdlflags='-Wl,-E'
    cccdlflags='-fPIC'
    lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector-strong'


Characteristics of this binary (from libperl): 
  Compile-time options:
    HAS_LONG_DOUBLE
    HAS_STRTOLD
    HAS_TIMES
    PERLIO_LAYERS
    PERL_COPY_ON_WRITE
    PERL_DONT_CREATE_GVSV
    PERL_HASH_FUNC_SIPHASH13
    PERL_HASH_USE_SBOX32
    PERL_MALLOC_WRAP
    PERL_OP_PARENT
    PERL_PRESERVE_IVUV
    PERL_USE_SAFE_PUTENV
    USE_64_BIT_ALL
    USE_64_BIT_INT
    USE_LARGE_FILES
    USE_LOCALE
    USE_LOCALE_COLLATE
    USE_LOCALE_CTYPE
    USE_LOCALE_NUMERIC
    USE_LOCALE_TIME
    USE_PERLIO
    USE_PERL_ATOF
  Built under linux
  Compiled at Jul  6 2024 19:31:58
  %ENV:
    PERL5LIB="/home/brano/.perlbrew/libs/perl-5.40@common/lib/perl5:/home/brano/lib/perl5/:/home/brano/lib/perl5/:/home/brano/lib/perl5/"
    PERLBREW_HOME="/home/brano/.perlbrew"
    PERLBREW_LIB="common"
    PERLBREW_MANPATH="/home/brano/.perlbrew/libs/perl-5.40@common/man:/home/brano/.cache/perlbrew/perls/perl-5.40/man"
    PERLBREW_PATH="/home/brano/.perlbrew/libs/perl-5.40@common/bin:/home/brano/.cache/perlbrew/bin:/home/brano/.cache/perlbrew/perls/perl-5.40/bin"
    PERLBREW_PERL="perl-5.40"
    PERLBREW_ROOT="/home/brano/.cache/perlbrew"
    PERLBREW_SHELLRC_VERSION="0.86"
    PERLBREW_VERSION="0.94"
    PERL_LOCAL_LIB_ROOT="/home/brano/.perlbrew/libs/perl-5.40@common"
    PERL_MB_OPT="--install_base /home/brano/.perlbrew/libs/perl-5.40@common"
    PERL_MM_OPT="INSTALL_BASE=/home/brano/.perlbrew/libs/perl-5.40@common"
  @INC:
    /home/brano/.perlbrew/libs/perl-5.40@common/lib/perl5/x86_64-linux
    /home/brano/.perlbrew/libs/perl-5.40@common/lib/perl5
    /home/brano/lib/perl5/
    /home/brano/lib/perl5/
    /home/brano/lib/perl5/
    /home/brano/.cache/perlbrew/perls/perl-5.40/lib/site_perl/5.40.0/x86_64-linux
    /home/brano/.cache/perlbrew/perls/perl-5.40/lib/site_perl/5.40.0
    /home/brano/.cache/perlbrew/perls/perl-5.40/lib/5.40.0/x86_64-linux
    /home/brano/.cache/perlbrew/perls/perl-5.40/lib/5.40.0

The text was updated successfully, but these errors were encountered:

demerphq · 2024-10-31T15:50:32Z

Yeah, this is an old issue. Since parens arent required on function calls there is a parse ambiguity between a `map anon-hash-constructor, LIST` and `map block LIST`. So the parser uses a heuristic to decide and in some cases gets it wrong. In the past we used to throw errors from "use" statements, but it seems the heuristic has been improved since then. A trick to disambiguate is to put a semicolon at the start of the block which forces the compiler to understand it is a block and not an anonymous hash constructor. $ perl -wle'my @x=map {; x => $_ } (1..3); print $x[0]' x $ perl -wle'my @x=map { x => $_ } (1..3); print $x[0]' syntax error at -e line 1, near "} (" Execution of -e aborted due to compilation errors.

…

On Thu, 31 Oct 2024 at 14:59, Branislav Zahradník ***@***.***> wrote: *Description* There is unexpected syntax error when using map (see example) *Steps to Reproduce* Run following code: package Foo { sub list { +{ name => 1 }, } }; my $self = bless {}, Foo::; my @list1 = map { $_->{name} => $_ } $self->list;my @LIST2 = map { "$_->{name}" => $_ } $self->list;my @List3 = map { name => $_ } $self->list; While list1 expression is ok, list2 and list3 ends with syntax error at ..., near "} $self" *Expected behavior* both list2 and list3 expressions should be valid *Perl configuration* tested with: - perl 5.28 - perl 5.38 - perl 5.40 (perlbrew perls, -V for 5.40) Summary of my perl5 (revision 5 version 40 subversion 0) configuration: Platform: osname=linux osvers=5.15.0-113-generic archname=x86_64-linux uname='linux happy-barney 5.15.0-113-generic #123-ubuntu smp mon jun 10 08:16:17 utc 2024 x86_64 x86_64 x86_64 gnulinux ' config_args='-de -Dprefix=/home/brano/.cache/perlbrew/perls/perl-5.40 -Aeval:scriptdir=/home/brano/.cache/perlbrew/perls/perl-5.40/bin' hint=recommended useposix=true d_sigaction=define useithreads=undef usemultiplicity=undef use64bitint=define use64bitall=define uselongdouble=undef usemymalloc=n default_inc_excludes_dot=define Compiler: cc='cc' ccflags ='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64' optimize='-O2' cppflags='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include' ccversion='' gccversion='11.4.0' gccosandvers='' intsize=4 longsize=8 ptrsize=8 doublesize=8 byteorder=12345678 doublekind=3 d_longlong=define longlongsize=8 d_longdbl=define longdblsize=16 longdblkind=3 ivtype='long' ivsize=8 nvtype='double' nvsize=8 Off_t='off_t' lseeksize=8 alignbytes=8 prototype=define Linker and Libraries: ld='cc' ldflags =' -fstack-protector-strong -L/usr/local/lib' libpth=/usr/local/lib /usr/lib/x86_64-linux-gnu /usr/lib /usr/lib64 libs=-lpthread -ldb -ldl -lm -lcrypt -lutil -lc perllibs=-lpthread -ldl -lm -lcrypt -lutil -lc libc=/lib/x86_64-linux-gnu/libc.so.6 so=so useshrplib=false libperl=libperl.a gnulibc_version='2.35' Dynamic Linking: dlsrc=dl_dlopen.xs dlext=so d_dlsymun=undef ccdlflags='-Wl,-E' cccdlflags='-fPIC' lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector-strong' Characteristics of this binary (from libperl): Compile-time options: HAS_LONG_DOUBLE HAS_STRTOLD HAS_TIMES PERLIO_LAYERS PERL_COPY_ON_WRITE PERL_DONT_CREATE_GVSV PERL_HASH_FUNC_SIPHASH13 PERL_HASH_USE_SBOX32 PERL_MALLOC_WRAP PERL_OP_PARENT PERL_PRESERVE_IVUV PERL_USE_SAFE_PUTENV USE_64_BIT_ALL USE_64_BIT_INT USE_LARGE_FILES USE_LOCALE USE_LOCALE_COLLATE USE_LOCALE_CTYPE USE_LOCALE_NUMERIC USE_LOCALE_TIME USE_PERLIO USE_PERL_ATOF Built under linux Compiled at Jul 6 2024 19:31:58 %ENV: ***@***.***/lib/perl5:/home/brano/lib/perl5/:/home/brano/lib/perl5/:/home/brano/lib/perl5/" PERLBREW_HOME="/home/brano/.perlbrew" PERLBREW_LIB="common" ***@***.***/man:/home/brano/.cache/perlbrew/perls/perl-5.40/man" ***@***.***/bin:/home/brano/.cache/perlbrew/bin:/home/brano/.cache/perlbrew/perls/perl-5.40/bin" PERLBREW_PERL="perl-5.40" PERLBREW_ROOT="/home/brano/.cache/perlbrew" PERLBREW_SHELLRC_VERSION="0.86" PERLBREW_VERSION="0.94" ***@***.***" PERL_MB_OPT="--install_base ***@***.***" ***@***.***" @inc: ***@***.***/lib/perl5/x86_64-linux ***@***.***/lib/perl5 /home/brano/lib/perl5/ /home/brano/lib/perl5/ /home/brano/lib/perl5/ /home/brano/.cache/perlbrew/perls/perl-5.40/lib/site_perl/5.40.0/x86_64-linux /home/brano/.cache/perlbrew/perls/perl-5.40/lib/site_perl/5.40.0 /home/brano/.cache/perlbrew/perls/perl-5.40/lib/5.40.0/x86_64-linux /home/brano/.cache/perlbrew/perls/perl-5.40/lib/5.40.0 — Reply to this email directly, view it on GitHub <#22716>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAZ5R4VYFPPP2V7PZ2UTYDZ6IZVXAVCNFSM6AAAAABQ6LIEXSVHI2DSMVQWIX3LMV43ASLTON2WKOZSGYZDMOJTHA4TKMA> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

-- perl -Mre=debug -e "/just|another|perl|hacker/"

demerphq · 2024-10-31T15:53:17Z

On Thu, 31 Oct 2024 at 16:50, demerphq ***@***.***> wrote: Yeah, this is an old issue. Since parens arent required on function calls there is a parse ambiguity between a `map anon-hash-constructor, LIST` and `map block LIST`.

I forgot to mention, this is documented in perlfunc, see perldoc -f map: "{" starts both hash references and blocks, so "map { ..." could be either the start of map BLOCK LIST or map EXPR, LIST. Because Perl doesn't look ahead for the closing "}" it has to take a guess at which it's dealing with based on what it finds just after the "{". Usually it gets it right, but if it doesn't it won't realize something is wrong until it gets to the "}" and encounters the missing (or unexpected) comma. The syntax error will be reported close to the "}", but you'll need to change something near the "{" such as using a unary "+" or semicolon to give Perl some help:

…

-- perl -Mre=debug -e "/just|another|perl|hacker/"

happy-barney · 2024-10-31T15:59:51Z

that explains { name => $_ } part
what about { "foo" => $_ } part ?

happy-barney · 2024-10-31T16:16:56Z

ok, so it looks like source of problem is yyl_leftcurly and it's treating of '{' as HASHBRACK magic to recognize between block and anonymous hash

so maybe in case of & prototype (not only map, grep, sort ..) block behaviour should be preferred unless +{ } is used ?

demerphq · 2024-10-31T17:16:49Z

On Thu, 31 Oct 2024 at 17:17, Branislav Zahradník ***@***.***> wrote: ok, so it looks like source of problem is yyl_leftcurly and it's treating of '{' as HASHBRACK magic to recognize between block and anonymous hash so maybe in case of & prototype (not only map, grep, sort ..) block behaviour should be preferred unless +{ } is used ?

I'd have thought that any significant change to this heuristic that is not a specific exception using a single token of lookahead that is absolutely robust would break at least some code out there. Yves

…

-- perl -Mre=debug -e "/just|another|perl|hacker/"

jkeenan · 2024-11-11T01:19:44Z

As I re-read this ticket, it appears that, although the syntax error reported may be surprising, it's not really "unexpected," as it pertains to a design flaw we're well aware of and don't really expect to be able to correct. If we do want to change things, we should discuss it on the mailing list first. In the mean time, I think this ticket is closable and am marking it as such.

happy-barney · 2024-11-11T07:00:53Z

@jkeenan flaw is due incapability of LALR(1) grammar to distinguish anonymous hash from block.
This can be mitigated by (but not only):

forcing leading + where structure is required
more expensive forward lookup (not only at second significant token but for first operator which is not comma)

Nov 11, 2024 · 2024-11-11T11:10:23Z

More look-ahead inside { } will not save you:

map { foo($_, 2, 3) } ...

This could plausibly be either a block (returning the return value(s) of foo() for each element) or a hashref constructor (initialized from the return values of foo()).

More look-ahead past { } will not save you:

map { foo($_) } /$x, $y, $z; #/g;

This could be the map BLOCK LIST form, the list being formed by repeatedly matching the regex $x, $y, $z; # against the default variable $_, or the map EXPR, LIST form, the list consisting of the variables $y and $z.

Requiring a leading + for the hashref interpretation would break existing code.

happy-barney · 2024-11-11T11:24:28Z

Requiring a leading + for the hashref interpretation would break existing code.

there are two ways how to prevent this:

feature
dialect (which btw will bring higher stability even to XS API, when properly implemented)

Meanwhile: + is easy way how to force {} to be hashref. So there should be also counterpart to be able to force {} to be block.

I'd be even fine with:

map do { } @list

Nov 11, 2024 · 2024-11-11T11:27:24Z

So there should be also counterpart to be able to force {} to be block.

That would be map {; ... } LIST. But map do { ... }, LIST also works.

happy-barney · 2024-11-11T11:32:43Z

oh, that didn't occur to me, nice one ... and a mess tidy in code base will introduce ... :-)

happy-barney added the Needs Triage label Oct 31, 2024

jkeenan removed the Needs Triage label Oct 31, 2024

jkeenan added the Closable? We might be able to close this ticket, but we need to check with the reporter label Nov 11, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Unexpected syntax error in map when first element is literal #22716

Unexpected syntax error in map when first element is literal #22716

happy-barney commented Oct 31, 2024 •

edited

Loading

demerphq commented Oct 31, 2024 via email

demerphq commented Oct 31, 2024 via email

happy-barney commented Oct 31, 2024

happy-barney commented Oct 31, 2024

demerphq commented Oct 31, 2024 via email

jkeenan commented Nov 11, 2024

happy-barney commented Nov 11, 2024

Nov 11, 2024

happy-barney commented Nov 11, 2024

Nov 11, 2024

happy-barney commented Nov 11, 2024

Unexpected syntax error in map when first element is literal #22716

Unexpected syntax error in map when first element is literal #22716

Comments

happy-barney commented Oct 31, 2024 • edited Loading

demerphq commented Oct 31, 2024 via email

demerphq commented Oct 31, 2024 via email

happy-barney commented Oct 31, 2024

happy-barney commented Oct 31, 2024

demerphq commented Oct 31, 2024 via email

jkeenan commented Nov 11, 2024

happy-barney commented Nov 11, 2024

Nov 11, 2024

happy-barney commented Nov 11, 2024

Nov 11, 2024

happy-barney commented Nov 11, 2024

happy-barney commented Oct 31, 2024 •

edited

Loading