[Templates] Syntactical Scalar call

[Andy Wardley]

Hi Paul,

Thanks once again for all your efforts.  I've been thinking about some of
these issues myself (well, for quite some time) and I'm in the process of
writing up a more formal definition of what I'm planning to do with the
language.  But seeing as you brought it up, I'll brain dump some of the
relevant bits that you've touched on.  In particular:

   * calling subroutines or object methods in list or scalar context

   * disambiguating between hash items (entries) and virtual methods

 > So - TT2 is plagued with the "all methods are called in scalar context" 
issue.

s/scalar/list, but yes, it's a plague of sorts.

I'm currently thinking that the default in TT3 will be to call in scalar
context.  The current support for list context and auto-collection of multiple
return values is all a bit too halfwit to work reliably and the generally
accepted wisdom is to return list references if you want your code to play
nicely with TT.

Of course, you may not always have control over the back-end code you're using
(Class::DBI for example), so I agree that we need some way to explicitly
invoke list context for a subroutine or method call.  But I think it makes
good sense for scalar context to be the default and have list context the
exception to the rule.

 > There have been recent patches to "fix" this, and there was even a new plugin
 > by Andy that provides for a scalar vmethod - though it requires a certain
 > level of voodoo to make it possible.  The scalar vmethod is a neat hack - but
 > it is also a little ambiguous as we already have an item vmethod.  Not only
 > that but it is the only vmethod that alters the chain that follows it.

True, although I'm thinking about making this kind of thing more common in
TT3.  e.g.

     [% hash.item('foo') %]      # regular method call
     [% hash.item.foo %]         # syntactic sugar using a monad

Dotops that expect at least one argument and receive none can return
a monad to read the next argument from the dotop chain.  This kind of
thing is really quite powerful as it implements a stateful object (that's
the monad bit) which can be used across "broken" dotop chains:

    [% items = hash.item %]
    [% items.foo %]     # same as hash.item.foo or hash.item('foo')
    [% items.bar %]

Anyway, I digress. The scalar vmethod certainly is a bit of voodoo that
doesn't really fit into TT2 as it is. Although it might fit into TT3 a little
better, it's still a sub-optimal solution to this particular problem. Apart
from anything else, you can't do this:

   [% scalar.my_sub %]

 > I don't think that any real allowance needs to be made for top level
 > functions.

Hmmm... I agree that you're more likely to have control and so it's less of a
problem, but I think that having something that works in all cases is to be
preferred. I think it's also more indicative of a good fit between the
underlying concept ("evaluate the next thing in list context") and the symbol
we use to represent it.  If the concept applies in both places (which it does)
then it's better to make the syntax work in both places than write
documentation explaining why it doesn't.

So if we accept that we *should* handle top-level functions, then it
effectively prevents us from using a "dot-something" operator if that
"something" can't appear without the dot prefix.

   [% foo.+bar %]     # OK
   [% +bar %]         # NOT OK - '+' is already a valid prefix/infix operator

I'm currently leaning towards '@' for obvious Perlish array reasons.

   [% @foo %]          # evaluate 'foo' in list context
   [% foo. at bar %]      # evaluate foo object method bar in list context
   [% foo at bar %]       # syntactic sugar for above

BTW, it also needs to play nicely with an interpolated '$key'.  e.g.

   [% search  = 'albums'
      results = artist.@$search    # same as artist. at albums
   %]

That doesn't look *too* bad, but I'd also like to offer the equivalent of the
.scalar. vmethod for people who prefer spelling things out. Only problem is we
can't call it .list. because we've already got one of those so we need another
name. 'slurp' is the best I could come up with. Other suggestions welcome.

   [% foo.slurp.bar %] ===> [% foo.slurp('bar') %] ==> [% foo at bar %]

 > In TT3, the plan is to merge filters and vmethods in-as-much as it is
 > possible.

Yep.  There will be no more filters.  Filtering will effectively be the
same thing as calling a virtual method. '|' will be the high precedence
operator and 'filter' will be the low precedence keyword which also acts
as the block directive (aka FILTER in TT2).

   [% foo | bar %]        # pipe foo through bar vmethod/filter

   [% foo filter bar %]   # same as above

   [% filter bar %]       # block form
        ...
   [% end %]

Or it might be 'pipe' instead of 'filter', but it's almost certainly going to
be lower case by default (along with all the other directives), because I'm
FED UP WITH TYPING THINGS IN UPPER CASE!!! (there are good reasons for TT2
using UPPER CASE keywords, but they're going away in TT3).

Anyway, I digress... again...

 > The only difference is that when a pipe
 > is used only vmethods or filters are searched for - no methods or hash values
 > are looked for.

There are actually two separate issues here.  There's the list vs scalar
context issue, but we also need some way to disambiguate between hash items
and vmethods.  The '|' works fine as a way to call vmethods only, and ties
in nicely with the homogenisation of vmethods and filters.

   [% foo filter html %] ==> [% foo | html %] ==> [% foo.html %]

But when hashes are concerned, we also need some way to say that we're
only interested in a hash item, and don't want a vmethod called.  For
example, say you want $hash->{ keys } which may, or may not be defined:

    [% IF hash.keys %]   # FAIL!  No cigar.

In this case, the IF will never resolve false because the keys virtual method
kicks in and returns a reference to a list of keys.  The "spelled out" way to
fix this is to use the .item virtual method.

    [% hash.item('foo') %]     # TT2/TT3
    [% hash.item.foo %]        # TT3

But that's sub-optimal because it invokes an extra method call just
to fetch a hash item that should be a simple operation. Not to mention the
fact that it's longer to type - it's bad Huffman Coding as Larry would say.

So I'm planning to adopt the #key fragment identifier (as used in URIs like
http://example.com/index.html#section2), for accessing hash (and list) items,
bypassing the vmethods altogether.

   [% hash#foo %]              # same as hash.item('foo') but more efficient
   [% list#3 %]                # same as list.item(3)

(actually, lists aren't a problem because their indices are all-numerical
which vmethods aren't, but consistency between lists and hashes is a good
thing).

The implication of this is that #comments will need to always have whitespace
before them (except at the start of a tag), but I don't think that's a major
problem as I imagine it's very unlikely for anyone to put their comments right
up against the thing they're commenting

   [% foo  # a comment
      foo # another comment
      foo#fragment                <-- this isn't a comment in TT3
   %]

Or we use something else instead of '#', but I've been through the entire
character set and I think this is the best fit, particularly seeing as it
has the same semantic meaning as how it is used in URIs (the sub-part of
the document identified by XXX vs the sub-part of the hash identified by
XXX).

And finally, I'm also thinking about making the dotop look for virtual
methods *first* before looking for hash items.  So hash.keys will always
resolve to the .keys vmethod, regardless of their being a 'keys' item or not.

   [% hash.keys %]             # vmethod, regardless of any 'keys' item
   [% hash#keys %]             # hash item, regardless of any 'keys' vmethod

I'm not yet sure if that's a good idea or not. I need to think about it some
more (particularly the performance implications), but I'm warming to the fact
that it's more deterministic (i.e. because you know up front that there's a
hash vmethod called 'keys' so you can be sure that hash.keys will be resolved
as a vmethod, whereas the current "hash item first, vmethod second" policy
depends on the contents of the hash which you can't grok up front).

 > So with further thought - I think the following should be used:
 >
 >    [% foo.method(bar) %]  # normal smart
 >    [% foo.*method(bar) %] # same thing (mnemonic 0 or more)
 >    [% foo.?method(bar) %] # scalar context (mnemonic 0 or 1)
 >    [% foo.+method(bar) %] # full list context - always a list (mnemonic 1 or
 > more)
 >    [% foo.!method(bar) %] # void context (this is optional)

And my list is currently looking like this:

      [% foo %]              # scalar context for sub/item
      [% @foo %]             # list context for sub/item

      [% obj.method(bar) %]  # scalar context
      [% obj. at method(bar) %] # list context
      [% obj at method(bar) %]  # sugar 4 above 4 rly lzy ppl

      [% hash.thing %]       # vmethod or item (or item/vm depending on order)
      [% hash#thing %]       # item only
      [% hash|thing %]       # vmethod only

Whadja think?  And what says the collective wisdom of the templates mailing
list?  Anyone got any better ideas?  Comments welcome as always.

A