pongo2: Template-engine and successor of pongo back to frontpage

Almost two years after releasing pongo, a Django-syntax like template-engine for Golang, I’m proud to announce that I’ve published a preview and beta of the successor of pongo: pongo2.

pongo2 is a complete rewrite and aims the same goal as pongo: being compatible with Django templates.

I put a lot of effort and new features into pongo2 including complex and nested function calls and powerful C-like expressions. Even though it’s still in beta and lacks of examples and documentation, I want to give you a short overview of what’s new and what’s the difference between pongo2-templates and Django-templates.

For those of you who are new to Django’s template system, please have a look on the official Django documentation.

I will improve the pongo2 documentation over time. You can look it up here on godoc.org.

At the end of this post you will find a migration tutorial.

New: Try pongo2 out in the pongo2 playground.

Update: pongo2 1.0 stable has been released.

C-like expressions

pongo2 supports C-like expressions in any situation. Quick examples:

{% if user.is_moderator && (user.moderation_level > 2 || user.moderation_level <= 0) %}
   ...
{% else %}
   ...
{% endif %}

Artithmetic operations and filters in expressions:

{% if my_integer + 5 > 10 && my_string|length - 3 < 15 %} ... {% endif %}

Expressions can be used in variables ({{ ... }}) as well:

{{ square_root ^ 2 }}

To check whether an item contains a specific element, the in-operator is available (so are function calls, see next section):

{% if !(user in banned_list) && user.is_verified() %} ... {% endif %}

Since pongo2 aims to be compatbile with Django templates it’s possible to use the keywords and, or and not for &&, || and ! respectively. You can use arithmetic operations in expressions like +, -, *, / and ^ (power). It is allowed to apply a filter on a variable or function call (like my_string|length) anytime.

Function calls

It’s now possible to call functions within any expression. Quick examples:

{% if (check_access("admin", userprofile.username) && userprofile.is_verified())
    || admin_happyhour %} ... {% endif %}

{% if helper_functions.is_server_online("florian-schlachter.de") %}Server is up!{% endif %}

{{ make_statuscode_verbose(
    user.lang, make_http_request(server.url, "/index.php").statuscode) }}

As you can see even nested functions are allowed. You can provide functions through the template Context. A special pongo2 type system makes it simple to write such functions:

func check_access(role, username *pongo2.Value) *pongo2.Value {
    // checks a users db (here a map) for a specific role- and username-entry
    _, allowed := users_db[role.String()][username.String()]

    // returns a bool whether access is granted or not
    return pongo2.AsValue(allowed)
}

All functions’ parameters types must be of either your own type or of type *pongo2.Value(no matter how many) and functions must return one value of either type *Value or your own one. pongo2’s Value-object provides a simple mechanism to receive and return data and is more fault-tolerant. When providing your own types (like int or string), the incoming arguments must match these types. pongo2.Values are accepting almost anything and allows you to convert them on the fly in your function. When using pongo2.Value, you can return any data you want using the AsValue() helper function. You can safely access any received data by using methods like String() or Integer(). You can even Slice() data (for arrays, slices or strings) or Iterate() over maps, array, slices and strings. See the documentation for the Value object for all methods being provided.

Numbers (integers and floats)

Update (July 3rd 2014): Clarification on where float literals are possible.

Number literals are mostly integers (see below for the exception!). To always be able to work with floats pongo2 introduces a special float filter which converts the given value (number, string) to a 64-bit float. You can use it like this:

{% if 3^3 == 27|float %}Yep, you're right!{% endif %}

(Since the power-operator always returns a 64-bit float, it’s neccessary to compare two floats here.)

To convert a float to an integer, pongo2 provides the integer filter which can be used in the same way.

In expressions a number notated like <number>.<number> will be interpreted by pongo2 as a float. So in addition to the first example the following if-condition is possible as well:

{% if 3^3 == 27.0 %}Yep, you're right!{% endif %}

Please note, that you can’t use a float to access a variable (or to address a field from a variable):

{{ myvar.4.5 }}

4 and 5 will not be interpreted as a 4.5 float, instead pongo2 will access the 4th element of myvar and then the 5th element of the resulting value of myvar.4.

Writing your own tags and filters

I’m very happy with the new tags and filters system. It’s very powerful and allows you to extend pongo2 in a very efficient way. It’s now possible to create tags and filters as 3rd-party applications and to integrate them into pongo2 by just adding their import path to the source like this:

"github.com/flosch/pongo2-super-duper-filters"

The imported filters are automatically being recognized by pongo2.

Filters

A filter function has always the following function signature:

func(in *Value, param *Value) (*Value, error)

You’re already familiar with pongo2’s Value-objects. The in parameter gives you access to the value which is being passed to the filter. Since it’s possible to pass a parameter to the filter (like this: "Hello there"|truncatewords:1) the param argument will give you access to that (1 in the example by accessing param.Integer()). Once you’ve finished processing the data, just return it by using the AsValue helper function. Whenever you’re encountering an error, you’re free to respond with a Golang error object.

To make your filter known to pongo2, you have to register the new filter function by calling RegisterFilter(). I highly recommend that you’re doing this in the init() function (in the same source file where you’ve declared the filter) which is always being called on program startup.

func init() {
    RegisterFilter("superfilter", filterSuperfilter)
}

Tags

Writing tags is slightly more difficult than writing filters because you have to parse the tags’ arguments. Luckily pongo2 provides a comprehensive parser toolkit which makes writing tags pretty easy.

Technically tags are basically consisting of a parser function and an execution function. The parser function gets called during the parsing phase, the execution function gets called whenever the template is being rendered. Since you often want to compute things for the tag (like parsing an expression and save it for later evaluation), it’s required that the parsing function returns a so-called NodeTag. It’s basically an empty struct with your private fields and which provides an Execute() function.

One would think that implementing an if-tag would be hard stuff, but in pongo2 it isn’t. I want to show and explain you the actual implementation of the if-tag.

The struct to save all relevant information for the if-tag looks like this:

type tagIfNode struct {
    condition   IEvaluator
    thenWrapper *NodeWrapper
    elseWrapper *NodeWrapper
}

The condition is provided by the user ({% if condition %}) and will be saved for later evaluation in the struct. The IEvaluator interface requires the object being saved here to provide an Evaluate() function:

Evaluate(*ExecutionContext) (*Value, error)

We will come back to this later.

thenWrapper and elseWrapper are of type *NodeWrapper which provides a neat interface to render the HTML or any other tags/variables wrapped by two tags. E. g. thenWrapper in the if-struct contains any stuff wrapped by {% if ... %} and either {% else %} or {% endif %}; if the user provided an else-tag, the elseWrapper contains any stuff which is wrapped by {% else %} and {% endif %}. pongo2’s parser toolkit provides a method to create such NodeWrappers easily: Parser.WrapUntilTag (see below).

With your tag’s registration you’re providing a parser function with the following signature:

func(doc *Parser, start *Token, arguments *Parser) (INodeTag, error)

The doc parameter provides access to the whole template document (i. e. to the WrapUntilTag() function), the start parameter provides information about the first so-called token (a lexical unit which includes a value, type and a position information) within a tag; in this case start contains the name of tag (field Val), the information that the name of the tag is an identifier (field Typ holding the value TokenIdentifier) as well as a Line and Col (column). arguments provides parsing access to any arguments provided to the tag by the user (in if’s case: the condition).

The parser for the if-tag looks like the following. I put verbose comments in it.

func tagIfParser(doc *Parser, start *Token, arguments *Parser) (INodeTag, error) {
    if_node := &tagIfNode{}

    // Parse condition
    // pongo2's parser already provides an easy way to parse 
    // any expression you're expecting somewhere through ParseExpression().
    // It returns an object which implements the IEvaluator interface 
    // (and therefore provides an Evaluate() method for later evaluation).
    condition, err := arguments.ParseExpression()
    if err != nil {
        return nil, err
    }
    if_node.condition = condition

    // We're checking whether the condition is syntactically correct.
    // If ParseExpression() didn't consumed all tokens, we're complaining 
    // a syntax error using the Error() function. This function automatically
    // creates a nice error message including position information.
    // We're checking for token consumption by calling Remaining() which
    // returns the number of unconsumed tokens.
    if arguments.Remaining() > 0 {
        return nil, arguments.Error("If-condition is malformed.", nil)
    }

    // Wrap then/else-blocks
    // As already explained, the parser provides a nice method WrapUntilTag()
    // which allows you to wrap any HTML or tags/filters within two corresponding
    // tags (i. e. an open and close tag like if/endif). You can provide
    // as many end tags as you want; the function only returns one wrapper to 
    // the endtag which comes first. You may have to check for which wrapper has been
    // returned and then call WrapUntilTag() again for the next block (like it's done here
    // in the example to save both the "then"-block and the "else"-block).
    // With the first WrapUntilTag() call we either want a wrapper to {% else %}
    // or {% endif %}, whatever end tag comes first.
    // It's currently *not* allowed by WrapUntilTag() to have any arguments in the endtag
    // (like {% else another_condition %}.
    wrapper, err := doc.WrapUntilTag("else", "endif")
    if err != nil {
        return nil, err
    }
    if_node.thenWrapper = wrapper

    // If the first end tag being discovered was an else-tag, we now have to
    // wrap the else-Block.
    if wrapper.Endtag == "else" {
        // if there's an else in the if-statement, we need the else-Block as well
        wrapper, err = doc.WrapUntilTag("endif")
        if err != nil {
            return nil, err
        }

        if_node.elseWrapper = wrapper
    }

    // Okay, we now parsed everything. Return the if-struct the pongo2.
    return if_node, nil
}

As explained it’s required to provide an Execute() function for the struct returned to pongo2. This is the Execute() function for the if-tag (with verbose comments):

func (node *tagIfNode) Execute(ctx *ExecutionContext) (string, error) {
    // We're required to return a rendered template as string or an error.

    // ctx contains both the public context provided by the pongo2 user and 
    // the private context which allows filter/tag developers to exchange data
    // between tags/filters.

    // First, we're evaluating the given and parsed condition by passing 
    // the current context to it.
    // Evaluate() returns a *Value object.
    result, err := node.condition.Evaluate(ctx)
    if err != nil {
        return "", err
    }

    // pongo2's Value provides a general IsTrue() function which
    // behaves like Python's evaluation:
    // FALSE: empty string/map/slice/array, 0 (number), false (bool)
    // TRUE: non-empty string/map/slice/array, any number (except 0), true (bool)
    if result.IsTrue() {
        // If the condition evaluates to TRUE, we're executing the "then"-Block by
        // calling Execute(). Luckily, the NodeWrapper.Execute() function already
        // returns (string, error).
        return node.thenWrapper.Execute(ctx)
    } else {
        // If the condition evaluates to FALSE, we're executing the "else"-Block
        // (if provided)
        if node.elseWrapper != nil {
            return node.elseWrapper.Execute(ctx)
        }
    }

    // If the "then"-Block wasn't executed (because the condition evaluated to FALSE)
    // and there was no "else"-Block provided, we're just returning an empty string.
    return "", nil
}

Finally we’re registering our tag in the pongo2 system:

func init() {
    RegisterTag("if", tagIfParser)
}

And we’re done!

How to migrate from pongo to pongo2

pongo2 is almost API-compatible with pongo. That means for you an easy migration by firstly changing your import path from "github.com/flosch/pongo" to "github.com/flosch/pongo2" and replacing all pongo occurrences by pongo2.

  1. In pongo2 the TemplateLocator has been removed. There is no need to pass a 2nd argument to FromFile() and FromString() (old and new) anymore; simply remove it. All parent templates for template inheritance are now being located automatically (relative to their child’s template path).

  2. The filter time_format is gone and has been replaced by the filters date and time (they behave identical). Please be aware that pongo2 is using (like pongo) the Golang datetime format string (instead of the one provided by Django). See Go’s time documentation for more about that.

  3. If you’re calling functions from within a pongo template, make sure you’re migrating all functions to accept *Value objects as parameters and that these functions are returning one *Value object as well (see above for more about *Value objects).

  4. In pongo2 the extends-tag only accepts the parent’s filename (as string) as the first argument (no static or whatsoever anymore).

  5. ExecutRW is being replaced by the more general ExecuteWriter() which now takes an io.Writer.

  6. Execute() and ExecuteWriter() don’t take a pointer to a pongo2.Context anymore (instead it’s pongo2.Context directly now).

That’s it! BTW, you can always enable the debugging mode with pongo2.SetDebug(true) (anywhere at application startup).

Conclusion

I hope you will give pongo2 a try. It’s still in beta, but my website already uses pongo2 and it works like a charm. It’s currently missing some more documentation and examples, but I’m on it. There’s still a lot to develop (especially built-in filters/tags), but the foundation is made.

As always, I’m happy about your feedback and I appreciate any contributions.


New comment

Comments are moderated and therefore are not published instantly.





Comments

No comments yet. Be the first! :-)