Feature: Use Jinja's new NativeEnvironment to get native types instead of strings!

[cognifloyd]

It's really hard to deal with magically converting from Jinja strings back into native python types. Let's just skip the conversion and use native python types.

How you ask? Jinja has merged a pretty cool PR: pallets/jinja#708
Here's a quote from the new doc:

The default jinja2.Environment renders templates to strings. With NativeEnvironment, rendering a template produces a native Python type.

This consists of three basic parts: NativeCodeGenerator, NativeTemplate, and NativeEnvironment. Check out the code. This will land in Jinja 2.10.

This comes from an ansible PR by @jctanner: ansible/ansible#23943

I suspect that using NativeEnvironment would resolve (or make a big dent in) these:

• Preserve type in ActionChain publish variables #1231 Preserve type in ActionChain published vars
• Support for complex types (lists, dictionaries, integers) for dynamic config values #2976 Complex types (other than string) in dynamic config
• for dynamic parameters (multi-type) or action-chain publishing allow explicit casting of jinja variables #3515 allow explicit casting of Jinja types
• Add extra_vars parsing directives due to Jinja StackStorm-Exchange/stackstorm-ansible#14 Parsing directives !JSON, !AST, and !INT that convert strings (like from Jinja) to python types.
• Plus, maybe it would be one component in allowing complex (ie non-string) data types in the datastore.

[Kami]

Thanks for letting us know about this.

It seems like a great overall feature which would help in a lot of scenarios. Having said that, the change will probably be quite big and we also need to research it to make sure it's actually a good for our use case (need to make sure it works across Python versions and not like pickle, need to research safety of this feature, etc).

Also, #2976 is not related to Jinja, but to the way we store and serialize values in the datastore.

[cognifloyd]

🎉 pallets/jinja#708 (comment)

[@davidism] just released Jinja 2.10 with this included.

https://pypi.python.org/pypi/Jinja2/2.10
http://jinja.pocoo.org/docs/2.10/nativetypes/

[nmaludy]

#3820 might also be related and/or fixed using this feature

[cognifloyd]

And here's how simple it is for Ansible to switch over to using NativeEnvironment:
ansible/ansible#32738
And here is his test playbook for using it:
https://gist.github.com/jctanner/32df50fe270096afc7fe2582e2d21b8c

[cognifloyd]

@Mierdin said this in #3909

This kicked off a long discussion internally, and we've decided to go for a more robust, exhaustive method of dealing with more complicated data types, likely to be implemented in the 2.6 release timeframe, perhaps 2.7.

That effort could be related to (or perhaps made easier with) this feature.

[cognifloyd]

This is a major-ish change - Maybe this can go in 3.0? Or make its way onto the roadmap? (Please)

[smemsh]

think this is what's implemented and merged in #32738, looks like it was relased in ansible v2.5