Integrate Twig into core: Implementation issue

Suggested commit message: Issue #1696786 by Fabianx, stevector, jenlampton, jwilson3, chx, Antoine Lafontaine, steveoliver, amateescu: Integrate Twig into core.

Part of meta-issues

#1788918: [GTD] [META] Prepare for Twig main engine core inclusion until December 1 →

Changes

1. twig.engine
2. the necessary glue code to make the render arrays work with Twig
3. node.twig and datetime.twig as examples of how things will look
4. temporary possibility for modules to provide .twig and .tpl.php to be removed once all .tpl.php in core modules are gone (merged from #1697854: Allow modules to provide both .twig and .tpl.php templates temporarily until twig is the default engine → )
5. extensive testing of the above
6. a bug fix and tests for drupal_find_theme_templates to find suggestions also for files with just one extension (merged from #1678808: Defining a new theme engine with templates without .tpl.[template extension] breaks template suggestions → )

Software Benefits

1. Security. Right now, perhaps aside from a rounding error, all custom themes are ridden with XSS holes. Most template-writing people are not PHP security savvy. Taking away PHP from them is a good thing. We will have autoescape on, check_plain()ing everything printed. This is obviously not complete or correct security in all cases but it's a world of a difference to the situation now where we hand a loaded gun to themers and tell them to hammer a nail with it. Note: the code to have autoescape on already exists in the sandbox.
2. Performance. Compiled classes instead of repeated includes of template files. . The patch authors benchmarked the patch and no difference is observed in the best case, even with compilation cache. See #1696786-100: Integrate Twig into core: Implementation issue →
3. Slightly cleaner templatesTemplate authors no longer call print function. Calls to hide/render/t are still required, as are if() statements.
4. Safety. Template authors can't use raw PHP so less trusted parties could contribute template changes ((e.g. Drupal Gardens site owners). Assumes you could teach and support these folks to author in Twig.

Performance regression and benefits follow-ups

#1825952: Turn on twig autoescape by default →
#1716048: Do not boot bundle classes on every request →
#1696786-160: Integrate Twig into core: Implementation issue →
#1815250: Type twig variables in PHP and remove superfluous render_var() and getAttribute() calls from the generated code →
#1778610: Remove the check for a link template from l(), have l() always output just a string. →

Community Benefits

1. Wide adoption. It's not just that it's used for Symfony -- it's actually used outside of PHP, even: Liquid for Ruby, Jinja2 and Django in Python. There's a TwigJS too.
2. The community stands behind it #1499460: [meta] New theme system →
3. New light syntax. Template authors do not have to know PHP now. Preprocess still requires PHP

Downside

1. Code complexity. This patch's reference workaround is clever but brain splitting. See TwigReference .
2. Code weight. The current patch (without caching or autoescape) is 40K which is not too bad, but the Twig engine, adds 385kb (based on #1591686-9: Add Twig itself → ).
3. Security benefit and complexity are undemonstrated. The patch currently disables disables autoescape. It is not clear how we can check_plain all variables indiscriminately. Clearly a lot of data has already been sanitized before template layer.
4. New syntax. Themers already have to know PHP in order to use preprocess system. They now have to learn Twig syntax.
5. IDE support needs plugins. PHP already has syntax highlight and syntax check in every IDE and text editor but Twig has support only with plugins. (http://twig.sensiolabs.org/doc/templates.html#ides-integration)

Follow up issues

• #1777532: Extend Twig_Loader_Filesystem instance to get .twig templates paths with hierarchy for theme dependencies →
• #1806546: Twig: Performance Testing / Enable twig cache by using the Drupal PHP Loading/Writing → from #1696786-101: Integrate Twig into core: Implementation issue →
• #1712444: Twig: Activate autoescape mode → from #1696786-133: Integrate Twig into core: Implementation issue →
• #1806538: Remove @todo about enabling strict_variables in Twig →
• #1815250: Type twig variables in PHP and remove superfluous render_var() and getAttribute() calls from the generated code →
• #1806524: Remove id selector from node.twig as it can be non-unique →
• #1806478: Make twig the default engine once all modules templates are converted from .tpl.php to .html.twig →
• #1805592: Remove static double theme engine code once all module provided .tpl.php files are converted to twig and twig is default →

Documentation for Twig Coding standards: http://drupal.org/node/1823416