This module uses an SQL query to load all terms. But it loads way too much terms because it always loads all translations and then overwrites the result array multiple times.
Later the the term gets fully loaded just to load the translation which was already in the SQL result. That's a performance issue.
At one place the term URL is used to build a link. This URL always points to the terms default translation instead of its translation which is used for the labels. That's a bug.
Limit the SQL results to the current and the default language instead of all languages if the locale module is active. In our case with 15 languages and big vocabularies that reduces the result set massively.
Avoid loading term entities just to translate their labels. The labels could be easily loaded via SQL where the untranslated labels are already taken from.
2.0
Code
I concur, the performance gain is impressive (1 to 2 seconds saved on a 4 to 5 seconds load time in our case). Thanks for the patch!