Include hierarchy when sorting by term weight

I wanted to sort taxonomy terms used in a facet by hand in a project, so I decided to use sorting by term weight for the purpose.

But the vocabulary in question has two levels, where only second level (L2) terms was used for facets, like this:

L1: T1
--> L2: T4
--> L2: T5
L1: T2
--> L2: T6
--> L2: T7

and so on. Now Drupal is writing the weights so that L2 terms begin always from 0. That leads to the fact that terms are sorted like T4, T6, T5, T7, which made no sense for the case.

What if I wanted terms sorted by their parent first, only then by their own weight? Or by hand regardless of their parent?

I guess the latter question is more difficult, but if it was enough to just take the parent weight into account, it would be fairly easy.