Give group permission to specific users without assigning a role

Problem/Motivation

I need to give group's(say Group A) full permission(like administer group) to a specific set of users. These users are members of a different group (say Group B).Both groups are of same type.

I need to give access to group A ,without assigning a new role for them in the group A as well in the site. User cannot be the members of Group A where access required. They need to remain as outsider.

Is there a way to do this in group by applying a patch.