We present an algebraic framework to study the time-optimal synthesis of arbitrary unitaries in SU(2), when the control set is restricted to rotations around two non-parallel axes in the Bloch sphere. Our method bypasses commonly used control-theoretical techniques and easily imposes necessary conditions on time-optimal sequences. In a straightforward fashion, we prove that time-optimal sequences are solely parametrized by three rotation angles and derive general bounds on those angles as a function of the relative rotation speed of each control and the angle between the axes. Results are substantially different whether both clockwise and counterclockwise rotations about the given axes are allowed, or only clockwise rotations. In the first case, we prove that any finite time-optimal sequence is composed at most of five control concatenations, while for the more restrictive case, we present scaling laws on the maximum length of any finite time-optimal sequence. The bounds we find for both cases are stricter than previously published ones and severely constrain the structure of time-optimal sequences, allowing for an efficient numerical search of the time-optimal solution. Our results can be used to find the time-optimal evolution of qubit systems under the action of the considered control set and thus potentially increase the number of realizable unitaries before decoherence.