While 100V line stuff exists, in the USA 70V is more common. Regardless of line voltage for stereo each "B" box would need four transformers ($5 each in USA), plus a pair of stereo L-pads ($13 each in USA), not to mention box, switches, and connectors. Interesting idea, maybe not the most cost-effective, certainly doable. Much cheaper if stereo is not needed!
Regarding Voyager10's other suggestions, I would say I agree completely, although a star topology may be impractical. From an efficiency viewpoint it would be better to use a 48V feed to minimize supply current, as well as Cat-6 cable, which has lower resistive loss than Cat-5. Each "B" box would then use an isolated step-down supply that allows for using a common high-efficiency class-d (stereo?) headphone amp for each output. In that case the use of higher impedance headphones is not required, so any cheap earbud would work.
Regarding Voyager10's other suggestions, I would say I agree completely, although a star topology may be impractical. From an efficiency viewpoint it would be better to use a 48V feed to minimize supply current, as well as Cat-6 cable, which has lower resistive loss than Cat-5. Each "B" box would then use an isolated step-down supply that allows for using a common high-efficiency class-d (stereo?) headphone amp for each output. In that case the use of higher impedance headphones is not required, so any cheap earbud would work.