The idea is to minimize loop area to minimize magnetic field pickup and magnetic radiation. It is far more difficult to shield conductors against magnetic fields than against electric fields, so minimizing area is the best strategy.
To get a feeling for the magnitude of these effects, take some power amp you have, with an ordinary single-ended input, hooked up to a speaker, and connect a loop of wire to the input between the hot and common. With enough gain you should hear plenty of mains hum in most any environment these days, with most of it probably radiated by the amp's power transformer. Move the loop around in different orientations and note the change in level and character of the sound.
Now make the loop area small by straightening out the length to make most of it two parallel closely spaced pieces. Note the reduction in hum. Now twist the conductors together. Note the additional reduction.
Note that, at audio frequencies at least, you have been shorting the hot to common all this time, as the wire resistance and inductance for a relatively short piece is small. If you were to cut the loop so to have a piece of wire attached to each of hot and common, you would have two electric field antennas, with the one tied to common having little effect other than a bit of partial shielding depending on its proximity to the hot wire. But a magnetic field can no longer couple to the wires, so the noise you will hear now is electric field induced. Note the change in the character to emphasize higher frequencies, especially in typical terrestrial environments with lots of higher frequency noise sources like fluorescents and switching power supplies. Your wires are in essence looking like plates of capacitors and are receiving electric fields with a sensititvity rising at about 6dB/octave. Now, if you place the two wires close together or twist them, you will get a slight reduction in electric field pickup as mentioned by Gus due to the partial shielding. But the single-ended input will spoil most of this improvement because there is no effective subtraction of the hot noise signal and the common one.
If you have a piece of equipment with balanced inputs, try the experiment above with it and note the generally higher immunity to electrical fields, since now, with equal impedances to common from hot+ and hot-, if each wire intercepts the same electric noise field the induced signals will cancel. But the loop of wire will still very effectively pick up magnetic fields since the induced signal is now differential.
Note that the capacitance between the two wires has gone up with the twisting. If inside an amp it is desired to minimize the capacitance then twisting a run of conductor with a common wire may not be good. For typical amps the signals internally are usually single-ended as as well. But for tube amps with ac filament supplies there's a lot of current at mains frequencies running around, and in this case the best strategy is to minimize the magnetic field emissions---the complementary problem that happily behaves the same way.
The idea is to minimize loop area.