Agree with the wikis on decoupling/coupling---always the way I understood the terms, with in fact the greater emphasis for "decoupling" on reducing interaction among subcircuits via the power rails.
Some mention of a series R or L-R between the power rail and the shunt cap would be helpful---but then I guess the wiki would be more properly titled "decoupling network".
As far as the noise pickup issue, besides the electric field interception, I've had situations where the simple fact of the larger loop area of the coupling capacitor and nearby plane or trace of signal return, or the other capacitor in a balanced arrangement, did me in. This was on the output (!) of a crossover system where the caps used were large copper foil/paper/oil types, and the charging current spikes were modulating the stray field of a nearby big toroid.
It seemed, in principle at least, that a compensating pickup coil could be arranged so as to cancel the fields. Better was just to move the transformer away, but this meant a bigger box or two-chassis system.
The discussion in the other forum about magnetic shielding is correct but somewhat incomplete, as it doesn't emphasize the importance of having a complete magnetic circuit for the shield. Just substituting ferromagnetic material in place of non-, where the latter was effective for electric fields, may well make the mag field pickup problem worse.
Sometimes you can contrive a given shape and size of such material to shape the field and get improvement though, as long as the interfering source is fixed in position. This won't help for magnetic radiation from other units in proximity though.