The best way to minimize effects on the signal is to have a high impedance meter interface, and balanced impedance to ground (maintain CMRR of balanced signals).
I just got delivery of some meter cards. Each set is a pair of small PCB's that mate together, and will fit in a 1u case vertically (8 pairs of them would fit no problem.).
One board is a board that holds 8 led's and 2 LM339 comparators and a resistor network to set the indicator levels (so you can do any scale you want.)
The other is a board that has a balanced line receiver circuit (with precisely balanced input impedance based upon figure 4 in this article
http://waltjung.org/PDFs/Op_Amps_in_Line_Driver_and_Receiver_Circuits_P2.pdf) it can run at very high impedance.
Board 2 has adjustable meter dynamics (you can set the rise time and fall time separately). This works for both mechanical meters or LED arrays, so you can make them act like VU or PPM (if the mechanical meter is fast enough). This works surprisingly well making cheap meters like the ones in the SM Audio unit)
Board 2 can drive board 1, or it can drive an actual meter.
Board 2 can support an LED peak/clip indicator, and has clip hold functionality/reset funtionality.
The boards can be daisy chained using ribon cable for power, voltage reference, clip reset/hold, so they would be easy to wire up.
This is the second revision of the cards, just got them the other day so they are untested as yet.
You could make better version of the SM Audio unit with 8 board 2's, and some of the meters they use (ebay $7.50 in small volume). That plus some TL074's and a few transistors and caps.