If I have a tube preamp and want to measure the overall gain I have read that I send a 1kHz tone to the input... connect the ch1 probe to the same spot and connect the ch2 probe to the output and measure the ratio between the two.

Then I should have number for calculating the gain (dBu???)... but I also have read that when calculating dBm I need the Resistance. But I do not know if it is the DCR of the input transformer or what?

Gain is unitless. If the output is 10 V when you feed in 1 V, then the gain is simply 10. Gain is a ratio of output to input. Gain of 1 is unity. A gain between 0 and 1 is attenuation (output less than input). A negative gain implies a polarity reversal.

Now, it is common to report gain in decibels, or dB, which is logarithmic. dB without a suffix (dBm, dBu, whatever) is unitless. Remember that gain is a ratio, and so too is gain in decibels. The formula to calculate gain in decibels is simple.

Gain (dB) = 20 log (Vout / Vin)

This is independent of impedance, termination, etc.

What is the difference between gain in dB and a number like +4 dBu? The latter isn't a gain, it's another way of expressing a voltage. Hidden in that expression, though, is that the number is "voltage with respect to some reference." And the reference is given as 0 dBu = 0.7746 volts rms and I suppose you can look up the derivation of that constant if it's interesting to you, and also the difference between dBu and dBm. The point, though, is that you never refer to gain in dBm or dBu or whatever, only dB.

So in the case of the thing you are trying to measure:

Put one 'scope probe on XLR Pin 2 of the output (and put the probe ground on the chassis), put the other probe on XLR Pin 2 of the input, measure the peak voltage of both, and do the math above, and that's your gain.

-a