A 15/16" spade bit (wood bit) in a drill press makes a perfect hole in aluminum panels. Clamp the panel between two pieces of MDF. The hole in the top piece of MDF acts as a guide to keep the bit from wandering. Line up the point of the bit with a pilot hole, clamp the panel, then position the predrilled MDF and clamp it. A drop or two of dish washing liquid makes a good lubricant. The cutting is done by the little points at the end of the bit, so if the material is too thick, it won't work. Home Depot sells individual spade bits. Cheaper than a punch, cleaner than a cheap step bit (though they work fine, too).
Drop a connector into the hole, secure it with tape, then use the screw holes as guides for drilling the mounting holes. Measuring never works for me...