Primer is designed to bond with unpainted surfaces like metal or plastic - base colors are not.
Primer is needed when you are painting a bare surface like metal, plastic, carbon fiber, polycarbonate, etc. Without primer, your paint will not hold or last. If your repair calls for a filler then you will need to apply primer before applying the paint.
If you are unsure whether you need it you are better off using it. Adding primer when it is not needed will not negatively affect your finish.