Just to add something to Andy's input. I think of what gun I will be shooting it in. A plastic gun will take less powder to cycle than say an all metal Sig normally. If I am loading for poly gun I will start lower than for a Sig 220 for example.
Also, is there leftover powder or excess residue? Do you tumble brass and w what media? Could your flash hole be plugged with media?