I have several hundred of the same cast from a Lee mold. Cast bullets invariably come out in varying weights and diameter. Even your lube will change them up a little, not to mention your alloy, or casting techniques.
I size all of mine and separate by weight to maintain consistently.
However for plinking and practice 4 grains difference in bullet weight isn't that noticeable.
I have never met a firearm that couldn't outshoot the shooter.
I find it much harder to get the lead bullets aligned in the case for seating resulting in bad accuracy than the weight of each bullet.