There are two things that come into play here-
The risk of a primer being hit and a round going off & the characteristic of powder changing when being tumbled.
I have dropped full containers of loaded ammo and had loaded rounds bouncing all over my concrete floor. They hit with 10 times the force that any bullet moving around in a dry tumbler. Could it happen? Maybe, but I find the odds to be astronomical for a primer to be hit hard enough to fire.
At one time there was a study done on a different website, where they tumbled rounds loaded with 4 or 5 different powders for a long period of time, weeks. They then showed before and after views of each powder under a microscope and there was no difference. So that myth was busted.
But every time I have seen this question posed on forums and boards, the answer has been "it's your decision to make."
One hint I can tell you is that corn cob media works much better as removing case lube than walnut media does.