Moldy Board: Please enlighten me!

I placed two Timber Piles outside my base, each with the same 6 tiles distance to other objects (I went for 6 to be save).
But, one pile was fine and the other one consisted only of Moldy Boards.
The only explantation I have is that I walked several times close to the "Moldy Board" Timber Pile with a Sled packed with structures (mostly Wooden Boxes, but also boulders and other stuff). So would a moving sled count as a structure that demolishes my Timber Pile boards? If so, I suspect the same applies to carrying a structure on your back in a 5 tile radius.
Is that really so?
But, one pile was fine and the other one consisted only of Moldy Boards.
The only explantation I have is that I walked several times close to the "Moldy Board" Timber Pile with a Sled packed with structures (mostly Wooden Boxes, but also boulders and other stuff). So would a moving sled count as a structure that demolishes my Timber Pile boards? If so, I suspect the same applies to carrying a structure on your back in a 5 tile radius.
Is that really so?