If you torture the data long enough, it will confess.

There's a risk that when you do a test you can wind up testing for enough different things that, just by chance, something looks like its true even if it really isn't. And if you just collect data or manipulate data that's already been collected you can find patterns, even if there aren't any.

Preregistration is one way to deal with this. Before you start you "call your shot" and say what you are going to try and how you will interpret the results. And then you stick to your plan.