Something super simple and dumb that I'm surprised I've never used beforehand!
To keep a 3d noise value from,say, a point-vop sticking to Geometry, make a rest-position value before the object is meant to start animating. There is a handy Rest SOP for this.