In the graph editor, right click the channel (eg. translate Y) and click Channels>Channel properties..
Here you can change the In and Out to whichever infinity-behaviour you'd like.
Something super simple and dumb that I'm surprised I've never used beforehand!
To keep a 3d noise value from,say, a point-vop sticking to Geometry, make a rest-position value before the object is meant to start animating. There is a handy Rest SOP for this.