Initial Observations
One of the first observations was that there what look to be harmonic relationships within the data. In these areas the mean pixel difference between frames jumps back and forth between two fairly constant arcs. It turns out these areas are pans, which are qualitatively different than areas where an object moves within the frame.
In this example the blue lines show the start of movement within the frame, and the red lines show a start of movement of the frame itself. I have yet to determine the cause of the distance between the two values. Probably the speed of the pan.