It turns out that calling Measure() on a child control inside ArrangeOverride() can cause an infinite layout invalidation loop. I created my own Measure() method in the child controls which I call from MeasureOverride() and from ArrangeOverride() in the scorePanel. Extra layout loops went away.

Line Forward/back working. Page forward/back and Start/End should be easy.