E. Cobham Brewer 18101897. Dictionary of Phrase and Fable. 1898.
En passant, laterally connected with the main subject. By-play is side or secondary play; By-lanes and streets are those which branch out of the main thoroughfare. The first by means passing from one to another, as in the phrase Day by day. Thus By-the-by is passing from the main subject to a by or secondary one.