E. Cobham Brewer 18101897. Dictionary of Phrase and Fable. 1898.
A bit of my mind, as Ill tell him a bit of my mind, Ill reprove him. Same word as bite, meaning a piece bitten off, hence a piece generally. (Anglo-Saxon, bitan, to bite.)
Bit by bit. A little at a time; piece-meal.
Not a bit, or Not the least bit. Not at all; not the least likely. This may be not a morsel, or not a doit, rap, or sou. Bit used to be a small Jamaica coin. We still talk of a threepenny-bit. Bit, of course, is the substantive of bite, as morsel (French morçeau) of mordre.