E. Cobham Brewer 18101897. Dictionary of Phrase and Fable. 1898.
Penny Saved (A).
A penny saved is twopence gained. In French, Un centime épargné en vaut deux.
Well, suppose a man asks twopence apiece for his oranges, and a haggler obtains hundred at a penny apiece, would he save 200 pence by his bargain? If so, let him go on spending, and he will soon become a millionaire. Or suppose, instead of paying £1,000 for a bad bet, I had not wagered any money at all, would this have been worth £2,000 to me?