E. Cobham Brewer 18101897. Dictionary of Phrase and Fable. 1898.
Fiddle-de-dee!
An exclamation signifying what you say is nonsense or moonshine. Fiddle-de-dee is meant to express the sound of a fiddle-string vocalised. Hence sound signifying nothing.