E. Cobham Brewer 18101897. Dictionary of Phrase and Fable. 1898.
Children (a word of contempt). Get away, you young fry. It means properly a crowd of young fishes, and its application to children should be limited to those that obstruct your path, crowd about you, or stand in your way. (French, frai, spawn.)
Nothing to fry with (French). Nothing to eat; nothing to live on. (See WIDE-NOSTRILS.)