E. Cobham Brewer 18101897. Dictionary of Phrase and Fable. 1898.
Drive at (To).
What are you driving at? What do you want to prove? What do you want me to infer? We say the wind drove against the sails, i.e. rushed or moved violently against them. Falstaff tells us of four rogues in buckram [who] let drive at him, where at means against or towards. What are you driving at? is, against or towards what object are you driving or moving?