"Historically speaking, there were four steps in the development of today's concept of the derivative, which I list here in chronological order. The derivative was first used; it was then discovered; it was then explored and developed; and it was finally defined. That is, examples of what we now recognize as derivatives first were used on an ad hoc basis in solving particular problems; then the general concept lying behind these uses was identified (as part of the invention of the calculus); then many properties of the derivative were explained and developed in applications both to mathematics and to physics; and finally, a rigorous definition was given and the concept of derivative was embedded in a rigorous theory.
[...]
"The span of time from Fermat to Weierstrass is over two hundred years. How did the concept of derivative develop? Fermat implicitly used it; Newton and Liebniz discovered it; Taylor, Euler, Maclaurin developed it; Lagrange named and characterized it; and only at the end of this long period of development did Cauchy and Weierstrass define it. This is certainly a complete reversal of the usual order of textbook exposition in mathematics, where one starts with a definition, then explores some results, and only then suggests applications.
[...]
"We need to remember that the rigorous definition is often the end, rather than the beginning, of a subject."
-- Judith V. Grabiner, "The Changing Concept of Change: The Derivative from Fermat to Weierstrass", April 1982 [I think this article might be interesting even to some people who don't know calculus, as long as you can understand the general shapes of the problems each generation of mathematicians was trying to solve ... but some parts will make much more sense if you're at least acquainted with the delta-epsilon construction calculus is often introduced with.]