a 1 lb bumbell up and down faster with the same amount of power(your arm) than you can a 10 lb dumbell. It's inertia, a heavier mass cone will not change directions as quickly as a lighter one, same concept as a lighter car being able to change direction more quickly than a heavy one, all things being equal.
Of coures, this depends on mass, not diameter. You could have a 10" with a super heavy cone/spider/voicoil assembly that would have more mass than a more lightly built 15".....