Confused on when to use virtual destructor

[bubbleshere]

I read things that are conflicting with one another. Many say only use virtual destructors when you use virtual functions or polymorphic class. Then I read classes are only polymorphic when they have virtual functions. But I read something else that you can declare virtual destructor if you have a derived class that may delete via a base pointer.

What is the proper thing to do without declaring virtual everywhere?

If I have a base class without virtual functions, should I only declare a virtual destructor if there is a delete via a base pointer or should I never use it since there is no virtual functions? I need the code to be as efficient as possible.

[JLBorges]

In general, for a class that is designed to be at the top of an inheritance hierarchy, provide a public user-declared virtual destructor.

Once a base class declares its destructor as virtual, it is not necessary that a derived class must also have a user-declared virtual destructor.

Note: This general rule should suffice for now.

This is for later:
A user-declared virtual destructor is not essential if there is a certainty that no object of a derived class is dynamically allocated other than in conjunction with std::shared_ptr. In addition, the destructor of a polymorphic base class can be protected and non-virual.

More information: See rule of zero / rule of five defaults at the bottom of this page
http://en.cppreference.com/w/cpp/language/rule_of_three