If you had a pointer and the it'll be used the entire time, is there any reason to delete it right before the program ends? If I recall, the OS does that?
Please do not rely on that, please, please. Delete everything you new.
Note: if you use something for the full duration of your program, store it on the stack.
What if your pointer stores VRAM memory (eg OpenGL buffer), will it be freed? Will you leak video ram on your system, will you leak memory on another system? Be correct to your users, and if it bothers you use smart pointers.
And this is a real problem in programming world. For example Delphi itself leaked Windows atoms, so in some time OS will stop working properly until reboot.
It is possible to leak OpenGL objects too. Which are shared across whole system. Only manual restart of OpenGL subsystem will help.
What if objects have a non-trivial destructor with side effects? Say, ifstream or network socket with non-flushed data? What if they manage system-wide resources?