If something like pointer arithmetic is done, it can be right or wrong - hence I don't think it should be restricted.
Following what I said above, dangerous features should not be restricted, they should be made cumbersome. The path of least resistance should be safe, and it should take effort to go the dangerous route.
Pointer arithmetic is plainly a mistake. That T * has no size component and that it can be arbitrarily added to to obtain another T * is just looking for trouble. The type of a single pointer and the type of a pointer to an array should be entirely unrelated. I would imagine it like