Best to just not include it in the first place if you do not require it. And it depends on what the header contains but you can expect a bigger file. If a dynamic link library is included it will slow down the program.
But if I just want to use a very simple function in the header
Well both really because regardless function you are trying to include it is going to perform some type of calculation or process (or you wouldn't be using it in the first place).
I wouldn't worry about that at this stage, processors are really fast. When you have a huge program and you want to trim the fat. that is when you can start to worry about it.
#including a header will not slow down your program or enlarge it.
It will, however, increase the compilation time as there is more code in the form of declarations/definitions to process. There may also be some additional time spent in the preprocessing phase of the compilation process.