So every time I want to use certain header files in a project, I have to add them to that particular project?
I thought the way it worked was there's a directory full of header files, then when you declare them in "#include" the compiler will just search for the header file and execute.
For standard C and C++ headers, you can just use #include and use them as-is. However, with your own headers for use with the project, they have to be somewhere that the compiler will find them (generally somewhere relative to the root directory of your project / makefile).
As for using libraries, it depends. Some people seem to prefer to copy the headers directly into their compiler's include directories, in which case they can be used simply with a #include. However, most of the time it makes more sense to leave them in separate locations on your disk for organisation reasons, which means that you'll need to add the location of the header files to your project yourself.