/usr/include is where all the things your 'OS provider' would put things either during the initial installation, or via whatever package manager you use to install other libraries that are supported up-stream.
/usr/local/include is where you would put things for anything you either compile yourself from source, or you obtain from other 3rd party sources.
If both are in your compiler search paths, it makes no difference.
Some Linux distros, like Ubuntu and Arch store their base system and additional packages in /usr. There is no /usr/local.
On BSD and many Unix systems, the "base system" is part of the OS. These are installed in /usr, but third party packages are installed in /usr/local so they're distinct.
Once upon a time, disk space was rare and expensive. /usr was often mounted from a larger host and shared. Thus the "base system" would be on /usr. Over time, workstations began to get larger local disks, and parts of it would get mounted under /usr/local. So /usr was remote and /usr/local was a local disk. Thus you could install whatever you wanted locally without affecting anyone else.
These days, you can get a 1Tb ssd for your phone. Disk space isn't rare, if not inexpensive. Most Linux systems install /usr locally, so there's no apparent need for /usr/local from that point of view.
It's worth noting that many 3rd party packages still set PREFIX=/usr/local by default, despite targetting Linux.
Libraries are probably packaged into two packages.
One has the *.so binary that dynamic linker seeks when an executable program loads.
Other (name has -dev ?) has the headers for the library that are needed when something is compiled and linked to use the library.