You don't install a language. You install a compiler.
You can open up notepad and type in any language. English, French, Italian, C++, Java, whatever. You don't need anything installed.
What you do need installed is a compiler. That is... something that reads your C++ text and translates it into a binary executable.
Visual Studio is a compiler. If you have VS installed, you already have everything you need installed and you will be able to write and compile C++ code.
If you want to put it on a thumb drive... then install Visual Studio to your thumb drive.