Open up Visual Studio Installer, click modify and under "individual components" into search box type "cmake"
Check the "C++ Cmake tools for Windows" and install it.
I don't know if this let's you debug cmake files, debugging cmake files is rather strange because cmake isn't a programming language, however this component provides you with syntax highlighting and with cmake.exe to execute your cmake.txt files.
Of course that doesn't debug the CMake file itself, it modifies whether a release or debug session is launched from an executable built using CMake.
It's amazing the information the internet has available when doing a 'net search instead of relying on others for answering questions. Hint, here's the DDG search I did.