Getting started using Catch2 in Visual Studio

[frek]

Hi all,
I'm using Visual Studio 2022 (version 17.1.1-community version) on my Windows 10 x64 machine and have just recently installed Catch2 for my IDE. I'm about to use that unit testing tool for the first time. Considering the following simple code I wish to use Catch2 for that purpose.
My purpose is to know first: how Catch2 works and second: how beneficial it is (since there's much about that being quite profitable and a must for good developers).

I wrote it this way:

#define CATCH_CONFIG_MAIN
#include <catch.hpp>

unsigned int Factorial(unsigned int number) {
	return number <= 1 ? number : Factorial(number - 1) * number;
}

TEST_CASE("Factorials are computed", "[factorial]") {
	REQUIRE(Factorial(0) == 1);
	REQUIRE(Factorial(1) == 1);
	REQUIRE(Factorial(2) == 2);
	REQUIRE(Factorial(3) == 6);
	REQUIRE(Factorial(10) == 3628800);
}

int main() {
	std::cout << Factorial(6) << '\n';

	return 0;
}

Two odd things:
1- We've got an output, 720, without having <iostream> included! Any reason?
2- When the project is run normally (by pressing F5), it runs as if there's no unit test. That is, nothing additional is shown as the output of Catch2.

[lastchance]

I fail to see any advantages whatsoever over just using assert().

[George P]

@lastchance, I've played around a bit with the example code samples included with the library. It makes the output look more colorful, literally, as well as violating more than a handful of C++ standards practices.

FYI, I installed the library yesterday, had never even heard of it before then.

#define CATCH_CONFIG_MAIN tells the framework to generate a main function.

Defining that macro AND having a regular main function causes linker problems. Visual Studio 2022 doesn't like the above code.

LINK : warning LNK4067: ambiguous entry point; selected 'mainCRTStartup'

So the VS linker is given a choice of TWO mains and ends-up default using the normal C++ main instead of the framework's main. All that purty unit-testing gobbledygook script is then ignored and might as well not be left in the code.

The library code examples, all of them, NEVER have a regularly defined main. main is smushed into the code by the framework.

I figured this out just by compiling the code, reading the compiler/linker output, as well as bothering to read the library's documentation and scanning the examples.

So where is #include <iostream> ? Buried deep in that catch.hpp header that also includes a handful of other C/C++ headers.

Easy to find by searching the header. If one bothers to be pro-active enough to actually look.

To be honest, from what I've seen by poking and mucking around this unit-testing library looks less like C++, it certainly isn't remotely standard C++, and more like a half-arsed scripting language framework leveraging C++ so it gets compiled.

The sole "advantage" is the testing works the same whether the code is compiled in either Debug and Release mode. Is that worth coding in IMO what is a VERY non-standard C++ manner?

Not for me. YMMV.

[frek]

After removing the main function and running the project it showed an error caught by the first REQUIRE, in a colored console page, so I changed the code this way and it showed a successful page this time.
return number <= 1 ? 1 : Factorial(number - 1) * number;

This determines that we have to be very clever in selecting exact REQUIREs for the project's testing. We can't rely on the tool for that cleverness (something I wished before dealing with unit tests, say, catch2).

But the question is, if the tool is that dummy that can't do anything tangibly helpful except performing a set of REQUIREs (something we could achieve using std::asserts) why to bother toying with it?

To me up to now, the sole (so-called) advantage of using unit tests, especially the current one, is some illusion of having a perfectly written code for the project at the end.

[MikeyBoy]

To me up to now, the sole (so-called) advantage of using unit tests, especially the current one, is some illusion of having a perfectly written code for the project at the end.

The advantage of unit tests is that they help you test small parts ("units") of your code in isolation, so that you can check that those parts are correctly providing the functionality they're supposed to provide to other parts of the code.

A good unit test framework provides features (usually implemented as macros) that help clearly express what things are being tested for, and helps you write unit tests that can be easily implemented, built and run, and that provide useful, well-formatted output when things fail.

In a large project, these things are extremely helpful. They're not some magical panacea that will make your code run bug-free, and there's certainly other sorts of testing you should be doing as well as unit tests, but it's pretty standard these days in the industry to use them.

Catch2 is about as useful as any unit test framework I've used, and unlike many others, allows you to express your test conditions as regular C++ conditions. If you don't like this particular framework, it's worth experimenting with some of the other well-used ones, like GoogleTest and CppUnit.

If you want to know more about what unit tests are and why people think they're useful, a simple Google search will throw up lots of information.

[frek]

Thank you MikeyBoy for your info. Actually your comment in another thread about Catch2 had to some degree impact on my decision to go for that unit test. I've read about that on the Web good enough to be interested and wish to achieve what's being talked about it here and there.
But act is more reliable than words. Let's get the example I sent here in this thread in the first post.
what does Catch2 do about that project - except for testing a few macros to see they return true or false (or something like that) that could equivalently be done through that number of asserts instead?

[MikeyBoy]

What that does in the code you've posted, is to generate a unit test that tests the Factorial function in isolation from the rest of your program.

Of course, in this case, there is no "rest of the program" - there's just that one function. But I assume you understand that this is a trivial example to test that you've set up Catch2 properly, not an example of a real-world project that one would actually write unit tests for, yes?

[George P]

MikeyBoy wrote:

...is to generate a unit test that tests the Factorial function in isolation from the rest of your program. (emphasis added)

*checks assumptions, busted.*

That is for me a very succinct explanation of how unit-testing works, and the why of it. I was very much mistaken on the concept of unit-testing, and the structure of how to craft test units.

I hadn't played around with any unit-test library before a couple of days ago, so I am learning as I go along.

Two aspects of programming I have shunned before, using the debugger as a matter of routine and unit-testing. Maybe I should take some time to learn them, just as much time and effort I expend to learn previously unknown features of C++.

[frek]

@MikeyBoy
So in that project it's doing what asserts can do. OK, let's consider another example:

#include <iostream>
#include <vector>

int solution(std::vector<int>& A) {
  auto N = A.size() + 1;

  return N * (N + 1) / 2 -
   std::accumulate(A.begin(), A.end(), 0);
}

int main() {

std::vector A{ 1, 3, 1, 4, 2, 3, 5, 4 };
std::cout << solution(A) << '\n';

return 0;
}

How to use Catch2 and see its benefits for this project, please?

[MikeyBoy]

How to use Catch2 and see its benefits for this project, please?

You clearly haven't understood what I said in my previous replies to you. That project is no different from your last one, as regards the value of writing unit tests. Why would you think it was different?

As I said in one of your previous threads, there is lots of stuff written online about where and how unit tests can be useful. Wouldn't your time be better spent making the effort to do a bit of reading about it?

I have to echo what George P told you in another thread: you really need to be a bit more self-sufficient, and start showing a bit more initiative. It's trivially easy to search for information online, and you will never be able to become a competend programmer without being able to find, read and digest information on your own.

It seems insane to me that you would go to all the effort of choosing a unit testing framework, and figuring out how to install it and integrate it with your IDE, without first having done some reading about unit testing in the first place.

I mean, why would you go to all that effort, if you didn't even know why you wanted to do unit testing in the first place?

[George P]

@MikeyBoy,

I can only speak for myself, but I went to all the effort of using vcpkg to get the Catch2 library was:

1. To learn myself how to use vcpkg. I had tried and tried before and ended up never getting the thing (vcpkg) to work that I could see.

2. So I could help someone with HARD-WON knowledge of how to do the task because I had sweated through it myself.

I can't say I am an expert on Catch2 or any unit-testing protocol at all. I'm very much a beginning noob at it, I must admit.

I understand (somewhat) the theory of unit-testing. Because I am a self-taught hobbyist I have not yet had reasons to do more than "gee, this is something I probably should look at and learn" until now. The code I usually write is really simplistic and rudimentary.

I am very curious and mostly insatiable about learning new things ON MY OWN after a bit of a "look over there" nudge from others here at CPlusPlus.

What you've mentioned here in this thread does give me a bit of a *light bulb over one's head!* dim flash of insight into why unit-testing can be useful and important.

Thank you. :)

[George P]

Something interesting about Visual Studio and unit-testing.....

Boost Test and Google Test are two individual (and optional) components available through the VS installer. Though the actual source files, headers and libraries need to be installed as well. What VS installs are a template code snippet.

Catch2 is best installed through the vcpkg app for complete integration with VS.

There are 1,859 libraries available via vcpkg, what appears to be 11 different unit-testing libraries. Including Boost-Test and gtest.

[wqking]

1- We've got an output, 720, without having <iostream> included! Any reason?

Maybe CATCH includes iostream already.

2- When the project is run normally (by pressing F5), it runs as if there's no unit test. That is, nothing additional is shown as the output of Catch2.

After you define CATCH_CONFIG_MAIN, you shouldn't write you own main function.

The simplest way to integrate CATCH to your project is to download its single header and drop it to your projects.
I've used CATCH2 in several of my open source libraries, such as my eventpp library. I won't look at other unit test frameworks, just my personal opinion.

[MikeyBoy]

George P wrote:

What you've mentioned here in this thread does give me a bit of a *light bulb over one's head!* dim flash of insight into why unit-testing can be useful and important.

Thank you. :)

No problem! It's one of those those things that doesn't really become very relevant until you're working on projects large enough to have significant integration issues. At that point, being able to separate out "bugs within a single unit" from "bugs in the ways units are interacting with each other" becomes really useful.

wqking wrote:

Maybe CATCH includes iostream already.

One of the jobs of a unit test framework, as I said earlier, is to output well-formatted messages about passes and failures. It's npo surprise thee that the Catch2 headers will #include <iostream> .