Compute a signature with private key in Server Key Exchange

[volang]

On https://tls.ulfheim.net/ there is an example showing how to compute a signature in the section of "Server Key Exchange".

https://i.ibb.co/Y7fbkDw/1.jpg (This image shows the Server Key Exchange section on the website that I refer to.)

Whatever I try I dont get the same output as the one on that website, I dont understand why.

I've tried storing the same data in two different ways, then use the same openssl command that they use on their example. None of the methods gave the same output.


Method 1.

char hex[] = { 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f, 0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17, 0x18, 0x19, 0x1a, 0x1b, 0x1c, 0x1d, 0x1e, 0x1f, 0x70, 0x71, 0x72, 0x73, 0x74, 0x75, 0x76, 0x77, 0x78, 0x79, 0x7a, 0x7b, 0x7c, 0x7d, 0x7e, 0x7f, 0x80, 0x81, 0x82, 0x83, 0x84, 0x85, 0x86, 0x87, 0x88, 0x89, 0x8a, 0x8b, 0x8c, 0x8d, 0x8e, 0x8f, 0x20, 0x9f, 0xd7, 0xad, 0x6d, 0xcf, 0xf4, 0x29, 0x8d, 0xd3, 0xf9, 0x6d, 0x5b, 0x1b, 0x2a, 0xf9, 0x10, 0xa0, 0x53, 0x5b, 0x14, 0x88, 0xd7, 0xf8, 0xfa, 0xbb, 0x34, 0x9a, 0x98, 0x28, 0x80, 0xb6, 0x15 };
ofstream myfile("c:/hex1.txt", ios::binary);
myfile.write(hex, sizeof hex);

then:

openssl dgst -hex -sign server.key -sha256 hex1.txt

Method 2.

I had this data stored in hex2.txt (as ASCII):

\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f\x70\x71\x72\x73\x74\x75\x76\x77\x78\x79\x7a\x7b\x7c\x7d\x7e\x7f\x80\x81\x82\x83\x84\x85\x86\x87\x88\x89\x8a\x8b\x8c\x8d\x8e\x8f\x20\x9f\xd7\xad\x6d\xcf\xf4\x29\x8d\xd3\xf9\x6d\x5b\x1b\x2a\xf9\x10\xa0\x53\x5b\x14\x88\xd7\xf8\xfa\xbb\x34\x9a\x98\x28\x80\xb6\x15

then:

openssl dgst -hex -sign server.key -sha256 hex2.txt

[jonnin]

you are confusing hex and text I think. you asked for binary file (ios::binary) but you said it was a text file (.txt).
if its binary, your command line junk won't like that at all.
so you likely want a text file.
you can then just do a regular cout statement.
cout << "00\\01\\02\\...FF";

if you want a binary file, look at it in a hex editor, but you can't use it for most command line stuff.

sha is rigged so the slightest change gives a very different answer...
even if you had the first section right, its missing the slashes.
The files have to be exact matches, including whitespace and all.

[volang]

Hello jonnin.

I dont think I understand your answer. But I tried what you wrote...

I put the following in hex3.txt:

00\\01\\02\\03\\04\\05\\06\\07\\08\\09\\0a\\0b\\0c\\0d\\0e\\0f\\10\\11\\12\\13\\14\\15\\16\\17\\18\\19\\1a\\1b\\1c\\1d\\1e\\1f\\70\\71\\72\\73\\74\\75\\76\\77\\78\\79\\7a\\7b\\7c\\7d\\7e\\7f\\80\\81\\82\\83\\84\\85\\86\\87\\88\\89\\8a\\8b\\8c\\8d\\8e\\8f\\20\\9f\\d7\\ad\\6d\\cf\\f4\\29\\8d\\d3\\f9\\6d\\5b\\1b\\2a\\f9\\10\\a0\\53\\5b\\14\\88\\d7\\f8\\fa\\bb\\34\\9a\\98\\28\\80\\b6\\15

then tried:
openssl dgst -hex -sign server.key -sha256 hex3.txt

Still not correct output

[volang]

I found the solution. Thanks for your time bro

[jonnin]

text files are a subset of binary files that only allow printable characters.
but more than that...
a floating point number, a double, 3.141592653589793 for example. '3' in text is at least 1 byte. '.' in text, the same. And for each digit, another. In binary, the whole number takes up 8 bytes (double) but you can't read it: its a mix of unprintable characters and printable, but its nonsense and looks nothing like the value. Similar for integers.
so you binary file setting is probably wrong.
----------------
'\\' is a single character, the '\' inside code. in a text file, \ is \ but in code the \ has a special meaning and you have to type 2 of them to tell it you want the actual \ and not treat it as a special signal.

Even if you got it working, you need to understand these things.