Communicating serialized objects over network

[see17]

Hi, I am a student working with this library https://github.com/microsoft/SealPIR/blob/master/main.cpp which demonstrates a "query" (L79) and "reply" (L90). These objects seem to be already serialized, and I want to run the same demo but over a local network. Can someone point me how to setup two instances of this code where one acts as a "server" and the other as a "client" and these objects being communicated from/to each other?

[doug4]

First of all, why do you think the objects are already serialized? The serialize and deserialize statements are commented out because everything is in the same process. Serialization and deserialization is used when one process creates the message/response and sends it to another process to process it. If the same process creates a query and then processes it, (in this case generate_reply(query, 0)), there is no reason to serialize it.

(Do you understand what serialization is?)

In order to make this work on a network, start by running it on 2 processes on the same machine. The client process should contain all of the code to (1) connect to the server, (2) generate and serialize the query (3) send the serialized query to the server (4) receive the serialized results (5) deserialize the results (6) check that the results are correct and (7) disconnect from the server.

The server (1) listens for connect and disconnect requests (and processes them accordingly), (2) listen for queries on established connections. For each query (a) receive serialized query, (b) deserialize query, (c) call generate_reply, (d) serialize result and (e) send serialized result back to correct client.

You can do the timing in either process or both.

This is not a trivial question. In addition to the above, you need to use (probably) a socket library for establishing the connection. You will also need to keep track of which client made each query so you know where to send the response.

So, you have lots of work ahead of you, but it can be done.

[see17]

Thank you for replying.
Yes I was thinking to simply comment out L85 on the client side and L86 on the server side and send query_ser over the network (similarly for reply). I just don't know where to start from, and looking for a simple tutorial-level code to import into this and send-receive query_ser.
I was looking at this tutorial
https://www.geeksforgeeks.org/socket-programming-cc/
which sends
char *hello = "Hello from server"
over the network. Can I just replace it with query_ser?

[doug4]

Yeah, that would be a good place to start.

Actually, start by getting the socket tutorial working first. Make sure you can send the Hello message from the client to the server and back again.

When that works, then add in the serialization/deserialization of data structures.

Make sure you do everything 1 step at a time.

[see17]

Ok so my server and client are communicating with the simple geeksforgeeks example, the "Hello" outputs are working as intended.
At this moment I just need to get a normal query through (without the reply). Here is modified server code:

https://pastebin.com/1eHbjEPP

and here is modified client code:

https://pastebin.com/ZKWGy7Yh


Basically I put query_ser in line 121 for the client code as
send(sock , query_ser , strlen(query_ser) , 0 );

Then for the server I try to recover query_ser in lines 142-143 and deserialize it in L148 and use it in the rest of the code. However I am getting iostream error.

What I am missing? Should I change the buffer size?

[jonnin]

post exact error message.

[see17]

Here is the error message which happens on the server side.

Waiting to receive query...
G��x04c�R�G�2z�^T|��-v�բ�
Query Received
terminate called after throwing an instance of 'std::__ios_failure'
  what():  basic_ios::clear: iostream error
Aborted (core dumped)

Apparently I am missing something on how the data are transferred.
In the client, after I generate the serialized query query_ser on L92 of type std::string, I feed it into L121 as
send(sock , query_ser.data() , strlen(query_ser.data()) , 0 );
However on the receiver side, on L146 I recover it as
query_ser = read( new_socket , buffer, 1024);
I think I am not recovering it correctly?

[see17]

Ok so I used the solution from this post
https://stackoverflow.com/questions/18463414/getting-segfault-due-to-string-data-type-variable-in-protobuf-server-and-client/18464632#18464632

But the std::string.assing() does not work for some reason for the server-receiver (L165).

Here's the current error message I'm getting, any help greatly appreciated!

main.cpp:165:36: error: no matching function for call to ‘std::__cxx11::basic_string<char>::assign(__gnu_cxx::__alloc_traits<std::allocator<unsigned char>, unsigned char>::value_type*, std::vector<unsigned char>::size_type)’
  165 |     tmp.assign(&(pkt[0]),pkt.size()); // Convert message data to a string
      |                                    ^
In file included from /usr/include/c++/9/string:55,
                 from /usr/include/c++/9/bits/locale_classes.h:40,
                 from /usr/include/c++/9/bits/ios_base.h:41,
                 from /usr/include/c++/9/ios:42,
                 from /usr/include/c++/9/ostream:38,
                 from /usr/include/c++/9/iostream:39,
                 from /usr/local/include/seal/biguint.h:6,
                 from /usr/local/include/seal/seal.h:6,
...

[mbozzi]

But the std::string::assign() does not work for some reason

Because unsigned char is not char

[see17]

Can you please explain how to fix? I'm not very experienced and confused. Don't see why it would work for the SO post and not for my code..

[JLBorges]

With

std::vector<unsigned char> pkt ;
std::string temp ;

Any of these would be ok:

// Convert message data to a string
temp = { pkt.begin(), pkt.end() } ;

temp.assign( pkt.begin(), pkt.end() ) ;

temp.clear() ;
for( char c : temp ) temp += c ;

[see17]

Thanks! that took care of this error.

Unfortunately the server-receiver still seems to receive a corrupted query, here's the error I get now, any idea on what can be wrong?
It seems that the received value for msgLength was supposed to be 65682, but the actual length of temp is zero..

...
Query Received
Query length: 65682
Actual query length: 0
Query: 
terminate called after throwing an instance of 'std::__ios_failure'
  what():  basic_ios::clear: iostream error
Aborted (core dumped)

Server code:
https://pastebin.com/1eHbjEPP
Client code:
https://pastebin.com/ZKWGy7Yh

[see17]

Hmm.. If I comment out the line
temp.clear() ;
then temp.size() will be equal to 131364, which is double the normal size (the query form is not correct though and results in different error)..

and if I just do

    temp = { pkt.begin(), pkt.end() } ;

    temp.assign( pkt.begin(), pkt.end() ) ;

then I get the correct size but still get a diffrent error related to the pir library

terminate called after throwing an instance of 'std::logic_error'
  what():  result ciphertext is transparent
Aborted (core dumped)

Is there any way to manually check if the reconstructed string temp on the server is identical to the sent string query_ser from the client? i.e print out the the last bit from the string?

[JLBorges]

> result ciphertext is transparent

See:

The problem is that you are subtracting a ciphertext from itself. This kind of operation results in a ciphertext that is identically zero; this is called a transparent ciphertext. Such a transparent ciphertext is not considered to be valid because the ciphertext reveals its underlying plaintext to anyone who sees it, even if they don't have the secret key. By default SEAL throws an exception when such a situation is encountered to protect you from a problem you may not have noticed. If you truly know what you are doing and want to enable the creation of transparent ciphertexts, you can configure SEAL with (the CMake flag) -DSEAL_THROW_ON_TRANSPARENT_CIPHERTEXT=OFF.

https://github.com/microsoft/SEAL/issues/224

[doug4]

I'll trust @JLBorges for the ciphertext statement. I have no idea about any of that.

However, note in his earlier post:

Any of these would be ok:

You appeared to try to do all of them, not just one of them.

// Convert message data to a string
temp = { pkt.begin(), pkt.end() } ;

Assigns string pkt to temp.

temp.assign( pkt.begin(), pkt.end() ) ;

Assigns string pkt to temp again. It just replaces the contents with the same contents.


The 3rd example actually has a bug in it. It should probably read

temp.clear() ;
for( char c : pkt ) temp += c ;

Explanation:

temp.clear() ;

Erases the contest of temp so that it has a length of 0.

for( char c : pkt) temp += c ;
Takes temp (whatever it contains) and append the contents of pkt to it.

When you tried to use all three methods at the same time, you ran into problems.

If you used the code as is (with the mistake), you would clear temp and then append all the character of temp onto itself. The resultant length of temp would still be 0.

When you didn't clear temp, you ended up appending temp to itself.

DON'T USE ALL THREE METHODS! Just 1. Those were just examples of ways to do it. And if you use the 3rd method, fix the typo first.

[seeplus]

The easiest way probably is to initialise temp when it's defined:

#include <string>
#include <iostream>
#include <vector>

int main() {
	const std::vector<unsigned char> pkt{'a', 'b', 'c', 'd'};

	const std::string temp {pkt.begin(), pkt.end()};

	std::cout << temp << '\n';
}

abcd

[see17]

Makes sense, thank you all for clarifying!

One more question: I also want to send over the network an object of type "octetStream" defined here
https://github.com/data61/MP-SPDZ/blob/master/Tools/octetStream.cpp
I want to use the same way as before, if I have an object o
octetStream testobj;

I want to send it as
send(new_socket,testobj ,msgLength ,0);
however I am not sure how to do it, i.e. which method to use from the above link and what is the exact syntax..
Can someone help with that as well?