Defining constant data in header

[helios]

Is there any way to define a large (let's say, a kilobyte) constant array such that
1. There won't be duplicate definition errors.
2. The values will be stored directly in the binary.
?

Regarding point #2, I know I can do this, for example:

inline const std::uint8_t *get_data(){
    static const std::uint8_t data[] = { /*...*/ };
    return data;
}

However, because static locals need thread-safe initialization, the implementation decided to use TLS to prevent double initialization. Because of various reasons that are not relevant right now, the image's thread information wasn't fully initialized and TLS didn't work properly, so the function that needed that data produced incorrect results.
On the other hand, using a normal global array

extern const std::uint8_t data[] = { /*...*/ };

makes data point directly to the read-only location in the binary, which is initialized even on-disk.

[mbozzi]

Probably I'm misunderstanding something, but won't an inline (namespace scope) variable fit the bill?
inline constexpr uint8_t data[] { 0x00, 0x01, 0x02, 0x03, 0x04 /* ... */ };

[helios]

Ooh, I didn't know inline could be used for data! Unfortunately my compiler rejects it.

[mbozzi]

Ooh, I didn't know inline could be used for data! Unfortunately my compiler rejects it.

Before C++17, you can use a static member of a class template to the same effect.

template <typename = void> struct my_data
{ static constexpr std::uint8_t value[] = { /**/ }; };

[malibor]

<removed because possibly not relevant>

[malibor]

<removed because not relevant>

[mbozzi]

@malibor, I removed my post before you replied, because I felt it wasn't entirely topical. I didn't mean to derail any discussion with you. However, I'm not certain what you're claiming about odr-usage, and I don't want to enter into another lengthy discussion about constexpr on @helios's thread. Maybe we could start an independent thread?

If multiple threads attempt to initialize the same static local variable concurrently, the initialization occurs exactly once.

Supposedly, yes, but repeating what's in the original post:

helios wrote:

However, because static locals need thread-safe initialization, the implementation decided to use TLS to prevent double initialization. Because of various reasons that are not relevant right now, the image's thread information wasn't fully initialized and TLS didn't work properly, so the function that needed that data produced incorrect results.

so what was your point about static constexpr in your sample code?

I was comparing @helios's original post with yours, which differs only in the use of const vs. constexpr.

So we can conclude that both your example code and my example code is not going to work for the problem helios described? [... Both static variables] will be initialized before function is first entered right?

The rules are different for static local variables.

A static local that is not constant-initialized is supposed to be initialized when control first reaches its declaration. A static local that is constant-initialized is supposed to be initialized when control first reaches the innermost enclosing block scope containing the declaration.

A static member of a class template is not local; it's essentially an inline variable. It's initialized before main is called when there's only one thread, and no risk of synchronization issues.

[malibor]

Didn't noticed you removed your post, and I don't want to debate this either, unfortunately I can't remove my post anymore...

Anyway I think Helios should better describe his problem and in more detail because it's unclear what exactly is causing the problem. the code he posted, and the solution (be it with template or without) with local static or class inline static is correct and should not cause any initialization issue in multithreaded program.

[helios]

(I haven't yet had a chance to test mbozzi's latest suggestion.)

Anyway I think Helios should better describe his problem and in more detail because it's unclear what exactly is causing the problem. the code he posted, and the solution (be it with template or without) with local static or class inline static is correct and should not cause any initialization issue in multithreaded program.

Yes, the code I posted is correct, but it assumes certain things about the environment that are not true in my case.

Since you're asking, I'm injecting my DLL into another process by first allocating a chunk of memory and then copying the DLL at the appropriate offsets, then doing the usual initialization that LdrLoadDll() would do. I'm doing this because the process has enabled a protection that prevents DLLs not signed (i.e. authored) by Microsoft from being loaded.
Now, the injected code works fine, but since it doesn't exist as a proper module that the system knows about, it doesn't get notified when a thread is created. Normally modules get a call to DllMain() with DLL_THREAD_ATTACH, which they can use to set up or update the environment. MSVC's C++ runtime presumably has some code that runs at that moment that enables static locals to work properly.

There are various other solutions I could try for this. For example, I could hook another module's DllMain() to catch the DLL_THREAD_ATTACH. If the injected DLL wasn't working at all I might consider that, but since everything other than an MD5 implementation (the static data in question is an MD5 lookup table) is working fine, and since most other solutions would be rather fragile, moving the data somewhere else is just much easier.

[jonnin]

dunno how you feel about questionable sites and coders, but its possible that the protection you mention has been cracked and you could just do things 'normally' by using the known crack to spoof the protection into going away. I don't know if its been done or not. A lot of microsoft stuff gets compromised rather quickly. This may be cleaner than trying to go fully behind its back as you are doing. ?

[helios]

I don't see any way that could work. The protection is almost certainly implemented by flipping a flag in the kernel's data structure which holds information about a process. Modifying kernel memory from user mode is no easy matter, and either way it would probably be patched in at most a few months, so it'd be even more fragile than what I'm doing now.

[jonnin]

Being patched out is a possible issue, for sure. Just wondering if it was a path worth looking at.