AVI file too big to stuff into c array?

[ineedastupidusername]

It seems like manipulating files is convoluted in any language. It's like... there's a sequence of bytes over there in a file, I want them as manipulable data in my program, how hard could that be? It should be like copy/paste, like myarray=open(myfile). ta da! now like myarray[3] should show the third byte. Instead you have all this "stream" and file pointers and top google result documentation seems like bot generated lawyer speak for basement dwelling tech geeks on pcp. Plus "bit manipulation" doesn't seem to exist, but rather BYTE manipulation. So to see what the last bit is you have to shift the whole byte and & it or something.

Anyway so I tried c instead of c++ (since c++ file manipulation seemed sucky at first glance, so did python), and the following successfully opens test.avi and saves the first 1000 characters to output.avi (which is broken cuz its only 1000 bytes). This short code took like 3 days of research ha ha (though a lot was reading c tutorials on pointers and stuff).

So the test.avi opened has 115,778,890 bytes.
Every spot in the code below that says "1000" I want to replace with 115,778,890 (unsigned char bytes[size]; etc) but it crashes cuz it's too large to make an array that size I guess? If so I don't get why, clearly the file itself has no problem existing at that size so why not a c array of that length also?

Plus if you look at fwrite:
https://www.techonthenet.com/c_language/standard_library_functions/stdio_h/fwrite.php
you need the pointer of such a long array to feed as a parameter to fwrite.

#include <stdio.h>

int main () {
  int i;
  FILE* file = fopen("test.avi", "rb");
  fseek(file, 0, SEEK_END);
  long int size = ftell(file);
  printf("file has %u bytes\n",size);
  fseek(file, 0, SEEK_SET);
  unsigned char bytes[1000];
  fread(&bytes, 1, 1000, file);
  for(i=0;i<1000;i++){
    printf("%c",bytes[i]);
  }
  fclose(file);

  file = fopen("output.avi", "wb");
  int bytes_written = fwrite(&bytes, sizeof(unsigned char), 1000, file);
  fclose(file);

  return 0;
}

[JLBorges]

The amount of storage for objects with automatic storage duration (typically the allocated on the run-time stack) is limited.

Try:

//  unsigned char bytes[1000];
static unsigned char bytes[ 256 * 1024 * 1024 ]; // array with static storage duration 

[ineedastupidusername]

Great, that worked, thanks. Also tried as a global variable (without "static") and that worked too, shown below, not sure if one is preferable.
When I used 256*1024*1024 it worked (and the output avi played in vlc media player fine too), however the windows folder said output.avi was 262144 kb whereas the original test.avi was 113066 kb (which, by the way, doesn't perfectly match the 115,778,890 from ftell(). Code below made windows folder say both are 113066 kb). So I guess bytes array needs exact bytes size. Dumb thing about that is size was calculated later in the code... yet you need size before it is calculated to put in the global. So you have to run program to calculate size then copy paste that into the global and re-run. (although program complained it needed a constant as bytes length (like a number instead of a variable) anyway).

#include <stdio.h>
unsigned char bytes[115778890];
int main () {
  int i;
  FILE* file = fopen("test.avi", "rb");
  fseek(file, 0, SEEK_END);
  long int size = ftell(file);
  printf("file has %u bytes\n",size);
  fseek(file, 0, SEEK_SET);
  size=115778890;
  //static unsigned char bytes[1024*1024*256];
  fread(&bytes,1,size,file);
  for(i=0;i<1000;i++){
    printf("%c",bytes[i]);
  }
  fclose(file);

  file = fopen("output.avi", "wb");
  int bytes_written = fwrite(&bytes, sizeof(unsigned char), size, file);
  fclose(file);

  return 0;
}

[JLBorges]

#include <stdio.h>
#include <stdbool.h>

bool copy_binary_file( const char* srce_file_name, const char* dest_file_name )
{
    FILE* srce = fopen( srce_file_name, "rb" ) ;
    if( srce != NULL )
    {
        FILE* dest = fopen( dest_file_name, "wb" ) ;
        if( dest != NULL )
        {
            unsigned char bytes[1024*4] ;
            unsigned long total_bytes = 0 ;
            size_t bytes_read = 0 ;
            while( ( bytes_read = fread( bytes, 1, sizeof(bytes), srce ) ) > 0 )
            {
                fwrite( bytes, 1, bytes_read, dest ) ;
                total_bytes += bytes_read ;
            }

            printf( "%lu bytes were copied\n", total_bytes ) ;
            fclose(dest) ;
            return total_bytes > 0 ;
        }

        else fprintf( stderr, "error opening output file %s\n", dest_file_name ) ;

        fclose(srce) ;
    }

    else fprintf( stderr, "error opening input file %s\n", srce_file_name ) ;

    return false ;
}

[JLBorges]

> Anyway so I tried c instead of c++ (since c++ file manipulation seemed sucky at first glance

It is a lot easier in C++:

#include <iostream>
#include <fstream>

bool copy_binary_file( const char* srce_file_name, const char* dest_file_name )
{
    if( std::ifstream srce{ srce_file_name, std::ios::binary } )
    {
        if( std::ofstream dest { dest_file_name, std::ios::binary } )
            return bool( dest << srce.rdbuf() ) ;

        else std::cerr << "error opening output file " << dest_file_name << '\n' ;
    }

    else std::cerr << "error opening input file " << srce_file_name << '\n' ;

    return false ;
}

[Chervil]

there's a sequence of bytes over there in a file, I want them as manipulable data in my program

If you want to read an entire file in binary mode, an easy way is to use a stringstream. The file contents is stored in the string data which is effectively an array of characters.

#include <iostream>
#include <string>
#include <fstream>
#include <sstream>


std::string read_binary(std::string filename)
{
    std::ifstream fin{ filename, std::ios::binary  };
    std::ostringstream oss;
    oss << fin.rdbuf();
    return oss.str();
}

int main()
{
    const char * input_filename  = "test.avi";
    const char * output_filename = "output.avi";
    
    std::string data = read_binary(input_filename);
    std::cout << "bytes read in = " << data.size() << '\n';  
    
    std::ofstream fout(output_filename, std::ios::binary);
    fout << data << std::flush;
    
    std::cout << "bytes written = " << fout.tellp() << '\n';
    
}

[benhart]

I'd recommend you to go slow with programming to support AVI as it tend to be a little bit complex and that can harm your code by generating bugs and errors that are hard to be detected. I think that unless you;re using some programs as help, such as checkmarx for example, it is very important to take it slow and know exactly what you do.

[ineedastupidusername]

I tried JLBorges c example (taught me that fwrite automatically moves the file position each call which I didn't know) and works fine (the c++ one had no errors but didn't seem to produce any output.avi file in the folder), and Chervil's c++ string stream works fine too. I guess I'll stick with c for now at least, every line of that c++ program above opens a whole can of worms of things you have to learn/know... ifstream,fin,oss,ostringstream,where all this stuff is in each namespace/library/ how they work etc.

This was just a starting test to make an output file from an input. Really I want to create my own frames/framerate/etc into an avi, I guess making it from scratch... or maybe just copying all the "crap" from a seed avi then editing the few necessary parts. So now I have to read (again) avi documentation.. which seemed poorly documented,confusing,etc, like I haven't even figured out what type of picture (bmp,jpg,etc) to use as a frame or where to put it yet. I might have to read 4 bytes but interpret them as a single 4 byte number rather than 4 individual bytes or something, not sure yet.