- Forum
- General C++ Programming
- Best technique for storing large amounts
Best technique for storing large amounts of data
As the title would suggest I'm looking for a really optimal way of storing high volumes of data. My requirements favor speed over compression, but a small footprint on disk would be great too. Anyways. I'm thinking an sql database is my best bet, but if there is another choice exposed to a c/c++ application that has a faster look-up and entry time I was hoping some one could suggest it.
I doubt anything will beat a database. I wouldn't spend too much thinking about this.
Well I was wondering if SQL was my only option. I've never done work with a database so I honestly have no idea.
closed account (1vRz3TCk)
| Best technique for storing large amounts of data |
What type of data?
When you say large amounts, what do you mean?
What do you want to do with it after you store it?
closed account (1yR4jE8b)
+1 to CodeMonkey
Just to add, when you say large amounts of data: do you mean, millions of small entries, or a few large entries.
Also, how many clients will be connecting to the database?
Just to add, when you say large amounts of data: do you mean, millions of small entries, or a few large entries.
Also, how many clients will be connecting to the database?
The data is more than likely going to be loads of small entries. What I'm doing is using a (possible combination of multiple) learning algorithm(s) to learn what a micro gesture would look like for an arbitrary user, and then find them when presented. The inputs to the program are Video and Audio.
But this isn't the ultimate goal for this project. Starting from this I want the AI to learn how to interpret the user's outward emotions. As in, what the user looks and sounds like they are feeling. Eventually I want to incorporate the data I've drawn from this into a virtual avatar that can emulate emotions. Though these two portions wont be possible till much further down the line.
EDIT: I didn't really explain very clearly what I would be doing with the database besides storing learned information. I want to do this using the Artificial Neural Network approach to machine learning. I'm thinking storing the neural net would be best done with a database, but as I need really quick look up times for the connections of the virtual synapses I was hoping for some good speed.
But this isn't the ultimate goal for this project. Starting from this I want the AI to learn how to interpret the user's outward emotions. As in, what the user looks and sounds like they are feeling. Eventually I want to incorporate the data I've drawn from this into a virtual avatar that can emulate emotions. Though these two portions wont be possible till much further down the line.
EDIT: I didn't really explain very clearly what I would be doing with the database besides storing learned information. I want to do this using the Artificial Neural Network approach to machine learning. I'm thinking storing the neural net would be best done with a database, but as I need really quick look up times for the connections of the virtual synapses I was hoping for some good speed.
Last edited on
Topic archived. No new replies allowed.