C++

Forum

 
Incorrect result

onetwo123 (74)
I check time of foo(). It I make a graph of the dependence of the average value on n I should get a parabola.
But now I have this result. Avarage grows up but in 7000 it falls. Why?:


Argument is 1000
Average time is 2.3 seconds
Standard deviation of time is 0.458258 seconds

Argument is 2000
Average time is 17.4 seconds
Standard deviation of time is 9.4784 seconds

Argument is 3000
Average time is 40 seconds
Standard deviation of time is 14.2548 seconds

Argument is 4000
Average time is 74.9 seconds
Standard deviation of time is 14.6045 seconds

Argument is 5000
Average time is 95.5 seconds
Standard deviation of time is 18.7949 seconds

Argument is 6000
Average time is 197 seconds
Standard deviation of time is 29.213 seconds

Argument is 7000
Average time is 153.3 seconds
Standard deviation of time is 35.1 seconds

Argument is 8000
Average time is 173.3 seconds
Standard deviation of time is 6.35689 seconds

Argument is 9000
Average time is 225.8 seconds
Standard deviation of time is 20.6824 seconds

Argument is 10000
Average time is 518.1 seconds
Standard deviation of time is 348.98 seconds


experiment.cpp
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
 
#include "experiment.h"
#include <chrono>
#include <iostream>
#include <thread>
#include<fstream>
#include <string>
#include <time.h> 
using namespace std::chrono;
using namespace std;

int experiment::foo(int n) {
    int s{ 0 };
    for (int i = 0; i < n; i++) {
        for (int j = 0; j < n; j++) {
            s += i * j;
        }
    }
    return s;
}

double experiment::measureTime(int n) {
    clock_t t;
    int f;
    t = clock();
    f = foo(n);
    t = clock() - t;
    return t;
}
void experiment::Experiment(double* arguments, size_t n, size_t z, double* means, double* stds) {
    statistics stat;
    double* times = new double[z];
    for (size_t i = 0; i < n; i++) {
        cout << "\nArgument is " << arguments[i] << endl;
        for (size_t j = 0; j < z; j++) {
            times[j] = measureTime(arguments[i]);
        }
        means[i] = stat.meanTime(times, z);
        stds[i] = stat.stdTime(times, z, means[i]);
    }
    delete[] times;
}


experiment.h
1
2
3
4
5
6
7
8
9
10
11
12
13
14
 
#pragma once
#include "statistics.h"

#ifndef EXPERIMENT
#define EXPERIMENT

class experiment {
public:
	int foo(int n);
	double measureTime(int n);
	void Experiment(double* arguments, size_t n, size_t z, double* means, double* stds);
};

#endif


statistics.cpp
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
 
#include "statistics.h"
#include "cmath"
#include <iostream>

using namespace std;

double statistics::meanTime(double* t, size_t z) {
    double s{ 0 };
    for (size_t i = 0; i < z; i++) s += t[i];
    cout << "Average time is " << s / z << " seconds " << endl;
    return  s / z;
}

double statistics::stdTime(double* t, size_t z, double mean) {
    double s{ 0 };
    for (size_t i = 0; i < z; i++) s += ((t[i] - mean) * (t[i] - mean));
    cout << "Standard deviation of time is " << sqrt(s / z) << " seconds " << endl;
    return  sqrt(s / z);
}


statistics.h
1
2
3
4
5
6
7
8
9
10
11
12
 
#pragma once

#ifndef STATISTICS
#define STATISTICS

class statistics {
public:
	double meanTime(double* array, size_t z);
	double stdTime(double* t, size_t z, double mean);
};
#endif


main.cpp
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
 
#include <iostream>
#include "experiment.h"
#include "statistics.h"

using namespace std;

int main() {
    experiment ex;
    size_t n{ 10 };
    size_t z{ 10 };

    double* arguments = new double[n];
    for (size_t i = 0; i < n; i++) arguments[i] = (i + 1) * 1000;
    double* means = new double[n];
    double* stds = new double[n];

    ex.Experiment(arguments, n, z, means, stds);

    delete[] arguments;
    delete[] means;
    delete[] stds;
    
    return 0;
}
Last edited on
keskiverto (10402)
No, it does not "fall in 7000". From numbers alone, intuitively, the 6000 is an outlier.
Did you plot the results? Did you fit a curve to the plot? Which point(s) deviate most?
What if you run the program again?

Ask yourself: Can something else running in the same machine at the same time affect the results?
onetwo123 (74)
No, it does not "fall in 7000". From numbers alone, intuitively, the 6000 is an outlier.


Okey, but how can I fix it?
keskiverto (10402)
Fix what? If the results are affected by something that is not in your control -- outside of your code, then what could you do?
onetwo123 (74)
Fix what? If the results are affected by something that is not in your control -- outside of your code, then what could you do?


Then could you check this code on your computer? I just think that problem in my code is.
Ganado (6811)
It's probably overflowing at high integers. So maybe change all your 'int's to 'long long's. The common theme in all of these threads is that I have no idea what's going on. Nobody knows what specifically is wrong, because it's not clear what is supposed to be happening.
Last edited on
onetwo123 (74)
So maybe change all your 'int's to 'long long's

I tried it.
The common theme in all of your answer is that It's probably overflowing (and it doesn't work)

Now I tried to rewrite code in one file and it works. But I need another files (experiment.cpp, statistics.cpp, etc.). Probably, it was a mistake 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
 
#include <iostream>
#include <chrono>
#include "cmath"

using namespace std;
using namespace std::chrono;

int foo(int n){
    int s{0};
    for (int i = 1; i <= n; ++i)
        for (int j = 1; j <= n; ++j)
            s += i * j;
    return s;
}

double measureTime(int n){
    steady_clock::time_point t1 = steady_clock::now();
    foo(n);
    steady_clock::time_point t2 = steady_clock::now();

    duration<double> time_span = duration_cast<duration<double>>(t2 - t1);
    double mTime = time_span.count();
    cout << "It took me " << mTime << " seconds" << endl;
    return mTime;
}

double meanTime(double* t, size_t z){
    double s{0};
    for (size_t i = 0; i < z; ++i)
        s += t[i];
    double aTime = s / z;
    cout << "Average time is " << aTime << " seconds" << endl;
    return aTime;
}

double stdTime(double* t, size_t z, double mean){
    double s{0};
    for (size_t i = 0; i < z; ++i)
        s += (t[i] - mean) * (t[i] - mean);
    double sTime = sqrt(s / z);
    cout << "Standard deviation of time is " << sTime << endl;
    return  sTime;
}

void experiment(int* arguments, size_t n, size_t z, double* means, double* stds){
    double* times = new double[z];
    for (size_t i = 0; i < n; ++i) {
        cout << "Argument is " << arguments[i] << endl;
        for (size_t j = 0; j < z; ++j)
            times[j] = measureTime(arguments[i]);
        means[i] = meanTime(times, z);
        stds[i] = stdTime(times, z, means[i]);
    }
    delete[] times;
}

int main(){
    size_t n{10};
    size_t z{10};
    int* arguments = new int[n];
    for (size_t i = 0; i < n; ++i)
        arguments[i] = (i + 1) * 1000;
    double* means = new double[n];
    double* stds = new double[n];
    experiment(arguments, n, z, means, stds);
    delete[] arguments;
    delete[] means;
    delete[] stds;
    return 0;
}
Last edited on
ne555 (10692)
> Ask yourself: Can something else running in the same machine at the same time affect the results?
clock() (which was used in the first code snip) is supposed to give you CPU time, so other process or i/o operations should have little effect on it
unless you are on windows, where they surprisingly managed to fuck it up and return wall clock time

> No, it does not "fall in 7000". From numbers alone, intuitively, the 6000 is an outlier.
>> Okey, but how can I fix it?
- increase the number of samples
- remove the outliers (alpha-trim)
- provide a clean environment (leave your experiment alone, don't start you 3d rendering)
- understand the error and deal with it

notice that your deviation is quite big too
onetwo123 (74)
Fix what? If the results are affected by something that is not in your control -- outside of your code, then what could you do?


It's probably overflowing at high integers. So maybe change all your 'int's to 'long long's. The common theme in all of these threads is that I have no idea what's going on. Nobody knows what specifically is wrong, because it's not clear what is supposed to be happening.


Now I tried it in code blocks and it works correctly. Before I worked in visual studio.

I think that question is closed. Thank you very much)
onetwo123 (74)
> Ask yourself: Can something else running in the same machine at the same time affect the results?
clock() (which was used in the first code snip) is supposed to give you CPU time, so other process or i/o operations should have little effect on it
unless you are on windows, where they surprisingly managed to fuck it up and return wall clock time

> No, it does not "fall in 7000". From numbers alone, intuitively, the 6000 is an outlier.
>> Okey, but how can I fix it?
- increase the number of samples
- remove the outliers (alpha-trim)
- provide a clean environment (leave your experiment alone, don't start you 3d rendering)
- understand the error and deal with it

notice that your deviation is quite big too


Thanks!
Topic archived. No new replies allowed.