How to compare paired count data?
$begingroup$
I am working with a machine learning approach that counts cars in images. I have a predicted dataset, which is the predicted output from the machine learning approach and a paired "true" dataset, which is the result of a human going through each image and counting the number of cars.
The following is a sample of what the datasets look like (note that the actual dataset has 2500 paired samples):
import pandas as pd
d = {'true': [0,0,0,1,1,0,1,0,0,0,0,0,0,0,4,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1],
'predicted': [0,0,0,0,0,0,1,0,0,0,0,0,0,0,2,2,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,1]}
df = pd.DataFrame(data=d)
true predicted
0 0 0
1 0 0
2 0 0
3 1 0
4 1 0
5 0 0
6 1 1
7 0 0
8 0 0
9 0 0
10 0 0
11 0 0
12 0 0
13 0 0
14 4 2
15 2 2
16 0 0
17 0 0
18 0 0
19 0 0
20 0 0
21 0 0
22 0 0
23 0 0
24 0 1
25 0 0
26 0 0
27 0 0
28 0 0
29 0 0
30 0 0
31 0 0
32 1 1
I am looking for a way to present the predicted approach to an audience so that they see if the predictions are statistically the same as the true observations and visualize any trends in the data (e.g. the predicted approach has a tendency to over or under predict). If these were categorical data, I would use a confusion matrix, however, I am not sure how to deal with these paired, discrete datasets that are heavily weighted with 0's.
What approach can I take to statistically compare the predicted vs true datasets?
machine-learning python pandas accuracy
asked Apr 16 at 2:47
BorealisBorealis
122213
$endgroup$
This question has an open bounty worth +50
reputation from Borealis ending in 7 days.
The question is widely applicable to a large audience. A detailed canonical answer is required to address all the concerns.
add a comment |
$begingroup$
I am working with a machine learning approach that counts cars in images. I have a predicted dataset, which is the predicted output from the machine learning approach and a paired "true" dataset, which is the result of a human going through each image and counting the number of cars.
The following is a sample of what the datasets look like (note that the actual dataset has 2500 paired samples):
import pandas as pd
d = {'true': [0,0,0,1,1,0,1,0,0,0,0,0,0,0,4,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1],
'predicted': [0,0,0,0,0,0,1,0,0,0,0,0,0,0,2,2,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,1]}
df = pd.DataFrame(data=d)
true predicted
0 0 0
1 0 0
2 0 0
3 1 0
4 1 0
5 0 0
6 1 1
7 0 0
8 0 0
9 0 0
10 0 0
11 0 0
12 0 0
13 0 0
14 4 2
15 2 2
16 0 0
17 0 0
18 0 0
19 0 0
20 0 0
21 0 0
22 0 0
23 0 0
24 0 1
25 0 0
26 0 0
27 0 0
28 0 0
29 0 0
30 0 0
31 0 0
32 1 1
I am looking for a way to present the predicted approach to an audience so that they see if the predictions are statistically the same as the true observations and visualize any trends in the data (e.g. the predicted approach has a tendency to over or under predict). If these were categorical data, I would use a confusion matrix, however, I am not sure how to deal with these paired, discrete datasets that are heavily weighted with 0's.
What approach can I take to statistically compare the predicted vs true datasets?
machine-learning python pandas accuracy
asked Apr 16 at 2:47
BorealisBorealis
122213
$endgroup$
This question has an open bounty worth +50
reputation from Borealis ending in 7 days.
The question is widely applicable to a large audience. A detailed canonical answer is required to address all the concerns.
add a comment |
$begingroup$
I am working with a machine learning approach that counts cars in images. I have a predicted dataset, which is the predicted output from the machine learning approach and a paired "true" dataset, which is the result of a human going through each image and counting the number of cars.
The following is a sample of what the datasets look like (note that the actual dataset has 2500 paired samples):
import pandas as pd
d = {'true': [0,0,0,1,1,0,1,0,0,0,0,0,0,0,4,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1],
'predicted': [0,0,0,0,0,0,1,0,0,0,0,0,0,0,2,2,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,1]}
df = pd.DataFrame(data=d)
true predicted
0 0 0
1 0 0
2 0 0
3 1 0
4 1 0
5 0 0
6 1 1
7 0 0
8 0 0
9 0 0
10 0 0
11 0 0
12 0 0
13 0 0
14 4 2
15 2 2
16 0 0
17 0 0
18 0 0
19 0 0
20 0 0
21 0 0
22 0 0
23 0 0
24 0 1
25 0 0
26 0 0
27 0 0
28 0 0
29 0 0
30 0 0
31 0 0
32 1 1
I am looking for a way to present the predicted approach to an audience so that they see if the predictions are statistically the same as the true observations and visualize any trends in the data (e.g. the predicted approach has a tendency to over or under predict). If these were categorical data, I would use a confusion matrix, however, I am not sure how to deal with these paired, discrete datasets that are heavily weighted with 0's.
What approach can I take to statistically compare the predicted vs true datasets?
machine-learning python pandas accuracy
asked Apr 16 at 2:47
BorealisBorealis
122213
$endgroup$
I am working with a machine learning approach that counts cars in images. I have a predicted dataset, which is the predicted output from the machine learning approach and a paired "true" dataset, which is the result of a human going through each image and counting the number of cars.
The following is a sample of what the datasets look like (note that the actual dataset has 2500 paired samples):
import pandas as pd
d = {'true': [0,0,0,1,1,0,1,0,0,0,0,0,0,0,4,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1],
'predicted': [0,0,0,0,0,0,1,0,0,0,0,0,0,0,2,2,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,1]}
df = pd.DataFrame(data=d)
true predicted
0 0 0
1 0 0
2 0 0
3 1 0
4 1 0
5 0 0
6 1 1
7 0 0
8 0 0
9 0 0
10 0 0
11 0 0
12 0 0
13 0 0
14 4 2
15 2 2
16 0 0
17 0 0
18 0 0
19 0 0
20 0 0
21 0 0
22 0 0
23 0 0
24 0 1
25 0 0
26 0 0
27 0 0
28 0 0
29 0 0
30 0 0
31 0 0
32 1 1
I am looking for a way to present the predicted approach to an audience so that they see if the predictions are statistically the same as the true observations and visualize any trends in the data (e.g. the predicted approach has a tendency to over or under predict). If these were categorical data, I would use a confusion matrix, however, I am not sure how to deal with these paired, discrete datasets that are heavily weighted with 0's.
What approach can I take to statistically compare the predicted vs true datasets?
machine-learning python pandas accuracy
machine-learning python pandas accuracy
asked Apr 16 at 2:47
BorealisBorealis
122213
asked Apr 16 at 2:47
BorealisBorealis
122213
edited 6 mins ago
Borealis
asked Apr 16 at 2:47
BorealisBorealis
122213
asked Apr 16 at 2:47
BorealisBorealis
122213
asked Apr 16 at 2:47
BorealisBorealis
122213
122213
This question has an open bounty worth +50
reputation from Borealis ending in 7 days.
The question is widely applicable to a large audience. A detailed canonical answer is required to address all the concerns.
This question has an open bounty worth +50
reputation from Borealis ending in 7 days.
The question is widely applicable to a large audience. A detailed canonical answer is required to address all the concerns.
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
$begingroup$
You can use a simple error measure of $sum (real.people-predicted.people)^2+sum (real.cars-predicted.cars)^2$, the kind of problem you are dealing with has this objective function as the solved one.
Actually, the algorithms implement this measure as their objective function.
answered Apr 16 at 3:23
Juan Esteban de la CalleJuan Esteban de la Calle
69122
$endgroup$
$begingroup$
This approach would yield two numbers--one for each class. Would the results of your approach, for example, "person" -7 and "car" +4 be sufficient to describe the predicted accuracy?
$endgroup$
– Borealis
Apr 16 at 4:31
$begingroup$
You are right, there is something to be corrected in the post. I edited it, I put the square in the difference, this way the errors will not substract.
$endgroup$
– Juan Esteban de la Calle
Apr 16 at 4:45
$begingroup$
I appreciate your help in this. I had to reword my question to clarify the problem I am trying to solve.
$endgroup$
– Borealis
2 mins ago
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "557"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f49363%2fhow-to-compare-paired-count-data%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
You can use a simple error measure of $sum (real.people-predicted.people)^2+sum (real.cars-predicted.cars)^2$, the kind of problem you are dealing with has this objective function as the solved one.
Actually, the algorithms implement this measure as their objective function.
answered Apr 16 at 3:23
Juan Esteban de la CalleJuan Esteban de la Calle
69122
$endgroup$
$begingroup$
This approach would yield two numbers--one for each class. Would the results of your approach, for example, "person" -7 and "car" +4 be sufficient to describe the predicted accuracy?
$endgroup$
– Borealis
Apr 16 at 4:31
$begingroup$
You are right, there is something to be corrected in the post. I edited it, I put the square in the difference, this way the errors will not substract.
$endgroup$
– Juan Esteban de la Calle
Apr 16 at 4:45
$begingroup$
I appreciate your help in this. I had to reword my question to clarify the problem I am trying to solve.
$endgroup$
– Borealis
2 mins ago
add a comment |
$begingroup$
You can use a simple error measure of $sum (real.people-predicted.people)^2+sum (real.cars-predicted.cars)^2$, the kind of problem you are dealing with has this objective function as the solved one.
Actually, the algorithms implement this measure as their objective function.
answered Apr 16 at 3:23
Juan Esteban de la CalleJuan Esteban de la Calle
69122
$endgroup$
$begingroup$
This approach would yield two numbers--one for each class. Would the results of your approach, for example, "person" -7 and "car" +4 be sufficient to describe the predicted accuracy?
$endgroup$
– Borealis
Apr 16 at 4:31
$begingroup$
You are right, there is something to be corrected in the post. I edited it, I put the square in the difference, this way the errors will not substract.
$endgroup$
– Juan Esteban de la Calle
Apr 16 at 4:45
$begingroup$
I appreciate your help in this. I had to reword my question to clarify the problem I am trying to solve.
$endgroup$
– Borealis
2 mins ago
add a comment |
$begingroup$
You can use a simple error measure of $sum (real.people-predicted.people)^2+sum (real.cars-predicted.cars)^2$, the kind of problem you are dealing with has this objective function as the solved one.
Actually, the algorithms implement this measure as their objective function.
answered Apr 16 at 3:23
Juan Esteban de la CalleJuan Esteban de la Calle
69122
$endgroup$
You can use a simple error measure of $sum (real.people-predicted.people)^2+sum (real.cars-predicted.cars)^2$, the kind of problem you are dealing with has this objective function as the solved one.
Actually, the algorithms implement this measure as their objective function.
answered Apr 16 at 3:23
Juan Esteban de la CalleJuan Esteban de la Calle
69122
edited Apr 16 at 4:46
answered Apr 16 at 3:23
Juan Esteban de la CalleJuan Esteban de la Calle
69122
answered Apr 16 at 3:23
Juan Esteban de la CalleJuan Esteban de la Calle
69122
answered Apr 16 at 3:23
Juan Esteban de la CalleJuan Esteban de la Calle
69122
69122
$begingroup$
This approach would yield two numbers--one for each class. Would the results of your approach, for example, "person" -7 and "car" +4 be sufficient to describe the predicted accuracy?
$endgroup$
– Borealis
Apr 16 at 4:31
$begingroup$
You are right, there is something to be corrected in the post. I edited it, I put the square in the difference, this way the errors will not substract.
$endgroup$
– Juan Esteban de la Calle
Apr 16 at 4:45
$begingroup$
I appreciate your help in this. I had to reword my question to clarify the problem I am trying to solve.
$endgroup$
– Borealis
2 mins ago
add a comment |
$begingroup$
This approach would yield two numbers--one for each class. Would the results of your approach, for example, "person" -7 and "car" +4 be sufficient to describe the predicted accuracy?
$endgroup$
– Borealis
Apr 16 at 4:31
$begingroup$
You are right, there is something to be corrected in the post. I edited it, I put the square in the difference, this way the errors will not substract.
$endgroup$
– Juan Esteban de la Calle
Apr 16 at 4:45
$begingroup$
I appreciate your help in this. I had to reword my question to clarify the problem I am trying to solve.
$endgroup$
– Borealis
2 mins ago
$begingroup$
This approach would yield two numbers--one for each class. Would the results of your approach, for example, "person" -7 and "car" +4 be sufficient to describe the predicted accuracy?
$endgroup$
– Borealis
Apr 16 at 4:31
$begingroup$
This approach would yield two numbers--one for each class. Would the results of your approach, for example, "person" -7 and "car" +4 be sufficient to describe the predicted accuracy?
$endgroup$
– Borealis
Apr 16 at 4:31
$begingroup$
You are right, there is something to be corrected in the post. I edited it, I put the square in the difference, this way the errors will not substract.
$endgroup$
– Juan Esteban de la Calle
Apr 16 at 4:45
$begingroup$
You are right, there is something to be corrected in the post. I edited it, I put the square in the difference, this way the errors will not substract.
$endgroup$
– Juan Esteban de la Calle
Apr 16 at 4:45
$begingroup$
I appreciate your help in this. I had to reword my question to clarify the problem I am trying to solve.
$endgroup$
– Borealis
2 mins ago
$begingroup$
I appreciate your help in this. I had to reword my question to clarify the problem I am trying to solve.
$endgroup$
– Borealis
2 mins ago
add a comment |
Thanks for contributing an answer to Data Science Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f49363%2fhow-to-compare-paired-count-data%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
