Choosing k value in KNN classifier?












1












$begingroup$


I'm working on classification problem and decided to use KNN classifier for the problem.



so if k=131 gave me auc of 0.689 and k=71 gave me auc of 0.682 what should be my ideal k?



Does choosing higher k means more usage of computational resource? if that's the case can I go with k=71. (or) should I always use K with maximum score no matter what?






machine-learning k-nn







share|improve this question









$endgroup$












  • $begingroup$
    So, are you calculating auc using cross-validation?
    $endgroup$
    – pythinker
    7 hours ago










  • $begingroup$
    @pythinker yes..
    $endgroup$
    – user214
    7 hours ago
















1












$begingroup$


I'm working on classification problem and decided to use KNN classifier for the problem.



so if k=131 gave me auc of 0.689 and k=71 gave me auc of 0.682 what should be my ideal k?



Does choosing higher k means more usage of computational resource? if that's the case can I go with k=71. (or) should I always use K with maximum score no matter what?






machine-learning k-nn







share|improve this question









$endgroup$












  • $begingroup$
    So, are you calculating auc using cross-validation?
    $endgroup$
    – pythinker
    7 hours ago










  • $begingroup$
    @pythinker yes..
    $endgroup$
    – user214
    7 hours ago














1












1








1





$begingroup$


I'm working on classification problem and decided to use KNN classifier for the problem.



so if k=131 gave me auc of 0.689 and k=71 gave me auc of 0.682 what should be my ideal k?



Does choosing higher k means more usage of computational resource? if that's the case can I go with k=71. (or) should I always use K with maximum score no matter what?






machine-learning k-nn







share|improve this question









$endgroup$




I'm working on classification problem and decided to use KNN classifier for the problem.



so if k=131 gave me auc of 0.689 and k=71 gave me auc of 0.682 what should be my ideal k?



Does choosing higher k means more usage of computational resource? if that's the case can I go with k=71. (or) should I always use K with maximum score no matter what?






machine-learning k-nn




machine-learning k-nn






share|improve this question













share|improve this question











share|improve this question




share|improve this question










asked 7 hours ago









user214user214

22318




22318












  • $begingroup$
    So, are you calculating auc using cross-validation?
    $endgroup$
    – pythinker
    7 hours ago










  • $begingroup$
    @pythinker yes..
    $endgroup$
    – user214
    7 hours ago


















  • $begingroup$
    So, are you calculating auc using cross-validation?
    $endgroup$
    – pythinker
    7 hours ago










  • $begingroup$
    @pythinker yes..
    $endgroup$
    – user214
    7 hours ago
















$begingroup$
So, are you calculating auc using cross-validation?
$endgroup$
– pythinker
7 hours ago




$begingroup$
So, are you calculating auc using cross-validation?
$endgroup$
– pythinker
7 hours ago












$begingroup$
@pythinker yes..
$endgroup$
– user214
7 hours ago




$begingroup$
@pythinker yes..
$endgroup$
– user214
7 hours ago










2 Answers
2






active

oldest

votes


















1












$begingroup$

Because knn is a non-parametric method, computational costs of choosing k, highly depends on the size of training data. If the size of training data is small, you can freely choose the k for which the best auc for validation dataset is achieved. In the case where you have a large training dataset, choosing large k can lead to huge computational complexity which is reflected in slow prediction for test data.






share|improve this answer









$endgroup$













  • $begingroup$
    does 100k rows and 8000 features qualify as big training data? Also choosing high k values means we are underfitting how can I know that i'm not underfitting when choosing high k values?
    $endgroup$
    – user214
    6 hours ago








  • 1




    $begingroup$
    Yes, that’s actually a big training dataset. To ensure that you are not underfitting or overfitting, you should check the performance of your model on the training and validation dataset, simultaneously. If it training score is low, you are underfitting. If training score is much higher than validation score, you are overfitting. The best case is when training and validation scores are close enough.
    $endgroup$
    – pythinker
    6 hours ago





















1












$begingroup$

I was taught the best way is to find the error for each k then plot them and look for the "elbow" on the plot.






share|improve this answer








New contributor




Stephen Ewing is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.






$endgroup$













  • $begingroup$
    So I used go with k=131
    $endgroup$
    – user214
    7 hours ago










  • $begingroup$
    It really depends. The higher your k the higher your chance of overfitting. So if you do every k from 2 to 200 and plot the error of all of them you use the k where the curve starts to flatten out.
    $endgroup$
    – Stephen Ewing
    7 hours ago












Your Answer





StackExchange.ifUsing("editor", function () {
return StackExchange.using("mathjaxEditing", function () {
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
});
});
}, "mathjax-editing");

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "557"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});


}
});














draft saved

draft discarded


















StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f48905%2fchoosing-k-value-in-knn-classifier%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown

























2 Answers
2






active

oldest

votes








2 Answers
2






active

oldest

votes









active

oldest

votes






active

oldest

votes









1












$begingroup$

Because knn is a non-parametric method, computational costs of choosing k, highly depends on the size of training data. If the size of training data is small, you can freely choose the k for which the best auc for validation dataset is achieved. In the case where you have a large training dataset, choosing large k can lead to huge computational complexity which is reflected in slow prediction for test data.






share|improve this answer









$endgroup$













  • $begingroup$
    does 100k rows and 8000 features qualify as big training data? Also choosing high k values means we are underfitting how can I know that i'm not underfitting when choosing high k values?
    $endgroup$
    – user214
    6 hours ago








  • 1




    $begingroup$
    Yes, that’s actually a big training dataset. To ensure that you are not underfitting or overfitting, you should check the performance of your model on the training and validation dataset, simultaneously. If it training score is low, you are underfitting. If training score is much higher than validation score, you are overfitting. The best case is when training and validation scores are close enough.
    $endgroup$
    – pythinker
    6 hours ago


















1












$begingroup$

Because knn is a non-parametric method, computational costs of choosing k, highly depends on the size of training data. If the size of training data is small, you can freely choose the k for which the best auc for validation dataset is achieved. In the case where you have a large training dataset, choosing large k can lead to huge computational complexity which is reflected in slow prediction for test data.






share|improve this answer









$endgroup$













  • $begingroup$
    does 100k rows and 8000 features qualify as big training data? Also choosing high k values means we are underfitting how can I know that i'm not underfitting when choosing high k values?
    $endgroup$
    – user214
    6 hours ago








  • 1




    $begingroup$
    Yes, that’s actually a big training dataset. To ensure that you are not underfitting or overfitting, you should check the performance of your model on the training and validation dataset, simultaneously. If it training score is low, you are underfitting. If training score is much higher than validation score, you are overfitting. The best case is when training and validation scores are close enough.
    $endgroup$
    – pythinker
    6 hours ago
















1












1








1





$begingroup$

Because knn is a non-parametric method, computational costs of choosing k, highly depends on the size of training data. If the size of training data is small, you can freely choose the k for which the best auc for validation dataset is achieved. In the case where you have a large training dataset, choosing large k can lead to huge computational complexity which is reflected in slow prediction for test data.






share|improve this answer









$endgroup$



Because knn is a non-parametric method, computational costs of choosing k, highly depends on the size of training data. If the size of training data is small, you can freely choose the k for which the best auc for validation dataset is achieved. In the case where you have a large training dataset, choosing large k can lead to huge computational complexity which is reflected in slow prediction for test data.







share|improve this answer












share|improve this answer



share|improve this answer










answered 6 hours ago









pythinkerpythinker

5431211




5431211












  • $begingroup$
    does 100k rows and 8000 features qualify as big training data? Also choosing high k values means we are underfitting how can I know that i'm not underfitting when choosing high k values?
    $endgroup$
    – user214
    6 hours ago








  • 1




    $begingroup$
    Yes, that’s actually a big training dataset. To ensure that you are not underfitting or overfitting, you should check the performance of your model on the training and validation dataset, simultaneously. If it training score is low, you are underfitting. If training score is much higher than validation score, you are overfitting. The best case is when training and validation scores are close enough.
    $endgroup$
    – pythinker
    6 hours ago




















  • $begingroup$
    does 100k rows and 8000 features qualify as big training data? Also choosing high k values means we are underfitting how can I know that i'm not underfitting when choosing high k values?
    $endgroup$
    – user214
    6 hours ago








  • 1




    $begingroup$
    Yes, that’s actually a big training dataset. To ensure that you are not underfitting or overfitting, you should check the performance of your model on the training and validation dataset, simultaneously. If it training score is low, you are underfitting. If training score is much higher than validation score, you are overfitting. The best case is when training and validation scores are close enough.
    $endgroup$
    – pythinker
    6 hours ago


















$begingroup$
does 100k rows and 8000 features qualify as big training data? Also choosing high k values means we are underfitting how can I know that i'm not underfitting when choosing high k values?
$endgroup$
– user214
6 hours ago






$begingroup$
does 100k rows and 8000 features qualify as big training data? Also choosing high k values means we are underfitting how can I know that i'm not underfitting when choosing high k values?
$endgroup$
– user214
6 hours ago






1




1




$begingroup$
Yes, that’s actually a big training dataset. To ensure that you are not underfitting or overfitting, you should check the performance of your model on the training and validation dataset, simultaneously. If it training score is low, you are underfitting. If training score is much higher than validation score, you are overfitting. The best case is when training and validation scores are close enough.
$endgroup$
– pythinker
6 hours ago






$begingroup$
Yes, that’s actually a big training dataset. To ensure that you are not underfitting or overfitting, you should check the performance of your model on the training and validation dataset, simultaneously. If it training score is low, you are underfitting. If training score is much higher than validation score, you are overfitting. The best case is when training and validation scores are close enough.
$endgroup$
– pythinker
6 hours ago













1












$begingroup$

I was taught the best way is to find the error for each k then plot them and look for the "elbow" on the plot.






share|improve this answer








New contributor




Stephen Ewing is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.






$endgroup$













  • $begingroup$
    So I used go with k=131
    $endgroup$
    – user214
    7 hours ago










  • $begingroup$
    It really depends. The higher your k the higher your chance of overfitting. So if you do every k from 2 to 200 and plot the error of all of them you use the k where the curve starts to flatten out.
    $endgroup$
    – Stephen Ewing
    7 hours ago
















1












$begingroup$

I was taught the best way is to find the error for each k then plot them and look for the "elbow" on the plot.






share|improve this answer








New contributor




Stephen Ewing is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.






$endgroup$













  • $begingroup$
    So I used go with k=131
    $endgroup$
    – user214
    7 hours ago










  • $begingroup$
    It really depends. The higher your k the higher your chance of overfitting. So if you do every k from 2 to 200 and plot the error of all of them you use the k where the curve starts to flatten out.
    $endgroup$
    – Stephen Ewing
    7 hours ago














1












1








1





$begingroup$

I was taught the best way is to find the error for each k then plot them and look for the "elbow" on the plot.






share|improve this answer








New contributor




Stephen Ewing is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.






$endgroup$



I was taught the best way is to find the error for each k then plot them and look for the "elbow" on the plot.







share|improve this answer








New contributor




Stephen Ewing is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.









share|improve this answer



share|improve this answer






New contributor




Stephen Ewing is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.









answered 7 hours ago









Stephen EwingStephen Ewing

112




112




New contributor




Stephen Ewing is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.





New contributor





Stephen Ewing is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.






Stephen Ewing is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.












  • $begingroup$
    So I used go with k=131
    $endgroup$
    – user214
    7 hours ago










  • $begingroup$
    It really depends. The higher your k the higher your chance of overfitting. So if you do every k from 2 to 200 and plot the error of all of them you use the k where the curve starts to flatten out.
    $endgroup$
    – Stephen Ewing
    7 hours ago


















  • $begingroup$
    So I used go with k=131
    $endgroup$
    – user214
    7 hours ago










  • $begingroup$
    It really depends. The higher your k the higher your chance of overfitting. So if you do every k from 2 to 200 and plot the error of all of them you use the k where the curve starts to flatten out.
    $endgroup$
    – Stephen Ewing
    7 hours ago
















$begingroup$
So I used go with k=131
$endgroup$
– user214
7 hours ago




$begingroup$
So I used go with k=131
$endgroup$
– user214
7 hours ago












$begingroup$
It really depends. The higher your k the higher your chance of overfitting. So if you do every k from 2 to 200 and plot the error of all of them you use the k where the curve starts to flatten out.
$endgroup$
– Stephen Ewing
7 hours ago




$begingroup$
It really depends. The higher your k the higher your chance of overfitting. So if you do every k from 2 to 200 and plot the error of all of them you use the k where the curve starts to flatten out.
$endgroup$
– Stephen Ewing
7 hours ago


















draft saved

draft discarded




















































Thanks for contributing an answer to Data Science Stack Exchange!


  • Please be sure to answer the question. Provide details and share your research!

But avoid …



  • Asking for help, clarification, or responding to other answers.

  • Making statements based on opinion; back them up with references or personal experience.


Use MathJax to format equations. MathJax reference.


To learn more, see our tips on writing great answers.




draft saved


draft discarded














StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f48905%2fchoosing-k-value-in-knn-classifier%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown





















































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown

































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown







-

Popular posts from this blog

Ponta tanko

Image
Usona ponta tanko M60A1 Ponta tanko estas, simile kiel kirasita savveturilo, inter specialaj kirasitaj transportiloj. Kiel pontaj tankoj eblas konsideri ĉiujn memmovigaj rimedoj ebligantaj superi obstaklojn ĝis larĝeco 24 metroj rekte en spaco de bataloperacoj. Ties ĉefa tasko estas ebligi al la propraj taĉmentoj daŭrigi en rapida antaŭenirado sen senbezona reteniĝo. El tio pli-malpli rezultas taktikaj-teknikaj postuloj, kiujn ili devas plenumi. Precipe temas pri kapablo operacii komune kun unue-sinsekvaj taĉmentoj kaj transpontigi obstaklojn dum kelke da minutoj. This page is only for reference, If you need detailed information, please check here
Read more

Tantalo (mitologio)

Image
Ĉi tiu artikolo temas pri mitologio. Por aliaj signifoj vidu la paĝon Tantalo (apartigilo) . Tantalo , laŭ Gioacchino Assereto. Laŭ la helena mitologio, Tantalo (el la greka Τάνταλος , ŝancelanta aŭ tre malfeliĉa ) estis filo de Zeŭso kaj la oceanidino Pluto, laŭ la plej divastigata versio. Li estis reĝo de Frigio [1] aŭ de la monto Sipilo, en Lidio (Malgrandazio) [2] kaj post sia morto li suferis eternan torturadon ĉe la Tartaro, la plej profunda parto de Hadeso, rezervita por la plej perversaj homoj. Tamen, kelkaj rakontoj asertas ke li estis reĝo de Argo aŭ de Korinto [3] , aŭ ke li migris norden el la monto Sipilo por akiri la tronon de Paflagonio, el kie Ilo forpelis lin pro tio, ke Tantalo forkaptis kaj seksperfortis lian fraton Ganimedon [4] , kaj tiel ofendis la diojn. Verŝajne li ankaŭ regis urbon en okcidenta Anatolio, kiun oni nomis Tantalis [5] pro li, aŭ Sipilo , pro la monto sur kies flanko ankoraŭ staras ĝiaj ruinaĵoj [6] . Laŭ la geografiisto Strabo,
Read more

Erzsébet Schaár

Image
Erzsébet Schaár Memportreto de Erzsébet Schaár en Pécs Personaj informoj Naskiĝo 27-an de julio 1908 ( 1908-07-27 ) en Budafok Morto 29-an de aŭgusto 1975 ( 1975-08-29 ) (67-jara) en Budapeŝto Ŝtataneco Hungario Profesio skulptisto Familianoj Edz(in)o Tibor Vilt v   •   d   •   r Erzsébet Schaár [erĵEbet ŝAr], laŭ hungarlingve kutima sinsekvo Schaár Erzsébet estis hungara skulptisto. Ŝia edzo estis Tibor Vilt. Déryné Széppataki Róza en Miskolc fare de Erzsébet Schaár Erzsébet Schaár [1] naskiĝis la 27-an de julio 1908 en Budafok, ŝi mortis la 29-an de aŭgusto 1975 en Budapeŝto. Enhavo 1 Biografio 2 Elektitaj solaj ekspozicioj 3 Elektitaj kolektivaj ekspozicioj 4 Elektitaj skulptaĵoj 5 Elektitaj disĉiploj 6 Fontoj 7 Referencoj Biografio | Erzsébet Schaár studis en Industriarta Lernejo, poste en Reĝa Belarta Altlernejo (1924-1926). Ŝia ĉefinstruisto estis Zsig
Read more