Are there real world applications where deep fully connected networks are better suited than ConvNets
$begingroup$
I would like to give some brief background for my question to avoid answers that explain the difference between fully connected nets and ConvNets.
I completed the first 3 courses in the deep learning specialization from deeplearning.ai (Coursera). Simultaneously I've worked through the first two home work assignments from Stanford's CS321n course for the Spring of 2017, as well as Michael Nielson's book on deep learning.
I believe I have a reasonable understanding of the architecture of both types of networks, and how to write them in python/tensoflow. I also implemented my own very simple fully connected feedfoward net in C where I was able to get 98% on MNIST.
I've also done some reading about this question in terms of which type of data is better suited for a ConvNet, and the answer I've come up with is similar to:
If the order of the input features matters to their meaning, then a
ConvNet will work best, but if you replace the position of one column
with another and the meaning is still intact, then a ConvNet will not
work.
This means that spreadsheet data (if I've understood correctly) is not suited for a ConvNet.
Unfortunately I don't know any deep learning practitioners to ask this question: are there fully connected deep neural networks in the wild that are currently in use? The best example I can think of is financial data. If that's true, what are some others?
deep-learning convnet deep-network
asked Jan 6 at 23:34
Sam HammamySam Hammamy
1062
$endgroup$
bumped to the homepage by Community♦ 1 hour ago
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
add a comment |
$begingroup$
I would like to give some brief background for my question to avoid answers that explain the difference between fully connected nets and ConvNets.
I completed the first 3 courses in the deep learning specialization from deeplearning.ai (Coursera). Simultaneously I've worked through the first two home work assignments from Stanford's CS321n course for the Spring of 2017, as well as Michael Nielson's book on deep learning.
I believe I have a reasonable understanding of the architecture of both types of networks, and how to write them in python/tensoflow. I also implemented my own very simple fully connected feedfoward net in C where I was able to get 98% on MNIST.
I've also done some reading about this question in terms of which type of data is better suited for a ConvNet, and the answer I've come up with is similar to:
If the order of the input features matters to their meaning, then a
ConvNet will work best, but if you replace the position of one column
with another and the meaning is still intact, then a ConvNet will not
work.
This means that spreadsheet data (if I've understood correctly) is not suited for a ConvNet.
Unfortunately I don't know any deep learning practitioners to ask this question: are there fully connected deep neural networks in the wild that are currently in use? The best example I can think of is financial data. If that's true, what are some others?
deep-learning convnet deep-network
asked Jan 6 at 23:34
Sam HammamySam Hammamy
1062
$endgroup$
bumped to the homepage by Community♦ 1 hour ago
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
$begingroup$
Are neural nets with dense layers in use? yes of course, that's one of the simplest possible layers, and found in almost any neural net architecture somewhere. CNNs also use them in additional to convolutional layers. What are you asking beyond that?
$endgroup$
– Sean Owen♦
Jan 8 at 1:20
add a comment |
$begingroup$
I would like to give some brief background for my question to avoid answers that explain the difference between fully connected nets and ConvNets.
I completed the first 3 courses in the deep learning specialization from deeplearning.ai (Coursera). Simultaneously I've worked through the first two home work assignments from Stanford's CS321n course for the Spring of 2017, as well as Michael Nielson's book on deep learning.
I believe I have a reasonable understanding of the architecture of both types of networks, and how to write them in python/tensoflow. I also implemented my own very simple fully connected feedfoward net in C where I was able to get 98% on MNIST.
I've also done some reading about this question in terms of which type of data is better suited for a ConvNet, and the answer I've come up with is similar to:
If the order of the input features matters to their meaning, then a
ConvNet will work best, but if you replace the position of one column
with another and the meaning is still intact, then a ConvNet will not
work.
This means that spreadsheet data (if I've understood correctly) is not suited for a ConvNet.
Unfortunately I don't know any deep learning practitioners to ask this question: are there fully connected deep neural networks in the wild that are currently in use? The best example I can think of is financial data. If that's true, what are some others?
deep-learning convnet deep-network
asked Jan 6 at 23:34
Sam HammamySam Hammamy
1062
$endgroup$
I would like to give some brief background for my question to avoid answers that explain the difference between fully connected nets and ConvNets.
I completed the first 3 courses in the deep learning specialization from deeplearning.ai (Coursera). Simultaneously I've worked through the first two home work assignments from Stanford's CS321n course for the Spring of 2017, as well as Michael Nielson's book on deep learning.
I believe I have a reasonable understanding of the architecture of both types of networks, and how to write them in python/tensoflow. I also implemented my own very simple fully connected feedfoward net in C where I was able to get 98% on MNIST.
I've also done some reading about this question in terms of which type of data is better suited for a ConvNet, and the answer I've come up with is similar to:
If the order of the input features matters to their meaning, then a
ConvNet will work best, but if you replace the position of one column
with another and the meaning is still intact, then a ConvNet will not
work.
This means that spreadsheet data (if I've understood correctly) is not suited for a ConvNet.
Unfortunately I don't know any deep learning practitioners to ask this question: are there fully connected deep neural networks in the wild that are currently in use? The best example I can think of is financial data. If that's true, what are some others?
deep-learning convnet deep-network
deep-learning convnet deep-network
asked Jan 6 at 23:34
Sam HammamySam Hammamy
1062
asked Jan 6 at 23:34
Sam HammamySam Hammamy
1062
asked Jan 6 at 23:34
Sam HammamySam Hammamy
1062
asked Jan 6 at 23:34
Sam HammamySam Hammamy
1062
asked Jan 6 at 23:34
Sam HammamySam Hammamy
1062
1062
bumped to the homepage by Community♦ 1 hour ago
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
bumped to the homepage by Community♦ 1 hour ago
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
$begingroup$
Are neural nets with dense layers in use? yes of course, that's one of the simplest possible layers, and found in almost any neural net architecture somewhere. CNNs also use them in additional to convolutional layers. What are you asking beyond that?
$endgroup$
– Sean Owen♦
Jan 8 at 1:20
add a comment |
$begingroup$
Are neural nets with dense layers in use? yes of course, that's one of the simplest possible layers, and found in almost any neural net architecture somewhere. CNNs also use them in additional to convolutional layers. What are you asking beyond that?
$endgroup$
– Sean Owen♦
Jan 8 at 1:20
$begingroup$
Are neural nets with dense layers in use? yes of course, that's one of the simplest possible layers, and found in almost any neural net architecture somewhere. CNNs also use them in additional to convolutional layers. What are you asking beyond that?
$endgroup$
– Sean Owen♦
Jan 8 at 1:20
$begingroup$
Are neural nets with dense layers in use? yes of course, that's one of the simplest possible layers, and found in almost any neural net architecture somewhere. CNNs also use them in additional to convolutional layers. What are you asking beyond that?
$endgroup$
– Sean Owen♦
Jan 8 at 1:20
add a comment |
1 Answer
1
active
oldest
votes
$begingroup$
Yes, there are plenty use case where a fully connected network is more appropriate.
ConvNet works because the weights are shared in its kernels. It makes a lot intuitive sense to share the weights in image recognition task, because a flower at the top left of a picture has the same local pattern as the flower at the right bottom of the picture. What we want is to learn of pattern that is local.
However, this logic no longer applies when you consider program such as the famous iris classification problem. The idea of "local" simply doesn't exist for such data set. each of the features (i.e, petal width, petal length etc) are all "standalone" concept. In those case, it simply doesn't make sense to use ConvNet.
BTW, you can use ConvNet on financial data as well. As financial problem usually involves a time dimension, where the concepts of "local" does exist.
answered Jan 7 at 0:28
Louis TLouis T
591219
$endgroup$
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
return StackExchange.using("mathjaxEditing", function () {
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
});
});
}, "mathjax-editing");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "557"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f43595%2fare-there-real-world-applications-where-deep-fully-connected-networks-are-better%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
Yes, there are plenty use case where a fully connected network is more appropriate.
ConvNet works because the weights are shared in its kernels. It makes a lot intuitive sense to share the weights in image recognition task, because a flower at the top left of a picture has the same local pattern as the flower at the right bottom of the picture. What we want is to learn of pattern that is local.
However, this logic no longer applies when you consider program such as the famous iris classification problem. The idea of "local" simply doesn't exist for such data set. each of the features (i.e, petal width, petal length etc) are all "standalone" concept. In those case, it simply doesn't make sense to use ConvNet.
BTW, you can use ConvNet on financial data as well. As financial problem usually involves a time dimension, where the concepts of "local" does exist.
answered Jan 7 at 0:28
Louis TLouis T
591219
$endgroup$
add a comment |
$begingroup$
Yes, there are plenty use case where a fully connected network is more appropriate.
ConvNet works because the weights are shared in its kernels. It makes a lot intuitive sense to share the weights in image recognition task, because a flower at the top left of a picture has the same local pattern as the flower at the right bottom of the picture. What we want is to learn of pattern that is local.
However, this logic no longer applies when you consider program such as the famous iris classification problem. The idea of "local" simply doesn't exist for such data set. each of the features (i.e, petal width, petal length etc) are all "standalone" concept. In those case, it simply doesn't make sense to use ConvNet.
BTW, you can use ConvNet on financial data as well. As financial problem usually involves a time dimension, where the concepts of "local" does exist.
answered Jan 7 at 0:28
Louis TLouis T
591219
$endgroup$
add a comment |
$begingroup$
Yes, there are plenty use case where a fully connected network is more appropriate.
ConvNet works because the weights are shared in its kernels. It makes a lot intuitive sense to share the weights in image recognition task, because a flower at the top left of a picture has the same local pattern as the flower at the right bottom of the picture. What we want is to learn of pattern that is local.
However, this logic no longer applies when you consider program such as the famous iris classification problem. The idea of "local" simply doesn't exist for such data set. each of the features (i.e, petal width, petal length etc) are all "standalone" concept. In those case, it simply doesn't make sense to use ConvNet.
BTW, you can use ConvNet on financial data as well. As financial problem usually involves a time dimension, where the concepts of "local" does exist.
answered Jan 7 at 0:28
Louis TLouis T
591219
$endgroup$
Yes, there are plenty use case where a fully connected network is more appropriate.
ConvNet works because the weights are shared in its kernels. It makes a lot intuitive sense to share the weights in image recognition task, because a flower at the top left of a picture has the same local pattern as the flower at the right bottom of the picture. What we want is to learn of pattern that is local.
However, this logic no longer applies when you consider program such as the famous iris classification problem. The idea of "local" simply doesn't exist for such data set. each of the features (i.e, petal width, petal length etc) are all "standalone" concept. In those case, it simply doesn't make sense to use ConvNet.
BTW, you can use ConvNet on financial data as well. As financial problem usually involves a time dimension, where the concepts of "local" does exist.
answered Jan 7 at 0:28
Louis TLouis T
591219
answered Jan 7 at 0:28
Louis TLouis T
591219
answered Jan 7 at 0:28
Louis TLouis T
591219
answered Jan 7 at 0:28
Louis TLouis T
591219
591219
add a comment |
add a comment |
Thanks for contributing an answer to Data Science Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f43595%2fare-there-real-world-applications-where-deep-fully-connected-networks-are-better%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
$begingroup$
Are neural nets with dense layers in use? yes of course, that's one of the simplest possible layers, and found in almost any neural net architecture somewhere. CNNs also use them in additional to convolutional layers. What are you asking beyond that?
$endgroup$
– Sean Owen♦
Jan 8 at 1:20