Reinforcement learning - generating a matrix of continuous values with varying size for test data generation
$begingroup$
Currently, I am using RL A3C algorithm for test data generation, where for a set of 30 functions written in C (mostly basic algorithms like Prime number checks, triangle validity, etc.) I try to generate a set of test data to have 100% coverage.
Now for this, I need to generate a matrix of these test data. To explain, lets say that we have 3 execution paths for the function and the function takes two parameters of integer. This means, that to have 100% code coverage, I need 3 function calls with different function parameters, hence a matrix of test data 3x2. But in reality, I can have a matrix of 12x5 and more, as the size of the matrix is varying.
What I have done now is, that with the help of this implementation: https://github.com/stefanbo92/A3C-Continuous of A3C I can generate a continuous value for one of the parameters of the function calls. I iterate through the functions, process the ASTs using a convolutional layer and generate a value for one cell in the matrix using a LSTM cell. Then I create the tests using this matrix and retrieve the coverage value. If the coverage is 90%+, I return a reward of 1, otherwise I return 0. I have also tried returning the coverage value as a reward, but with little to no changes in results.
Right now, I try to change the value separately for one cell in the matrix at a time and then generate a test and then get the coverage value. I think that this doesn't allow the agent to efficiently train on the data, as the action is only associated to the one value of the matrix, not the matrix in general (or the current row of the matrix). The current results only confirm this, as after around 1200 training episodes the coverage and reward stagnated and even dropped from the initial value.
As such, I would like my agent to not generate a continuous integer value for just one cell of the matrix, but ideally the entire matrix as one action.
I have no clue how to do this for varying matrix size, which I require, and I thought of using some sort of encoding/decoding, but have no clue as to have it could be achieved.
Any kind of advice is appreciated as to how to deal with this issue.
Thanks in advance.
neural-network reinforcement-learning recurrent-neural-net markov-process actor-critic
asked 14 mins ago
CegyCegy
61
$endgroup$
add a comment |
$begingroup$
Currently, I am using RL A3C algorithm for test data generation, where for a set of 30 functions written in C (mostly basic algorithms like Prime number checks, triangle validity, etc.) I try to generate a set of test data to have 100% coverage.
Now for this, I need to generate a matrix of these test data. To explain, lets say that we have 3 execution paths for the function and the function takes two parameters of integer. This means, that to have 100% code coverage, I need 3 function calls with different function parameters, hence a matrix of test data 3x2. But in reality, I can have a matrix of 12x5 and more, as the size of the matrix is varying.
What I have done now is, that with the help of this implementation: https://github.com/stefanbo92/A3C-Continuous of A3C I can generate a continuous value for one of the parameters of the function calls. I iterate through the functions, process the ASTs using a convolutional layer and generate a value for one cell in the matrix using a LSTM cell. Then I create the tests using this matrix and retrieve the coverage value. If the coverage is 90%+, I return a reward of 1, otherwise I return 0. I have also tried returning the coverage value as a reward, but with little to no changes in results.
Right now, I try to change the value separately for one cell in the matrix at a time and then generate a test and then get the coverage value. I think that this doesn't allow the agent to efficiently train on the data, as the action is only associated to the one value of the matrix, not the matrix in general (or the current row of the matrix). The current results only confirm this, as after around 1200 training episodes the coverage and reward stagnated and even dropped from the initial value.
As such, I would like my agent to not generate a continuous integer value for just one cell of the matrix, but ideally the entire matrix as one action.
I have no clue how to do this for varying matrix size, which I require, and I thought of using some sort of encoding/decoding, but have no clue as to have it could be achieved.
Any kind of advice is appreciated as to how to deal with this issue.
Thanks in advance.
neural-network reinforcement-learning recurrent-neural-net markov-process actor-critic
asked 14 mins ago
CegyCegy
61
$endgroup$
add a comment |
$begingroup$
Currently, I am using RL A3C algorithm for test data generation, where for a set of 30 functions written in C (mostly basic algorithms like Prime number checks, triangle validity, etc.) I try to generate a set of test data to have 100% coverage.
Now for this, I need to generate a matrix of these test data. To explain, lets say that we have 3 execution paths for the function and the function takes two parameters of integer. This means, that to have 100% code coverage, I need 3 function calls with different function parameters, hence a matrix of test data 3x2. But in reality, I can have a matrix of 12x5 and more, as the size of the matrix is varying.
What I have done now is, that with the help of this implementation: https://github.com/stefanbo92/A3C-Continuous of A3C I can generate a continuous value for one of the parameters of the function calls. I iterate through the functions, process the ASTs using a convolutional layer and generate a value for one cell in the matrix using a LSTM cell. Then I create the tests using this matrix and retrieve the coverage value. If the coverage is 90%+, I return a reward of 1, otherwise I return 0. I have also tried returning the coverage value as a reward, but with little to no changes in results.
Right now, I try to change the value separately for one cell in the matrix at a time and then generate a test and then get the coverage value. I think that this doesn't allow the agent to efficiently train on the data, as the action is only associated to the one value of the matrix, not the matrix in general (or the current row of the matrix). The current results only confirm this, as after around 1200 training episodes the coverage and reward stagnated and even dropped from the initial value.
As such, I would like my agent to not generate a continuous integer value for just one cell of the matrix, but ideally the entire matrix as one action.
I have no clue how to do this for varying matrix size, which I require, and I thought of using some sort of encoding/decoding, but have no clue as to have it could be achieved.
Any kind of advice is appreciated as to how to deal with this issue.
Thanks in advance.
neural-network reinforcement-learning recurrent-neural-net markov-process actor-critic
asked 14 mins ago
CegyCegy
61
$endgroup$
Currently, I am using RL A3C algorithm for test data generation, where for a set of 30 functions written in C (mostly basic algorithms like Prime number checks, triangle validity, etc.) I try to generate a set of test data to have 100% coverage.
Now for this, I need to generate a matrix of these test data. To explain, lets say that we have 3 execution paths for the function and the function takes two parameters of integer. This means, that to have 100% code coverage, I need 3 function calls with different function parameters, hence a matrix of test data 3x2. But in reality, I can have a matrix of 12x5 and more, as the size of the matrix is varying.
What I have done now is, that with the help of this implementation: https://github.com/stefanbo92/A3C-Continuous of A3C I can generate a continuous value for one of the parameters of the function calls. I iterate through the functions, process the ASTs using a convolutional layer and generate a value for one cell in the matrix using a LSTM cell. Then I create the tests using this matrix and retrieve the coverage value. If the coverage is 90%+, I return a reward of 1, otherwise I return 0. I have also tried returning the coverage value as a reward, but with little to no changes in results.
Right now, I try to change the value separately for one cell in the matrix at a time and then generate a test and then get the coverage value. I think that this doesn't allow the agent to efficiently train on the data, as the action is only associated to the one value of the matrix, not the matrix in general (or the current row of the matrix). The current results only confirm this, as after around 1200 training episodes the coverage and reward stagnated and even dropped from the initial value.
As such, I would like my agent to not generate a continuous integer value for just one cell of the matrix, but ideally the entire matrix as one action.
I have no clue how to do this for varying matrix size, which I require, and I thought of using some sort of encoding/decoding, but have no clue as to have it could be achieved.
Any kind of advice is appreciated as to how to deal with this issue.
Thanks in advance.
neural-network reinforcement-learning recurrent-neural-net markov-process actor-critic
neural-network reinforcement-learning recurrent-neural-net markov-process actor-critic
asked 14 mins ago
CegyCegy
61
asked 14 mins ago
CegyCegy
61
asked 14 mins ago
CegyCegy
61
asked 14 mins ago
CegyCegy
61
asked 14 mins ago
CegyCegy
61
61
add a comment |
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function () {
return StackExchange.using("mathjaxEditing", function () {
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
});
});
}, "mathjax-editing");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "557"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f47201%2freinforcement-learning-generating-a-matrix-of-continuous-values-with-varying-s%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Data Science Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f47201%2freinforcement-learning-generating-a-matrix-of-continuous-values-with-varying-s%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
