Machine Learning for User Modeling in an Interactive Genetic Algorithm for the Next Release Problem Allysson Allex Ara´ ujo and Matheus Paix˜ao State University of Cear´a - UECE Brazil
August 26, 2014
(goes.uece.br)
ML for User Modeling in an IGA for the NRP
August 26, 2014
1 / 20
Summary
1
Introduction
2
An Interactive Genetic Algorithm for the Next Release Problem
3
Empirical Study
4
A Machine Learning Approach for User Modelling
5
Conclusions and Future Work
(goes.uece.br)
ML for User Modeling in an IGA for the NRP
August 26, 2014
2 / 20
Introduction
Introduction
During an iterative and incremental software development process, there are some problems that are complex to deal with. It may be mentioned the widely known Next Release Problem (NRP), which is based on the maximization of the customer satisfaction while respecting a predefined budget. The current mono-objective SBSE approaches to the NRP can be considered as decision-making tools, where the requirements’ selection for the next release is automatically made. Certain reluctance from the users to accept such results.
(goes.uece.br)
ML for User Modeling in an IGA for the NRP
August 26, 2014
3 / 20
Introduction
Introduction The Interactive Evolutionary Computation, is supported by two components: human evaluation and computational search through GAs. The IGA follows the same concepts of a traditional GA, the difference is regarded to the solution evaluation process. Despite the human involvement being interesting and attractive to the search process, it is also the cause the human fatigue. This paper proposes the use of a machine learning technique to model the requirements engineer preferences during the use of the IGA for the NRP. (goes.uece.br)
ML for User Modeling in an IGA for the NRP
August 26, 2014
4 / 20
An Interactive Genetic Algorithm for the Next Release Problem
An Interactive Genetic Algorithm for the NRP Consider a set of requirements R = {r1 , r2 , . . . , rN }. Each requirement ri has an importance value vi and an effort cost ci . maximize: α.score(X ) + β.she(X )
(1)
subject to: cost(X ) ≤ budget
(2)
where, score(X ) =
cost(X ) =
N X
vi xi
(3)
i=1 N X
ci xi
(4)
i=1
The decision variable xi = 1 indicates that ri is included in the next release and xi = 0 otherwise. (goes.uece.br)
ML for User Modeling in an IGA for the NRP
August 26, 2014
5 / 20
Empirical Study
Empirical Study Settings In order to represent the requirements engineer, a human simulator was developed. Based on a certain evaluation profile, the simulator defines a “target-individual�, which represents what the requirements engineer would consider as an optimal release. The requirements to be included in the target-individual are chosen based on a certain percentage. Three different profiles were used: Random: the requirements in the target-individual are randomly defined. Lower Score: the target-individual is formed by the requirements with least score. Higher Cost: the target-individual is formed by the requirements with highest cost. (goes.uece.br)
ML for User Modeling in an IGA for the NRP
August 26, 2014
6 / 20
Empirical Study
Empirical Study Settings
The evaluations for each individual are provided according to the similarity to the target-individual. If the individual’s requirements are: totally different from target-individual, the evaluation is minimal. equal to the target-individual, the evaluation is maximum. the evaluation is proportionally given for the other possibilities.
For each weights configuration and instance, was collected the: average of the non-normalized score(X). average similarity degree of the final solution.
(goes.uece.br)
ML for User Modeling in an IGA for the NRP
August 26, 2014
7 / 20
Empirical Study
Empirical Study Settings
The similarity degree represents how similar is a candidate solution to the target-individual. Consider a set of 6 requirements with a target-individual [100011]. The possible solution [110110], for example, presents 3 equal requirements to the target-individual (r1 , r3 and r5 ). So this solution would present a similarity degree of 3/6 = 0.5.
(goes.uece.br)
ML for User Modeling in an IGA for the NRP
August 26, 2014
8 / 20
Empirical Study
Empirical Study Settings
Therefore, the empirical study was conducted in order to answer the following research question: RQ: What is the influence of the evaluation profile in the search process?
(goes.uece.br)
ML for User Modeling in an IGA for the NRP
August 26, 2014
9 / 20
Empirical Study
Results and Analyzes
Figure: Empirical Study Results - Score [ I 50 ] (goes.uece.br)
ML for User Modeling in an IGA for the NRP
August 26, 2014
10 / 20
Empirical Study
Results and Analyzes
Figure: Empirical Study Results - Score [ I 100 ] (goes.uece.br)
ML for User Modeling in an IGA for the NRP
August 26, 2014
11 / 20
Empirical Study
Results and Analyzes
Figure: Empirical Study Results - Similarity Degree [ I 50 ] (goes.uece.br)
ML for User Modeling in an IGA for the NRP
August 26, 2014
12 / 20
Empirical Study
Results and Analyzes
Figure: Empirical Study Results - Similarity Degree [ I 100 ] (goes.uece.br)
ML for User Modeling in an IGA for the NRP
August 26, 2014
13 / 20
Empirical Study
Results and Analyzes
Given these results, it is stated the final solutions are considerably influenced by the evaluations profiles, in a way they tend to get closer to the target-individuals, answering the research question.
(goes.uece.br)
ML for User Modeling in an IGA for the NRP
August 26, 2014
14 / 20
A Machine Learning Approach for User Modelling
A Machine Learning Approach for User Modelling The IGA proposed is capable of incorporating the user preferences in the search process. But there is the human fatigue problem, which makes an interactive approach unfeasible when the number of interactions is high. In order to handle this difficulty, this paper proposes a machine learning technique to model the user evaluation profile. The learning model would use the individuals, and the respective human evaluations, as a training set, replacing the requirements engineer after a while. (goes.uece.br)
ML for User Modeling in an IGA for the NRP
August 26, 2014
15 / 20
A Machine Learning Approach for User Modelling
A Machine Learning Approach for User Modelling
Figure: The Architecture of the IGA with a Learning Model
(goes.uece.br)
ML for User Modeling in an IGA for the NRP
August 26, 2014
16 / 20
A Machine Learning Approach for User Modelling
A Machine Learning Approach for User Modelling
When incorporating a learning model to the IGA, it is expected that: the results remain consistent to the evaluation profile. significantly decreasing the number of questions to the requirements engineer.
Currently, the learning model is under development and it is expected to be tested with different learning techniques to assess which adapts better to the proposal.
(goes.uece.br)
ML for User Modeling in an IGA for the NRP
August 26, 2014
17 / 20
Conclusions and Future Work
Conclusions and Future Work When search techniques are used to tackle such problems, it becomes difficult to incorporate the user’s preferences to the process The objective of this work was to develop a feasible IGA proposal as a solution to the NRP, but it could also deal with the human fatigue. It is proposed a machine learning technique, which is able to model the requirements engineer preferences and replace him throughout the search process. Intermediate IGA results show that it is able to successfully incorporate the requirements engineer knowledge in the final solutions. (goes.uece.br)
ML for User Modeling in an IGA for the NRP
August 26, 2014
18 / 20
Conclusions and Future Work
Conclusions and Future Work
As future works, it is expected to: finalize the implementation and tests related to the learning model; assess different ways in which the requirements engineer can evaluate the solutions consider interdependencies between requirements; apply the IGA proposed for multiobjective version of the NRP.
(goes.uece.br)
ML for User Modeling in an IGA for the NRP
August 26, 2014
19 / 20
Conclusions and Future Work
Thank You!
Questions?
Optimization Group in Software Engineering goes.uece.br
(goes.uece.br)
ML for User Modeling in an IGA for the NRP
August 26, 2014
20 / 20