Comments (5)
The authors introduce an approach to automatically generate tests to detect redundant loop traversals in Java, aiming to effectively locate potential performance issues.
The artifact works as advertised. Everything is easy to install and all the instructions are straightforward. The authors have done an excellent job documenting everything at an appropriate level of detail. The artifact is very useful since it allows the generation of tests on the fly for any source code. I would encourage the authors to also think of making their work available as a web service or an IDE plugin.
I'm happy to recommend the inclusion of this dataset in the online repository, the social media campaign and the award session.
from fse16.
This artifact consists of a technique that addresses the problem of automatically generating tests to detect redundant loop traversals. According to the authors, redundant loop traversal is one of the primary sources for performance issues in many Java libraries.
Insightful
The idea of the paper is interesting, timely, and relevant for this FSE 2016 artifact track.
The paper is well motivated and the problem is clearly stated.
Useful
The authors leverage dynamic analysis to generate targeted tests that expose loop inefficiencies. The aim is to overcome shortcomings of previous works that heavily rely on manual work and analysis of classes, which is tedious, impractical and time consuming.
The technique seems to be cost effective.
Usable
The authors adhere to the FSE artifact track guidelines and particularly the three required dimensions: insightfulness, usefulness, and usability.
The given artifact consists of a script to execute all the benchmarks.
Details related to the artifact package are provided including the hardware requirements, installation, and experimental setup.
The artifact is available online and provided in the form of compressed tar.gz file.
Additional Comments:
In section 1.2, the authors claim that their technique enhances the effectiveness of the state of the art approaches to detect similar problems. Did you run a comparative study to draw such a conclusion and—or confirm these findings? If yes, please add a reference to the paper/study.
What about this effectiveness? How it was measured? And what about the results in terms of numbers? Could you please add information about how accurate is the suggested technique?
The authors state that their technique has detected a number of bugs across several Java libraries. And that the reported bugs are confirmed and fixed by developers. Could you please add information about this? Plus, a reference to the work?
You mentioned “benchmarks” or “all benchmarks” across the paper. Please add a note in the paper about what these benchmarks are/consists of.
Minor:
In section 1.2: please replace “targetted tests” by “targeted tests”.
Overall, I believe this artifact can be of great value to the SE community especially the one working in the testing area. And I therefore mark it as an accept. I evaluate it as ‘Maybe Gold’.
from fse16.
@Latifa-Guerrouj -- Thanks for your review.
- For the concerns mentioned under additional comments, is it a review of the paper or that of the artifact?
- Aren't these details already available in the Experimental Section (Sec 5) of the paper?
- Is the suggestion to include these details once again in the writeup of the artifact?
- Was there any problem in reproducing the results mentioned in the paper?
from fse16.
The artifact is a package to reproduce the experiments in the
corresponding paper. A detailed tutorial to reproduce the experiments
is included. The artifact also includes instructions on configuring a
virtual machine.
I found the instructions easy to follow and could reproduce the
experiments.
The artifact is necessary to support the empirical results used in the
paper and might be useful for researchers interested in conducting
additional experiments.
from fse16.
Note these labels are still "under discussion" and are still subject to change prior to the final notifications Friday.
from fse16.
Related Issues (20)
- Sung_JSdep HOT 8
- Yoga_PTRacer HOT 9
- Xu_PyPredictive HOT 4
- Su_Dyclink HOT 23
- Xu_PyProbaTyping HOT 4
- silva_whyWeRefactorDataset HOT 5
- Sui_SUPA HOT 6
- reif_libcg HOT 9
- Wang_WebRanz HOT 9
- ApelEtal_SpecificationDecomposition HOT 4
- for olga: next steps HOT 4
- Wang_FPSO HOT 8
- Regarding open review, if done again, I would ALWAYS....
- Regarding open review, if done again, I would NEVER....
- Regarding open review, the most USELESS thing was....
- Regarding open review, the most USEFUL thing was....
- Regarding open review: overall, what went WRONG was....
- Regarding open review: overall, what went RIGHT was....
- survey on value (or otherwise) of open review
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from fse16.