The ABM collections
The ABM collections are a sets of real-world, open-source collections that can be used for testing software tools and analyses.Collection 1: Java business web applications
Version 1.0 of this collection was created on February 16th, 2016, from GitHub.It contains 139 projects, and 100 executables
Full download links: projects - executables
An index of the projects can be found here. It lists the available projects and executables, with metadata such as libraries and build information.
The source code of each project should contain the precise licence information. If not, please refer to the corresponding GitHub webpage.
The scripts used to extract this dataset are available here.
The libraries whitelist is provided with the index of the projects.
Why was ABM created?
The ABM is a methodology designed to make benchmark creation and maintenance easier. By creating this methodology, we wish to address the following concerns:- Outdated benchmark suites: Well-established benchmark suites are often created by hand, and remain unchanged for years. As a result, analyses and tools are tested on outdated code, which is not representative of real-world software. By automatically constructing the collection, the code base can be regularly updated to provide up-to-date test projects.
- Analysis/tool scope: Benchmark suites are very general, and aim to cover many different types of software. On the other hand, analyses and tools are sometimes targeted at specific programs, such as Java web applications for example. Evaluating them on other types of software is insubstantial. When mining GitHub, the scope of the collection can be restricted to the target of the analysis/tool, and provide an adapted, yet still representative collection of up-to-date applications. The current collection targets Java web applications, and the same automated mining methodology can be applied to different types of target programs.
Can I contribute?
With the ABM methodology, we strive to create more dynamic, easily updatable benchmark suites that are adapted to the needs of different target domains. The overhead of this semi-automated methodology is the manual creation of the scripts and filters. We are happy to collect and share scripts and filters on this webpage. To submit your own ABM scripts, please contact us by email.Contact information
Lisa Nguyen Quang Do (lisa.nguyen@iem.fraunhofer.de)Michael Eichberg (eichberg@informatik.tu-darmstadt.de)
Eric Bodden (eric.bodden@upb.de)
Impressum
This research was supported by a Fraunhofer Attract grant as well as the Heinz Nixdorf Foundation.