rar-mfs is an algorithm for feature selection and can be employed to select features from labled data sets.
The Relevance and Redundancy Framework (RaR), which is the theory behind the implementation, is a novel feature selection algorithm that
- works on large data sets (polynomial runtime),
- can handle differently typed features (e.g. nominal features and continoues features), and
- handles multivariate correlations.
The tool is written in scala and uses the weka framework to load and handle data sets. You can either run it independently providing the data as an .arff
or .csv
file or you can include the algorithm as a (maven / ivy) dependency in your project. As an example data set we use heart-c.
The project is published to maven central (link). To depend on the project use:
-
maven
<dependency> <groupId>de.hpi.kddm</groupId> <artifactId>rar-mfs_2.11</artifactId> <version>1.0.2</version> </dependency>
-
sbt:
libraryDependencies += "de.hpi.kddm" %% "rar-mfs" % "1.0.2"
To run the algorithm use
import de.hpi.kddm.rar._
// ...
val dataSet = de.hpi.kddm.rar.Runner.loadCSVDataSet(new File("heart-c.csv", isNormalized = false, "")
val algorithm = new RaRSearch(
HicsContrastPramsFA(numIterations = config.samples, maxRetries = 1, alphaFixed = config.alpha, maxInstances = 1000),
RaRParamsFixed(k = 5, numberOfMonteCarlosFixed = 5000, parallelismFactor = 4))
algorithm.selectFeatures(dataSet)
-
EITHER download the prebuild binary which requires only an installation of a recent java version (>= 6)
- download the prebuild jar from the releases tab (latest)
- run
java -jar rar-mfs-1.0.2.jar--help
Using the prebuild jar, here is an example usage:
rar-mfs > java -jar rar-mfs-1.0.2.jar arff --samples 100 --subsetSize 5 --nonorm heart-c.arff Feature Ranking: 1 - age (12) 2 - sex (8) 3 - cp (11) ...
-
OR build the repository on your own:
- make sure sbt is installed
- clone repository
- run
sbt run
Simple example using sbt directly after cloning the repository:
rar-mfs > sbt "run arff --samples 100 --subsetSize 5 --nonorm heart-c.arff" Feature Ranking: 1 - age (12) 2 - sex (8) 3 - cp (11) ...
To speed up the algorithm, consider using a fast solver such as Gurobi (http://www.gurobi.com/). Install the solver and put the provided gurobi.jar
into the java classpath.
Abstract overview of the different steps of the proposed feature selection algorithm:
The Relevance and Redundancy ranking framework (RaR) is a method able to handle large scale data sets and data sets with mixed features. Instead of directly selecting a subset, a feature ranking gives a more detailed overview into the relevance of the features.
The method consists of a multistep approach where we
-
repeatedly sample subsets from the whole feature space and examine their relevance and redundancy: exploration of the search space to gather more and more knowledge about the relevance and redundancy of features
-
decude scores for features based on the scores of the subsets
-
create the best possible ranking given the sampled insights.
Parameter | Default value | Description |
---|---|---|
m - contrast iterations | 100 | Number of different slices to evaluate while comparing marginal and conditional probabilities |
alpha - subspace slice size | 0.01 | Percentage of all instances to use as part of a slice which is used to compare distributions |
n - sampling itertations | 1000 | Number of different subsets to select in the sampling phase |
k - sample set size | 5 | Maximum size of the subsets to be selected in the sampling phase |
Current license is CC BY-NC 4.0. The complete license can be found at creativecommons. For commercial licensing please contact the author.