# Welcome to mulearn’s documentation!¶

$$\mu$$-learn (read _mooh-learn_, code mulearn) is a python package implementing the methodology for data-driven induction of fuzzy sets described in

• D. Malchiodi and W. Pedrycz, Learning Membership Functions for Fuzzy Sets through Modified Support Vector Clustering, in F. Masulli, G. Pasi e R. Yager (Eds.), Fuzzy Logic and Applications. 10th International Workshop, WILF 2013, Genoa, Italy, November 19–22, 2013. Proceedings, Vol. 8256, Springer International Publishing, Switzerland, Lecture Notes on Artificial Intelligence, 2013;

• D. Malchiodi and A. G. B. Tettamanzi, Predicting the Possibilistic Score of OWL Axioms through Modified Support Vector Clustering, in H. Haddad, R. L. Wainwright e R. Chbeir (Eds.), SAC’18: Proceedings of the 33rd Annual ACM Symposium on Applied Computing, ACM (ISBN 9781450351911), 1984–1991, 2018.

The package API is described here below.

# mulearn¶

class mulearn.FuzzyInductor(c=1, k=GaussianKernel(), fuzzifier=ExponentialFuzzifier(), solver=GurobiSolver(), random_state=None)[source]

FuzzyInductor class.

__init__(c=1, k=GaussianKernel(), fuzzifier=ExponentialFuzzifier(), solver=GurobiSolver(), random_state=None)[source]

Create an instance of FuzzyInductor.

Parameters
fit(X, y, warm_start=False)[source]

Induce the membership function starting from a labeled sample.

Parameters
• X (iterable of float vectors having the same length) – Vectors in data space.

• y (iterable of float having the same length of X) – Membership for the vectors in X.

• warm_start (bool) – flag triggering the non reinitialization of independent variables of the optimization problem, defaults to None.

Raises

ValueError if the values in y are not between 0 and 1, if X and have different lengths, or if X contains elements of different lengths.

Returns

self – the trained model.

decision_function(X)[source]

Compute predictions for the membership function.

Parameters

X (iterable of float vectors having the same length) – Vectors in data space.

Returns

array of float – the predictions for each value in X.

predict(X, alpha=None)[source]

Compute predictions for membership to the set.

Predictions are either computed through the membership function (when alpha is set to a float in [0, 1]) or obtained via an $$\alpha$$-cut on the same function (when alpha is set to None).

Parameters
• X (iterable of float vectors having the same length) – Vectors in data space.

• alpha (float) – $$\alpha$$-cut value, defaults to None.

Raises

ValueError if alpha is set to a value different from None and not included in $$[0, 1]$$.

Returns

array of int – the predictions for each value in X.

score(X, y, **kwargs)[source]

Compute the fuzzifier score.

Score is obtained as the opposite of MSE between predicted membership values and labels.

Parameters
• X (iterable of float vectors having the same length) – Vectors in data space.

• y (iterable of float having the same length of X) – Labels containing the gold standard membership values for the vectors in X.

Returns

float – opposite of MSE between the predictions for the elements in X w.r.t. the labels in y.

# kernel¶

Kernel implementation.

This module implements the kernel used in mulearn.

class mulearn.kernel.Kernel[source]

Base kernel class.

__init__()[source]

Create an instance of Kernel.

compute(arg_1, arg_2)[source]

Compute the kernel value, given two arguments.

Parameters
• arg_1 (Object) – First kernel argument.

• arg_2 (Object) – Second kernel argument.

Raises

NotImplementedError (Kernel is abstract)

Returns

float – kernel value.

classmethod get_default()[source]

Return the default kernel.

Returns

LinearKernel() – the default kernel.

class mulearn.kernel.LinearKernel[source]

Linear kernel class.

compute(arg_1, arg_2)[source]

Compute the kernel value.

The value $$k(x_1, x_2)$$ of a linear kernel is equal to the dot product $$x_1 \cdot x_2$$, that is to $$\sum_{i=1}^n (x_1)_i (x_2)_i$$, $$n$$ being the common dimension of $$x_1$$ and $$x_2$$.

Parameters
• arg_1 (iterable of float) – First kernel argument.

• arg_2 (iterable of float) – Second kernel argument.

Returns

float – kernel value.

class mulearn.kernel.PolynomialKernel(degree)[source]

Polynomial kernel class.

__init__(degree)[source]

Create an instance of PolynomialKernel.

Parameters

degree (int) – degree of the polynomial kernel.

Raises

ValueError if degree is not an integer or if it has a negative value.

compute(arg_1, arg_2)[source]

Compute the kernel value.

The value $$k(x_1, x_2)$$ of a polynomial kernel is equal to the quantity $$(x_1 \cdot x_2 + 1)^d$$, $$d$$ being the polynomial degree of the kernel.

Parameters
• arg_1 (iterable of float) – First kernel argument.

• arg_2 (iterable of float) – Second kernel argument.

Returns

float – kernel value.

class mulearn.kernel.HomogeneousPolynomialKernel(degree)[source]

Homogeneous polynomial kernel class.

__init__(degree)[source]

Create an instance of HomogeneousPolynomialKernel.

Parameters

degree (int) – degree of the polynomial kernel.

Raises

ValueError if degree is not an integer or if it has a negative value.

compute(arg_1, arg_2)[source]

Compute the kernel value.

The value $$k(x_1, x_2)$$ of a homogeneous polynomial kernel is intended as the quantity $$(x_1 \cdot x_2)^d$$, $$d$$ being the polynomial degree of the kernel.

Parameters
• arg_1 (iterable of float) – First kernel argument.

• arg_2 (iterable of float) – Second kernel argument.

Returns

float – kernel value.

class mulearn.kernel.GaussianKernel(sigma=1)[source]

Gaussian kernel class.

__init__(sigma=1)[source]

Create an instance of GaussianKernel.

Parameters

sigma (float) – gaussian standard deviation, defaults to 1.

Raises

ValueError if sigma has a negative value.

compute(arg_1, arg_2)[source]

Compute the kernel value.

The value $$k(x_1, x_2)$$ of a gaussian kernel is intended as the quantity $$\mathrm e^{-\frac{||x_1 - x_2||^2}{2 \sigma^2}}$$, $$\sigma$$ being the kernel standard deviation.

Parameters
• arg_1 (iterable of float) – First kernel argument.

• arg_2 (iterable of float) – Second kernel argument.

Returns

float – kernel value.

class mulearn.kernel.HyperbolicKernel(scale=1, offset=0)[source]

Hyperbolic kernel class.

__init__(scale=1, offset=0)[source]

Create an instance of HyperbolicKernel.

Parameters
• scale (float) – scale constant, defaults to 1.

• offset (float) – offset constant, defaults to 0.

compute(arg_1, arg_2)[source]

Compute the kernel value.

The value $$k(x_1, x_2)$$ of a hyperbolic kernel is intended as the quantity $$\tanh(\alpha x_1 \cdot x_2 + \beta)$$, $$\alpha$$ and $$\beta$$ being the scale and offset parameters, respectively.

Parameters
• arg_1 (iterable of float) – First kernel argument.

• arg_2 (iterable of float) – Second kernel argument.

Returns

float – kernel value.

class mulearn.kernel.PrecomputedKernel(kernel_computations)[source]

Precomputed kernel class.

__init__(kernel_computations)[source]

Create an instance of PrecomputedKernel.

Parameters

kernel_computations (square matrix of float elements) – kernel computations.

Raises

ValueError if kernel_computations is not a square bidimensional array.

compute(arg_1, arg_2)[source]

Compute the kernel value.

The value of a precomputed kernel is retrieved according to the indices of the corresponding objects. Note that each index should be enclosed within an iterable in order to be compatible with sklearn.

Parameters
• arg_1 (iterable of float) – First kernel argument.

• arg_2 (iterable of float) – Second kernel argument.

Returns

float – kernel value.

# fuzzifier¶

This module implements fuzzifiers used in mulearn.

class mulearn.fuzzifier.Fuzzifier[source]

Base class for fuzzifiers.

The base class for fuzzifiers is Fuzzifier: it exposes a basic constructor which is called from the subclasses, and two methods get_membership (returning the membership function inferred from data) and get_profile computing information exploitable in order to visualize the fuzzifier in graphical form.

__init__()[source]

Create an instance of Fuzzifier.

get_membership()[source]

Return the induced membership function.

Raises

NotFittedError if fit has not been called

Returns

function – the induced membership function

get_profile(X)[source]

Return information about the learnt membership function profile.

The profile of a membership function $$\mu: X \rightarrow [0, 1]$$ is intended here as the associated function $$p: \mathbb R^+ \rightarrow [0, 1]$$ still returning membership degrees, but considering its arguments in the feature space. More precisely, if X contains the values $$x_1, \dots, x_n$$, $$R^2$$ is the function mapping any point in data space into the squared distance between its image and the center $$a$$ of the learnt fuzzy set in feature space, the function get_profile computes the following information about $$p$$:

• a list $$r_\mathrm{data} = [ R^2(x_i), i = 1, \dots, n]$$ containing the distances between the images of the points in X and $$a$$;

• a list $$\tilde{r}_\mathrm{data}$$ containing 200 possible distances between $$a$$ and the image of a point in data space, evenly distributed between $$0$$ and $$\max r_{\mathrm{data}}$$;

• a list $$e = [\hat\mu(r_i), r_i \in \tilde{r}_{\mathrm{data}}]$$ gathering the profile values for each element in $$\tilde{r}_{\mathrm{data}}$$.

This information can be used in order to graphically show the membership profile, which is always plottable, whereas the membership function isn’t mostly of the time (unless the elements in X are either one- or bidimensional vectors).

Parameters

X (iterable of float vectors having the same length) – Vectors in data space.

Returns

list – $$[r_{\mathrm{data}}, \tilde{r}_\mathrm{data}, e]$$.

class mulearn.fuzzifier.CrispFuzzifier(profile='fixed')[source]

Crisp fuzzifier.

Fuzzifier corresponding to a crisp (classical) set: membership is always equal to either $$0$$ or $$1$$.

__init__(profile='fixed')[source]

Create an instance of CrispFuzzifier.

Parameters

profile (str) – method to be used in order to build the fuzzifier profile: ‘fixed’ relies on the radius of the sphere defining the fuzzy set core, while ‘infer’ fits a generic threshold function on the provided examples.

fit(X, y)[source]

Fit the fuzzifier on training data.

Parameters
• X (iterable of float vectors having the same length) – Vectors in data space.

• y (vector of floats having the same length of X) – membership degrees of the values in X.

Raises

ValueError if self.profile is not set either to ‘fixed’ or to ‘infer’.

The fitting process is done considering a threshold-based membership function, in turn corresponding to a threshold-based profile of the form

$\begin{split}p(r) = \begin{cases} 1 & \text{if r \leq r_\text{crisp},} \\ 0 & \text{otherwise.} \end{cases}\end{split}$

The threshold $$r_\text{crisp}$$ is set to the learnt square radius of the sphere when the profile attribute of the class have been set to ‘fixed’, and induced via interpolation of X and y attributes when it is has been set to ‘infer’.

class mulearn.fuzzifier.LinearFuzzifier(profile='fixed')[source]

Crisp fuzzifier.

Fuzzifier corresponding to a fuzzy set whose membership in feature space linearly decreases from 1 to 0.

__init__(profile='fixed')[source]

Create an instance of LinearFuzzifier.

Parameters

profile (str) – method to be used in order to build the fuzzifier profile: ‘fixed’ relies on the radius of the sphere defining the fuzzy set core, while ‘infer’ fits the profile function on the provided examples.

fit(X, y)[source]

Fit the fuzzifier on training data.

Parameters
• X (iterable of float vectors having the same length) – Vectors in data space.

• y (vector of floats having the same length of X) – membership degrees of the values in X.

Raises

ValueError if self.profile is not set either to ‘fixed’ or to ‘infer’.

The fitting process is done considering a membership function linearly decreasing from $$1$$ to $$0$$, in turn corresponding to a profile having the general form

$\begin{split}p(r) = \begin{cases} 1 & \text{if r \leq r_1,} \\ l(r) & \text{if r_1 < r \leq r_0,} \\ 0 & \text{otherwise.} \end{cases}\end{split}$

The free parameters are chosen in order to guarantee continuity; moreover, when the profile attribute of the class have been set to ‘fixed’ the membership profile will be equal to 0.5 when $$r$$ is equal to the learnt square radius of the sphere, and induced via interpolation of X and y when it is has been set to ‘infer’.

class mulearn.fuzzifier.ExponentialFuzzifier(profile='fixed', alpha=- 1)[source]

Exponential fuzzifier.

Fuzzifier corresponding to a fuzzy set whose membership in feature space exponentially decreases from 1 to 0.

__init__(profile='fixed', alpha=- 1)[source]

Create an instance of ExponentialFuzzifier.

Parameters
• profile (str) – method to be used in order to build the fuzzifier profile: ‘fixed’ relies on the radius of the sphere defining the fuzzy set core, ‘infer’ fits the profile function on the provided examples, and ‘alpha’ allows for manually setting the exponential decay via the alpha parameter.

• alpha (float) – fixed exponential decay of the fuzzifier.

fit(X, y)[source]

Fit the fuzzifier on training data.

Parameters
• X (iterable of float vectors having the same length) – Vectors in data space.

• y (vector of floats having the same length of X) – membership degrees of the values in X.

Raises

ValueError if self.profile is not set either to ‘fixed’, ‘infer’, or ‘alpha’.

In this fuzzifier, the function that transforms the square distance between the center of the learnt sphere and the image of a point in the original space into a membership degree has the form

$\begin{split}\mu(r) = \begin{cases} 1 & \text{if r \leq r_1,} \\ e(r) & \text{otherwise,} \end{cases}\end{split}$

where $$e$$ is an exponential function decreasing from 1 to 0. The shape of this function is chosen so that the membership profile will be equal to 0.5 when $$r$$ is equal to the learnt square radius of the sphere, and induced via interpolation of X and y when it is has been set to ‘infer’; finally, when the parameter is set to ‘alpha’ the exponential decay of $$e$$ is manually set via the alpha parameter of the class constructor.

class mulearn.fuzzifier.QuantileConstantPiecewiseFuzzifier[source]

Quantile-based constant piecewise fuzzifier.

Fuzzifier corresponding to a fuzzy set with a piecewise constant membership function, whose steps are defined according to the quartiles of the squared distances between images of points and center of the learnt sphere.

__init__()[source]

Create an instance of QuantileConstantPiecewiseFuzzifier

fit(X, y)[source]

Fit the fuzzifier on training data.

Parameters
• X (iterable of float vectors having the same length) – Vectors in data space.

• y (vector of floats having the same length of X) – membership degrees of the values in X.

The piecewise membership function is built so that its steps are chosen according to the quartiles of square distances between images of the points in X center of the learnt sphere.

class mulearn.fuzzifier.QuantileLinearPiecewiseFuzzifier[source]

Quantile-based linear piecewise fuzzifier.

Fuzzifier corresponding to a fuzzy set with a piecewise linear membership function, whose steps are defined according to the quartiles of the squared distances between images of points and center of the learnt sphere.

__init__()[source]

Create an instance of QuantileLinearPiecewiseFuzzifier.

fit(X, y)[source]

Fit the fuzzifier on training data.

Parameters
• X (iterable of float vectors having the same length) – Vectors in data space.

• y (vector of floats having the same length of X) – membership degrees of the values in X.

The piecewise membership function is built so that its steps are chosen according to the quartiles of square distances between images of the points in X center of the learnt sphere.

# optimization¶

Implementation of optimization procedures.

This module module contains the implementations of the optimization processes behind fuzzy inference.

Once loaded, the module preliminarily verifies that some libraries are installed (notably, Gurobi and TensorFlow), emitting a warning otherwise. Note that at least one of these libraries is needed in order to solve the optimization problems involved in the fuzzy inference process.

The module also checks the availability of tqdm, which is used in order to graphically depict the progress of some learning processes using a progress bar. However, this package is not strictly needed: if it is not installed, the above mentioned progress bars will not be displayed.

class mulearn.optimization.Solver[source]

Abstract solver for optimization problems.

The base class for solvers is Solver: it exposes a method solve which delegates the numerical optimization process to an abstract method solve_problem and subsequently clips the results to the boundaries of the feasible region.

solve(xs, mus, c, k)[source]

Solve optimization phase.

Build and solve the constrained optimization problem on the basis of the fuzzy learning procedure.

Parameters
• xs (iterable) – Objects in training set.

• mus (iterable) – Membership values for the objects in xs.

• c (float) – constant managing the trade-off in joint radius/error optimization.

• k (mulearn.kernel.Kernel) – Kernel function to be used.

Raises

ValueError if c is non-positive or if xs and mus have different lengths.

Returns

list – optimal values for the independent variables of the problem.

class mulearn.optimization.GurobiSolver(time_limit=600, adjustment=0, initial_values=None)[source]

Solver based on gurobi.

Using this class requires that gurobi is installed and activated with a software key. The library is available at no cost for academic purposes (see https://www.gurobi.com/downloads/end-user-license-agreement-academic/). Alongside the library, also its interface to python should be installed, via the gurobipy package.

__init__(time_limit=600, adjustment=0, initial_values=None)[source]

Build an object of type GurobiSolver.

Parameters
• time_limit (int) – Maximum time (in seconds) before stopping iterative optimization, defaults to 10*60.

• adjustment (float or ‘auto’) – Adjustment value to be used with non-PSD matrices, defaults to 0. Specifying ‘auto’ instead than a numeric value will automatically trigger the optimal adjustment if needed.

• initial_values (iterable of floats or None) – Initial values for variables of the optimization problem, defaults to None.

solve_problem(xs, mus, c, k)[source]

Optimize via gurobi.

Build and solve the constrained optimization problem at the basis of the fuzzy learning procedure using the gurobi API.

Parameters
• xs (iterable) – objects in training set.

• mus (iterable) – membership values for the objects in xs.

• c (float) – constant managing the trade-off in joint radius/error optimization.

• k (mulearn.kernel.Kernel) – kernel function to be used.

Raises

ValueError if optimization fails or if gurobi is not installed

Returns

list – optimal values for the independent variables of the problem.

class mulearn.optimization.TensorFlowSolver(initial_values='random', init_bound=0.1, n_iter=100, optimizer=None, tracker=<function trange>, penalization=10)[source]

Solver based on TensorFlow.

Using this class requires that TensorFlow 2.X is installed.

__init__(initial_values='random', init_bound=0.1, n_iter=100, optimizer=None, tracker=<function trange>, penalization=10)[source]

Build an object of type TensorFlowSolver.

Parameters
• initial_values (str or iterable of float) – values to be used for initializing the independent variables, either randomly (if set to ‘random’) or seting them to a given sequence of initial values (if set to an iterable of floats), defaults to ‘random’.

• init_bound (float) – Absolute value of the extremes of the interval used for random initialization of independent variables, defaults to 0.1.

• n_iter (int) – Number of iterations of the optimization process, defaults to 100.

• optimizer (tf.keras.optimizers.Optimizer) – Optimization algorithm to be used, defaults to Adam with learning rate=1e-4 if tensorflow is available, to None otherwise.

• tracker (object) – Tool to graphically depict the optimization progress, defaults to tqdm.trange if tqdm is available, to range (i.e., no graphical tool) otherwise.

• penalization (float) – Lagrange penalization for the equality constraint in the original problem, defaults to 10.

solve_problem(xs, mus, c, k)[source]

Optimize via TensorFlow.

Build and solve the constrained optimization problem on the basis of the fuzzy learning procedure using the TensorFlow API.

Parameters
• xs (iterable) – objects in training set.

• mus (iterable) – membership values for the objects in xs.

• c (float) – constant managing the trade-off in joint radius/error optimization.

• k (mulearn.kernel.Kernel) – kernel function to be used.

Raises

ValueError if optimization fails or if TensorFlow is not installed

Returns

list – optimal values for the independent variables of the problem.

# distributions¶

Mulearn distributions.

This module implements the kernel and fuzzifier distributions used in randomized model selection.

class mulearn.distributions.GaussianKernelDistribution(low=0, high=1)[source]

Uniform distribution for gaussian kernels.

__init__(low=0, high=1)[source]

Build an object of type GaussianKernelDistribution.

Parameters
Raises

ValueError if the arguments low and high do not define an interval (that is, low is not lower than high).

rvs(*args, **kwargs)[source]

Generate a Gaussian kernel with uniformly distributed parameter.

Returns

mulearn.kernel.GaussianKernel – Gaussian kernel having a parameter uniformly chosen at random in the interval having self.low and self.high as extremes.

class mulearn.distributions.ExponentialFuzzifierDistribution(low=0, high=1)[source]

Uniform distribution for gaussian kernels.

__init__(low=0, high=1)[source]

Build an object of type ExponentialFuzzifierDistribution.

Parameters
Raises

ValueError if the arguments low and high do not define an interval (that is, low is not lower than high).

rvs(*args, **kwargs)[source]

Generate an Exponential fuzzifier with uniformly distributed parameter.

Returns

mulearn.fuzzifier.ExponentialFuzzifier` – Exponential fuzzifier having a parameter uniformly chosen at random in the interval having self.low and self.high as extremes.