Skip to content

M-Estimation

Problem Formulation

M-estimation is the generalized estimation task: given a per-datapoint loss function \(\ell(z, \theta)\) and a dataset of \(n\) samples \(z_1, \dots, z_n\), estimate the parameter that minimizes the empirical risk:

\[\hat{\theta} = \arg\min_{\theta \in \mathbb{R}^d} \frac{1}{n} \sum_{i=1}^{n} \ell(z_i, \theta)\]

Many estimation problems are special cases: squared-residual loss recovers the mean and linear regression, absolute-residual loss recovers the median. The loss and gradient functions are part of the problem specification (Metadata), not the private data (Input) — tasks that reuse M-estimation algorithms supply them through a lifted class (make_lifted_algorithm, see docs/custom-tasks.md § "Reusing Algorithms Across Tasks"), whose lift_metadata/lift_input maps transform their data into the M-estimation format. Under differential privacy we want a release \(\tilde{\theta}\) that satisfies \(\varepsilon\)-DP or \((\varepsilon, \delta)\)-DP while keeping the empirical risk low.

API Reference

Types, algorithms (with their privacy arguments + hyperparameter defaults), instance generators, and metrics are documented from the source docstrings on the M-Estimation API Reference page.

Usage Example

from dp_testing_platform import ApproxDPBudget
from dp_testing_platform.tasks.m_estimation.instance_generators import (
    GaussianCovariatesAndNoise,
)
from dp_testing_platform.tasks.m_estimation.algorithms import (
    GradientDescent,
    DPGradientDescent,
)

# The generator supplies the loss/grad functions in metadata along with the data.
gen = GaussianCovariatesAndNoise(dim=3, n_samples=200, metric="empirical_loss")
metadata, input_data, metric = gen.generate(seed=0)

# Non-private baseline
budget = ApproxDPBudget(1.0, 1e-5)
base_out = GradientDescent().run(metadata, input_data, budget, seed=0)

# DP gradient descent
dp_out = DPGradientDescent().run(metadata, input_data, budget, seed=0)

print(f"Baseline empirical loss: {metric(base_out):.4f}")
print(f"DP empirical loss:       {metric(dp_out):.4f}")