M-Estimation — API Reference

Auto-generated from the source docstrings. For the problem formulation and a usage example, see the M-Estimation task page.

Types

dp_testing_platform.tasks.m_estimation.types

Typed dataclasses for the M-estimation task.

MEstimationMetadata dataclass

Metadata for M-estimation.

Attributes:

Name	Type	Description
dim	int	Dimension of the parameter theta.
n_samples	int	Number of samples in the dataset.
loss	LossFn	Loss function. Takes (data, theta) and returns per-sample losses.
grad_loss	GradFn	Gradient function. Takes (data, theta) and returns per-sample gradients.
domain	Domain | None	Domain constraint on the data records. If None, unbounded. A lift from a bounded child task forwards its constraint here so a domain-aware M-estimation algorithm can use it (e.g. to tighten a clip); public problem-spec, never private data.

MEstimationInput dataclass

Input for M-estimation.

Contains only the private data. All fields are sample-indexed arrays where the first dimension corresponds to n_samples.

Attributes:

Name	Type	Description
data	Array2D	Data matrix, shape (n_samples, data_dim).

MEstimationOutput dataclass

Output for M-estimation.

Attributes:

Name	Type	Description
theta	Array1D	Estimated parameter vector, shape (dim,).

Algorithms

dp_testing_platform.tasks.m_estimation.algorithms.erm

Non-private empirical risk minimisation for M-estimation.

EmpiricalRiskMinimization

Bases: BaseAlgorithm

Non-private empirical risk minimization (baseline).

Minimizes empirical loss using scipy.optimize. No privacy guarantees. Standard/textbook M-estimation baseline; not a DP mechanism and has no single canonical paper.

Reference

Non-private baseline; no DP source paper.

dp_testing_platform.tasks.m_estimation.algorithms.gradient_descent

Simple gradient descent algorithm for M-estimation.

GradientDescent

Bases: BaseAlgorithm

Non-private full-batch gradient descent with Armijo backtracking (baseline).

Computes the gradient over all samples each step and picks the step size by backtracking line search: starting from eta, the step is halved until the mean loss satisfies the Armijo sufficient-decrease condition. The mean loss is therefore monotonically non-increasing, so the iterates cannot diverge regardless of the data's scaling — a fixed step size has no problem-agnostic safe default (the stability threshold depends on the data's curvature), which produced astronomically large "successful" outputs on unnormalized real datasets. No privacy guarantees. Standard/textbook first-order optimizer (gradient descent with Armijo backtracking line search); not a DP mechanism and has no single canonical paper.

Reference

Non-private baseline; no DP source paper.

Hyperparameters

eta: Initial (maximum) step size for the line search. Default: 1.0. num_steps: Number of gradient steps. Default: 10.

dp_testing_platform.tasks.m_estimation.algorithms.sgd

Stochastic gradient descent for M-estimation.

StochasticGradientDescent

Bases: BaseAlgorithm

Non-private stochastic gradient descent with a stochastic Armijo line search (baseline).

Samples one datapoint per step and picks the step size by backtracking on that datapoint's own loss until the Armijo sufficient-decrease condition holds (stochastic line search in the style of Vaswani et al., "Painless Stochastic Gradient: Interpolation, Line-Search, and Convergence Rates", arXiv:1905.09997). This bounds every update — a fixed step size has no problem-agnostic safe default (the stability threshold depends on per-sample curvature), which produced divergent "successful" outputs on unnormalized real datasets. No privacy guarantees.

Reference

Non-private baseline; no DP source paper. The stochastic Armijo line search follows Vaswani, Mishkin, Laradji, Schmidt, Gidel & Lacoste- Julien, "Painless Stochastic Gradient: Interpolation, Line-Search, and Convergence Rates", NeurIPS 2019. https://arxiv.org/abs/1905.09997

Hyperparameters

eta: Initial (maximum) step size for each step's line search. Default: 1.0. num_steps: Number of gradient steps. Default: 100.

dp_testing_platform.tasks.m_estimation.algorithms.dp_gradient_descent

Differentially private gradient descent for M-estimation.

DPGradientDescent

Bases: BaseAlgorithm

Differentially private gradient descent (the DP-SGD lineage).

Per-example gradient clipping with Gaussian noise. Satisfies (eps, delta)-DP.

The clip-then-add-Gaussian-noise gradient template is the DP-SGD mechanism of Abadi et al., "Deep Learning with Differential Privacy", ACM CCS 2016 (arXiv:1607.00133). Each step releases the SUM of the per-example clipped gradients plus Gaussian noise, then divides by the public sample count n. Under the add/remove-one neighboring relation the released noisy SUM has L2 sensitivity grad_clip -- adding/removing one record changes the sum by a single clipped term of norm <= grad_clip (the replace-one bound would be 2*grad_clip; the factor of 2 is dropped) -- so the per-coordinate noise std is noise_multiplier * grad_clip; the /n is post-processing by a public constant and does not enter the sensitivity. Equivalently the std on the n-divided gradient is noise_multiplier * grad_clip / n.

Two batch regimes, selected by subsample_rate (q):

• Full batch (q == 1.0): every step uses all n rows. The per-step noise multiplier is calibrated through the dp-accounting RDP seam (core.accounting.noise_multiplier_for_composed_gaussian) so that num_steps identical Gaussian steps compose to exactly (eps, delta)-DP, realizing the full-batch zCDP bound of Lemma 2.1 of Brown et al., AISTATS 2024 (arXiv:2402.13531).
• Poisson-subsampled (q < 1.0): each step draws a fresh independent Poisson lot -- every row joins the lot i.i.d. with probability q (mask = rng.random(n) < q) -- and the clipped lot-gradient sum is the released quantity. The per-step noise multiplier is calibrated through the amplified seam (noise_multiplier_for_subsampled_gaussian), which uses the Sampled Gaussian Mechanism RDP of Mironov, Talwar, Zhang, "Renyi Differential Privacy of the Sampled Gaussian Mechanism", 2019 (arXiv:1908.10530) -- privacy amplification by subsampling lets the calibrator spend less noise for the same (eps, delta). The amplification is only honest if the lot is true independent Poisson sampling (not a fixed-size or shuffle-and-slice lot) under the add/remove-one relation; this implementation does exactly that. The released-sum sensitivity is still grad_clip and n is still the public /n post-processing constant.

The released transcript is the num_steps noisy gradient steps; the returned estimate is a function of the resulting iterate trajectory selected by iterate_aggregation (last iterate, suffix/tail average, or full Polyak average). Aggregation is post-processing of the (already privately released) trajectory and is privacy-neutral: it does not change the num_steps-step composition, the noise calibration, or the per-step noise draw -- by DP post-processing immunity any deterministic function of the same transcript costs zero additional privacy. The only regime where releasing the last iterate is itself privacy-load-bearing is amplification-by-iteration (Feldman, Mironov, Talwar, Thakurta, 2018, arXiv:1808.06651), which would require NOT releasing the intermediate iterates -- a separate algorithm we do not implement.

Reference

Abadi, Chu, Goodfellow, McMahan, Mironov, Talwar & Zhang, "Deep Learning with Differential Privacy", ACM CCS 2016. https://arxiv.org/abs/1607.00133 Full-batch zCDP noise calibration (Lemma 2.1): Brown, Hopkins, Kamath & Pawar, "Private Gradient Descent for Linear Regression: Tighter Error Bounds and Instance-Specific Uncertainty Estimation", AISTATS 2024. https://arxiv.org/abs/2402.13531 Poisson-subsampled amplification (Sampled Gaussian Mechanism RDP): Mironov, Talwar & Zhang, "Renyi Differential Privacy of the Sampled Gaussian Mechanism", 2019. https://arxiv.org/abs/1908.10530

Hyperparameters

C_grad_clip: Clipping constant. Clips gradients to norm C_grad_clip * sqrt(d). Default: 1.0. eta: Learning rate. Default: 0.1. num_steps: Number of gradient steps. Default: 10. subsample_rate: Poisson lot rate q in (0, 1]. q=1.0 is full batch. Default: 1.0. iterate_aggregation: Which function of the iterate trajectory to release ("last", "tail_average", "polyak"). Post-processing; privacy-neutral. Default: "last".

Instance generators

dp_testing_platform.tasks.m_estimation.instance_generators.gaussian_instance_generator

Gaussian covariates and noise instance generator for M-estimation.

GaussianCovariatesAndNoise

Bases: InstanceGenerator

Gaussian instance generator for M-estimation.

Generates y = X @ beta + noise where X ~ N(0, Sigma). Supports squared or absolute residual loss functions.

Metrics

dp_testing_platform.tasks.m_estimation.metrics

Metric registry for the M-estimation task.

empirical_loss

empirical_loss(output: MEstimationOutput, evaluation_metadata: MEstimationMetadata, evaluation_input: MEstimationInput) -> float

Average loss of theta over evaluation_input.

dp_clipped_empirical_loss

dp_clipped_empirical_loss(output: MEstimationOutput, evaluation_metadata: MEstimationMetadata, evaluation_input: MEstimationInput, budget: DPBudget, seed: int, clipping_bound: float) -> float

DP clipped sum of per-sample losses with Laplace noise.

Generalization of LinearRegression's dp_clipped_SSE to arbitrary loss functions.

Parameters:

Name	Type	Description	Default
output	MEstimationOutput	Algorithm output (has .theta)	required
evaluation_metadata	MEstimationMetadata	Contains the loss function	required
evaluation_input	MEstimationInput	Evaluation input data	required
budget	DPBudget	Privacy allowance for the Laplace release.	required
seed	int	Seed for noise generation.	required
clipping_bound	float	Clipping bound for per-sample losses	required

Returns:

Type	Description
float	Noisy, clipped sum of per-sample losses

dp_median_empirical_loss

dp_median_empirical_loss(output: MEstimationOutput, evaluation_metadata: MEstimationMetadata, evaluation_input: MEstimationInput, budget: DPBudget, seed: int, beta: float) -> float

DP median of per-sample losses using Unbounded Quantile mechanism.

Generalization of LinearRegression's dp_median_SSE to arbitrary loss functions.

Parameters:

Name	Type	Description	Default
output	MEstimationOutput	Algorithm output (has .theta)	required
evaluation_metadata	MEstimationMetadata	Contains the loss function	required
evaluation_input	MEstimationInput	Evaluation input data	required
budget	DPBudget	Privacy allowance delegated to the quantile mechanism.	required
seed	int	Seed for noise generation.	required
beta	float	Multiplicative step size for the Unbounded Quantile candidate grid	required

Returns:

Type	Description
float	Noisy median of per-sample losses