Hyperparameter tuning

DP algorithms often expose clip multipliers, step sizes, or grid radii whose best values depend on the problem. The platform separates two fundamentally different uses of private data:

Tool	Privacy status	Appropriate use
`tune_algorithm_with_optuna`	Selection is not privacy-accounted	Development, utility ceilings, and designing a search grid
`DPHyperparameterTuned` subclasses	Selection is paid from the offered typed budget	Producing a tuned release with a DP guarantee

Choosing a winner after inspecting private-data scores is itself a release. Running every candidate with a DP algorithm does not make an unaccounted argmin private.

Search grids

Every tuner consumes a {parameter: candidates} mapping. A finite list means exact discrete choices. Liu–Talwar and Papernot–Steinke additionally accept {"lower": x, "upper": y} for continuous sampling; exhaustive composition does not. The wrapped algorithm's HYPERPARAM_INFO supplies defaults and whether a numeric range is sampled on a linear or logarithmic scale.

from dp_testing_platform.tasks.linear_regression.algorithms import VanillaSSP

grid = VanillaSSP.default_search_grid(n_points=5)
# Or specify exact choices:
grid = {"C_x_clip": [0.5, 1.0, 2.0], "C_y_clip": [1.0]}

Non-private Optuna tuning

tune_algorithm_with_optuna passes the same concrete budget to each trial, but it does not charge for selecting the winning trial. Its output is therefore not a private release.

from dp_testing_platform import (
    ApproxDPBudget,
    LinearRegression,
    tune_algorithm_with_optuna,
)
from dp_testing_platform.tasks.linear_regression.algorithms import VanillaSSP
from dp_testing_platform.tasks.linear_regression.instance_generators import (
    GaussianCovariatesAndNoise,
)

gen = GaussianCovariatesAndNoise(dim=2, n_samples=400)

def instances():
    return [gen.generate(seed) for seed in (0, 1)]

tuned, study = tune_algorithm_with_optuna(
    task=LinearRegression,
    alg_class=VanillaSSP,
    param_grid={"C_x_clip": [0.5, 1.0, 2.0], "C_y_clip": [1.0]},
    instances=instances,
    budget=ApproxDPBudget(1.0, 1e-5),
    num_trials=5,
    num_alg_samples=2,
    seed=0,
    time_limit=30.0,
)

Optuna is optional: install it with pip install "dp_testing_platform[tuning]".

Private tuning

A DPHyperparameterTuned object is a normal algorithm with the canonical interface:

run(metadata, input, budget: DPBudget, seed: int)

It splits the input into disjoint tuning and final slices, privately selects on the tuning slice, and runs the selected configuration on the final slice. Parallel composition lets both disjoint slices receive the same outer allowance. Selection is not free: it uses data and mechanism-specific accounting inside the tuning slice.

All DP tuners share these constructor arguments:

Argument	Meaning
`alg_class`	Inner algorithm class or dotted import path
`alg_hyperparam_grid`	Discrete candidates, or a supported continuous range
`quality_metric`	Registered DP selection cost; lower is better
`sample_split`	Fraction reserved for tuning (default `0.1`)
`cheating`	Debug-only reuse of full data for the final fit; not a private release

The DP quality metric receives the same typed per-run budget as the candidate and a required integer seed. It must account for every private-data access in its score.

Family routing

The tuner preserves budget families until its theorem requires a particular spend:

Tuner	Accepted outer budgets	Per-candidate allowance
`DPSelectionLiuTalwar`	pure, approximate, or zCDP	theorem-specific approximate split; otherwise a sound pure spend followed by `PureDPBudget(eps/3)`
`DPExhaustiveCompose`	pure, approximate, or zCDP	same family with every parameter divided by the number of candidates
`DPSelectionPapernotSteinke`	approximate DP only	exact calibrated `ZCDPBudget(rho_run)` charged by the executed P&S route

Consumers that cannot realize the resulting family refuse structurally. The tuners do not smuggle a zCDP charge through an (eps, delta) pair.

Liu–Talwar random stopping

DPSelectionLiuTalwar repeatedly samples and privately scores configurations, stopping with probability gamma after each round. Approximate-DP budgets follow Liu and Talwar's Theorem 3.4 allocation and finite trial cap. Pure-DP budgets use one third of epsilon per round; zCDP budgets enter that route only through the sound pure-spend conversion.

from dp_testing_platform import ApproxDPBudget, DPSelectionLiuTalwar
from dp_testing_platform.tasks.linear_regression.algorithms import VanillaSSP
from dp_testing_platform.tasks.linear_regression.instance_generators import (
    GaussianCovariatesAndNoise,
)

tuner = DPSelectionLiuTalwar(
    alg_class=VanillaSSP,
    alg_hyperparam_grid={"C_x_clip": [0.5, 1.0, 2.0], "C_y_clip": [1.0]},
    quality_metric={"name": "dp_clipped_SSE", "args": {"clipping_bound": 100.0}},
    gamma=0.3,
)
metadata, input, _ = GaussianCovariatesAndNoise(dim=2, n_samples=400).generate(0)
output = tuner.run(metadata, input, ApproxDPBudget(10.0, 1e-5), seed=0)

Exhaustive composition

DPExhaustiveCompose evaluates the Cartesian product once. For m candidates it uses eps/m, (eps/m, delta/m), or rho/m, matching the offered family. Selecting the minimum of already-private scores is post-processing. The grid grows multiplicatively, so keep it small.

from dp_testing_platform import ZCDPBudget
from dp_testing_platform.hyperparameter_tuning import DPExhaustiveCompose

tuner = DPExhaustiveCompose(
    alg_class=VanillaSSP,
    alg_hyperparam_grid={"C_x_clip": [0.5, 1.0, 2.0]},
    quality_metric={
        "name": "dp_clipped_SSE_gaussian",
        "args": {"clipping_bound": 100.0},
    },
)
output = tuner.run(metadata, input, ZCDPBudget(1.0), seed=0)

Papernot–Steinke exact zCDP handoff

DPSelectionPapernotSteinke accepts an ApproxDPBudget because its RDP selection theorem yields an approximate-DP guarantee. It calibrates the executed Poisson or TNB route, then hands the exact charged ZCDPBudget(rho_run) to both the base algorithm and its DP quality metric. The reported accounting metadata records that same rho. There is no interim epsilon allocation and no fallback to a route that was not run.

from dp_testing_platform import ApproxDPBudget, DPSelectionPapernotSteinke

tuner = DPSelectionPapernotSteinke(
    alg_class=VanillaSSP,
    alg_hyperparam_grid={"C_x_clip": [0.5, 1.0, 2.0]},
    quality_metric={
        "name": "dp_clipped_SSE_gaussian",
        "args": {"clipping_bound": 100.0},
    },
    repeat_dist="poisson",
)
output = tuner.run(metadata, input, ApproxDPBudget(10.0, 1e-5), seed=0)

The inner algorithm and quality metric must actually support a zCDP spend. If the executed route cannot fit the outer allowance, the tuner raises NoRelease instead of under-noising or silently changing mechanisms.

Sweeping a tuner

Put the tuner in algs and use concrete budgets like any other algorithm:

from dp_testing_platform import ApproxDPBudget, LinearRegression, run_experiment

run_experiment(
    task=LinearRegression,
    algs=[tuner],
    instance_generators=[GaussianCovariatesAndNoise(dim=2, n_samples=400)],
    budgets=[ApproxDPBudget(10.0, 1e-5)],
    alg_seeds=2,
    instance_generator_seeds=[0],
    workdir="results/tuned_sweep",
)

Operational cautions

DP tuning generally sacrifices utility relative to public, hand-chosen hyperparameters because selection consumes a data slice and privacy budget.
A NoRelease candidate scores as worst and cannot win; ordinary exceptions remain bugs and abort the selection.
Quality scores must be finite and DP. Candidate selection is only as sound as the registered metric and its family-specific accounting.

See Privacy budgets, Running evaluations, and the hyperparameter tuning API.