• Technical • long

Machine-Learned Surrogates and Amortized Parametric Optimization for Stochastic Least-Cost Feed Formulation

Where machine-learned surrogates and amortized/parametric optimization actually help stochastic least-cost feed formulation — nonlinear margin maximization, active-set prediction, and warm-starts — with the cases where LP is already enough.

#machine-learning#optimization#surrogate-models#feed-formulation#linear-programming#stochastic#amortized-optimization

Part of a three-piece series. Companion references: Post-Training Techniques and Training Platforms (the techniques and tools) and Azure GPU and CPU Cost Reference (what they cost to run).

TL;DR

  • The single most defensible ML play is predicting the active set / optimal basis to amortize very large families of near-identical solves (thousands of formulas/day × price scenarios × plants), because the reduced system then recovers the exact optimum. This is a proven technique in power systems (2–3 orders of magnitude speedups) but its transfer to feed is unproven and its payoff only materializes at high solve volume, which must be measured first.
  • Multiparametric LP (mpLP) is the exact, certificate-carrying answer to “parametrics,” but it collapses combinatorially past ~5–10 varying parameters. Feed formulation routinely varies far more than that (dozens of ingredient prices + nutrient uncertainties), so exact mpLP is out — which is precisely the regime where approximate amortization becomes the only tractable option, but also where verification burden is highest.

Key Findings

  1. The deterministic single-ration LP is genuinely trivial and should never be surrogated. Feed LPs are small (tens of ingredients, tens of nutrient constraints) and solve exactly in milliseconds via revised simplex, returning reduced costs and a managed basis. Imitating this with a neural net is value-destroying.
  2. The parametric/stochastic family is where compute genuinely grows. Moving from one solve to (a) chance-constrained/stochastic formulation (nonlinear), (b) scenario-based Monte Carlo (hundreds–thousands of scenarios), (c) multi-plant/multi-period/multi-product MILP (integer batching, transport, capacity), and (d) margin-maximization over a nonlinear animal-response function all move the problem from milliseconds to genuinely hard. This is where the value concentrates.
  3. Multiparametric programming is the classical non-ML answer, and it is exact. mpLP/mpQP (Gal & Nedoma 1972; Bemporad, Morari, Dua, Pistikopoulos, Automatica 2002) pre-compute the entire optimal solution map offline as a piecewise-affine function over polyhedral “critical regions”; online, the method does a table lookup. Explicit MPC is the canonical success story. But the number of critical regions is upper-bounded by 2^q (q = number of constraints) and grows combinatorially. Per Alessio & Bemporad, “A Survey on Explicit Model Predictive Control” (2009): “In practice, explicit MPC is limited to relatively small problems (typically one/two inputs, up to five-ten states, up to three/four free control moves) but allows one to reach very high sampling frequencies (up to 1 MHz).” Feed formulation with dozens of varying prices and nutrient uncertainties is far outside the exact-mpLP tractable regime.
  4. Active-set / optimal-basis learning is the sweet spot of amortization. Because a non-degenerate LP with p variables is fully determined by p active constraints, predicting which constraints bind reduces the problem to a single linear solve that recovers the exact optimum. In power systems, only a handful of distinct active sets recur across tens of thousands of instances (often 1–78 active sets across 24–1888-bus systems in Deka & Misra), classifiers hit near-100% top-3 accuracy, and Bertsimas & Stellato (“Online Mixed-Integer Optimization in Milliseconds,” INFORMS J. Computing 2022, 34(4):2229–2248) report: “We benchmark our method against the state-of-the-art solver Gurobi obtaining up to two to three orders of magnitude speedups on examples from fuel cell energy management, sparse portfolio optimization, and motion planning with obstacle avoidance.” Caveat: the “few active sets” result is distribution-dependent — under wider, multi-parameter perturbations counts explode into the thousands.
  5. Chance-constrained/stochastic feed formulation is a mature, 50-year-old domain technique that is economically real but adoption-poor. Charnes & Cooper (1959); St. Pierre & Harvey (1986, Journal of Dairy Science) single- and joint-chance-constrained models; Roush, Zhang, Pesti. A deterministic LP meets requirements with only ~50% probability — per Pesti & Seila (1999, J. Applied Poultry Research 8(1):110–121): “Average composition values are input; therefore, the confidence in meeting nutrient restrictions is 50%.” Stochastic programming reaches 90–99% confidence and typically produces cheaper feed than naive additive safety margins because it uses the total variance of the finished feed rather than stacking per-ingredient margins. Adoption is low due to software, data, training, and KPI barriers (Moss et al. 2024 Australian survey); notably, commercial stochastic software was offered as early as Spring 1997 at ~$18,000, underscoring that cost/complexity — not concept novelty — has gated uptake.
  6. Ingredient nutrient variability is large, measurable, and costly. Published CVs for crude protein: corn ~8.7%, wheat ~19.1%, DDGS ~5.4% (Tahir et al. 2012); DDGS standardized ileal digestible lysine CV ~13.5%; DDGS starch/crude-fat CVs the highest. This variance is exactly what safety margins over-buy and what stochastic formulation prices correctly.
  7. The nonlinear margin-maximization problem is where LP genuinely breaks down — and where an ML surrogate of the animal response model (not the optimizer) is the most defensible ML opportunity. Guevara (2004, Poultry Science 83(2):147–151) fitted quadratic responses and used nonlinear programming to maximize margin over feed cost, finding optimal broiler energy density of 2.8–3.3 Mcal/kg and concluding “an energy level does not need to be set,” with the nonlinear model reducing losses vs LP especially under adverse chicken prices (tested by moving broiler, corn, fish meal and soybean meal prices by ±25%).

Details

A. Multiparametric programming — the exact classical answer, and why it doesn’t scale to feed

Multiparametric programming (mpLP/mpQP) solves an optimization problem once, symbolically, producing the optimal solution as an explicit piecewise-affine function of the parameters θ. The parameter space is partitioned into critical regions — polyhedra within which the optimal active set is invariant and the optimizer is an affine function of θ. Online, the method locates which region θ falls in and evaluates a linear formula. Theory: Gal & Nedoma (1972) for mpLP; Fiacco’s Basic Sensitivity Theorem and Bemporad, Morari, Dua & Pistikopoulos (Automatica 2002) for mpQP. Tools: the Multi-Parametric Toolbox (MPT3, Kvasnica et al.) and the POP/Parametric OPtimization toolbox (Pistikopoulos group). Explicit MPC — precomputing a constrained LQR control law as a lookup table — is the canonical industrial success, reaching sampling rates up to ~1 MHz.

The fatal limitation for feed: combinatorial region explosion. The number of critical regions M is upper-bounded by 2^q where q is the number of constraints (Alessio & Bemporad 2009). Each additional parameter increases q via state/horizon dimensions; storage per region and online point-location cost both scale with parameter dimension n_θ. The canonical practitioner limit is stark: “explicit MPC is limited to relatively small problems (typically one/two inputs, up to five-ten states, up to three/four free control moves)” (Alessio & Bemporad 2009). Empirically, even modest problems generate thousands of regions (e.g. ~4000 regions for a 16-mode nanosatellite docking controller).

Implication for feed formulation: A feed LP that varies, say, 30 ingredient prices + 10 uncertain nutrient values simultaneously has a parameter dimension of ~40 — an order of magnitude beyond the exact-mpLP tractable frontier. Exact mpLP is therefore NOT viable for the full parametric feed problem. It is viable and underused for low-dimensional slices: e.g., parametric cost as a function of 1–3 key ingredient prices, which is exactly what “price maps” and “parametric evaluation / step supply curves” in commercial feed software already approximate. Recent work on approximate mpLP and region-reduction (fitting-based simplification in MPT3, region-free/active-set-storage methods, constrained-zonotope reformulations) pushes the frontier but does not change the order of magnitude.

The crucial comparison — mpLP vs neural surrogate: If mpLP gives an exact piecewise-affine map with certificates, a neural surrogate is never preferable when mpLP is tractable. The neural approach becomes the only option precisely when the parameter dimension (roughly >5–10) makes the exact partition explode — which is the feed case. That is also where the exactness and certificates are lost, shifting the burden onto verification (Section B).

B. Amortized optimization and learning the parametric solution map

Amos’s Tutorial on Amortized Optimization (Foundations and Trends in ML, 2023) formalizes learning a mapping from problem parameters to solutions, exploiting shared structure across similar instances; reported speedups reach “many orders of magnitude” in domains like variational inference and control. The relevant question for feed is which form of amortization to use.

Regressing the full solution directly (optimization proxies) is the most-studied but least appropriate for feed. In AC-OPF, proxies like CANOS reach within ~1% of optimal cost in 33–65 ms. But three problems dominate: (a) naive nets violate hard constraints → infeasible rations, unacceptable when constraints encode animal health/regulatory minimums; (b) active-set transitions cause abrupt, hard-to-predict geometry changes; (c) worst-case performance is orders of magnitude worse than average and proxies are vulnerable to adversarial inputs (multiple OPF verification papers).

Predicting the active set / optimal basis is far more promising. Misra, Roald & Ng (“Learning for Constrained Optimization: Identifying Optimal Active Constraint Sets,” INFORMS J. Computing 2022) and Deka & Misra (2019) show that although possible active sets grow as 2^q, only a handful actually occur in engineered systems — as few as 1–3 active sets across 50,000 samples for several power grids, up to 78 for a 300-bus case. A neural classifier reaches “close to 100%” top-3 accuracy; once the active set is known, an LP optimum is recovered by a single linear solve — exact, feasible, optimal. Bertsimas & Stellato (2022) generalize this to a “strategy” (active set + integer assignment), report between a few hundred and <10,000 distinct strategies across problems, and achieve two-to-three orders of magnitude speedups (e.g. sparse portfolio: MLOPT 0.175 s vs Gurobi 2063.8 s). Caveat: their raw top-1 accuracy can be <10%; feasibility/optimality come from evaluating the top-k candidate strategies in parallel and picking the best feasible one, and the “few active sets” property degrades badly under wide, multi-parameter perturbation (counts into the thousands — arXiv:1911.06784). For feed, this means active-set learning works best for narrow families (same formula re-solved as prices drift a little), not for arbitrary parameter sweeps.

Warm-starting is the cheap baseline any ML method must beat. In rolling reformulation the previous solution/basis is a natural warm start; simplex re-optimization from a warm basis after a small price change is typically a handful of pivots — already near-instant. Learned warm starts in power systems report only 2–25× speedups and, crucially, inherit feasibility/optimality from the classical solver’s termination. For a millisecond feed LP, warm-started simplex almost certainly dominates any learned proxy on a cost/risk basis. ML only adds value when the solve is genuinely expensive (stochastic/MILP/nonlinear), not for the base LP.

Hard-constraint enforcement and repair: DC3 (Donti, Rolnick & Kolter, ICLR 2021) enforces feasibility via differentiable completion (satisfy equalities by solving for dependent variables) + unrolled gradient correction (inequalities), achieving near-optimal feasible solutions on AC-OPF. Differentiable optimization layers — OptNet, cvxpylayers, DiffOpt — embed the solve as a network layer for end-to-end training. Projection/feasibility-restoration layers can guarantee constraint satisfaction. These matter because in feed, an infeasible or nutrient-deficient ration is a safety/regulatory event, not a rounding error.

Verification is possible but expensive. OPF proxy verification (Nellikkath & Chatzivasileiadis; compact optimality-verification formulations; scalable MIP-based worst-case violation) can certify bounds on a learned surrogate’s constraint violation/optimality gap — verified up to ~793-bus AC-OPF. But verification is computationally heavy and often “overly pessimistic,” and self-certifying primal-dual proxies are an active research area. For a regulated feed context this means: a bare neural surrogate cannot be trusted for committed formulas without either (a) an exact solver confirming the final formula, or (b) formal verification that must be built and maintained.

C. Stochastic / non-deterministic feed formulation — the domain literature

Chance-constrained programming (CCP) is the classical treatment of ingredient nutrient variability: replace “nutrient i ≥ requirement” with “P(nutrient i ≥ requirement) ≥ α.” Charnes & Cooper (1959) introduced CCP; St. Pierre & Harvey (1986, Journal of Dairy Science) developed single- and joint-chance-constrained ration models, comparing linear approximations against exact nonlinear solutions using 1970–79 prices for 17 ingredients. The deterministic equivalent is generally nonlinear (convex), which is why the feed industry historically used linear approximations (subtracting a multiple of the standard deviation as a safety margin).

Why stochastic formulation is economically real: A conventional LP using only mean composition meets each nutrient requirement with only ~50% probability (Pesti & Seila 1999: “Average composition values are input; therefore, the confidence in meeting nutrient restrictions is 50%.”). Stochastic programming delivers 90–99% confidence (Stochastic Programming Model for Lactating Cattle, 2023). Counter-intuitively it often yields cheaper feed than additive safety margins, because it uses the total standard deviation of the finished feed (accounting for how ingredient variances combine) rather than stacking a margin on every ingredient independently — additive margins over-formulate.

Multi-objective / goal programming: Zhang & Roush (2002, Poultry Science) multiple-objective programming for reducing nutrient variation; multiobjective stochastic programming for pig feed (JORS 2009) confronts cost against probability of meeting requirements. Sustainability/footprint objectives (cost vs performance vs environmental impact vs ingredient sustainability) are increasingly added.

Scenario-based / Monte Carlo and where compute explodes: Sample Average Approximation (SAA) approximates the distribution with a finite scenario set; decision variables grow linearly with scenarios and the MILP search tree grows worse, so intractability arrives quickly. Scenario reduction (Dupačová, Heitsch & Römisch Wasserstein-based methods; problem-driven reduction; Latin hypercube; progressive hedging) can cut a set to ~1–2% of samples while recovering the SAA solution — this is a concrete place ML (scenario selection/reduction) legitimately adds value.

Ingredient nutrient variability (measured CVs):

Ingredient / nutrientCoefficient of variationSource
Corn, crude protein~8.7%Tahir et al. 2012
Wheat, crude protein~19.1%Tahir et al. 2012
DDGS, crude protein~5.4% (395-sample survey: 8.9%)Tahir et al. 2012 / Wilson review
Soybean meal, crude protein<2%OSU Extension (St-Pierre)
DDGS, SID lysine~13.5%Olukosi & Adebiyi 2013
DDGS, starch / crude fat~45% / ~17%Pedersen et al. 2014 (NIRS, 72 samples from 21 U.S. ethanol plants)
Corn / wheat / barley calcium (across import origins)65.7% / 57.4% / 28.8%432-sample cereal-import study (PMC4823223)

Practical impact figures from feed-mill industry sources (indicative, not peer-reviewed): a ±10% ingredient price move alters formula cost by ~$8–12/tonne; a 1–2% matrix adjustment for moisture shifts diet cost by ~$3–6/tonne; simultaneous multi-formula optimization (“Multi-Mix”) is reported to reduce overall feed cost by 2–3%.

Multi-plant/multi-period/multi-product: These become genuinely hard MILPs (integer batching, transport, plant capacity, sequence-dependent changeovers). Grain/blending MILP and enterprise-wide optimization (Grossmann) literatures report simultaneous production+distribution planning cost savings up to ~20% and solve times that reach hours at scale — the regime where decomposition (Benders, progressive hedging) and potentially learned subproblem surrogates pay off.

Nonlinear biological response: Guevara (2004, Poultry Science) fitted quadratic weight-gain/intake responses to energy density and used nonlinear programming to maximize margin over feed cost, letting optimal nutrient densities emerge rather than be pre-set; optimal energy density ranged 2.8–3.3 Mcal/kg and the nonlinear model reduced losses vs LP especially under adverse chicken prices. This is where classic LP genuinely fails — and where a differentiable ML surrogate of the response function, wrapped by an NLP/LP optimizer, is the most defensible ML contribution.

D. Where the ML actually belongs — ranked

  1. ML on the INPUTS (predict-then-optimize) — HIGHEST ROI, LOWEST RISK. NIRS + ML for real-time ingredient composition is already commercial (Evonik AMINONIR for >60 ingredients; micro-NIRS with PLSR/ANN achieving R²cv ~0.90, RPD>3 for protein/fat/moisture; scan times <20 s, ±0.2% protein accuracy). Feeding better means AND variances into a chance-constrained LP directly reduces over-formulation. Decision-focused learning / SPO+ (Elmachtoub & Grigas 2022) goes further: train the price/composition predictor to minimize decision regret rather than prediction MSE, because small prediction errors can cause large decision errors. PyEPO implements this. Reported gains are largest when the prediction model is misspecified — likely in noisy ingredient-price forecasting.
  2. ML surrogate of the ANIMAL RESPONSE MODEL inside margin-maximization — HIGH ROI, MEDIUM RISK. Replace the fixed nutrient requirement with a learned response surface, keep an exact NLP/LP doing the optimization. This attacks the one place LP genuinely breaks down and keeps the optimizer auditable.
  3. ML predicting the ACTIVE SET to accelerate large parametric families — MEDIUM ROI, MEDIUM RISK, HIGH UPSIDE AT VOLUME. The exactness-recovering property (predict basis → single linear solve → exact optimum) makes this the safest way to replace solves. Pays off only at high volume/narrow families; must beat warm-started simplex.
  4. ML surrogate for fast interactive what-if / sensitivity exploration — MEDIUM ROI, LOW RISK. A cheap surrogate for real-time UI responsiveness, with the exact solver confirming any committed formula. Low risk because the exact solve gates commitment.
  5. ML for scenario reduction in stochastic programming — MEDIUM ROI, LOW RISK. Directly attacks the compute explosion in SAA; well-precedented.
  6. Full neural replacement of the parametric solve — LOWEST ROI, HIGHEST RISK. Justified only if (a) solves are individually expensive (nonlinear/MILP), (b) volume is enormous, (c) latency is binding, AND (d) formal verification plus an exact-solver backstop are in place. None of these clearly hold for base feed LPs.

The underexploited classical win — LP duals and sensitivity analysis. Every feed LP already yields shadow prices (marginal value of relaxing each nutrient constraint), reduced costs / marginal price changes (how far an excluded ingredient’s price must fall to enter), and RHS/objective ranging (over what price range the current basis stays optimal). Roush et al. showed Excel Solver sensitivity output gives exactly this for both LP (shadow price) and stochastic (Lagrange multiplier / reduced gradient) formulations. “Parametric evaluation” and “price maps” already trace formula cost across ingredient price ranges. A large fraction of the sought-after “parametric value” is available exactly, today, with zero ML — this should be exhausted before building surrogates.

E. Quantified decision criteria

  • Parameter dimension where exact mpLP dies → surrogate becomes the only option: roughly >5–10 simultaneously varying parameters (Alessio & Bemporad 2009). Feed problems varying dozens of prices/nutrients are firmly in surrogate territory if the full parametric map is required — but the full map is rarely necessary (see duals/ranging).
  • Solve volume where amortization pays back training/validation: No feed-specific figure exists; extrapolating from power systems, amortization pays at thousands+ near-identical solves per day against a stable formula/constraint structure (so the trained model isn’t invalidated by frequent re-specification). The actual daily solve count and formula-structure churn should be measured first.
  • Solve time where a surrogate is worth it: When individual solves exceed roughly seconds-to-minutes (stochastic/MILP/nonlinear), not milliseconds (base LP). A 100–1000× speedup on a 1 ms solve saves nothing usable; on a 10-minute stochastic MILP it is transformative.
  • Acceptable constraint-violation / optimality gap in feed: For committed production formulas, effectively zero tolerance on hard nutrient minimums and regulatory limits (animal health + compliance) — hence any surrogate must be gated by an exact solve or formal verification. For interactive what-if exploration, a ~1% optimality gap and small transient violations are tolerable because nothing is committed. Power-system proxies target ε ≈ 10⁻⁴; average optimality gaps ~0.03–1% are achievable, but worst-case violations are orders of magnitude worse — which is disqualifying without a backstop.
  • Expected economic benefit (for comparison against the ML opportunity): Stochastic vs LP-with-margins typically yields cheaper feed at higher nutrient-compliance confidence (90–99% vs ~50%); simultaneous multi-formula optimization ~2–3% feed cost reduction; integrated production+distribution planning up to ~20%; better composition data avoids ~$3–12/tonne swings from moisture/price misestimation. These simpler wins are large relative to any speed-driven ML benefit.

Recommendations

Stage 0 — Instrument before building (weeks). Measure: (a) actual solves/day and how many are near-identical re-solves of a stable formula vs genuinely new specs; (b) wall-clock per solve for the hardest cases (stochastic, multi-plant); (c) how often formula structure changes (which invalidates trained models). These three numbers decide everything below. Threshold to proceed to ML-for-speed: >~1000 near-identical solves/day AND hardest solves >seconds AND stable structure.

Stage 1 — Exhaust the free classical wins first (now). Surface LP duals/shadow prices, reduced costs, RHS/objective ranging, and multi-ingredient price maps in AutoCalc’s UI. Add chance-constrained / stochastic formulation using ingredient variances typically already available from QC/NIRS. This captures most of the “parametric/non-deterministic value” with no ML and full auditability. Benchmark: does stochastic formulation change committed formulas and reduce cost-at-confidence vs current margin practice? If yes, this alone justifies the program.

Stage 2 — Improve the inputs (next). Deploy/upgrade NIRS-driven composition and variance estimation; pilot decision-focused (SPO+) price/composition prediction on one species/phase, measuring decision regret vs current MSE-trained forecasts. Benchmark: measurable reduction in realized formula cost or nutrient-shortfall events on the pilot species/phase.

Stage 3 — Surrogate the response model, not the solver (if margin-maximization is a goal). Build an ML animal-response surface, optimize margin-over-feed-cost with an exact NLP/LP. Keep the optimizer auditable.

Stage 4 — Only if Stage 0 thresholds are met: active-set/basis learning for the large parametric family. Prototype an active-set classifier on the highest-volume, most stable formula family; recover the exact optimum via the reduced linear solve; require it to beat warm-started simplex on cost and latency before scaling. Gate every committed formula with an exact solve.

Never: deploy a bare neural surrogate as the source of truth for committed production formulas without an exact-solver backstop or formal verification.

Caveats

  • Transfer risk is the dominant uncertainty. Almost all quantified learned-optimization evidence (speedups, active-set counts, feasibility/gap figures) comes from power systems (OPF), process systems engineering, and portfolio optimization — not feed. The structure transfers (LPs with parametric RHS/objective), but the specific numbers (few active sets, 2–3 orders of magnitude speedups) may not. The “few recurring active sets” property in particular degrades sharply under wide multi-parameter variation, which feed price/nutrient sweeps resemble.
  • Feed LPs are small, so the absolute time saved per solve is tiny; the amortization argument rests entirely on volume and on the harder stochastic/MILP/nonlinear variants — verify the volume empirically.
  • Adoption evidence is thin and mostly potential, not realized. The feed-formulation stochastic/max-profit literature repeatedly reports barriers (software, data, training, KPIs) rather than large deployed savings; the economic-benefit figures are often illustrative examples, not audited field results. Several $/tonne figures come from extension/industry articles rather than peer-reviewed field trials — treat them as indicative.
  • Regulatory/assurance burden is asymmetric. In a Cargill-scale ANH context, a nutrient-deficient or non-compliant committed ration is a materially worse outcome than a slightly suboptimal cost. This asymmetry is why the exact solver should remain the arbiter of committed formulas and why full neural replacement is disfavored regardless of speedups.
  • mpLP tractability limits (“5–10 parameters”) are practitioner rules of thumb, not hard theorems; the hard result is the 2^q worst-case region bound. Approximate mpLP may extend the frontier somewhat.

Bottom line

The common instinct — that “the real value is on parametrics / non-deterministic behavior” — is correct about where the interesting problem lives and wrong if it implies the answer is a learned surrogate of the solve. The parametric/stochastic family is genuinely where compute and value concentrate. But the highest-ROI, lowest-risk responses are, in order: (1) exploit the LP duality/ranging already available; (2) adopt chance-constrained/stochastic formulation with better NIRS-driven variance data; (3) apply decision-focused prediction to inputs; (4) surrogate the animal response model, not the optimizer. Learning the active set to amortize large families is the one place a learned surrogate is both principled and exactness-preserving — but only justify it after measuring solve volume, and always keep an exact solver as the arbiter of any committed formula.

Read the companion references:

Part of the Univrs research ecosystem: