This Neural Method Searches Only Feasible Functions

A feasible function is part of the model

A neural optimizer normally produces a function first and asks whether it broke the rules afterward. The rules might be an integral balance, a pointwise bound, a boundary value, or a differential equation. Physics-informed neural networks typically place a differential-equation residual in the training objective, alongside data or boundary terms.^{1, 2} Hintermüller and Ning's paper starts one step earlier: change the function the network is allowed to produce.

Write the constrained problem as minimizing an objective J(x) over an admissible set K. CNP introduces a parameterized map T_M : ℝ^M → X with two properties: every output lies in K, and as the parameter dimension M grows, the family of outputs becomes dense in K. Training then minimizes J(T_M(θ)) over ordinary network parameters.¹

That separation is the method's main idea. A penalty method can make violations expensive without making them impossible. CNP spends some of the parameterization on geometry, so the loss can focus on the task objective. The rest of the paper supplies the geometry for increasingly structured sets.

The polyhedron supplies the scaffold

Start with finitely many linear inequalities on a Hilbert space of functions: P = {x ∈ X : ⟨α_i,x⟩ ≤ b_i}. The constraint normals span a finite-dimensional space V_α. The paper's Theorem 2.3 says that every such polyhedron can be split into three pieces: an unconstrained orthogonal complement, a bounded convex hull of vertices, and an unbounded cone generated by extreme rays.^{1, 3}

The CNP is just that decomposition turned into a neural layer. Let N(·,θ) be the ordinary network, let \hat{\alpha}_i be an orthonormal basis for V_α, let y_i be the vertices, and let z_i be the extreme rays.

The first bracket is not a learned projection. The inner products are computed from the network and subtracted explicitly. The second bracket is a convex combination. A softmax is a convenient choice for its coefficients, but it returns strictly positive values, so it can approach a simplex boundary without attaining it. The paper's recursive sin² construction reaches the entire simplex. The last bracket uses squared or exponential scalar maps to make the ray weights nonnegative.¹

This geometry also explains the computational division of labor. Vertices and rays are computed once in the finite-dimensional constraint space. During training, the network supplies the infinite-dimensional-looking part, while a few trainable scalars choose the feasible finite-dimensional part. The authors' Example 5.1 code follows that pattern: it forms analytic correction terms and squared scalar parameters around a residual network rather than adding a constraint-violation term to the loss.⁴

Squares turn pointwise rules into identities

The same idea works without a polyhedral decomposition. If a scalar quantity must be nonnegative, square an unconstrained output. If it must stay between two functions, put the square inside a bounded scalar transform. If several phases must be nonnegative and sum to one, spend each successive fraction of the remaining mass. The map carries the constraint, so no penalty is needed to tell the optimizer what “inside” means.¹

Boundary conditions can be composed with these maps. A network multiplied by a nonnegative function w that vanishes on the boundary has zero boundary trace. In the paper's state-bound example, the square is applied to Nw + \sqrt{\psi} so the resulting state stays below ψ while also inheriting the boundary condition. This is an architectural constraint, not a claim that the network has learned to respect a large penalty weight.

The distinction matters because “hard constraints” is overloaded. Lu and colleagues' hPINN paper uses the name for a method that imposes constraints with penalty and augmented-Lagrangian terms.⁵ CNP uses the stricter sense: for the encoded inequality, the output map has no infeasible value to return. That stricter sense still has limits. A construction for one class of constraints does not automatically encode a second, coupled class.

The PDE disappears when the control is a function of the state

A typical neural optimal-control setup gives the state and control separate networks and asks a loss to reduce the PDE residual. Other methods approximate a coupled optimality system with state, control, adjoint, and sometimes multiplier networks.^{2, 6, 7} CNP takes a narrower route when the equation can be solved algebraically for the control. If u = G(y), substitute it into the objective and optimize only over a CNP for the admissible state:

For the paper's Poisson example, -\Delta y = f + u, the state is zero on the boundary, and the state must obey y ≤ 1. The parameterization is y = 1 - (Nw + 1)², followed by u = -\Delta y - f. Automatic differentiation supplies the Laplacian. Every sampled value of y satisfies the upper bound, and the control is defined so the differential equation holds at that point. There is no separate PDE-residual penalty and no adjoint or multiplier network in this reduced objective.^{1, 4}

The reduction is exact only inside its domain of definition. The paper requires enough regularity for G(y) to belong to the control space. For Poisson, its sufficient example is y ∈ H²(Ω) ∩ H₀¹(Ω) with f ∈ L²(Ω). A weakly feasible state on a nonsmooth domain need not be in H², so the naive reduced formula can ask the network for derivatives the solution does not have. That is why the paper treats the reduced form as a structured method, not a generic replacement for PDE solvers. The training remains a neural objective minimization, in the same broad variational family as methods such as Deep Ritz, but the constraint handling is different.^{1, 8}

Feasibility is exact; success is still empirical

The paper's answer to low regularity is to give the network the part it would otherwise struggle to learn. On an L-shaped domain, a re-entrant corner creates a known singular mode. The singularity-enriched ansatz writes the state as a smooth learned remainder plus trainable multiples of analytic corner functions, then applies the PDE operator to that sum. This is the same general lesson as singularity-enriched PINNs: when the solution has a known non-smooth component, put that component in the ansatz instead of asking a smooth network to discover it from residuals alone.^{1, 9}

The numerical comparisons are promising but scoped. In Example 5.1, which has two integral constraints, CNP reaches relative L² errors of 1.90 × 10⁻³ for the function and 6.27 × 10⁻³ for its derivative. Penalty runs at β = 10, 100, 1000 produce function errors of 1.52 × 10⁻², 3.29 × 10⁻³, and 2.24 × 10⁻², with derivative errors from 3.29 × 10⁻² to 1.59 × 10⁻¹. On the L-shaped test, singularity enrichment lowers the state/control errors from 2.81 × 10⁻² / 4.32 × 10⁻² to 4.02 × 10⁻³ / 1.06 × 10⁻². These are the paper's synthetic/reference-solution tests, not a random-data benchmark.¹

The strongest caveat appears in the complementarity example. Squared networks make the state and multiplier nonnegative, and exact reduction handles the PDE. But the orthogonality condition ⟨y,ξ⟩ = 0 is still enforced with a penalty of β = 100. The reported state, multiplier, and control errors are about 1.00 × 10⁻³, 2.07 × 10⁻², and 1.05 × 10⁻², respectively. That result shows how CNP can remove several difficult constraints while leaving a coupled relation for a future construction; it does not show that every constraint has become hard by construction.¹

The fair summary is therefore precise. CNP moves selected feasibility conditions into the parameterization, proves density for its families under explicit assumptions, and reports lower errors or lighter models on the problems tested. It does not supply a general convergence theorem for Adam, eliminate the need to understand state regularity, or make non-separable and nonlinear PDE control problems disappear. The paper's real contribution is a reusable design rule: if the feasible set has structure you can parameterize, make the network search that structure directly and leave the optimizer fewer ways to be wrong.¹