""" ===================================================== Optimization and root finding (:mod:`scipy.optimize`) ===================================================== .. currentmodule:: scipy.optimize .. toctree:: :hidden: optimize.cython_optimize SciPy ``optimize`` provides functions for minimizing (or maximizing) objective functions, possibly subject to constraints. It includes solvers for nonlinear problems (with support for both local and global optimization algorithms), linear programming, constrained and nonlinear least-squares, root finding, and curve fitting. Common functions and objects, shared across different solvers, are: .. autosummary:: :toctree: generated/ show_options - Show specific options optimization solvers. OptimizeResult - The optimization result returned by some optimizers. OptimizeWarning - The optimization encountered problems. Optimization ============ Scalar functions optimization ----------------------------- .. autosummary:: :toctree: generated/ minimize_scalar - Interface for minimizers of univariate functions The `minimize_scalar` function supports the following methods: .. toctree:: optimize.minimize_scalar-brent optimize.minimize_scalar-bounded optimize.minimize_scalar-golden Local (multivariate) optimization --------------------------------- .. autosummary:: :toctree: generated/ minimize - Interface for minimizers of multivariate functions. The `minimize` function supports the following methods: .. toctree:: optimize.minimize-neldermead optimize.minimize-powell optimize.minimize-cg optimize.minimize-bfgs optimize.minimize-newtoncg optimize.minimize-lbfgsb optimize.minimize-tnc optimize.minimize-cobyla optimize.minimize-cobyqa optimize.minimize-slsqp optimize.minimize-trustconstr optimize.minimize-dogleg optimize.minimize-trustncg optimize.minimize-trustkrylov optimize.minimize-trustexact Constraints are passed to `minimize` function as a single object or as a list of objects from the following classes: .. autosummary:: :toctree: generated/ NonlinearConstraint - Class defining general nonlinear constraints. LinearConstraint - Class defining general linear constraints. Simple bound constraints are handled separately and there is a special class for them: .. autosummary:: :toctree: generated/ Bounds - Bound constraints. Quasi-Newton strategies implementing `HessianUpdateStrategy` interface can be used to approximate the Hessian in `minimize` function (available only for the 'trust-constr' method). Available quasi-Newton methods implementing this interface are: .. autosummary:: :toctree: generated/ BFGS - Broyden-Fletcher-Goldfarb-Shanno (BFGS) Hessian update strategy. SR1 - Symmetric-rank-1 Hessian update strategy. .. _global_optimization: Global optimization ------------------- .. autosummary:: :toctree: generated/ basinhopping - Basinhopping stochastic optimizer. brute - Brute force searching optimizer. differential_evolution - Stochastic optimizer using differential evolution. shgo - Simplicial homology global optimizer. dual_annealing - Dual annealing stochastic optimizer. direct - DIRECT (Dividing Rectangles) optimizer. Least-squares and curve fitting =============================== Nonlinear least-squares ----------------------- .. autosummary:: :toctree: generated/ least_squares - Solve a nonlinear least-squares problem with bounds on the variables. Linear least-squares -------------------- .. autosummary:: :toctree: generated/ nnls - Linear least-squares problem with non-negativity constraint. lsq_linear - Linear least-squares problem with bound constraints. isotonic_regression - Least squares problem of isotonic regression via PAVA. Curve fitting ------------- .. autosummary:: :toctree: generated/ curve_fit -- Fit curve to a set of points. Root finding ============ Scalar functions ---------------- .. autosummary:: :toctree: generated/ root_scalar - Unified interface for nonlinear solvers of scalar functions. brentq - quadratic interpolation Brent method. brenth - Brent method, modified by Harris with hyperbolic extrapolation. ridder - Ridder's method. bisect - Bisection method. newton - Newton's method (also Secant and Halley's methods). toms748 - Alefeld, Potra & Shi Algorithm 748. RootResults - The root finding result returned by some root finders. The `root_scalar` function supports the following methods: .. toctree:: optimize.root_scalar-brentq optimize.root_scalar-brenth optimize.root_scalar-bisect optimize.root_scalar-ridder optimize.root_scalar-newton optimize.root_scalar-toms748 optimize.root_scalar-secant optimize.root_scalar-halley The table below lists situations and appropriate methods, along with *asymptotic* convergence rates per iteration (and per function evaluation) for successful convergence to a simple root(*). Bisection is the slowest of them all, adding one bit of accuracy for each function evaluation, but is guaranteed to converge. The other bracketing methods all (eventually) increase the number of accurate bits by about 50% for every function evaluation. The derivative-based methods, all built on `newton`, can converge quite quickly if the initial value is close to the root. They can also be applied to functions defined on (a subset of) the complex plane. +-------------+----------+----------+-----------+-------------+-------------+----------------+ | Domain of f | Bracket? | Derivatives? | Solvers | Convergence | + + +----------+-----------+ +-------------+----------------+ | | | `fprime` | `fprime2` | | Guaranteed? | Rate(s)(*) | +=============+==========+==========+===========+=============+=============+================+ | `R` | Yes | N/A | N/A | - bisection | - Yes | - 1 "Linear" | | | | | | - brentq | - Yes | - >=1, <= 1.62 | | | | | | - brenth | - Yes | - >=1, <= 1.62 | | | | | | - ridder | - Yes | - 2.0 (1.41) | | | | | | - toms748 | - Yes | - 2.7 (1.65) | +-------------+----------+----------+-----------+-------------+-------------+----------------+ | `R` or `C` | No | No | No | secant | No | 1.62 (1.62) | +-------------+----------+----------+-----------+-------------+-------------+----------------+ | `R` or `C` | No | Yes | No | newton | No | 2.00 (1.41) | +-------------+----------+----------+-----------+-------------+-------------+----------------+ | `R` or `C` | No | Yes | Yes | halley | No | 3.00 (1.44) | +-------------+----------+----------+-----------+-------------+-------------+----------------+ .. seealso:: `scipy.optimize.cython_optimize` -- Typed Cython versions of root finding functions Fixed point finding: .. autosummary:: :toctree: generated/ fixed_point - Single-variable fixed-point solver. Multidimensional ---------------- .. autosummary:: :toctree: generated/ root - Unified interface for nonlinear solvers of multivariate functions. The `root` function supports the following methods: .. toctree:: optimize.root-hybr optimize.root-lm optimize.root-broyden1 optimize.root-broyden2 optimize.root-anderson optimize.root-linearmixing optimize.root-diagbroyden optimize.root-excitingmixing optimize.root-krylov optimize.root-dfsane Linear programming / MILP ========================= .. autosummary:: :toctree: generated/ milp -- Mixed integer linear programming. linprog -- Unified interface for minimizers of linear programming problems. The `linprog` function supports the following methods: .. toctree:: optimize.linprog-simplex optimize.linprog-interior-point optimize.linprog-revised_simplex optimize.linprog-highs-ipm optimize.linprog-highs-ds optimize.linprog-highs The simplex, interior-point, and revised simplex methods support callback functions, such as: .. autosummary:: :toctree: generated/ linprog_verbose_callback -- Sample callback function for linprog (simplex). Assignment problems =================== .. autosummary:: :toctree: generated/ linear_sum_assignment -- Solves the linear-sum assignment problem. quadratic_assignment -- Solves the quadratic assignment problem. The `quadratic_assignment` function supports the following methods: .. toctree:: optimize.qap-faq optimize.qap-2opt Utilities ========= Finite-difference approximation ------------------------------- .. autosummary:: :toctree: generated/ approx_fprime - Approximate the gradient of a scalar function. check_grad - Check the supplied derivative using finite differences. Line search ----------- .. autosummary:: :toctree: generated/ bracket - Bracket a minimum, given two starting points. line_search - Return a step that satisfies the strong Wolfe conditions. Hessian approximation --------------------- .. autosummary:: :toctree: generated/ LbfgsInvHessProduct - Linear operator for L-BFGS approximate inverse Hessian. HessianUpdateStrategy - Interface for implementing Hessian update strategies Benchmark problems ------------------ .. autosummary:: :toctree: generated/ rosen - The Rosenbrock function. rosen_der - The derivative of the Rosenbrock function. rosen_hess - The Hessian matrix of the Rosenbrock function. rosen_hess_prod - Product of the Rosenbrock Hessian with a vector. Legacy functions ================ The functions below are not recommended for use in new scripts; all of these methods are accessible via a newer, more consistent interfaces, provided by the interfaces above. Optimization ------------ General-purpose multivariate methods: .. autosummary:: :toctree: generated/ fmin - Nelder-Mead Simplex algorithm. fmin_powell - Powell's (modified) conjugate direction method. fmin_cg - Non-linear (Polak-Ribiere) conjugate gradient algorithm. fmin_bfgs - Quasi-Newton method (Broydon-Fletcher-Goldfarb-Shanno). fmin_ncg - Line-search Newton Conjugate Gradient. Constrained multivariate methods: .. autosummary:: :toctree: generated/ fmin_l_bfgs_b - Zhu, Byrd, and Nocedal's constrained optimizer. fmin_tnc - Truncated Newton code. fmin_cobyla - Constrained optimization by linear approximation. fmin_slsqp - Minimization using sequential least-squares programming. Univariate (scalar) minimization methods: .. autosummary:: :toctree: generated/ fminbound - Bounded minimization of a scalar function. brent - 1-D function minimization using Brent method. golden - 1-D function minimization using Golden Section method. Least-squares ------------- .. autosummary:: :toctree: generated/ leastsq - Minimize the sum of squares of M equations in N unknowns. Root finding ------------ General nonlinear solvers: .. autosummary:: :toctree: generated/ fsolve - Non-linear multivariable equation solver. broyden1 - Broyden's first method. broyden2 - Broyden's second method. NoConvergence - Exception raised when nonlinear solver does not converge. Large-scale nonlinear solvers: .. autosummary:: :toctree: generated/ newton_krylov anderson BroydenFirst InverseJacobian KrylovJacobian Simple iteration solvers: .. autosummary:: :toctree: generated/ excitingmixing linearmixing diagbroyden """ # noqa: E501 from ._optimize import * from ._minimize import * from ._root import * from ._root_scalar import * from ._minpack_py import * from ._zeros_py import * from ._lbfgsb_py import fmin_l_bfgs_b, LbfgsInvHessProduct from ._tnc import fmin_tnc from ._cobyla_py import fmin_cobyla from ._nonlin import * from ._slsqp_py import fmin_slsqp from ._nnls import nnls from ._basinhopping import basinhopping from ._linprog import linprog, linprog_verbose_callback from ._lsap import linear_sum_assignment from ._differentialevolution import differential_evolution from ._lsq import least_squares, lsq_linear from ._isotonic import isotonic_regression from ._constraints import (NonlinearConstraint, LinearConstraint, Bounds) from ._hessian_update_strategy import HessianUpdateStrategy, BFGS, SR1 from ._shgo import shgo from ._dual_annealing import dual_annealing from ._qap import quadratic_assignment from ._direct_py import direct from ._milp import milp # Deprecated namespaces, to be removed in v2.0.0 from . import ( cobyla, lbfgsb, linesearch, minpack, minpack2, moduleTNC, nonlin, optimize, slsqp, tnc, zeros ) __all__ = [s for s in dir() if not s.startswith('_')] from scipy._lib._testutils import PytestTester test = PytestTester(__name__) del PytestTester