Abstract
The multiexponential analysis problem of fitting kinetic models to time-resolved spectra is often solved using gradient-based algorithms that treat the spectral parameters as conditionally linear. We make a comparison of the two most-applied such algorithms, alternating least squares and variable projection. A numerical study examines computational efficiency and linear approximation standard error estimates. A new derivation of the Fisher information matrix under the full Golub-Pereyra gradient allows a numerical comparison of parameter precision under variable projection variants. Under the criteria of efficiency, quality of standard error estimates and parameter precision, we conclude that the Kaufman variable projection technique performs well, while techniques based on alternating least squares have significant disadvantages for application in the problem domain.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Al-Khayyal F. (1990). Jointly constrained bilinear programs and related problems: An overview. Comput. Math. Appl. 19: 53–62
Audet C., Hansen P., Jaumard B. and Savard G. (2000). A branch and cut algorithm for non-convex quadratically constrained quadratic programming. Math. Program. 87: 131–152
Basu S. and Bresler Y. (2000). Stability of Nonlinear Least Squares Problems and the Cramer-Rao Bound. IEEE T. Signal Proces. 48: 3426–3436
Bates, D.M., Lindstrom, M.J.: Nonlinear Least Squares with Conditionally Linear Parameters. In Proceedings of the Statistical Computing Section. 152–157. American Statistical Association, New York (1986)
Bates D.M. and Watts D.G. (1988). Nonlinear regression analysis and its applications. John Wiley & Sons, New York
Bijlsma S., Boelens H.F.M., Hoefsloot H.C.J. and Smilde A.K. (2000). Estimating reaction rate constants: comparison between traditional curve fitting and curve resolution. Anal. Chimica Acta, 419: 197–2007
Bijlsma S., Boelens H.F.M., Hoefsloot H.C.J. and Smilde A.K. (2002). Constrained least squares methods for estimating reaction rate constants from spectroscopic data. J. Chemometr. 16: 28–40
Böckmann C. (1995). A modification of the trust-region Gauss-Newton method to solve separable nonlinear least squares problems. J. Math. Syst. Estimat. Control. 5: 111–115
Brimberg J., Hansen P. and Mladenović N. (2002). A note on reduction of quadratic and bilinear programs with equality constraints. J. Global Optim. 22: 39–47
Dioumaev A.K. (1997). Evaluation of intrinsic chemical kinetics and transient product spectra from time-resolved spectroscopic data. Biophys. Chem. 67: 1–25
Gay, D., Kaufman, L.: Tradeoffs in Algorithms for Separable and Block Separable Nonlinear Least Squares. In Vichnevetsky, R., Miller, J.J.H. (eds.) IMACS ’91, Proceedings of the 13th World Congress on Computational and Applied Mathematics, pp.157–158. Criterion Press, Dublin (1991)
Godfrey K. (1983). Compartmental Models and Their Application. Academic Press, London
Golub G. and Pereyra V. (2003). Separable nonlinear least squares: the variable projection method and its applications. Inverse Problems 19: R1–R26
Golub, G.H., LeVeque, R.J.: Extensions and uses of the variable projection algorithm for solving nonlinear least squares problems. Proc. Army Num. Anal. Comp. Conf. ARO Report 79–3, 1–12 (1979)
Golub, G.H., Pereyra, V.: The differentiation of pseudo-inverses and nonlinear least squares problems whose variables separate, Tech. rep., Stanford University, Department of Computer Science (1972)
Golub G.H. and Pereyra V. (1973). The differentiation of pseudoinverses and nonlinear least squares problems whose variables separate. SIAM J. Num. Anal. 10: 413–432
Horst, R., Tuy, H.: Global Optimization: Deterministic Approaches. Springer-Verlag, Berlin, 3rd edn (1996)
Istratov A.A. and Vyvenko O.F. (1999). Exponential analysis in physical phenomena. Rev. Sci. Instrum. 70: 1233–1257
Jandanklang P., Maeder M. and Whitson A.C. (2001). Target transform fitting: a new method for the non-linear fitting of multivariate data with separable parameters. J. Chemometr. 15: 886–9383
Kaufman L. (1975). A variable projection method for solving separable nonlinear least squares problems. BIT. 15: 49–57
Nagle J.F. (1991). Solving Complex Photocycle Kinetics – Theory and Direct Method. Biophys. J. 59: 476–487
Osborne M.R. and Smyth G.K. (1995). A modified Prony algorithm for exponential function fitting. SIAM J. Sci. Comput. 16: 119–138
Parks, T.A.: Reducible Nonlinear Programming Problems, Tech. Rep. Technical Report TR85-08, Department of Computational and Applied Mathematics, Rice University, USA (1985)
Petersson, J., Holmström, K.: A Review of the Parameter Estimation Problem of Fitting Positive Exponential Sums to Empirical Data, Tech. Rep. Technical Report IMa-TOM-1997-08, Department of Mathematics and Physics, Märlardalen University, Sweden (1997)
Petersson, J., Holmström K.: Initial Values for the Exponential Sum Least Squares Fitting Problem, Tech. Rep. Technical Report IMa-TOM-1998-01, Department of Mathematics and Physics, Märlardalen University, Sweden (1998)
R Development Core Team: R: A Language and Environment for Statistical Computing, R Foundation for Statistical Computing, Vienna, Austria, http://www.R-project.org, ISBN 3-900051-07-0 (2004)
Ruhe A. and Wedin P.A. (1980). Algorithms for Separable Nonlinear Least Squares Problems. SIAM Rev. 22: 318–337
Seber G.A.F. and Wild C.J. (2003). Nonlinear regression. John Wiley & Sons, New Jersey
Smyth G.K. (1996). Partitioned algorithms for maximum likelihood and other nonlinear estimation. Stat. Computing. 6: 201–216
Bos A. (1980). A class of small sample nonlinear least squares problems. Automatica. 16: 487–490
van den Bos, A., Swarte J.H.: Resolvability of the parameters of multiexponentials and other sum models. IEEE T. Signal Process. 41, 313–322 (1993)
Stokkum I.H.M. (1997). Parameter Precision in Global Analysis of Time-Resolved Spectra. IEEE T. Instrum. Measure. 46: 764–768
van Stokkum, I.H.M., Larsen, D.S., van Grondelle, R.: Global and target analysis of time-resolved spectra. Biochim. Biophys. Acta. 1657, 82–104, and erratum, 1658, 262 (2004)
Varah J. (1985). On Fitting Exponentials by Nonlinear Least Squares. SIAM J. Sci. Stat. Comput. 1: 30–44
Wohlleben W., Buckup T., Herek J.L., Cogdell R.J. and Motzkus M. (2003). Multichannel Carotenoid Deactivation in Photosynthetic Light Harvesting as Identified by an Evolutionary Target Analysis. Biophys. J. 85: 442–450
Wold H. and Lyttkens E. (1969). Nonlinear iterative partial least squares (NIPALS) estimation procedures. Bulletin. Int. Stat. Institut. 43: 29–51
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Open Access This is an open access article distributed under the terms of the Creative Commons Attribution Noncommercial License ( https://creativecommons.org/licenses/by-nc/2.0 ), which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.
About this article
Cite this article
Mullen, K.M., Vengris, M. & van Stokkum, I.H.M. Algorithms for separable nonlinear least squares with application to modelling time-resolved spectra. J Glob Optim 38, 201–213 (2007). https://doi.org/10.1007/s10898-006-9071-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10898-006-9071-7