Abstract
Data assimilation is an important tool in many geophysical applications. One of many key elements of data assimilation algorithms is the measurement error that determines the weighting of the data in the cost function to be minimized. Although the algorithms used for data assimilation treat the measurement uncertainty as known, it is in many cases estimated or set based on some expert opinion. Here we treat the measurement uncertainty as a hyperparameter in a fully Bayesian hierarchical model and derive a new class of iterative ensemble methods for data assimilation where the measurement uncertainty is integrated out. The proposed algorithms are compared with the standard iterative ensemble smoother on a 2D synthetic reservoir model.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Anthes, R., Rieckh, T.: Estimating observation and model error variances using multiple data sets. Atmos. Meas. Tech. 11(7), 4239–4260 (2018). https://doi.org/10.5194/amt-11-4239-2018. (https://amt.copernicus.org/articles/11/4239/2018/)
Bocquet, M., Sakov, P.: An iterative ensemble Kalman smoother. Q. J. R. Meteorol. Soc. 140(682), 1521–1535 (2012)
Casella, G.: An introduction to empirical Bayes data analysis. Am. Stat. 39(2), 83–87 (1985). ISSN 00031305. http://www.jstor.org/stable/2682801
Chen, S.-Y., Huang, C.-Y., Kuo, Y.-H., Sokolovskiy, S.: Observational error estimation of formosat-3/cosmic GPS radio occultation data. Mon. Weather. Rev. 139(3), 853–865 (2011). https://doi.org/10.1175/2010MWR3260.1. (https://journals.ametsoc.org/view/journals/mwre/139/3/2010mwr3260.1.xml.)
Daley, R.: Atmospheric Data Analysis. Cambridge University Press, Cambridge Atmospheric and Space Science Series (1993). ISSN 00031305. http://www.jstor.org/stable/2682801
Desroziers, G., Ivanov, S.: Diagnosis and adaptive tuning of observation-error parameters in a variational assimilation. Q. J. R. Meteorol. Soc. 127(574), 1433–1452 (2001). https://doi.org/10.1002/qj.49712757417. (https://rmets.onlinelibrary.wiley.com/doi/abs/10.1002/qj.49712757417)
Desroziers, G., Berre, L., Chapnik, B., Poli, P.: Diagnosis of observation, background and analysis-error statistics in observation space. Q. J. R. Meteorol. Soc. 131(613), 3385–3396 (2005). https://doi.org/10.1256/qj.05.108. (https://rmets.onlinelibrary.wiley.com/doi/abs/10.1256/qj.05.108)
Gelman, A., Carlin, J., Stern, H., Dunson, D., Vehtari, A., Rubin, D.: Bayesian Data Analysis (3rd ed.). Chapman and Hall/CRC (2013)
Gray, J., Allan, D.: A method for estimating the frequency stability of an individual oscillator. In: 28th Annual Symposium on Frequency Control, pp. 243-246. (1974). https://doi.org/10.1109/FREQ.1974.200027
Hollingsworth, A., Lönnberg, P.: The statistical structure of short-range forecast errors as determined from radiosonde data. part i: The wind field. Tellus A 38A(2), 111–136 (1986). https://doi.org/10.1111/j.1600-0870.1986.tb00460.x. https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1600-0870.1986.tb00460.x
Jaynes, E.: Probability Theory: The Logic of Science. Cambridge University Press Cambridge (2003)
Khaki, M., Ait-El-Fquih, B., Hoteit, I., Forootan, E., Awange, J., Kuhn, M.: Unsupervised ensemble Kalman filtering with an uncertain constraint for land hydrological data assimilation. J. Hydrol. 564, 175–190 (2018). ISSN 0022–1694. https://doi.org/10.1016/j.jhydrol.2018.06.080. https://www.sciencedirect.com/science/article/pii/S002216941830502X
Luo, X., Bhakta, T.: Estimating observation error covariance matrix of seismic data from a perspective of image denoising. Comput. Geosci. 21(2), 205–222 (2017)
Mehra, R.: On the identification of variances and adaptive Kalman filtering. IEEE Trans. Autom. Control. 15(2), 175–184 (1970). https://doi.org/10.1109/TAC.1970.1099422
Mehra, R.: On the identification of variances and adaptive Kalman filtering. IEEE Trans. Autom. Control. 15(2), 175–184 (1970). https://doi.org/10.1109/TAC.1970.1099422
Oliver, D.S., Reynolds, A.C., Liu, N.: Inverse theory for petroleum reservoir characterization and history matching. Cambridge, (2008)
Oliver, D.S., Fossum, K., Bhakta T., Sandø, I., Nævdal, G., Lorentzen, R.J.: 4d seismic history matching. J. Petrol. Sci. Eng. 207, 109119 (2021). ISSN 0920-4105. https://doi.org/10.1016/j.petrol.2021.109119. https://www.sciencedirect.com/science/article/pii/S0920410521007750
Raanes, P.N., Stordal, A.S., Evensen, G.: Revising the stochastic iterative ensemble smoother. Nonlinear Process. Geophys. 26(3), 325–338 (2019). https://doi.org/10.5194/npg-26-325-2019. (https://npg.copernicus.org/articles/26/325/2019/)
Raboudi, N.F., Ait-El-Fquih, B., Hoteit, I.: Online estimation of colored observation-noise parameters within an ensemble Kalman filtering framework. Q. J. R. Meteorol. Soc. n/a, (n/a) (2023). https://doi.org/10.1002/qj.4484. https://rmets.onlinelibrary.wiley.com/doi/abs/10.1002/qj.4484
Rasmussen, A.F., Sandve, T. H., Bao, K., Lauser, A., Hove, J., Skaflestad, B., Klöfkorn, R., Blatt, M., Rustad, A.B., Sævareid, O., Lie, K.-A., Thune, A.: The open porous media flow reservoir simulator. Comput Math Appl 81, 159–185 (2021). ISSN 0898-1221. https://doi.org/10.1016/j.camwa.2020.05.014. https://www.sciencedirect.com/science/article/pii/S0898122120302182. Development and Application of Open-source Software for Problems with Numerical PDEs
Rawlinson, N., Fichtner, A., Sambridge, M., Young, M.K.: Seismic tomography and the assessment of uncertainty. Adv. Geophys. 55, 1–76 (2014). ISSN 0065–2687. https://doi.org/10.1016/bs.agph.2014.08.001. https://www.sciencedirect.com/science/article/pii/S0065268714000028
Rue, H., Martino, S., Chopin, N.: Approximate Bayesian inference for latent Gaussian models by using integrated nested laplace approximations. J. R. Stat. Soc. Ser. B Stat. Methodol. 71(2), 319–392 (2009). https://doi.org/10.1111/j.1467-9868.2008.00700.x. https://rss.onlinelibrary.wiley.com/doi/abs/10.1111/j.1467-9868.2008.00700.x
Simon, D.: Optimal state estimation: Kalman, H infinity, and nonlinear approaches. John Wiley & Sons (2006)
Sornette, D., Ide, K.: The Kalman–lévy filter. Physica D Nonlinear Phenom 151(2), 142–174 (2001). ISSN 0167-2789. https://doi.org/10.1016/S0167-2789(01)00228-7. https://www.sciencedirect.com/science/article/pii/S0167278901002287
Tilmann, F.J., Sadeghisorkhani, H., Mauerberger, A.: Another look at the treatment of data uncertainty in Markov chain Monte Carlo inversion and other probabilistic methods. Geophys. J. Int. 222, 388–405, 05 (2020). ISSN 0956-540X. https://doi.org/10.1093/gji/ggaa168
Ueno, G., Nakamura, N.: Bayesian estimation of the observation-error covariance matrix in ensemble-based filters. Q. J. R. Meteorol. Soc. 142(698), 2055–2080 (2016). https://doi.org/10.1002/qj.2803. (https://rmets.onlinelibrary.wiley.com/doi/abs/10.1002/qj.2803)
Acknowledgements
The authors acknowledge financial support from the NORCE research project “Assimilating 4D Seismic Data: Big Data Into Big Models" which is funded by industry partners, Equinor Energy AS, Repsol Norge AS, Shell Global Solutions International B.V., TotalEnergiesEP Norge AS, WintershallDeaNorge AS and AkerBP ASA, as well as the Research Council of Norway (PETROMAKS2).
Funding
Open access funding provided by NORCE Norwegian Research Centre AS.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Stordal, A.S., Lorentzen, R.J. & Fossum, K. Marginalized iterative ensemble smoothers for data assimilation. Comput Geosci 27, 975–986 (2023). https://doi.org/10.1007/s10596-023-10242-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10596-023-10242-1