Abstract
Self-adaptive methods which permit step-sizes being selected self-adaptively are effective methods for solving some important problems, e.g., variational inequality problems. We devote this paper to developing and improving the self-adaptive methods for solving the split feasibility problem. A new improved self-adaptive method is introduced for solving the split feasibility problem. As a special case, the minimum norm solution of the split feasibility problem can be approached iteratively.
MSC:47J25, 47J20, 49N45, 65J15.
Similar content being viewed by others
1 Introduction
As we know, the original split feasibility problem (SFP) was introduced firstly by Censor and Elfving [1], and has received much attention since its inception in 1994. This is due to its applications in signal processing and image reconstruction, with particular progress in intensity-modulated radiation therapy; please, see [2–6].
Since the SFP is a special case of the convex feasibility problem (CFP), which is to find a point in the nonempty intersection of finitely many closed and convex sets, we next briefly review some historic approaches which relate to the CFP. The CFP is an important problem because many real-world inversion or estimation problems in engineering as well as in mathematics can be cast into this framework; see, e.g., Combettes [7], Bauschke and Borwein [8] and Kiwiel [9]. Traditionally, iterative projection methods for solving the CFP employ orthogonal projections onto convex sets (i.e., nearest point projections with respect to the Euclidean distance function); see, e.g., [10–14]. Much work has been done with generalized distance functions and the generalized projections associated with them suggested by Bregman [15].
In 1994, Censor and Elfving [1] investigated the use of different kinds of generalized projections in a single iterative process for solving the SFP. Their proposal is an iterative algorithm, which involves the computation of the inverse of a matrix, which is known to be a difficult task. That is why Byrne [16, 17] proposed the so-called CQ algorithm, which generates a sequence by a recursive procedure with suitable step-size. The CQ algorithm only involves the computations of the projections onto the sets C and Q, respectively, and is therefore implementable in the case where these projections have closed-form expressions (e.g., C and Q are the closed balls or half-spaces). There is a large number of references on the CQ method in the literature; see, for instance, [18–34]. However, we have to remark that the determination of the step-size depends on the operator (matrix) norm (or the dominant eigenvalue of a matrix product). This means that in order to implement the CQ algorithm, one has first to compute (or, at least, to estimate) the matrix norm of an operator, which is in general not an easy work in practice.
To overcome the above difficulty, the so-called self-adaptive method which permits step-size being selected self-adaptively was developed. Note that this method is the application of the projection method of Goldstein [35], Levitin and Polyak [36] to a suitable variational inequality problem, which is among the simplest numerical methods for solving variational inequality problems. Nevertheless, the efficiency of this projection method depends strongly on the choice of the step-size parameter. If one chooses a small parameter such that it guarantees the convergence of the iterative sequence, the recursion leads to slow speed of convergence. On the other hand, if one chooses a large step-size to improve the speed of convergence, the generated sequence may not converge. In real applications to solving variational inequality problems, the Lipschitz constant may be difficult to estimate, even if the underlying mapping is linear, the case such as the SFP. Some self-adaptive methods for solving variational inequality problems have been developed according to the original Goldstein-Levitin-Polyak method [35, 36]. See, e.g., [37–45].
Motivated by the self-adaptive strategy, Zhang et al. [45] proposed their method by using variable step-sizes instead of the fixed step-sizes as in Censor et al. [46]. Also, a self-adaptive projection method was introduced by Zhao and Yang [29], and it was adopted by using the Armijo-like searches. The advantage of these algorithms lies in the fact that neither prior information about the matrix norm A nor any other conditions on Q and A are required, and still convergence is guaranteed.
In this paper, we further develop and improve the self-adaptive methods for solving the SFP. An improved self-adaptive method is introduced for solving the SFP. As a special case, the minimum norm solution of the SFP can be approached iteratively.
2 Framework and preliminary results
Let and be two Hilbert spaces, and let C and Q be two closed and convex subsets of and , respectively. Let be a bounded linear operator. The split feasibility problem (SFP) is to find a point such that
Next, we use Γ to denote the solution set of the SFP, i.e., .
In 1994, Censor and Elfving [1] investigated the use of different kinds of generalized projections in a single iterative process for solving the SFP. They were the first to propose the following algorithm which involved the computation of the inverse :
where C and Q are closed and convex sets in , while A is a full rank matrix and . Note that is not easily executed. Consequently, Byrne [16, 17] proposed the so-called CQ algorithm which generates a sequence by the recursive procedure
where the step-size is chosen in the interval . It is remarkable that the CQ algorithm only involves the computations of the projections and onto the sets C and Q, respectively, and is therefore implementable in the case where and have closed-form expressions (e.g., C and Q are the closed balls or half-spaces). However, we observe that the determination of the step-size depends on the operator (matrix) norm (or the largest eigenvalue of ). This means that for practical implementation of the CQ algorithm, one has first to compute (or, at least, to estimate) the matrix norm of A, which is in general not an easy task in practice.
To overcome the above difficulty, the so-called self-adaptive method which permits step-size being selected self-adaptively was developed. If we set
then the convex objective f is differentiable and has a Lipschitz gradient given by
Thus, the CQ algorithm (2) can be obtained by minimizing the following convex minimization problem
We know that a point is a stationary point of problem (3) if it satisfies
Thus, we can use a gradient projection algorithm below to solve the (SFP)
where , the step-size at iteration n, is chosen in the interval , where L is the Lipschitz constant of ∇f.
The above method (5) has to be thought of as the application of the projection method of Goldstein [35], Levitin and Polyak [36] to the variational inequality problem (4), which is among the simplest numerical methods for solving variational inequality problems. Nevertheless, the efficiency of this projection method depends greatly on the choice of the parameter . A small guarantees the convergence of the iterative sequence, but the recursion leads to slow speed of convergence. On the other hand, a large step-size will improve the speed of convergence, but the generated sequence may not converge. In real applications for solving variational inequality problems, the Lipschitz constant may be difficult to estimate, even if the underlying mapping is linear, the case such as the SFP.
The methods in Zhang et al. [45] and Censor et al. [46] were proposed for solving the multiple-sets split feasibility problem.
Algorithm 2.1 S1. Given a nonnegative sequence such that , , , , , , and arbitrary initial point . Set and .
S2. Find the smallest nonnegative integer such that and
which satisfies
S3. If
then set ; otherwise, set .
S4. If , stop; otherwise, set and go to S2.
The following self-adaptive projection method was introduced by Zhao and Yang [29], and it was adopted by using the Armijo-like searches.
Algorithm 2.2 Given constants , and . Let be arbitrary. For , calculate
where and is the smallest nonnegative integer l such that
The advantage of Algorithm 2.1 and Algorithm 2.2 lies in the fact that neither prior information about the matrix norm A nor any other conditions on Q and A are required, and still convergence is guaranteed.
We shall introduce our improved self-adaptive method for solving the SFP. In this respect, we need the ingredients introduced right now.
Let C be a nonempty closed convex subset of a real Hilbert space H. A mapping is called nonexpansive if
A mapping is said to be δ-contractive if there exists a constant such that
Recall that the (nearest point or metric) projection from H onto C, denoted by , assigns to each the unique point with the property
It is well known that the metric projection of H onto C has the following basic properties:
-
(a)
for all ;
-
(b)
for every ;
-
(c)
for all and .
Next we adopt the following notation:
-
means that converges strongly to x;
-
means that converges weakly to x;
-
is the weak ω-limit set of the sequence .
Recall that a function is called convex if
It is known that a differentiable function f is convex if and only if the following relation holds:
Recall that an element is said to be a subgradient of at x if
If the function has at least one subgradient at x, it is said to be subdifferentiable at x. The set of subgradients of f at the point x is called the subdifferential of f at x, and is denoted by . A function f is called subdifferentiable if it is subdifferentiable at all . If f is convex and differentiable, then its gradient and subgradient coincide. A function is said to be weakly lower semi-continuous (w-lsc) at x if implies
f is said to be w-lsc on H if it is w-lsc at every point .
The first lemma is easy to prove.
Lemma 2.1 [14]
Let . Then
-
(i)
f is convex and differentiable;
-
(ii)
f is w-lsc on C.
Lemma 2.2 [47]
Given . Then solves the SFP if and only if solves the fixed point equation
where .
Lemma 2.3 [48]
Assume that is a sequence of nonnegative real numbers such that
where is a sequence in and is a sequence such that
-
(1)
;
-
(2)
or .
Then .
Lemma 2.4 [49]
Let be a sequence of real numbers that does not decrease at infinity, in the sense that there exists a subsequence of such that for all . For every , define an integer sequence as
Then as and for all
3 Main results
In this section we state and prove our main results.
Let C and Q be nonempty closed convex subsets of real Hilbert spaces and , respectively. Let be a δ-contraction with . Let be a bounded linear operator.
Algorithm 3.1 For given , assume that has been constructed. If , then stop and is a solution of SFP (1). Otherwise, continue and compute by the recursion
where and .
Theorem 3.1 Suppose that the SFP is consistent, that is, . Assume that the following conditions hold:
-
(a)
and ;
-
(b)
.
Then defined by (6) converges strongly to z, which solves the following variational inequality:
Proof First, it is obvious that the solution of the variational inequality (7) is unique (by the strong monotonicity of according to the related results in variational inequality), denoted by z. Then . We may assume that the sequence is infinite, that is, Algorithm 3.1 does not terminate in a finite number of iterations. Thus, for all n. From (6), we have
By the convexity of f (Lemma 2.1) and the fact that for , we deduce that
Using the inequality for all , we have
From (8)-(10), we get
By induction, we deduce
Hence, is bounded.
By using the firm nonexpansivity of , we derive that
It follows that
Next, we will prove that following the ideas in [49]. Set for all . Since and , we may assume, without loss of generality, that for some . Thus, we can rewrite (11) as
Now, we consider two possible cases.
Case 1. Assume that is eventually decreasing, i.e., there exists such that is decreasing for . In this case, must be convergent, and from (12) it follows that
where is a constant such that . Letting in (13), we get
Since is bounded, there exists a subsequence of converging weakly to .
From the weak lower semicontinuity of f, we have
Hence, , i.e., . This indicates that
Furthermore, due to the property of the projection (c),
From (12), we obtain
Applying Lemma 2.3 to (14), we get .
Case 2. Assume is not eventually decreasing. That is, there exists an integer such that . Thus, we can define an integer sequence for all as follows:
Clearly, is a non-decreasing sequence such that as and
for all . In this case, we derive from (13) that
It follows that
This implies that every weak cluster point of is in the solution set Γ; i.e., .
On the other hand, we note that
from which we can deduce that
Since , we have from (12) that
Combining (15) and (16) yields
and hence
From (14), we have
Thus,
From Lemma 2.4, we have
Therefore, . That is, . This completes the proof. □
From Theorem 3.1, we can deduce easily the following algorithm and corollary.
Algorithm 3.2 For given , assume that has been constructed. If , then stop and is a solution of SFP (1). Otherwise, continue and compute by the recursion
where and .
Theorem 3.2 Suppose that the SFP is consistent, that is, . Assume that the following conditions hold:
-
(a)
and ;
-
(b)
.
Then defined by (17) converges strongly to the minimum norm solution of the SFP.
4 Concluding remarks
This work contains our study dedicated to developing and improving self-adaptive methods for solving the split feasibility problem. We have introduced our improved self-adaptive method for solving the split feasibility problem. As a special case, the minimum norm solution of the split feasibility problem can be approached iteratively. This study is motivated by relevant applications for solving many real-world problems, which give rise to mathematical models in the sphere of variational inequality problems.
References
Censor Y, Elfving T: A multiprojection algorithm using Bregman projections in a product space. Numer. Algorithms 1994, 8: 221–239. 10.1007/BF02142692
Censor Y, Bortfeld T, Martin B, Trofimov A: A unified approach for inversion problems in intensity-modulated radiation therapy. Phys. Med. Biol. 2006, 51: 2353–2365. 10.1088/0031-9155/51/10/001
Stark H (Ed): Image Recovery: Theory and Applications. Academic Press, San Diego; 1987.
Ceng LC, Ansari QH, Yao JC: An extragradient method for solving split feasibility and fixed point problems. Comput. Math. Appl. 2012, 64: 633–642. 10.1016/j.camwa.2011.12.074
Ceng LC, Ansari QH, Yao JC: Mann type iterative methods for finding a common solution of split feasibility and fixed point problems. Positivity 2012, 16: 471–495. 10.1007/s11117-012-0174-8
Ceng LC, Ansari QH, Yao JC: Relaxed extragradient methods for finding minimum-norm solutions of the split feasibility problem. Nonlinear Anal. 2012, 75: 2116–2125. 10.1016/j.na.2011.10.012
Combettes PL: The foundations of set theoretic estimation. Proc. IEEE 1993, 81: 182–208.
Bauschke HH, Borwein JM: On projection algorithms for solving convex feasibility problems. SIAM Rev. 1996, 38: 367–426. 10.1137/S0036144593251710
Kiwiel, KC: Block-iterative surrogate projection methods for convex feasibility problems. Technical report, Systems Research Inst., Polish Academy of Sciences, Newelska 6, 01–447, Warsaw, Poland (December 1992)
Butnariu D, Censor Y: On the behavior of a block-iterative projection method for solving convex feasibility problems. Int. J. Comput. Math. 1990, 34: 79–94. 10.1080/00207169008803865
Censor Y: Row-action methods for huge and sparse systems and their applications. SIAM Rev. 1981, 23: 444–464. 10.1137/1023097
Gubin LG, Polyak BT, Raik EV: The method of projection for finding the common point of convex sets. U.S.S.R. Comput. Math. Math. Phys. 1967, 7: 1–24.
Han SP: A successive projection method. Math. Program. 1988, 40: 1–14.
Iusem AN, De Pierro AR: On the convergence of Han’s method for convex programming with quadratic objective. Math. Program. 1991, 52: 265–284. 10.1007/BF01582891
Bregman LM: The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming. U.S.S.R. Comput. Math. Math. Phys. 1967, 7: 200–217.
Byrne C: Iterative oblique projection onto convex subsets and the split feasibility problem. Inverse Probl. 2002, 18: 441–453. 10.1088/0266-5611/18/2/310
Byrne C: A unified treatment of some iterative algorithms in signal processing and image reconstruction. Inverse Probl. 2004, 20: 103–120. 10.1088/0266-5611/20/1/006
Byrne C: Bregman-Legendre multidistance projection algorithms for convex feasibility and optimization. In Inherently Parallel Algorithms in Feasibility and Optimization and Their Applications. Edited by: Butnariu D, Censor Y, Reich S. Elsevier, Amsterdam; 2001:87–100.
Censor Y, Segal A: The split common fixed point problem for directed operators. J. Convex Anal. 2009, 16: 587–600.
Dang Y, Gao Y: The strong convergence of a KM-CQ-like algorithm for a split feasibility problem. Inverse Probl. 2011., 27: Article ID 015007
Fukushima M: A relaxed projection method for variational inequalities. Math. Program. 1986, 35: 58–70. 10.1007/BF01589441
Qu B, Xiu N: A note on the CQ algorithm for the split feasibility problem. Inverse Probl. 2005, 21: 1655–1665. 10.1088/0266-5611/21/5/009
Wang F, Xu HK: Approximating curve and strong convergence of the CQ algorithm for the split feasibility problem. J. Inequal. Appl. 2010. 10.1155/2010/102085
Wang F, Xu HK: Cyclic algorithms for split feasibility problems in Hilbert spaces. Nonlinear Anal. 2011, 74: 4105–4111. 10.1016/j.na.2011.03.044
Wang Z, Yang Q, Yang Y: The relaxed inexact projection methods for the split feasibility problem. Appl. Math. Comput. 2010. 10.1016/j.amc.2010.11.058
Xu HK: A variable Krasnosel’skii-Mann algorithm and the multiple-set split feasibility problem. Inverse Probl. 2006, 22: 2021–2034. 10.1088/0266-5611/22/6/007
Xu HK: Averaged mappings and the gradient-projection algorithm. J. Optim. Theory Appl. 2011, 150: 360–378. 10.1007/s10957-011-9837-z
Yang Q, Zhao J: Several solution methods for the split feasibility problem. Inverse Probl. 2005, 21: 1791–1799. 10.1088/0266-5611/21/5/017
Zhao J, Yang Q: Self-adaptive projection methods for the multiple-sets split feasibility problem. Inverse Probl. 2011., 27: Article ID 035009
Yao Y, Wu J, Liou YC: Regularized methods for the split feasibility problem. Abstr. Appl. Anal. 2012., 2012: Article ID 140679
Ceng LC, Ansari QH, Yao JC: Extragradient-projection method for solving constrained convex minimization problems. Numer. Algebra Control Optim. 2011, 1: 341–359.
Ceng LC, Ansari QH, Yao JC: Some iterative methods for finding fixed points and for solving constrained convex minimization problems. Nonlinear Anal. 2011, 74: 5286–5302. 10.1016/j.na.2011.05.005
Ceng LC, Ansari QH, Wen CF: Multi-step implicit iterative methods with regularization for minimization problems and fixed point problems. J. Inequal. Appl. 2013., 2013: Article ID 240 10.1186/1029-242X-2013-240
Ceng LC, Ansari QH, Wen CF: Implicit relaxed and hybrid methods with regularization for minimization problems and asymptotically strict pseudocontractive mappings in the intermediate sense. Abstr. Appl. Anal. 2013., 2013: Article ID 854297
Goldstein AA: Convex programming in Hilbert space. Bull. Am. Math. Soc. 1964, 70: 709–710. 10.1090/S0002-9904-1964-11178-2
Levitin ES, Polyak BT: Constrained minimization problems. U.S.S.R. Comput. Math. Math. Phys. 1966, 6: 1–50.
Han D: Solving linear variational inequality problems by a self-adaptive projection method. Appl. Math. Comput. 2006, 182: 1765–1771. 10.1016/j.amc.2006.06.013
Han D: Inexact operator splitting methods with self-adaptive strategy for variational inequality problems. J. Optim. Theory Appl. 2007, 132: 227–243. 10.1007/s10957-006-9060-5
Han D, Sun W: A new modified Goldstein-Levitin-Polyak projection method for variational inequality problems. Comput. Math. Appl. 2004, 47: 1817–1825. 10.1016/j.camwa.2003.12.002
He BS, He X, Liu H, Wu T: Self-adaptive projection method for co-coercive variational inequalities. Eur. J. Oper. Res. 2009, 196: 43–48. 10.1016/j.ejor.2008.03.004
He BS, Yang H, Meng Q, Han D: Modified Goldstein-Levitin-Polyak projection method for asymmetric strong monotone variational inequalities. J. Optim. Theory Appl. 2002, 112: 129–143. 10.1023/A:1013048729944
He BS, Yang H, Wang SL: Alternating direction method with self-adaptive penalty parameters for monotone variational inequalities. J. Optim. Theory Appl. 2000, 106: 337–356. 10.1023/A:1004603514434
Liao LZ, Wang SL: A self-adaptive projection and contraction method for monotone symmetric linear variational inequalities. Comput. Math. Appl. 2002, 43: 41–48. 10.1016/S0898-1221(01)00269-3
Yang Q: On variable-step relaxed projection algorithm for variational inequalities. J. Math. Anal. Appl. 2005, 302: 166–179. 10.1016/j.jmaa.2004.07.048
Zhang W, Han D, Li Z: A self-adaptive projection method for solving the multiple-sets split feasibility problem. Inverse Probl. 2009., 25: Article ID 115001
Censor Y, Elfving T, Kopf N, Bortfeld T: The multiple-sets split feasibility problem and its applications for inverse problems. Inverse Probl. 2005, 21: 2071–2084. 10.1088/0266-5611/21/6/017
Xu HK: Iterative methods for the split feasibility problem in infinite-dimensional Hilbert spaces. Inverse Probl. 2010., 26: Article ID 105018
Xu HK: Iterative algorithms for nonlinear operators. J. Lond. Math. Soc. 2002, 66: 240–256. 10.1112/S0024610702003332
Mainge PE: Strong convergence of projected subgradient methods for nonsmooth and nonstrictly convex minimization. Set-Valued Anal. 2008, 16: 899–912. 10.1007/s11228-008-0102-z
Acknowledgements
The first author was supported in part by NSFC 11071279 and NSFC 71161001-G0105. The third author was partially supported by NSC 101-2628-E-230-001-MY3.
Author information
Authors and Affiliations
Corresponding author
Additional information
Competing interests
The authors declare that they have no competing interests.
Authors’ contributions
The authors completed the paper together. All authors read and approved the final manuscript.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 2.0 International License ( https://creativecommons.org/licenses/by/2.0 ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
About this article
Cite this article
Yao, Y., Postolache, M. & Liou, YC. Strong convergence of a self-adaptive method for the split feasibility problem. Fixed Point Theory Appl 2013, 201 (2013). https://doi.org/10.1186/1687-1812-2013-201
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/1687-1812-2013-201