Local convergence of a two-step Gauss-Newton Werner-type method for solving least squares problems

Abstract.

The aim of this paper is to extend the applicability of a two-step Gauss-Newton-Werner-type method (TGNWTM) for solving nonlinear least squares problems. The radius of convergence, error bounds and the information on the location of the solution are improved under the same information as in earlier studies. Numerical examples further validate the theoretical results.

1. Introduction

Let $i,j$ be natural numbers with $j\ge i.$ Let also $\mathrm{\Omega }$ be an open and convex subset of ${ℝ}^j.$ We are concerned with the solution $p$ of the least squares problem [4, 5, 6, 7, 8, 9]:

(1.1) $$\underset{x\in \mathrm{\Omega }}{\min }f(x):=\frac{1}{2}F{(x)}^{T}F(x),$$

where $F:\mathrm{\Omega }⟶{ℝ}^j$ is a Fréchet-differentiable mapping. Numerous problems can be brought in the form (1.1) using Mathematical Modeling [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]. The closed form solutions can only be found in special cases. That explains why most solution methods for problem (1.1) are iterative. Let $x_0,y_0\in \mathrm{\Omega }$ and set $z=\frac{x_0+y_0}{2}.$ In the present study, we provide the local convergence analysis of GNWTM defined for each $n=0,1,2,\mathrm{\dots }$ by

$x_{n+1}$ $=$ $x_n-A_{n}F(x_n)$

(1.2) $y_{n+1}$ $=$ $x_{n+1}-A_{n}F(x_{n+1})$

$z_n$ $=$ ${\displaystyle \frac{x_n+y_n}{2}},$

where $A_n={[F^{\prime }{(z_n)}^T{F}^{\prime }(z_n)]}^{-1}{F}^{\prime }{(z_n)}^T.$ If $i=j,$ TGNWTM reduces to a Gauss-Newton-Werner type method [3, 8, 9]. Notice that in each iteration the inversion of ${[F^{\prime }{(z_n)}^T{F}^{\prime }(z_n)]}^{-1}$ is required only once. Therefore, the computational cost is essentially the same as in the Gauss- Newton method. The $L{L}^T$ decomposition of ${[F^{\prime }{(z_n)}^T{F}^{\prime }(z_n)]}^{-1}$ costs $O(n^3)$ floating-point operations (Flops) leading to the computation of $x_{n+1}.$ It then follows from the second substep of method (1) that $O(n^2)$ Flops are needed for the computation of $y_{n+1}.$

The local convergence analysis of method (1) was given in the elegant paper by Shakhno et.al. in [9] (see also related work in [3, 8]). Their convergence analysis uses average Lipschitz continuity condition as well as Lipschitz conditions.

Using the concept of the average Lipschitz continuity [12] and our new idea of restricted convergence domains, we present a local convergence analysis with the following advantages (A) over works using the similar information [3, 4, 8, 9, 11, 12, 13]:

1. Larger radius of convergence;

2. Tighter error bounds on the distances $\Vert x_n-p\Vert ;$

3. An at least as precise information on the location of the solution $p.$

Achieving (a)-(c) is very important in computational sciences, since: (a)^′ We obtain a wider choice of initial guesses; (b)^′ Fewer iterates are required to obtain a desired error tolerance; (c)^′ Better information about the ball of convergence is obtained.

The rest of the paper is structured as follows: Section 2 contains the local convergence analysis of method (1) whereas special cases and the applications are presented in the concluding Section 3.

2. Local convergence analysis

Set to be the open ball in ${ℝ}^j$ and by $\overline{U}(w,\rho )$ to denote its closure. Let $R>0.$ Define parameter $R_1$ by $R_1:=sup\{t\in [0,R]:\overline{U}(p,t)\subset \mathrm{\Omega }\}.$ The convergence analysis of numerous iterative methods has been given using the following concept due to Wang [12]:

Definition 1.

Mapping $F:\overline{U}(p,R_1)⟶{ℝ}^i$ satisfies the Lipschitz condition with $L_1$ average on $U(p,R_1)$ if

$$\Vert F^{\prime }(x)-F^{\prime }(y)\Vert \le {\int }_0^{\Vert x-y\Vert }{L}_1(u)𝑑u\text{for each}x,y\in \overline{U}(p,R_1),$$

where $L_1$ is a positive non-decreasing function.

It turns out that the convergence analysis of iterative methods based on the preceding notion can be improved as follows:

Definition 2.

The mapping $F:\overline{U}(p,R_1)⟶{ℝ}^i$ satisfies the center-Lipschitz condition with $L_0$ average on $U(p,R_1)$ if

$$\Vert F^{\prime }(x)-F^{\prime }(p)\Vert \le {\int }_0^{\Vert x-p\Vert }{L}_0(u)𝑑u\text{for each}x\in \overline{U}(p,R_1),$$

where $L_0$ is a positive non-decreasing function.

Clearly, we have that

(2.1) $$L_0(u)\le L_1(u)\text{for each}u\in [0,R_1],$$

and $\frac{L_0}{L_1}$ can be arbitrary small [2, 3, 4]. Let $\beta >0$ be a parameter. Suppose that equation

(2.2) $$\beta {\int }_0^t{L}_0(u)𝑑u=1$$

has positive solutions. Denote by $R_0$ the smallest such solution. Notice for example that $R_0$ exists, if

(2.3) $$\beta L_0(R_1)R_1\ge 1.$$

Indeed, function $g(t):=\beta {\int }_0^{t}L(u)𝑑u-1$ is such that and $g(R_1)=\beta L(R_1)R_1-1>0.$ The existence of $R_0$ follows from the intermediate value theorem.

Definition 3.

The mapping $F:\overline{U}(p,R_1)⟶{ℝ}^i$ satisfies the restricted Lipschitz condition with $L$ average on $U(p,R_0)$ if

$$\Vert F^{\prime }(x)-F^{\prime }(y)\Vert \le {\int }_0^{\Vert x-y\Vert }L(u)𝑑u\text{for each}x,y\in \overline{U}(p,R_0),$$

where $L$ is a positive non-decreasing function.

We have that

(2.4) $$L(u)\le L_1(u)\text{for each}u\in [0,R_0],$$

since $R_0\le R_1.$ Throughout this paper, we suppose that

(2.5) $$L_0(u)\le L(u)\text{for each}u\in [0,R_0],$$

unless, otherwise stated. Otherwise, i.e., if

(2.6) $$L(u)\le L_0(u)\text{for each}u\in [0,R_0],$$

then the results that follow hold with $L_0$ replacing $L.$ Moreover, we need the definitions:

Definition 4 ([12]).

Let $F:\overline{U}(p,R_1)⟶{ℝ}^i$ be a twice Fréchet-differentiable mapping. We say that mapping $F$ satisfies the Lipschitz condition with $M_1$ average on $U(p,R_1)$ if

$$\Vert F^{\prime \prime }(x)-F^{\prime \prime }(y)\Vert \le {\int }_0^{\Vert x-y\Vert }{M}_1(u)𝑑u\text{for each}x,y\in \overline{U}(p,R_1),$$

where $M_1$ is a positive nondecreasing function.

Definition 5.

Let $F:\overline{U}(p,R_0)⟶{ℝ}^i$ be twice Fréchet-differentiable mapping. We say that mapping $F$ satisfies the restricted Lipschitz condition with $M$ average on $U(p,R_0)$ if

$$\Vert F^{\prime \prime }(x)-F^{\prime \prime }(y)\Vert \le {\int }_0^{\Vert x-y\Vert }M(u)𝑑u\text{for each}x,y\in \overline{U}(p,R_0),$$

where $M$ is a positive nondecreasing function.

We have that

(2.7) $$M(u)\le M_1(u)\text{for each}u\in [0,R_0].$$

It is worth noticing that the definition of functions $L$ and $M$ (based on $L_0$ and $R_0$) was not possible in earlier studies using $L_1$ and $M_1.$ That is, $L=L(L_0,R_0,R_1),M=M(L_0,R_0,R_1),$ whereas $L_1=L_1(R_1)$ and $M_1=M_1(R_1).$ It turns out that $L_0$ can replace the less precise $L$ in the computation of the upper bounds on the inverses of the operators involved and $\overline{U}(p,R_0),L,M$ can replace $\overline{U}(p,R_1),L_1,M_1,$ respectively in the proofs of such results. Moreover, notice that the iterates $x_n$ lie in $\overline{U}(p,R_0)$ which is a more precise location than $\overline{U}(p,R_1)$ used in earlier studies [2, 3, 4, 8, 11, 12, 13]. We shall make the paper as self contained as possible by stating some standard auxiliary concepts and results.

Denote by ${ℝ}^{i\times j}$ the set of all $i\times j$ matrices. The Moore-Penrose pseudo-inverse is defined by $A^{†}={(A^{T}A)}^{-1}{A}^T$ for each full rank $A\in {ℝ}^{i\times j}$ [6].

Lemma 2.1 ([2, 6]).

Let $A,A_1\in {ℝ}^{m\times n}.$ Assume that and $rank(A)=rank(A_2).$ Then,

$$\Vert A_2^{†}\Vert \le \frac{\Vert A^{†}\Vert }{1-\Vert A^{†}\Vert \Vert A_1\Vert }.$$

If $rank(A)=rank(A_2)=\min \{m,n\},$ the following holds

$$\Vert A_2-A^{†}\Vert \le \frac{\sqrt{2}{\Vert A^{†}\Vert }^2\Vert A_1\Vert }{1-\Vert A^{†}\Vert \Vert A_1\Vert }.$$

Lemma 2.2 ([5]).

Let $A,A_1\in {ℝ}^{m\times n}.$ Assume that and $rank(A)=n,$ then $rank(A_2)=n.$

Lemma 2.3 ([12]).

Let $\phi (t)=\frac{1}{t}{\int }_0^{t}P(u)𝑑u,0\le t\le \rho ,$ where $P(u)$ is a positive integrable function and monotonically non-decreasing on $[0,\rho ].$ Then, $\phi (t)$ is monotonically non-decreasing with respect to $t.$

Lemma 2.4 ([12]).

Let $\psi (t)=\frac{1}{t^3}{\int }_0^{t}Q(u)𝑑u,0\le t\le \rho ,$ where $Q(u)$ is a positive integrable function and monotonically non-decreasing on $[0,\rho ].$ Then, $\psi (t)$ is monotonically non-decreasing with respect to $t.$

As in [9], it is convenient for the local convergence analysis that follows to introduce some functions and parameters:

$\alpha$ $=$ $\parallel F(p)\parallel ,\beta =\parallel (F^{\prime T}{F}^{\prime -1}{F}^{\prime T}\parallel ,$

$d(x)$ $=$ $\Vert x-p\Vert ,s_0=\max \{d(x_0),d(y_0)\},$

$\mu (t)$ $=$ $\mu (L_0,L,M)(t)$

$=$ ${\displaystyle \frac{\beta }{8}}{\displaystyle {\int }_0^t}M(u){(t-u)}^2𝑑u$

$+\beta t\left({\displaystyle {\int }_0^{\frac{3}{2}t}}L(u)𝑑u+{\displaystyle {\int }_0^t}{L}_0(u)𝑑u\right)+\sqrt{2}\alpha {\beta }^2{\displaystyle {\int }_0^t}{L}_0(u)𝑑u-t,$

$\gamma$ $=$ $\gamma (L_0,L,M)$

$=$ ${\displaystyle \frac{\beta {\int }_0^{d(x_0)}M(u){(d(x_0)-u)}^2𝑑u}{8d(x_0)\left(1-\beta {\int }_0^{d(x_0)}{L}_0(u)𝑑u\right)}}$

$+{\displaystyle \frac{\beta d(x_0){\int }_0^{\frac{d(x_0)+d(y_0)}{2}}L(u)𝑑u}{\frac{2d(x_0)+d(y_0)}{3}\left(1-\beta {\int }_0^{d(z_0)}{L}_0(u)𝑑u\right)}}$

$\delta$ $=$ ${\displaystyle \frac{\beta {\int }_0^{d(x_0)}M(u){(d(x_0)-u)}^2𝑑u}{8d{(x_0)}^3\left(1-\beta {\int }_0^{d(z_0)}{L}_0(u)𝑑u\right)}},$

$\tau$ $=$ ${\displaystyle \frac{\sqrt{2}\alpha {\beta }^2{\int }_0^{d(z_0)}{L}_0(u)𝑑u}{d(z_0)\left(1-\beta {\int }_0^{d(z_0)}{L}_0(u)𝑑u\right)}},$

$\lambda$ $=$ ${\displaystyle \frac{\beta {\int }_0^{\frac{d(x_0)+d(y_0)}{2}}L(u)𝑑u}{\frac{2d(x_0)+d(y_0)}{3}\left(1-\beta {\int }_0^{d(z_0)}{L}_0(u)𝑑u\right)}},$

$e_{n+1}^1$ $=$ $\delta d{(x_n)}^3+\lambda d(x_n)d(y_n)+\tau d(z_n),$

$e_{n+1}^2$ $=$ $\delta d{(x_{n+1})}^3+\frac{\lambda }{3}(d(x_n)+d(y_n)+d(x_{n+1}))d(x_{n+1})+\tau d(z_n),$

and

$$s_{n+1}=\max \{d(x_{n+1}),d(y_{n+1})\}.$$

Notice that if $L_0=L=L_1$ and $M=M_1,$ then the preceding definitions reduce to the corresponding ones in [9].

The local convergence analysis is based on the conditions ($𝒞$):

1. Mapping $F:\overline{U}(p,R_1)⟶{ℝ}^i$ is twice Fréchet-differentiable, $F^{\prime }(p)$ has full rank and $p$ solves problem (1.1).

2. $F^{\prime }(x)$ satisfies: the center-Lipschitz condition with $L_0$ average on $\overline{U}(p,R_1)$ and the restricted Lipschitz condition with $L$ average on $\overline{U}(p,R_0);$ $F^{\prime \prime }(x)$ satisfies the restricted Lipschitz condition with $M$ average on $\overline{U}(p,R_0),$ where $L_0,L$ and $M$ are positive non-decreasing functions on $[0,\frac{3R_0}{2}].$

3. Function $\mu$ has a minimal zero $R^{\ast }$ in $[0,R_0],$ which also satisfies

Then, we can show the following local convergence result for TGNWTM under the conditions ($𝒞$) and the preceding notation.

Theorem 2.5.

Suppose that conditions ($𝒞$) hold. Then, sequences $\{x_n\},$ $\{y_n\},\{z_n\}$ generated for $x_0,y_0\in \overline{U}(p,R^{\ast })-\{p\}$ by TGNWTM are well defined in $\overline{U}(p,R^{\ast }),$ remain in $\overline{U}(p,R^{\ast })$ for each $n=0,1,2,\mathrm{\dots }$ and converge to $p.$ Moreover, the following estimates hold

(2.8) $$d(x_{n+1})\le e_{n+1}^1,$$

(2.9) $$d(y_{n+1})\le e_{n+1}^2,$$

and

(2.10) $$s_{n+1}\le \gamma s_k\le \mathrm{\dots }\le {\gamma }^{n+1}{s}_0.$$

Proof. The proof follows the corresponding one in [9] but there are differences where we use ($L_0,L$), $M$ instead of $L_1,M_1,$ respectively used in [9]. We shall use mathematical induction to show that iterates $\{x_k\},$ $\{y_k\},\{z_k\}$ are well defined converge to $p$ and the error estimates (2.8)–(2.10) are satisfied. Using TGNWTM for $n=0,$ we can write

$x_1-p$ $=$ $x_0-p-{[F^{\prime }{(z_0)}^T{F}^{\prime }(z_0)]}^{-1}{F}^{\prime }(z_0)F(x_0)$

$=$ ${[F^{\prime }{(z_0)}^T{F}^{\prime }(z_0)]}^{-1}{F}^{\prime }{(z_0)}^T[F^{\prime }(z_0)(x_0-p)-F(x_0)+F(p)]$

$+[F^{\prime T}{F}^{\prime -1}{F}^{\prime T}F(p)-{[F^{\prime }{(z_0)}^T{F}^{\prime }(z_0)]}^{-1}{F}^{\prime }{(z_0)}^{T}F(p)$

$=$ ${[F^{\prime }{(z_0)}^T{F}^{\prime }(z_0)]}^{-1}{F}^{\prime }{(z_0)}^{T}J\left(x_0\right)$

$+[F^{\prime T}{F}^{\prime -1}{F}^{\prime T}F(p)-{[F^{\prime }{(z_0)}^T{F}^{\prime }(z_0)]}^{-1}{F}^{\prime }{(z_0)}^{T}F(p),$

and

$y_1-p$ $=$ $x_1-p-{[F^{\prime }{(z_0)}^T{F}^{\prime }(z_0)]}^{-1}{F}^{\prime }{(z_0)}^{T}F(x_1)$

$=$ ${[F^{\prime }{(z_0)}^T{F}^{\prime }(z_0)]}^{-1}{F}^{\prime }{(z_0)}^T[F^{\prime }(z_0)(x_1-p)-F(x_1)+F(p)]$

$+[F^{\prime T}{F}^{\prime -1}{F}^{\prime T}F(p)-{[F^{\prime }{(z_0)}^T{F}^{\prime }(z_0)]}^{-1}{F}^{\prime }{(z_0)}^{T}F(p)$

$=$ ${[F^{\prime }{(z_0)}^T{F}^{\prime }(z_0)]}^{-1}{F}^{\prime }{(z_0)}^{T}J\left(x_1\right)$

$+[F^{\prime T}{F}^{\prime -1}{F}^{\prime T}F(p)-{[F^{\prime }{(z_0)}^T{F}^{\prime }(z_0)]}^{-1}{F}^{\prime }(z_0)F(p),$

where

$J\left(x_i\right)$ $=$ $F^{\prime }\left(\frac{x_i+p}{2}\right)(x_i-p)-F(x_i)+F(p)$

$+\left(F^{\prime }(z_0)-F^{\prime }\left(\frac{x_i+p}{2}\right)\right)(x_i-p).$

In view of the estimate

$F(x)-F(y)-F^{\prime }\left(\frac{x+y}{2}\right)(x-y)$ $={\displaystyle {\int }_0^1}\frac{1-t}{4}(F^{\prime \prime }(\frac{x+y}{2}+\frac{t}{2}(x-y))$

$-F^{\prime \prime }(\frac{x+y}{2}+\frac{t}{2}(y-x)))(x-y){}^{2}dt,$

for $x=p$ and $y=x_0,$ we obtain in turn

$\Vert F(p)-F(x_0)-F^{\prime }\left(\frac{x_0+p}{2}\right)(p-x_0)\Vert$

$={\displaystyle \frac{1}{4}}\parallel {\displaystyle {\int }_0^1}(1-t)[F^{\prime \prime }(\frac{x_0+p}{2}+\frac{t}{2}(p-x_0))$

$-F^{\prime \prime }(\frac{x_0+p}{2}+\frac{t}{2}(x_0-p))](p-x_0){}^{2}dt\parallel$

$\le {\displaystyle {\int }_0^1}(1-t){\displaystyle {\int }_0^{t\Vert x_0-p\Vert }}M(u)𝑑u{\Vert x_0-p\Vert }^2𝑑t$

$={\displaystyle \frac{1}{8}}{\displaystyle {\int }_0^{d(x_0)}}M(u){\left(1-\frac{u}{d(x_0)}\right)}^2𝑑ud{(x_0)}^2$

$={\displaystyle \frac{1}{8}}{\displaystyle {\int }_0^{d(x_0)}}M(u){(d(x_0)-u)}^2𝑑u,$

and

$$\Vert F^{\prime }\left(\frac{x_0+y_0}{2}\right)-F^{\prime }\left(\frac{x_0+p}{2}\right)\Vert \le {\int }_0^{d(y_0)/2}L(u)𝑑u.$$

By the central Lipschitz condition, we have that

$$\Vert {\left(F^{\prime }{\left(p\right)}^T{F}^{\prime }\left(p\right)\right)}^{-1}{F}^{\prime }{\left(p\right)}^T\Vert \Vert F^{\prime }(x)-F^{\prime }(p)\Vert \le \beta {\int }_0^{d(x)}{L}_0(u)𝑑u.$$

Moreover, by 2.1 and 2.2 and (${𝒞}_1$), for all $x\in U(p,R^{\ast }),$ we get

$$\Vert {\left(F^{\prime }{\left(x\right)}^T{F}^{\prime }\left(x\right)\right)}^{-1}{F}^{\prime }{\left(x\right)}^T\Vert \le \frac{\beta }{1-\beta {\int }_0^{d(x)}{L}_0(u)𝑑u}$$

and

$$\Vert {\left(F^{\prime }{\left(x\right)}^T{F}^{\prime }\left(x\right)\right)}^{-1}{F}^{\prime }{\left(x\right)}^T-{\left(F^{\prime }{\left(p\right)}^T{F}^{\prime }\left(p\right)\right)}^{-1}{F}^{\prime }{\left(p\right)}^T\Vert \le \frac{\sqrt{2}{\beta }^2{\int }_0^{d(x)}{L}_0(u)𝑑u}{1-\beta {\int }_0^{d(x)}{L}_0(u)𝑑u}.$$

By the monotonicity of $L(u)$ and $M(u)$ with 2.3 and 2.4, functions $\frac{1}{t}{\int }_0^{t}L(u)𝑑u$ and $\frac{1}{t^3}{\int }_0^{t}M(u){(t-u)}^2𝑑u$ are non-decreasing in $t.$ That is, by (${𝒞}_3$)

$\gamma$ $=$ ${\displaystyle \frac{1}{R_0}}\left[{\displaystyle \frac{\beta {\int }_0^{R_0}M(u){(R_0-u)}^2𝑑u}{8(1-\beta {\int }_0^{R_0}L(u)𝑑u)}}+{\displaystyle \frac{\beta R_0{\int }_0^{\frac{3}{2}{R}_0}L(u)𝑑u}{1-\beta {\int }_0^{R_0}{L}_0(u)du)}}+{\displaystyle \frac{\sqrt{2}\alpha {\beta }^2{\int }_0^{R_0}{L}_0(u)𝑑u}{1-\beta {\int }_0^{R_0}{L}_0(u)du)}}\right]$

${\displaystyle \frac{1}{R}}\left[{\displaystyle \frac{\beta {\int }_0^{R}M(u){(R-u)}^2𝑑u}{8(1-\beta {\int }_0^R{L}_0(u)𝑑u)}}+{\displaystyle \frac{\beta R{\int }_0^{\frac{3}{2}R}L(u)𝑑u}{1-\beta {\int }_0^R{L}_0(u)du)}}+{\displaystyle \frac{\sqrt{2}\alpha {\beta }^2{\int }_0^R{L}_0(u)𝑑u}{1-\beta {\int }_0^R{L}_0(u)du)}}\right]$

$\le$ $1.$

Thus, by 2.1–2.9 and condition (${𝒞}_2$), we have in turn

$\Vert x_1-p\Vert$ $\le$ $\Vert {[F^{\prime }{(z_0)}^T{F}^{\prime }(z_0)]}^{-1}{F}^{\prime }{(z_0)}^T\Vert$

$\times \parallel {\displaystyle {\int }_0^1}(1-t)[F^{\prime \prime }(\frac{x_0+p}{2}+\frac{t}{2}(p-x_0))$

$-F^{\prime \prime }(\frac{x_0+p}{2}+\frac{t}{2}(x_0-p))](p-x_0){}^{2}dt\parallel$

$+\parallel [F^{\prime T}{F}^{\prime -1}{F}^{\prime T}F(p)$

$-{[F^{\prime }{(z_0)}^T{F}^{\prime }(z_0)]}^{-1}{F}^{\prime }{(z_0)}^{T}F(p)\parallel$

$\le$ ${\displaystyle \frac{\beta d{(x_0)}^3{\int }_0^{d(x_0)}M(u){(d(x_0)-u)}^2𝑑u}{8d{(x_0)}^3(1-\beta {\int }_0^{d(x_0)}{L}_0(u)𝑑u)}}$

$+{\displaystyle \frac{\beta d(x_0)d(y_0){\int }_0^{d(y_0)/2}L(u)𝑑u}{d(y_0)(1-\beta {\int }_0^{d(x_0)}{L}_0(u)𝑑u)}}+{\displaystyle \frac{\sqrt{2}\alpha {\beta }^2{\int }_0^{d(x_0)}{L}_0(u)𝑑u}{d(z_0)(1-\beta {\int }_0^{d(x_0)}{L}_0(u)𝑑u)}}$

In an analogous way, we get in turn

$\Vert y_1-p\Vert$ $\le$ $\Vert {[F^{\prime }{(z_0)}^T{F}^{\prime }(z_0)]}^{-1}{F}^{\prime }{(z_0)}^T\Vert$

$\times \parallel F^{\prime }\left(\frac{x_1+p}{2}\right)(x_1-p)-F(x_1)+F(p)$

$+(F^{\prime }(z_0)-F^{\prime }\left(\frac{x_1+p}{2}\right))(x_1-p)\parallel$

$+\parallel [F^{\prime T}{F}^{\prime -1}{F}^{\prime T}F(p)-{[F^{\prime }{(z_0)}^T{F}^{\prime }(z_0)]}^{-1}{F}^{\prime }{(z_0)}^{T}F(p)\parallel$

$\le$ ${\displaystyle \frac{\beta d{(x_1)}^3{\int }_0^{d(x_1)}M(u){(d(x_1)-u)}^2𝑑u}{8d{(x_1)}^3\left(1-\beta {\int }_0^{d(x_1)}{L}_0(u)𝑑u\right)}}$

$+{\displaystyle \frac{\beta d(x_1)d(z_0^{\prime }){\int }_0^{d(z_0)/2}L(u)𝑑u}{d(z_0^{\prime })\left(1-\beta {\int }_0^{d(z_0)}{L}_0(u)𝑑u\right)}}+{\displaystyle \frac{\sqrt{2}\alpha {\beta }^2{\int }_0^{d(z_0)}{L}_0(u)𝑑u}{d(z_0)\left(1-\beta {\int }_0^{d(z_0)}{L}_0(u)𝑑u\right)}}$

$\le$ $\delta d{(x_1)}^3+\frac{\lambda }{3}d(x_1)(d(x_0)+d(y_0)+d(x_1))+\tau d(z_0)$

hold, where $d(z_0^{\prime })=(d(x_0)+d(y_0)+d(x_1))/2,$ so $x_1,y_1\in U(p,R)$ and we also have that

$$R_1=\max \{\Vert x_1-p\Vert ,\Vert y_1-p\Vert \}\le \delta R_0,$$

so (2.10) is satisfied. Suppose that $x_k,y_k\in U(p,R)$ and (2.10) hold for $k>0.$ By TGNWTM for $k+1$ we get in turn that

$\Vert x_{k+1}-p\Vert$ $\le$ ${\displaystyle \frac{\beta d{(x_k)}^3{\int }_0^{d(x_k)}M(u){(d(x_k)-u)}^2𝑑u}{8d{(x_k)}^3\left(1-\beta {\int }_0^{d(x_k)}{L}_0(u)𝑑u\right)}}$

$+{\displaystyle \frac{\beta d(x_k)d(y_k){\int }_0^{d(y_k)/2}L(u)𝑑u}{d(y_k)\left(1-\beta {\int }_0^{d(y_k)}{L}_0(u)𝑑u\right)}}+{\displaystyle \frac{\sqrt{2}\alpha {\beta }^{2}d(z_k){\int }_0^{d(z_k)}{L}_0(u)𝑑u}{d(z_k)\left(1-\beta {\int }_0^{d(z_k)}{L}_0(u)𝑑u\right)}}$

$\le$ ${\displaystyle \frac{\beta d{(x_k)}^3{\int }_0^{d(x_0)}M(u){(d(x_0)-u)}^2𝑑u}{8d{(x_0)}^3\left(1-\beta {\int }_0^{d(x_0)}{L}_0(u)𝑑u\right)}}$

$+{\displaystyle \frac{\beta d(x_k)d(y_k){\int }_0^{d(y_0)/2}{L}_0(u)𝑑u}{d(y_0)\left(1-\beta {\int }_0^{d(y_0)}{L}_0(u)𝑑u\right)}}+{\displaystyle \frac{\sqrt{2}\alpha {\beta }^{2}d(z_k){\int }_0^{d(z_0)}{L}_0(u)𝑑u}{d(z_0)\left(1-\beta {\int }_0^{d(z_0)}{L}_0(u)𝑑u\right)}}$

$\le$

and

$\Vert y_{k+1}-p\Vert$ $\le$ ${\displaystyle \frac{\beta d{(x_{k+1})}^3{\int }_0^{d(x_{k+1})}M(u){(d(x_{k+1})-u)}^2𝑑u}{8d{(x_{k+1})}^3\left(1-\beta {\int }_0^{d(x_k)}{L}_0(u)𝑑u\right)}}$

$+{\displaystyle \frac{\beta d(x_{k+1})d(z_k^{\prime }){\int }_0^{d(z_k^{\prime })/2}L(u)𝑑u}{d(z_k^{\prime })\left(1-\beta {\int }_0^{d(z_k)}{L}_0(u)𝑑u\right)}}+{\displaystyle \frac{\sqrt{2}\alpha {\beta }^{2}d(z_k){\int }_0^{d(z_k)}L(u)𝑑u}{d(z_k)\left(1-\beta {\int }_0^{d(z_k)}{L}_0(u)𝑑u\right)}}$

$\le$ ${\displaystyle \frac{\beta d{(x_{k+1})}^3{\int }_0^{d(x_0)}M(u){(d(x_0)-u)}^2𝑑u}{8d{(x_0)}^3\left(1-\beta {\int }_0^{d(x_0)}{L}_0(u)𝑑u\right)}}$

$+{\displaystyle \frac{\beta d(x_{k+1})d(z_k^{\prime }){\int }_0^{d(z{0}^{\prime })}L(u)𝑑u}{d(z_0^{\prime })\left(1-\beta {\int }_0^{d(z_0)}{L}_0(u)𝑑u\right)}}+{\displaystyle \frac{\sqrt{2}\alpha {\beta }^{2}d(z_k){\int }_0^{d(z_0)}{L}_0(u)𝑑u}{d(z_0)\left(1-\beta {\int }_0^{d(z_0)}{L}_0(u)𝑑u\right)}}$

$\le$ $\delta d{(x_{k+1})}^3+\frac{\lambda }{3}(d(x_k)+d(y_k)+d(x_{k+1}))d(x_{k+1})+\tau d(z_k)$

where $d(z_k^{\prime })=(d(x_k)+d(y_k)+d(x_{k+1}))/2.$ Furthermore, we obtain

$$R_{k+1}=\max \{\Vert x_{k+1}-p\Vert ,\Vert y_{k+1}-p\Vert \}\le \delta R_k\le {\delta }^2{R}_{k-1}\le \mathrm{\dots }{\delta }^{k+1}{R}_0,$$

so

$$x_{k+1},y_{k+1}\in U(p,R),\text{(}\text{<a href="\#S2.E8" title="Equation 2.8 ‣ Theorem 2.5. ‣ 2. Local convergence analysis ‣ Local convergence of a two-step Gauss-Newton Werner-type method for solving least squares problems" class="ltx\_ref"><span class="ltx\_text ltx\_ref\_tag">2.8</span></a>}\text{), (}\text{<a href="\#S2.E10" title="Equation 2.10 ‣ Theorem 2.5. ‣ 2. Local convergence analysis ‣ Local convergence of a two-step Gauss-Newton Werner-type method for solving least squares problems" class="ltx\_ref"><span class="ltx\_text ltx\_ref\_tag">2.10</span></a>}\text{) hold,}$$

${lim}_{k⟶\mathrm{\infty }}{x}_k=p$ and ${lim}_{k⟶\mathrm{\infty }}{y}_k=p.$

$\mathrm{\square }$

Concerning the uniqueness of the solution $p$ we have:

Proposition 2.6.

Under the conditions ($𝒞$) further suppose that

(2.11)

where $\overline{\beta }=\parallel [F^{\prime T}{F}^{\prime -1}\parallel .$ Then, limit point $p$ is the only solution of problem (1.1) in $\overline{U}(p,R^{\ast }).$

The proof follows from the corresponding one in [5] but we only use the center-Lipschitz condition.

3. Special cases and applications

Remark 3.1.

1. Set $\alpha =\Vert F(p)\Vert =0$ in 2.5 and 2.6 to obtain the results in the case of zero residual.

2. If $L_0,L,M$ are constants, then we can obtain results of special cases.

3. In the literature functions $L_1$ and $M_1$ are used instead of $L$ and $M,$ respectively [3, 5, 8, 9, 12]. Let us compare ratios and ball of convergence. Notice that in view of (2.1), (2.4), (2.5) and (2.7), we have

   (3.1) $$\mu (L_0,L,M)(t)\le \mu (L_1,L_1,M_1)(t),$$

   and

   (3.2) $$\gamma (L_0,L,M)(t)\le \gamma (L_1,L_1,M_1)(t),$$

   so

   (3.3) $$R^{\ast }(L_1,L_1,M_1)(t)\le R^{\ast }(L_0,L,M)(t).$$

Therefore, our radius of convergence is larger and our ratio of convergence is smaller. Moreover the information on the location of the solution $p$ is more precise, since only $L$ is used in (2.11) [9]. Notice that these advantages are obtained under the same computational cost, since in practice the computation of $L_1$ and $M_1$ require the computation of the rest of the functions $L_0,L$ and $M$ as special cases.

Remark 6.

In particular, using the error estimates, it follows that for $\alpha =0$ we have $\tau =0$ and

$d(x_{k+1})$ $\le$ $d(x_k)(\delta d{(x_k)}^2+\lambda d(y_k))$

$d(y_{k+1})$ $\le$ $d(x_{k+1})\left[\delta d{(x_{k+1})}^2+\frac{\lambda }{3}\left(d(x_k)+d(x_{k+1})+d(y_k)\right)\right]$

$\le$ $d(x_{k+1})\left[\left(\delta d(x_k)+\frac{2\lambda }{3}\right)d(x_k)+\frac{\lambda d(y_k)}{3}\right]$

$\le$ $d(x_{k+1})d(x_k)(\delta R^{\ast }+\lambda )$

$=$ $d(x_{k+1})d(x_k){\mathrm{\ell }}_1.$

Also, for sufficiently large $k$,

$d(x_{k+1})$ $\le$ $d(x_k)(\delta d{(x_k)}^2+\lambda d(y_k))$

$\le$ $d(x_k)(\delta d{(x_k)}^2+\lambda {\mathrm{\ell }}_{1}d(x_k)d(x_{k-1}))$

$\le$ $d{(x_k)}^{2}d(x_{k-1})(\delta +\lambda {\mathrm{\ell }}_1)$

$=$ $d{(x_k)}^{2}d(x_{k-1}){\mathrm{\ell }}_2,$

leading to the equation

$$x^2-2x-1=0,$$

so the order of iterative method (1) is the positive root of the preceding equation which is $1+\sqrt{2}.$

Next, we present an example to show that (3.1)–(3.3) hold as strict inequalities justifying the advantages as claimed at the introduction of this study.

EXAMPLE 3.2.

Let $X={ℝ}^3,D=\overline{U}(0,1),p={(0,0,0)}^T.$ Define function $F$ on $D$ for $w={(x,y,z)}^T$ by

$$F(w)={(e^x-1,\frac{e-1}{2}{y}^2+y,z)}^T.$$

Then, the Fréchet-derivative is given by

Notice that using the (2.9) conditions, we get $L_0=e-1,L=M=e^{1/L_0},L_1=M_1=e,\beta =1,i=j=3,\alpha =0.$ Then

which justify the improvements as stated in the introduction of this paper.