Coupled-decompositions: exploiting primal–dual interactions in convex optimization problems

Abstract

Decomposition techniques implement the so-called “divide and conquer” in convex optimization problems, being primal and dual decompositions the two classical approaches. Although both solutions achieve the goal of splitting the original program into several smaller problems (called the subproblems), these techniques exhibit in general slow speed of convergence. This is a limiting factor in practice and in order to circumvent this drawback, we develop in this article the coupled-decompositions method. As a result, the number of iterations can be reduced by more than one order of magnitude. Furthermore, the new technique is self-adjustable, i.e., it does not depend on user-defined parameters, as opposite to what happens with classical strategies. Given that in signal processing applied to communications and networking we usually deal with a variety of problems that exhibit certain coupling structures, our method is useful to design decentralized as well as centralized optimization schemes with advantages over the existing techniques in the literature. In this article, we expose there different resource allocation problems where the proposed method is successfully applied.

1 Introduction

Convex optimization theory [1, 2] has provided in the last decades a powerful framework to solve optimization problems in many distinct areas. Besides the numerous applications existing in the signal processing literature, it is also possible to find examples in topics such as filter design, machine learning, or finance among others. This great success has been motivated by (i) convex optimization provides relevant insights into each specific problem, thanks to a mature theoretical framework, (ii) some problems can be solved analytically or semi-analytically applying the so-called Karush–Kuhn–Tucker (KKT) optimality conditions, and (iii) efficient numerical methods, e.g., interior point methods, have been developed to solve generic convex problems in polynomial time.

In many engineering areas, optimization problems with a partially coupled structure arise. In particular, we consider programs where the objective can be expressed as a sum of functions that depend on disjoint sets of variables, which are additionally coupled by the problem constraints (e.g., [3–5]). The optimization of such programs is the topic addressed by decomposition methods [6] and a common strategy is to split the original problem into several smaller subproblems that are somehow coordinated until they reach the optimal solution. Additionally and as a by-product, the resulting methods can deal more naturally with decentralized implementations [7, 8].

However, existing decomposition methods exhibit some drawbacks in practice. Roughly speaking, the speed of convergence of the algorithms is in general slow (this can be appreciated, for instance, in the numerical examples of [6]) and furthermore, it is necessary to manually adjust the step-size used in the successive updates of the algorithms. Since there is no universal rule to do that optimally, the performance of the methods is compromised [9]. In order to overcome these drawbacks, we introduce a novel technique, the coupled-decompositions method (CDM). It can be applied to decentralized implementations and furthermore, due to its superior computational performance in terms of convergence speed, the new technique is also competitive when compared to well-established centralized methods.

In the following, we synthesize the main contributions of this article: (i) development of new interactions between the primal and dual domains in convex decomposition problems, (ii) development of a new method based on these novel interactions for problems with a single coupling constraint, (iii) convergence proof of the proposed method, (iv) further analysis of the method when it is applied to a subset of the problems of interest, and (v) presentation of numerical examples that show the benefits of having an unsupervised and efficient solution (in terms of both computational cost and convergence speed).

The remainder of the article is organized as follows. Section 2 formulates the type of problems that we deal with and it also reviews the classical decomposition techniques. Section 3 describes the proposed CDM and proves its convergence to the optimal solution whereas Section 4 provides further analysis on the proposed method when the problem is particularized. Finally, Section 5 presents numerical examples of the proposed method and Section 6 concludes the article.

2 Problem formulation and existing solutions

In this section, we first define the type of problems that we deal with throughout the text. Thereafter, the existing decomposition techniques in the literature are reviewed.

2.1 Problem formulation

Let us consider the following optimization problem,

$$\begin{array}{cc}\underset{\left\{{\mathit{x}}_j\right\}}{\text{min}}& \sum _{j=1}^J{f}_j\left({\mathit{x}}_j\right)\\ \mathrm{s.t.}& {\mathit{x}}_j\in {\mathcal{X}}_j,& j=1,\dots ,J\\ \sum _{j=1}^J{h}_j\left({\mathit{x}}_j\right)\le C\end{array}$$

(1)

with variables ${\mathit{x}}_j\in {\mathbb{R}}^{n_j}$. The functions $f_j,h_j:{\mathbb{R}}^{n_j}\to \mathbb{R}$ are assumed convex and differentiable in the sets ${\mathcal{X}}_j$, also convex and compact too. These sets are defined as ${\mathcal{X}}_j=\left\{{\mathit{x}}_j\right|{\mathit{g}}_j\left({\mathit{x}}_j\right)\preccurlyeq 0\}$ ^a with ${\mathit{g}}_j\left({\mathit{x}}_j\right)={\left[g_j^1\right({\mathit{x}}_j),\dots ,g_j^{G_j}({\mathit{x}}_j\left)\right]}^T$, where the functions $g_j^k:{\mathbb{R}}^{n_j}\to \mathbb{R}$ are convex and differentiable. Therefore, Equation (1) defines a convex problem and if we further assume that its feasible region has non-empty relative interior, then strong duality holds.

Note that we may interpret (1) as the distribution of a quantity C of resources among J entities where the j th entity aims to set the values of the variables in x _j (constrained to lie in ${\mathcal{X}}_j$) in order to minimize the global cost function $\sum _{j=1}^J{f}_j\left({\mathit{x}}_j\right)$ without exceeding the coupling constraint $\sum _{j=1}^J{h}_j\left({\mathit{x}}_j\right)\le C$. The presented formulation applies, among others, to fair dynamic bandwidth allocation (DBA) in point-to-multipoint networks [4], to problems related with multiple-input multiple-output design [3, 10, 11] or problems related to OFDM system design [12, 13].

The problem in (1) is suitable for a dual decomposition approach and also for a primal decomposition if it is adequately reformulated. In the next sections, those classical solutions are reviewed.

2.2 Primal decomposition

Let us consider the following modified version of (1),

$$\begin{array}{cc}\underset{\left\{{\mathit{x}}_j\right\},\mathit{y}}{\text{min}}& \sum _{j=1}^J{f}_j\left({\mathit{x}}_j\right)\\ \mathrm{s.t.}& {\mathit{x}}_j\in {\mathcal{X}}_j,& j=1,\dots ,J\\ h_j\left({\mathit{x}}_j\right)\le y_j,& j=1,\dots ,J\\ \sum _{j=1}^J{y}_j\le C\\ \mathit{y}\in \mathcal{Y},& \mathcal{Y}={\mathcal{Y}}_1\times \dots \times {\mathcal{Y}}_J\end{array}$$

(2)

where we have introduced the coupling variables y= [y ₁,…,y _J ]^T. The subsets ${\mathcal{Y}}_j\in \mathbb{R}$ are defined as the images of ${\mathcal{X}}_j$ through the functions h _j , i.e., $h_j:{\mathcal{X}}_j\to {\mathcal{Y}}_j$. Since the functions h _j are convex over ${\mathcal{X}}_j$ and so continuous, the subsets ${\mathcal{Y}}_j$ are guaranteed to be compact ([14], Th. 5.2.2). Therefore, each ${\mathcal{Y}}_j$ has both a minimum and a maximum.

In primal decomposition, we assume that the coupling variables are fixed to a given value $\mathit{y}\in \mathcal{Y}$ (more details can be found in [15], Sec. 6.4.2). Then, the problem in (2) is solved as J independent problems in the variables x _j . They are called the subproblems and they are expressed as

$$p_j\left(y_j\right)=\left\{\begin{array}{cc}\underset{{\mathit{x}}_j}{\text{min}}& f_j\left({\mathit{x}}_j\right)\\ \mathrm{s.t.}& h_j\left({\mathit{x}}_j\right)\le y_j\\ {\mathit{x}}_j\in {\mathcal{X}}_j\end{array}\right.$$

(3)

Interestingly, we know from ([15], Sec. 5.4.4) that −λ _j , i.e., minus the Lagrange multiplier associated to the constraint h _j (x _j )≤y _j , is in fact a subgradient^b of p _j at y _j .

Having defined the primal subproblems, we can rewrite (2) as

$$\begin{array}{cc}\underset{\left\{y_j\right\}}{\text{min}}& \sum _{j=1}^J{p}_j\left(y_j\right)\\ \mathrm{s.t.}& \sum _{j=1}^J{y}_j\le C\\ \mathit{y}\in \mathcal{Y}\end{array}$$

(4)

and (4) is referred to as the primal master problem. Note that since the subgradients of the primal subproblems are obtained at no cost, we can use a projected gradient approach ([15], Sec. 2.3) to solve the problem. In other words, the following recursion (k indexes iterations)

$${\mathit{y}}^{k+1}={\left[{\mathit{y}}^k-{\alpha }^k{\mathit{s}}^k\right]}^{‡}$$

(5)

with ${\mathit{s}}^k=-{\left[{\lambda }_1^{\ast }\right(y_1^k),\dots ,{\lambda }_J^{\ast }(y_J^k\left)\right]}^T$ and where [·]^‡is the projection onto the feasible set (i.e., $\mathit{y}\in \left\{\mathit{y}\right|\mathit{y}\in \mathcal{Y},\sum _{j=1}^J{y}_j\le C\}$) converges to y ^∗. The interested reader can find more details about primal decomposition in ([15], Sec. 6.4.2) and also in [6]. However, note that it is necessary to appropriately adjust the value of α ^k in order to guarantee the convergence to the desired solution [6, 9].

2.3 Dual decomposition

Dual decomposition is the dual-domain alternative to primal decomposition. Let us compute the partial Lagrangian of (1) by means of relaxing only the coupling constraint

$$q(\mu )=\sum _{j=1}^J\underset{{\mathit{x}}_j\in {\mathcal{X}}_j}{\text{min}}\left\{f_j\right({\mathit{x}}_j)+\mu h_j({\mathit{x}}_j\left)\right\}-\mu C$$

(6)

Clearly, the problem in (6) decouples into J independent problems, called the dual subproblems and defined as

$$q_j(\mu )=\left\{\begin{array}{cc}\underset{{\mathit{x}}_j}{\text{min}}& f_j\left({\mathit{x}}_j\right)+\mu h_j\left({\mathit{x}}_j\right)\\ \mathrm{s.t.}& {\mathit{x}}_j\in {\mathcal{X}}_j\end{array}\right.$$

(7)

Note that the dual subproblems are convex programs for μ ≥ 0 and that given a value of μ, the values of the variables in x _j are found after solving the subproblems in (7) for j = 1,…,J, which can be computed in parallel. In particular, the optimal values of the primal variables, i.e., {x _j }, are obtained from an optimal value of the dual variable, i.e., μ ^∗.

Using the dual subproblems, the dual master problem is written as

$$\begin{array}{cc}\underset{\mu }{\text{max}}& \sum _{j=1}^J{q}_j(\mu )-\mathrm{\mu C}\\ \mathrm{s.t.}& \mu \ge 0\end{array}$$

(8)

and, as in primal decomposition, a projected gradient approach can be applied ([15], Sec. 6.4.1) to finally get μ ^∗. The recursion is

$${\mu }^{k+1}={[{\mu }^k+{\alpha }^k{s}^k]}^{+}$$

(9)

where k indexes iterations and [a]⁺= max{0,a}. As well as in primal decomposition, it can be shown that a subgradient of q _j at μ ^k is readily found as^c $h_j\left({\mathit{x}}_j^{\ast }\left({\mu }^k\right)\right)$ once the dual subproblems are solved ([15], Sec. 6.1) and therefore, a subgradient of q at μ ^k is given by $s^k=\sum _{j=1}^J{h}_j\left({\mathit{x}}_j^{\ast }\left({\mu }^k\right)\right)-C$. Finally, note that a user-defined step-size is also necessary in dual decomposition and, as we discuss later, this is a serious drawback of classical decomposition methods in practice.

2.4 Primal–dual techniques

There is a huge list of methods in the literature that are termed primal–dual but, to the best of our knowledge, the essentials in our proposed CDM have not been established previously. In general, all the reviewed methods suffer from (i) slow speed of convergence to the optimal solution (this restricts the number of practical applications), (ii) no consideration for the separated nature of the problem (i.e., the techniques are not decomposition-based approaches), and/or (iii) the decentralized implementation of the methods is not taken into account. On the contrary, all these aspects are addressed in the proposed CDM.

A first group of existing primal–dual techniques focus on iteratively finding a saddle-point of the Lagrangian, which is a convex and concave function of the primal and dual variables, respectively. Although these methods were not originally conceived from a decomposition perspective, they can be applied to the problems of interest in this article (and also implemented in a decentralized manner). Among these techniques, we find the classical work of Arrow et al. [16] or the more recent Mean Value Cross (MVC) decompositions method [17, 18]. However, both techniques need to fix an step-size (explicit or implicit as in the MVC decompositions method) and, as a consequence, they penalize in terms of convergence speed in practice.

In a second group of techniques we include all the possible combinations of classical primal and dual decompositions, as described in [6]. The idea in this case is to solve some parts of the problem with a primal decomposition approach while other parts are tackled by means of a dual decomposition. Therefore, these solutions do not consider full primal–dual interactions as in the proposed CDM, where each part considers both domains simultaneously. Furthermore, they also suffer from slow convergence speeds due, in part, to the manually adjusted step-sizes. However, it is important to remark that in the last decade a significant progress has been made in dual-decomposition-based solutions using smoothing [19] or path-following [20] strategies, improving the number of iterations of the classical dual decomposition by an order of magnitude. Notwithstanding, these methods tackle problems with linear constraints and are not designed under a decentralized implementation perspective.

Finally, let us mention the primal–dual interior point methods ([2], Sec. 11.7) and its variants [21, 22] as the third group of primal–dual approaches. In this case, the basic idea is to iteratively solve the KKT conditions of the problem using numerical methods typically applied to the resolution of systems of nonlinear equations such as the Newton method. These techniques have received great attention during the past years due to their good performance in terms of convergence speed when used in generic convex problems. However, since they were not conceived to exploit the separability of the problem (if it exists), it is not straightforward to derive decentralized solutions from this third group of techniques (one of the goals in this article).

3 The CDM

In order to overcome the detected drawbacks, we design our CDM with the aim to (i) exploit the primal and dual domains in convex optimization problems and (ii) simultaneously benefit from the separability of the problem in order to derive decentralized solutions. Although there are solutions in the literature that exploit both solution domains as discussed, the development of a fast technique satisfying (i) and (ii) is still pending. In the following, we first describe our proposed CDM and thereafter, we prove that the iterates of the method convergence to the optimal solution.

3.1 Description of the method

The proposed method has four building blocks: the primal subproblems, the dual subproblems, the primal projection, and the dual projection. These blocks are connected as depicted in Figure 1 and in what follows, we describe the actions taken at each step of the method and we provide a summary of the technique in algorithmic form. Thereafter, the convergence of the successive updates of the CDM, i.e., μ ^k, towards an optimal value of the dual variable, i.e., μ ^∗, is proved (and the same is valid for the rest of variables, primal, and dual).

Figure 1

Block diagram of the CDM.

3.1.1 Step 1: dual subproblems

From μ ^k, the primal value $y_j^k$ is obtained after solving the following convex optimization problem in d _j

$$d_j\left({\mu }^k\right)=\left\{\begin{array}{cc}\underset{{\mathit{x}}_j,y_j^k}{\text{min}}& f_j\left({\mathit{x}}_j\right)+{\mu }^k{y}_j^k\\ \mathrm{s.t.}& h_j\left({\mathit{x}}_j\right)\le y_j^k\\ {\mathit{x}}_j\in {\mathcal{X}}_j\end{array}\right.$$

(10)

Note that d _j (μ ^k) coincides with (7) if we substitute $y_j^k$ by h _j (x _j ). Note also that ${\lambda }_j^k$, i.e., the dual variable associated to the constraint $h_j\left({\mathit{x}}_j\right)\le y_j^k$, always takes the value of μ ^k. This can be checked using one of the KKT optimality conditions of the problem as follows

$$\frac{\partial L({\mathit{x}}_j,y_j^k,{\lambda }_j^k,\dots )}{\partial y_j^k}={\mu }^k-{\lambda }_j^k=0\to {\lambda }_j^k={\mu }^k$$

(11)

where $L({\mathit{x}}_j,y_j^k,{\lambda }_j^k,\dots )$ stands for the Lagrangian function of the problem. The interested reader can find more details on the Lagrangian function as well as on the KKT optimality conditions of convex problems in ([2], Sec. 5.1, Sec. 5.5).

3.1.2 Step 2: primal projection

In the second step of the method, the values in $y_j^k$ from all the subproblems are grouped in ${\mathit{y}}^k={[y_1^k,\dots ,y_J^k]}^T$ and projected to the subset $\mathcal{Y}\cap \left\{\mathit{y}\right|\sum _i{y}_i=C\}$ if μ ^k> 0 and to the subset $\mathcal{Y}\cap \left\{\mathit{y}\right|\sum _i{y}_i\le C\}$ otherwise. Note that both projections force the values in ${\widehat{\mathit{y}}}^k$ to be feasible and that the choice of the projection subset depending on μ is in accordance with the complementary slackness constraint $\mu (\sum _{j=1}^J{y}_j-C)=0$ of the problem.

In the most usual case, that is, for μ ^k> 0, the following convex problem has to be solved

$$\begin{array}{cc}\underset{{\widehat{\mathit{y}}}^k}{\text{min}}& \left|\right|{\mathit{y}}^k-{\widehat{\mathit{y}}}^k|{|}^2\\ \mathrm{s.t.}& \sum _{j=1}^J{ŷ}_j^k=C\\ {\widehat{\mathit{y}}}^k\in \mathcal{Y}\end{array}$$

(12)

which can be done semi-analytically as discussed in “Proof of Proposition 2” in Appendix.

3.1.3 Step 3: primal subproblems

The j th primal subproblem is defined as

$$p_j\left({ŷ}_j^k\right)=\left\{\begin{array}{cc}\underset{{\mathit{x}}_j}{\text{min}}& f_j\left({\mathit{x}}_j\right)\\ \mathrm{s.t.}& h_j\left({\mathit{x}}_j\right)\le {ŷ}_j^k\\ {\mathit{x}}_j\in {\mathcal{X}}_j\end{array}\right.$$

(13)

and it can be solved once ${ŷ}_j^k$ is available. In this case, we are interested in the optimal value of the Lagrange multiplier associated to $h_j\left({\mathit{x}}_j\right)\le {ŷ}_j^k$, that is, ${\widehat{\lambda }}_j^k$. As later discussed in Section 3.4, the step 4 of the method uses only the values of ${\widehat{\lambda }}_j^k$ that result from ${ŷ}_j^k\notin \mathbf{bd}{\mathcal{Y}}_j$, where $\mathbf{bd}\mathcal{A}$ stands for the boundary of the subset $\mathcal{A}$. The selected values are then grouped in the list $\left\{{\breve{\lambda }}_j^k\right\}$, which is the input of the dual projection. Note that if $\sum _{j=1}^J{ŷ}_j^k=C$, then the list is guaranteed to be non-empty as shown in Proposition 3 (Section 3.4). Besides, it is important to solve the primal subproblem in (13) according to its dual version in (10). In other words, if ${ŷ}_j^k$ is fixed in the j th primal subproblem then ${\widehat{\lambda }}_j^k$ (not necessarily unique) is accepted as valid only if $d_j\left({\widehat{\lambda }}_j^k\right)$ gives $y_j^k={ŷ}_j^k$.

3.1.4 Step 4: dual projection

If $\sum _{j=1}^J{ŷ}_j^k=C$, a new update of μ, i.e., μ ^{k + 1}, is obtained as the solution of the following optimization problem

$$\begin{array}{cc}\underset{{\mu }^{k+1}}{\text{min}}& \left|\right|{\mu }^{k+1}-{\mu }^k|{|}^2\\ \mathrm{s.t.}& {\mu }^{k+1}\in \left\{{\breve{\lambda }}_j^k\right\}\end{array}$$

(14)

In other words, μ ^{k + 1} takes the value in $\left\{{\breve{\lambda }}_j^k\right\}$ that is the closest to μ ^k. As discussed in Section 3.3, this is equivalent to set ${\mu }^{k+1}=\text{min}\left\{\underset{j}{\overset{k}{\breve{\lambda }}}\right\}$ if μ ^k< μ ^∗ and ${\mu }^{k+1}=\text{max}\left\{\underset{j}{\overset{k}{\breve{\lambda }}}\right\}$ if μ ^k> μ ^∗.

If $\sum _{j=1}^J{ŷ}_j^k<C$ then μ ^{k + 1} is fixed to 0, which is in accordance with the complementary slackness constraint $\mu (\sum _{j=1}^J{y}_j-C)=0$.

3.2 The CDM in algorithmic form

Let us consider without loss of generality a decentralized implementation of the proposed method with a controller and J independent participants. Each participant is able to solve the corresponding primal and dual subproblems whereas the task of the controller is to compute the primal and dual projections. Note that both operations involve simple computations as discussed in the steps 2 and 4 above. The proposed CDM is then summarized in the following algorithm,

Choose an initial value for μ ⁰ and repeat

• 1. The controller sends μ ^k to the participants, which compute d _j (μ ^k) in (10) and return $y_j^k$.

• 2. With ${\mathit{y}}^k=[y_1^k,y_2^k,\dots ,y_J^k]$, the controller computes ${\mathit{ŷ}}^k$ using the primal projection (step 1 above) and sends ${ŷ}_j^k$ to the participants if ${ŷ}_j^k\notin \mathbf{bd}{\mathcal{Y}}_j$.

• 3. The participants compute $p_j\left({ŷ}_j^k\right)$ in (13) and return ${\widehat{\lambda }}_j^k$ to the controller.

• 4. The controller fixes μ ^{k + 1} to the received value that is closer to μ ^k.

Until convergence.

3.3 Resource–price interpretation

Often in convex optimization, primal variables are interpreted as resources and dual variables as prices to be paid for them. In the sequel, we revisit the proposed technique under this resource–price perspective. Initially, a global price μ ^k is fixed and sent to the parts. Given that price, the parts estimate the amount of resources they want to buy. Intuitively, there will be a deficit of resources (a total request over C) if the price is too low and an excess if it is too high. In both cases, the primal projection corrects the allocation in order to distribute all the available resources among the parts. However, there is no guarantee that the distribution follows a common market law. In order to correct the situation, the primal subproblems estimate the price to be paid for the new resource allocation and, in case the individual prices differ, the dual projection fixes a new common price μ ^{k + 1} in order to advance towards a consensus price μ ^∗.

3.4 Proof of the method

Before proving that the successive updates of the proposed method converge to the optimal solution, let us establish the relationship between primal and dual variables in the subproblems with the following proposition.

Proposition 1. Take the jth primal subproblem p _j in (13) and the jth dual subproblem d _j in (10) of the CDM. Then, the following two statements hold: (i) ${\widehat{\lambda }}_j^k\left({ŷ}_j^k\right)$ is non-increasing on ${ŷ}_j^k$ in (13) and (ii) $y_j^k\left({\mu }^k\right)$ is non-increasing on μ ^k in (10).

Proof. See “Proof of Proposition 1” in Appendix. □

Next, the goal is to verify that the primal and dual projections effectively coordinate the subproblems towards the optimal solution. Let us assume, without loss of generality, that the initial guess is μ ⁰ = 0 so that μ ⁰ ≤ μ ^∗. From that value, the CDM starts by solving the dual subproblems in (10) in order to obtain y ⁰. As a result, there are two possibilities, namely, (i) $\sum _j{y}_j^0\le C$ and (ii) $\sum _j{y}_j^0>C$. In the first situation, μ ⁰ as well as y ⁰ and the corresponding values in {x _j } are optimal. Note that the subproblems are in this case decoupled and therefore the individual optimization carried out in the dual subproblems is globally optimal, too. For the sake of brevity, we do not discuss here what are the outputs of the following steps and iterations of the method, but it can be checked that the solution remains unaltered as expected. In the second case, μ ⁰ = 0 is clearly non-optimal and in the sequel we show how the successive updates of μ ^kconverge to an optimal value of the dual variable, that is, μ ^∗ > 0.

Let us revisit then a complete iteration of the method starting at the dual subproblems in (10) with μ ^k< μ ^∗, which holds at least for k = 0. Since $y_j^k$ is a non-increasing function of μ ^kin the j th dual subproblem (see Proposition 1), μ ^k< μ ^∗ and $y_j^k\left({\mu }^{\ast }\right)=y_j^{\ast }$, it is true that $y_j^k\ge y_j^{\ast }$. Moreover, if we take into account that $\sum _{j=1}^J{y}_j^{\ast }=C$ (we are considering the case where the optimal solution is coupled), we can establish that $\sum _{j=1}^J{y}_j^k>C$ unless y ^k= y ^∗.

Thereafter, it is verified in the second step of the method (primal projection) that ${\widehat{\mathit{y}}}^k\preccurlyeq {\mathit{y}}^k$ (${ŷ}_j^k<y_j^k$ for some j) according to Proposition 2 next.

Proposition 2. Given the optimization problem in (12) and y ^k≽ y ^∗ (y ^k≠ y ^∗), its optimal solution can be expressed as ${\widehat{\mathit{y}}}^k={\mathit{y}}^k-\mathit{r}$ with r≽ 0(r _j > 0 for some j).

Proof. See “Proof of Proposition 2” in Appendix. □

In the third step of the method, the j th primal subproblem defined in (13) computes the individual price ${\widehat{\lambda }}_j^k$ and the list of individual prices $\left\{{\widehat{\lambda }}_j^k\right\}$ is constructed with the values obtained from the J independent subproblems, indexed by j = 1,…,J. Note, however, that our main interest is not in the prices ${\widehat{\lambda }}_j^k$ but in finding a global consensus price μ ^∗. Fortunately, if we come back to the problem definition in (2), we notice that there is a dependence between the dual variable associated to the constraint h _j (x _j )≤ y _j , i.e., λ _j , and the dual variable associated to the constraint $\sum _{j=1}^J{y}_j\le C$, i.e., μ (in terms of the proposed algorithm, ${ŷ}_j^k$, ${\widehat{\lambda }}_j^k$ and μ ^k play the role of y _j , λ _j and μ, respectively). This dependance motivates in our algorithm the selection of some of the values in the list $\left\{{\widehat{\lambda }}_j^k\right\}$. To be more specific, the value ${\widehat{\lambda }}_j^k$ is chosen if the corresponding primal variable ${ŷ}_j^k$ satisfies ${ŷ}_j^k\notin \mathbf{bd}{\mathcal{Y}}_j$ as discussed next.

Let us first write the Lagrangian of the problem in (2), that is

$$\begin{array}{c}\begin{array}{c}L\left(\right\{{\mathit{x}}_j\},\mathit{y},\mathit{\lambda },\mu ,\{{\mathit{\xi }}_j\},\{{\mathit{\psi }}_j\left\}\right)=\sum _{j=1}^J{f}_j\left({\mathit{x}}_j\right)+\sum _{j=1}^J{\mathit{\xi }}_j^T{\mathit{g}}_j\left({\mathit{x}}_j\right)\\ +\sum _{j=1}^J{\mathit{\psi }}_j^T{\mathit{q}}_j\left(y_j\right)+\sum _{j=1}^J{\lambda }_j\left(h_j\left({\mathit{x}}_j\right)-y_j\right)+\mu (\sum _{j=1}^J{y}_j-C)\end{array}\end{array}$$

(15)

where the set of convex functions q _j (y _j ) with associated Lagrange multipliers ψ _j define the subset ${\mathcal{Y}}_j$. From the Lagrangian function we derive some of the KKT optimality conditions of the convex optimization problem as far as the optimal values of the variables form a saddle-point in the function plot. In particular, let us consider the following condition

$$\frac{\partial L}{\partial y_j}=\mu -{\lambda }_j+{\mathit{\psi }}_j^T\frac{\partial }{\partial y_j}{\mathit{q}}_j\left(y_j\right)=0$$

(16)

that reveals

$$\mu ={\lambda }_j-{{\mathit{\psi }}_j}^T\frac{\partial }{\partial y_j}{\mathit{q}}_j\left(y_j\right)$$

(17)

This equality is not very useful in general and neither from an algorithmic point of view because the values of the multipliers in ψ _j are unknown. However, we can make use of the following complementary slackness conditions ([2], Sec. 5.5.2) of the problem, compactly written as ψ _j ⊙ q _j (y _j ) = 0,^d and observe that if $y_j\notin \mathbf{bd}{\mathcal{Y}}_j$ then ${\mathit{q}}_j\left(y_j\right)\prec 0$ and consequently ψ _j = 0. In that case, the link between μ and λ _j is clear,

$$\mu ={\lambda }_j\text{if}{y}_j\notin \mathbf{bd}{\mathcal{Y}}_j$$

(18)

Back to the algorithm, this result motivates the use of ${\widehat{\lambda }}_j^k$ only if it is derived from ${ŷ}_j^k\notin \mathbf{bd}{\mathcal{Y}}_j$ and so a new list $\left\{{\breve{\lambda }}_j^k\right\}$ that contains all these suitable dual values is constructed. Besides, it is necessary to guarantee that the new list $\left\{{\breve{\lambda }}_j^k\right\}$ is non-empty or, equivalently, that after the dual projection at least one value in ${\widehat{\mathit{y}}}^k$ satisfies ${ŷ}_j^k\notin \mathbf{bd}{\mathcal{Y}}_j$. This is the result of Proposition 3 next.

Proposition 3. Let ${\widehat{\mathit{y}}}^k={\mathit{y}}^k-\mathit{r}$ (${\widehat{\mathit{y}}}^k\ne {\mathit{y}}^{\ast }$ ) be a primal point resulting from the primal projection of the CDM with the value of r≽ 0 suitable to fulfill $\sum _{j=1}^J{ŷ}_j^k=C$, ${\widehat{\mathit{y}}}^k\in \mathcal{Y}$. Then, at least one value in $\left\{{ŷ}_j^k\right\}$ verifies ${ŷ}_j^k\notin \mathbf{bd}{\mathcal{Y}}_j$ and also ${ŷ}_j^k>y_j^{\ast }$.

Proof. See “Proof of Proposition 3” in Appendix. □

Finally, we need to prove that the last step of the method, i.e., the dual projection in (14), is able to find an update of the global price μ from the list $\left\{{\breve{\lambda }}_j^k\right\}$ such that μ ^k→ k → ∞ μ ^∗. Since we have assumed that primal and dual subproblems are reciprocal in the sense that they agree on the values of the dual variables ${\widehat{\lambda }}_j^k$ and ${\lambda }_j^k$ when $y_j^k={ŷ}_j^k$ (see Section 3.1, step 3), a consequence is that ${\breve{\lambda }}_j^k\left(y_j^{\ast }\right)$ computed in (13) equals ${\lambda }_j^{\ast }={\mu }^{\ast }$ as well as $y_j^k\left({\mu }^{\ast }\right)=y_j^{\ast }$ in (10). Note that we have intentionally written ${\breve{\lambda }}_j^k$ instead of ${\widehat{\lambda }}_j^k$ because our focus is only on the primal subproblems with ${ŷ}_j^k\notin \mathbf{bd}{\mathcal{Y}}_j$, which ensures λ _j = μ according to (18). Additionally, the following two claims can be made: (i) all the values in $\left\{{\breve{\lambda }}_j^k\right\}$ satisfy ${\breve{\lambda }}_j^k\ge {\mu }^k$ and (ii) at least one value in the list verifies ${\breve{\lambda }}_j^k\le {\mu }^{\ast }$. The first statement uses Proposition 1 and in particular that $\widehat{{\lambda }_j^k}$ (or $\breve{{\lambda }_j^k}$ equivalently) is a non-increasing function of $\widehat{y_j^k}$ in the primal subproblems. Recalling that $p_j\left(y_j^k\right)$ in (13) would produce ${\lambda }_j^k$ as inner Lagrange multiplier and that ${\lambda }_j^k={\mu }^k$ according to (11), it is true that ${\breve{\lambda }}_j^k\ge {\mu }^k$ since $\widehat{y_j^k}\le y_j^k$ (as a result of the primal projection). The second statement is verified in a similar manner taking into account that at least one value in $\left\{{ŷ}_j^k\right\}$ verifies ${ŷ}_j^k\notin \mathbf{bd}{\mathcal{Y}}_j$ and also ${ŷ}_j^k>y_j^{\ast }$ (see Proposition 3). Since ${\breve{\lambda }}_j^k\left(y_j^{\ast }\right)={\lambda }_j^{\ast }={\mu }^{\ast }$ in the j th primal subproblem, Proposition 1 establishes that ${\breve{\lambda }}_j^k\le {\mu }^{\ast }$.

Figure 2a explains the effects of the three steps of the CDM graphically from the dual domain point of view. Each bar represents an entity (J in total) and a point in that bar indicates the value of the dual variable ${\lambda }_j^k$ or ${\widehat{\lambda }}_j^k$. The highest the point the highest the value. At the beginning of the k th iteration, the dual subproblems enforce ${\lambda }_j^k={\mu }^k\forall j$ and translate these dual values to the primal variables in y ^k. Immediately after the primal projection, the corrected values in ${\widehat{\mathit{y}}}^k$ are converted again to dual variables, i.e., $\left\{{\widehat{\lambda }}_j^k\right\}$. In the figure, we appreciate the effect of the primal projection on the Lagrange multipliers of interest. In short, we notice that (i) all values increase and (ii) there is at least one value below μ ^∗.

Figure 2

Primal and dual projections lead the updates of μ towards μ ^∗.

The role of the dual projection in (14) is then to update to μ ^{k + 1} by selecting the closest value to μ ^k from the list $\left\{{\breve{\lambda }}_j^k\right\}$, that is, ${\mu }^{k+1}=\text{min}\left\{\underset{j}{\overset{k}{\breve{\lambda }}}\right\}$ if μ ^k< μ ^∗, as depicted in Figure 2b. Together with the previous results, i.e., ${\breve{\lambda }}_j^k\ge {\mu }^k$ and ${\breve{\lambda }}_j^k\le {\mu }^{\ast }$, the new update verifies μ ^{k + 1}∈ [μ ^k,μ ^∗] and thus our initial hypothesis (μ ^k< μ ^∗) is also satisfied for the next iteration unless μ ^{k + 1}= μ ^∗. Therefore, successive iterations confirm μ ^k→ k → ∞ μ ^∗ and, accordingly, ${\widehat{\mathit{y}}}^k\stackrel{k\to \infty }{\to }{\mathit{y}}_j^{\ast }\forall j$. This concludes the proof of the proposed method.

4 Convergence rate analysis and stopping criterion

This section provides additional insights into the proposed CDM by means of the following particularization of (1),

$$\begin{array}{cc}\underset{\left\{x_j\right\},\mathit{y}}{\text{min}}& \sum _{j=1}^J{f}_j\left(x_j\right)\\ \mathrm{s.t.}& x_j\in {\mathcal{X}}_j,& j=1,\dots ,J\\ h_j\left(x_j\right)\le y_j,& j=1,\dots ,J\\ \sum _{j=1}^J{y}_j\le C\\ \mathit{y}\in \mathcal{Y},& \mathcal{Y}={\mathcal{Y}}_1\times \dots \times {\mathcal{Y}}_J\end{array}$$

(19)

where the variables in {x _j } as well as the subsets in $\left\{{\mathcal{X}}_j\right\}$ are uni-dimensional. To be precise, not all the problems that can be formulated as in (19) are considered in the following convergence analysis but only those with the following dependance between the primal variable y _j and the dual variable λ _j in the subproblems of the CDM, still interesting as far as usual problems in the literature exhibit that relationship (see some examples in Section 5),

$$\begin{array}{ll}{y}_j=a_j{\left({\lambda }_j\right)}^{-\alpha }+b_j,& \text{for a certain value of}\alpha >0,\\ a_j>0\text{and}{b}_j\in \mathbb{R}\end{array}$$

(20)

In the general case, the relationship between y _j and λ _j can be established again, thanks to the KKT optimality conditions of the problem. Therefore, let us construct the Lagrangian of (19), that is,

$$\begin{array}{cc}L\left(\right\{x_j\},\mathit{y},\{{\lambda }_j\},\mu ,\{{\mathit{\xi }}_j\left\}\right)=& \sum _{j=1}^J{f}_j\left(x_j\right)+\mu (\sum _{j=1}^J{y}_j-C)\\ +\sum _{j=1}^J{\lambda }_j\left(h_j\left(x_j\right)-y_j\right)\\ +\sum _{j=1}^J{\mathit{\xi }}_j^T\left({\mathit{g}}_j\right(x_j\left)\right)+\sum _{j=1}^J{\mathit{\psi }}_j^T{\mathit{q}}_j\left(y_j\right)\end{array}$$

(21)

and consider the following optimality condition,

$$\frac{\partial L}{\partial x_j}={\dot{f}}_j\left(x_j\right)+{\lambda }_j{\dot{h}}_j\left(x_j\right)+{\mathit{\xi }}_j^T\frac{\partial }{\partial x_j}{\mathit{g}}_j\left(x_j\right)=0$$

(22)

where $\dot{f}$ and $\dot{h}$ stand for the first derivatives of the functions f and h, respectively. Note that if $x_j\notin \mathbf{bd}{\mathcal{X}}_j$ then ξ _j = 0 due to complementary slackness and

$${\lambda }_j=-\frac{{\dot{f}}_j\left(x_j\right)}{{\dot{h}}_j\left(x_j\right)}$$

(23)

Moreover, if the constraint h (x _j ) ≤ y _j is satisfied with equality (the usual case as we consider coupled problems) then $x_j=h_j^{-1}\left(y_j\right)$ and the relationship between λ _j and y _j is established.

Finally, as we show in Section 5, (20) is found for common functions f and h appearing in usual problems. Furthermore, the convergence rate of the proposed method can be derived assuming (20) and a stopping criterion that enhances the performance of the CDM can be designed. These two issues are developed in the following subsections.

4.1 Convergence rate analysis

In order to find out the convergence rate of the proposed method, let us compare the value of |(μ ^k)^{− α}−(μ ^∗)^{− α}| in two successive iterations, i.e., k and k + 1. First, let us classify the optimal primal variables $\left\{y_j^{\ast }\right\}$ into three groups: ${\mathcal{I}}^{\ast }$ includes the indexes j corresponding to the variables that satisfy $y_j^{\ast }=\text{inf}{\mathcal{Y}}_j$, ${\mathcal{S}}^{\ast }$ embraces the indexes where $y_j^{\ast }=\text{sup}{\mathcal{Y}}_j$ and finally, ${\mathcal{A}}^{\ast }$ contains the remaining indexes, i.e., those associated to $y_j\notin \mathbf{bd}{\mathcal{Y}}_j$. Using (20) and recalling the optimality condition ${\lambda }_j^{\ast }={\mu }^{\ast }$ seen in (11), it is true that

$$y_j^{\ast }=\left\{\begin{array}{cc}{a}_j{\left({\mu }^{\ast }\right)}^{-\alpha }+b_j& j\in {\mathcal{A}}^{\ast }\\ m_j& j\in {\mathcal{I}}^{\ast }\\ d_j& j\in {\mathcal{S}}^{\ast }\end{array}\right.$$

(24)

where $m_j=\text{inf}{\mathcal{Y}}_j$ and $d_j=\text{sup}{\mathcal{Y}}_j$. Assuming that $\sum _{j=1}^J{y}_j^{\ast }=C$ is fulfilled, we get

$${\left({\mu }^{\ast }\right)}^{-\alpha }=\frac{C-\sum _{j\in {\mathcal{A}}^{\ast }}{b}_j-\sum _{j\in {\mathcal{I}}^{\ast }}{m}_j-\sum _{j\in {\mathcal{S}}^{\ast }}{d}_j}{\sum _{j\in {\mathcal{A}}^{\ast }}{a}_j}$$

(25)

For any other value μ ^k≠ μ ^∗ we define

$$y_j^k=\left\{\begin{array}{cc}{a}_j{\left({\mu }^k\right)}^{-\alpha }+b_j& j\in {\mathcal{A}}^k\\ m_j& j\in {\mathcal{I}}^k\\ d_j& j\in {\mathcal{S}}^k\end{array}\right.$$

(26)

where the subsets ${\mathcal{A}}^k$, ${\mathcal{I}}^k$, and ${\mathcal{S}}^k$ are defined likewise ${\mathcal{A}}^{\ast }$, ${\mathcal{I}}^{\ast }$, and ${\mathcal{S}}^{\ast }$ but refer to the indexes of the variables in $\left\{y_j^k\right\}$.

Let us assume μ ^k< μ ^∗ and let us obtain $\left\{y_j^k\right\}$ from (26). Clearly, since (μ ^k)^−α> (μ ^∗)^−α, it holds that $y_j^k\ge y_j^{\ast }\forall j$. As a result of the primal projection in (12), now with the objective value modified by the weighting matrix W= [1/a ₁,…,1/a _J ]^T, i.e., ${\mathit{W}}^{1/2}\left|\right|{\mathit{y}}^k-{\widehat{\mathit{y}}}^k|{|}^2$, the corrected ${ŷ}_j^k$ values can be expressed as

$${ŷ}_j^k=\left\{\begin{array}{cc}{a}_j{\left({\mu }^k\right)}^{-\alpha }+b_j-a_{j}K& j\in {\mathcal{A}}^k\\ m_j& j\in {\mathcal{I}}^k\\ d_j-a_{j}K& j\in {\mathcal{S}}^k\end{array}\right.$$

(27)

for the value of K > 0 to be determined. The proof is very similar to the case W= I in “Proof of Proposition 1” in Appendix and the convergence of the method is not affected. We use this projection in this particularized version simply because it offers better performance and we did not use it before just because we had no means to find a better weighting matrix than the identity matrix.

At the third step of the method, i.e., the dual subproblems, the reduced list $\left\{{\breve{\lambda }}_j^k\right\}$ is obtained from the values ${ŷ}_j^k$ in (27) with $j\in {\mathcal{A}}^k\cup {\mathcal{S}}^k$. In other words, reversing (20) we find

$${\left({\breve{\lambda }}_j^k\right)}^{-\alpha }=\frac{{ŷ}_j^k-b_j}{a_j}=\left\{\begin{array}{cc}{\left({\mu }^k\right)}^{-\alpha }-K,& \forall j\in {\mathcal{A}}^k\\ \frac{d_j-b_j}{a_j}-K,& \forall j\in {\mathcal{S}}^k\end{array}\right.$$

(28)

Finally, in the dual projection we select the minimum value in $\left\{{\breve{\lambda }}_j^k\right\}$, which is in this case the closest to μ ^kgiven μ ^k< μ ^∗ because μ ^{k + 1}∈[μ ^k,μ ^∗] (see Section 3.3),

$${\mu }^{k+1}=\text{min}\left\{\underset{j}{\overset{k}{\breve{\lambda }}}\right\}$$

(29)

or equivalently,

$${\left({\mu }^{k+1}\right)}^{-\alpha }=\text{max}\left\{\underset{j}{\overset{k}{\breve{\lambda }}}\right\}={\left({\mu }^k\right)}^{-\alpha }-K$$

(30)

since $\frac{d_j-b_j}{a_j}$ is always lower than (μ ^k)^{− α} or otherwise $\frac{d_j-b_j}{a_j}-K$ would belong to ${\mathcal{A}}^k$. Note in (27) that the definition of the subsets ${\mathcal{A}}^k$ and ${\mathcal{S}}^k$ implies d _j < a _j (μ ^k)^−α+ b _j .

Using the previous results, we can state that

$$|{\left({\mu }^{k+1}\right)}^{-\alpha }-{\left({\mu }^{\ast }\right)}^{-\alpha }|=|{\left({\mu }^k\right)}^{-\alpha }-{\left({\mu }^{\ast }\right)}^{-\alpha }-K|$$

(31)

This can be further refined if K is developed using (27) and $\sum _{j=1}^J{ŷ}_j^k=C$,

$$\begin{array}{ll}K& =\frac{{\left({\mu }^k\right)}^{-\alpha }\sum _{j\in {\mathcal{A}}^k}{a}_j+\sum _{j\in {\mathcal{A}}^k}{b}_j+\sum _{j\in {\mathcal{I}}^k}{m}_j+\sum _{j\in {\mathcal{S}}^k}{d}_j-C}{\sum _{j\in {\mathcal{A}}^k\cup {\mathcal{S}}^k}{a}_j}\\ =\frac{\sum _{j\in {\mathcal{A}}^k}{a}_j}{\sum _{j\in {\mathcal{A}}^k\cup {\mathcal{S}}^k}{a}_j}{\left({\mu }^k\right)}^{-\alpha }-\frac{\sum _{j\in {\mathcal{A}}^k}{a}_j}{\sum _{j\in {\mathcal{A}}^k\cup {\mathcal{S}}^k}{a}_j}\\ \times \left[\frac{C-\sum _{j\in {\mathcal{A}}^k}{b}_j-\sum _{j\in {\mathcal{I}}^k}{m}_j-\sum _{j\in {\mathcal{S}}^k}{d}_j}{\sum _{j\in {\mathcal{A}}^k}{a}_j}\right]\end{array}$$

(32)

Particularly, note that the expression within brackets in (32) is exactly (μ ^∗)^−α when the subsets ${\mathcal{A}}^k$, ${\mathcal{I}}^k$ and ${\mathcal{S}}^k$ coincide with the optimal ones. We say that the algorithm is in the optimal zone when the sets $({\mathcal{A}}^k$, ${\mathcal{I}}^k$,${\mathcal{S}}^k)$ coincide with $({\mathcal{A}}^{\ast }$, ${\mathcal{I}}^{\ast }$,${\mathcal{S}}^{\ast })$.

Finally, we can conclude that the speed of convergence within the optimal zone obeys the following rule, which is obtained by plugging (32) into (31),

$$\begin{array}{ll}|{\left({\mu }^{k+1}\right)}^{-\alpha }-{\left({\mu }^{\ast }\right)}^{-\alpha }|=& |{\left({\mu }^k\right)}^{-\alpha }-{\left({\mu }^{\ast }\right)}^{-\alpha }|\\ \times \left(1-\frac{\sum _{j\in {\mathcal{A}}^{\ast }}{a}_j}{\sum _{j\in {\mathcal{A}}^{\ast }\cup {\mathcal{S}}^{\ast }}{a}_j}\right)\end{array}$$

(33)

In other words, (μ ^k)^−α converges linearly to (μ ^∗)^−α except when ${\mathcal{S}}^{\ast }=\{\varnothing \}$, showing superlinear convergence. Alternatively, if the initial hypothesis is μ ⁰> μ ^∗, the convergence is also linear expect for ${\mathcal{I}}^{\ast }=\{\varnothing \}$, in which case it is superlinear. Note in both cases that since (1) and (19) are assumed coupled problems, ${\mathcal{A}}^{\ast }\ne \{\varnothing \}$.

4.2 Stopping criterion

The previous convergence rule in (33) can be used to define a stopping criterion for the CDM. It is based on the particular evolution followed by μ ^k inside the optimal zone. For that purpose, let us take three consecutive values of μ, i.e., μ ^k, μ ^{k + 1}, and μ ^{k + 2}, all of them in the optimal zone. The successive application of (33) leads to

$$\begin{array}{ll}\hfill |{\left({\mu }^{k+l}\right)}^{-\alpha }-{\left({\mu }^{\ast }\right)}^{-\alpha }|=& |{\left({\mu }^k\right)}^{-\alpha }-{\left({\mu }^{\ast }\right)}^{-\alpha }|\\ \times {\left(1-\frac{\sum _{j\in {\mathcal{A}}^{\ast }}{a}_j}{\sum _{j\in {\mathcal{A}}^{\ast }\cup {\mathcal{S}}^{\ast }}{a}_j}\right)}^l\\ \hfill l=& \{0,1,2\}\end{array}$$

(34)

From (34) it is verified that

$$\frac{{\left({\mu }^{k+2}\right)}^{-\alpha }-{\left({\mu }^{k+1}\right)}^{-\alpha }}{{\left({\mu }^{k+1}\right)}^{-\alpha }-{\left({\mu }^k\right)}^{-\alpha }}=1-\frac{\sum _{j\in {\mathcal{A}}^{\ast }}{a}_j}{\sum _{j\in {\mathcal{A}}^{\ast }\cup {\mathcal{S}}^{\ast }}{a}_j}$$

(35)

and therefore, in the optimal zone, the left side of (35) is a constant number regardless of k. From the practical point of view and thanks to this result, we can monitor the evolution of

$$S{C}^k=\frac{{\left({\mu }^{k+2}\right)}^{-\alpha }-{\left({\mu }^{k+1}\right)}^{-\alpha }}{{\left({\mu }^{k+1}\right)}^{-\alpha }-{\left({\mu }^k\right)}^{-\alpha }},\forall k$$

(36)

and stop the iterations when S C ^k stabilizes to a constant value. Afterwards, the optimal solution is readily obtained since at that point we know which allocations saturate to either m _j or d _j and the exact value of μ ^∗ can be computed by means of (25).

4.3 Graphical comparison among decomposition techniques

In the sequel, we include a graphical comparison among decomposition techniques and the goal is to highlight the manner in which the different methods operate in essence. We do this with the support of the following toy optimization problem

$$\begin{array}{cc}\underset{x_1,x_2,y_1,y_2}{\text{min}}& a_1{(x_1-c_1)}^2+a_2{(x_2-c_2)}^2\\ x_i\le y_i,& i=1,2\\ \mathrm{s.t.}& y_1+y_2\le C\\ 0\le x_i\le x_i^{\mathit{\text{max}}},& i=1,2\\ 0\le y_i\le x_i^{\mathit{\text{max}}},& i=1,2\end{array}$$

(37)

where we have included the variables y _i to match the formulation of the proposed CDM and a primal decomposition as well. In Figure 3, we compare our proposed method to the classical decomposition techniques. In all the cases, the feasibility region of the problem in terms of the variables y ₁,y ₂ is marked in grey. Also, the contour lines of the objective function (centered at c= [c ₁,c ₂]^T) are represented in the plots (even we know that the dependance of the objective function is on x ₁,x ₂ instead of y ₁,y ₂).

Figure 3

Comparison among decomposition techniques. Toy example.

As depicted in the figure, a primal decomposition approach updates y ^kby adding the subgradient to the point and, if the result is not feasible, a projection corrects the situation by finding the closest point in the feasible set. In the figure, note that arrows represent subgradients and dashed lines projections. In this way, the successive projections tend to the optimal solution, i.e., y ^∗. Next, let us consider a dual decomposition approach. In order to analyze it from the perspective of the primal variables, we need to establish first the relationship between y _j and λ _j and also between λ _j and μ. For that purpose, we consider again the Lagrangian of the problem, that is

$$\begin{array}{ll}L(\mathit{x},\mathit{y},& \mathit{\lambda },\mu ,{\xi }_1,{\xi }_2,{\mathit{\psi }}_1,{\mathit{\psi }}_2)\\ \hfill =& a_1{(x_1-c_1)}^2+a_2{(x_2-c_2)}^2+{\mathit{\lambda }}^T(\mathit{x}-\mathit{y})\\ +\mu (y_1+y_2-C)\\ -\sum _{i=1}^2{\xi }_{1,i}{x}_i+\sum _{i=1}^2{\xi }_{1,i}(x_i-x_i^{\mathit{\text{max}}})\\ -\sum _{i=1}^2{\psi }_{1,i}{y}_i+\sum _{i=1}^2{\psi }_{1,i}(y_i-y_i^{\mathit{\text{max}}})\end{array}$$

(38)

For the case $x_i\in (0,x_i^{\mathit{\text{max}}}),i=1,2$ and y _i = x _i , the dual variables in ψ _i and ξ _i satisfy ψ _i = ξ _i = 0,i = 1,2 due to slackness (note that $y_i^{\mathit{\text{max}}}=x_i^{\text{max}},i=1,2$). In that conditions, the KKT optimality condition ∂ L/∂ x _i = 0 forces

$$x_i=c_i-\frac{{\lambda }_i}{a_i}$$

(39)

Furthermore, if we take into account that y _i = x _i in the case of interest and λ _i = μ due to the KKT optimality condition ∂ L/∂ y _i = 0, then we verify that

$$y_i(\mu )=c_i-\frac{\mu }{a_i}$$

(40)

The dashed line in Figure 3 shows all the points that can be obtained by changing the value of μ in (40). Note in particular that for μ = 0 the optimum of the unconstrained version of (38) is achieved. Using the dual decomposition technique (in the figure we start with μ ⁰ = 0), the successive updates move along this line (the orientation and module defined by the subgradient) until the optimal solution is achieved.

In the two classical decomposition approaches, primal and dual decomposition, a good election of the step-size that modifies the length of the subgradient plays a central role and it is recommended to choose a value that diminishes with the iteration number in order to prevent the successive updates from moving indefinitely around the optimal solution without reaching it. This issue is in fact an important practical impairment of both solutions. Notwithstanding, the method we propose in this article avoids the usage of a user-defined step-size. As the reader can appreciate in the figure, once the initial guess y ⁰ derived from μ ⁰ = 0 is projected to the feasible subset, the proposed method finds out several candidates to update μ ⁰ to μ ¹ (two in this case, i.e., ${\breve{\lambda }}_1^1$ and ${\breve{\lambda }}_2^1$). These two candidates provide the possible updates ${\mathit{y}}^1\left({\breve{\lambda }}_1^1\right)$ and ${\mathit{y}}^1\left({\breve{\lambda }}_2^1\right)$ and the method always chooses the dual candidate that provides the smallest possible update. This operation guarantees that the primal update, i.e., y ¹ in this case, remains in the same half-space (with respect to the frontier y ₁ + y ₂ = C). Note with this simple example the interesting feature of the proposed method in comparison with the other techniques, that is, the step is automatically controlled.

5 Applications and numerical results

In the sequel, we present three different applications of the proposed method, the first two are related to power allocation problems and the third one deals with DBA in satellite networks. In the first problem, a decentralized solution is required to reduce the amount of signaling information. In the second one, a centralized implementation of the CDM is used to solve a time-varying water-filling problem and the aim is to show the benefits of having an unsupervised method in that changing conditions. Finally, the third example shows the advantages of the proposed technique when a small allocation time is required in order to accommodate a large number of users.

5.1 Decentralized power allocation for cognitive radios

Let us consider a communication device that is able to establish simultaneous communication links by joining several networks or using multiple channels within the same system (e.g., this is possible in IEEE 802.11n). To do so, the device integrates multiple radio transceivers [23, 24] which, at their turn, operate over multiple subchannels or subcarriers in order to combat the multipath fading (see Figure 4). We assume that the device can sense the wireless channel and determine the non-used subcarriers in each subsystem, as it is usual in cognitive scenarios. Furthermore, each transceiver is able to optimally allocate the available power among its subcarriers using the water-filling solution. This is advantageous from the system design point of view because we can employ off-the-shelf radio transceivers and simply balance the device power among them. Finally, there is a central controller that performs the distribution task, being the global objective to maximize the total sum rate capacity. Note that, depending on the signal strength and capacity in each subsystem, some of the transceivers may remain temporarily idle.

Figure 4

Example of a cognitive device.

The problem can be formulated for M radio transceivers as

$$\begin{array}{cc}\underset{\left\{P_i\right\}}{\text{max}}& \sum _{i=1}^M{r}_i\left(P_i\right)\\ \mathrm{s.t.}& \sum _{i=1}^M{P}_i\le P_T\\ P_i\ge 0,& i=1,\dots ,M\end{array}$$

(41)

where r _i (P _i ) is the transmission rate of the i th transceiver when power P _i is allocated to it and P _T is the total available power. Each of the transmission rates is actually the result of another optimization problem, that is,

$$r_i\left(P_i\right)=\left\{\begin{array}{cc}\underset{\left\{p_j^i\right\}}{\text{max}}& \sum _{j=1}^{N_s^i}B{W}_i\text{log}\left(1+\frac{\underset{j}{\overset{i}{p}}{H}_j^i}{N_{0}B{W}_i}\right)\\ \mathrm{s.t.}& p_j^i\ge 0\\ \sum _{j=1}^{N_s^i}{p}_j^i\le P_i\end{array}\right.$$

(42)

where $N_s^i$ is the number of subcarriers of the i th radio, B W _i stands for subcarrier bandwidth, N ₀ is the noise power spectral density, and $H_j^i$ is the channel gain at the j th subcarrier of the i th transceiver.

Note that given the separability of the problem, i.e., there are many independent transceivers coupled by a total power constraint, a decentralized optimization method is adequate both from a mathematical and a practical point of view. In this approach, the controller decides the total power per transceiver and each subsystem computes its own optimal allocation. The application of the CDM to solve (41) is briefly detailed next.

Given μ ^k< μ ^∗, each transceiver computes the following dual subproblem,

$$\begin{array}{cc}\underset{\left\{p_{i,j}\right\},P_i^k}{\text{max}}& \sum _{j=1}^{N_s^i}B{W}_i\text{log}\left(1+\frac{p_{i,j}{H}_{i,j}}{N_{0}B{W}_i}\right)-{\mu }^k{P}_i^k\\ \mathrm{s.t.}& \sum _{j=1}^{N_s^i}{p}_{i,j}\le P_i^k\\ p_{i,j}\ge 0\end{array}$$

(43)

and the application of the KKT conditions gives the solution

$$\begin{array}{c}{p}_{i,j}=B{W}_i{\left(\frac{1}{{\mu }^k}-\frac{N_0}{H_{i,j}}\right)}^{+}\\ P_i^k=\sum _{j=1}^{N_s^i}{p}_{i,j}\end{array}$$

(44)

As a result of the dual subproblems we obtain ${\mathit{P}}^k={[P_1^k,\dots ,P_M^k]}^T$ and we use it as the input for the primal projection, that is,

$$\begin{array}{cc}\underset{\left\{{\widehat{P}}_i^k\right\}}{\text{min}}& \left|\right|{\widehat{\mathit{P}}}^k-{\mathit{P}}^k|{|}^2\\ \mathrm{s.t.}& \sum _{i=1}^M{\widehat{P}}_i^k=P_T\\ {\widehat{P}}_i^k\ge 0\end{array}$$

(45)

where ${\widehat{\mathit{P}}}^k={[{\widehat{P}}_1^k,\dots ,{\widehat{P}}_M^k]}^T$.

The corrected values ${\widehat{P}}_i^k$ are then used in the primal subproblems defined in (42) and each transceiver computes the optimal allocation by its own. As a result of the primal subproblems, the Lagrange multipliers associated to the constraints $\sum _{j=1}^{N_s^i}{p}_{i,j}\le {\widehat{P}}_i^k$ at the k th iteration, i.e., ${\widehat{\lambda }}_i^k$, are obtained. Discarding the values that result from ${\widehat{P}}_i^k=0$, we obtain the reduced list $\left\{{\breve{\lambda }}_i^k\right\}$ and finally, the dual projection updates μ ^k from $\left\{{\breve{\lambda }}_i^k\right\}$ by doing

$${\mu }^{k+1}=\text{min}\left\{\underset{i}{\overset{k}{\breve{\lambda }}}\right\}$$

(46)

A completely different approach is to gather all the information at the controller and to compute there the optimal power allocation. Afterwards, the result is sent back to the transceivers. Note that this centralized solution has an important drawback in terms of signaling because the powers in all the subcarriers and all the transceivers need to be exchanged. On the contrary, decentralized solutions benefit from transceiver-level signaling. In the numerical results below, we compare the CDM to other approaches.

5.1.1 Numerical results

We consider a device with three different OFDM transmitters. The first transmitter employs 256 subcarriers spanning a total bandwidth of 1.536 MHz (6 kHz per subchannel), the second one has 256 subcarriers as well and 3.072 MHz of bandwidth (12 kHz per subchannel) and the third one manages 128 subcarriers in 1.28 MHz of bandwidth (10 kHz per subchannel), so that a total of 640 subcarriers and 5.888 MHz have to be controlled (see Table 1). We assume frequency selective Rayleigh-fading channels in all three systems with a channel length of 20 taps and an exponential power delay profile where the delay spread is 1 ms. Mean channel gain is 0 dB in system 1, −10 dB in system 2, and −5 dB in system 3. Moreover, we assume that the noise power spectral density is flat over frequency with $N_0={\sigma }_n^2/B{W}_1$, being B W ₁ the subcarrier bandwidth in system 1. Initially, we set up a uniform power allocation in all the methods and the total available power is always $P_T\left(\text{dB}\right)={\sigma }_n^2\left(\text{dB}\right)+10\underset{10}{\text{log}}\left(640\right)+5$.

Table 1 Description of the subsystems

Figure 5 shows a multi-system water-filling allocation example. The plot at the top depicts one channel realization for the three systems whereas the plot at the bottom shows the optimal power allocation. As expected, most of the power is allocated to transceivers 1 and 3, which are the ones that have the best channel condition. On the contrary, subsystem 2 only allocates power to a few subcarriers that have the highest channel gains. Notwithstanding, in absolute terms, transceiver 2 receives quite a large allocation in order to exploit the higher subcarrier bandwidth.

Figure 5

Distributed water-filling example. Top: channel gains. Bottom: power allocation. In f 1 – f 3 we have the initial band frequencies of subsystems 1 – 3.

Figure 6 shows the evolution of the Normalized Mean Squared Error (NMSE) in the power allocation with respect to the number of messages exchanged between the transmission subsystems and the central controller. The optimal power allocation is computed using the bisection method (relative error below 10⁻⁵). We compare the proposed CDM to the classical primal and dual decomposition techniques, the classical primal–dual algorithm of Arrow et al. [16] and also to a centralized approach. The classical decomposition techniques use ${\alpha }^k=1/\sqrt{k}$ as step-size and the Arrow–Hurwicz method initializes the value in the dual variable μ to 0, the primal variables with a uniform power allocation and the step-size is fixed to 0.1. On the one hand, results show that the proposed CDM is the best option, whereas the remaining alternatives require at least to double the amount of signaling in order to achieve the same allocation error. On the other hand, note that the classical decomposition techniques as well as the primal–dual approach are penalized in terms of convergence speed even taking into account that we have manually adjusted the step-size of each method in order to achieve the best possible result. Finally, note also that a centralized approach is not efficient at all as far as the allocation error becomes small enough only when the entire allocation has been transmitted. This requires 640 messages in our case to send the channel gains to the controller and 640 messages more to return the optimal power allocation values to the radios.

Figure 6

NMSE versus number of signaling messages.

5.2 Power allocation in a conventional OFDM transmission

In the following, we apply our method to a classical water-filling problem where a decentralized solution is not necessary. In this occasion, we are interested in the adaptability of the method in time-varying scenarios.

Let us consider the well-known single-user water-filling solution over parallel Gaussian channels ([25], Sec. 10.4), which provides the optimal power allocation to the subcarriers of an OFDM-based system in order to maximize the mutual information given a total power constraint. Mathematically,

$$\begin{array}{ll}\underset{\left\{p_i\right\}}{\text{max}}& \sum _{i=1}^{N_s}\text{log}\left(1+\frac{p_i}{\underset{n_i}{\overset{2}{\sigma }}}\right)\\ \mathrm{s.t.}& p_i\ge 0\\ \sum _{i=1}^{N_s}{p}_i\le P\end{array}$$

(47)

where N _s is the total number of subcarriers or parallel channels in the system, P is the total transmission power, ${\sigma }_{n_i}^2$ is the noise variance in the i th subcarrier and p _i stands for the allocated power. The application of the KKT optimality conditions to (47) leads to the solution

$$p_i={\left(\frac{1}{\mu }-{\sigma }_{n_i}^2\right)}^{+}$$

(48)

where (a)⁺ = max {0,a} and $\frac{1}{\mu }$ is denoted as the water-level and shall be chosen in order to satisfy the total power constraint. Typically, the bisection method is employed to find μ ^∗. However, note that (47) can be rewritten in the form of (2) and also (19). Therefore, we can apply the proposed CDM as well. Indeed, (48) and the relationship in (20) match if we identify p _i with y _i and μ with λ _i (remember that the required relationship applies only to $y_i\notin \mathbf{bd}{\mathcal{Y}}_i$, that is, y _i = p _i > 0).

5.2.1 Numerical results

Let us assume N _s = 512 subcarriers. The channel is time-varying and frequency selective; it has 20 taps. The power delay profile is assumed exponential with a delay spread of 1 ms and the baseband sampling time is 1 μ s. We compare now the proposed CDM to the bisection method and also to the classical primal–dual algorithm in [16]. It is remarkable that the CDM requires no modification at all (it is completely unsupervised) and the same holds for the primal–dual algorithm. On the contrary, the bisection method requires a slight modification to be able to track the time-varying scenario. For that purpose, we introduce the updating factor α _u . Initially, the method is applied as usual, that is, having the initial hypothesis on ${\mu }_l^0$ and ${\mu }_u^0$ (two values that are below and above μ ^∗, respectively), we compute ${\mu }^1=1/2({\mu }_l^0+{\mu }_u^0)$ and we update ${\mu }_l^1$ to μ ¹ if $\sum _{i=1}^{N_s}{p}_i\left({\mu }^1\right)>P$ or ${\mu }_h^1$ to μ ¹ otherwise. In the subsequent iterations, given that the channel is time-varying, we need to check first if ${\mu }_l^k$ and ${\mu }_h^k$ are still valid. If $\sum _{i=1}^{N_s}{p}_i\left({\mu }_l^k\right)>P$ is not accomplished, we update ${\mu }_l^k$ to $\frac{{\mu }_l^k}{{\alpha }_u}$ and we repeat this while $\sum _{i=1}^{N_s}{p}_i\left({\mu }_l^k\right)>P$. Similarly, if $\sum _{i=1}^{N_s}{p}_i\left({\mu }_u^k\right)<P$ is not attained, we modify ${\mu }_u^k$ to ${\alpha }_u·{\mu }_u^k$ and we repeat this while $\sum _{i=1}^{N_s}{p}_i\left({\mu }_u^k\right)<P$. Then, we compute ${\mu }^{k+1}=1/2({\mu }_l^k+{\mu }_u^k)$ and we update the hypothesis accordingly, as in the normal version of the technique.

Figure 7 plots the NMSE of the power allocation for both methods as a function of the mean SNR. As in the previous application example, we compute the optimal power allocation using the bisection method (relative error below 10⁻⁵). Moreover, all the algorithms are initialized to the optimal solution for the current channel condition, α _u = 1.05 in the bisection technique, the step-size is fixed to 0.001 in the Arrow–Hurwicz method and we have considered three different channel velocities, namely, (i) $T_c^1=10·T_{\text{CDM}}$, (ii) $T_c^2=100·T_{\text{CDM}}$, and (iii) $T_c^3=1000·T_{\text{CDM}}$, where T _CDM is the time taken by one complete iteration of the CDM and $T_c^i$ is the coherence time of the channel at the i th scenario. Note that we have manually adjusted α _u in the bisection method and the step-size in the primal–dual algorithm in order to achieve the best possible performance at the worst channel condition, that is, when the channel coherence time is the smallest one, i.e., $T_c^1$.

Figure 7

Normalized MSE of the power allocation versus SNR in a time-varying channel.

Results show that the CDM usually outperforms the bisection method and it is far better than the primal–dual algorithm. Indeed, it performs worse than the bisection only for $T_c^1$ and at low SNR. Note that since the CDM has no user-defined parameter, it automatically adapts to the different channel velocities. On the contrary, this adaptation does not occur in the other two methods. This is reflected in Figure 7, where, for example, the bisection method saturates to an NMSE around 10⁻⁴ for $T_c^1$, $T_c^2$, and $T_c^3$ as the SNR grows.

5.3 Fair DBA

The fair DBA problem arises in many-to-one communication systems [26, 27] and the goal is to fairly distribute the available bandwidth. In many cases and specially in systems with a huge number of users [28], the computational cost of the techniques plays an important role. Additionally, let us remark that recent works on the topic aim at providing mechanisms for QoS differentiation [4, 29] to modify a plain fair allocation. Therefore, we consider the following network utility maximization (NUM) formulation to solve a fair DBA problem,

$$\begin{array}{ll}\underset{\left\{r_j\right\}}{\text{max}}& \sum _{j=1}^N{U}_j(r_j;p_j)\\ \mathrm{s.t.}& m_j\le r_j\le d_j,& j=1,\dots ,N\\ \sum _{j=1}^N{r}_j\le B\end{array},$$

(49)

where B is the available bandwidth, r _j is the rate allocated to the j th flow, and U _j is the j th utility function (the terms bandwidth and rate are used interchangeably). The parameters m _j , d _j (with 0 ≤ m _j < d _j ), and p _j > 0 are used to define the QoS requirements for each ongoing connection and they represent the minimum necessary rate, the required (maximum) bandwidth and the priority of the j th flow, respectively. Furthermore, we assume that $\sum _j{m}_j<B<\sum _j{d}_j$, i.e., the problem is coupled. As argued before, the utility functions can adequately be chosen in order to achieve a fair distribution of resources in different degrees. The following family of functions parameterized by γ

$$U_j(r_j;p_j,\gamma )=\left\{\begin{array}{ll}{p}_j\text{log}\left(r_j\right),& \gamma =1\\ p_j\frac{r_j^{(1-\gamma )}}{1-\gamma },& \gamma \ne 1\end{array}\right.$$

(50)

define different types of fairness, being γ → ∞ (max–min fairness) and γ = 1 (proportional fairness) the most relevant ones [29].

Note that (49) can be rewritten in the form of (19) and in particular, the problem is strictly convex and we assume that strong duality holds, i.e., there is at least one strictly feasible point. Therefore, we can apply the KKT optimality conditions to solve (49) semi-analytically. In this case, the optimal rates must verify^e

$$r_j^{\ast }(\mu )={\left[{\left(\frac{p_j}{\mu }\right)}^{\frac{1}{\gamma }}\right]}_{m_j}^{d_j}$$

(51)

and the optimal value of μ is such that $\sum _{j=1}^N{r}_j^{\ast }\left({\mu }^{\ast }\right)=B$. The bisection method is a classical technique widely used in the literature in order to approximate μ ^∗ but, alternatively, we can also apply the enhanced version of the CDM. Specifically, by adding the new variables {y _j } and identifying f _j (r _j ) with − U _j (r _j ) and h _j (r _j ) = r _j , (23) together with $r_j=h_j^{-1}\left(y_j\right)$, (51) turns into

$$y_j={\left(\frac{p_j}{{\lambda }_j}\right)}^{\frac{1}{\gamma }}$$

(52)

when m _j < y _j < d _j and has the required form in (20). Therefore, once the subsets ${\mathcal{S}}^{\ast }$, ${\mathcal{I}}^{\ast }$, and ${\mathcal{A}}^{\ast }$ are known, the optimal value of μ is readily found according to (25) as

$${\mu }^{\ast }={\left[\frac{\sum _{i\in {\mathcal{A}}^{\ast }}\sqrt[\gamma ]{p_i}}{B-\sum _{i\in {\mathcal{I}}^{\ast }}{m}_i-\sum _{i\in {\mathcal{I}}^{\ast }}{d}_i}\right]}^{\gamma }.$$

(53)

5.3.1 Numerical results

Let us draw the values of m _j from an integer uniform distribution between 0 and 10. Each request d _j is obtained summing m _j and an integer random number between 0 and 100. The priority values p _j are drawn from a uniform distribution that takes values between 0.25 and 5 in steps of 0.25 and γ = 1. Figure 8 plots the mean allocation time, i.e., execution time, of the CDM when centrally computed in combination with the stopping criterion defined in Section 4.2. The algorithm has been executed in a Intel ^Ⓒ Core 2 Duo CPU running at 2.2 GHz and programmed in Matlab ^Ⓒ. We have considered three different values for the total available bandwidth, namely $B1=\sum _j{m}_j+0.25\sum _j{d}_j$, $B2=\sum _j{m}_j+0.5\sum _j{d}_j$, and $B3=\sum _j{m}_j+0.75\sum _j{d}_j$. The results of the CDM have been compared to the classical bisection method and to the hypothesis testing method [30]. Since the allocation time is not sensitive to the available capacity for the latter methods, in Figure 8 we distinguish among B 1, B 2, and B 3 only for the CDM.

Figure 8

Allocation time.

In order to provide a fair comparison among the methods, it is necessary to take into account the accuracy with respect to the optimal solution. The hypothesis testing strategy always achieves the exact optimal solution (see the details in [30]). The bisection method has been adjusted to achieve a relative error in the allocation lower than 10⁻⁶, or in other words,

$$\frac{\left|\right|{\mathit{r}}^{\mathit{\text{BI}}}-{\mathit{r}}^{\ast }\left|\right|}{\left|\right|{\mathit{r}}^{\ast }\left|\right|}\le 10^{-6},$$

(54)

where r ^∗ is the optimal allocation (which can be obtained with the hypothesis testing method) and r ^BIis the allocation achieved by the bisection method. Initially, the two hypothesis for the values of μ are 0 and 10. In the CDM we stop the iterations when

$$\frac{|S{C}^{k+1}-S{C}^k|}{\left|S{C}^{k+1}\right|}\le 10^{-2}.$$

(55)

Note that as the number of users grows, the difference in time between the proposed method and the others also grows, specially when the system is more restricted in terms of capacity, i.e., for B 1. In this case, the CDM is able to compute the allocation in half the time required by classical methods. In terms of accuracy in the solution, (55) gives the exact optimal solution for B 1, B 2 and a relative allocation error lower than 10⁻⁴ for B 3. Overall, the solution is good in practice; it is optimal in capacity-constrained scenarios and nearly optimal in less critical situations. In order to illustrate the selection of the threshold for the stopping criterion, we plot in Figure 9 the evolution of the relative error and allocation time as a function of the accuracy in the stopping criterion. Note that a threshold of 10⁻² provides a good trade-off between both performance metrics for the worst scenario, i.e., for B 3. Finally, if we consider a higher available bandwidth, e.g., 99% of the whole system demand, this threshold value keeps the allocation time small as in B 3 at the expenses of a higher allocation error (around 5%). However, the accuracy degradation appears only in this extreme case and it is not critical in practice as far as all the users nearly reach their demands.

Figure 9

Precision adjustment of the stopping criterion.

6 Conclusions and future work

This article has contributed with novel decomposition ideas that efficiently intertwine the classical primal and dual decomposition approaches in a single iteration of a new technique, called the CDM. It solves generic convex optimization problems that have one coupling constraint with the known advantages of decomposition-based approaches, that is, the implementation of decentralized solutions. However, it reduces the number of iterations by more than one order of magnitude with respect to the classical primal and dual decomposition solutions and furthermore, it is completely unsupervised, that is, there is no parameter that requires a manual adjustment. Moreover, when the problem is particularized (but still of interest), additional results regarding the convergence rate of the proposed technique are achieved and an stopping criterion that enhances the performance of the method (in terms of the number of iterations required to achieve the optimal solution) is derived.

The proposed method has been tested in three different problems, two dealing with power allocation in OFDM-based systems and a third one dealing with DBA. In the first two cases, the goal is to find the well-known water-filling solution in power. In one case, we benefit from a decentralized approach that suits the system architecture whereas in the other case, the proposed method is applied to a conventional OFDM transmission that deals with a time-varying channel. In both examples, we have compared our solution to other decomposition strategies and our approach performs significantly better than the available alternatives when a decentralized solution is required. In particular, our results show that the signaling requirements can be reduced at least by a factor 2. Moreover, in the centralized application of the method, that is, the conventional OFDM transmission, the proposed method benefits from being unsupervised and the channel variability does not compromise the performance as classical methods do. In particular, and at the high SNR regime, the difference in the NMSE of the power allocation between the proposed method and the bisection is more than two orders of magnitude in all the explored scenarios.

Finally, when applied to a NUM problem and thanks to the enhanced version of the method, the proposed technique reduces the computational time by a factor 2 with respect to well-established techniques such as the bisection method. This reduction is very important in systems having a large number of users (as it happens in satellite communication networks), where the bandwidth allocation has to be computed in a short-time interval.

Appendix

Proof of Proposition 1

For the sake of simplicity, in what follows we obviate the iteration index k as well as the modifier $\widehat{(·)}$ in ${ŷ}_j^k$, ${\widehat{\lambda }}_j^k$, and μ ^k. Let us consider first the dependance of λ _j on y _j in the primal subproblems. Interestingly, the function p _j in (13) has already been studied in the convex literature and it is known as the primal function ([15], Sec. 5.4.4). We recall here two main results related to the primal function: (i) p _j defines a convex function over the set ${\mathcal{P}}_j$ defined as ${\mathcal{P}}_j=\left\{y_j\right|p_j\left(y_j\right)<\infty \}$ and (ii) the optimal value of the dual variable associated to the constraint h _j (x _j ) ≤ y _j and with opposite sign, say $-{\lambda }_j^{\ast }\left(y_j\right)$, is a subgradient of p _j at y _j . In our case, note that ${\mathcal{Y}}_j\subseteq {\mathcal{P}}_j\forall j$ and thus, these results can be applied to (13). Furthermore, since p _j is convex in the region of interest, it is guaranteed to be continuous although not necessarily differentiable. However, given that the objective function as well as all the constraints in the problem (finite number of them) are differentiable, then for each singular value in ${\mathcal{Y}}_j$ we can find an open interval that includes it where p _j is differentiable (except of course at the singular point). Then, taking into account that the first derivative of a convex function is non-decreasing by definition and noting that the subgradient equals to the gradient where the function is differentiable, we obtain the desired result. In other words, $-{\lambda }_j^{\ast }\left(y_j\right)$ is non-decreasing in the intervals where p _j is differentiable just because the subgradient and the first derivative coincide whereas it takes a value in-between the right and left derivatives at the singular points, thus preserving the non-decreasing property. Finally, by removing the minus sign we can state that ${\lambda }_j^{\ast }\left(y_j\right)$ is non-increasing on y _j .

Second, we prove that y _j (μ) is non-increasing on μ in (10). Let us first rewrite (10) as

$$d_j(\mu )=\underset{y_j}{\text{min}}\left\{p_j\left(y_j\right)+\mu y_j\right\}$$

(56)

Then take any two values of μ, say μ ₁ and μ ₂, that accomplish: (i) 0 ≤ μ ₁ < μ ₂ and (ii) there exist two values in ${\mathcal{Y}}_j$, $y_{j,1}^{\ast }$ and $y_{j,2}^{\ast }$, such that $d_j\left({\mu }_1\right)=p_j\left(y_{j,1}^{\ast }\right)+{\mu }_1{y}_{j,1}^{\ast }$ and $d_j\left({\mu }_2\right)=p_j\left(y_{j,2}^{\ast }\right)+{\mu }_2{y}_{j,2}^{\ast }$. In other words, $y_{j,1}^{\ast }$ and $y_{j,2}^{\ast }$ are minimizers of d _j (μ ₁) and d _j (μ ₂), respectively. Now, since $y_{j,1}^{\ast }$ is not necessarily a minimizer of d _j (μ ₂), we can establish the following inequality,

$$\begin{array}{ll}{d}_j\left({\mu }_2\right)& =p_j\left(y_{j,2}^{\ast }\right)+{\mu }_1{y}_{j,2}^{\ast }+({\mu }_2-{\mu }_1)y_{j,2}^{\ast }\\ \le p_j\left(y_{j,1}^{\ast }\right)+{\mu }_1{y}_{j,1}^{\ast }+({\mu }_2-{\mu }_1)y_{j,1}^{\ast }\end{array}$$

(57)

Next, we prove by contradiction assuming $y_{j,2}^{\ast }>y_{j,1}^{\ast }$. In this case, it is true that (i) $({\mu }_2-{\mu }_1)y_{j,2}^{\ast }>({\mu }_2-{\mu }_1)y_{j,1}^{\ast }$ since μ ₂ − μ ₁ > 0 and (ii) $p_j\left(y_{j,2}^{\ast }\right)+{\mu }_1{y}_{j,2}^{\ast }\ge p_j\left(y_{j,1}^{\ast }\right)+{\mu }_1{y}_{j,1}^{\ast }$ since $y_{j,1}^{\ast }$ is a minimizer of d _j (μ ₁). Finally, observations (i) and (ii) together contradict the inequality in (57) and therefore $y_{j,2}^{\ast }$ must necessarily satisfy $y_{j,2}^{\ast }\le y_{j,1}^{\ast }$. This proves our second statement.

Proof of Proposition 2

Let us apply the KKT optimality conditions corresponding to (12). The Lagrangian is

$$\begin{array}{cc}L\left({\widehat{\mathit{y}}}^k,\mu ,\mathit{\alpha },\mathit{\beta }\right)=& \sum _{j=1}^J{\left(y_j^k-{ŷ}_j^k\right)}^2+\mu \left(\sum _{j=1}^J{ŷ}_j^k-C\right)\\ +{\mathit{\alpha }}^T({\mathit{y}}_{\mathit{\text{min}}}-\widehat{\mathit{y}})+{\mathit{\beta }}^T(\widehat{\mathit{y}}-{\mathit{y}}_{\mathit{\text{max}}})\end{array}$$

(58)

where ${\mathit{y}}_{\mathit{\text{min}}}=[\text{min}{\mathcal{Y}}_1,\dots ,\text{min}{\mathcal{Y}}_J]$ and ${\mathit{y}}_{\mathit{\text{max}}}=[\text{max}{\mathcal{Y}}_1,\dots ,\text{max}{\mathcal{Y}}_J]$ and, if we look at the optimality condition $\partial L/\partial {ŷ}_j^k=0$, we get

$${ŷ}_j^k=y_j^k-\mu -{\beta }_j+{\alpha }_j,j=1,\dots ,J$$

(59)

Therefore, assuming that the j th optimal primal value, i.e., ${ŷ}_j^{k,\ast }$, lies inside the interval $(\text{min}{\mathcal{Y}}_j,\text{max}{\mathcal{Y}}_j)$, then β _j = α _j = 0 due to slackness and ${ŷ}_j^k=y_j^k-\mu$ (with $\mu \in \mathbb{R}$). If this is not the case, then either ${ŷ}_j^{k,\ast }=\text{min}{\mathcal{Y}}_j$ or ${ŷ}_j^{k,\ast }=\text{max}{\mathcal{Y}}_j$. In both cases, note that we can choose an adequate value of the free dual multipliers α _j or β _j , respectively, in order to satisfy (59). Finally, taking these results into account, we can conclude that the solution to the primal projection is

$${ŷ}_j^k={\left[y_j^k-\mu \right]}_{\text{min}{\mathcal{Y}}_j}^{\text{max}{\mathcal{Y}}_j}$$

(60)

where μ is adjusted to accomplish $\sum _{j=1}^J{ŷ}_j^k=C$. Since y ^k≽y ^∗ ($\sum \underset{j}{\overset{k}{y}}>C$ unless $y_j^k=y_j^{\ast }\forall j$) and $\sum _{j=1}^J{y}_j^{\ast }=C$, it is necessary that μ > 0.

Proof of Proposition 3

From Proposition 2 it is verified that non-optimal values y that attain $\sum _{j=1}^J{y}_j>C$ diminish its value in the primal projection in order to achieve $\sum _{j=1}^J{ŷ}_j=C$ unless $y_j=\text{min}{\mathcal{Y}}_j$, in which case $\widehat{y_j}=y_j$. Let us distinguish two subsets of variables: $\mathcal{I}$ includes the indexes of the values y _j that attain $y_j=\text{min}{\mathcal{Y}}_j$ and $\bar{\mathcal{I}}$ the rest. Note that $\bar{\mathcal{I}}$ exactly contains the indexes of the $\left\{\breve{{\lambda }_j}\right\}$ candidates used in the dual projection. Then it is true that

$$\widehat{y_j}\le y_j^{\ast },\forall j\in \mathcal{I}\Rightarrow \sum _{j\in \mathcal{I}}{ŷ}_j\le \sum _{j\in \mathcal{I}}{y}_j^{\ast },$$

(61)

and therefore,

$$\sum _{j\in \bar{\mathcal{I}}}{ŷ}_j>\sum _{j\in \bar{\mathcal{I}}}{y}_j^{\ast }$$

(62)

since the equality constraints $\sum _{j=1}^J{ŷ}_j=\sum _{j=1}^J{y}_j^{\ast }=C$ are always fulfilled. This fact assures that there is at least one value ${ŷ}_j$ with $j\in \bar{\mathcal{I}}$ that attains ${ŷ}_j>y_j^{\ast }$ unless all the values are optimal yet.

Endnotes

^aNotation: ≼, ≺, ≽, and ≻ stand for component-wise inequalities.^bThe vector s is a subgradient of the function $\mathit{f}:{\mathbb{R}}^n\to \mathbb{R}$ at $\mathit{x}\in {\mathbb{R}}^n$ if $\mathit{f}\left(\mathit{y}\right)\ge \mathit{f}\left(\mathit{x}\right)+{(\mathit{y}-\mathit{x})}^T\mathit{s},\forall \mathit{y}\in {\mathbb{R}}^n$. If f is differentiable at x, the subgradient s and the gradient ∇f(x) coincide. Otherwise, there exist many subgradients.^cThe notation ${\mathit{x}}_j^{\ast }\left({\mu }^k\right)$ stands for the optimal solution of the j th subproblem given μ ^k.^dNotation: ⊙ stands for vector element-wise product. If a= [a ₁,a ₂,…,a _N ]^T and b= [b ₁,b ₂,…,b _N ]^T, then a⊙ b= [a ₁·b ₁,a ₂·b ₂,…,a _N · b _N ]^T.^eNotation: ${\left[a\right]}_{m_j}^{d_j}$ equals a if m _j < a < d _j , m _j if a ≤ m _j and d _j if a ≥ d _j .