Incentive-compatible demand-side management for smart grids based on review strategies

3.1 Power system

We consider a smart grid system with multiple consumers and one energy producer, e.g., a utility company. These consumers receive electricity from the same aggregator which distributes electricity to the consumers. Each consumer is equipped with an energy consumption scheduler (ECS) for scheduling the household energy consumption. A smart meter is connected to the set of consumers from which it collects and analyzes the energy consumption. This smart meter gathers (almost) accurate readings automatically, at requested time intervals, and relays them to the utility company. Using this information, the aggregator (utility company) can adjust its energy generation, purchase, and transmission accordingly. The communication between the utility company and the consumers’ smart meters is done through the local area network (LAN) by using appropriate communication protocols. Let \(\mathcal {N}\) denote the set of consumers that share the same aggregator, where the number of consumers is N. Figure 1 illustrates the system model.

where b _peak>b _off−peak>0 and B(·) is a benefit function.

where κ is the revenue/cost ratio. If κ=1, then the billing system is budget-balanced and the utility company charges the consumers only the generating/providing energy costs for the utility (i.e., the utility company does not make money and it serves simply as a benevolent energy provider). If κ>1, then the difference between the total charges to the consumers and the total energy cost represent the profit made by the utility company. In this paper, the protocol designer is considered to be benevolent and represent the consumer’s interests rather than maximizing the utility company’s profit. Thus, κ=1 in the subsequent analysis.

We make the following standard assumptions on the benefit function and cost functions throughout this paper.

where ε is induced by the monitoring noise of the usage pattern. Because we assume that the schedulable demand vector d is fixed, the price p(D _peak,D _off−peak) only depends on the consumption scheduling actions a of the consumers. Therefore, we alternatively write the price p(a) as a function of the scheduling action profile p(a). Note that p(a) is the expected price if a is taken and \(\hat {p}({\boldsymbol {a}})\) is the actual realized price if a is taken in the noisy environment. When consumers are making scheduling decisions, only p(a) is important since consumers can only compute the expected price but not the actual price which has not been realized yet.

A billing cycle consists of L days. At the beginning of each billing cycle, the consumers determine the consumption scheduling actions for the next L days. At the end of each billing cycle, the utility company posts bills as well as the electricity price of the previous cycle. Using the pricing information, the consumers are able to infer the aggregate daily usage pattern in the last L days. However, since the price does not perfectly reflect the aggregate usage pattern due to the noise term, the consumers’ knowledge of the aggregate usage pattern is imperfect.

3.2 Energy consumption game: stage game

Since consumers are self-interested, they will want to selfishly maximize their own utilities. We use Nash equilibrium as the solution concept of this energy consumption game.

In NE, no agent can improve its own utility by unilaterally changing its own action. Theorem 1 proves the existence of NE of the considered energy consumption game.

It is well known that NE is often not efficient in terms of Pareto-optimality. If an action profile is Pareto-optimal, then no consumer can gain a higher utility without decreasing at least one other consumer’s utility by using a different action profile. We provide a formal definition as follows.

Even though a PO action profile a ^PO is superior to a ^NE, consumers do not have incentives to automatically adopt this action profile in the energy consumption game since they are self-interested. This is because for any action profile that is not a NE, there will always be some consumers who want to schedule a different amount of energy consumption in peak hours to increase their own utilities. In order to provide incentives for the consumers to play the PO action profile, in this paper, we will design a protocol that exploits the ongoing nature of consumers’ energy consumption interactions. For notational simplicity, we denote u _n (a ^NE) as \(u_{n}^{\text {NE}}\) and u _n (a ^PO) as \(u_{n}^{\text {PO}}\). Before we proceed to that, we provide a simple example to illustrate the difference between a ^NE,a ^PO, and the inefficiency of a ^NE.

Example (Two consumers). Consider two consumers: N=2. Let κ=1,b _peak=2,b _off−peak=1,d ₁=d ₂=1 and B(x)=x. The cost functions are simple quadratic functions as in [22]: C _peak(x)=x ²,C _off−peak=0.5x ². The utility of consumer n is therefore u _n (a ₁,a ₂)=1+a _n −0.5[(a ₁+a ₂)²+0.5(2−a ₁−a ₂)²].

In this two-consumer game example, the unique symmetric NE action profile is a1NE=a2NE=2/3. That is, both consumers schedule 2/3 of their energy consumption in peak hours and no one wants to unilaterally schedule a different amount since that will only reduce its own utility. Thus, the corresponding utilities are u1NE=u2NE=2/3. However, the sum utility is maximized when both users choose the action profile a1PO=a2PO=0.5, which is the unique symmetric PO action profile. The corresponding utilities are u1PO=u2PO=5/4. The sum utility obtained when taking this action profile is 87.5 % higher than that by taking the NE action profile (i.e., 4/3), thereby indicating the inefficiency of NE.

3.3 Repeated energy consumption game

That is, the long-term utility is the normalized sum of the discounted expected stage utilities which are induced by the strategy where the expectation is taken over the probability over different sequences of action profiles \(\left \{{\boldsymbol {a}}^{t}\right \}_{t=1}^{\infty }\).

Figure 3 illustrates the timeline of the system and the decision flow of consumers.

3.4 Problem formulation

The constraint requires that all consumers have incentives to follow the protocol Ψ.

4.1 Protocols based on review strategies

In a protocol based on review strategies, there are two types of phases: review phases and punishment phases. Each review phase consists of one billing cycle. Since the length of the billing cycle is designed by the protocol designer, the length of a review phase is a design parameter. Each punishment phase consists of a certain integer number K of billing cycles, and hence, it has KL time slots (i.e., days). The protocol designer recommends a review strategy σ ^R for all consumers as follows: the recommended scheduling action profile is a ^PO for each day in the review phase; the recommended scheduling action profile is a ^NE for each day in the punishment phase. At the end of each billing cycle in the review phase, a statistical test is performed (either by the consumers themselves or the utility company) on the prices of the previous cycle to determine whether other consumers followed or not the recommended strategy. A signal \(z \in \mathcal {Z} = \{0,1\}\) is generated based on the test results with z=1 representing that all consumers followed the recommended strategy and z=0 representing that some consumers deviated from the recommended strategy. If z=1, the system moves to another review phase; if z=0, the system moves to the punishment phase. When a punishment phase ends, the system automatically moves to a new review phase. This is the protocol based on review strategies depicted in Fig. 4.

Importantly, note that the utilities obtained by the consumers in the punishment phase are lower than in the review phase and they are the same as in the scenario where there was no review strategy protocol being deployed. Therefore, the review strategy-based protocol is guaranteed to achieve better system performance than that which can be obtained without deploying the protocol. However, the designer still needs to carefully design the protocol such that the system enters the punishment phase as rarely as possible.

Given this review strategy structure, the protocol can be fully characterized by the length of the review phase (i.e., the billing cycle length) L, the length of the punishment phase KL and the statistical test G. Therefore, we write a protocol based on review strategies as Ψ(L,K,G).

4.2 Performance metrics

To evaluate the protocols based on review strategies, we need to define several performance metrics. First, the protocol should be IC since the consumers are self-interested and will only follow recommendations when this is in their self-interest. Particularly, consumers should have incentives to take the PO scheduling actions in the review phase.

Proposition 1 states that, given the daily desired energy demand of a consumer, if the designer recommends to it the PO scheduling action, then the self-interested consumer will schedule no less than the PO energy consumption in peak hours in order to maximize its own utility assuming other consumers are complying with the recommended PO scheduling. Note that this applies even in the case when the consumer’s daily demand is low since the consumer is simply re-scheduling its desired amount of energy consumption to different hours. In the two-consumer example, given the recommended PO action profile a ₁=a ₂=0.5, a consumer can only increase its utility by scheduling more than half of its energy consumption in peak hours to increase its own utility. Therefore, we only need to focus on the case \({a^{d}_{n}} > a^{\text {PO}}_{n}\) when considering consumers’ incentive problems and denote the corresponding utility by \({u^{d}_{n}}\) for consumer n. Note that consuming more energy in peak hours increases the price and, hence, consumer n’s own payment is increased. However, because the costs are shared among the entire consumer set, consumer n is still able to receive a higher utility \({u^{d}_{n}} > u^{\text {PO}}_{n}\) by unilaterally increasing its own consumption in peak hours when ∇ _n u _n (a ^PO)>0.

At the beginning of each billing cycle, the consumers determine the scheduling actions for this billing cycle based on the previous energy consumption history. We will focus on constant strategies during a billing cycle, i.e., consumer n chooses a constant scheduling action every day during a billing cycle. However, our analysis can also be extended to more sophisticated strategies (i.e., the consumer may use different scheduling actions during a billing cycle) by taking the equivalent average scheduling action. Let U _n (σ ^R |s=1) and U _n (σ ^R |s=0) denote the long-term utilities of consumer n at the beginning of a review phase and a punishment phase, respectively, if all consumers follow the strategy σ. Let \(\tilde {\sigma }^{R}_{n}\) denote a strategy where consumer n deviates to some \({a^{d}_{n}} > a^{\text {PO}}_{n}\) in the current review phase and follow the review strategy afterwards and all consumers follow σ ^R all the time. The following proposition characterizes the IC condition of a protocol.

Proposition 2 uses the one-shot deviation principle in repeated games and shows that if a consumer cannot gain by unilaterally deviating from the recommended strategy σ ^R in the current billing cycle and following afterwards, it cannot gain by switching to any other strategies either and vice versa. In the punishment phase, the recommended strategy is the NE scheduling action profile a ^NE, and hence, the consumers will not have incentives to deviate from the recommended action. Therefore, we only need to check whether the consumers have incentives to take the recommended PO profile a ^PO, i.e., whether the one-shot deviation principle holds in the review phase. The left-hand side of (10) represents the long-term utility by following σ ^R , and the right-hand side represents the long-term utility for consumer n by deviating to \({a^{d}_{n}}\) only in the current review phase while all other consumers follow σ ^R . In Section 4.2, we will use Proposition 2 to construct IC protocols based on review strategies.

The goal of the protocol designer is to maximize the sum utility of all the users. The maximum sum utility is achieved when all consumers take the PO scheduling actions in each time slot, denoted by \(V^{*} = \sum _{n} u_{n}^{\text {PO}}\). We call V ^∗ the “first-best” utility which yields the maximum sum utility for the consumers. The efficiency loss of a protocol is defined by C(Ψ)=(V ^∗−V(Ψ))/V ^∗.

A Δ-PO protocol yields a sum utility no less than 1−Δ of the first-best sum utility. If a protocol Ψ prescribes the scheduling action profile a ^PO every day regardless of the history, then V(Ψ)=V ^∗ and thus Ψ achieves the first-best sum utility (0-PO). However, such a protocol is not IC, and hence, the first-best is not achievable.

4.3 Statistical test

If there are some consumers who deviated by consuming more than the recommended amount of energy during the peak hours, the price of that day is increased⁴. Due to the monitoring noise, the actual price is \(\hat {p}\left ({a^{d}_{n}}, {\boldsymbol {a}}^{\text {PO}}_{-n}\right) = p\left ({a^{d}_{n}}, {\boldsymbol {a}}^{\text {PO}}_{-n}\right) + \epsilon \).

Note that our design can also be extended to analyze the collusion problem where a subset of consumers colludes in order to gain higher utilities. In that case, we can simply take the consumers who collude as a single consumer with its demand being the aggregate demand.

The following proposition characterizes the impact of the threshold and the billing cycle length on the error probabilities.

Proposition 3 informs the protocol designer’s selection of the statistical test (i.e., the test threshold). It also proves that, in order to accurately detect self-interested consumers’ excessive usage in peak hours, a longer billing cycle should be chosen such that the monitoring mitigated. However, a very long billing cycle also reduces the consumers’ valuation of utilities in the future billing cycles, and hence, the punishment may not be strong enough to induce consumers’ compliance. Therefore, the optimal billing cycle as well as the punishment length should be carefully designed to induce the optimal performance of the smart grid system.

5.1 Incentive-compatibility

The first term in (15) is the utility gain by the deviation in the current review phase which is larger than \(u^{\text {PO}}_{n}\). The second term is the continuation utility after the review phase. With probability 1−q _M,n (L,p _th) the deviation is detected, so the system moves to a punishment phase; with probability q _M,n (L,p _th) the system remains in the review phase.

Let us examine the physical meaning of this incentive ratio. Essentially, the numerator represents the long-term utility gain due to deviation, and the denominator represents the maximal long-term utility loss due to the punishment. To enforce consumers to cooperatively optimize their energy consumption, this loss should be positive and larger than the gain obtained when deviating. Therefore, the incentive ratio should be in the range [0,1]. It is worth noting that g _n (L,p _th ) should be strictly less than 1 since the denominator is only an upper bound on the loss induced by the punishment but not the actual loss (which depends on L,K,p _th). Theorem 2 provides a condition such that a protocol based on review strategy is IC against a deviation action \({a^{d}_{n}}\). This condition serves as a guideline for the choices of the proper protocol parameters L,K, and p _th.

5.2 Optimal protocol parameters

We first determine the efficiency of a given protocol. If a protocol is IC, then all consumers follow the recommended strategy and play the PO action profile. Therefore, the efficiency depends on the probability that the system is in review phases and punishment phases due to monitoring errors. Denote these two probabilities by η _R (Ψ) and η _P (Ψ)=1−η _R (Ψ), respectively. The efficiency of an IC protocol is thus \(V(\Psi) = \sum _{n} (u^{\text {PO}}_{n} \eta _{R}(\Psi) + u^{\text {NE}}_{n} \eta _{P}(\Psi))\) where η _R (Ψ) is determined in the following Lemma.

Proof.

Solving the stationary distribution of the Markov chain in Fig. 6 yields the result. The transition probabilities of the chain are R-R with probability 1−q _F , R-P with q _F , P-P and P-R with probability 1.

Lemma implies that in order to maximize the system efficiency, the protocol designer should choose L,K,p _th such that K q _F (L,p _th) is minimized subject to IC conditions in Theorem 2. These design parameters are coupled in a complex manner, and thus, to find the optimal design parameters, it is better to work backwards.

In general, the statistical test threshold p _th has two opposite effects on K ^∗(L,p _th)q _F (L,p _th). Minimizing q _F (L,p _th) often leads to a large q _M,n (L,p _th), and hence, q _F (L,p _th)+q _M,n (L,p _th) may also be large. This by (16) induces a large K ^∗(L,p _th). Therefore, the protocol designer has to trade-off when selecting p _th between minimizing the false alarm probability q _F (L,p _th ) and the punishment phase length K ^∗(L,p _th).

Step 3. The previous two steps provide the optimal \(p^{*}_{\text {th}}\) and \(K^{*}(L, p_{\text {th}}^{*})\) given the review phase length L. However, the space of L includes all positive integer numbers and is infinite. In the following, we determine the upper bound on L such that an IC protocol can be designed.

Proposition 4 leads to a crucial trade-off of the review phase length. On one hand, the protocol designer wants to choose a longer review phase period L because it improves the monitoring accuracy, and hence, there is a smaller probability that the system goes into a punishment phase due to monitoring errors. On the other hand, a longer review increases the current gain of a consumer in the review phase upon deviation while it reduces the future loss due to the punishment because of the discounting of the future utility. This requires a stronger punishment (longer punishment phase), and hence, it induces more energy consumption in peak hours as the system stays longer in the punishment phase. More importantly, if the review phase is too long, then even the strongest punishment (i.e., a trigger strategy that prescribes to stay in the punishment phase forever upon deviations) is not able to provide the sufficient incentives for the consumers to schedule the PO energy consumption in the current review phase. Therefore, given consumers’ valuation of the future utility (i.e., the discount factor δ) and the structure of the stage game (i.e., \({u^{d}_{n}}, u^{\text {PO}}_{n}, u^{\text {NE}}_{n},\forall n \in \mathcal {N}\)), there is a maximum length of the review phase, and hence, the billing cycle should not be too long. To make the protocol IC, the protocol designer must choose a review phase no longer than the upper bound determined in Proposition 4. The optimal review phase length L (i.e., billing cycle) is thus such that it minimizes the product of \(K^{*}\left (L, p^{*}_{\text {th}}\right)\) and \(q_{F}\left (p^{*}_{\text {th}}, L\right)\) which have been determined in the previous two steps for a fixed L. Based on the above three design steps, we propose a greedy algorithm (presented in Table 2) to determine the optimal protocols based on review strategies which requires only finite iterations on L. If the candidate threshold p _th belongs to a continuous interval, then we need to quantize p _th to solve (19) in finite iterations, and hence, the algorithm leads to a sub-optimal protocol.

5.3 Performance evaluation

In the previous subsection, we determine the optimal protocol design. In this subsection, we characterize the performance of these optimal protocols. Specifically, we are interested in determining whether a protocol based on review strategies Ψ can be Δ-PO.

Recall that the first-best efficiency is \(V^{*} = \sum _{n} u^{\text {PO}}_{n}\), and the sum utility of a given IC protocol can be determined using the result in Proposition 3. Therefore, if η _R (Ψ) is close enough to 1, then (V(Ψ)−V ^∗)/V ^∗ can be made less than a given Δ. According to Proposition 3, this can be done if q _F is close enough to 0 and if K is finite. That is, an accurate enough statistical test is required. By Proposition 4, if L is long enough, then the monitoring can be accurate enough. However, a long L can be chosen only when the consumers valuate the future utilities sufficiently high enough. The following theorem characterizes the condition when a protocol is Δ-PO.

Theorem 3 characterizes the asymptotical performance of the protocols based on review strategies. Specifically, when the consumers highly value their future utilities, then the protocol designer can design a protocol based on review strategies whose sum utility is close enough to the first-best. That is, consumers will comply with the recommended scheduling most of the time.

The corollary states that if the consumers do not discount their future utilities, then the protocol designer can design a review strategy-based protocol which is IC and also asymptotically achieves the full efficiency.

Remark: Our analysis depends on the deviation strategy a ^d . To compute \({a^{d}_{n}}\), we consider the unilateral deviation by consumer n at the optimal action profile a ^OPT, i.e., \({a^{d}_{n}} = a^{*}_{n} = \arg \max _{a_{n}} \mu _{n}\left (a_{n}; {\boldsymbol {a}}^{\text {OPT}}_{-n}\right)\). In this way, a consumer chooses to deviate to the action that maximizes its own utility. However, if the consumer is “smarter”, it can deviate to a slightly lower action than \(a^{*}_{n}\) to avoid being detected but still gains some increased utility (of course, since there is noise, the probability of being detected is higher if its selected action is closer to \({a^{d}_{n}}\)). Hence, a more practical way to set \({a^{d}_{n}}\) is by setting a maximal tolerable deviation action \({a^{d}_{n}} = (1+\gamma)a^{\text {OPT}}_{n}\) where γ<1 depends on the maximal tolerable social welfare loss. Since, by doing this the (one-shot) social welfare loss is at most \(u_{n}\left ({\boldsymbol {a}}^{\text {OPT}}_{n}\right) - u_{n}\left ((1+\gamma){\boldsymbol {a}}^{\text {OPT}}\right)\), the designer can determine γ according to the maximal tolerable social welfare loss and set \({a^{d}_{n}}\) accordingly.