Jul 10, 2020 System Convergence

A fundamental premise of most applied transport models is the existence and uniqueness of an equilibrium solution that balances demand and supply . The demand consists of the people that travel in the transport system and on the defined network, whereas the supply consists of the resulting level-of-service attributes (e.g., travel time and cost) offered to travellers. An important source of complexity is the congestion, which causes increasing demand to affect travel time in a non-linear way. Transport models most often involve separate models for traffic assignment and demand modelling. As a result, two different equilibrium mechanisms are involved, (i) the internal traffic assignment equilibrium, and (ii) the external equilibrium loop between the assignment model and the demand model. The main objective of the paper will be to analyse convergence performance of the external loop. Firstly, we investigate the method of repeated approximations (MRA) and the method of Successive Averages (MSA). Moreover, we discuss variations of the MSA algorithm, including weighted MSA, MSA with memory reset, and MSA with Polyak step-size. Secondly, we discuss the possibility of using polynomial smoothing. Finally, we perform a sequence of simulation tests on a “toy” network to investigate convergence properties of the different algorithms.


INTRODUCTION
In transport modelling, one of the most fundamental equilibrium principles is the internal route choice equilibrium, where a route choice model (demand) iterates with a time-flow model (supply). The complexity of this iteration scheme arise because increasing demand cause a disproportional increasing in travel time, which in turn will reduce the demand. It is generally recognised that the method of repeated approximations (MRA), which is basically a simple iteration scheme where the levelof-service variables is fed directly to the route-choice and vice versa, may exhibit an unstable pattern and lead to cyclic unstable solutions. It can be shown that the contraction region, e.g. the region for which a starting solution will render stable convergence, depends on the slope of the demand and supply curve. Generally, as the slope (i.e., and ) between the curves increases, the contraction region shrinks. To obtain stable convergence various techniques including the Method of Successive Averages (MSA) have been proposed. Convergence of the MSA under fairly weak regularity conditions was shown in Robbins and Monro (1951).
The iteration between demand and assignmentthe external equilibriumare in many models either decoupled or follow the MRA principle. However, as demand models are often based on logit or probit models, and thus conform to the way demand is represented in stochastic assignment models, there is reason to believe that convergence problems could also be expected in the external equilibrium loop. The intuitive explanation is that, if an iterative solution algorithm may not converge in traffic assignment with fixed demand (base OD-matrix), adding the complexity of variable demand makes the problem even more difficult to solve. Another strong motivation for be concerned with the extern loop convergence relates to the computational effort. As the external equilibrium loop involves running a complete assignment model combined with a complete demand model, iterations are much more costly than for the inner loop. This does not justify a simple iteration scheme for the sake of simplicity. As only 3 to 8 iterations may be possible in practice, it is important that these are spent wisely.
An additional motivation for the investigation of the external loop is that the authors recently have experienced "occasional" convergence problems in practice in several large-scale models when applying simple MRA.
In the paper, we first give an introduction to fix point algorithms (in Section 2) In Section 3, we turn our focus to the convergence of the external loop and discuss averaging methods and introduce the concept of polynomial smoothing. Section 4, is concerned with simulation experiments on a medium sized "toy network" and we investigate convergence performance of the various algorithms under a variety of settings. In Section 5 we offer a summary and conclusions.

INTRODUCTION TO FIX POINT ALGORITHMS
Let demand be represented as a continuous vector function and a non-empty, compact and convex set. By Brouwer theorem it has at least one fixedpoints (existence). If it can also be proved that at most a fixed-point exist (uniqueness), the unique fixed-point may be found through many algorithms, whose general specification can be written as; (1) The algorithm is based on a starting solution , and a "duly" defined matrix (see among many others Kelley, 1995).
The method of repeated approximations (MRA), as in the Banach theorem, is given by , but it may be proven to converge only for contractions (or for strictly nonexpansive functions).
The Newton method (also referred to as the method of tangent approximations), is based on and will usually convergence fast provided that the starting solution is close enough to the searched fixed-point. The Broyden method is a kind of secant approximations, where matrix is update at each iteration, from ; it gives some computation advantages with respect to the Newton method since derivates need to be computed only at first iteration.
The method of successive averages (MSA) is given by with , and convergence conditions are given by the Blum theorem (Blum, 1954) or by an extension in Cantarella (1997). If the function is computed through Monte Carlo simulation which only provides an unbiased estimation of the value, only almost sure convergence can be considered. Since all intermediate solutions in the sequence are feasible, algorithms based on MSA are often called feasible, and the current solution may be considered an approximation to the searched equilibrium flows. These algorithms are also called simple since they only require computation of all the involved functions and will not require computation of derivative and no need of matrix algebra during iterations (Cantarella and Cascetta, 2009).
The "success" of the MSA (and variants of the MSA) in the sense that it converge is due to the fact that "averaging" form a contraction principle. Although it is not a contraction in a strict mathematical sense, it is a contraction principle which will lead to convergence as the number of iterations goes to infinity. The principle of MSA is illustrated in Figure 1   illustrates how averaging will form a contraction and that the MSA is particular efficient in the start of the iterative scheme. The draw-back of the MSA is that the speed of convergence tends to be relative slow as we move towards the equilibrium. Figure 2 also illustrate how the MRA may fail to converge if the slope of the demand and supply curve is too steep.

Route choice fix-point formulation
A natural starting point for the discussion of external convergence is to consider convergence of the inner loop. Firstly, because the external loop is conditioned on the inner loop and secondly, because many of the principles applied in the inner loop is parallel to what may be applied in the external loop.
As a starting point, we will consider a simple route choice example. Consider a simple system of two routes connecting two nodes. We will assume that each route is only represented by a single link as illustrated below in Figure 2. We assume a simple BPR formula for the time-flow relationship, Where define the loading on route and the capacity. Generally we will assume that and . In (2) it is assumed that the two routes have the same free-flow travel time .
Furthermore, we may define the route-choice probability on link as a multinomial logit model (3) If we consider a fixed demand problem, where the OD matrix is fixed and equal to , the demand on the two links defined by for is given by the OD demand multiplied by the corresponding route choice probability, hence; (4) (5) Mathematically, the system can be written as a fix-point system (6)   (7) The equilibrium can be found as the solution to this dynamic system, e.g.
. The dynamic properties of the dynamic system can be investigated by evaluating the determinant of the Jacobian matrix; As the current system is to-dimensional it is easy to assess that the dynamic properties depends on the slopes of the demand and supply curve.

CONVERGENCE WHEN DEMAND IS VARIABLE
The convergence of the external loop will generally be smoother than convergence of the inner loop. This is because the supply function is represented as a weighted averaged of LoS over all routes between OD pairs. The averaging tends to dampen volatility caused by single stretches.
Below, we will investigate the convergence of the external in sequential convergence scheme, where an external demand model interacts with an internal assignment model. Hence, we do not consider an integrated approach (Cantarella, 1997), as this is rarely used in practise for large scale modelling. More, specifically, we apply an iterative scheme consisting of five steps as illustrated below; Step1: Calculation of Initial Level-of-service (traffic assignment of base-line matrices).
Step2: Demand model (outer-loop), based on possible scenario data and initial LoS (Step 1).
Step4: Calculation of Level-of-service (assignment of new demand matrices, the Inner-loop).

1 Averaging methods
As discussed in the introduction, averaging is a popular method to obtain convergence under mild regularity conditions. Applying the MSA in the external loop is straightforward and implies that demand at iteration , is represented as a moving average (9) Demand will in this example represent the set of OD pairs. In the original paper by Robbins and Monro (1951) it was suggested that . With this parameterisation of , the MSA puts most weight on the "history" and less weight on the current iteration. This generally works quite well if we have a "spiderconvergence" as illustrated in Figure 1 where the iteration scheme jumps between the curves and in particular for the first couple of iterations. However, it tends to be less efficient as the iteration number increases. The problem is that the complete iteration history is inherited in the new demand update including the very "noisy" first iterations. Figure 3 illustrates an example where the performance of the MSA will be particular weak, namely when the slope of the supply curve are relative flat towards the equilibrium. As this is usually the case for low and medium congested sections of the network, this type of convergence behaviour will usually account for a majority of the network loading.
To deal with this issue several methods has been proposed. The underlying idea of most of these methods is to define a sequence that conform to the regularity conditions (Blum, 1954) and where the weight of the first part of the iteration process is gradually damped.
A simple idea was made by Cascetta and Postorino (2001) as they suggest resetting the iteration history at some points in the iterative process. If we let be the number of MSA iterations before reset, the resetting can be accomplished by defining a new iteration index equal to; (10) The MSA is then simply defined with step-size , which will produce a repeated sequence Cascetta and Postorino (2001) suggest resetting the history for every 5 steps, however, the optimal choice is strongly network dependent and will also depend on the acceptable precision level for the final iteration and the number of iterations we are willing to run. It should also be said, however, that in order for the resetting approach to be consistent with the regularity conditions (Blum's theorem) stating that and , there should always be a point from where the reset is no longer used. Moreover, choosing a value of which is to small can be dangerous and it is generally not recommended to use values less than 5. Polyak and Juditsky (1990) introduce an alternative step-size for the MSA equal to . Compared to the original suggestion by Robbins and Monro (1951), this specification put more weight on the newest iterations which should be preferable in this context.
A weighted MSA approach was considered in Liu et al. (2009) where is given by (12) Where . Clearly, for the ordinary MSA emerge, however, as increases more emphasis is put on the latest iterations.

Polynomial smoothing
As discussed in the previous section, many heuristic MSA variants can be proposed. Although some of these are fairly efficient the efficiency is generally a function of the network and the structure of the OD. It is therefore relevant to consider approaches, which are "self-adjusted" and adopts with respect to the curvature of the iteration scheme. Interestingly, many OD pairs will converge in a smooth and monotone way, where the iterative process "crawl" on the supply curve and stays under the equilibrium all the time. This situation was illustrated in Figure 3. Typically, these situations are characteristic by monotonous convergence with decreasing absolute slope.
An obvious idea is therefore to consider polynomial smoothing of the demand curve. We are not considering smoothing of demand as a function of supply in a classical sense as the supply curve cannot be represented as a curve and may not even be monotone and continuous. Rather we suggest considering the iteration scheme as a mapping from the domain of iterations to the external demand model, e.g. (13) The external loop, whether it is iterated as MSA or MRA, can be considered as a set of equally spaced data points, in the sense that we assume an equal space between the iterations. Due to the equally spaced data points, we can apply a numerical smoothing process as suggested in Savitzky and Golay (1964).
The smoothing will require a set of data points, which is why the strategy is to first run a MSA variant for at least three iterations, and then do a 2-order polynomial smoothing. As the MSA tends to be fairly efficient during the first iterations this makes good sense. By solving we attain a minimum or maximum and may then be considered as the next step on the demand curve. Hence, in Figure 4 is the estimated starting point prior to iteration 4, which in turn will produce the true point . If the curve is monotone decreasing (or increasing) it is possibly to "jump" quite a few iterations as illustrated in Figure 4.
The polynomial fit for the first there points is straightforward. If we choose the smoothing point as the latest point, hence , we may define a change of variable, e.g.
. A polynomial is now defined for (14) The "convolution" coefficients can now be calculated by solving the following equation system (15) Where represent the data points and the Jacobian. As we consider only a second-order polynomial, the Jacobian is simply given by (16) A suggested iteration scheme of the smoothing approach might be; Step1: Process three points using MSA.
Step2: Fit a second-order polynomial on the available data points developed around the most recent point.
Step3: Find such that and let it be the starting point prior to the next iteration.
Step4: Carry out the next iteration of the external loop and attain the next data point.
Step5: Construct a new data point by doing MSA (or a variant) based on the previous data points.
There are a couple of issues related to the suggested iteration scheme that should be discussed. Firstly, it should be pointed out that although the smoothing approach might work well when the iterative process is smooth it will work quite badly (for only three points) when the iteration process oscillate. An idea might be to assign MSA to oscillating entries and smoothing to smoothly developing entries, however, this is complicated by the fact that the convergence type may change while iterating.
Another issue is that the above algorithm may fail to converge if we do not embed the MSA in Step 5. Clearly, if we "jump" iterations, running an ordinary MSA with will automatically put more emphasis on the newest iterations compared to a traditional MSA iteration scheme. An example is that we jump from to , which in turn will cause the MSA to represent for the fourth point compared to if a normal MSA had been processed.
More importantly, however, the sequence will be a function of the curvature of the demand model and in this sense be "self-adjusted" and less sensitive to external parameters.

SIMULATION EXPERIMENTS
To analyse how various specifications affect the convergence of the external loop, a small "toy" network has been constructed as illustrated in Figure 5 below. The network is represented 9 nodes (or zones), which by elimination of inter-zone traffic gives 72 OD pairs connected by 16 arcs. There is no assumption of limitations in the network (e.g., closed links or one-way links) and the total combination of routes is 3,804 representing 22,404 connected arcs. As a result, the average route consists of 5.88 arcs.
Different OD matrix structures have been generated randomly to reflect different OD flow patterns and congestion levels.
The arc cost table is presented below in Table 1. In the simulation experiment, for the inner loop, we consider a full stochastic loading on all routes for each loop in the assignment (approximately 50 routes per OD pair). This is in contrast to a normal assignment model, where routes are sampled sequentially in the MSA loop. As a result, each iteration conform to approximately 50 inner-loop iterations and hence, the convergence is fast and will normally fully converge after 10 iterations.
In the following we will address four issues in a range of simulation experiments.

Non-convergence of the outer loop
The first and very obvious issue is whether we need to consider convergence problems in the outer loop at all? As many models have applied MRA iteration schemes with success it is a question whether it is possible to construct a counter example?
However, it turns out to be straightforward to generate an infinite number of counter examples where the external loop will diverge. If the slopes of the demand and supply curves are moderately flat, usually the performance of the MRA will be quite good and generally better than the MSA. However, as the slope increases the divergence is amplified and at some point it will be cyclic unstable.
Rather than illustrating this in a separate figure we refer to Figure 8, which indicate a nearly cyclic unstable MRA convergence and in all cases converge very slowly.

Benchmark of averaging methods
In the following we will present a series of convergence benchmarks for the MRA and five averaging methods.
A first and interesting observation is that although the MRA may fail to converge in certain cases it will generally be quite efficient compared to many averaging methods if it converges. The main explanation for this is that many averaging methods will tend to converge slowly when the slope of the supply curve up to the point where it crosses the demand curve is relative flat. Figure 6: Convergence performance measured in terms of % deviation from equilibrium of six methods (iterations in the horizontal axis); "normal" congestion. The weighted MSA with performs quite well for only 3 iterations. The MSA with reset is also efficient; however, it requires that we pass the reset point (in this case 5 iterations). In fact, from an infinitesimal point of view the MSA reset performs best from 10 iterations and out. Figure 7 below illustrate a situation where the congestion level is higher. For the BPR , logit demand parameter (route choice) is -0.5 and logit parameter for demand model is -1.3. As can be seen, the performance decline for all of the methods. It is however interesting to see that the weighted MSA with ( ) is generally relative efficient for event a small number of iterations. The MSA with reset tends to be efficient after the reset point of 5 iterations.  Generally, the performance of the MSA as well as the MSA with Polyak step size and weighted MSA ( ) decline and tend to be unacceptable for a small number of iterations. However, the weighted MSA ( ) and MSA with reset perform well. The best performing method is again the weighted MSA with . The MSA with reset is again quite good, although "jumps" can occur, especially around reset points. Also, note the oscillating pattern for the MRA and the extreme slow convergence of the MSA.

MSA with reset
As indicated in the simulation experiments, the MSA with reset will generally perform reasonable well. However, it requires as a minimum that the number of iterations exceed the first reset point and at best 2 or 3 reset points. Also as we saw in Figure  8, "jumps" can occur.
An obvious question is what reset point is optimal? Although the answer cannot be answered in any precise way as it is network dependent, simulation can give indications on how choice of reset point affect the iteration history. Figure 7: Convergence performance of MSA reset with different reset points; "hyper" congestion. Figure 7 suggest an interesting finding, namely that the choice of reset point, whether it is 2, 3, 4, 5, 6 or 7 are not so important even in a "hyper" congestion regime. What is important is to choose the correct stoppage point after the reset point. Given that iterations are costly, the optimal point seems to always be 1 plus the reset point. Hence, for a reset point of 2 we should stop after the third iteration, for a reset point of 5, we should stop at 6, etc. The finding has been consistent for a range of different random OD matrices.
To fully investigate whether this is due to "peculiarities" in the data or point to a more general finding needs to be assessed in more details.

Weighted MSA
Another interesting question is whether it may be efficient to further decrease the emphasis on "history" in the weighted sequence; hence increase the value of to 3 or 4. Again we turn to the "hyper" congestion regime from Figure 9 as it provides the greatest challenge to the algorithms. As can be seen, there is an optimal value somewhere between 1 and 4. However, seems to be the more efficient choice.

SUMMARY AND CONCLUSIONS
The present paper has investigated the convergence between a demand model as represented by a linear-in-parameter logit model and an assignment model. Whereas much research has been invested in the inner-loop convergence of the assignment model, the convergence of the external loop has not been given any attention to our knowledge. This somehow contradicts the fact that the external loop iterations are far more costly as they (for every iteration) involve a complete assignment convergence.
All though the results of the current paper relates to a "toy network" and should be seen as "work in progress" a range of interesting indications has been revealed.
-The common MRA algorithm will usually be relative efficient in low-congested networks, however, convergence is not guaranteed and cyclic unstable convergence may be present. Generally, as the congestion level increase, the performance of the MRA decline. -The MSA in it its original form with will generally converge very slowly and in particular when the iteration scheme are non-oscillating.
-A number of MSA variants has been analysed to deal with the slow MSA convergence. It has been found that; o A weighted MSA ( ) is particular efficient, even in situations with few iterations and "hyper" congestion.
o A MSA with reset is also very efficient, although the reset point and the iteration stoppage point needs to be chosen carefully. It appears to be efficient to stop the algorithm one iteration after the reset points irrespectively of the reset length.
o MSA with Polyak step size and weighted MSA with ( ) is significantly better than the MSA but also significantly worth than the two methods mentioned above.
It has also been pointed out that the MSA variants, although some of these may be efficient, will be network dependent and as a result require MSA parameters to be fitted to specific applications. As this heuristic principle is not very appealing it has been suggested to derive methods that take the curvature of the demand function into account. As this principle will make the sequence a function of demand it will be "self-adjusting".

Further research
The paper represents "work in progress" and additional research needs to be carried out along several lines;