Optimizing Model Performance

This is the second of a three-part series on calibration and optimization. The first part can be accessed here. The third part can be accessed here.

Part one of this series explored calibrating a model using the Stella® Architect/Professional optimization feature (introduced in version 1.5). This post will explore using this same feature for single-objective optimization.

The Challenge

Last year, I explored the Shifting the Burden archetype with the example of using credit card debt to pay current expenses. In that case, adding feedback between expenses and an ability to pay allowed the credit card debt to stabilize. What if we instead wanted to maximize our savings and/or minimize our debt? The original model is shown below for reference and can be downloaded here (with the optimization configurations described below).

Most of these parameters cannot be adjusted by the person trying to control their debt. The interest rate and the minimum payment parameters are set by the credit card company. Let’s assume the income cannot easily be adjusted either, for example, by working overtime or taking on a second job (as we did before). That leaves only the spending rate, which we varied last time, and the minimum bank balance. The latter, you may argue, is set by the bank, but different banks and different bank accounts have different minimums…before they impose a charge. The balance can still fall below the level the bank sets.

We will perform two separate optimizations:  maximize savings (i.e., bank balance) and minimize credit card debt. We should not expect them to give the same results.

To start, set up two different Payoff functions in the Payoff tab of the Model Analysis Tools tab (this is described in part 1). Name the first one “Savings” and add Bank Balance to it. Name the second one “Debt” and add Credit Card Debt to it. Leave the weight set at one and the Payoff time range set to Final value for both payoffs.

Move to the optimization tab and name this optimization “Maximize Savings.” Choose the following settings, leaving the remaining parameters at their default values:

• Optimization method:  Differential Evolution (DE)
• Generations:  20
• Population size:  10

Also place a checkmark next to the payoff “Savings.” Scroll down to the Optimization Parameters sections to add the two parameters to vary. First add:

• Variable:  spending rate
• Min value:  1500
• Max value:  5000

Then add:

• Variable:  minimum bank balance
• Min value:  100
• Max value:  1000

Before running the optimization, what do you expect to happen? Press the O-Run button in the run toolbar at the bottom left of the window. The hopefully-not-too-surprising result pops up in the log: The best payoff (remember, the payoff is the Bank Balance at the end of the simulation) is 24,500 when spending rate is 1500 and minimum bank balance is 490. It should be no surprise that the greatest savings occur when the spending rate is the smallest (and below the income of 2000). But what effect did minimum bank balance have? Let’s run it again! Same payoff and spending rate, but now minimum bank balance is 561. Running it three more times, I get 860, 956, and 103 (you will get different values as we did not set the random number seed).

This is incredible! How can we get different values for only one of the parameters, but still get the same payoff? Easy: The payoff is insensitive to the value of minimum bank balance. It makes no difference in the outcome. We don’t need to adjust this parameter to optimize this system (but it gave us something fun to explore).

What about Powell?

Last time, Powell found the solution much quicker than DE. How does it fare this time? To test this, you must first Restore All Devices (press the U-turn button on the run toolbar, or choose it from the Model menu). We must do this because the Powell method (and Stepper) starts from the current set of parameters. Since the parameters have been set to the optimum already, it won’t be much of a test unless we set them back to their original values (which Restore All Devices will do). Then change the Optimization method to Powell, retaining all of the default parameters, and press O-Run again.

Powell, unfortunately, cannot find the solution to this problem. It returns a payoff of only 1000 with a spending rate of 2990. Also, in comparison to the DE solution where Credit Card Debt stayed at zero, Credit Card Debt is now rising uncontrollably, reaching 35,700 at the end of the simulation (you can see this on the diagram). The problem for Powell is that the fitness landscape is flat over most of its region, which is one of the areas where Powell does not perform well (refer to the Stella documentation for its limitations).

The best solutions for this problem are all restricted to low values of spending rate, so another method, Grid, which does a gridded search of the fitness landscape, should work (as DE did). Choose Grid as the Optimization method and press O-Run (we do not have to Restore All Devices because Grid, like DE, ignores the current parameter settings). The Grid method finds the optimal solution, a payoff of 24,500, in 121 runs. This is in contrast to DE’s 200 runs, but I overspecified what DE requires to guarantee it always fully converges. By setting a tolerance, DE should converge in roughly the same number of runs. However, due to the nature of this particular fitness landscape (which has its optimum at the minimum parameter value), I can improve Grid’s performance to only 11 runs by changing its Search type from Exhaustive to Latin Hypercube (note that while DE initializes using Latin Hypercube, it then explores each region).

As you can see, there are a lot of considerations when setting up an optimization. Luckily, it’s easy and not time-consuming to run a few simple experiments to hone in on the best approach. By default, I always start with DE, but that is my bias based on the fact it converges quickly to the global optimum. Once I know where that optimum is, I can find another method that achieves the same result faster and use it to optimize many different scenarios.

Does Minimizing Debt Change Things?

Perhaps our main goal should be to minimize our debt. We will discover, though, that this is not a sufficiently strong condition to give us optimal performance.

To test this, create a new optimization (press the green +) and name it “Minimize Debt.” Choose the following settings, leaving the remaining parameters at their default values:

• Optimization method:  Differential Evolution (DE)
• Direction:  minimize
• Generations:  20
• Population size:  10

Also place a checkmark next to the payoff “Debt.” Scroll down to the Optimization Parameters sections to add the two parameters to vary (these are the same settings as before). First add:

• Variable:  spending rate
• Min value:  1500
• Max value:  5000

Then add:

• Variable:  minimum bank balance
• Min value:  100
• Max value:  1000

Before running the optimization, what do you expect to happen? Scroll up to the payoff selection area and press the “Activate and Run” button beneath it (this activates this set of optimization parameters and then runs the optimization; alternatively, you can turn on the Active checkbox and use O-Run as we have up to now). The payoff is now 0 (recall this is Credit Card Debt) with a spending rate of 1588. It remains insensitive to minimum bank balance, but run it a few more times and you will see something interesting: a different spending rate for each optimization. After 1588, I got 2121, 1530, an 1790. These had corresponding ending Bank Balances of 21,400, 2760, 23,500, and 14,400, none of which is optimal.

As I said above, this minimal debt condition is not sufficiently strong to optimize the performance of the system as whole. Once Credit Card Debt goes to zero, the system cannot detect any improvement between one set of parameters and another: they are all equally good solutions.

It is worth noting that Powell can solve this optimization. Since Powell always maximizes, to test this you will need to change the Debt payoff definition to use a weight of –1 (as shown in part 1 – or create a new payoff with this weight so you can easily compare to DE’s results).

Can We Minimize Debt and Maximize Savings?

This is technically the domain of multiobjective optimization. However, it is possible to combine parameters in a weighted payoff that would allow you to approximate this with a single-objective optimization. In addition, since we know Credit Card Debt goes to zero long before we reach the system optimum, we are optimizing only one objective once we pass that threshold. It is like doing a two-level sort, e.g., by last name and then for identical last names, by first name. However, in this case, we are first optimizing by Debt (sort of – it’s more complicated than this), but once all Debt payoffs reach zero (the same value), we are optimizing by Savings.

I should point out that this is a trivial exercise that will return the same results as when we optimized for Savings. Nevertheless, here is how you would do it (for example, if you didn’t know that optimizing for one specific variable would give the best results).

We need to define a new payoff, so return to the Payoff tab and press the green +. Name this Payoff “Debt then Savings.” First add Credit Card Debt and give it a weight of –2 (minimize debt with twice the weight/important of savings). Then add Bank Balance with the default weight of 1.

Return to the Maximize Savings optimization (saved using DE), but select the payoff “Debt then Savings” rather than “Savings”. If it is not Active, check this box and then press the O-Run button. The best solution has a payoff of 24,500 when spending rate is 1500 – as we expected.

Stay tuned for the next post where we’ll set up a multiobjective optimization using DE.