Some Financial Analysis in Software Engineering

Warning: Lots of navel-gazing in this post.

Example of Financial Analysis of Two Methods for Detecting Software Bugs (from Vienneau 1995)

Consider a specific industry, namely the production of software. (This industry would correspond to a column in an input-output matrix.) Many methods have been proposed for improving the productivity, the quality, and management control in software development, operations, and management. And, as software became a more important component in, for example, embedded systems, various measures, and various methods for choosing among possible measures, have been proposed for software processes and products. Some time ago, some researchers argued that analyses of such metrics should be oriented towards economic analysis of, for example, efforts to introduce new methods into the software lifecycle. I think of the following two articles as early exponents of this thesis:

• Sandra A. Slaughter, Donald E. Harter, and Mayuram S. Krishnan. Evaluating the Cost of Software Quality, Communications of the ACM. V. 41, No. 8 (Aug. 1998): pp. 67-73.
• Robert L. Vienneau. The Present Value of Software Maintenance, Journal of Parametrics (April 1995).

Slaughter et al. is more high level and is published in a more prestigious journal. My article is more a tutorial.

One can apply economics to analyze specific decisions during the software lifecycle. For instance, the following article looks at when testing should stop and software should be released:

• Robert L. Vienneau. The Cost of Testing Software, Proceedings of the Annual Reliability and Maintainability Symposium, Orlando, FL (29-31 Jan. 1991).

The optimal release date can be justified by a simple marginal rule: the final phase of testing should stop, and software should be released, when the cost of an increment of testing reaches or exceeds the marginal benefit to gained from that increment of testing. The marginal benefit can be conceptualized as the product of the expected number of bugs to be observed during that increment of time and the savings in finding those bugs during test, instead of operations. (Various software reliability models exist for estimating the parameters needed to implement this rule.)

I have not been aggressive in promoting my work in software engineering economics. Once in a while, I search the web to see who, if anybody, is referencing this stuff. This post concludes with a number of papers, theses, and selected technical reports citing one or the other of the above papers, or related work from me. I do not know why so many of these references are so recent; perhaps software processes must attain a certain level of maturity before these financial analyses become useful.

• Majed Alyahya, Rodina Ahmad, and Sai Peck Lee. Impact of CMMI-Based Process Maturity Levels on Effort, Productivity and Diseconomy of Scale, International Arab Journal of Information Technology. V. 9, No. 4 (Jul. 2012): pp. 352-360
• Hanna Farnstrand. Introducing Structured Testing, Lund University (16 Jun. 2008).
• Matt Ganis. Agile Methods: Fact or Fiction, Trenton Computer Festival. (Apr. 2010).
• Warren Harrison, David Raffo, John Settle, Nancy Eickelmann. Technology Review: Adapting Financial Measures: Making a Business Case for Software Process Improvement, Software Quality Journal. V. 8, No. 3 (1999): pp. 211-231.
• Chin-Yu Huang, Jung-Hua Lo, Sy-Yen Kuo, and Michael R. Lyu. Optimal Allocation of Testing-Resource Considering Cost, Reliability, and Testing Effort, Proceedings of the 10th IEEE Pacific Rim International Symposium on Dependable Computing (PRDC'04) (2004): pp. 103-112.
• Chin-Yu Huang and Jung-Hua Lo. Optimal Resource Allocation for Cost and Reliability of Modular Software Systems in the Testing Phase, Journal of Systems and Software. V. 79, Iss. 5 (May 2006): pp. 653-664.
• Zhengrui Jiang, Sumit Sarkar, and Varghese S. Jacob. Postrelease Testing and Software Release Policy for Enterprise-Level Systems, Information Systems Research. V. 23, No. 4 (Dec. 2012).
• J. Dennis Lawrence. Software Reliability and Safety in Nuclear Reactor Protection Systems, Lawrence Livermore National Laboratory, UCRL-ID-114839 (11 Jun. 1993).
• Tuomas Paasonen. Methods for Improving the Maintainability of Application Software. Master Thesis, Aalto University (13 April 2012).
• David Raffo and Tim Menzies. Evaluating the Impact of a New Technology Using Simulation: The Case for Mining Software Repositories. [I am unsure of the status of this paper].
• Tobias Scherner and Lothar Fritsch. Early Evaluation of Security Functionality in Software Projects - some Experience on using the Common Criteria in a Quality Management Process, Johann Wolfgang Goethe - Universitat, Franfurt am Main (2007).
• Leon van Delft. Automated GUI Testing in Industry. Master Thesis, Delft University of Technology (28 Dec. 2011).

Update (28 February 2013): Added Huang and Lo (2006) to remind me to check if my search with Google Scholar yielded an accurate reference.

Update (13 March 2013): Added Huang et al. (2004).

Empirical Bifurcation Diagram For Kaldor Model

Figure 1: An Empirically-Constructed Bifurcation Diagram

I thought I would try to summarize what I have learned about the qualitative behavior of Kaldor's business cycle model. (Although I had some preparation, I learned much of the mathematics in this post during this analysis.) Figure 1, above, shows (mostly) bifurcations found empirically, while Figure 2 below shows bifurcations that can be found analytically. The normalized model is specified by four parameters, the speed of adjustment of national income to the difference between aggregate demand and supply, the depreciation rate of capital, the (average and marginal) propensity to save out of income, and the cost of adjusting the capital stock to the desired level. Figures 1 and 2 are drawn for a constant depreciation rate (δ = 1/5) and a constant cost of adjustment (γ = 3/5). The abscissa in the figures represents different levels of the speed of adjustment (α), while the ordinate represents different levels of the propensity to save (σ).

Figure 2: An Analytically-Constructed Bifurcation Diagram

The Kaldor model has two (endogenously determined) state variables, the normalized level of the capital stock (k_t) and the normalized flow of national income (y_t). A bifurcation analysis determines regions in the parameter space in which the flows for the state variables qualitatively differ. A bifurcation is a manifold in the parameter space in which some such qualitative difference arises. The figures above show selected bifurcations, where each bifurcation is represented by a line or curve in the figure. (I do not claim that the bifurcations shown are the complete set of bifurcations that arises, even in the part of the parameter space shown. For example, I ignore homoclinic bifurcations of points along a limit cycle with the (in)stability of a saddle point.)

The horizontal line near the top of Figure 1 is shown near the bottom of Figure 2. This line represents a pitchfork bifurcation. The model has one fixed point (stationary state), at the origin of the state space, above the line. The model has three fixed points below the line, one at the origin, and the other two symmetrically located around the origin in the first and third quadrants. As I understand it, the origin always has saddle-point (in)stability below this line.

The blue curve in Figure 2 arcing upward to the right from the horizontal line is a Neimark-Sacker bifurcation. (The Neimark-Sacker bifurcation is the discrete time analog to the Hopf bifurcation.) To the left of this curve, the fixed point at the origin is stable. The origin loses its stability at the Niemark-Sacker bifurcation, and it throws out a stable limit cycle to the right. A limit cycle corresponds to a business cycle in the modeled economy.

I think that the bifurcations in the region with three fixed points, below the horizontal line representing the pitchfork bifurcation, are more complicated and more difficult to understand. In the lower left of Figure 1, the two symmetrical fixed points not at the origin are stable. One can find each fixed point's basin of attraction in the state space, that is, those values of the state variables such that a trajectory in the state space started with those values converges to the given fixed point. As I understand it the two basins of attraction cover the entire state space in this region of the parameter space.

Figure 1 shows three (hard to distinguish) curves coming down from the horizontal line and curving to the right. All of these curves intersect the horizontal line at the same point. And that point is also the intersection with that horizontal line of the curve above the horizontal line representing the Neimark-Sacker bifurcation I have previously described. (I have no idea how you would formally prove this.)

The lowest of these curves sloping downward to the right represents a bifurcation in which a limit cycle with saddle point stability appears. Along this curve in the parameter space, a region exists in the state space in which trajectories approach the limit cycle, only to ultimately diverge to one of the fixed points away from the origin.

The next higher curve sloping downward to the right in Figure 1 is almost impossible to tell apart from the curve that I have just described. This curve represents a homoclinic bifurcation. The basins of attraction of the fixed points away from the origin no longer combine to cover the state space. At the homoclinic bifurcation, the stable and unstable sets of the origin merge. Between the lower curve and this curve, the limit cycle with saddle-point stability bifurcates to form (at least) two limit cycles, one stable and one unstable. The unstable limit cycle is the boundary of the union of the basins of attraction of the two stable fixed points. Just above this curve, the border of the basins of attraction bifurcates, to form two disjoint unstable limit cycles. The stable limit cycle remains in the state space, enclosing the fixed point at the origin and these unstable limit cycles.

The highest curve sloping downward to the right in Figure 1 is another Neimark-Sacker bifurcation. The fixed points away from the origin lose their stability at this bifurcation. Their basins of attraction disappear. The unstable limit cycles forming the border of each basin of attraction are absorbed into the corresponding limit point. A stable limit cycle remains. So on the right of Figure 1, both above and below the horizontal line, a stable limit cycle exists, even though the number of fixed points varies with the propensity to save.

I was surprised at the diverse and complex behavior that economists have found in the Kaldor model, a model of the business cycle that is nearly three-quarters of a century old.

Inconsistencies In Keynes's General Theory And In General Equilibrium Theory?

1.0 Introduction

In this post, I raise some questions about how short-period and long-period equilibrium relate. I consider this issue in two frameworks:

• Keynes's General Theory.
• The Arrow-Debreu model of intertemporal equilibrium.

The issue in the first framework is at least partly a question of hermeneutics - can one read Keynes such that his theory is internally consistent? The issue in the second framework is, as I understand it, a question of mathematics.

2.0 Keynes's General Theory

It seems to me, Keynes, in his definition of long-period equilibrium simultaneously postulates that agents have common and divergent expectations. Can one find a consistent long-period equilibrium concept in the General Theory that includes both production and financial markets? I think of Post Keynesians, such as Joan Robinson, as having emphasized the extension of the General Theory to the long period. But I am not sure I can find a good resolution in the literature. I think my question is close to Jan Kregel's (1985) point.

Keynes defines long period equilibrium in Chapter 5, "Expectation as Determining Output and Employment":

"If we suppose a state of expectation to continue for a sufficient length of time for the effect on employment to have worked itself out so completely that there is, broadly speaking, no piece of employment going on which would not have taken place if the new state of expectation had always existed, the steady level of employment thus attained may be called the long-period employment¹ corresponding to that state of expectation. It follows that, although expectation may change so frequently that the actual level of employment has never had time to reach the long-period employment corresponding to the existing state of expectation, nevertheless every state of expectation has its definite corresponding level of long-period employment...

...past expectations, which have not yet worked themselves out, are embodied in the to-day's capital equipment with reference to which the entrepreneur has to make to-day's decisions, and only influence his decisions in so far as they are so embodied. It follows, therefore..., to-day's employment can be correctly described as being governed by to-day's expectations taken in conjunction with to-day's capital equipment.

¹ it is not necessary that the level of long-period employment should be constant, i.e. long-period conditions are not necessarily static. For example, a steady increase in wealth or population may constitute a part of the unchanging expectation. The only condition is that the existing expectations should have been foreseen sufficiently far ahead." -- John Maynard Keyes (1936): pp. 48-50.

Keynes analyzes financial markets in Chapter 12, "The State of Long-Term Expectation". It is in this chapter that he introduces his distinction between speculation and enterprise, as well as likening the markets for shares (stocks) and bonds to a beauty contest in which participants are not trying to pick the prettiest entrant, but rather the entrant who will be thought prettiest by common opinion, when all who are picking the entrant are looking at their choice from this perspective.

My claim is that the Chapter 5 definition requires common expectations, while the chapter 12 analysis presumes a diversity of expectations. In fact, according to Keynes, the momentary stability of financial markets depends on a balance of bulls and bears and, thus, divergent expectations.

3.0 General Equilibrium Theory

In the Arrow-Debreu model, an equilibrium can be considered a path through (logical) time. Some of those paths, in some simplified models, are steady states, in which each industry expands at the same rate of growth. One can read Von Neumann as setting forth the production side of such a model of a stationary state, if one so chooses.

In general, an equilibrium path in the Arrow-Debreu model is a (very) short-period equilibrium. The initial endowment of capital goods is taken as given, and expectations of the agents in the model are pre-reconciled. (Questions, perhaps unanswerable, exist about how such an equilibrium can be achieved.) One can ask about the limit behavior of each equilibrium path, as time increases without bound. Some of these paths might converge to stationary states, and some might diverge.

I associate the Turnpike Theorem with Paul Samuelson. I turn to either Dorfman, Samuelson, and Solow (1958) or, for example, Dixit (1976), when I want to read an exposition of this theorem. As I understand it, this theorem implies that stationary states, generically, have saddle point (in)stability in the Arrow-Debreu model.

I am also aware of the Sonnenschein-Mantel-Debreu theorem. As I understand it, this theorem implies almost any dynamics are possible in the Arrow-Debreu model. Some such dynamics, then, should be consistent with stationary states that are locally stable. Another dynamics in the model should be consistent with stationary states that are locally unstable. And some configuration of parameters should be consistent with multiple equilibrium mixing any combination of stable steady states, unstable steady states, and saddle-point steady states.

My understanding of the implications of the Turnpike and Sonnenschein-Mantel-Debreu theorems seems to be inconsistent. Where do I go wrong?

References

• A. K. Dixit (1976). The Theory of Equilibrium Growth. Oxford University Press.
• Robert Dorfman, Paul A. Samuelson and Robert M. Solow (1958). Linear Programming and Economic Analysis, Dover.
• John Maynard Keynes (1936).The General Theory of Employment, Interest and Money. Harcourt, Brace and Company.
• J. A. Kregel (1985). Hamlet without the Prince: Cambridge Macroeconomics without Money, American Economic Review. V. 75, No. 2 (May): pp. 133-139.
• J. Von Neumann (1945-1946). A Model of General Economic Equilibrium, Review of Economic Studies. V. 13, No. 1: pp. 1-9.

Against "Science", "Reality", And "Free Will"

1.0 Introduction

You may have noticed. I am not overly fond of neoclassical economics. But today I thought I would talk about criticisms you might find in the blogosphere that I find unpersuasive. That is, I do not like certain one-line assertions, without additional elaborations. I make no attempt to demonstrate here that some make these assertions.

2.0 "Economics Is Not Science"

If you are not arguing about the history, philosophy, or sociology of science, why would you care if a particular field is a science? Should you not be more concerned if the arguments in a field tend to be persuasive, if the norms in the field lead to such arguments? I can see a role here for classifying types of assumptions. One can argue about whether economists put forth supposedly substantial theories that cannot be falsified by any logical or empirical findings. Likewise, perhaps some communities of economics are not as quick as they should be to discard empirically falsified theories. Or one could ask if whatever laws are supposed to be embedded in economic models are restricted to certain institutional and historical instances of capitalism. I hope in putting forth criticisms along these lines, I try to provide concrete examples, not just abstract claims.

3.0 "Economic Theory Does Not Correspond To Reality"

What does use of the word "reality" add to an argument about the persuasiveness or non-persuasiveness of a certain set of doctrines? (I do not mean here to downgrade Tony Lawson's research into ontology and economics. In particular, I do not have a problem with the idea that economic systems are invariably open systems. I guess this idea is in tension with my simultaneous interest in natural experiments.)

4.0 "People Have Free-Will; Thus, Economies Cannot Be Modeled With Mathematics"

I am also not fond of the claim that, since people have free will, one cannot apply mathematics to economics. First, I think at least some applications of mathematics in economics are about algorithms and accounting conventions. I do not see how ideas about consumer choice are relevant to much of this work. Second, I tend to think of the distinction between free will versus determinism as one of those tired dualisms that the linguistic turn in philosophy should have dissolved. I usually cite work drawing on Ludwig Wittgenstein for this sort of point. But let me mention J. L. Austin's "A Plea For Excuses" as being directly relevant for an analysis of when an action is voluntary and of when an agent is responsible or blameworthy for what they do. Austin argues for distinctions that you might not initially see. Third, I agree that describing agents as if they calculate how to obtain a maximum utility curve, given preferences and constraints does not leave room for genuine individual choice. One might try to problematize individual choice and seek more sophisticated models. Such an approach does not necessitate the rejection of mathematics.

Distinctive Elements of Keynes' General Theory

I suggest the following are distinctive features¹ of Keynes' General Theory:

• An investment function, independent of savings (as in these examples).
• An argument for the possible existence of an equilibrium with widespread unemployment.
• A setting in historical time, not logical time.
• A claim that interest rates are to be explained by monetary and financial markets, not by causes in the "real" economy.

Paul Davidson rightly notes that Keynes strove to make his theory compatible with any degree of monopoly. This differs from Michal Kalecki, who just assumed the degree of monopoly empirically existing. But Kalecki and Keynes had different objectives; Kalecki was not as interested in internal criticism of existing neoclassical theory. Nevertheless, I think Keynes' assumption of an independent investment function may have reflected existing corporate structures. In particular, Keynes was aware of the separation of ownership from control in joint stock companies. This awareness has something to do with what Keynes meant by the "socialization of investment." Maybe it also facilitated his awareness of the possibility of investment decisions as an independent driving force for the economy.

Keynes offers a theory of a (Marshallian) short run equilibrium. For most of the book, capacity is given. Keynes analysis addresses the question of what level that capacity will be operated at. (In chapter 5, he does offer a theory of a long run equilibrium, in which the expectations that guided the installation of the present array of capital equipment have persisted since their production and are currently being satisfied.) In offering a theory of equilibrium, he needs to answer why the multiplier process does not create a cumulative process that goes on unceasingly. Somewhere I have read that Keynes' ability to answer this question distinguishes him from some potential precursors.

The distinction between historical time and logical time includes the distinction between uncertainty and risk. If I recall correctly, Hyman Minsky describes Keynes setting as having a business cycle background.

I am deliberately not putting forth the liquidity preference theory of interest as a distinctive feature. Keynes needed liquidity preference to explain why interest was a monetary phenomenon, not a matter of bringing the real economy into equilibrium. And the theory certainly generated lots of discussion. But once you see that interest rates are a monetary feature, must you retain that specific theory? Can I not think of the long run interest rate as what emerges from a succession of short runs, where the interest rate emerges in those short runs from speculators disagreeing over their views of the long run? In others words, cannot one accept interest rates as some mixture of being held up by its own bootstraps and of conventions, perhaps supported by the monetary authority?

1. I do not here make any claim about originality. This post was inspired by reading Daniel Kuehn and Jonathan Catalan noodling over the question of what was original in The General Theory.