Is the con gone from Econometrics?

[BWV]

Is the "con" gone from Econometrics?

(spillover from the thread on the Easterly piece)

Ed Leamer wrote a famous piece in the early 80s decrying the ad hoc BS nature of most econometric research entitled "Lets take the Con out of Econometrics"

Focusing on the poorly constructed and contradictory findings in the 70s over the relationship between crime and capital punishment, he substantiated the quip that "doing econometrics is like trying to learn the laws of electricity by playing the radio"


https://management.ucsd.edu/faculty/directory/valkanov/classes/mfe/docs/Leamer_1983.pdf


Last year, Angrist and Pishke, a couple of econometricians at MIT wrote a paper discussing how better research design was "taking the con from econometrics". Their focus was primarily on microeconomic studies and the paper expressed frustration with the current failed state of quantitative macroeconomics but provided some rays of hope

http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1639809


Ed Leamer wrote in response that randomization was not sufficient and their hopes of a better empirical and quantitative macroeconomics was illusionary and at best what this quantitative research was doing was simply satisfying a basic human need to "seek patterns and tell stories"

http://www.anderson.ucla.edu/faculty/edward.leamer/selected_research/Tantalus by Leamer.pdf

We economists trudge relentlessly toward Asymptopia, where data are unlimited
and estimates are consistent, where the laws of large numbers apply perfectly and
where the full intricacies of the economy are completely revealed. But it’s a frustrating
journey, since, no matter how far we travel, Asymptopia remains infifi nitely far
away. Worst of all, when we feel pumped up with our progress, a tectonic shift can
occur, like the Panic of 2008, making it seem as though our long journey has left us
disappointingly close to the State of Complete Ignorance whence we began.

The pointlessness of much of our daily activity makes us receptive when the
Priests of our tribe ring the bells and announce a shortened path to Asymptopia.
(Remember the Cowles Foundation offering asymptotic properties of simultaneous
equations estimates and structural parameters?) We may listen, but we don’t hear,
when the Priests warn that the new direction is only for those with Faith, those
with complete belief in the Assumptions of the Path. It often takes years down the
Path, but sooner or later, someone articulates the concerns that gnaw away in each of us and asks if the Assumptions are valid. (T. C. Liu (1960) and Christopher Sims
(1980) were the ones who proclaimed that the Cowles Emperor had no clothes.)
Small seeds of doubt in each of us inevitably turn to despair and we abandon that
direction and seek another.

and concludes

Ignorance is a formidable foe, and to have hope of even modest victories, we
economists need to use every resource and every weapon we can muster, including
thought experiments (theory), and the analysis of data from nonexperiments,
accidental experiments, and designed experiments. We should be celebrating the
small genuine victories of the economists who use their tools most effectively, and
we should dial back our adoration of those who can carry the biggest and brightest
and least-understood weapons. We would benefifi t from some serious humility, and
from burning our “Mission Accomplished” banners. It’s never going to happen.

 

[Pyrrhus]

Repost as I think is relevant.

I do not think that multilinear regressions are particulary sophisticated tools and of course they are always flexible enough to serve the biases of whoever is using them. Page 22 of Easterly's paper discusses the well recognized problems with partial correlations of equally plausible variables along with bivariate and multivariate regressions on several factors. I think he would agree that that part is hardly the centerpiece of the paper as some economist with a contrary view could construct a different model that fit their biases. You cannot control for every variable, nor even suppose that all the important ones are measurable. Can you name one important controversy in economics that has been resolved through the use of econometric methods? I am not aware of any.

"Classical" Regression models are only a type of method used in econometrics, that's of course without mention the different extensions done by econometricians to handle different types of data (cross sectional, time series, repeated observations...). Other econometric advancements include Mixture logit, Cointegration, Tobit, GARCH, hybrid choice and others. Regardless of the mathematical sophistication, and its flexibility to handle different types of data generating processes such as random parameters, or corner solutions. The philosophy in statistical modeling is always parsimony. This of course can be tested for nested and nonnested models as well. Thus, I do take offend of you calling years of advancements in statistical theory... crude statistical tools.

Of course, Econometric modeling without an underlying theory is futile, and it requires validation with the data, and thus data must be available to cross-validate such estimates. However, models based on economic theory have solid grounds as the assumptions may be corroborated, as well as the expectations of the researchers with regards to the direction of the effects of the variables. This does not mean that such models do not undergo cross-validation, and thus new data is acquired were the predictions of the model can be tested vs. observed choices.

You mention that a problem with the econometric modeling is the omitted variables bias. This is but one problem (others include state dependence, feedback, endogenous sample design, heteroskedasticity...), yes. However, there are MANY methods developed that may reduce the bias that range from econometric "patches" such as instrumental variables, proxy variables to experimental designs like those advocated by experimental economics. Furthermore, again the importance is the magnitude of the bias how much effect does the variable inclusion has on the estimates and that the results will be spurious at best.

Finally, I don't think is useful to call econometric development gibberish, and just write it off as well as its results. This is not only pedantic, but it undermines years of theoretical and empirical research on economics. You need to understand that a lot of the development on econometrics has been done to deal with such types of problems, and with many proving successful to warrant significant consideration. Thus, once the econometric advancement is done, it disseminates and new papers will incorporate the changes and test on previous data to reverify previous results already published (a requirement of many economic journals is that the authors provide data, and any statistical routine they programmed because it might not be available along with the paper), and also use on new data to offer more flexibility in the model.

 

[Pyrrhus]

As I've mentioned before, Doing Econometric modeling without an underlying economic theory is for the most part futile as there are no grounds to understand why a functional form or why a specific variable should have any effect on the study of interest.

Now, what you refer and others have referred as "Con" in Economics is just part of the research process. In economics, it is always a debate to accept the results of one empirical study. It takes SEVERAL studies with similar findings to accept a result.

An example in Transportation Economics is the Value of Travel Time Savings. For years since the 1960s, economists have been trying to link theoretically and quantify empirical the marginal rate of substitution of travelers to travel time, which may change by time of day, purpose of trip, income level, and other factors. There was a junction in 1970s, where MANY econometric studies were presenting very different results, and the variation among estimates was significant. However, there was no theoretical basis for such estimates. This lead to the development of the Time allocation models, where the value of travel time savings could be understood by reformulating the consumer problem by adding constraints related to a Time budget, and minimum consumption time per activity. Both of which have also been studied and questioned before.

Furthermore, there's a reason economic journals have required for years now that empirical studies must include DATASETs, and also any routine coded by the analyst if its not widely available. Thus, other economist can download those datasets, and perform their own analysis and see if the results hold water so to speak.

 

[WhoWee]

(spillover from the thread on the Easterly piece)

Ed Leamer wrote a famous piece in the early 80s decrying the ad hoc BS nature of most econometric research entitled "Lets take the Con out of Econometrics"

Focusing on the poorly constructed and contradictory findings in the 70s over the relationship between crime and capital punishment, he substantiated the quip that "doing econometrics is like trying to learn the laws of electricity by playing the radio"


https://management.ucsd.edu/faculty/directory/valkanov/classes/mfe/docs/Leamer_1983.pdf


Last year, Angrist and Pishke, a couple of econometricians at MIT wrote a paper discussing how better research design was "taking the con from econometrics". Their focus was primarily on microeconomic studies and the paper expressed frustration with the current failed state of quantitative macroeconomics but provided some rays of hope

http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1639809


Ed Leamer wrote in response that randomization was not sufficient and their hopes of a better empirical and quantitative macroeconomics was illusionary and at best what this quantitative research was doing was simply satisfying a basic human need to "seek patterns and tell stories"

http://www.anderson.ucla.edu/faculty/edward.leamer/selected_research/Tantalus by Leamer.pdf

and concludes

We had a similar problem with the other thread - what is the focus of this thread - do you want to discuss the articles - or do you have something else in mind?

 

[BWV]

As I've mentioned before, Doing Econometric modeling without an underlying economic theory is for the most part futile as there are no grounds to understand why a functional form or why a specific variable should have any effect on the study of interest.

Now, what you refer and others have referred as "Con" in Economics is just part of the research process. In economics, it is always a debate to accept the results of one empirical study. It takes SEVERAL studies with similar findings to accept a result.

An example in Transportation Economics is the Value of Travel Time Savings. For years since the 1960s, economists have been trying to link theoretically and quantify empirical the marginal rate of substitution of travelers to travel time, which may change by time of day, purpose of trip, income level, and other factors. There was a junction in 1970s, where MANY econometric studies were presenting very different results, and the variation among estimates was significant. However, there was no theoretical basis for such estimates. This lead to the development of the Time allocation models, where the value of travel time savings could be understood by reformulating the consumer problem by adding constraints related to a Time budget, and minimum consumption time per activity. Both of which have also been studied and questioned before.

Furthermore, there's a reason economic journals have required for years now that empirical studies must include DATASETs, and also any routine coded by the analyst if its not widely available. Thus, other economist can download those datasets, and perform their own analysis and see if the results hold water so to speak.

It notable that the examples you provide are closer to operations management issues, where quantitative techniques have tended to work very well and not from macroeconomics or the financial economics. Paul Samuelson launched the mathematics arms race in the 1950s and the results have been mixed, at best, in regards to macroeconomics and financial economics. None of the ideological differences that existed 50 years ago have been resolved by better quantitative tools or data analysis

 

[Pyrrhus]

My examples are from my area of applied research which is transportation economics.

Not sure what you mean by "ideological differences". There's mainstream econonics and fringe economics as far as I know.

Better quantitative tools have been paramount in understanding the forms of the data generating process, and thus computational advances have allowed for econometric models that were known but not used in the past because of tractability to be used today!

 

[BWV]

My examples are from my area of applied research which is transportation economics.

Not sure what you mean by "ideological differences". There's mainstream econonics and fringe economics as far as I know.

Better quantitative tools have been paramount in understanding the forms of the data generating process, and thus computational advances have allowed for econometric models that were known but not used in the past because of tractability to be used today!

Right, so you are in a discipline that has lent itself very well to sophisticated quantitative tools. Other areas of economics have had less successby ideological differences I mean the main schools of thought - Chicago, Keynesian etc

 

[Pyrrhus]

Right, so you are in a discipline that has lent itself very well to sophisticated quantitative tools. Other areas of economics have had less successby ideological differences I mean the main schools of thought - Chicago, Keynesian etc

Which areas do you refer to? Criminal Behavior? Macroeconometrics?...

I've already talked about the problems with Macroeconomics. I think the most important are the representative agent assumption, and the disconnection with Microeconomics. How can you explain aggregate outcomes of the interaction of economic agents while ignoring microeconomics, which basically deals with such agent behaviors.

In terms of Macroeconometrics, the advances has been set on analyzing time series data, and the formulation of models such as GARCH. These are models popular in financial analyses as well.

Yes, the Saltwater vs. Freshwater in Mainstream economics, I suppose. As for the most part instruction of mainstream economics is not that different across universities.

 

[skilgannonau]

We had a similar problem with the other thread - what is the focus of this thread - do you want to discuss the articles - or do you have something else in mind?

I agree. It seems to me half the threads on this board are created by people with an axe to grind about one issue or another.

It notable that the examples you provide are closer to operations management issues, where quantitative techniques have tended to work very well and not from macroeconomics or the financial economics. Paul Samuelson launched the mathematics arms race in the 1950s and the results have been mixed, at best, in regards to macroeconomics and financial economics.

What 'mathematics arms race' are you referring to? I didn't realize economics was fighting a metaphorical mathematics war with other disciplines.

None of the ideological differences that existed 50 years ago have been resolved by better quantitative tools or data analysis

BMV you need to distinguish between economists when they are acting as academics and when they act as politicians, lobbyists, and idealogues because the message they send is very different in those roles. The Chicago v Keynesian debate (ended in the 60s and 70s) and the claim of 'ideologoical differences' in general is so cliche. Politicians and mainstream media might have an interest in overplaying the significance of 'ideological differences' but the academic discipline is apolitical. The contentious debates within the discipline are methodological ones eg. about representative agents, sticky prices etc surprise surprise. I can't think of a single discipline that doesn't have debates about theory. By definition you would expect academic disciplines to have these debates otherwise they would be fact. You also keep on referring to macro but the fact is while most of society see macro as the face of economics it's only one part of it. Micro isn't sexy to the politicians or the media but it is arguably more influential yet I don't see anyone ranting about the methodological approach used in micro. In general the only time I see the 'ideological differences' about macro and micro is when the various the ideological camps whether they be Liberterians or left wing Liberals choose to selectively use economic theory for their political agenda.

 

[BWV]

BMV you need to distinguish between economists when they are acting as academics and when they act as politicians, lobbyists, and idealogues because the message they send is very different in those roles. The Chicago v Keynesian debate (ended in the 60s and 70s) and the claim of 'ideologoical differences' in general is so cliche. Politicians and mainstream media might have an interest in overplaying the significance of 'ideological differences' but the academic discipline is apolitical. The contentious debates within the discipline are methodological ones eg. about representative agents, sticky prices etc surprise surprise. I can't think of a single discipline that doesn't have debates about theory. By definition you would expect academic disciplines to have these debates otherwise they would be fact. You also keep on referring to macro but the fact is while most of society see macro as the face of economics it's only one part of it. Micro isn't sexy to the politicians or the media but it is arguably more influential yet I don't see anyone ranting about the methodological approach used in micro. In general the only time I see the 'ideological differences' about macro and micro is when the various the ideological camps whether they be Liberterians or left wing Liberals choose to selectively use economic theory for their political agenda.

Don't know why putting up works by actual economists here for discussion somehow constitutes having an axe to grind.

Econometrics is most commonly associated with time series studies of macro data, not microeconomics so the general direction of this thread should be obvious - even if you did not look at the links - I am not really discussing micro here.

You deny there are not serious ideological differences or camps in macroeconomics, financial economics or development economics? Basically anywhere there is an intersection of public policy and economics you find serious ideological differences. This is no just some arcane methodological debate - actual public policy gets made from this work. An example is how Leamer's original paper referred to the faulty methodology in studies that found a link between a reduction in murder rates and the death penalty and contributed to the Supreme Court reinstating executions in 1976. On the other hand, 35 years later, you have the greater skepticism of statistical methodology expressed by the court on the link between violent video games and actual violence in the recent Brown vs. Entertainment Merchants Association

 

[skilgannonau]

Don't know why putting up works by actual economists here for discussion somehow constitutes having an axe to grind.

Econometrics is most commonly associated with time series studies of macro data, not microeconomics so the general direction of this thread should be obvious - even if you did not look at the links - I am not really discussing micro here.

You deny there are not serious ideological differences or camps in macroeconomics, financial economics or development economics? Basically anywhere there is an intersection of public policy and economics you find serious ideological differences. This is no just some arcane methodological debate - actual public policy gets made from this work. An example is how Leamer's original paper referred to the faulty methodology in studies that found a link between a reduction in murder rates and the death penalty and contributed to the Supreme Court reinstating executions in 1976. On the other hand, 35 years later, you have the greater skepticism of statistical methodology expressed by the court on the link between violent video games and actual violence in the recent Brown vs. Entertainment Merchants Association

Saying econometrics is most commonly associated with time series studies of macro data is like saying mathematics is most commonly associated with calculus. http://en.wikipedia.org/wiki/Econometrics" is a whole field encompassing statistical methods in cross-sectional, time-series, and panel data. Choosing to define econometrics to mean ‘time series studies of macro data’ is disingenuous especially given the Ed Learner article you refer to criticising metrics in your OP doesn’t even refer to time series or macro and even in the quote here you refer crime which is an issue studied in micro, not macro.

Correct me if I’m but none of your articles refer to ideology in economics or ideology dictating economic research. If anything they are as you would say having an ‘arcane methodological debate’. You say you don’t have an axe to grind yet in your OP you mention nothing about ideology in economics but bring it up as a defence to a response by Pyrrhus about methodological matters. You assert

None of the ideological differences that existed 50 years ago have been resolved by better quantitative tools or data analysis

yet you don’t refer to any sources about what these ideological differences are except making references to Chicago v Keynesianism debate which ended decades ago and was a methodological debate when debated by academic economists (i.e. when economists are acting as academics as opposed to politicians and idealogues).

I believe there are methodological differences between economists in any of the fields not just macro, financial econ, or development econ, as you would expect in any academic discipline. I can only presume ideology would influence policy making given policy is inherently a political activity but I think it’s important to make a distinction between economists when they acting as academics and when they are acting as politicians, lobbyists, and idealogues. I don’t know why the Court made the decisions that they did but making assertions about the direct link between econometric methodology and the Courts decision making is I think drawing a long bow. At the end of the day academics can’t control how people use research. Politicians and ideologues will use whatever tools available to them push their agenda whether that be economic theory, statistics, history, psychology, physics, biology, or whatever. Trying to subscribe the acts of judges, policy makers, and politicians to academic economists or to any academics for that matter is ridiculous.

 

[John Creighto]

What 'mathematics arms race' are you referring to? I didn't realize economics was fighting a metaphorical mathematics war with other disciplines.

The term is physics envy. Social sciences like the rigor in mathematics because it helps to legitimize research. However – ad hoc application of mathematical techniques can: obfuscate the point, be misapplied, or lose relevancy by a focusing on an overly contributed and overly minute situations. While math may tell us under what conditions a given methodology is legitimate, it does not tell us if in the real world is close enough to these assumptions to justify the technique.

BMV you need to distinguish between economists when they are acting as academics and when they act as politicians, lobbyists, and ideologues

I agree. Of course the term ideologue is subjective. We all start are reasoning from certain core principles .

because the message they send is very different in those roles.

A good academic will more conservatively state a point when perusing a minor area of research. What makes the research relevant is how widely applicable it is. It is also as equally as valid an academic pursuit to tie to gather various research into a system of ideas but such wide encompassing viewpoints are by definition ideologies and thus can’t be apolitical.

You also keep on referring to macro but the fact is while most of society see macro as the face of economics it's only one part of it.

The face the public (includes undergrads not just lay people) sees is relevant. Karl Marx wrote the predominant ideas of the times are those of the ruling class. I think though this may be too big an idea to get into at this point so I’ll leave it at that.

Micro isn't sexy to the politicians or the media but it is arguably more influential yet I don't see anyone ranting about the methodological approach used in micro.

How about, the weakness of simple supply and demand models, artificially created scarcity, negative externatilities, barriers to entry (patients, regulations economies of scale), vendor lock in, abuse of buying power…. Well, we teach these ideas to undergraduates and some through mass media the wisdom of the market is still held to a large degree sacrosanct . Consequently, my complaint about micro would be about teaching relevance. It is great if people know the theory but if they can’t tie it to real world issues then are people learning about these problems with the theory simply as an after thought.

 

[Pyrrhus]

The term is physics envy. Social sciences like the rigor in mathematics because it helps to legitimize research. However – ad hoc application of mathematical techniques can: obfuscate the point, be misapplied, or lose relevancy by a focusing on an overly contributed and overly minute situations. While math may tell us under what conditions a given methodology is legitimate, it does not tell us if in the real world is close enough to these assumptions to justify the technique.

This comment seems to insinuate that social scientist do not look at the data. In other words, mathematical models of social scientist are not based on data. Furthermore, it says that any social science uses mathematics to imitate natural science research. This is terribly shortsighted (and insulting to social scientists), you do know what is the role of mathematical modeling in science?, and why it is ubiquitous in every line of research. Physics is not the only field where mathematics "works".

A good academic will more conservatively state a point when perusing a minor area of research. What makes the research relevant is how widely applicable it is. It is also as equally as valid an academic pursuit to tie to gather various research into a system of ideas but such wide encompassing viewpoints are by definition ideologies and thus can’t be apolitical.

You need to clear this. It is unreadable.

The face the public (includes undergrads not just lay people) sees is relevant. Karl Marx wrote the predominant ideas of the times are those of the ruling class. I think though this may be too big an idea to get into at this point so I’ll leave it at that.

What is your point?. Obviously, the individuals outside of a field only have a vague notion of the contents of the field. For example, the perception of physics is tied to Newton and Einstein for the most part.

How about, the weakness of simple supply and demand models, artificially created scarcity, negative externatilities, barriers to entry (patients, regulations economies of scale), vendor lock in, abuse of buying power…. Well, we teach these ideas to undergraduates and some through mass media the wisdom of the market is still held to a large degree sacrosanct . Consequently, my complaint about micro would be about teaching relevance. It is great if people know the theory but if they can’t tie it to real world issues then are people learning about these problems with the theory simply as an after thought.

This paragraph does not make any sense.

Economics is not just a "library field" where papers and books are written and simply go inside a bookcase. The ideas of economics are applied in today's world. Examples exist such as demand prediction, cost benefit analyses, auctions... I am not sure what you mean by teaching "relevance". You do know that Economists (as in students graduating from ECONOMICS) apply those ideas, and also may expand those ideas as needed when they are practicing economists. Furthermore, I am not sure what else can be taught to the general public?. You do realize that you must spent years studying economics just like any other profession such as physics. You cannot expect the general public to understand economics by watching a TV show of 1 hour about a certain topic.

 

[John Creighto]

Originally Posted by John Creighto
The term is physics envy. Social sciences like the rigor in mathematics because it helps to legitimize research. However – ad hoc application of mathematical techniques can: obfuscate the point, be misapplied, or lose relevancy by a focusing on an overly contributed and overly minute situations. While math may tell us under what conditions a given methodology is legitimate, it does not tell us if in the real world is close enough to these assumptions to justify the technique.

This comment seems to insinuate that social scientist do not look at the data. In other words, mathematical models of social scientist are not based on data. Furthermore, it says that any social science uses mathematics to imitate natural science research. This is terribly shortsighted (and insulting to social scientists), you do know what is the role of mathematical modeling in science?, and why it is ubiquitous in every line of research. Physics is not the only field where mathematics "works".

What I am saying is that inductive reasoning is limited. A pig may inductively conclude that a farmer isn’t likely to slaughter him. However, in reality it is the case that the farmer isn’t likely to not slaughter the pig. We can not tell from the Data if it is ergodic over the period of time we measure it. We can only look for apparent convergence in our estimates.

I’m not saying it isn’t useful but I think that it is overly valued in social science and especially over valued in humanities. I’ll have more to say on this later as and will also have more to say on the rest of your comments.