Here are the concerns I have put to Professors Oberholzer and Strumpf.

 

What are your results?

 

You claim in your abstract that downloads have a zero impact on sales although they are precisely estimated.

 

This doesn’t seem to be an accurate description of the results. It seems more accurate to say that the results indicate that downloads have a large or very large positive impact on sales. The difference between your description and mine depends on whether each record is given equal weight or not.

 

Your regressions give each record album equal weight in the analysis. I am sure you will agree, upon reflection, that this is not appropriate to answer a question about the overall impact of downloads on sales. If the impact of downloads was expected to be the same for each album it wouldn’t matter, but in table 13 the impact of downloads is clearly different (and higher) for popular albums than for less popular albums. Since popular albums represent the large majority of total sales (85% by my count), giving equal weight to all albums causes you to come to erroneous conclusions regarding the overall impact of downloads. I discus that a few paragraphs down.

 

Before we get to that, however, let us start with the results in Table 11.

 

I think you pass over this table rather too quickly. I think this table is telling you something you might not want to hear. Kennedy’s econometric text (rather wisely, I think) says you are lucky when you get a wrong sign on a variable since it alerts you to problems with your statistical analysis. In this case, you are getting a result that is certainly unreasonable. Although you don’t say much about these results, your only concern seems to be your claim that the instruments aren’t particularly strong. Since they are both significant, however, I am not even sure upon what basis you say that they are less precise than those of table 12. It can’t be the low R-squareds or t-tests of the instruments since there seems to be very little difference between Tables 11 and the results you are more happy with in Table 12.

 

In table 11, the coefficient is significantly positive, at 1.467. This means that each download increased album sales by 1467 units (since sales are measured in thousands). Since you state (page 22) that each of measured download represents 71,000 downloads in the US, this extrapolates into each download increasing album sales by .021 units.  Thus each 48 downloads would translate into one additional sale of an album. With the estimate (page 22, citing Crupnick) of .8 billion US downloads per month, or 9.6 billion per year, this would translate into additional sales of 198 million albums that were caused by downloading. These estimates are for 2002. Footnote 11 tell us that the number of downloads increased by one third since the study was completed. This implies a larger increase in sales for 2003. Since total US sales of albums are about 750 million in 2003, this would seem to suggest a problem with the estimates since it implies that were it not for downloads, sales would be less than 500 million. In other words, sales of albums would have dropped by over 50% since 1999 for no apparent reason except for the positive impact of downloads. This is rather difficult to accept as being even remotely plausible.

 

Yet I would have thought that stronger alarms would have gone off because the instruments were supposed to control for simultaneity, which biases the coefficient on downloads upward, yet this coefficient increased when you used your instruments. It seems to me that your instruments are not controlling for simultaneity at all. Which might also explain how you can generate a result that is almost absurdly unrealistic. And this result doesn’t account for the possibly stronger positive impact on high selling albums that might make the estimate even higher.

 

In Table 12 you have your two preferred models with coefficients close to zero, -.014 and +.088. The -.014 implies that it would take 5000 downloads to decrease sales by 1 album, as you report.  The “preferred” measurement (.088) would translate into a gain due to downloading of 12 million albums, which is fairly small.

 

But this is too low an estimate because Table 13 tells us that the impact on sales due to downloading is much larger and more positive for popular albums. In table 13 albums are broken into quartiles, with a monotonic increase in the coefficient going from the least to most popular quartile. The quartile of largest sellers has a coefficient of .468, whereas the quartile with the second largest sellers has a coefficient of .084. One wonders if the data were broken down into 8 or 10 segments, what the coefficient of the largest segment might be [I think you should examine this]. At any rate, a coefficient of .468 implies that every 150 downloads leads to a sale of a CD. Plugging in the numbers, this would be an increase of 63 million albums due to downloading. Of course, I am applying the results from the top quartile to the entire market. We know from Table 9, however, that the top quartile represents 85% of sales. And downloading has a larger impact every year. Downloading, therefore, if we were to believe these numbers, is responsible for increasing album sales by 10% or more.

 

Although this result cannot be dismissed outright, it is not the ‘zero’ impact that you have used to describe your results. And it too throws off, in my mind at least, warning signs that something isn’t right with your analysis. Again, I suspect that your instruments are not doing the job. But I have other concerns with the analysis as well that have nothing to do with controlling for simultaneity.

 

 

Overarching problem: Fallacy of Composition

 

By using records as the object of analysis (as opposed to, say, genres or the entire industry)  I think you run the danger of being unable to answer the question at all.

 

To make my concern as simple as possible, assume there is no simultaneity between downloads and sales. Assume the regression result found in column 1 of table 11 or 12 were believable. You have a positive coefficient on downloads meaning that recordings with large numbers of downloads also have large sales. Assert causality if you wish – increases in downloads increase the sales of those records being downloaded. Would this demonstrate that overall sales increase when downloads increase? I don’t think so.

 

What this result (the positive coefficient) would demonstrate, if it were true, is that individual recordings benefit from downloading. But that doesn’t necessarily translate into the entire industry benefiting from downloading.

 

Let say that downloading does increase the sales of a record. Perhaps downloading creates a buzz  for record x and people buying records buy those records with buzz. Or perhaps it allows consumers to sample records and consumers get to find out that they prefer record x to record y. Thus heavy downloading makes record x more successful than record y. The coefficient on downloads would be positive in a regression on record sales. I would suggest that your positive coefficient could be entirely consistent with record sales being severely harmed by downloads, or being strongly benefited, and thus wouldn’t answer the question about the overall impact of downloading.

 

Think of advertising as an analogy: If advertising expenditures were used to explain record sales, we would also expect to find a positive coefficient using records as the object of the analysis. But that wouldn’t mean that advertising increased overall record sales. The impact across brands would almost surely be more positive than the impact over the entire industry. We know, from the television cigarette-advertising ban in 1971 and the studies that followed, that advertising did not increase overall cigarette sales. Similarly, political advertising clearly increases the market share of individual candidates, but it need not, and probably does not, increase the total voter turnout. Yet a cross section regression would show that when candidates increase their spending they get more votes (or higher poll numbers).

 

I have a recent paper that looks at the issue of the impact of radio broadcast on record sales. Although radio play obviously has a positive impact on market share of individual records, its impact on record sales appears to be zero or negative.

 

The results from a cross section of the type that you perform cannot, in my opinion, be generalized into any conclusion about the impact of downloads on overall record sales. I think you can fix this problem by converting cross section information into impacts on time series sales.

 

 

 

Secondary Issues: Your Instruments

 

In Table 11 your instruments are different for each album. In Table 12, your instruments only have 17 observations, one for each week. That doesn’t seem like much information on which to explain the downloading behavior toward 670 albums. I am particularly concerned about the German school holiday variable. To start, I am surprised that the coefficient was even positive. I looked at German school holidays and I see that there are usually 12 days in October plus the typical Christmas holiday. Yet according to Table 3, October is when downloads were lowest. Is there something else going on here? Would you mind providing data on the number of German kids on vacation for each of your 17 weeks?

 

How important are the files of German school children to American downloaders? We really do not know. You only provide data on the total files of Germans used by Americans. 

 

You suggest that these files are more available to Americans when German school children are on vacation. That is because you assume that German school kids leave their computers on when they are at home. But there are two problems with this story. First, Germany is 7 or so hours ahead of most American time zones, so Americans are sleeping while German school kids are at school. When the German highschooler wakes up at 10am on his vacation days, even the most resilient American night-owls are heading off to bed. The availability of these files doesn’t seem terribly relevant to Americans when they are asleep. Perhaps you would then suggest that Americans download files while sleeping. But then German schoolkids could leave their computers on while at school. There are numerous possibilities..

 

Plus, German university students, when they come home for vacation (if they stay at the university) are likely to not be making their files available since they are not using their computers, which presumably stay at school with no reason to be turned on (after all, you assume the high school kids turn off their computers when not in use). So this is a countervailing influence, and we don’t know which is stronger. Also, one would think that the university students’ computers at school were more likely to have high speed connections making them more influential than their numbers. Since your time sample includes the Christmas holidays going home for the holidays is a real possibility. Also, I thought most European countries charge by the minute for local calls, so that German kids wouldn’t be expected to make their files available at home for hours on end anyway.

 

I also thought that the way these peer-to-peer systems work, if some of the computers disappear, new computers take their place in the network. In other words, an American user might have contact with 4000 or so nodes, and if some of these nodes close down, computers further on the fringe become part of his available network. That is how the workings of these networks has been explained to me (see the papers by Krishnan et. al.).

 

Staying with the German kids for one more moment, and the problems with this instrument, you assume that a decrease in files from German kids shifts the supply curve for downloaders, making downloads more difficult. But the supply of MP3s is not like the supply in ordinary markets. Markets occur when goods are scarce, which is why the price is positive. The supply of MP3s might be sufficiently large that there is no scarcity for most files. In that case the impact of files available from German school children may not impact downloads any more than everyone in Germany taking a deep breath would impact the air available for breathing in the US. In other words, there may well be excess capacity in the system and removing German schoolchildren from the system may have no impact. You do get a positive coefficient on the German holiday variable, but that may just be due to the fact that some holidays are common between the countries, and Christmas is a big holiday in both countries.

 

 

I have less to say about your congestion variables. Since the congestion variable is measured weekly, how much variation during the week does it hide? I do wonder, whether Internet usage is like phone usage – heaviest during the day when businesses are in operation. I would have thought that music downloading was likely to take place at home during off-peak usage. If so, the congestion variables impacted by the daytime activity (in Europe and the US) might not have much meaning for US downloading at all (when Europe is asleep and American businesses are closed). Another concern comes from the fact that you say that peer-to-peer networks use up 25% of all Internet bandwidth. If peer-to-peer does occur during the day, then is it possible that, when the Internet is congested it is due to a spike in peer-to-peer file sharing? There seems to be a potential simultaneity problem with this variable, which is an instrument being used to solve a simultaneity problem with another variable.

 

Suggestion for future research:

 

Your data should allow you to calculate the ratio of downloads/sales for each genre of music [you would need to take the albums in “new” and “current” and have them categorized in their musical genre, however]. Your current tables provide some evidence that ‘hard’ and ‘alternative’ are much more heavily downloaded than other categories. This ratio might differ across genres because different genres have different audiences that participate in downloading to different degrees. The problem, of course, is that there aren’t as many genres as we would like. We can take those ratios and see how record sales have fared in the last five years for each genre. If genres with the highest download intensity grow at the same rate as other genres, then the evidence supports a claim of no harm. If they grow more slowly, then the claim of harm is supported. Plus, we would get point estimates of the impact of downloading on sales, if there is harm, although the metric would be a bit unusual.

Since these genres might have different secular growth rates, we can use Arbitron [or Nielsen] data (which provides detailed audience size by genre for each of the last few years) as a control for the growth of genres.

This test is simple and clean. Unfortunately, the number of data points will be small, so it might not be as informative as we might like.

If you are not interested in doing it, I would hope you would send me your data so that I could do it.

 

 

Finally, one of the reasons I find your results hard to believe is that they go against so much other evidence. You were quoted in the Times as saying you expected to find a negative impact (of course the Times made several factual errors in discussing my work, but I presume that quote was correct). There are good reasons for that belief.

 

There is a lot of evidence to think that harm is occurring. Peer-to-peer file sharing and CD burners started in the late 90s. The decline in sales follows quite closely. Economic theory would suggest that it is unlikely that a consumer would purchase a song if he already has a perfectly functioning copy. The sampling hypothesis (which does not imply an increase in sales, by the way) can only hold if consumers are extremely honest or if they are too lazy to burn a CD or download the multiple files required to duplicate a CD, and prefer instead to drive to the store to purchase a CD after listening on their computer to some of the music from the CD. This might describe some downloaders, but it is hard to believe it describes all downloaders.

 

The decline that has occurred in CD sales is unusually large. It has occurred in most European countries as well as the US. The file sharing hypothesis fits the events well. There is no obvious explanation besides file sharing to explain this large change in CD purchasing behavior. When our statistics provide results that do not seem right we need to look particularly carefully at what we have done, as I am sure you would agree.