ESTIMATING NEW PRODUCT DEMAND FROM BIASED SURVEY DATA Roger Klein and Robert Sherman Bellcore Abstract Market researchers often conduct surveys asking respondents to estimate their future demand for new products. However, projected demand may exhibit systematic bias. For example, the more respondents like a product, the more they may exaggerate their demand. We found evidence of such exaggeration in a recent survey of demand for a potential new video product. In this paper, we develop a computationally tractable procedure that corrects for a general form of systematic bias in demand projections. This general form is characterized by a monotonic transformation of projected demand, and covers exaggeration bias as a special case. 1 1. Introduction Many popular econometric models have the form Λ(Y ) = X 0 β0 + u (1) where Y is a response variable, X is a vector of predictor variables, β0 is a vector of unknown parameters, u is an error term, and Λ is a monotonic transformation. The Box-Cox model (Box and Cox, 1964) is a famous example, where Λ is known up to a single real-valued parameter and u is normally distributed with mean zero and unknown variance. All unknown parameters are estimated using maximum likelihood. Horowitz (1992) presents a kernel-based method for estimating both Λ and the distribution function of u without making √ any parametric assumptions about their functional forms. His estimators are nconsistent, converge to Gaussian processes, and can be used to estimate the quantiles of Y given X. In this paper, we consider the following model: Λ(Y ) = (X 0 β0 + u){X 0 β0 + u > c} (2) where the constant c is known to equal either 0 or −∞. This model allows a monotonic function of Y to equal a censored regression when c = 0, and reduces to (1) when c = −∞. We will argue that (2) with c = 0 is a useful model for new product demand based on survey data. In this context, Y denotes reported demand and Λ(Y ) denotes actual future demand. The inverse function, Λ−1 , may be interpreted as a reporting function mapping actual demand into reported demand, Y . We develop a procedure for estimating β0 , the variance of u, and Λ without making any parametric assumptions about the functional form of Λ. However, we assume that the distribution of u is known up to scale. We obtain √ n-consistent estimates of the model parameters and show that the estimate of √ Λ converges to a Gaussian process at a n rate. These estimates can be used to produce reliable estimates of new product demand. While our procedure places heavier demands on u than the kernel-based method of Horowitz, it places lighter demands on Λ, and the latter fact is crucial for the application we consider. The kernel estimators are based on a representation of Λ that holds only if each distribution function of Y (given X 0 β0 ) is differentiable (Horowitz, p. 5). Further, each distribution function must have at least √ 3 derivatives for the estimator of Λ to control asymptotic bias and so achieve n-consistency (Horowitz, pp. 10–11). This is needed even if the distribution of u is known. In our application, the response variable only takes on nonnegative integer values. Consequently, each conditional distribution function of Y is a step function and so not even √ continuous. Our procedure covers applications like this without sacrificing n-consistency. We also note that the kernel-based estimator only applies when c = −∞ (Horowitz, p. 21). For our application, we need a procedure that covers the case c = 0. The 2 procedure we develop covers this case and generalizes immediately to cover the case c = −∞. Finally, we note that kernel-based methods are much more computationally intensive than the procedure proposed here. In order to motivate the use of model (2), we consider a recent survey of demand for a potential new video product. Respondents are asked to estimate the average number of times per month they would use the product if it were offered, and there is a charge for each use. They report nonnegative integer values. Figure 1 gives a histogram of the responses. Because of the proprietary nature of the data, reported quantities have been masked by dividing them by their median value. From this figure, one would suspect that individuals reporting high levels of demand are exaggerating.1 For example, 7% of those surveyed reported a level of demand more than three times the median level. Moving out in the tail of this distribution, there are people who report a level of demand exceeding median demand by factors of 6, 10, 13, and even 20. Given the nature of this particular new product, such levels of demand are highly suspect. Though it is not possible to discern from the masked data, median and lower levels of demand agree with what one might expect for the type of product being surveyed. We also note that nearly 20% of the respondents reported zero demand for the product. Write Y for projected demand and Q for actual future demand (quantity). From the description of the data just given, it seems reasonable to assume that there exists a function Λ defined on [0, ∞) and satisfying the following conditions: (1) Λ(Y ) = Q where Λ is strictly increasing. (2) Λ(0) = 0 and Λ(s) = s for some known, positive number s. Assumption (1) requires that large projected demands correspond to large actual demands, but imposes no further structure on Λ. By freeing Λ from parametric restrictions like those imposed by the Box-Cox formulation, we allow for different reporting regimes: Y < Λ1 (Y ) , Y ∈ A1 : Underreporting Y = Λ2 (Y ) , Y ∈ A2 : Accurate Reporting Λ(Y ) ≡ Y > Λ3 (Y ) , Y ∈ A3 : Overreporting The regime regions Ai must partition the real line, but need not be known or contiguous. In addition, Λ need not be continuous, even at transition points from one regime to another. This type of freedom in modeling Λ is desirable in 1 Exaggeration is common in surveys on demand for new products. This phenomenon may be due to new product enthusiasm, an attempt to influence the decision to market the product, a desire to please the interviewer, or the tendency for people to be less sensitive to total costs in a survey than they would be if they were making actual purchases. There may be other plausible explanations. For whatever reason or combination of reasons, the tendency to exaggerate is an acknowledged problem in surveys of this type. 3 the context of our data: we are skeptical about the magnitude of high reported quantities but are reluctant to impose the form of the relationship between Y and Λ(Y ), beyond requiring that the relationship be monotonic. Note, however, that we do not assume that misreporting exists: Λ(Y ) = Y satisfies assumptions (1) and (2) given above. Without assumption (2), Λ can only be identified up to location and scale. For full identification, we require that Λ be known a priori at two distinct nonnegative points. Zero is a natural point to choose in the context of new product demand: Λ(0) = 0 says that people who project zero demand will, in fact, not use the product in the future. The assumption Λ(s) = s for s known and positive is a key identifying condition. It says that there is a known, nonzero “safety point” at which demand is accurately reported. It is instructive to contrast the method of Horowitz with ours regarding assumptions needed to identify Λ(Y ). Both methods require location and scale assumptions to identify Λ(Y ). Horowitz (pp. 10–11) assumes that Λ(Y ) is known at one point and that one of the slope parameters in model (1) is known up to sign. For purposes of making inferences on Y , which is natural in the context of actual market data, for both methods, location and scale assumptions serve merely as convenient normalizations. Indeed, any location and scale normalizations would suffice. However, in the application we consider in this paper, we are not directly interested in reported demand, Y , but rather in actual demand, Λ(Y ). Assumption (2) is substantive in this context. Accordingly, we have developed graphical and formal safety point tests in Sections 3 and 4. We now further sketch out the import of assumptions (1) and (2). As mentioned above, nearly 20% of the respondents in the survey projected zero demand for the product. It is quite common in surveys of this type to see a significant fraction of respondents report zero demand. Using this fact in conjunction with assumptions (1) and (2), we can correctly assign each reported quantity to one of three regions as follows: Region I : Y =0 ⇐⇒ Q = 0 Region II : 0 < Y ≤ s ⇐⇒ 0 < Q ≤ s Region III : Y > s ⇐⇒ Q > s In Section 2, we show how to estimate the parameters of a demand model for Q using only information on the region containing Y . The classification scheme above shows that this region contains the corresponding Q value. We do not use any other information about reported quantities, and consequently, provide an estimation method that is insensitive to reporting bias of the form characterized by assumptions (1) and (2). The new product demand model we assume is the standard Tobit (Tobin (1958)) model: Q = (X 0 β0 + u){X 0 β0 + u > 0} . 4 (3) The disturbance term u is assumed to be independent of X and normally distributed with mean zero and unknown positive variance σ02 . The distributional assumption on u is important, because various estimated quantities of interest will be inconsistent if it does not hold. In the concluding section, we mention work on a semiparametric version of the procedure developed in the next section that makes no parametric assumptions on the error term. Other assumptions implicit in (3) can also be relaxed. For example, we can permit quantities to depend nonlinearly on X. We can also replace Q in (3) with a known, strictly increasing function, F (Q). However, in order to test whether F is correctly specified, more information is needed to distinguish F from Λ. For example, we could assume that F is the log transformation and that there is a known interval of accurate reporting. Proceeding with assumptions (1), (2), and (3), in the next section we provide a method for estimating the parameters β0 and σ0 . Using these parameter estimates, we then show how to recover an estimate of Λ. Our focus will be on recovering an estimate of Λ. There are two main reasons for this. First, knowledge of this function should prove useful in survey design, by defining the nature and extent of misreporting. This, in turn, may suggest ways of designing future surveys to avert or at least minimize this problem. Second, by applying an estimate of Λ to reported quantities, we can recover estimates of actual demand. This can greatly facilitate making revenue forecasts for the product being surveyed. In Section 3, we report the results of several simulation experiments illustrating the performance of the estimation method. We also provide graphical tests of the safety point assumption. Section 4 provides a formal statistical test of this assumption. In Section 5, we apply the method to the video application discussed at the beginning of this section. Section 6 collects results on the asymptotic properties of the parameter estimates, the estimate of Λ, and the safety point tests. In particular, √ we show that the parameter estimates and the pointwise estimates of Λ are n-consistent and asymptotically normally distributed, that the estimate of Λ converges uniformly on compact intervals at √ rate n1/2−δ for any δ > 0, and that n(Λ̂ − Λ) converges in distribution to a mean-zero Gaussian process on compact intervals. Finally, Section 7 provides a summary and directions for future research. 2. The Orbit Procedure In this section we present the Orbit procedure, so-called because it borrows features from an ordered choice model (Amemiya, 1985, Chapter 9) and the Tobit model defined in (3).2 It is a 2-stage procedure in which we first estimate 2 After completing this paper, we learned that Goldberger first used the name Orbit perhaps as far back as 1964 to refer to a method, due to Orcutt, for estimating a Tobit model subject to sample selection. See Kiefer (1989) for a reference. 5 the parameters of the Tobit model, and then use these estimates to recover an estimate of the function Λ at points of interest. Let (Y1 , X1 ), . . . , (Yn , Xn ) denote a sample of n independent observations from the model defined by assumptions (1) through (3) from the last section. Write Zi for (Yi , Xi ) and z ≡ (y, x) for an element of SZ , the support of Zi . Write θ for (β, σ), θ0 for (β0 , σ0 ), and Θ for a compact subset of IRk ⊗ IR+ . For each t > 0, z in SZ , λ > 0, and θ in Θ, define 0 xβ ft (z, λ, θ) = {y = 0} log Φ − σ 0 λ − x0 β xβ + {0 < y ≤ t} log Φ −Φ − σ σ 0 λ−xβ + {y > t} log 1 − Φ σ where Φ denotes the cumulative distribution function of a standard normal random variable. Write Pn for the empirical measure that places mass n1 at each Zi , and note that Pn fs (·, s, θ) defines a log-likelihood function for the data. Define θ̂(s) = argmax Pn fs (·, s, θ) . Θ We call θ̂(s) an Orbit maximum likelihood estimator of θ0 . Standard arguments √ sketched out in Section 6 show that θ̂(s) is n-consistent for θ0 and asymptotically normally distributed. For each t > 0, let Λt denote a compact subset of IR+ containing Λ(t). We estimate Λ(t) with Λ̂(t; s) = argmax Pn ft (·, λ, θ̂(s)) . (4) Λt Straightforward arguments show that Λ(t) maximizes IEft (·, λ, θ0 ), the function to which Pn ft (·, λ, θ̂(s)) converges uniformly. It then readily follows that Λ̂(t; s) consistently estimates Λ(t). We call Λ̂(t; s) an Orbit estimator of Λ(t) and the 2-stage procedure which produces both θ̂(s) and Λ̂(t; s), the Orbit procedure. Note that Λ̂(Yi ; s) is a natural estimate of Qi , the actual demand of the ith individual in the sample. Repeating this calculation for each of the reported quantities yields estimates of actual quantities. Remark 1. The Orbit procedure separates parameter estimation from estimating the function Λ. For this reason, the computational burden involved in using (4) to recover estimates of actual demand is slight. Even if Λ̂(Y ; s) is computed for each positive value of Y in the sample, each of these optimizations is over a single variable, and so can be performed very quickly. For example, using the MAXLIK routine in GAUSS on a 486DX2/66 PC, we did 500 such optimizations well within an hour. Moreover, for some applications, even if the 6 sample size is large, the number of distinct positive Y values in the sample may be quite small. In this situation, the procedure is extremely fast. For example, the sample size for the application considered in Section 5 is around 1000, but because of rounding, the number of distinct positive Y values is around 20. We produced the corresponding estimates of actual demand in less than a minute. Remark 2. There are at least two other ways to estimate the function Λ.3 The first applies only when there are no explanatory variables in the model. It is based on the observation that Λ(t) = α0 + σ0 Φ−1 (IP {Y ≤ t}) where α0 is the intercept in the model. An estimate of Λ(t) can be obtained by substituting Orbit estimates for α0 and σ0 , and substituting the corresponding sample proportion for IP [Y ≤ t]. Since the sample proportion can be viewed as a (nonparametric) maximum likelihood estimator of IP [Y ≤ t], it readily follows that this estimator is equivalent to the Orbit estimator. However, since no second-stage optimization is required, this alternative method is easier and faster to implement than Orbit. The second method applies when explanatory variables are in the model and involves simultaneously estimating Λ at all the points of interest, say, t1 , . . . , tk , rather than one at a time. The underlying model is: Z=j if tj−1 < Y ≤ tj j = 0, 1, . . . , k + 1 where t−1 = −∞, t0 = 0, and tk+1 = ∞ and Λ(tj−1 ) − X 0 β0 Λ(tj ) − X 0 β0 −Φ . IP [Z = j] = Φ σ0 σ0 After substituting Orbit estimates for β0 and σ0 , estimates of the Λ(tj ) are obtained through the usual maximum likelihood procedure for ordered qualitative response models. Like the first alternative, this method is equivalent to the Orbit procedure when only an intercept is fit. When explanatory variables are in the model, this second method is different from Orbit. While we have yet to determine the relative efficiencies of this second method and Orbit,4 there is evidence that the Orbit procedure provides a significant computational edge, especially when the number of tj values is large. In one simulation, Orbit estimated 15 Λ(tj ) values in about 30 seconds using the GAUSS MAXLIK routine running on a 486DX2/66 machine. Using the Orbit estimates as starting values, the second method required over 45 minutes to converge, even though the final estimates were close to the Orbit estimates. 3 We are grateful to two referees for suggesting these alternative methods. most efficient procedure would involve simultaneously estimating β0 , σ0 , and Λ at points of interest. Such a method, however, would be computationally burdensome. 4 The 7 When the Λ(t)’s are estimated sequentially rather than simultaneously, the natural starting value for estimating Λ(t(k) ) is the estimate of Λ(t(k−1) ), where t(k) is the kth largest of the tj ’s. When Λ(t(k) ) and Λ(t(k−1) ) are close, the kth maximization in the second stage of Orbit is very fast. Remark 3. Λ̂(s; s) is constrained to equal s. To see this, for t > 0, z ∈ SZ , µ > 0, and γ ∈ IRk , define gt (z, µ, γ) = {y = 0} log Φ(−x0 γ) + {0 < y ≤ t} log [Φ(µ − x0 γ) − Φ(−x0 γ)] + {y > t} log [1 − Φ(µ − x0 γ)] . Note that gt (·, λ/σ, β/σ) = ft (·, λ, θ). Since Pn fs (·, s, θ) is maximized at θ̂(s), Pn gs (·, µ, β̂(s)/σ̂(s)) is maximized at µ = s/σ̂(s). Consequently, Pn fs (·, λ, θ̂(s)) is maximized at λ = s. Remark 4. We show in Section 6 that if Λ(t) = t in a neighborhood of a safety point s, then it is possible to replace s with any consistent estimator without affecting the asymptotic distributions of the Orbit estimators. For example, one may be willing to assume a priori that Λ(t) = t in a neighborhood of a specified population quantile q of the marginal distribution of Y . The result says that θ̂(q) and Λ̂(t; q) have the same respective asymptotic distributions as θ̂(q̂) and Λ̂(t; q̂) where q̂ is the sample quantile corresponding to q. This result is also useful when applying a test of the safety point assumption developed in Section 4. Remark 5. After applying the Orbit procedure, one may find accurate reporting over an entire range of Y values. A likelihood-based procedure that exploits this extra information will yield a more efficient estimator of θ0 , which, in turn, will lead to a more efficient estimator of Λ(t) in the second stage of the Orbit procedure. For example, after applying Orbit one may find that an interval of the form [0, s0 ] is safe. In this case, one may then estimate a truncated Tobit model to obtain a more efficient estimate of θ0 . However, to minimize assumptions on Λ, we chose not to assume a priori the existence of a safety interval. Remark 6. For positive t values less than the minimum positive Y value in the sample, there are no observations for which 0 < Y ≤ t. Similarly, for t values greater than or equal to the maximum Y value in the sample, there are no observations for which Y > t. For such t values, the objective function Pn ft (·, λ, θ̂(s)) degenerates in the sense that it is maximized at ±∞. As a result, it is not possible to produce corresponding Orbit estimates of Λ(t). 3. Simulation Results 8 In this section, we discuss the results of several simulations exploring various aspects of the estimator Λ̂ defined in (4). The designs were chosen to facilitate comparison with results obtained for the application presented in Section 4. For each of the simulations, the sample size is 1000 and Q = (2 + 2X + 2u){2 + 2X + 2u > 0} (5) where X and u are independent, each having a standard normal distribution. Thus, β0 = (2, 2), σ0 = 2, and the distribution of Q is a mixture of a point mass at zero and a N (2, 8) distribution truncated at zero. There are about 25% zeros in a typical sample from this mixture distribution. Also, for each simulation, we let the interval [0, 4] be an “accuracy region” – a region of accurate reporting. The point 4 corresponds to about the 65th to 70th percentile of the positive Q values. This setup corresponds roughly to what we observed in the application in Section 4. In the first simulation, we investigate the performance of the estimator when there is linear exaggeration beyond the point 4. Write M for the function Λ−1 . We take Y = M (Q) = Q{Q ≤ 4} + (2Q − 4){Q > 4} . (6) Results of this simulation appear in Figures 2 and 3. The plots in Figure 2 illustrate the performance of the estimator when the safety point, s, falls within the accuracy region at the 50th percentile, or median, of the positive Y values. The corresponding plots in Figure 3 illustrate performance when s falls outside the accuracy region, at the 90th percentile of the positive Y values. Turn to Figure 2. In the upper left-hand corner, three quantities are plotted against the estimated quantities Q̂ = Λ̂(Y ; s) where Y comes from equation (6) and the safety point s is approximately equal to 3: the points plot Y vs. Q̂, giving an estimate of the function M ; the piecewise-linear dashed curve is a plot of M (Q̂) vs. Q̂; the straight dashed line is the reference line Q̂ vs. Q̂. The vertical line indicates the position of the safety point, s. Notice the close correspondence between M (Q̂) and Y values, with only a slight increase in variation associated with reported quantities in the extreme tails of the Y distribution. In the upper right-hand corner we see the point plot of Q̂ vs. Q superimposed over the dashed reference plot of Q vs. Q, where Q comes from equation (5). We see that estimated quantities very accurately track actual quantities. In the lower left-hand corner we show a plot of centered first differences of Y vs. Q̂. This plot gives an estimate of the derivative of the function M in equation (6). Notice that the estimated derivative at the safety point is very close to unity, as it should be. Finally, for each t > 0, define σ̂(t) ≡ σ̂(s) Λ̂(t; s) ·t (7) and note that σ̂(t) should be close to σ0 in a neighborhood of s if there is an accuracy region about s. This is confirmed by the simulation: the plot of σ̂(Y ) 9 vs. Y in the lower right-hand corner of Figure 2 shows a region of relative constancy about s, and this constant value is very close to σ0 = 2. Figure 3 shows what can happen when the safety point lies outside the accuracy region. The 4 plots have the same format as in Figure 2, and are based on the same simulated data. The only difference is that the point s is chosen to be the 90th percentile of the positive Y values - approximately equal to 8. The plot in the upper right-hand corner shows that Q̂ overestimates Q everywhere. We see the corresponding problem in the upper left-hand plot. Notice, however, that the two diagnostic plots on the bottom of the page alert us that something is wrong. In the lower left-hand plot the estimate of the derivative at s is far from unity, suggesting that we select another s from a region where the derivative estimates appear roughly constant. Similarly, the lack of relative constancy about σ̂(s) in the lower right-hand plot suggests a similar course of action. We performed a second simulation following exactly the same pattern as the first but with quadratic exaggeration beyond the point 4. Specifically, we took Y = M (Q) = Q{Q ≤ 4} + (Q2 − 12){Q > 4} . (8) The diagnostic plots told the same story as their counterparts in Figures 2 and 3: M (Q̂) and Q̂ did a very good job of estimating Y and Q, respectively, when s was chosen correctly. The plots clearly signaled a problem when s was misspecified. In another simulation of the same model, we considered the case of no exaggeration, namely, Y = Q. Once again, Q̂ accurately tracked Q, with minor deviations in the extreme tails of the Y distribution. Because of the similarity of the results with linear exaggeration, the plots associated with quadratic and no exaggeration are not reproduced in this paper. 4. Safety Point Tests Assumptions (1), (2), and (3) are sufficient to estimate β0 , σ0 , and Λ. However, since Λ̂(s; s) is constrained to equal s (Remark 3 in Section 2), it is impossible to test the assumption Λ(s) = s without more information. This is a critical assumption, since the asymptotic bias incurred from estimating Λ(t) with (4) can be shown to equal Λ(t) [1 − s/Λ(s)] . This bias is zero when Λ(s) = s, but can be substantial if Λ(s) is not close to s. In order to test Λ(s) = s we add the following condition to assumption (2): Λ(t) = t on an interval containing s. 10 Since the procedure used to estimate Λ̂(t; s) imposes no constraints on points near s, departures from the above condition can be easily diagnosed, both graphically, as in the last section, and with a formal statistical test. We present a formal test in this section. t . Recall the definition of σ̂(t) given in (7). For each t > 0 define σ(t) = σ0 · Λ(t) Let a and b be positive real numbers with a < b. We say that the interval [a, b] is a safety interval if the following conditions hold: (A) Λ(t) = t on [a, b]. (B) Λ0 (t) = 1 on (a, b). (C) σ(t) = σ0 on [a, b]. Of course, condition (A) implies (B) and is equivalent to (C). Still, because of the constraint on Λ̂(s; s), it is useful to separate these conditions. We start by assuming that the point s and a set of neighboring points all lie in a safety interval. We then develop a χ2 test of an implication of this assumption. For ease of exposition, we construct a statistic that tests an implication of condition (C). Simple functions of this statistic can be used to test corresponding implications of conditions (A) and (B). Let S ≡ (s1 , . . . , sk )0 be a vector of positive real numbers hypothesized to be in a safety interval containing s. We exclude s from S. We wish to test the following hypothesis: H0 : σ(s) = σ(s1 ) = · · · = σ(sk ) . Note that H0 is implied by condition (C) with [a, b] containing s and all the si ’s. However, H0 is not equivalent to (C) since H0 can hold without (C) holding. However, if H0 does not hold, then (C) is violated. Write σ̂(S) = (1/σ̂(s1 ), . . . , 1/σ̂(sk ))0 and 1 for a column vector of k ones. Consider the statistic √ Cn = n[σ̂(s) · σ̂(S) − 1] . If s and the components of S all lie in a safety interval, then there exists a nonsingular matrix Ω such that Cn =⇒ N (0, Ω) where the symbol =⇒ denotes convergence in distribution and 0 denotes the zero vector in IRk . A proof of this result, exhibiting the explicit form of Ω, is given in Section 6. A consistent estimator of Ω, denoted Ω̂, is also presented there. Deduce that Cn0 Ω̂−1 Cn =⇒ χ2k . The statistic Cn0 Ω̂−1 Cn can be used to test H0 . 11 Turn to Figure 5, and consider the upper right-hand plot. The data is a replication of a simulation of quadratic exaggeration defined in (8) in Section 3. For this data, we plot the square root of the test statistic Cn Ω̂−1 Cn against Orbit estimates of actual quantities, denoted Q̂. That is, the ordinate of the ith point in this plot is the square root of Cn Ω̂−1 Cn where S is the single point Λ̂(Yi ; s). The dotted vertical line corresponds to the safety point s, approximately equal to 3. The dotted horizontal line is the line y = 1.96 and corresponds to the 95th percentile of the distribution of the square root of a χ21 random variable. As expected, the values of the test statistic for points within the accuracy region [0, 4] are generally not significant at the 5% level, but are highly significant at this level for points much greater than 4.5 Choosing S = (1, 2, 4)0 , we get a test value of 1.57 for Cn Ω̂−1 Cn . This value is the 33rd percentile of the distribution of a χ23 random variable. Some care must be exercised in interpreting the results of a test based on Cn Ω̂−1 Cn . As noted above, if one rejects H0 then one must reject condition (C). However, if one accepts H0 one need not accept (C). This follows from the fact that having σ(t) constant on an interval is necessary but not sufficient for that interval to be safe. To see this, consider M (t) = t{t ≤ 4} + 2t{t > 4}. The function σ(t) will equal σ0 on [0, 4] and 2σ0 on (4, ∞). At this point, the practitioner must judge, based on knowledge of the application, whether an apparent region of constancy actually corresponds to a safety interval. Finally, note that one can view the hypothesized safety points s1 , . . . , sk as population quantiles of the unconditional distribution of Y . At the end of Section 6 we show that the si ’s can be replaced by the corresponding sample quantiles (or any consistent estimators) without affecting the asymptotic distribution of the χ2 test. We use this fact in the next section when we test the safety point assumption in the context of survey data. 5. An Application In this section, we present the results of applying the Orbit procedure developed in Section 2 to the survey data on demand for a potential new video product described in the introduction. Because of the proprietary nature of the data, we cannot, at present, identify either the new product or the exogenous variables entering the demand model. In this survey, respondents are asked to estimate the average number of times per month they would use the product if it were offered, and there is a charge for each use. They report nonnegative integer values. Let Y ∗ denote average projected monthly demand and let Q denote average actual monthly demand. We assume that there exists a strictly increasing function Λ such that Λ(Y ∗ ) = Q . 5 This statement is made informally. We make no claims about the asymptotic distribution of the maximum of the individual test statistics. 12 As before, we assume that Q = (X 0 β0 + u){X 0 β0 + u > 0} where X is a vector of explanatory variables, β0 is a vector of unknown parameters, and the random variable u is normally distributed with mean zero and unknown variance σ02 , and is independent of X. Since Y ∗ is an average, we assume that its positive part is continuously distributed. Therefore, we do not observe Y ∗ , but rather a rounded version of it, denoted Y . As a matter of convenience, we shall assume that respondents round their Y ∗ ’s up to the nearest integer.6 It follows that for each positive integer k, Y ≤ k ⇐⇒ Y ∗ ≤ k ⇐⇒ Q ≤ Λ(k) . Therefore, with Λ(Y ) = Λ(Y ∗ ) on the support of Y , we can proceed to estimate quantities of interest under assumptions (1), (2), and (3). Table 1 gives Orbit estimates and t-ratios. The safety point for the masked quantities (reported quantities divided by their median) is chosen to be unity, corresponding to the median of the positive reported quantities. The results are based on a sample of the form (Y1 , X1 ), . . . , (Yn , Xn ) where n = 922. We experimented with adding other pertinent variables to the model but the improvement in fit was negligible. In addition, we tried fitting appropriately transformed variables, certain interaction terms, and higher order polynomial effects but with largely the same result. None of the alternative models we estimated led to a noticeable change in the estimate of Λ. Given the inherent coarseness of the data (σ̂(1) = 2.39), we feel that this final main effects model is a reasonable one. The fact that the signs of the estimated parameters as well as the correlation matrix for the estimated parameters are believable also supports this claim. So, with the Orbit estimator θ̂(1) in hand, we now apply (4) from the last section to estimate Q. The results appear in Figure 4. In the upper left-hand corner is a point plot of Y vs. Q̂ superimposed over the dashed reference line Q̂ vs. Q̂, where Y stands for reported demand and Q̂ stands for estimates of actual demand obtained from applying (4) to the Y values. The position of the safety point, s = 1, is indicated with a vertical line. This plot suggests that reported estimates of average usage are reliable from zero to around the point 1.3 or possibly 1.7 (about the 75th percentile of the Y values), but then begin to take off. The diagnostic plots also seem to confirm that unity is a proper choice of safety point. The estimate of M 0 (1) in the upper right-hand plot is very close to one, and there appears to be stability about unity in the plot of σ̂(Y ) vs. Y . 6 We find that the choice of rounding scheme has little effect on estimation. Therefore, in framing our assumptions, we are guided by convenience: we assume respondents round up. By adopting this convention, we avoid having to make any changes in assumptions, objective functions, or interpretation of results from the last section. Very small, but annoying, changes have to be made for other rounding schemes. 13 The lower right-hand plot is a histogram of Q̂ values where the range of the horizontal axis is the range of the corresponding Y values. The difference in ranges is striking. It is also interesting to compare the other three plots in this figure with the corresponding plots from the simulations in Section 3. In a rough, qualitative sense, it would appear that the type of exaggeration present in this data is somewhere between linear and quadratic beyond the point 1.7. Finally, refer to Figure 5. In the lower left-hand corner, we plot the square root of the test statistic An Σ̂−1 An against Orbit estimates of actual quantities. The dotted vertical line indicates the point s, equal to unity. The dotted horizontal line indicates the 95th percentile of the distribution of the square root of a χ21 random variable. Write (s−− , s− ) for the points adjacent to unity from below and (s+ , s++ ) for the points adjacent from above. That is, s−− < s− < 1 < s+ < s++ . At the 5% level, the points s−− , s+ , and s++ individually and jointly pass the corresponding χ2 tests. However, s− , the point just below unity, fails the χ21 test at the 5% level. Consequently, there is some doubt about unity as a safety point. To further probe the matter, we repeat the entire Orbit procedure, this time taking the point s+ as a new potential safety point. The corresponding χ21 test results appear in the lower right-hand plot of Figure 5. The vector S = (1, s++ ) passes the χ22 test at the 5% level with a p-value of .82. We conclude that there is not enough evidence to reject the hypothesis that σ(1) = σ(s+ ) = σ(s++ ) . Notice that there are no other apparent intervals of constancy in the lower lefthand plot in Figure 4. This and properties of the application suggest that it is reasonable to view the interval [1, s++ ] as a safety interval. That unity is a reasonable safety point is further supported by a p-value of .46 for its associated χ21 test value. 6. Asymptotic Properties In this section, we establish some asymptotic properties of the Orbit estimates and the safety √ point tests. In particular, we show that the estimates of θ0 and Λ(t) are n-consistent and asymptotically normally distributed, that the estimate of Λ converges uniformly on compact intervals at rate n1/2−δ for √ any δ > 0, and that n(Λ̂ − Λ) converges in distribution to a mean-zero Gaussian process on compact intervals. We first present results for the case where the safety point s is known. Subsequently, we will slightly strengthen our assumptions to include a small safe interval about an unknown, but consistently estimable point s. For example, s might be chosen a priori to be the median of the marginal distribution of the positive Y values, as in the application in Section 5. For this case, we show that we may replace s with any consistent 14 estimator without affecting the asymptotic properties of the Orbit estimates and safety point tests. We begin with the case where the safety point s is known. Review the notation introduced at the beginning of Section 2. The following assumptions are sufficient for the consistency and asymptotic normality results: A1. Z1 , . . . , Zn is a sample of independent observations from the model described by assumptions (1), (2), and (3) in the introduction. A2. The support of X, the vector of explanatory variables in (3), is bounded. A3. θ0 is an interior point of Θ, a compact subset of IRk ⊗ IR+ . A4. Λ(t) is an interior point of Λt , a compact subset of IR+ . Assumption A1 describes the data and the model. Assumption A2 is made solely for convenience, and guarantees that probabilities that are arguments of the log function stay bounded away from zero. A3 and A4 are standard assumptions, ensuring consistency and a limiting normal distribution for the estimators. If A1 through A3 hold, then standard arguments show that θ̂(s) converges in probability to θ0 . For example, a simple piece of calculus shows that IEfs (·, s, θ) is uniquely maximized at θ0 . Andrews (1987) shows that Pn fs (·, s, θ) converges uniformly in probability to IEfs (·, s, θ). Consistency then follows from Amemiya (1985, pp.106–107). Let λ denote a positive real number. Define the following derivative opera∂ ∂ ; ∇θ ≡ ∂θ ; ∇λθ ≡ ∇θ [∇λ ]; ∇θθ ≡ ∇θ [∇θ ]; ∇λλ ≡ ∇λ [∇λ ]. tors: ∇λ ≡ ∂λ Theorem 1: If A1 through A3 hold, then √ √ n(θ̂(s) − θ0 ) = nPn gs (·, θ0 ) + op (1) where gs (z, θ) = −[Hs (θ)]−1 ∇θ fs (z, s, θ) and Hs (θ) = IE∇θθ fs (·, s, θ) . Theorem 1 follows from standard Taylor expansion arguments. See, for example, Amemiya (1985, pp.111–114). Write 0 for the zero vector in IRk . The symbol =⇒ denotes convergence in distribution. Corollary: If A1 through A3 hold, then √ n[θ̂(s) − θ0 ] =⇒ N (0, −[Hs (θ0 )]−1 ) . 15 Turn to Λ̂(t; s). If A1 through A4 hold, then straightforward arguments, similar to those referred to in relation to θ̂(s), show that for each t > 0, Λ̂(t; s) converges in probability to Λ(t). For each t > 0, λ in Λt , and θ in Θ, define mt (λ, θ) = IE∇λθ ft (·, λ, θ) and Ht (λ, θ) = IE∇λλ ft (·, λ, θ) . Theorem 2: If A1 through A4 hold, then for each t > 0, √ √ n[Λ̂(t; s) − Λ(t)] = nPn hst (·, Λ(t), θ0 ) + op (1) where hst (z, λ, θ) = −[∇λ ft (z, λ, θ) + [gs (z, θ)]0 mt (λ, θ)]/Ht (λ, θ) . Theorem 2 follows from standard Taylor expansion arguments. The term gs (z, θ0 )]0 mt (Λ(t), θ0 ) corrects the variance of Λ̂(t; s) for the fact that θ0 is known with error. We can use Theorem 2 to derive the distribution of the test statistic Cn0 Ω̂−1 Cn defined in Section 4. Note that for each t > 0, [σ̂(s)/σ̂(t) − 1] = [Λ(t; s) − t] /t . Recall from Section 4 that S denotes the set of k points under test. Let D denote the diagonal matrix with the components of S along the diagonal. Corollary 2.1: Suppose A1 through A4 hold. If the components of S lie within a safety interval, then Cn =⇒ N (0, Ω) where Ω = D−1 ΣD−1 and the ijth element of Σ is given by IEhssi (·, Λ(si ), θ0 ) hssj (·, Λ(sj ), θ0 ) . We assume that the matrix Σ in Corollary 2.1 is invertible. The ijth element of Σ can be consistently estimated by replacing Λ(t) with Λ̂(t), θ0 with θ̂(s), and expectations with the corresponding sample averages. Let Σ̂ denote the 16 corresponding matrix estimator and write Ω̂ for D−1 Σ̂D−1 . Since Σ̂ consistently estimates Σ, we have that Cn0 Ω̂−1 Cn =⇒ χ2k . This fact justifies the test developed in Section 4. Corollary 2.2: If A1 through A4 hold, then for each t > 0, √ n[Λ̂(t; s) − Λ(t)] =⇒ N (0, Vt (Λ(t), θ0 ; s)) where Vt (λ, θ; s) = IE[hst (·, λ, θ)]2 . A simple calculation shows that Vt (Λ(t), θ0 ; s) is finite for each t > 0. Similar calculations show that Vt (Λ(t), θ0 ; s) converges to zero as t converges to zero and converges to infinity as t goes to infinity. Also, Vs (Λ(s), θ0 ; s) = 0, since Λ̂(s; s) = Λ(s) = s. These facts are illustrated in the upper left-hand plot in Figure 5. The data for the plot come from a simulation replicating quadratic exaggeration defined in (8) in Section 3. The vertical line indicates the safety point, approximately equal to 3. The square root of the sample analogue of VY (Λ(Y ), θ0 , ; s) is plotted against Q̂. The shape of the function Λ(t) defines the nature and extent of misreporting. Our next results concern the uniform convergence of Λ̂(t; s) to Λ(t). Such uniform results are useful for inferring the shape of Λ(t) from Λ̂(t; s). A simple calculation shows that for each t > 0, Λ̂(t; s) − Λ(t) = − Pn ∇λ ft (·, Λ(t), θ̂(s)) Pn ∇λλ ft (·, λn , θ̂(s)) for λn between Λ(t; s) and Λ(t). Fix positive numbers r < ρ. A straightforward calculation shows that inf Pn ∇λλ ft (·, λn , θ̂(s)) > 0 n,r≤t≤ρ and Pn ∇λ ft (·, Λ(t), θ̂(s)) = Ht (Λ(t), θ0 ) Pn hst (·, Λ(t), θ0 ) √ + Op (1/ n) (9) uniformly over t in the set [r, ρ]. Another straightforward calculation shows that sup z∈SZ ,r≤t≤ρ |Ht (Λ(t), θ0 ) hst (z, Λ(t), θ0 )| < ∞ . 17 Thus, the average in (9) is of mean-zero, independent, identically distributed, bounded random variables. Theorem 3: Fix positive numbers r < ρ. If A1 through A4 hold, then for each δ > 0, n1/2−δ sup Λ̂(t; s) − Λ(t) = op (1) . r≤t≤ρ The proof of Theorem 3 follows from the preceding remarks together with Lemma 1 in Appendix A of Klein and Spady (1993). Their Lemma 1 is based on arguments in Lemma 2 of Bhattacharya (1967). √ For each t ≥ 0, write Γn (t; s) for n(Λ̂(t; s)−Λ(t)). Fix positive numbers r < ρ. Apply Lemma 15 and Theorem 21 in Pollard (1984, Chapter VII) together with Theorem 2 and Theorem 3 to see that the process {Γn (t; s) : r ≤ t ≤ ρ} converges in distribution to a Gaussian process {Γ(t; s) : r ≤ t ≤ ρ} satisfying Γ(s; s) = 0 and having covariance kernel C(t, τ ) = IEhst (·, Λ(t), θ0 ) hsτ (·, Λ(τ ), θ0 ) . Finally, suppose we are willing to assume a priori that Λ(t) = t in a neighborhood of, say, a specified population quantile of the marginal distribution of Y . Let s denote this population quantile and ŝ the corresponding sample quantile. Note that ŝ consistently estimates s. We now show that θ̂(ŝ) has the same asymptotic distribution as θ̂(s). This, the fact that Λ̂(t; ŝ) depends on ŝ only through θ̂(ŝ), and standard uniform convergence results will imply that Λ̂(t; ŝ) = Λ(t) − Pn ∇λ ft (·, Λ(t), θ0 ) + (θ̂(ŝ) − θ0 ) mt (Λ(t), θ0 ) 1 + op ( √ ) . Ht (Λ(t), θ0 ) n The distributional result for θ̂(ŝ) will then imply that Λ̂(t; ŝ) has the same asymptotic distribution as Λ̂(t; s). We first show that θ̂(ŝ) converges in probability to θ0 . Since ŝ consistently estimates s, there must exist a sequence {δn } of positive numbers converging to zero as n tends to infinity for which IP {|ŝ − s| > δn } → 0. It follows that |θ̂(ŝ) − θ0 | ≤ |θ̂(t) − θ0 | + op (1) . sup |t−s|≤δn A simple calculation shows that θ̂(ŝ) − θ0 = −[Pn ∇θθ fŝ (·, ŝ, θ∗ )]−1 Pn ∇θ fŝ (·, ŝ, θ0 ) where θ∗ is between θ̂(ŝ) and θ0 . Note that for any point t satisfying Λ(t) = t, IE∇θ ft (·, t, θ0 ) = 0 . 18 (10) Since Λ(t) = t near s, eventually Λ(t) = t for all t within δn of s. The Euclidean property of the class of functions {∇θ ft (·, t, θ0 ) : |t − s| ≤ δn } follows easily from an application of Lemmas 2.4, 2.12, and 2.14 in Pakes and Pollard (1989). It then follows from their Lemma 2.8 that sup |Pn ∇θ ft (·, t, θ0 )| = op (1) . |t−s|≤δn A similar argument shows that sup |Pn ∇θθ ft (·, t, θ) − IE∇θθ ft (·, t, θ)| = op (1) . |t−s|≤δn , θ∈Θ It follows from the continuity of IE∇θθ ft (·, t, θ) as a function of t that sup |IE∇θθ ft (·, t, θ) − IE∇θθ fs (·, s, θ)| = o(1) . |t−s|≤δn , θ∈Θ The strict concavity of IEfs (·, s, θ) and the compactness of Θ ensure the boundedness of supΘ [IE∇θθ fs (·, s, θ)]−1 . Deduce that θ̂(ŝ) consistently estimates θ0 . √ Apply the consistency result to see that n[θ̂(ŝ) − θ̂(s)] equals √ −[IE∇θθ fs (·, s, θ0 )]−1 n[Pn ∇θ fŝ (·, ŝ, θ0 ) − Pn ∇θ fs (·, s, θ0 )] + op (1) . Arguing as before, we have √ n|θ̂(ŝ) − θ̂(s)| ≤ √ sup n|θ̂(t) − θ̂(s)| + op (1) . |t−s|≤δn Apply (10) once more along with Lemma 2.17 in Pakes and Pollard (1989) to √ see that n[θ̂(ŝ) − θ̂(s)] has order op (1). 7. Conclusions In this paper, we consider the problem of estimating demand for a new product in the presence of biased demand projections. The form of the bias is characterized by a strictly increasing function Λ of projected demand. We develop a two-stage procedure, called Orbit, for estimating (1) the parameters of a standard Tobit model for actual future demand, and (2) the function, Λ. We make no parametric assumptions about the functional form √ of Λ. Nor do we require that Λ be continuous. The Orbit estimates are n-consistent and asymptotically normally distributed, the estimate √ of Λ converges uniformly on compact sets at rate n1/2−δ for any δ > 0, and n(Λ̂ − Λ) converges in distribution to a zero-mean Gaussian process on compact intervals. Moreover, the procedure is computationally tractable. To apply the Orbit procedure, there must exist a positive safety point at which reported quantities equal actual quantities. This point must either be 19 known a priori, or if unknown, must be consistently estimable and contained in a safe open interval. We provide graphical and formal tests of this assumption. When a given safety point is incorrect, the graphical tests can suggest a proper choice of safety point. In our simulations, we examine reporting mechanisms for which reported quantities equal actual quantities on a given interval. Beyond a threshold point, we let reported quantities overstate actual quantities in a variety of ways. In each simulation, with a sample size of 1000 observations, Orbit accurately recovers the the relationship between reported and actual quantities. We also apply the Orbit procedure to survey data on a potential new video product. Under our model assumptions, the survey respondents report demand projections that exaggerate actual future demand beyond 1.7 times the median level, corresponding to about 25% of the sample. This level of exaggeration falls somewhere between linear and quadratic exaggeration, as discussed in Section 4. The standard Tobit model for actual future demand assumes normality of the model’s error term. It is possible to test this assumption, and when it fails, to apply a semiparametric version of Orbit that does not require making any parametric assumptions about the distribution of the error term. We are currently investigating the theoretical properties of this procedure and its finite sample performance. 20 REFERENCES Amemiya, T. (1985): Advanced Econometrics. Harvard University Press, Cambridge, Mass. Andrews, D. W. K. (1987): “Consistency in Nonlinear Econometric Models: A Generic Uniform Law of Large Numbers,” Econometrica, 55, 1465–1471. Bhattacharya, P. K. (1967): “Estimation of a Probability Density Function and its Derivatives,” The Indian Journal of Statistics: Series A, 373–383. Box, G. E. P. and Cox, D. R. (1964): “An Analysis of Transformations,” Journal of the Royal Statistical Society, Series B, 34, 187–220. Horowitz, J. L. (1992): “Semiparametric Estimation of a Regression Model with an Unknown Transformation of the Dependent Variable,” Working Paper #92–28, Department of Economics, University of Iowa, Iowa City, IA. Kiefer, N. M.(1989): “The ET Interview: Arthur S. Goldberger,” Econometric Theory, 5, 133-160. Klein, R. W. and Spady, R. H. (1993): “An Efficient Semiparametric Estimator for Binary Response Models,” Econometrica, 61, 387–421. Pakes, A. and Pollard, D. (1989). Simulation and the Asymptotics of Optimization Estimators. Econometrica, 57, 1027–1057. Pollard D. (1984). Convergence of Stochastic Processes. Springer, New York. Tobin, J. (1958): “Estimation of Relationships for Limited Dependent Variables,” Econometrica, 26, 24–36. 21

© Copyright 2021 Paperzz