Search-Engine Advertising: Dynamic Auctions under Performance-Based Pricing Hemant K. Bhargava B, hemantb@ucdavis.edu Juan (Jane) Fend B, jane.feng@cba.ufl.edu Hemant K. Bhargava is Professor of Management and of Computer Science at The University of California at Davis. He received his Ph.D. and M.S. from The Wharton School in Decision Sciences in 1990. His recent work on quality uncertainty in the context of IT services and products explains how firms can employ contingency pricing schemes in order to better manage the effects of quality uncertainty and improve their competitive position. His research interest in the economics of information systems and IT industry covers sponsored search in recommenders and search engines, stockout compensation practices, performance-based pricing for IT goods, product versioning, pricing strategies for grid computing, and information structures in contract design. Dr. Bhargava received the Menneken Prize for Excellence in Research at the Naval Postgraduate School, and his research on DecisionNet won best-paper awards at leading information systems conferences. Juan Feng is Assistant Professor of Information Systems & Operations Management at the University of Florida, Warrington College of Business. In 2003, she received her doctorate degree in Management Science & Information Systems from Penn State University, with a dual title in Operations Research. In 2002, she was the Penn State representative to the Doctoral Student Consortium, held in Barcelona. That same year Jane received the e-Business Research Center’s Doctoral Support Award. Jane’s current research is in economics of information systems, sponsored search auctions in the information gatekeepers, and pricing and competition strategies.
In the last 15 years, the Internet has spurred on a vast collection of online information services - real-time traffic, weather, news, stocks, audio and video streaming, travel search, shopbots, recommendation services, photo and video sharing, social networking. And the biggest gorilla of them all, Internet search engines (the leading search engine, Google, had an average of 140,000,000 searches per day during August 2007† ). The dominant revenue source for search engines and many other online information services is advertising by merchants who hope to reap monetary rewards through trade with users of these services. This form of advertising is commonly referred to as “sponsored search,” capturing the idea that some of the search results (or, generally, information content) returned by these services are paid by third-party sponsors: communication of these “relevant” or targeted messages enables the search engine to offer a free service to end-users. Other popular industry terms for this practice are paid search and preferential placement. The first use of sponsored results (in 1998) is credited to the search engine GoTo.com, which later becomes Overture Services which was acquired by Yahoo! in 2003. Sponsored search is widely accepted to be the driving factor behind the meteoric rise in commercial value of the current industry leader, Google. It is behind the multi-billion dollar valuations of popular social networking sites, and has also † Nielsen
NetRatings, 2007, as quoted on SearchEngineWatch.com.
ICS News B
entered the realm of inter-personal phone calls: recently, both wireless and fixed-line telephone operators have begun offering free service in exchange for advertising messages determined by monitoring the conversation. While sponsored results are observed in many information services, this article uses the context of search engines for ease of exposition. Sponsored search advertising, having hatched over a foundation of modern information technologies, has several innovative characteristics that make it very data, computation, and algorithm intensive. First, the advertisements (or “messages”) are contextual rather than general and non-personal. The mapping from search query to advertisement is made in real-time based on contextual information from the current search as well as past behavior. Second, advertisements tend to be very fine-grained (e.g., for a specific product out of tens of thousands that a merchant may carry, vs. simply advertising the brand). Merchants therefore have to manage their advertising expenditures over thousands of messages. The search engines, in turn, manage inventories of hundreds of thousands of keywords or search phrases. Third, the advertising price for each keyword is discovered mostly through online, repetitive, auctions. The search engine announces a mechanism or format for the auction — a separate auction is conducted for each keyword — and merchants must place bids on each keyword they wish to compete on (these bids are considered independent, even though merchants’ preferences may be interrelated). Fourth, the advertising payment is “performancebased” meaning that the actual fee is based not just on impressions of the ad (as in traditional advertising) but on some measure of advertising performance (the commonly used measure is “clickthrough” but firms are experimenting with other measures of conversion such as page-views and product sales). Fifth, for each keyword, the merchants’ bids and the search engines’ selection of winners — and more generally, the choice of auction format — must be determined under uncertainty about one crucial set of variables, the performance of each merchant’s ad. Collectively, these five characteristics lead to several novel and complex optimization problems. Let us begin with a simplified statement of this practice. Consider a single auction for a single sponsored slot on a single keyword. I merchants are interested in contacting consumers who search on this keyword. The search engine awards the slot in order to maximize its expected value. Under performance-based pricing, the expected revenue from awarding the slot to merchant i is a function of merchant i’s clickthrough rate, which the search engine can (imperfectly) estimate, and the merchant’s bids. This appears like a straightforward expected value maximization problem: compute all the expected revenues and pick the highest one. However, in reality, the same keyword is searched thousands of times a day or week. If the search engine’s initial estimates of clickthrough rates are incorrect (as is likely the case), a myopic allocation scheme (award all exposures to the
Fall 2007
Page 1
merchant with highest expected revenue) is predictably subFor example, ZiXXo employs “pay-per-print� while Ingeoptimal. This raises the need to refine the estimates of clicknio uses a “pay-per-call� model. The more “aggressive� through rates. But such learning requires the search engine to the performance measure, the less likely that the search provide some exposures to merchants that have, for that parengine can accurately estimate or control the probability ticular exposure, low expected revenue. This creates a tradeoff of successful performance. Hence the choice of unit for between exploring and exploiting available information. How pricing has fundamental implications on other aspects of to best allocate the slots over a number of searches in order to auction design. optimally balance exploration and exploitation? Readers will readily recognize the multi-arm bandit problem here. The auc- Determining the winner. The simplest mechanism for ranking bidders is “rank by bid� (used in recent years by GoTo, tion design and analysis is further complicated because each Overture, and Yahoo!), which may be reasonable when the search is an opportunity to allocate not just one but several winner pays per impression. But under performance-based slots. These slots are not identical, rather they are vertically pricing, the slot generates revenue only when some user differentiated variants (its safe to assume that all merchants takes an action (such as a click), raising the need to weed agree that the top-ranked slot garners more user attention than out frivolous bids that may have high value but generate lower-ranked slots). Lower-ranked slots are subject to the “law no click-through revenue (Yahoo! employed manual edof declining clickthrough rates�. This adds to the complexity itorial filtering when they used this scheme). Recognizof revenue maximization in the single auction problem. Moreing this, some search engines sort the bidders by a product over, this entire analysis depends on how merchants respond to Vi = pi Bi which models the expected payoff from awardthe auction rules set by the search engine. Merchants’ bidding ing the slot to merchant i (this approach has been employed strategy, the allocation of the sponsored slots, and the search by Google). Vi can be thought of as the value of merchant engine’s revenues can differ substantially based on the auction i to the search engine. Several papers have found that it is format, the analysis of which is in turn complicated due to the profitable to rank the winners by pi Bi [5, 10, 6]. factors mentioned above. The rest of this article elaborates on the problem characteristics and challenges outlined above. Pricing the slot. In terms of the payment scheme (what price R to charge the winner), the “pay your bid� scheme (firstAuction Design: How and Whom to Allocate Sponsored Slots? price auction, used in recent years by GoTo, Overture, and Online auctions have become the dominant method for alYahoo!) requires the winner ( j = arg maxi Vi ) to pay B j locating sponsored slots. Consider again a single auction for a per click. Or, the search engine may use “pay an incresingle sponsored slot on a single keyword, of value to I merment over next-highest bid�. This would be straightforchants. Suppose merchant i has value bi for each customer ward in a “pay-per-impression� model (set R j to the value contact directed by the search engine; this is the merchant’s Bk + where k is the next highest bidder after j). Howreservation price for the slot. This may be the average profit ever, performance-based pricing introduces a small twist: on a sale divided by the number of contacts needed to make a should the rule be applied to bid amount Bi (the well-understood sale, or may represent the cost of customer acquisition using an measure) or to expected revenue pi Ri ? These two might alternative channel. Let Bi be the amount that merchant i bids produce very different results because the pi ’s may be subin the auction, which depends on bi and the auction format. stantially different across the advertisers, moreover the esThe auction format determines how merchants bid, but we igtimates may be incorrect. For example (focusing on ex ante nore the nuances of the winner determination rule and meranalysis), suppose p = 0.04 with B1 = 4 (expected rev1 chants’ bidding strategy in order to focus on other complexienue $0.16 per impression), while p2 = 0.01 with B2 = 9 ties unique to this problem. Let pi denote the search engine’s (expected revenue $0.09 per impression). Merchant 1 wins estimate of merchant i’s clickthrough probability (in practice, the slot, but how much should he pay per click? Applyclickthrough rates typically range from 0 to 3%). Let Ri deing “next highest bid plus a penny� to B ’s would yield i note the price-per-click that the search engine can charge if it $10.01 (obviously meaningless because merchant 1 is willallocates the slot to merchant i. Then the expected payoff from ing to pay no more than $4 per click), whereas applying awarding the slot to merchant i is pi Ri . The search engine’s the rule to expected revenue would require R = $2.50 per auction design problem — how to select the slot winner, what click. But this second, more reasonable, approach also to charge — poses several design choices. poses problems when there are multiple slots to allocate. Suppose p3 = 0.01 with B3 = 7 (expected revenue $0.07 The unit for charging. While advertising fees were traditionper impression). So, now, the second slot (which is awarded ally linked to the number of impressions, internet-based to merchant 2) would require the winner to generate an exadvertising is dominated by performance-based pricing. The pected revenue of $0.08, with a per-click price of $8. Here, commonly used “performance� measure is simply a “click�, we have a seemingly strange outcome that the merchant in but the industry is experimenting with more aggressive acthe top slot pays $2.50 per click while one in the second tions such as page views, downloads, and product sale [8].
ICS News B
Fall 2007
Page 2
slot pays a higher per-click fee of $8. A third alternative is to set R j to be the minimum of the price obtained under the two approaches. This reduces the search engine’s revenue but has desirable properties. It is simple and fair: the merchant that occupies rank i pays (a small increment) more per click than the merchant at rank i + 1. The higher rank can produce substantially higher expected revenue because the slot (if awarded to merchants) is likely to generate more clicks. The property that, on a per click basis, a lower ranked merchant pays less than a higher-ranked one, reduces merchants’ incentives for bidshading. The overall challenge in auction design is to develop a complete specification of the auction format such that it achieves some objective (maximize the search engine’s profit, or the advertiser’s payoff, or the consumer’s welfare, or the total social welfare, etc.). However, deriving this complete specification and its associated equilibrium outcomes are non-trivial because (1) the number of slots available and the number of bidders participating in the auction are usually more than just a few, (2) advertisers from different industry and backgrounds may have quite different strategic objectives when participating in such auctions (for example, some advertisers may be interested in raising awareness/impression, and some may prefer to signal their high quality levels; some advertisers may prefer the top-rank position, while some may prefer may prefer a lower rank to attract more serious consumers, etc.), and (3) as a result, bidders may have quite different daily budgets to participate in these auctions. Moreover, there are dynamic features that are hard to completely capture in the commonly used static models: for example, new advertisers may participate in these auctions any time (while others advertisers have sufficiently long history and experience), and bidders have the flexibility of adjusting individual bids in real time. Thus, in reality, focusing purely on designing the optimal auction may not be sufficiently productive. Equally worthwhile is comparative analysis of intuitive designs or those observed in practice. Factors that deserve attention while studying an auction mechanism include the following. • Clickthrough rates and revenue at each position. Experience in several domains (travel search, movies, listing guides) indicates that higher ranks get, ceteris paribus, more attention and click rates. Some studies of search auction design assume that the clicks received by an advertisement do not depend on the identity of the advertiser. However, for the user with a search problem, all merchant listings are not equal. The “more relevant” merchants are likely to get higher clicks at the same rank than less relevant ones. This raises the complexity of estimating expected revenues, winner determination, and pricing. • The advertiser’s optimal bidding behavior with a certain budget, given he/she participates in a cluster of keywords. It ICS News B
may be wise to adjust bids/budget based on the cost/benefit of each auction, which may include different competition status (current price for different ranks) of these keywords, and the click-throughs/conversions attracted by these keywords. This is a complicated problem because bidders may compete for different advertising positions for different keywords. In addition, in the presence of budget constraints, merchants might game their bids in a second-price auction: a lower-value bidder may have incentive to bid high in initial rounds (note that this does not actually cause it pay a higher price for its own slot) to quickly force its competition to run out of its daily budget and drop out, thereby enabling it to win the high slot for a low price in the future. • The outcome of the bidding prices and winning ranks for different advertisers. These usually depend on the auction mechanism in consideration. For example, studies of Yahoo!’s first price auction [11, 1] have found that the winning ranks and bidding prices are frequently updated and the prices form a cycle, whereas the prices and ranks are relatively stable in Google’s second price auction [9, 4]. • The correlation between the quality/relevance of the winners and their winning ranks. The literature on “advertising as signaling” predicts that because the high quality advertiser benefits more from attracting more demand, it has bigger incentive than an advertiser with a lower quality level [7]. This makes it more likely that a higher quality advertiser wins a higher position. While this might be the case in general when advertising serves as a signal, other features might affect this outcome, such as the accuracy of search results presented by the search engine [3], the type of item being searched (for example, experience or search goods [2]), the different roles of advertising (serving as a signaling mechanism or raising awareness, [12]), or whether the product price is fixed or allowed to be adjusted according to the bidding prices. Dynamics: Learning and Revising Clickthrough Probabilities Now consider the repetitive nature of sponsored slot auctions for keywords. Each keyword may be searched by web users hundreds or thousands of times a day or week. During this time the search engine may receive bids from a number of merchants, some of whom participate throughout, and others that arrive after certain number of periods, or may exits earlier. The search engine’s optimal allocations, and revenue, depend on the clickthrough rates (pi ’s), hence on the accuracy with which the search engine estimates future click rates. However, the search engine may know little about its “new customers” (merchants who haven’t participated previously), and also about existing merchants who have not received sufficient exposure (perhaps due to low initial estimates of their
Fall 2007
Page 3
click rate). Thus it is imperative to consider the tradeoff between exploring the quality of merchants (which requires allocating the slot even to merchants with low expected revenues) and exploiting available information (but awarding the slot to low-value merchants has an expected revenue loss). This is a multi-arm bandit problem, with the additional complications that a) each trial (or “play”) involves making multiple allocations (the K slots available for sponsored search), and b) not all ranks are equivalent in the quality of information they generate. For example, a click produced by a merchant who occupies the top slot provides less information about its quality than a click achieved by a merchant in the 10th rank (an advertisement attracts more attention at a top rank than at a lower ranked); similarly, a “failure” (no click) on the top slot conveys more than a failure at the 10th . How, then, to allocate the slots over a number of searches in order to reasonably balance exploration and exploitation? A naïve strategy for making this tradeoff is practiced in many fields (including in sponsored search engines): conduct a series of trials to cover a subset of the planning period, and employ the information gained through the trials in making future decisions. For example, suppose the search engine has 5 slots to allocate over each of 10,000 searches. It might designate a trial period of, say, 1000 searches where the 5 slots are rotated amongst the 10 most promising merchants. This trial period ranks the merchants according to the expected revenue they produce, and the search engine then awards slots for each of the remaining 9,000 searches according to this ranking. The problem with this strategy is that (even though it exploits well the information learnt from the first 1,000 searches) it fails to explore enough: the winner of the first 1,000 trials may not in fact be the best merchant. This example highlights opportunities for dynamic optimization techniques such as evolutionary optimization, dynamic programming, and reinforcement learning. In a dynamic approach, allocations at time t would consider all the information gained till time t (i.e., not just in a predefined trial period), but in addition, prior allocations would aim not just to optimize the local revenue for that period but to produce additional information for future periods. A simple modification of the naïve approach is to conduct a series of trials which give exposure to a wide base of merchants, and then continue to update clickthrough estimates based on actual observations. What, then, is the ideal length of the trial period, and should all observations be treated uniformly or differently based on the rank? Feng, et al. [5] proposed a weighted revision rule in which the clickscore of a merchant was increased by a factor δ j , where δ the observed attention decay factor and j denoting the position at which a click was received. They experimentally compared dynamic performance of allocation mechanisms on two dimensions, 1) the length of trial period (the trial period covers sufficient searches to ensure that each merchant occupies each slot at least once), and 2) the use of
ICS News B
weighted and unweighted revision rules. The experiment involved bids by 5 merchants for 5 sponsored slots, over 1000 searches. Merchants’ reservation prices and clickthrough rates followed a random distribution (covering a range of correlations between these two parameters) and the outcomes were produced by aggregating data from 200 runs for each dynamic allocation mechanism. The measures of dynamic performance included 1) the frequency of convergence (to the true click rates for each merchant), 2) the speed of convergence, and 3) total profit for the entire period. Their analysis demonstrated that the weighted revision rule had both a higher frequency of convergence and shorter time to learn the true click rates (see Figure HB-1).
Figure HB-1. Convergence and Learning Performance for Different Dynamic Mechanisms
Management of Sponsored Search The above presentation of the problem employed a simplified scenario with three parties: users, advertisers, and the search engine. In practice there is at least a fourth crucial player: firms that act as advertising brokers (often this role is played by the major search engines). And when sponsored search is managed by a broker, e.g., Overture’s relationship with MSN, Google’s relationship with AOL, and LookSmart’s relationship with Lycos—the broker is a fourth party to consider in the balance. The search engine’s (or broker’s) most direct goal is to maximize revenue. However, revenue is entirely dependent on keeping both advertisers and users from defecting to other search engines. So, the sponsored search mechanism design problem must simultaneously consider a number of factors, including direct revenue, utility for users, utility for advertisers, and, in the case of broker-affiliate relationships, utility for the affiliate. It’s worth noting that simply maximizing current revenue may not be a good strategy for the search engine as these factors are interdependent in the long run. For example, advertising revenues, including the sponsored search revenue, depends on the search engine’s volume
Fall 2007
Page 4
of traffic, which in turn affect the number of advertisers the [3] H.K. Bhargava and J. Feng, Sponsored Results in Intelligent Recommenders: Impact on Quality Signaling and Consumer search engine attracts. Therefore, the utility for users and adWelfare, Working Paper, Business School, University of Calivertisers are important factors to consider in the long run. fornia at Davis, 2007. Other management challenges include detecting and ignoring robot clicks and fraudulent clicks by people with malicious [4] B. Edelman, M. Ostrovsky, and M. Schwarz, Internet Advertising and the Generalized Second Price Auction: Selling Bilintent — for example a competing advertiser who wants to lions of Dollars Worth of Keywords, American Economic Reforce costs onto their competitor, or an affiliate who actually view 97:1 (2007), 242–259. benefits monetarily from additional clicks. Click fraud is a serious challenge faced by the search engine under the pay- [5] J. Feng, H.K. Bhargava, and D. Pennock, Implementing Sponsored Search in Web Search Engines: Computational Evaluaper-click pricing scheme. Advertisers are discontent with how tion of Alternative Mechanisms, INFORMS Journal on Comclick fraud is current managed and some lawsuits are filed. The puting 19:1 (2007), 137–148. ultimate solution to this problem remains to be investigated, and it needs to consider the mis-alignment of incentives of the [6] D. Liu and J. Chen, Designing Online Auctions with Past Performance Information, Decision Support Systems 42:3 (2006), advertising brokers, the search engine, and the advertisers. 1307–1320. Data measurement issues also remain to be addressed. Ironically, advertising on the Internet has sharpened some of the [7] P. Milgrom and J. Roberts, Price and Advertising Signals of Product Quality, Journal of Political Economy 94:4 (1986), data measurement problems associated with the commercial 796–821. aspects of advertising. In traditional print and media advertising, inadequate data measurement — how many copies of [8] The Ultimate Marketing Machine, The Economist 380:8485 the newspaper were sold? how many people viewed a particu(2006), 61–64. lar ad in the paper? how many were influenced by it? — has [9] H.R. Varian, Position Auctions, International Journal of Indusbeen a known evil. The Internet promised to change that, but trial Organization, to appear. substantial concerns exist about how to measure the number [10] T. Weber and Z. Zheng, A Model of Search Intermediaries and of impressions or actions for an ad. This raises need for better Paid Referrals, Information Systems Research, to appear. measurement techniques, perhaps supported by new informa[11] X.M. Zhang and J. Feng, Price cycles in online advertising auction structures, as well as new algorithmic techniques that can tions, Proceedings of the 26th International Conference on Inaccurately estimate the relevant outcome variables using actual formation Systems (ICIS), Las Vegas, NV, 2005. measurements of reliable proxy variables. Finally, scale is a substantive aspect of complexity and the [12] H. Zhao, Raising Awareness and Signaling Quality to Uninformed Consumers: A Price-Advertising Model, Marketing need for smart/robust optimization methods. As noted earlier, Science 19:4 (2000), 390–396. Google alone gets about 40,000 searches per minute. A typical search engine may have a library of 100,000 keywords that advertisers can bid on, and there may be 100 merchants competing for, say, 10 slots for each keyword. Dynamic methods for learning and probability must, depending on factors such as frequency of search for a keyword, comprise a planning horizon of days or weeks. Given the low clickthrough rates (on the order of 1%), learning methods require a large number of observations. With every search, complex computations are required to apply the observations of user actions into revision of estimated clickthrough rates. For these reasons, sponsored search has attracted researchers in economics and auction theory, computer science, combinatorial optimization, and largescale computing. References [1] K. Asdemir, Bidding Patterns in Search Engine Auctions, In Second ACM Workshop on Sponsored Search Auctions, ACM Press, New York, NY, 2006. [2] A. Animesh, V. Ramachandran, and S. Viswanathan, An Empirical Investigation of the Performance of Online Sponsored Search Markets, In Proceedings of the 9th International Conference on Electronic Commerce (ICEC), ACM Press, New York, NY, 2007, 153–160.
ICS News B
Fall 2007
Page 5