When the number of customers who have a movie at the the top of their queue is greater than the number of copies available, Netflix must decide which customers to allocate the available copies to.
I have determined that Netflix uses the number of movies you rented in your previous billing period or periods combined with your disc plan (3 out, 5 out, etc.) to determine your priority in getting movies. As your cost-per-disc to Netflix increases due to more frequent rentals, you will have less of a chance of receiving a low-availability movie compared to an individual who has a lower cost to Netflix.
Put it another way, if customer X and Y are both in the 5 disc out plan and X rented 14 discs vs. Y's 11 discs in the previous month, Y would have priority over X when they are both competing for the same movie. A side effect of this is that trial and new customers will have far fewer problems getting movies, especially new releases, versus the majority of established customers. Essentially these new and low cost customers can "cut in line" ahead of other customers.
This report describes how I came to this conclusion using automated data collection on netflix.com. While Netflix initially did not admit the practice, Netflix public relations and some of their customer service representatives are now acknowledging that customers who rent fewer movies receive priority.
Around January 2003 I started seeing "wait times" on numerous movies in my queue skyrocket. In particular some movies which were recent, mainstream, and well advertised releases were impossible to get. About a Boy was one such example. Thus I began a quest to determine what was going on. It just did not make sense that some of the popular movies in my queue were so hard to get. I searched USENET but came up empty.
The first thing I did was create a new trial account for my wife. I added three movies that I was having difficulty getting in my account. The following table shows the differences in availability between the two accounts:
| Movie |
Availability | |
| Original Account ("A") | New Trial Account ("B") | |
| About a Boy | Short Wait | Now |
| Sweet Home Alabama | Very Long Wait | Now |
| Trees Lounge | Long Wait | Now |
The snapshot was taken when the trial account was opened on the evening of 3/5/2003. Both accounts use Colleyville, TX as their service center. Here is a screen shot showing the differences:
In the screenshot above Netscape was used to display account "A" while Internet Explorer was used to simultaneously show account "B".
The difference in availabilities translates directly into your ability to receive a movie. About a Boy and Sweet Home Alabama were shipped to account "B" on 3/6. Trees Lounge did not ship until 3/10, however. But note that I had only kept these three movies in the "B" queue during the early days of my testing. I am uncertain whether it would have skipped over Trees Lounge for another movie even though Trees Lounge was listed as available "Now." This is one area I wish I would have tested a little more with some of the other accounts I had access to. I did test purging my "A" queue down to a few movies and this had no effect on availability; however, I did not keep only a small set of hard to get movies in it to see what happens when all movies in the queue are not available. Does Netflix sit on your queue or give you one of these movies anyway?
As an aside, Serving Sara and Undercover Brother are, in my humble opinion, terrible movies and I'm just a little embarrassed they were in my queue above. Not that Sweet Home Alabama is Oscar worthy, but who can't smile at Reese Witherspoon?
My initial thought was that new accounts are given priority, to get you hooked on the service and make your trial and perhaps first paid month a pleasant experience. And in a way they are, but I think through a different route than simply account age. But in this I am getting ahead of myself. More on what I think is going on behind the scenes at Netflix later.
I am not a statistician. I am not a scientist. By training I am a software engineer who has also dabbled in product management, software architecture, and low-level IT management. It has been a very long time since my basic statistics courses in college. My intention is not to create the perfect scientific study of this. My goal was simply to understand and document some very odd behavior in the Netflix system with as little effort as possible. It is very possible my conclusions are incorrect; however, I have spent significant effort to insure that, in the least, the data presented here is not flawed and is as accurate as possible.
To investigate this anomaly further I set about creating some Perl scripts to:
I am not making the scripts available. The data collected from Netflix can be browsed here. All data is in XML format and very self explanatory. I imported the comparison results into Excel and further summarized it and charted it. The Excel 97-2000 workbook is available here. You could launch this workbook to view the charts directly within Excel. An Adobe Acrobat version of the Excel workbook is available here.
I created a list of 45 movies in my queue that did not have an availability of "Now." Initially I only added these movies to account "B." I later added these same 45 movies to three other pre-existing friend and family accounts, which will be referred to as accounts "C", "D", and "E". Three of these movies eventually shipped to account "A" or "B" during the period I was collecting data, so in the end there were only 43 movies that were in common to all five accounts during the entire test period.
Keep in mind that most movies in my queue had a "Now" availability. The following table summarizes the state of my queue before testing. Future releases have been excluded.
| Availability | Number
of Movies |
Percentage of Total |
| Now | 214 | 73% |
| Short Wait | 48 | 16% |
| Long Wait | 19 | 6% |
| Very Long Wait | 12 | 4% |
| 293 |
This report does not attempt to determine what percentage or class of movies Netflix has availability problems for. It focuses solely on how movies are allocated when there are availability problems.
The five test accounts used three different Netflix service centers. A service center is the place where you are most likely to receive a DVD from and where you return your DVDs to. Two of the accounts were in the 5 movie out plan ("A" and "C") while the rest were in the 3 movie out plan ("B", "D", and "E").
An overview of the test accounts:
At first I manually started my collection script at various times of the day. I eventually used the UN*X cron scheduler to collect queue data every evening at 8:00 PM CT. Netflix batch processing seemed to be finished by this time, with availabilities changing Sunday through Thursday evenings.
Note that in a few cases these pre-cron batches were run after midnight and then another batch was run in the evening on the same date. This does not cause any problems but I wanted to point this out in case anyone inspects the raw data closely.
To aid in comparing the availability of movies in one queue to another, I created an "availability score" for each queue. I assigned a weighting to the different levels of availability
| Availability | Weighting |
| Now | 0 |
| Short Wait | 1 |
| Long Wait | 2 |
| Very Long Wait | 3 |
A queue was then assigned an overall "availability score" by adding up the weighting of each movie. So lower availability scores mean more movies are available, higher scores mean more movies are unavailable. To put it another way: low score good, high score bad!
The following table illustrates the differences between the five test accounts for one sample day (March 14th). On this date each queue had 44 movies in common. The availability score is shown in parenthesis besides the account ID.
| Movie | A (46) | B (12) | C (27) | D (27) | E (6) |
It is not clear how if any your service center location affects movie availability. In the example above "C" & "D" have the exact same availability status for each movie, yet one is in Houston, TX and the other is in Lansing, MI.
As discussed earlier, at first I thought that account age might be used to determine availability. However, account "E" was not really using their account. The account was 13 months old and had an extremely low (good) availability score. My brother had set up my Mom with an account and she was just sitting on movies. In one Netflix billing period she had only rented 2 movies and in another only 1!! Needless to say I have since cancelled the account: those were some very expensive rentals!
So what is affecting the availability score? I believe it is the number of movies you rented during your last billing cycle, or possibly last 2 or 3 billing cycle(s). Your billing cycle is shown on http://www.netflix.com/BillingHistory.
The following chart tracks availability scores during the entire test period for accounts "A" and "B". As I started testing I stopped returning movies from account "A" and started using account "B" exclusively. Thus over time A's number of rentals decreased and B's increased. The chart dramatically shows how on the first day of the new billing period for "A" (March 31st), the availability score dropped from an average of 42 for the previous billing period to an average of 25 for the new billing period. Likewise, when a new billing period kicked in for "B" (April 15th), it's availability score jumped from 13 to 40.
Remember: low score good (easier to get movies), high score bad (harder to get movies).
Each billing cycle is plotted separately and has a different symbol for a data point. For example, "A1" is in one billing cycle while "A2" is in another. For the test account I created for my wife, I showed the 10 day free trial as B1, the first paid month as B2. and the second paid month as B3.
The following table summarizes the availability stats for each test account:
The number of rentals in the previous billing period is charted against average availability score for the current billing period below. Accounts in the 3 movie plan are shown in orange while those in the 5 movie plan are shown in blue.
In version 1.0 of this report I indicated that I thought the rental plan you are in is not affecting the availability score. I.e. if customer A who is on a 5 disc plan and customer B who is on a 3 disc plan had the same rental habits over a period of time, they would have the same availability rating.
A slashdot poster suggested average rental price was in fact what is affecting availability, and this it would take into account the various rental plans. Or at least in the unlimited plans. The following chart shows per-disc rental cost charted against availability score. I have charted two data sets: the previous billing period average disc cost and the average disc cost over the last two billing periods.
The points stuck on the X axis are for "B" during the trial and 1st month. The clustering of both 3 and 5 disc plan samples in the same area does indicate that your rental plan is part of the equation! This is great news.
Given the very small data set, it is difficult to tell whether Netflix simply places renters into different priority "buckets" or fine tunes the algorithm on a per-customer basis. Note that as of the 1st quarter of 2003, on average each customer rents 5.5 discs per month. I do not know whether this is an average across all plans or only the $19.95 plan. Assuming a $19.95 plan, this translates to $3.63 per disc. This would put the average customer in the second cluster from the top above.
I present one final chart showing the availability scores of all 5 test accounts during the test period. It is really not that useful but does show how availability tends to move in lockstep between accounts. For example, a movie that was "Very Long Wait" on one account and "Long Wait" on another might change to "Long Wait" on the former and "Short Wait" on the later.
When reading the following, keep in mind that Netflix's greatest per-customer variable cost is postage and handling.
Some possible reasons for Netflix tweaking availability on a customer-by-customer basis based on rental history:
But I see problems ahead:
One other gem I gleaned from listening to the conference call. An analyst actually noticed worse availability for various movies in his queue and questioned Netflix about it. They attributed this to growth that was faster than expected and that things will improve. So Netflix may see what I describe here as a temporary issue.
What are some possible ways Netflix could "tweak" the system?
This report did not address any delays between when Netflix receives a movie and where/when it sends a new movie from your queue. Please see the discussion of "throttling" in the report's postscript.
This report was first published on April 22 2003 and appeared on Slashdot.Org, a techie news site, the next day. Many Netflix customers contacted Netflix for comment. As has been the case in the past, Netflix customer service representatives (CSRs) did not acknowledge this practice. A copy of a letter sent by a CSR to a Slashdot reader on April 23rd is available here. In this letter the CSR states that "all subscribers are treated equally, including those on free trials."
On April 25th a Slashdot reader received this response. An excerpt from the letter follows:
When demand temporarily is greater than supply we ship to customers who rent fewer movies because they have fewer opportunities to get a particular movie. We do this to give the best experience to as many members as possible.
A journalist received a similar statement from the Netflix public relations department on April 25th. Another customer, however, received a letter from Netflix on May 5th stating that "newer customers [do] not get better service." A consistent message from Netflix that is available in their online customer documentation and FAQs would help clear this up.
Early in the morning of May 1st I sent Netflix public relations a request for comment and some additional questions. I have not received a response as of late May 5th.
Netflix should:
More data collection or a response from Netflix would be necessary to determine whether only the previous month or the previous 2-3 months are used to determine availability.
In January 2005 Netflix updated their terms of service to state that customer usage impacts both DVD availability and shipping. Some excerpts as of March 26 2005:
We reserve the right to allocate and ship DVDs to you in any manner that we, in our sole and absolute discretion, determine. In addition, we will, in our sole and absolute discretion, determine the quantity of DVDs we purchase for any particular title and the level of staffing and number of shipments to be processed at each distribution center. As a result, we may not always send you the first choice from your queue, and we may not ship out your next DVD on the same day that we receive one from you. Our goal is to ship you the DVDs listed highest in your queue. We try to ship you DVDs from the distribution center closest to you so that you get movies quickly. Often, on the same day that we receive a DVD from you, we will ship the next available DVD from your queue. In certain instances, your next available DVD will not ship until the next business day following our receipt of your returned movie. This can occur, for example, when your top choices are not available to you from your closest distribution center or the number of shipments to be processed by the distribution center on that day has been exceeded. When this happens, your DVD will ship on the next business day and may come from an alternate distribution center.
In determining priority for shipping and inventory allocation, we give priority to those members who receive the fewest DVDs through our service. As a result, those subscribers who receive the most movies may experience next-day shipping and receive movies lower in their queue more often than our other subscribers.
The second paragraph was updated some point after March 2005. As of February 10 2006 it now reads:
In determining priority for shipping and inventory allocation, we give priority to those members who receive the fewest DVDs through our service. As a result, those subscribers who receive the most movies may experience that (i) the shipment of their next available DVD occurs at least one business day following return of their previously viewed movie, (ii) delivery takes longer, as the shipments may not be processed from their local distribution center and (iii) they receive movies lower in their queue more often than our other subscribers.
One Netflix practice not discussed in this report is the tweaking of the rental return and ship process for different customers ("turnaround"). As stated above, heavy renters may have a longer turnaround time than low renting customers. For example, low renters may get same day turnaround, while a heavy renter may have to an extra day.
Netflix can also choose to ship a heavy renter's DVD from a more remote distribution center. Either of these practices will slow the turnaround cycle for frequent renters, thus saving on all important postage in addition to keeping the infrequent renters happy.
Finally, many users have reported that DVDs are checked in to Netflix more slowly as their usage goes up. For example, a customer close to a distribution might normally expect an e-mail from Netflix letting them know Netflix received their return one business day after they placed the DVD in the mail. But many customers say they start to see much longer "check in" times as they rent more.
Numerous Netflix customers have contacted me in the past with anecdotal evidence of the above. Manuel Villanueva is Netflix's worst nightmare, renting on average 30 movies per month. Manuel reports very long turnaround times as a result of his heavy usage and has even developed a simple calculator to estimate turnaround time. I have not done any testing to confirm this, however, and have been renting so little that I am now a very profitable customer!
Netflix's increasing of turnaround time for heavy renters has been dubbed "throttling" by the Netflix user community. Google "netflix throttling" to learn more.
This study was mentioned in a recent LA Times article, which stated I had "watched about 20 movies a month". Just an FYI that my wife and I had only rented 14 movies in the month before I started to question the long wait times. There were 28 days during this billing cycle, so we averaged out to watching 3.5 movies per week. But we were on the 5 out plan at the time. So really Netflix cost model wise, our renting was equivalent to someone on the popular 3 at a time plan renting 2.1 movies a week.
My wife and I are no longer Netflix customers. Soon after I wrote the report, work projects and our hobbies kicked back into high gear and our DVD viewing decreased dramatically. And we enjoyed going to the "real" movies more and more.
Reverse Disclosure: I am in no way affiliated with Netflix, Blockbuster, or any DVD rental or entertainment company. I have never owned, shorted, etc. any Netflix common stock, derivatives, etc., or any of it's competitors. And I have no plans to do so. I have received no compensation to do this study or make follow-ups.