What do the ratings mean?

The ratings represent the video streaming quality you can expect (at least 90% of the time) when you watch YouTube on an Internet Service Provider in a specific area.

YouTube HD Verified: Users on YouTube HD Verified networks should expect smooth playback most of the time when watching high-definition YouTube videos (720p and above).
Standard Definition: Users on networks rated as Standard Definition should expect smooth playback on standard-definition YouTube videos (360p) and may experience occasional interruptions on high-definition YouTube videos (720p and above).
Lower Definition: Users on networks rated as Lower Definition may experience fuzzy picture quality and frequent interruptions while playing YouTube videos at 360p and above.

Why is the data useful?

Rather than being based on data from a small sample of users, this report is based on billions of YouTube videos watched across thousands of ISPs.

We look at how quickly all YouTube video data was loaded over the last 30 days.
We segment the results by ISP and by geographical location.
We determine what the minimum available speed was at least 90% of the time.

How do you protect my privacy?

This is anonymized data about everyone watching YouTube on an ISP.

Ratings are centered around networks, not users.
All samples are completely anonymized and no user information is stored or used.
We will only show the results for a geographic area big enough to have a lot of users.

Abstract

There are many factors that can independently affect user experience on the internet, including - but not limited to - a user's internet connection speed, reliability of the access network, availability and load characteristics of the application servers and in some cases, the configuration of the users' in-home network. Individual and isolated measures like access speed or server capacity do not capture the real user experience. An end to end, application-level performance measurement that includes all the influencers in the equation is the right approach to measure and quantify the true internet user experience.

Presented here is a methodology to rate Internet Service Providers (ISPs) in terms of YouTube video capability, based on sustained application level performance measurements. The objective is to present a rating that is meaningful, easy to understand and one that closely reflects the real world internet experience.

Methodology

Measurements

A typical YouTube video playback consists of a YouTube client (player) fetching video bytes in a streaming fashion from a YouTube server (CDN), in one or more requests (e.g. HTTP GET). The first step in determining ISP ratings is to measure the sustained speed at which these video bytes are transferred from server to the client. To measure the achieved application level throughput (goodput), the following are recorded for each request:

1) Request Identity: The originating request's timestamp, access network (e.g. network block, autonomous system number of ISP) and the coarse geographical location (e.g. country, metro), derived from client attributes such as IP address, User Agent, etc. Note that the IP to location translation done by our automated systems may return a location that is incorrect for some users.
2) Response Size: The number of application bytes (including application headers but excluding any kernel level overhead) transferred by the server to the client, in response to the request.
3) Response Time: The time taken to service the request by server, including network transmission time (all bytes acknowledged by the receiver).

Based on these measurements, the goodput for a given request 'R' is computed using the formula below. Each measured request is considered a goodput sample.

Goodput

R

= Response Size

R

/ Response Time

R

Ratings

Ratings are derived by aggregating relevant goodput samples recorded in the measurements phase. The methodology supports ratings to be computed at various levels of granularity, for the selected dimensions. For example, the rating for an ISP could be calculated for various time slices (e.g. hour, day, week, month) and/or at various geographical levels (e.g. country, province, metro, city).

For a given time period 'T' (e.g. trailing 30 days) and a geographical location 'L' (e.g. San Francisco, CA, USA), the rating for an ISP 'P' (e.g. Comcast) is computed as follows:

1) Aggregate relevant samples: Collect all goodput samples that are identified with 'T', 'L' and 'P'. i.e. requests that were originated from within the target ISP, from the target location in the target timeframe.
2) Compute GAT Volume (Goodput Above Threshold): Place each goodput sample in one of the three capability buckets - HD (High Definition), SD (Standard definition) or LD (Lower definition) based on the goodput thresholds specified in the table below. This provides the raw aggregated GAT Volume for the selected dimensions.

GAT Bucket	Goodput Threshold	Reasoning
HD (High Definition)	> 2.5 Mbps	Minimum goodput required to sustain an average YouTube HD video playback at 720p resolution
SD (Standard Definition)	0.7 to 2.5 Mbps	Minimum goodput required to sustain an average YouTube SD video playback at 360p resolution
LD (Lower Definition)	< 0.7 Mbps	Goodput too low to sustain YouTube SD video playback at 360p resolution

Volume (GAT)

HD Samples

SD Samples

LD Samples

3) Define Rating Criteria: The ISP rating criteria is defined in terms of minimum level of GAT volume requirement for each rating level. Since this metric is designed to reflect consistency and reliability of the ISP's network, the bar needs to be set at a level that captures sustained performance rather than typical (average) performance. To that effect we define three rating scales: GAT-90 (90% of requests above threshold), GAT-95 (95% of requests above threshold), GAT-99 (99% of requests above threshold) to reflect different levels of reliability.

The following table defines the criteria used to determine the final ISP rating in our methodology, using GAT-90. The 90% bar is chosen after careful consideration of observed practical performance in the field. The bar will closely follow the evolving network capability over time.

Rating	Criteria (GAT-90)	Reasoning
HD (High Definition)	90+% samples are marked HD	Network offers consistent and reliable YouTube HD (720p) performance
SD (Standard Definition)	90+% samples are marked at least SD	Network offers consistent and reliable YouTube SD (360p) performance
LD (Lower Definition)	Neither of the Above	Network offers unreliable YouTube performance

4) Assign Rating: Stack the three GAT buckets from step 2 to convert raw GAT volume into GAT percentage. Determine the final rating using the criteria defined in step 3. In the illustration below, the assigned rating is SD (standard definition) since 90% or more requests meet this criteria. (i.e. the network is capable of offering at least SD quality for more than 90% of the served requests.)

Percentage

100

90

80

70

60

50

40

30

20

10

0

Rating at 90%: SD

Privacy

The ratings are centered around networks, not users. All goodput samples are completely anonymized and no user information (e.g. browser cookies, IP address) is persisted or used directly in the rating algorithm. Furthermore, if the aggregated sample volume for the selected geo level and time interval is below a certain threshold, the algorithm would fall back to using more coarse grained dimensions (i.e. aggregate by broader geo and/or time interval) that meet the minimum size requirements to compute the rating.

Frequently Asked Questions

1) ISPs sell many different internet products that promise different speeds. Does the methodology rate each product and speed separately?
Each ISP can provide appropriate product names and speeds for its different offerings, and the labels in the report depend on the information we have from the ISP. In some cases, ISPs will provide us information that allows us to identify and rate the different products and speeds. In other cases, we don’t have detailed information about the speeds offered to consumers, so we provide a general rating of the product across all speeds.
2) Why is the bar set high at 90%? Why not use mean or median?
At Google, we strongly believe that consistent and reliable network performance is key to a great user experience on the internet. Hence the choice of 90% bar. I.e. one can expect the rated level of performance 9 out of 10 times. While mean and median values are useful to represent average network performance, they are not well suited for the purpose of the rating metric which tries to capture the sustained level of the network performance.
3) Does this methodology recognize and reward networks that offer sustained high performance during high service usage hours?
Yes. Since the GAT volume is request (load) based, high YouTube consumption time periods (e.g. evenings) would contribute more towards the GAT volume pool, thereby offering a disproportionate influence on the final rating. The methodology is naturally normalized for usage.
4) What type of server requests are qualified to be included in ratings computation?
Requests serving video in modern delivery formats (e.g. DASH, HLS) are included in the ratings computation. Legacy video formats (e.g. non-adaptive progressive playbacks, RTSP) are excluded due to lack of high measurement confidence in older streaming delivery mechanisms. This is done in the interest of maintaining high degree of accuracy in computed ISP ratings.
5) What are factors within the user’s control that may influence their video quality experience?
Individual results may vary depending on factors such as: the number of connected users or devices, your hardware, your software and software configuration, your connection to the internet (wireless of fixed), and whether you have exceeded your allotted data quota.
6) For networks: Why aren’t my network’s results displaying?
We strive to provide the most accurate results to our users. Using certain data thresholds and heuristics we only include networks available for residential use. If your results are not displaying or if you believe your results have been incorrectly categorized, please contact us at: hdverified@google.com