You are here: Lean Six Sigma Home » Queueing Theory » YouTube’s Queueing Properties

YouTube’s Queueing Properties

by Pete Abilla on March 1, 2007

Interested in a free 25+ eBook on the 7 Wastes? Please DOWNLOAD HERE.

YouTube has many intriguing queueing properties.  This article will primarily look at the mpeg-to-swf conversion and study out the queueing properties of that process.

I’ll show the basic process of how to upload a video on Youtube, explain the Queueing mechanics that goes on behind-the-scenes, propose a few books in case you’re interested in Queueing Theory applications, and then finally show a video tutorial on an introduction to Queueing Theory and Models.

I make many assumptions in this article, so feel free to poke holes in what I’m arguing.

Assumptions and Data

Below is a very simple, high-level process map of the steps to upload a video on YouTube:

In summary, the user does the following:
  1. User uploads video
  2. User waits while video is uploaded
  3. Upload is completed
  4. User waits for video to be converted from MPEG to SWF
  5. Conversion from MPEG to SWF completes

Based on a small sample of 20 uploaded videos of an average file size of 3.5 MB, I calculate the mean for uploads to be ~180 seconds per MB and ~240 seconds per MB for the conversion.

Based on YouTube’s own disclosure, we also know that there are on average 65,000 video uploads per day.  We do not know the average file size.

Queueing Properties

Important items to note when studying the queueing properties of a system are the following:

  1. λ = Arrival Rate, or more specific, the time between arrivals.  For most queues, we can assume that the arrival distribution can be approximated by a Poisson distribution; which means that the time between arrivals are not deterministic, but random.
  2. μ = Service Rate, or more specific the time for a arrival to be serviced.

A poisson distribution typically looks skewed to the left or to the right — that is because the mean and the standard deviation is the same.  Here’s a standard picture of a poisson for server utilization:

What we see above is that as there are more simultaneous connections, there is a subsequent arrival rate batching — represented by the poisson curve above.

Given the data and notation, we can now attempt to better understand the queueing properties of the MPEG-to-SWF conversion.  Remember: the data I have assumes several things and, is most likely, completely off the mark.  But, it’s an attempt and, if anything, it’s fun to try.

Presuppostions Redux

So, 65,000 video uploads on a 12 hour day, gives us the following:

λ = 65,000 / 720 minutes = (90 / minute)
μ = 3.5 * 240 seconds = 840 seconds; 840 seconds / 60 = (14 conversion / minute)

Arguably, 14 conversion / minute is very low.  Let’s just assume that YouTube average service rate is 200 conversions / minute.   Given that, we can now learn about the queueing properties, which I describe below.

Average Number of Videos Waiting to be Converted

The equation to learn about the average number of files in the conversion process is the following:

Cw = (λ2 / μ(μ – λ))

So,

Cw = [(8100) / (200(200 - 90)] = .36

So, given the assumptions above, not even 1 video is waiting to be converted from MPEG-to-SWF.

Average Number of Videos in the Conversion Process

Cs = (μ – λ)

So,

Cs = [(90) / (200 - 90)] = .81

This means, given the assumptions above, that an any point in time, there is 1 video in the conversion system.

Average Time Spent Waiting

Tw = (λ / μ(μ – λ))

So, we get:

Tw = [(90) / (200(200 - 90))] = .004

This means as videos enter the conversion process, there is hardly any waiting — they are served almost immediately.

Average Time Spent in the System

Ts = (1 / (μ – λ))

So, we get,

Ts = (1 / (200 – 90)) = .009

This means, then, that as videos are uploaded and enter the conversion queue, they are served almost immediately, without any waiting.

Weaknesses in Analysis

Okay, the numbers above are pretty much pulled out of the clouds.  But, if we had real data, then you could just plug them into the equations above.  My guess is, though, that even if we had real numbers, the results would be really close to what I show above.  Why?  Well, for each input into the YouTube system, one could argue that it has very little impact on resources — this is a common property in telephony and in server modeling.  I see the same thing going on with YouTube.  The biggest challenge for YouTube is not computing resources, but storage capacity.

queueing theory computer applications, operations research, optimization, modeling, econometrics

Computer Applications, Volume 2, Queueing Systems by Leonard Kleinrock
Pete Abilla
www.shmula.com
Book Review
Oct 04, 2008
Rating: 5/5

(From Amazon) Queueing Systems Volume 1: Theory Leonard Kleinrock – This book presents and develops methods from queueing theory in sufficient depth so that students and professionals may apply these methods to many modern engineering problems, as well as conduct creative research in the field. It provides a long-needed alternative both to highly mathematical texts and to those which are simplistic or limited in approach. Written in mathematical language, it avoids the “theorem-proof” technique: instead, it guides the reader through a step-by-step, intuitively motivated yet precise development leading to a natural discovery of results. Queueing Systems, Volume I covers material ranging from a refresher on transform and probability theory through the treatment of advanced queueing systems. It is divided into four sections: 1) preliminaries; 2) elementary queueing theory; 3) intermediate queueing theory; and 4) advanced material. Important features of Queueing Systems, Volume 1: Theory include:

  • techniques of duality, collective marks
  • queueing networks
  • complete appendix on z-transforms and Laplace transforms
  • an entire appendix on probability theory, providing the notation and main results needed throughout the text
  • definition and use of a new and convenient graphical notation for describing the arrival and departure of customers to a queueing system
  • a Venn diagram classification of many common stochastic processes

Fundamentals of Queueing Theory Second Edition Donald Gross and Carl M. Harris This graduated, meticulous look at queueing fundamentals developed from the authors’ lecture notes presents all aspects of the methodology-including Simple Markovian birth-death queueing models; advanced Markovian models; networks, series, and cyclic queues; models with general arrival or service patterns; bounds, approximations, and numerical techniques; and simulation-in a style suitable to courses of study of widely varying depth and duration. This Second Edition features new expansions and abridgements which enhance pedagogical use: new material on numerical solution techniques for both steady-state and transient solutions; changes in simulation language and new results in statistical analysis; and more. Complete with a solutions manual, here is a comprehensive, rigorous introduction to the basics of the discipline.

I highly recommend this book to lean practitioners, operations researchers in any industry – highly relevant for economics and optimization and also for computer programmers interested in queueing theory.

search terms for this article:

queueing theory lectures on youtube, queuing theory VIDEo lectures, queuing theory youtube, fundamentals of queueing theory download, queuing theory models, probalility and queing theory lecture videos, queuing theory lectures videos, queuing theory tutorial pdf, queuing theory pdf, queueing theory videos, theoretical probability you tube, operations research lecture notes pdf queuing, operations research problems and solutions pdf free download queueing thoery, QUEUEING MODELS video tutorials, you tube video-queueing network, youtube queuing theory, queuing theory for dummies pdf, queueing video, queuing theory in networking in you tube, queueing theory video tutorials, queuing theory markovian models- ebooks download, queueing theory video tutorial, queueing theory video lectures, queuing models AND ADVANCE QUEUING MODELS PDF, queuing theory for computer system youtube, queuing modeling in operation research pdf file, queuing models features, queuing model video lectures, queuing models video tutorial, queuing model simulation you tube, queuing systems leonard kleinrock pdf, queuing theory application tutorial, queuing model in or, queuing system volume 1 pdf, queuing theory online video tuition, video tutorial queuing theory, waiting line and queuing theory lectures on youtube, waiting line model tutorial, waiting line theory you tube videos, you tube queuing theory, youtube file properties, youtube queue theory, youtube queueing theory waiting lines, youtube tutorial queue theory, youtubr queueing theory, video lecture about queuing theory pdf, video for learning probabilty queuing thory, single server queueing system youtube, queuing theory problems on you tube, youtube (queuing theory), queuing theory tutorial u tube, queuing theory vedio lecture, queuing theory vedio lectures, queuing theory video clip, queuing theory videos, simulation modelling operations research video lectures, simulation of queing system using monte carlo- you tube, simulation of queuing system you tube, دانلود فیلم رياضي دوم راهنمايي اصل, queueing theory video, advanced queueing models classes, freedown lood manual solution of queueing theory, fundamentals of queueing theory pdf by donald gross, fundamentals of queueing theory solution manual pdf, gross donald and carl m harris fundamentals of queueing theory pdf, important features of queuing, important features of queuing model, kleinrock, kleinrock queueing systems pdf, lecture in probability and queuing theory, lecure note on poison distribution using excel, markov chains queueing theory for dummies, free lectures on queueing theory, free download queueing systems volume i: theory, Advanced queuing models notes, applications to queueing systems simulation pdf tutorial, average youtube file size, average youtube video mb size, Computer Applications Volume 2 Queueing Systems ebookee, continuous time markov chain with rates, damodar gujarati basic econometrics, deviance on youtube, disneyland queu management you tube, download fundamentals of queueing theory: solutions manual, Elementry characteristics of queing theory, mathematical models of process control pdf free, non markovian queues and queue networks video lectures, non-markovian queues and queue networks tutorial YOUTUBE, queue theory video, Queueing models Poisson process problems and their solutions video lectures, queueing networks video tutorials, queueing system leonard kleinrock book free download, queueing systems kleinrock ebook free download, queueing systems volume 1 kleinrock download free, queueing systems volume 1: theory download, Queueing Systems Volume 2 Computer Applications download free, Queueing Systems Volume 2: Computer Applications ebook, queueing theory by donald gross download, queueing theory kleinrock free download, queue theory tutorials

Related Articles:


This post was written by

{ 1 comment… read it below or add one }

Don November 27, 2008 at 1:17 pm

I am studying this area as part of my MBA program. The potentially novel use is to apply it to the area of Information security server availability and data integrity modeling.

Do you have suggestions on how successive queue modeling might be done between computer networking inputs and Operating System resource management functions? If so, defenses against distributed denial of service attacks against servers could be devised.

Sorry for geeking out, I hope this still made sense.

Best Wishes,

Don Turnblade
CISSP
MBA (Expected 2009)

Reply

Leave a Comment