Recent from talks
Yao's principle
Knowledge base stats:
Talk channels stats:
Members stats:
Yao's principle
In computational complexity theory, Yao's principle (also called Yao's minimax principle or Yao's lemma) relates the performance of randomized algorithms to deterministic (non-random) algorithms. It states that, for certain classes of algorithms, and certain measures of the performance of the algorithms, the following two quantities are equal:
Yao's principle is often used to prove limitations on the performance of randomized algorithms, by finding a probability distribution on inputs that is difficult for deterministic algorithms, and inferring that randomized algorithms have the same limitation on their worst case performance.
This principle is named after Andrew Yao, who first proposed it in a 1977 paper. It is closely related to the minimax theorem in the theory of zero-sum games, and to the duality theory of linear programs.
Yao's principle is formulated in terms of an arbitrary real valued cost measure of an algorithm on an input , such as its running time, for which one wishes to study the expected value over randomized algorithms and random inputs. The algorithms used in this cost measure are drawn from a finite set of deterministic algorithms; a typical way to make a problem have only a finite set of algorithms is to restrict its inputs to a single size. The inputs, too, should be drawn from a finite set , which can be made finite in the same way. Then, each probability distribution over corresponds to a randomized algorithm that first makes a random choice from according to that distribution and then follows the chosen algorithm; in this way, the class of randomized algorithms for the same problem can be modeled as the class of all probability distributions over . Finally, the formulation of Yao's principle involves the class of all probability distributions on inputs in , denoted as . Then, Yao's principle states that:
Here, is notation for the expected value, and means that is a random variable distributed according to . The left hand side of the formula gives the optimal performance that can be obtained by a deterministic algorithm on a random input (its average-case complexity), for a probability distribution on inputs that is as hard as possible and with being the algorithm that performs best against . The right hand side gives the optimal performance that can be obtained by a random algorithm on a deterministic input (its expected complexity), when has the best performance on its worst case inputs, and when is a worst case input to . The finiteness of and allows and to be interpreted as simplices of probability vectors, whose compactness implies that the minima and maxima in these formulas exist.
Another version of Yao's principle weakens it from an equality to an inequality, but at the same time generalizes it by relaxing the requirement that the algorithms and inputs come from a finite set. The direction of the inequality allows it to be used when a specific input distribution has been shown to be hard for deterministic algorithms, converting it into a lower bound on the cost of all randomized algorithms. In this version, for every input distribution , and for every randomized algorithm in , That is, the best possible deterministic performance against distribution is a lower bound for the performance of each randomized algorithm against its worst-case input. This version of Yao's principle can be proven through the chain of inequalities each of which can be shown using only linearity of expectation and the principle that for all distributions. By avoiding maximization and minimization over and , this version of Yao's principle can apply in some cases where or are not finite. Although this direction of inequality is the direction needed for proving lower bounds on randomized algorithms, the equality version of Yao's principle, when it is available, can also be useful in these proofs. The equality of the principle implies that there is no loss of generality in using the principle to prove lower bounds: whatever the actual best randomized algorithm might be, there is some input distribution through which a matching lower bound on its complexity can be proven.
When the cost denotes the running time of an algorithm, Yao's principle states that the best possible running time of a deterministic algorithm, on a hard input distribution, gives a lower bound for the expected time of any Las Vegas algorithm on its worst-case input. Here, a Las Vegas algorithm is a randomized algorithm whose runtime may vary, but for which the result is always correct. For example, this form of Yao's principle has been used to prove the optimality of certain Monte Carlo tree search algorithms for the exact evaluation of game trees.
Hub AI
Yao's principle AI simulator
(@Yao's principle_simulator)
Yao's principle
In computational complexity theory, Yao's principle (also called Yao's minimax principle or Yao's lemma) relates the performance of randomized algorithms to deterministic (non-random) algorithms. It states that, for certain classes of algorithms, and certain measures of the performance of the algorithms, the following two quantities are equal:
Yao's principle is often used to prove limitations on the performance of randomized algorithms, by finding a probability distribution on inputs that is difficult for deterministic algorithms, and inferring that randomized algorithms have the same limitation on their worst case performance.
This principle is named after Andrew Yao, who first proposed it in a 1977 paper. It is closely related to the minimax theorem in the theory of zero-sum games, and to the duality theory of linear programs.
Yao's principle is formulated in terms of an arbitrary real valued cost measure of an algorithm on an input , such as its running time, for which one wishes to study the expected value over randomized algorithms and random inputs. The algorithms used in this cost measure are drawn from a finite set of deterministic algorithms; a typical way to make a problem have only a finite set of algorithms is to restrict its inputs to a single size. The inputs, too, should be drawn from a finite set , which can be made finite in the same way. Then, each probability distribution over corresponds to a randomized algorithm that first makes a random choice from according to that distribution and then follows the chosen algorithm; in this way, the class of randomized algorithms for the same problem can be modeled as the class of all probability distributions over . Finally, the formulation of Yao's principle involves the class of all probability distributions on inputs in , denoted as . Then, Yao's principle states that:
Here, is notation for the expected value, and means that is a random variable distributed according to . The left hand side of the formula gives the optimal performance that can be obtained by a deterministic algorithm on a random input (its average-case complexity), for a probability distribution on inputs that is as hard as possible and with being the algorithm that performs best against . The right hand side gives the optimal performance that can be obtained by a random algorithm on a deterministic input (its expected complexity), when has the best performance on its worst case inputs, and when is a worst case input to . The finiteness of and allows and to be interpreted as simplices of probability vectors, whose compactness implies that the minima and maxima in these formulas exist.
Another version of Yao's principle weakens it from an equality to an inequality, but at the same time generalizes it by relaxing the requirement that the algorithms and inputs come from a finite set. The direction of the inequality allows it to be used when a specific input distribution has been shown to be hard for deterministic algorithms, converting it into a lower bound on the cost of all randomized algorithms. In this version, for every input distribution , and for every randomized algorithm in , That is, the best possible deterministic performance against distribution is a lower bound for the performance of each randomized algorithm against its worst-case input. This version of Yao's principle can be proven through the chain of inequalities each of which can be shown using only linearity of expectation and the principle that for all distributions. By avoiding maximization and minimization over and , this version of Yao's principle can apply in some cases where or are not finite. Although this direction of inequality is the direction needed for proving lower bounds on randomized algorithms, the equality version of Yao's principle, when it is available, can also be useful in these proofs. The equality of the principle implies that there is no loss of generality in using the principle to prove lower bounds: whatever the actual best randomized algorithm might be, there is some input distribution through which a matching lower bound on its complexity can be proven.
When the cost denotes the running time of an algorithm, Yao's principle states that the best possible running time of a deterministic algorithm, on a hard input distribution, gives a lower bound for the expected time of any Las Vegas algorithm on its worst-case input. Here, a Las Vegas algorithm is a randomized algorithm whose runtime may vary, but for which the result is always correct. For example, this form of Yao's principle has been used to prove the optimality of certain Monte Carlo tree search algorithms for the exact evaluation of game trees.