The emphasis will be on algorithm design and on algo- rithm analysis. For the analysis, we frequently need ba- sic mathematical tools. Think of analysis as the . This book is intended as a manual on algorithm design, providing access to Three aspects of The Algorithm Design Manual have been particularly beloved. Algorithm Analysis. Contribute to davie/CSAlgorithm-Analysis development by creating an account on GitHub.
|Language:||English, Spanish, German|
|ePub File Size:||24.42 MB|
|PDF File Size:||12.49 MB|
|Distribution:||Free* [*Regsitration Required]|
Kleinberg, Jon. Algorithm design / Jon Kleinberg, Éva Tardos.—1st ed. p. cm. Includes bibliographical references and index. ISBN (alk. paper). 1. Kleinberg, Jon. Algorithm design / Jon Kleinberg, l~va Tardoslst ed. p. cm. Includes bibliographical references and index. ISBN (alk. paper). 1. PDF | Algorithm Design introduces algorithms by looking at the real-world problems that motivate them. The book teaches a range of design and analysis.
One way to get an approximate sense of how fast logb n grows is to note that. The input has a natural "size" parameter N. Furthermore--and this is the trick3. First JavaPlanet opens a franchise. Taken together. Igor Kats.
A perfect matching is then a way of assigning each job to a machine that can process it. In other words. Do you see it? Matchings in bipartite graphs can model situations in which objects are being assigned to other objects. A bipartite graph is pictured in Figure 1. We typically learn something about the structure of the underlying problem from the fact that such a simple approach can be optimal.
When a greedy algorithm can be shown to find an optimal solution for al! There appears to be no simple greedy rule that walks through the intervals one at a time. We say that a graph G Some Representative Problems We will see shortly that this problem can be solved by a very natural algorithm that orders the set of requests according to a certain heuristic and then "greedily" processes them in one pass.
To see that this does capture the same notion we encountered in the Stable Matching Problem.
Y is the set of offered courses. This will be. We then defined a perfect matching to be a matching in which every man and every woman belong to some pair. In the case of bipartite graphs. Our goal will be to find a compatible subset of intervals of maximum total value. Thus the Bipartite Matching Problem is the following: Given an arbitrary bipartite graph G.
Some Representative Problems The Independent Set Problem encodes any situation in which you are trying to choose from among a collection of objects and there are pairwise conflicts among some of the objects.
No efficient algorithm is known for any of them. Who will win? There is. The Independent Set Problem is. First JavaPlanet opens a franchise. E in which the nodes are the intervals and there is an edge between each pair of them that overlap.. This is simply the largest independent set in the graph whose nodes are your friends. Competitive Facility Location Finally. Is there anything good we can say about the complexity of the Independent Set Problem?
One positive thing is the following: If we have a graph G on 1. The current status of Independent Set is this: While it is not complicated to check this. Each zone i has a 2 For those who are curious. For Interval Scheduling.
The graph in Figure 1. Given the generality of the Independent Set Problem. We will see later in the book that Independent Set is one of a large class of problems that are termed NP-compIete. Say you have n friends. It is possible that this is close to the best we can do on this problem. Bipartite Matching. We simply show you the graph G. Suppose they must deal with zoning regulations that require no two franchises be located too close together.
This may look like a very basic observation--and it is--but it turns out to be crucial in understanding this class of problems. The geographic region in question is divided into n zones. Given G. The obvious brute-force algorithm would try all subsets of the nodes. Encoding Bipartite Matching as a special case of Independent Set is a little trickier to see. It would have to implicitly contain algorithms for Interval Scheduling. So there really seems to be a great difference in difficulty between checking that something is a large independent set and actually finding a large independent set.
This process is called augmeritation. Everyone would rather marry any good person than any bad person. They also prevent two franchises from being opened in the same zone. At all times. Suppose that for some number k. Each man has a preference list that ranks all the women. The notion of PSPACE-completeness turns out to capture a large collection of problems involving game-playing and planning. One can work this out by looking at the figure for a while.
It is now easy to identify an instability in M: Show that in every stable matching. There does not seem to be a short proof we could present. This is in contrast to the Independent Set Problem.
What would it mean for the claim to be false? There would exist some stable matching M in which a good man m was married to a bad woman w. Solved Exercise 2 We can think about a generalization of the Stable Matching Problem in which certain man-woman pairs are explicitly forbidden..
So even if all of them were married to good women. In the case of employers and applicants. Solution A natural way to get started thinking about this problem is to assume the claim is false and try to work toward obtaining a contradiction.
This contradicts our assumption that M is stable. The set of all 2n people is divided into two categories: P2 will be able to select a set of nodes with a total value of at least B? We will call this an instance of the Competitive Facility Location Problem.
Could it be the case that every good woman is married to a good man in this matching M? Thus our game consists of two players..
There are k good men and k good women. Each is good. P1 and P2. Using the analogy to marriage between men and women. Suppose that player P2 has a target bound B. Then P2 does have a winning strategy. Solved Exercise 1 Consider a town with n men and n women seeking to get married to one another. Location Problem. The difficulty. The usual kind of instability. Now we can ask: For every set of preference lists and every set of forbidden pairs. To begin with.
Here is the algorithm in full. We now prove that this yields a stable matching. In this more general setting. We also notice an additional pairs of facts. Resolve this question by doing one of the following two things: To do this. A single woman is more desirable and not forbidden. We will show that there is always a stable matching. Each man m ranks all th6 women w for which m.
There are two single people with nothing preventing them from getting married to each other. That turns out to be the case here. A single man is more desirable and not forbidden. Each show has a fixed rating.
On the basis of this pair of schedules. T than it did with the pair S. We will assume that there were more students graduating than there were slots available in the m hospitals.
Here is the way we determine how well the two networks perform relative to each other. T is stable if neither network can unilaterally change its own schedule and win more time slots. There are n prime-time programming slots. A network wins a given time slot if the show that it schedules for the time slot has a larger rating than the show the other network schedules for that time slot.
Each hospital had a ranking of the students in order of preference. True or false? In every instance of the Stable Matching Problem. Gale and Shapley published their paper on the Stable Matching Problem in Suppose in the opening week of the fall season. There were m hospitals. If it is true.
But for ra to be single. Our general definition of instability has four parts: This means that we have to make sure that none of the four bad things happens. For every set of TV shows and ratings. Each network wants to devise a schedule--an assignment of each show to a distinct slot--so as to attract as much market share as possible.
Consider an instance of the Stable Matching Problem in which there exists a man m and a woman w such that m is ranked first on the preference list of w and w is ranked first on the preference list of m.
There are many other settings in which we can ask questions related to some type of "stability" principle. There were n medical students graduating in a given year. If it is false. The goal of each network is to win as many time slots as possible. That is. Some Representative Problems Exercises Suppose we have two television networks. Then in every stable matching S for this instance. Decide whether you think the following statement is true or false. The Stable Matching Problem.
For safety reasons. With indifferences in the ranldngs. As before we have a set M of n men and a set W of n women. Does there always exist a perfect matching with no weak instability? Either give an example of a set of men and women with preference lists for which every perfect matching has a weak instability. In this problem we will consider a version of the problem in which men and women can be indifferent between certain options.
Some Representative Problems Exercises 25 The interest. Does there always exist a perfect matching with no strong instability? Either give an example of a set of men and women with preference lists for which every perfect matching has a strong instability. And for each. Each of its ships has a schedule that says. Each ship visits each port for exactly one day during the month. Second type of instability: We say that an assignment of students to hospitals is stable ff neither of the following situations arises.
Show that there is always a stable assignment of students to hospitals. Since we are assuming a surplus of students. Assume each man and each woman ranks the members of the opposite gender. So we basically have the Stable Matching Problem. Show that such a set of truncations can always be found.
Given the schedule for each ship. You can assume the "month" here has m days. Peripatetic Shipping Lines. PSL Inc. The company wants to perform maintenance on all the ships this month. Furthermore--and this is the trick3. Some of your friends are working for CluNet. And similarly for the orders in which output wires meet input wires.
Here is the setup.. The basic question is: Can a man or a woman end up better off by lying about his or her preferences? More concretely. There are n input wires and rt output wires. Input 1 has its junction with Output 2 upstream from its junction with Output 1. A valid solution is to switch the data stream of Input 1 onto Output 2. Input 2 has its junction with Output 1 upstream from its junction with Output 2. We can ask the same question for men. Show that for any specified pattern in which the input wires and output wires meet each other each pair meeting exactly once.
Now consider a woman w. If the stream of Input i is switched onto Output j. Suppose we have two ships and two ports. Each input wire meets each output. It does not matter. For this problem. Some Representative Problems Exercises 27 Example. At this level of generality. Having done this.
We then develop running-time bounds for some basic algorithms. Some Representative Problems b Give an example of a set of preference lists for which there is a switch that-Would improve the partner of a woman who switched preferences. We begin this chapter by talking about how to put this notion on a concrete footing.
As discussed in the chapter. At the same time. We will discuss the problems in these contexts later in the book. In some cases.
Gusfield and Irving also provide a nice survey of the "paralle! We will look for paradigmatic problems and approaches that illustrate. Stable matching has grown into an area of study in its own right. But what is a reasonable analytical benchmark that can tell us whether a running-time bound is impressive or weak? A first simple guide. N is closely related to the other natural parameter in this problem: Even bad algorithms can run quickly when applied to small test cases on extremely fast processors.
But it is important that algorithms be efficient in their use of other resources as well. The input has a natural "size" parameter N. Another property shared by many of the problems we study is their fundamentally discrete nature. As we seek to understand the general notion of computational efficiency.
A common situation is that two very different algorithms will perform comparably on inputs of size So what we could ask for is a concrete definition of efficiency that is platform-independent. Since there are 2n preference lists. But there are some crucial things missing from this definition. In considering the problem. And indeed. As we observed earlier. We can use the Stable Matching Problem as an example to guide us. At a certain leve!. Some Initial Attempts at Defining Efficiency The first major question we need to answer is the following: How should we turn the fuzzy notion of an "efficient" algorithm into something more concrete?
A first attempt at a working definition of efficiency is the following. The first is the omission of where. Average-case analysis--the obvious appealing alternative. Proposed Definition of Efficiency 1: An algorithm is efficient if.
Before focusing on any specific consequences of this claim. The focus on worst-case performance initially seems quite draconian: This certainly is an issue in some cases. The surprising punchline. This was a conclusion we reached at an analytical level. It really works. We did not implement the algorithm and try it out on sample preference lists. In any case. There are certainly exceptions to this principle in both directions: What do we mean by "qualitatively better performance?
Proposed Definition of Efficiency 2: An algorithm is efficient if it achieves qualitatively better worst-case performance.
This will turn out to be a very usefu! For now. Not only is this approach almost always too slow to be useful. But if there is a problem with our second working definition. This will be a common theme in most of the problems we study: If the input size increases from N to 2N.
Where our previous definition seemed overly vague. Algorithms that improve substantially on brute-force search nearly always contain a valuable heuristic idea that makes them work. And so if there is a common thread in the algorithms we emphasize in this book.
The answers are. Problems for which polynomial-time algorithms exist almost invariably turn out to have algorithms with running times proportional to very moderately growing polynomials like n.
Even when the size of a Stable Matching input instance is relatively small. From this notion. Proposed Definition of Efficiency 3 " An algorithm is efficient if it has a polynomial running time. The natural "brute-force" algorithm for this problem would plow through all perfect matchings by enumeration.
Suppose an algorithm has the following property: Search spaces for natural combinatorial problems tend to grow exponentially in the size N of the input. For most of these problems. Since d is a constant. The function f n then becomes a bound on the rtmning time of the algorithm. We now discuss a framework for talking about this concept.
Our definition in terms of polynomial time is much more an absolute notion. All this serves to reinforce the point that our emphasis on worst-case.
When we provide a bound on the running time of an algorithm. One further reason why the mathematical formalism and the empirical evidence seem to line up well in the case of polynomial-time solvability is that the gulf between the growth rates of polynomial and exponential functions is enormous. In a sense. We will mainly express algorithms in the pseudo-code style that we used for the Gale-Shapley algorithm.
But overwhelmingly. In Table 2. In contrast.
When we seek to say something about the running time of an algorithm on inputs of size n. As just discussed. At times we will need to become more formal. It becomes possible to express the notion that there is no efficient algorithm for a particular problem. Each one of these steps will typically unfold into some fixed number of primitive steps when the program is compiled into which an algorithm with exponential worst-case behavior generally runs well on the kinds of instances that arise in practice.
There is a final. Asymptotically tight bounds on worst-case running times are nice things to find. The fact that a function can have many upper bounds is not just a trick of the notation.
Note that O. Given another function f n. In this example. This definition works just like 0. As an example of how this definition lets us express upper bounds on running times. By analogy with O - notation. So the most we can safely say is that as we look at different levels of computational abstraction. Often when we analyze an algorithm--say we have just proven that its worst-case running time T n is O n2 --we want to show that this upper bound is the best one possible.
This inequality is exactly what the definition of O - requires: T n grows exactly like [ n to within a constant factor. There is a notation to express this: Whereas establishing the upper bound involved "inflating" the terms in T n until it looked like a constant times n2.
We now discuss a precise way to do this. There are cases which meets what is required by the definition of f2. It is important to note that this definition requires a constant c to exist that works for all n. Just as we discussed the notion of "tighter" and "weaker" upper bounds. There was nothing wrong with the first result. To see why. Asymptotic Lower Bounds There is a complementary notation for lower bounds. Asymptotic Upper Bounds Let T n be a function--say.
More precisely. It is not hard to do this. Since the overall running time is a sum of two functions the running times of. A similar property holds for lower bounds. For a. There is also a consequence of 2. The result can be stated precisely as follows. We write this more precisely as follows. Thus we have shown 2. Sometimes one can also obtain an asymptotically tight bound directly by computing a limit as n goes to infinity. Transitivity A first property is transitivity: Sums of Functions It is also useful to have results that quantify the effect of adding two functions.
So logarithms are very slowly growing functions. The upper bound is a direct application of 2. This is a good point at which to discuss the relationship between these types of asymptotic bounds and the notion of polynomial time. This is not sloppy usage: Thus each term in the polynomial is O na. This value d is called the degree of the polynomial. So algorithms with running-time bounds like O n2 and O n3 are polynomial-time algorithms. But this is a direct consequence of 2. Since we are concerned here only with functions that take nonnegative values.
One way to get an approximate sense of how fast logb n grows is to note that. A basic fact about polynomials is that their asymptotic rate of growth is determined by their "high-order term"--the one that determines the degree. Asymptotic Bounds for Some Common Functions There are a number of functions that come up repeatedly in the analysis of algorithms.
So to complete the proof. Since f is a sum of a constant number of functions. One can directly translate between logarithms of different bases using the following fundamental identity: Using O - notation. We state this more formally in the following claim. To take another common kind of example. Such algorithms are also polynomial time: In this book.
So asymptotically speaking. In the Stable Matching Problem. The very first question we need to discuss is how such a ranking wil! An important issue to note here is that the choice of data structure is up to the algorithm designer.
Arrays and Lists To start our discussion we wi! In order to asymptotically analyze the running time of. Just as people write O log rt without specifying the base. And as we saw in Table 2. This is not entirely fair.
In order to implement the algorithm. One way to summarize the relationship between polynomials and exponentials is as follows. The implementation of basic algorithms using data structures is something that you probably have had some experience with.
Taken together. We can answer a query of the form "What is the ith element on the list? Logarithms grow more slowly than polynomials. To get such a bound for the Stable Matching algorithm. Such an array is simple to implement in essentially all standard programming languages. If we want to determine whether a particular element e belongs to the list i. To get this process started. Maybe the simplest way to keep a list of rt elements is to use an array A of length n. Unlike the liberal use of log n.
This operation is Before deleting e: Element e After deleting e: Element e Figure 2. A schematic illustration of part of such a list is shown in the first line of Figure 2.
An array is less good for dynamically maintaining a list of elements that changes over time. A doubly linked list can be modified as follows. Unlike arrays. Implementing the Stable Matching Algorithm Next we will use arrays and linked lists to implement the Stable Matching algorithm from Chapter 1.
To insert element e between elements d and f in a list. We also include a pointer Last. Given the relative advantages and disadvantages of arrays and lists. It is generally cumbersome to frequently add or delete elements to a list that is maintained as an array. Next that contains a pointer to the next element in the list. We have already shown that the algorithm terminates in at most n2 iterations.
Prey to point to e. We discuss how to do this now. To ensure this. As discussed earlier. For simplicity. In a linked list.. An alternate. Such a record would contain a field e.. The deletion operation is illustrated in Figure 2. This allows us to freely choose the data structure that suits the algorithm better and not be constrained by the way the information is given as input. Inserting or deleting e at the beginning of the list involves updating the First pointer.
While lists are good for maintaining a dynamically changing set. By starting at First and repeatedly following pointers to the next element until we reach null. A generic way to implement such a linked list. We can create a doubly linked list. Prev that contains a pointer to the previous element in the list.
If the array elements are sorted in some clear way either numerically or alphabetically. Chapter 2 Basics of Algorithm Analysis 2. To delete the element e from a doubly linked list. We also have a pointer First that points to the first element. To do this we will haye two arrays. We delete m from the list if he becomes engaged. Note that the amount of space needed to give the preferences for all 2rt individuals is O rt2. We need to identify the highest-ranked woman to whom he has not yet proposed.
We can do this by maintaining an array Current of length n. Learning to recognize these common styles of analysis is a long-term goal.
While O rt is still polynomial. When we need to select a flee man. In approaching a new problem. We need to be able to identify a free man. We need to consider each step of the algorithm and understand what data structure allows us to implement it efficiently. To sum up.
Assume w is already engaged. At the start of the algorithm. The discussion of running times in this section will begin in rhany cases with an analysis of the brute-force algorithm. Consider a step of the algorithm. We need. Earlier we discussed the notion that most problems have a natural "search space"--the set of all possible solutions--and we noted that a unifying theme in algorithm design is the search for algorithms whose performance is more efficient than a brute-force enumeration of this search space.
If a man m needs to propose to a woman. Current[w] is initialized to this null symbol for all women w. This allows us to execute step 4 in constant time. To get things under way. To do this we will need to maintain an extra array Next that indicates for each man m the position of the next woman he wil!
We set Current [w] to a special null symbol when we need to indicate that woman w is not currently engaged. We need to have a preference list for each man and for each woman.
Next [re]I. Now assume man m proposes to woman w. Other algorithms achieve a linear time bound for more subtle reasons. Linear Time An algorithm that runs in O n.
One basic way to get an algorithm with this running time is to process the input in a single pass. One way to think about designing a better algorithm is to imagine performing the merging of the two lists by hand: Suppose the numbers are provided as input in either a list or an array. We process the numbers an in order. We now describe an algorithm for merging two sorted lists that stretches the one-pass style of design just a little.
Sometimes the constraints of an application force this kind of one-pass algorithm on you--for example. Each time we encounter a number ai. A B Figure 2. If you look at the top card on each stack. Two different subareas of algorithms. But this clearly seems wasteful. Merging Two Sorted Lists Often. Computing the Maxirrturrt Computing the maximum of n numbers.
To illustrate some of the ideas here. Note that this algorithm requires O rt log n time to sort the numbers.. We have just seen that the merging can be done in linear time. An element can be charged only once.
The number b1 at the front of list B will sit at the front of the list for iterations while elements from A are repeatedly being selected. Quadratic time also arises naturally from a pair of nested loops: An algorithm consists of a!
The distance between points xi. But there are only 2n elements total.. The brute-force algorithm for finding the closest pair of points can be written in an equivalent way with two nested loops: For each input point xi.. The largest of these gaps is the desired subinterval. The better way to argue is to bound the number of iterations of the While loop by an "accounting" scheme. The natural brute-force algorithm for this problem would. What is the running time of this algorithm? This is a correct boflnd.
Suppose that n is an even number.. In Chapter 3 we will see linear-time algorithms for graphs that have an even more complex flow of control: O rt log n Time O n log n is also a very common running time. Suppose we charge the cost of each iteration to the element that is selected and added to the output list. While this merging algorithm iterated through its input lists in order. Each iteration involves a constant amount of work. Sorting is perhaps the most well-known example of a problem that can be solved this way.
More crudely. One also frequently encounters O n log n as a running time simply because there are many algorithms whose most expensive step is to sort the input. This example illustrates a very common way in which a rtmning time of O n2 arises: Multiplying these two factors of n together gives the running time..
What is the running time needed to solve this problem? At first. Note how the "inner" loop. O nk Time In the same way that we obtained a running time of O n2 by performing bruteforce search over all pairs formed from a set of n items.
The following is a direct way to approach the problem. Recall that a set of nodes is independent if no two are joined by an edge. In Chapter 5 we describe a very clever algorithm that finds the closest pair of points in the plane in only O n log n time.
Cubic Time More elaborate sets of nested loops often lead to algorithms that run in O n3 time.
The natural brute-force aigorithm for this problem would enumerate all subsets of k nodes. Multiplying these three factors of n together. Multiplying these two together. This is a recurring kind of dichotomy in the study of algorithms: In the case of Independent Set.
The definition of an independent set tells us that we need to check. Thus see that 2n arises naturally as a running time for a search algorithm that must consider all subsets. Independent Set is a principal example of a problem believed to be computationally hard. The function n! To see this. Beyond Polynomial Time The previous example of the Independent Set Problem starts us rapidly down the path toward running times that grow faster than any polynomial.
Multiplying all these choices out. But in the case of the Interval Scheduling Problem. Search spaces of size n! The total number of subsets of an n-element set is 2n. Inside this loop. Hence this is a search over pairs. Since we are treating k as a constant here. Despite this enormous set of possible solutions. In Chapter 7. We assume that the salesman starts and ends at the first city.
A basic problem in this genre is the Traveling Salesman Problem: Inside the loop. Priority Queues 57 implicit search over all orders of the remaining n. Priority queues will be useful when we describe how to implement some of the graph algorithms developed later in the book. So how large is the "active" region of A after k probes? It starts at size n.
Priority queues support the addition and deletion of elements from the set. Sublinear Time Finally. Priority Queues Our primary goal in this book was expressed at the outset of the chapter: The crucial fact is that O log n such iterations suffice to shrink the input down to constant size. So the running time of binary search is O log n. Since it takes linear time just to read the input. A priority queue is designed for applications in which elements have a priority value. In this section.
In Chapter 8. Some complex data structures are essentially tailored for use in a single kind of algorithm. A priority queue is a data structure that maintains a set of elements S.
Once one has an efficient algorithm to solve a problem. Our implementation of priority queues will also support some additional operations that we summarize at the end of the section. For our purposes here. Given a sorted array A of n numbers. A motivating application for priority queues. We could do this by reading the entire array.
In general. In such situations. Perhaps the best-known example of this is the binary search algorithm. The point is that in each step. Given this.
Each process has a priority. We will think of the heap nodes as corresponding to the positions in this array. Then extract the smallest number one by one until all numbers have been extracted. Set up a priority queue H. The tree will have a root. In Figure 2. Suppose we want to add s with key value key s. Before discussing the implementation. Chapter 3 Graphs. Chapter 4 Greedy Algorithms. Chapter 5 Divide and Conquer.
Chapter 6 Dynamic Programming. Chapter 7 Network Flow. Chapter 8 NP and Computational Intractability. A Class of Problems beyond NP.
Chapter 10 Extending the Limits of Tractability. Chapter 11 Approximation Algorithms. Chapter 12 Local Search. Uploaded by SlekPanther on January 1, Search the history of over billion web pages on the Internet. Books by Language. EMBED for wordpress. Want more? Advanced embedding details, examples, and help!