09:23:14 >>Professor:  good morning everyone.  We will get started in one minute.  Good morning everybody.  Let me start the recording.  I hope you all had a nice weekend.  The solutions, by the way, to home work one have now been posted.  Please do take a look at those.  If you have any -- if you are unsure about the problems from home work one and if there's any confusion you have on home work two it may be helpful to look those over as well.
09:23:53 One point I want to make about the solution I which is mentioned in the solution file itself is in general our solutions will be given in somewhat greater detail than we ask you to do.  Don't worry if your solutions are not quite as detailed as those.  We are trying to make our solutions as useful as possible.  We sometimes go in to a greater depth than we asked you to on the question.  Just to recap last time we ended by briefly introducing this concept of a priority Q.
09:24:33 It's a useful data structure that's important when we talk about graph algorithms which we will start later this week.  A priority Q is a way of keeping track of some set of items.  Each of which has a key which is some number.  We assume it's an integer or floating number which indicates priority in the Q.  If the keys were integers maybe priority one would be the highest priority.  If we have an item with priority ten that's less important item in the Q.
09:25:11 The data structure is useful for when you need to keep track of things the algorithm has to do.  Some of which need to do before others.  This priority gives you a way to add things in to the Q in any order, but then pop them off of the Q in priority order.  It could be you already have a number of things in the the last, in the Q.  At some later point you realize there's something I have to do urgently.  You can add it in the Q with a high priority.
09:25:46 Or a small numerical key.  It would be removed from the Q earlier.  Something could be in the Q for a long time but if it has higher priority it will be handled first.  This is first in first out Q.  The order in which you add things to the Q is the same order they come off the Q.  The thing that's been in the Q for the longest period of time would be the first thing you process.  Here it's not which item has been on the Q for the longest period of time.
09:26:38 It's which item on the Q has the highest priority or smallest key value.  That's the priority Q.  We will see applications of these Q's later this week.  Let's define more formally what the operations that a priority Q needs to support.  I will write PQ for priority Q.  There's different things you need to do with a priority Q.  One of the most obvious is we need to add items to the Q.  We call that the insert operation.  This is adding a new item.
09:27:30 Let's say X.  With key K.  We are adding a new item in to the Q with a given key value.  That's the basic way you would add things onto the Q.  How do you get things off?  We have a few different variants.  Sometimes you want the highest priority without removing it from the Q.  That's finding the minimum.  This is just finding the item with the least key value.  You may have ten items in the Q.  This is returning the one that has the smallest key.
09:28:25 Most often what you will do when you find the smallest or highest priority items, let's say the one with the smallest key, is you do something to it and remove it from the Q because you don't need to process it anymore.  We have a delete Min operation.  Which is like find Min but it removes it.  This is the equivalent of the pop operation for a normal Q.  This is what removes the next thing to process from the Q.  Okay.
09:29:06 Sometimes, for some algorithms you don't necessarily just want to delete the item with the smallest priority.  It could be something you added to the Q you don't want to deal with anymore.  You may want to remove another item.  We have a generic delete which is remove a particular item from the Q.  Here we will assume you have some kind of pointer to the item.  You have some reference that's specific to the data structure.
09:30:01 That identifies item X letting us remove it from the Q.  We say given a pointer.  Depending on what type of data structure you have this could be various types of things.  You can't see what I'm writing.  I forgot to spotlight the video.  Okay.  Can you see this now?  No.  Okay.  Some of you can.  Some can't.  Okay.  It looks like most of you can see this.  Hopefully, it's working.  If you have trouble you could try disconnecting and reconnecting.
09:30:49 It looks like most people can see it.  The difference between the delete Min and delete operations here is that the delete Min you don't specify which item you want to remove.  It automatically deletes what has the least key.  In the delete key you are specifying a specific item.  Something you added you are saying I don't want this in the Q.  The last main operation we need for some algorithms you need the ability to change the priority of an item after you add it to the Q.
09:31:37 For that we have a decrease key operation.  Which is what you expect.  Change the key of item X to a smaller value K.  The reason that we are saying you are just decreasing the key value rather than changing it to some other value like maybe you could increase it is for some data structures it's more efficient to do the (INDISCERNIBLE).  We need the ability to decrease and make it higher priority than it was before.  That's what the decrease operation is doing.
09:32:33 Decrease key says take item X in the Q and give it to higher priority than before.  Decrease the key value to a smaller value.  I will mention here, although I don't think we will use any algorithms that use this, but sometimes for some algorithms you need a further operation merge or sometimes called meld or join sometimes.  Which is combine two disjoin priority Q's.  You could imagine you have some priority Q that has items in it.
09:33:04 You have a second priority Q with different items in it.  Can we combine them for a single priority Q?  That's the merge or meld operation which I will not talk much about because I don't think we need it.  That's also an operation people look at.  A question here, why don't we have a look up function?  The equivalent of a look up function here for us is to find Min.  Find the element which has the least key.  If you want to look up the key for a particular item,
09:33:38 We are assuming that's built in.  If you have appointer to the item we are basically storing along with each of the items the key value.  That's a free operation.  Look up the key of a given item.  We don't have a separate operation for that.  If you want to do that you could just -- if the data structure didn't actually provide that for you, you could have a hash table to store the key value.  Usually that's not necessary in the algorithms we are talking about.
09:34:17 In whatever data structure the algorithm has for storing these items we will store the keys with the items.  Here we are concerned with how can we efficiently find the cheapest element.  The highest priority element and look it up an remove it as we are constantly adding new elements so the contents of the Q is changing.  Before we talk about implementation of a priority Q any questions about what these operations are supposed to do?
09:35:20 Okay.  This is what we want for a priority Q.  We need to have away of keeping track of items with keys so you can implement these five different operations.  How can we actually do this?  There are several different, actually more than several.  There's many different data structures that can be used to implement a priority Q.  I will talk about several.  There's a couple or more sophisticated data structures for this described in CLRS.
09:36:11 There's further data structures that have been developed over the years.  This is an important type of data structure.  There's been a lot of research on how to most efficiently implement a priority Q.  We will mention some of the most basic yet most useful data structures for this.  One thing I should mention which is if you are looking up priority Qs in some other text books or in CL RS.  In some text books they reverse the ordering of the keys.
09:36:51 What you are trying to do with each operation is find the item with the greatest key value or delete the item with the greatest key value.  They have a find max operation or delete max operation.  When you implement this you flip all the comparisons.  Instead of less than you have greater than.  The way you implement it is other wise the same.  Formally the type of priority Q we are looking at is called a Min Q.  Min priority Q.
09:37:37 Sometimes you will see a max priority Q which is just this where everything has been flipped.  It's find max, delete max and increase key.  Don't get confused.  It's the same data structure but with the keys flipped around.  Instead of higher keys being higher priority it (INDISCERNIBLE).  How do we implement a priority Q?  One of the basic implementations is to have a list of elements.  You could have an unassorted list.
09:38:22 What does this mean?  The idea is in the list we have items and keys stored in a list.  We have a linked list.  The first element of the linked list has item one with its key and item two with its key, et cetera.  If we had a linked list like this the advantage of such a representation is some of these operations can now be done quickly.  For example, if you want to insert an item that's constant time.  All you have to do is add it on the end of the list.
09:38:53 You can do that in constant time.  Similarly, if you want to delete a particular item that's constant time.  You just go to the part of the linked list and you have appointer there and remove that item from the list that's constant time.  Decrease key.  Also constant time.  Which just go to the particular item X using the pointer an change the key that's stored there.  The problem with just having an unassorted linked list is these minimum operations.
09:39:35 They require us to search through the list because it's not in any given order.  If we want to find the order with the least key order we have to iterate through the entire list potentially.  In order to find the least thing we have to iterate through the entire list.  Keep track of the smallest key we found and return that smallest item.  Find Min and delete Min are linear time.  I will summarize that.  This gives you constant time.  Insert, delete and (INDISCERNIBLE).
09:40:39 But linear time you find Min and delete Min.  Do you see that?  If you have no order in the linked list it's fast to add things and remove them and change their key, but since there's no order if you want to find the minimum you have to do a linear time reversal of the items.  If you have a large number of items and you are looking up the smallest item each time this could be very slow.  Questions on this?  Why is decrease key constant time?
09:41:23 We are asumming for delete and decrease key when identifying item X we have some pointer to item X.  In a linked list that means we have appointer to the element, the node of the link list where item X is stored.  That means delete constant time.  We use the pointer to find the node where it's stored and remove it to the list and up date on either side if we have a double link list.  Where as, decrease key if we have appointer to item X we have to change the key stored
09:42:09 At the node.  At the node for item X we store the key item.  If we change the key we change the number.  That's a constant time operation.  Okay.  That's the advantage of having this linked list not be in any order.  If we change the key of a particular element or if we add a new element with a lower key we don't rearrange anything we put them where ever.  If you have inserting and deleting and decrease keys in our algorithm but don't need find or delete Min,
09:42:51 This is a good way to implement a priority Q.  We will see examples of that.  If you do find Min and delete Min this will be slow.  Every one of those operations is linear time.  One thing you could try to improve on this would be why don't we sort this list.  If we keep the list in order then it will be quick to find the cheapest item.  The item with the least key.  This is another implementation.  It's a assorted link list.  We store items in the key in the link list like before.
09:43:45 Now we enforce the linked list has to be in order.  In increasing order of the key.  Now, if we have this list in order that now means that find Min and delete Min will be constant time.  The minimum is the first item in the list.  It's always there in the front.  We have constant time.  Delete will still have constant time delete because you have to delete the item from the list.  Find Min and delete Min will be constant time.
09:44:23 The Min is at the front of the list.  Finding it constant time.  Removing it constant time.  On the other hand, now that it's assorted inserting and decrease key will take more time.  If we insert something we can't just put it it in any where in the list.  We have to keep this list in assorted order.  We have to go through the list and find what's the right place to put the new item.  Potentially, again we have to search through the entire list to find the correct place to put in the item.
09:45:06 That will take linear time.  Insert will take linear time in the worse case.  Likewise with decrease key because you could have had an item toward the end of the list.  Now that you decreased the key you have to search backward through the list to find the new place to put it.  It could move through the list.  This is the trade off you get if you switch from an unassorted link list to a assorted one.  That speeds up these operations that involve finding the minimum.
09:45:55 Inserting and changing the key of an item could take linear time in the worst case.  Do you see that?  Is there a benefit to using one over the other?  Absolutely.  It's not like one is better than the other.  Which one is better depends on how you use the priority Q.  Imagine you had some algorithm where you use a priority Q internally.  If your algorithm for example needs to do a large number of find and delete Mins but few insertings and decrease keys.
09:46:27 Then maybe you want to go with a assorted link list.  Most of the operations you do are constant time compared to linear time.  It's not that one is better than the other.  It's a trade off depending on how often you use each of these five operations one may be faster than the other.  We are going to see this later in the week when we see graph algorithms that use priority Q's.  Depending on how the algorithm works one of these may make more sense than the other.
09:47:13 Another question here.  Why would decrease key be linear time here?  Finding the current key of the item is constant time.  We are assuming in decrease key.  These operations delete and decrease key these that operate on a particular item X we are assuming you are given appointer to that item.  Given appointer to the node in the link list where the item is stored.  Here we are storing the keys with the items in the nodes of the link list.
09:47:43 If I say decrease the key of a given item looking up that item finding the node for it that's constant time.  The issue is now that we have changed the key, let's say the key was five, now we are decreasing it to two, now we have to do a potentially linear search through the linked list to find where to put that so the link list is in order again.  If we up date the key value we change it from five to two this linked list may not be assorted anymore.
09:48:30 Imagine we can look at an example of that.  Suppose we had a linked list that looked like this.  If this was our linked list before and now we apply a decrease key operation to this item which has priority five and we decrease it to priority one then we need to up date this linked list.  If we just up dated the linked list like this it's no longer assorted.  Our find Min operation which looks at the first item of the list would return the wrong result if this is all we did.
09:49:20 We have to transform the list to move one to the front of the list.  This is the part that will take linear time in the worse case.  We have to look through the whole list to find the correct place to store this one.  If there was another item here with priority zero then we wouldn't move this to the front.  It has to be between zero and two.  That's why this decrease key can take linear time.  Make sense.  Okay.  These are two data structures we could use.
09:50:18 One more that is very important is a more sophisticated data structure you saw mainly a heap or in particular a Min heap.  Just to refresh your memory the idea of a heap is a tree data structure where we maintain the in varying in the tree the parent of the node always has a smaller value than its children or than the node itself.  For example, we could have a heap that looks like this.  Maybe we have two at the root here.  We have three as one child.
09:51:09 The left child and five is the right child.  This would be an example of a heap.  I should draw the arrows the other way.  We represent a heap with child pointers.  For every node in the heap it has a left and right child potentially.  This would be an example of a tree with the heap property because in deed if we look at every node its parent has a smaller value.  Three is here but its parent has two, et cetera.  The idea in the heap is if we add a new item in the heap, we have to
09:51:55 Up date the tree in such a way that the heap property is preserved.  If we added in a new element that had had priority or a key value then we may up date this by adding in a new node here for example.  If we added another key with value six then we would up date it like this for instance.  It's some tree such that as you move up wards the values are decreasing.  As you learned in one zero one, there are log rhythm time operations to add nodes in to the heap.
09:52:46 Because the idea of the heap is we maintain it it as a binary tree that is balanced.  It doesn't have linear depth going off in one direction.  It's spread out so the depth of the tree is log rhythmic in the nodes of the tree.  That means for any particular node the number of links you have to follow to get up to the parent is at most log rhythmic.  If we do a change key operation on six, if we change the priority of six up to one we have to move it up ward in the heap.
09:53:17 If we change the six to one this violates the heap property.  The one here its parent would not have a smaller value.  It's three.  We actually exchange the three and one.  The one comes up here.  That still wouldn't satisfy the heap property because now you have one here and two there.  We exchange one and two.  Then you have a tree with the heap property.  Because you have one here, two, four and three.  I will not go through that whole procedure again.
09:53:57 You learned it.  If you want to brush up on that it's also discussed in both of our text books.  The take away for the heap is it's at most log rhythmic time to change the value of an element and restore the heap property.  Likewise log rhythmic time to add a new element in and to delete an item from the heap as well, but the nice thing about the heap is the minimum is also at the top.  Its constant time to find the minimum element because it's up at the top.
09:54:44 And deleting the minimum is also log rhythmic time because there's a deleting procedure for heaps where we have to fix at most the positions of log rhythmic many elements here.  That's also log rhythmic time.  To summarize the run times you get from the heap is you have now constant time find Min.  Because the minimum is always at the top of this heap.  Then you have log rhythmic time everything else.  Inserting takes log rhythmic time in the worse case.
09:55:35 Deleting in the worse case delete Min and decrease key.  Those all take log rhythmic time.  A question here.  Shouldn't six be a child of five?  No actually.  If six were a child of five that would still be a tree with the heap property in terms of every child being smaller than its parent, but the most common way to set up heaps is you add everything as left most as you can.  If we didn't have the six on here then the way we would add six is you add it in the left most child position possible.
09:56:12 You can't add it here because that's already occupied.  You could add it here.  That's further to the left than adding it as the left child of five.  If we add an extra element it would go under five here.  Yeah.  The usual way that people implement heaps is you keep it close to a binary tree filling things from the left.  With when you need to you add a row on the tree and increase the depth by one.  You add things in on the bottom most level from the left moving to the right.
09:56:51 What you may be thinking of is a binary search tree.  That's different.  In a binary search tree we assume everything on the left is smaller than the root.  Everything on the right is larger than the root.  A heap is different.  In a heap we care that parents are smaller than children.  Any tree that satisfies that is good.  To keep the tree balanced we are adding things in from the left trying to keep the tree balanced.  Trying to keep the tree as close to a complete binary try as possible.
09:57:25 If you would like a summary of heap I will post online when I fill in the lecture topics table for today the sections of the text book where you can read up on heaps.  The take away message here is that for a heap almost everything is log rhythmic time which is worse than constant time but better than linear time.  Imagine you have a million elements in your heap.  A linear time operation would take on the order of a million steps.
09:58:12 Log rhythmic is six steps.  Log of ten to the six is six.  Log rhythmic time is not constant time but it it's small in general of the unless you have a huge amount of data log rhythmic is fast.  Heap is nice in that it doesn't have (INAUDIBLE).  The way they are in the lists here.  Conversely, it doesn't have as many operations being constant time.  There's a trade off.  There's situations where even one of these simple data structures can be faster than using this sophisticated heap.
09:58:51 A quick question.  Is a disadvantage of the heap that it has a greater space complexity?  Do you need more memory to implement a heap than a link list?  If you do a good job of implementing the heap if you think of how to efficiently encode the heap it doesn't need anymore memory than the link list.  You can encode the heap in an array where you have a convention.  You put the root as the left most node of the array.  Then the children are the next two items.
09:59:24 These four are the next four.  You don't need more than the amount of memory you need in a link list where it's linear for each item you have the item is key and forward and backward pointers.  You don't need more than that for the heap.  Where in fact, in the heap if you are implementing using an array like I indicated you don't need the pointers.  You can do everything with array index arithmetic.  You don't need to store pointers at all.
10:00:13 In that way heap uses less memory than link lists where you need forward and backward pointers.  Good question.  All these data structures use linear memory.  None is significantly better than the other in terms of memory.  It's the run time of these various operations.  Okay.  Just to check to make sure you understand all of this let's have a quick poll.  I will fire this up.  The poll is if you have an algorithm using this priority Q.
10:01:51 You have to do many more additions and deleteings than finding or deleting the minimum what implementation is most efficient?  Let's finish up here.  All right most of you have voted.  Thank you.  Let's take a look.  Most of you say the unassorted list.  You are correct.  How are we to think about this question?  The idea is if we need to do a lot of inserting and deleting let's think about the total run time that would be required for the algorithm for these operations.
10:02:25 If we look at the unassorted list inserting and deleting are both constant time for that data structure.  At the cost of find and delete Min being linear time, but in our question if we are doing many more inserting and deleting than find and delete Min the over all run time could be quite efficient.  For example suppose you did a constant number of these linear time operations but a quadratic number of inserting and deleting.
10:03:04 Our over all run time would be quadratic because we have quadratic times constant time.  Where as, if we used the assorted link list, if we did a quadratic number of (INDISCERNIBLE) we have quadratic number of these which takes linear time.  Now, a heap you may think your gut reaction may be maybe we should always go with a heap.  In general, it has a better trade off in terms of the run times of these operations.  None of them take linear time,
10:03:55 In fact, if you have a large number of inserting and deleting even this simple unassorted link list could be more effective than the heap.  Each inserting and deleting in the heap takes log rhythmic time where as it's constant time here.  If you have few slow linear time operations the unassorted list is better than the heap.  The answer is the unsort list.  Even though it's a simple data structure.  To emphasize that, suppose we had, if you had N squared insertings.
10:04:51 And only constant find Mins then if you work out the run time of these, let's do that.  If we have N squared inserting and constant (INDISCERNIBLE) what's the run time of these data structures going to be?  If we have the unassorted list it would be N squared inserting times the run time of the inserting which for unassorted list is constant time.  Plus a constant number we will say zero of one times find Min which is linear time.
10:05:38 That's O of N.  The over all work we have to do is order N squared.  I could make this theta.  If we have the assorted list we have N squared times the run time of the inserting for a sort list which is in the worse case theta of N plus we have a constant number of these find Mins which for the assorted list are fast.  Those are constant time.  The problem is you can see the trade off here.  We sped up this operation.  We slowed down this one.
10:06:29 Since there are many more of these it will hurt us in the run time.  This is theta of N cubed.  If we tried it for the heap what would we get?  We have N squared times these insertings which are theta of log N.  We have a constant number of the find Mins which are constant time for heap.  You look at the top.  This is going to be theta N log N.  Sorry.  N squared log N.  The question here, how is the number of inserting operations related to the elements in the list?
10:07:15 It's a good point.  If we were actually being careful here then we need to keep track of the fact that as (INAUDIBLE) the size of the array or size of the data structure is changing.  That will affect the run time of the find Min operation.  Here I glossed over that.  In the example here if we did the find Min we have N squared elements then the find Min would take N squared time.  This assumes there's only N elements in the list.  Maybe he did N of these assertings and find Min.
10:07:54 You are right.  We should be careful about that.  In this case it will not change the result.  You are right if you are doing a sequence of operations like this you do need to keep track of the size of the data structure overtime.  I will mention that in just a minute.  That leads to the next thing we will talk about.  Before I say that let me recap here.  The take away message from this analysis is that depending on how you are using the data structure,
10:08:40 Different implementation maybe faster.  In this example even though the heap gives (INDISCERNIBLE).  In this case you have a lot of inserting and few find Mins it turns out the unsorted list is faster asymptoticly than the heap.  The cost of going from linear time operation to a constant time one which it should help a lot didn't matter because we are only doing have you operations.  We are doing a lot of operations where we went from constant to log rhythmic time.
10:09:37 Our total run time got hurt.  I will write that down.  The best data structure depends on how the priority Q is being used.  This is always something to keep in mind when you are thinking about an algorithm.  It's fine to think about just using an abstract priority Q without worrying about how exactly you are going to implement it.  The implementation details of how you set up the priority Q like are you using a list or heap could change the run time of the algorithm.
10:10:12 Think of which implementation works for your case of the we will see examples of by choosing between these we can improve the run time.  If you say let's always use a heap even though that's a good option in general and if you don't know anything about your problem you maybe should stick with the heap if you know more about how you are going to use the data structure you may be able to do better with some other implementation.
10:10:44 Let me mention one last thing about the run times of these.  I will summarize this in a table and mention other data structures we will not get to but exist out there so you can use them when you are thinking about the run time of an algorithm.  Then we will get back to this point which was brought up in the chat which is a great one.  You have to keep track of as you are doing this sequence of operations how is the size of the data structure changing.
10:12:02 To summarize here let's look at the run time.  Let's put the run times of these different data structures in a table.  Run times of various priority Q implementation.  We have all the operations that we looked at before.  We have insert.  We have delete.  We have find Min.  Delete Min and decrease key.  Those are our operations.  We have various data structures that we can use to implement the priority Q.  We have the unassorted list.
10:12:55 We have the assorted list and we have the heap.  Let's fill these in.  For the unassorted list remember it's constant time to add and delete things, but to find the minimum could be linear time.  This is on the order I'm going to put for all these I will put the theta of the run time for the worst case.  This would be, I will write it up here.  I will write for all of these entries it's going to be theta of F of N.  I will give F where N is the number of elements in the Q.
10:13:40 N equals number of elements in the Q.  Inserting is a constant time.  Deleting is constant time, but find and delete Min can take linear time.  Decrease key is also constant time.  I'm leaving out the theta here so I don't have to write that over again.  That means theta of one, theta of one, theta of N and theta of one.  Where as the assorted list that made find minimum and delete Min constant time.  Deleting was still constant time.
10:14:31 Now the inserting and decrease key can take linear time where as for the heap find Min was constant time because you look at the top of the heap.  Anything else requires this log rhythmic algorithm to fix up the heap.  Inserting can take log N steps.  Deleting likewise.  Delete Min and (INDISCERNIBLE) because you may have to move things around in the heap to fix up the heap property.  The last thing that I want to mention about priority Q's is that there are other more sophisticated da structure
10:15:19 We don't have time to talk about in this class but can improve on this.  One of the most famous such data structures which is described in CLRS I will put the chapter number in the notes online is the Fibonacci heap.  This heap implementation is somewhat similar to, it's related to a normal heap it's a tree data structure.  You want to have children have smaller keys than -- sorry larger keys than their parents.  As you move up ward in the tree you get smaller keys.
10:15:52 The Fibonacci heap is implemented in a sophisticated way that insures all the operations that don't involve deleting an element actually take constant time on average.  Not just on average but in the sense a stronger sense of not average in the sense of there's some randomness it's not a random algorithm but on average in the sense that if you have a long series of operation the average time over all of those operations is constant.
10:16:42 That's what we will talk about in the last ten minutes is this concept of average run time.  Let me write the run times first.  All the operations except the deleting ones take constant time.  Inserting is constant time.  Delete is log N.  Find Min is constant time.  Delete Min is log N.  Decrease key is constant time.  This is asymptoticly strictly better than the plain heap.  It's more difficult to implement.  We will not talk about it here.
10:17:19 If you are computing the run time of an algorithm and it uses a priority Q and you want to know how fast can you implement it using any priority Q this Fibonacci heap run time achieves here are asymptoticly optimal.  If you want to keep these operations at constant time it's not possible to improve these beyond log rhythmic time.  If you are looking for the best data structure you can use a Fibonacci heap in place of a heap.
10:17:54 Notice there are situations where it makes sense to use an unassorted list like the one we had before where you have a very large number of deleteings or the example we had before is a large number of inserting.  If you have a large number of deleting those take log rhythmic time in the Fibonacci heap but are constant in the list.  There's cases to use the list even though you have the Fibonacci heap.  We are not going to cover the Fibonacci heap in class.
10:18:55 It exists and gives you these run times.  If you want to learn more there's a chapter about it in CLRS.  I will put the reference online.  As I said, there's a bit of a caveat for some of these operations.  They are not necessarily actually constant time, but they are constant time averageed over a larger sequence of operations.  In the case of in particular the find Min.  Not find Min.  Where is it?  It's the delete Min and the decrease key.  This one.
10:19:34 And this one.  These two run times are called amortized time.  What does this mean?  Amortized time means if you do a bunch of these operations it's not necessarily the case that the decrease key operation for example necessarily takes constant time.  It may sometimes take maybe linear time, but averaged over the whole sequence of operations you are doing it does take constant time.  Sometimes you do this operation it may take many steps.
10:20:49 If this takes many steps some of the other operations will take fewer steps.  On average it works out to constant time.  That's the concept of amortized time which we will talk about next.  The idea of amortized run time is instead of counting the run times of individual operations we count up the run time of a sequence of operations and then average it out over all of them.  Instead of finding worst case bounds for individual operations.
10:21:43 Find the worst case over any sequence of say M operations and average over them.  If we add it up over all of the sequence then we just divide by M.  The idea is if you are actually running an algorithm using some data structure let's say you are using a Fibonacci heap or a heap.  You are not just going to do one of these operations.  You will do some sequence.  Maybe you insert ten things and delete five and delete Min three times and insert more things.
10:22:18 You do more delete Mins et cetera.  All you care about for your algorithm is what is the total run time.  The amount of time of each individual operation is not actually important.  These bounds on the run times of individual operations are useful because if you know your algorithm does at most fifty deleteings you know in the heap will you have fifty times log N.  Fine.  Sometimes you can get a better bound by looking at what is the worst case over a sequence of operations.
10:23:10 Rather than taking the worst case for a single operation and assuming the worst case happens every time.  It can happen if you have a long sequence of operations it's not possible to get the worst case every time.  This can yield better bounds if the worst case for an individual operation can't happen every time.  I will give an example of this.  What I mean by better bounds is it can give a smaller worst case run time.  If you did a naive analysis,
10:23:50 You would say this will take N squared time.  If you do this careful analysis where you look at any possible sequence of operations you will find it can never do worse than linear time.  Let me show you an example.  One note I will mention if this word amortized seems strange this is from the word used in finance.  If you have, let's say you want to buy a house and can't afford to buy it directly you get a mortgage that
10:24:21 amortizes the house for thirty years.  It let's you pay an average cost over a longer time.  That's the same sense we are talking about here.  We don't look at run times of a single operation.  We are looking at a sequence of operations over a long algorithm.  We are going to add up the run time over that whole sequence then once we do that you divide by the number of operations to find how long each one takes on average.
10:25:08 There's no probability here.  It's an average overtime.  Not an average over randomness in the algorithm.  Let me show you a simple example and we will talk more about it on Wednesday.  An example is automatically expanding arrays.  Those of you who programmed in Python or C plus plus for example know that there's a type of array that grows automatically as you add more elements to it.  In C plus plus it's called a vector.
10:26:18 Or the Python list.  These are both stored internally as an array, but the problem with an array is it has affixed size.  What you do is when the array fills up you allocate a new array that's bigger than the old one and copy everything in there.  The idea is you allocate affixed size array to store a list when it fills up allocate a new array of twice the size is the most common.  Copy the old array in to it.  We are out of time.  I will not talk about the analysis.
10:26:58 I will come back to it Wednesday.  The idea is as you insert things in to the array normally it's constant time.  You set element one, two, three, et cetera.  Once the list runs out of space you have to reallocate one twice the size and copy everything older.  That's a linear operation.  You have to do this when the array fills up.  Since you double the size of the array each time the duping averages out.  On average the time it takes for inserting is still constant time.
10:27:31 Even though it's mostly constant time but occasionly linearly time.  It's still constant time because you do the linear time thing when the array doubles in size.  When it doubles the amount of time you need to double it is getting less.  Over all, it will end up being constant time amortized over the size of the array.  We will come back to this Wednesday.  I will hang around for awhile if you have further questions.  See you.