Dynamic programming is an optimisation technique. It aims to optimise and simplify a problem-solving process by breaking down a complex problem into more straightforward problems. Its flexibility and adaptability make the dynamic programming approach a powerful and versatile problem-solving technique. Understanding dynamic programming, however, requires intuitive & intelligent thinking.
This article attempts to elucidate DP using elementary examples & offers an overview of dynamic programming problems.
A Brief Intro
A central aspect of dynamic programming is the multistage & recursive nature of the entire optimisation procedure. The dynamic programming framework can be used to solve a vast variety of problems using numerous different kinds of optimisation techniques.
Every Dynamic Programming problem has a specific schema to follow:
- The problem is disintegrable and can be broken down into optimal sub-problems.
- We can define the problem and the solution’s value by expressing it in terms of optimal solutions to smaller subproblems.
- We can compute the value of the optimal solution through a bottom-up approach.
- The computed information can be used to find the optimal solution.
We can think of dynamic programming as a table-filling algorithm: we already know the calculations we need to do & have them stored. So we need to pick the best order to do them in & fill the table sequentially and ignore the ones we don’t have to fill in.
Dynamic programming approaches any problem in either of the two following manners.
- Bottom Up – The bottom-up approach uses the tabulation technique to implement a dynamic programming process. It does not use recursion, removing the risk of any memory overflow or additional overhead. Instead, tabulation techniques solve problems and store the results in a matrix.
This is an iterative approach based on the simple idea: the solution of a more significant problem depends on the solution of smaller sub-problems
- Top Down – Recursion plays a significant role over here. Solutions employ recursion, and the solutions at every stage or sub-problem are stored in some data structure. The top-down approach checks whether a particular subproblem has been solved. If yes, then the stored result is used. This is the memorised version of a recursive solution.
Dynamic programming can feel a little convoluted for beginners. You may need professional Global Assignment Help if you struggle with your DP assignments or coursework. Examples are the best way to understand dynamic programming from scratch.
Following is one of the most infamous examples of them all.
Understanding Dynamic Programming Using The Infamous Stage-Coach Problem
The Stagecoach Problem was developed by Professor Harvey M. Wagner while teaching at Stanford University. This is an elementary example for illustrating the working & features of dynamic programming. It goes something like this:
During the colonisation era of the United States of America, the journey toward the West was fraught with dangers. A fortune seeker in the state of Missouri decides to go west to take part in the gold rush during the 19th century. The journey requires the stagecoach to traverse many uncharted territories fraught with dangers.
- His starting point and destination are fixed, but there are multiple routes between the sources and the destination.
- Every route connects one state to another.
- There are four stages involved, from the traveller’s point of embarkation at state A (Missouri) to his destination J (California)
- Every route has a cost associated with it, as per the value of the life insurance policy offered for a particular route due to the risks involved in traversing.
The fortune seeker wishes to find the safest to reach California.
Here’s the entire map à
So, our task is to find the route with the least cost.
The Solution
At first glance, we may be tempted to choose the routes with the least values at every stage. For example, this strategy gives us A à B à F à I à J with a total cost of 13. However, the overall savings may be more significant if we make sacrifices at one stage.
The trial-and-error method is ineffective & inefficient, and there are many possible routes available.
For the stagecoach problem, we use dynamic programming by considering the following small problem: The fortune seeker has completed the majority of the journey & has just one stage to complete.
Formulation
- Let xn (n=1, 2,3,4) be decision variables denoting the immediate destination at stage n, the nth stage to be undertaken. So, a generic route will be A à x1 à x2 à x3 à x4, where x4 is J.
- And, let fn(s, xn) be the total cost function of traversing the remaining stages, where s is the current stage, xn is the immediate destination & n denotes the stage. Given s and n, let x*n denote the route that minimises cost function fn(s, xn) and let the corresponding minimum cost function be denoted by f*n(s, xn).
Thus, f*n(s) = fn(s, x*n) and fn(s, xn) = immediate cost (stage n) + minimum future cost (stages n + 1 onward) = Csxn +f*n+1 (xn)
- The value of Csxn is obtained from preceding calculations, or route information tables by where s is the current stage and xn is the immediate destination.
- The objective is to find f*1(A) and then the successive route. Dynamic programming finds it by successively finding f*4(s), f*3(s), and f*2(s) for every possible state and then finally using f*2(s) to determine f*1(A).
The Procedure
When the fortune seeker has just the final stage left, the fourth (n=4) stage, the route is determined entirely by the current state and the final destination. At the fourth stage, the current state s can be either H or I, and the final destination is x4= J, so the final stagecoach run is s à J. Therefore, the f*4 (s) = f4(s, J) = Cs, J, the minimum path from s to J, which is the immediate solution to the fourth stage, is as follows=
Fig: Stage 4 Table & Diagram
Next up, when the fortune seeker has two more stages, calculations become more extensive.
As may be evident, we are using the bottoms-up DP approach to find the optimal solution.
For example, when they are at stage F, they can go to either H or I at costs 6 and 3, respectively. However, if they choose state H, the minimum additional cost acquired is f*4(H) = 3, as seen from the above table. Thus, the total cost is 6 + 3 =9 in the case of H and 3+4=7 in the case of I. Therefore, the optimal choice is x*3 = I as it offers f*3(F)=7.
Fig: Stage 3 table and diagram
Similarly, we must calculate for s= E & G. Different combinations of Cij and f*4(s).
In stage 2, we again use the same approach. In this case, f2(s, x2)= Csx2 + f*3(x2).
Fig: Stage 2 diagram and table
We can go to E, F or G from B, C, or D. As usual, the current or immediate costs need to be added with the minimum additional cost of every node at stage 3, as per the stage 3 table. As evident from above, we can either go to E or F from both B and D.
Read Also: Study Abroad for Indian student
At the final stage of the problem ( n=1), when the fortune seeker has just begun his journey, the overall calculations become simpler as there is just one state, source A.
The calculations are similar, with immediate costs added to the minimum additional costs of every destination.
Finally, the ultimate optimal solution for the stagecoach problem can be determined by combining the results of the previous tables.
- The result for the stage 1 problem should have the fortune seeker going from A to either C or D.
- If C is chosen for stage 2, the route with minimal immediate & additional cost is E.
- If E has been chosen at stage 2, then at stage 3, the best choice is H.
- And, when s= H at stage 3, then for stage 4, the ideal node is J.
The optimal route is A à C à E à H à J à T.
We can find two extra optimal routes if we choose D instead of C. All of them yield a COMPLETE ROUTE COST of 11.
The final diagram is as follows:
And that’s about it for this write-up. I hope it was an informative and exciting read. If you need more help, seek help from professional assignment help services experts.
All the best!
Author-Bio: Natasha Stevenson is a computer science professor from a major public university in London, the UK. She is also a part-time tutor at MyAssignmenthelp.com, a leading assignment and essay writers service.
Apart from this if you are interested to know more about Web Development then visit our technology category
