|
1 | | -# 1-DD |
| 1 | +# 1-D Dynamic Programming |
2 | 2 |
|
3 | | -**Dynamic Programming (DP)** is a powerful algorithmic technique for solving problems by breaking them down into overlapping subproblems. Each subproblem is solved only once, and its solution is stored (usually in a table or array) to avoid redundant computations. DP is especially effective for problems involving optimization, counting, and partitioning. |
| 3 | +**One-liner.** State indexed by **one** parameter (position / value / count); transition reuses smaller subproblems. |
4 | 4 |
|
5 | | -### Key Concepts in Dynamic Programming |
6 | | - |
7 | | -1. **Optimal Substructure**: A problem exhibits optimal substructure if its optimal solution can be constructed from the optimal solutions of its subproblems. |
8 | | - |
9 | | -2. **Overlapping Subproblems**: A problem has overlapping subproblems if it can be broken down into subproblems that are solved multiple times. DP is efficient for such problems because it avoids redundant calculations by storing results. |
10 | | - |
11 | | -3. **Memoization vs. Tabulation**: |
12 | | - - **Memoization**: A top-down approach where results of subproblems are stored as they are computed (usually with recursion and a cache). |
13 | | - - **Tabulation**: A bottom-up approach where solutions to subproblems are built iteratively, filling a table from smaller to larger subproblems. |
14 | | - |
15 | | ---- |
16 | | - |
17 | | -### Common Types of DP Problems |
18 | | - |
19 | | -1. **Fibonacci Sequence** |
20 | | -2. **Knapsack Problems** |
21 | | -3. **Longest Common Subsequence (LCS)** |
22 | | -4. **Edit Distance** |
23 | | -5. **Coin Change** |
24 | | -6. **Partition Problem** |
25 | | -7. **Minimum Path Sum** |
26 | | -8. **Matrix Chain Multiplication** |
27 | | - |
28 | | -Let’s explore these with explanations, examples, and code. |
29 | | - |
30 | | ---- |
31 | | - |
32 | | -### 1. Fibonacci Sequence |
33 | | - |
34 | | -**Problem**: Find the nth Fibonacci number, where `Fib(n) = Fib(n-1) + Fib(n-2)` and `Fib(0) = 0`, `Fib(1) = 1`. |
35 | | - |
36 | | -**Recursive DP (Memoization)**: |
37 | | - |
38 | | -```python |
39 | | -def fib_memo(n, memo={}): |
40 | | - if n in memo: |
41 | | - return memo[n] |
42 | | - if n <= 1: |
43 | | - return n |
44 | | - memo[n] = fib_memo(n-1, memo) + fib_memo(n-2, memo) |
45 | | - return memo[n] |
46 | | - |
47 | | -# Example usage |
48 | | -print(fib_memo(10)) # Output: 55 |
| 5 | +## Visual |
49 | 6 | ``` |
| 7 | +House Robber nums = [2, 7, 9, 3, 1] |
50 | 8 |
|
51 | | -**Iterative DP (Tabulation)**: |
| 9 | + i: 0 1 2 3 4 |
| 10 | + nums: 2 7 9 3 1 |
| 11 | + dp: 2 7 11 11 12 dp[i] = max(dp[i-1], dp[i-2]+nums[i]) |
| 12 | + ↑↑↑ "skip i" vs "rob i + best up to i-2" |
52 | 13 |
|
53 | | -```python |
54 | | -def fib_tab(n): |
55 | | - if n <= 1: |
56 | | - return n |
57 | | - dp = [0, 1] |
58 | | - for i in range(2, n + 1): |
59 | | - dp.append(dp[i-1] + dp[i-2]) |
60 | | - return dp[n] |
61 | | - |
62 | | -# Example usage |
63 | | -print(fib_tab(10)) # Output: 55 |
64 | | -``` |
65 | | - |
66 | | -**Time Complexity**: O(n) for both approaches. |
67 | | - |
68 | | ---- |
69 | | - |
70 | | -### 2. Knapsack Problem |
71 | | - |
72 | | -**Problem**: Given a list of items with weights and values, and a knapsack with a weight capacity `W`, maximize the value in the knapsack without exceeding `W`. Each item can only be included once (0/1 Knapsack). |
| 14 | + Space-optimized: only prev2, prev1 kept → O(1) memory. |
73 | 15 |
|
74 | | -**Solution Approach (Tabulation)**: |
75 | | -- Use a DP table where `dp[i][j]` represents the maximum value achievable with the first `i` items and weight limit `j`. |
76 | | - |
77 | | -**Code Example**: |
78 | | - |
79 | | -```python |
80 | | -def knapsack(weights, values, W): |
81 | | - n = len(weights) |
82 | | - dp = [[0] * (W + 1) for _ in range(n + 1)] |
83 | | - |
84 | | - for i in range(1, n + 1): |
85 | | - for j in range(W + 1): |
86 | | - if weights[i-1] <= j: |
87 | | - dp[i][j] = max(dp[i-1][j], values[i-1] + dp[i-1][j - weights[i-1]]) |
88 | | - else: |
89 | | - dp[i][j] = dp[i-1][j] |
90 | | - |
91 | | - return dp[n][W] |
92 | | - |
93 | | -# Example usage |
94 | | -weights = [1, 2, 3] |
95 | | -values = [10, 15, 40] |
96 | | -W = 5 |
97 | | -print(knapsack(weights, values, W)) # Output: 55 |
| 16 | +Recursion tree without memo (Fibonacci-style): |
| 17 | + f(5) ──┬── f(4) ──┬── f(3) ─── f(2) ─── f(1) |
| 18 | + │ └── f(2) ──┘ ← recomputed |
| 19 | + └── f(3) ─── f(2) ← recomputed again |
| 20 | + ⇒ memoize and the tree becomes a chain (O(n)) |
98 | 21 | ``` |
99 | 22 |
|
100 | | -**Time Complexity**: O(n * W), where `n` is the number of items and `W` is the knapsack capacity. |
101 | | - |
102 | | ---- |
103 | | - |
104 | | -### 3. Longest Common Subsequence (LCS) |
105 | | - |
106 | | -**Problem**: Given two strings, find the length of their longest common subsequence (a sequence of characters that appears in both strings in the same order). |
107 | | - |
108 | | -**Solution Approach (Tabulation)**: |
109 | | -- Use a 2D DP table where `dp[i][j]` represents the length of the LCS for substrings of the first `i` characters of string `A` and the first `j` characters of string `B`. |
110 | | - |
111 | | -**Code Example**: |
112 | | - |
| 23 | +## Mental model |
| 24 | +DP is **brute force with a notebook**: write down each subproblem the first time you solve it, never compute it twice. The mental jump from recursion to DP is realizing the recursion tree has *overlap*. In real systems this is exactly what memoization, build caches, incremental computation, and query plan caches do — same shape, different vocabulary. |
| 25 | + |
| 26 | +## Recognize it |
| 27 | +- "min / max / count of ways" over a sequence |
| 28 | +- "decisions per index", optimal substructure |
| 29 | +- problem feels like backtracking but you only need **a number**, not the solutions |
| 30 | +- overlapping subproblems (you'd recompute the same state in naive recursion) |
| 31 | + |
| 32 | +## Pick DP over... |
| 33 | +| Instead of | Use DP when | |
| 34 | +|---|---| |
| 35 | +| backtracking | you only need a count / min / max, not the solutions themselves | |
| 36 | +| greedy | a local choice can hurt later (e.g. coin change with weird denominations) | |
| 37 | +| math closed-form | recurrence doesn't simplify | |
| 38 | +| BFS | states have weights, not unit cost | |
| 39 | + |
| 40 | +## The 5-step recipe (do this on EVERY DP problem) |
| 41 | +1. **State** — what does `dp[i]` mean in one sentence? (write it down) |
| 42 | +2. **Transition** — how do `dp[i]` and smaller states relate? (the recurrence) |
| 43 | +3. **Base case** — what's `dp[0]` (and maybe `dp[1]`)? |
| 44 | +4. **Order** — bottom-up (tabulation) or top-down (memo)? |
| 45 | +5. **Answer** — is it `dp[n]`, `dp[n-1]`, `max(dp)`, …? |
| 46 | + |
| 47 | +## Top-down (memoization) |
113 | 48 | ```python |
114 | | -def lcs(A, B): |
115 | | - m, n = len(A), len(B) |
116 | | - dp = [[0] * (n + 1) for _ in range(m + 1)] |
117 | | - |
118 | | - for i in range(1, m + 1): |
119 | | - for j in range(1, n + 1): |
120 | | - if A[i-1] == B[j-1]: |
121 | | - dp[i][j] = dp[i-1][j-1] + 1 |
122 | | - else: |
123 | | - dp[i][j] = max(dp[i-1][j], dp[i][j-1]) |
124 | | - |
125 | | - return dp[m][n] |
| 49 | +from functools import cache |
126 | 50 |
|
127 | | -# Example usage |
128 | | -print(lcs("ABCBDAB", "BDCAB")) # Output: 4 |
| 51 | +@cache |
| 52 | +def f(i): |
| 53 | + if i <= 0: return base(i) |
| 54 | + return combine(f(i-1), f(i-2), ...) |
129 | 55 | ``` |
130 | 56 |
|
131 | | -**Time Complexity**: O(m * n), where `m` and `n` are the lengths of the strings. |
132 | | - |
133 | | ---- |
134 | | - |
135 | | -### 4. Edit Distance |
136 | | - |
137 | | -**Problem**: Given two strings `A` and `B`, find the minimum number of operations required to convert `A` into `B`. Allowed operations are insertion, deletion, and substitution. |
138 | | - |
139 | | -**Solution Approach (Tabulation)**: |
140 | | -- Use a 2D DP table where `dp[i][j]` represents the minimum edit distance between the first `i` characters of `A` and the first `j` characters of `B`. |
141 | | - |
142 | | -**Code Example**: |
143 | | - |
| 57 | +## Bottom-up (tabulation) |
144 | 58 | ```python |
145 | | -def edit_distance(A, B): |
146 | | - m, n = len(A), len(B) |
147 | | - dp = [[0] * (n + 1) for _ in range(m + 1)] |
148 | | - |
149 | | - for i in range(m + 1): |
150 | | - for j in range(n + 1): |
151 | | - if i == 0: |
152 | | - dp[i][j] = j |
153 | | - elif j == 0: |
154 | | - dp[i][j] = i |
155 | | - elif A[i-1] == B[j-1]: |
156 | | - dp[i][j] = dp[i-1][j-1] |
157 | | - else: |
158 | | - dp[i][j] = 1 + min(dp[i-1][j], dp[i][j-1], dp[i-1][j-1]) |
159 | | - |
160 | | - return dp[m][n] |
161 | | - |
162 | | -# Example usage |
163 | | -print(edit_distance("sunday", "saturday")) # Output: 3 |
| 59 | +dp = [0] * (n + 1) |
| 60 | +dp[0] = base |
| 61 | +for i in range(1, n + 1): |
| 62 | + dp[i] = recurrence(dp[i-1], dp[i-2], ...) |
| 63 | +return dp[n] |
164 | 64 | ``` |
165 | 65 |
|
166 | | -**Time Complexity**: O(m * n). |
167 | | - |
168 | | ---- |
169 | | - |
170 | | -### 5. Coin Change |
171 | | - |
172 | | -**Problem**: Given an integer array `coins` representing coin denominations and an integer `amount`, find the minimum number of coins that make up `amount`. |
173 | | - |
174 | | -**Solution Approach (Tabulation)**: |
175 | | -- Use a 1D DP array where `dp[i]` represents the minimum number of coins needed for amount `i`. |
176 | | - |
177 | | -**Code Example**: |
178 | | - |
| 66 | +## Space-optimized (keep last-k window) |
179 | 67 | ```python |
180 | | -def coin_change(coins, amount): |
181 | | - dp = [float('inf')] * (amount + 1) |
182 | | - dp[0] = 0 |
| 68 | +prev2, prev1 = base0, base1 |
| 69 | +for i in range(2, n + 1): |
| 70 | + cur = recurrence(prev1, prev2) |
| 71 | + prev2, prev1 = prev1, cur |
| 72 | +return prev1 |
| 73 | +``` |
183 | 74 |
|
184 | | - for coin in coins: |
185 | | - for i in range(coin, amount + 1): |
186 | | - dp[i] = min(dp[i], dp[i - coin] + 1) |
| 75 | +## The 1-D pattern zoo |
187 | 76 |
|
188 | | - return dp[amount] if dp[amount] != float('inf') else -1 |
| 77 | +**Fibonacci / Climbing stairs** — `dp[i] = dp[i-1] + dp[i-2]`. |
189 | 78 |
|
190 | | -# Example usage |
191 | | -print(coin_change([1, 2, 5], 11)) # Output: 3 |
192 | | -``` |
| 79 | +**House Robber** — `dp[i] = max(dp[i-1], dp[i-2] + nums[i])`. |
193 | 80 |
|
194 | | -**Time Complexity**: O(n * amount), where `n` is the number of coins. |
| 81 | +**House Robber II (circular)** — run rob on `nums[:-1]` and `nums[1:]`, take max. |
195 | 82 |
|
196 | | ---- |
| 83 | +**Min cost climbing stairs** — `dp[i] = min(dp[i-1], dp[i-2]) + cost[i]`. |
197 | 84 |
|
198 | | -### 6. Minimum Path Sum |
| 85 | +**Decode ways** — `dp[i] = (dp[i-1] if s[i] != '0') + (dp[i-2] if 10 ≤ int(s[i-1:i+1]) ≤ 26)`. |
199 | 86 |
|
200 | | -**Problem**: Given a grid of non-negative integers, find a path from the top-left to the bottom-right that minimizes the sum of the numbers along the path. You can only move right or down. |
| 87 | +**Word break** — `dp[i] = any(dp[j] and s[j:i] in dict for j < i)`. |
201 | 88 |
|
202 | | -**Solution Approach (Tabulation)**: |
203 | | -- Use a 2D DP table where `dp[i][j]` represents the minimum path sum to reach cell `(i, j)`. |
| 89 | +**Coin change (min coins, unlimited)** — `dp[a] = min(dp[a - c] + 1 for c in coins)`; init `dp[0]=0`, others `inf`. |
204 | 90 |
|
205 | | -**Code Example**: |
| 91 | +**Coin change 2 (count combinations)** — outer loop **coin**, inner loop **amount** (order matters for combinations not permutations!). |
206 | 92 |
|
207 | | -```python |
208 | | -def min_path_sum(grid): |
209 | | - m, n = len(grid), len(grid[0]) |
210 | | - dp = [[0] * n for _ in range(m)] |
211 | | - dp[0][0] = grid[0][0] |
212 | | - |
213 | | - for i in range(1, m): |
214 | | - dp[i][0] = dp[i-1][0] + grid[i][0] |
215 | | - for j in range(1, n): |
216 | | - dp[0][j] = dp[0][j-1] + grid[0][j] |
217 | | - |
218 | | - for i in range(1, m): |
219 | | - for j in range(1, n): |
220 | | - dp[i][j] = min(dp[i-1][j], dp[i][j-1]) + grid[i][j] |
221 | | - |
222 | | - return dp[m-1][n-1] |
223 | | - |
224 | | -# Example usage |
225 | | -grid = [ |
226 | | - [1, 3, 1], |
227 | | - [1, 5, 1], |
228 | | - [4, 2, 1] |
229 | | -] |
230 | | -print(min_path_sum(grid)) # Output: 7 |
231 | | -``` |
| 93 | +**Longest Increasing Subsequence (LIS)**: |
| 94 | +- O(n²): `dp[i] = 1 + max(dp[j] for j<i if nums[j] < nums[i], default=0)`. |
| 95 | +- O(n log n): `bisect_left` into a "tails" array — the array's **length** is the answer (its contents are not the actual LIS). |
232 | 96 |
|
233 | | -**Time Complexity**: O(m * n), where `m` and `n` are the grid dimensions. |
| 97 | +**Partition equal subset sum** — subset-sum DP: `dp[s] = True if dp[s - num] OR dp[s]`. Iterate amounts **backward** to avoid reuse. |
234 | 98 |
|
235 | | ---- |
| 99 | +**Maximum subarray (Kadane)** — `cur = max(x, cur + x)`; `best = max(best, cur)`. |
236 | 100 |
|
237 | | -### Summary of Dynamic Programming |
| 101 | +**Maximum product subarray** — track `(min, max)` because negatives flip. |
238 | 102 |
|
239 | | -| **Problem** | **Approach** | ** |
| 103 | +## Edge cases |
| 104 | +- empty input — base case `dp[0] = 0` / `1` / problem-defined |
| 105 | +- single element — make sure transitions don't out-of-bounds index |
| 106 | +- `inf` sentinel for "unreachable" in min-DP (`coin change`); guard the final answer |
| 107 | +- coin change with no solution → return `-1` (check `dp[amount] == inf`) |
| 108 | +- LIS with duplicates: strict (`<`) vs non-strict (`≤`) → `bisect_left` vs `bisect_right` |
| 109 | +- maximum product with all negatives / containing zeros — Kadane's track-both fixes this |
| 110 | +- 0/1 vs unbounded knapsack — iterate amount **backward** for 0/1, **forward** for unbounded |
240 | 111 |
|
241 | | -Complexity** | **Explanation** | |
242 | | -|-----------------------|--------------------------|-------------------|--------------------------------------------------| |
243 | | -| Fibonacci Sequence | Memoization or Tabulation| O(n) | Avoid redundant calculations for each Fib(n). | |
244 | | -| Knapsack | Tabulation | O(n * W) | Maximize value without exceeding weight W. | |
245 | | -| Longest Common Subseq | Tabulation | O(m * n) | Finds the longest subsequence present in both. | |
246 | | -| Edit Distance | Tabulation | O(m * n) | Transform one string to another with minimal ops.| |
247 | | -| Coin Change | Tabulation | O(n * amount) | Find min coins for given amount. | |
248 | | -| Minimum Path Sum | Tabulation | O(m * n) | Minimize the sum from top-left to bottom-right. | |
| 112 | +## Complexity |
| 113 | +- Linear transitions: O(n) time, O(n) or O(1) space. |
| 114 | +- Knapsack-style: O(n · W). |
| 115 | +- LIS O(n log n) via patience sorting. |
| 116 | +- Coin change: O(amount · |coins|). |
249 | 117 |
|
250 | | -Dynamic Programming is versatile for solving a wide variety of problems, especially those with overlapping subproblems and optimal substructure. Its efficiency often lies in converting exponential recursive problems to polynomial solutions. |
| 118 | +## Gotchas |
| 119 | +- **State definition vague** → recurrence will be wrong. Pin down what `dp[i]` means *exactly* (inclusive? ending here? up to here?). |
| 120 | +- For "ending at i" vs "up to i" — completely different recurrences for problems like max subarray. |
| 121 | +- Iteration order matters for 0/1 vs unbounded — flip it and you've changed the problem. |
| 122 | +- Use `@cache` (Python 3.9+) for trivial top-down, but watch out for unhashable args. |
| 123 | +- Initialize `inf` only for **min** problems; `-inf` or `0` for max/count. Wrong sentinel breaks everything. |
| 124 | +- LIS O(n log n) gives you the **length**, not the subsequence — reconstructing needs back-pointers. |
0 commit comments