connect dots ...

Part one

basic concepts

• DP, a general, powerful algorithm design technique
• DP ≈ a careful brute force
  • try all possibilities ,carefully
  • get polynomial time
• DP ≈ subproblems + reuse
• DP ≈ recursion + memoization + guessing
  • memoize(remember) & re-use solutions to subproblems that help solve the problem

Fibonacci

problems:

F₁ = F₂ = 1
F_n = F_n-1 + F_n-2
Goal: compute F_n

Naive recursive algorithms

fib(n):
  if n <= 2: f = 1
  else: f = fib(n-1) + fib(n+2)
  return f

• time complexity
  • EXPONENTIAL TIME
  • T(n): the time of compute F(n)
  • T(n) = T(n-1) + T(n-2) + Θ(1) >= F_n ≈ Φⁿ >= 2T(n-2) = Θ(2^n/2)

Memoized DP algorithm

memo = {}
fib(n):
  if n in memo: return memo[n]
  if n <= 2: f = 1
  else: f = fib(n-1) + fib(n-2)
  memo[n] = f
  return f

• idea is simple: whenever we compute a Fibonacci number, we put it into a dictionary.And then we need to compute the nth Fibonacci we check, is it already in the dictionary? Did we already solve this problem? If so, return that answer. Otherwise, compute it.
• fib(k) only recurses the first time it's called.∀k
  • memoized calls cost O(1)
  • non-memoized calls is n:
  • non-recursive work per call:Θ(1)
    => time = Θ(n)
  • time = subproblems * time/subproblem

Bottom-up DP algorithm

fib = {}
for k in range(1, n+1):
  if k <= 2: f = 1
  else: f = fib[k-1] + fib[k-2]
  fib[k] = f
return fib[n]

• exactly same computation
• topological sort of subproblem dependency DAG
• save space (only need the last two value)

shortest paths

problems:

δ(s, v)∀v

tool:

Guessing, a very powerful tool.a tried and tested method for solving any problem

• suppose you don't know something, but you'd like to know it.
• so what's the answer to this question?I don't know.
• How am I going to answer the question?Guess!
• Don't try any guess.try them all. & take the best one

There are some incoming edges to vertex v. The shortest path to v comes through one of those edges. We don't know which one. So before solving the problem of finding shortest path to v we have solve smaller problems of finding shortest path to the parent vertices. If we have 2 incoming edges u1 -> v and u2 -> v, then we have to first find the shortest paths to vertices u1 and u2. Once we know them we can choose one edge between u1 -> v and u2 -> v which will result in the shortest path to v.

guessing & DAG view

• infinite time on graphs with cycles
  DAGS: O(v+e)

  • use formula: time = subproblems * time/subproblem
  • number of subproblems: v
  • how much time do I spend per subproblem? : incoming edges to v. indegree(v)

• if you are a acyclic:time = subproblems * time/subproblem

• if you are a cyclic: make it to acyclic

DAGS

cyclic graph

Shortest Paths

Memoized Fibonacci

DAG

遗留

• Bellman-Fords's algorithm

PS

因为教授是自己非常喜欢的一位导师，他总是能够用轻松自如的态度和浅显的语言表述来授课，但记录到一半的时候会怀疑自己投入大量时间来这样做的价值。——毕竟还有很多事情需要去做。
到底是会做dp的题就OK了，还是细细解剖每一个细节，争取得到比DP本身更多的东西？比如思维方式，比如逻辑推理，比如面对一般性的问题如何去处理，又比如智慧本身。
再记录更多的时候，会得到一种喜悦。可以通过模仿，可以通过不断演练，都可以。其他需要我去做的事情皆黯淡无光。此时此刻，聚光灯，打在舞台正中央，打在我身上。