adds policy iteration

Author: samagra14

md/Policy-Iteration.md

@@ -18,3 +18,24 @@ __function__ POLICY-ITERATION(_mdp_) __returns__ a policy
 
 ---
 __Figure ??__ The policy iteration algorithm for calculating an optimal policy.
+
+---
+
+## AIMA4e
+__function__ POLICY-ITERATION(_mdp_) __returns__ a policy
+ __inputs__: _mdp_, an MDP with states _S_, actions _A_(_s_), transition model _P_(_s′_ | _s_, _a_)
+ __local variables__: _U_, a vector of utilities for states in _S_, initially zero
+        _π_, a policy vector indexed by state, initially random
+
+ __repeat__
+   _U_ ← POLICY\-EVALUATION(_π_, _U_, _mdp_)
+   _unchanged?_ ← true
+   __for each__ state _s_ __in__ _S_ __do__
+&emsp;&emsp;&emsp;&emsp;&emsp;_a <sup> &#x2a; </sup>_ &larr; argmax<sub>_a_ &isin; _A_(_s_)</sub> Q-VALUE(_mdp_,_s_,_a_,_U_)
+&emsp;&emsp;&emsp;&emsp;&emsp;__if__ Q-VALUE(_mdp_,_s_,_a<sup>&#x2a;</sup>_,_U_) &gt; Q-VALUE(_mdp_,_s_,_&pi;_\[_s_\],_U_) __then do__  
+&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;_&pi;_\[_s_\] &larr; _a<sup>&#x2a;</sup>_ ; _unchanged?_ &larr; false  
+&emsp;__until__ _unchanged?_
+&emsp;__return__ _&pi;_
+
+---
+__Figure ??__ The policy iteration algorithm for calculating an optimal policy.

md/Value-Iteration.md

@@ -18,3 +18,25 @@ __function__ VALUE-ITERATION(_mdp_, _ε_) __returns__ a utility function
 
 ---
 __Figure ??__ The value iteration algorithm for calculating utilities of states. The termination condition is from Equation (__??__).
+
+---
+
+## AIMA4e
+__function__ VALUE-ITERATION(_mdp_, _ε_) __returns__ a utility function
+ __inputs__: _mdp_, an MDP with states _S_, actions _A_(_s_), transition model _P_(_s′_ | _s_, _a_),
+      rewards _R_(_s_,_a_,_s′_), discount _γ_
+   _ε_, the maximum error allowed in the utility of any state
+ __local variables__: _U_, _U′_, vectors of utilities for states in _S_, initially zero
+        _δ_, the maximum change in the utility of any state in an iteration
+
+ __repeat__
+   _U_ ← _U′_; _δ_ ← 0
+   __for each__ state _s_ in _S_ __do__
+&emsp;&emsp;&emsp;&emsp;&emsp;_U&prime;_\[_s_\] &larr; max<sub>_a_ &isin; _A_(_s_)</sub> Q-VALUE(_mdp_,_s_,_a_,_U_)
+&emsp;&emsp;&emsp;&emsp;&emsp;__if__ &vert; _U&prime;_\[_s_\] &minus; _U_\[_s_\]  &vert; &gt; _&delta;_ __then__ _&delta;_ &larr; &vert; _U&prime;_\[_s_\] &minus; _U_\[_s_\]  &vert;
+&emsp;__until__ _&delta;_ &lt; _&epsi;_(1 &minus; _&gamma;_)&sol;_&gamma;_
+&emsp;__return__ _U_
+
+---
+__Figure ??__ The value iteration algorithm for calculating utilities of states. The termination condition is from Equation (__??__).
+~