Teen-Different commited on
Commit
cf14949
·
verified ·
1 Parent(s): 716c31e

Upload 25 files

Browse files
.gitattributes CHANGED
@@ -33,3 +33,6 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ images/all_algos_testing.png filter=lfs diff=lfs merge=lfs -text
37
+ images/all_algos.png filter=lfs diff=lfs merge=lfs -text
38
+ Report.pdf filter=lfs diff=lfs merge=lfs -text
Enhanced Environemnt Simulation.gif ADDED
README.md CHANGED
@@ -1,3 +1,230 @@
1
- ---
2
- license: mit
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # RxRovers: Roaming for Rapid Relief
2
+
3
+ <span style="color: blue;">Dynamic Obstacles and Path Optimization
4
+
5
+ ![Trained Agents in Action](images/trained_agents.gif)
6
+
7
+ ## Project Overview
8
+
9
+ "RxRovers: Roaming for Rapid Relief" aims to integrate advanced reinforcement learning (RL) into the healthcare domain, specifically focusing on optimizing medical supply delivery within hospital settings. This project seeks to deploy autonomous agents, RxRovers, which are programmed to navigate through hospital corridors efficiently, dodging any potential obstacles (dynamic and static) to ensure the timely distribution of medicines. This innovation can significantly enhance patient care and outcomes.
10
+
11
+ ## Team Members
12
+
13
+ - Charvi Kusuma [GitHub](https://github.com/kcharvi)
14
+ - Tarun Reddi [GitHub](https://github.com/REDDITARUN)
15
+
16
+ ### Real-World Application
17
+
18
+ The RxRovers project directly addresses a real-world healthcare problem: simulating delivery of medical supplies efficiently within hospital environments. This project integrates RL into the healthcare domain to enhance this delivery system, directly benefiting patient care and outcomes.
19
+
20
+ ### Complex Navigation Challenges
21
+
22
+ The project’s challenges mirror real-world navigation complexities in hospital settings:
23
+
24
+ - **Dynamic and Static Obstacles**: Accurate representation of unpredictable hospital environments.
25
+ - **Path Planning**: Optimization to ensure timely and efficient delivery of medical supplies.
26
+
27
+ ### Adaptable Environment
28
+
29
+ The project's environment can be modified to reflect various real-world hospital layouts, demonstrating its adaptability to different healthcare settings.
30
+
31
+ ## Objectives
32
+
33
+ 1. Develop RL agents capable of autonomously navigating hospital environments while delivering medical supplies.
34
+ 2. Optimize path planning strategies to ensure timely and efficient delivery of medicines, while avoiding obstacles such as equipment, humans, and environmental constraints.
35
+ 3. Enhance the visual representation and user experience of the simulated hospital environment to improve engagement and realism.
36
+ 4. Conduct comparative analysis of various reinforcement learning algorithms to identify the optimal approach for medical supply delivery optimization within hospital environments.
37
+
38
+ ## Environment
39
+
40
+ ### Simulated Hospital Layouts
41
+
42
+ The project creates a hospital environment that mirrors real-world scenarios. The environment includes features such as:
43
+
44
+ - **Hospital Corridors**: Multiple corridors, rooms, and operation desks.
45
+ - **Obstacles**: Dynamic obstacles (humans, other rovers) and static obstacles (rooms, walls).
46
+ - **RxRovers**: Autonomous agents programmed to navigate these environments, avoid obstacles, and reach designated destinations.
47
+
48
+ ### Initial Stage
49
+
50
+ - **Grid**: 9×9 grid simulating rovers navigating on a grid.
51
+ - **Setup**: Initialized action and observation spaces, set up grid size, starting points, destinations, and parameters such as rewards and penalties for actions and events.
52
+
53
+ ### Refined Version
54
+
55
+ - **Grid Size**: 15×15 grid representing the hospital environment.
56
+ - **Rovers**: Two agents represented by blue squares.
57
+ - **Targets**: Yellow target destinations for delivering medicine.
58
+ - **Actions**: Move down, up, left, right, or stay still (5 possible actions).
59
+ - **Operation Desks**: Dark green squares representing starting points.
60
+ - **Rooms**: Black squares indicating static obstacles.
61
+ - **Human**: Purple square representing a moving human obstacle.
62
+ - **Observation Space**: Positions of the Rovers, human, rooms, operation desks, and the grid boundary.
63
+ - **Rewards**:
64
+ - +30 for moving closer to targets.
65
+ - +100 for reaching destinations.
66
+ - -15 for collisions or moving out of grid bounds.
67
+ - -5 for waiting near the human obstacle.
68
+ - -20 for moving away from targets.
69
+ - **Termination**: Episode ends if both Rovers reach their targets or maximum time steps (20) are reached.
70
+
71
+ ![Environment Image](images/enhanced-environment.png)
72
+
73
+ ## Algorithms
74
+
75
+ ### Q-Learning (QL)
76
+
77
+ Implemented Q-Learning for a rover agent navigating a grid environment.
78
+
79
+ - **Settings**:
80
+ - State Representation: Positions of the two rovers and the human.
81
+ - Action Space: Move up, down, left, right, or stay in place.
82
+ - Rewards: Based on interactions (moving closer to target, collisions, reaching target).
83
+ - Done Flag: Episode terminates when both rovers reach targets or max time steps are reached.
84
+ - **Hyperparameters**:
85
+ - Alpha (Learning Rate): 1e-4
86
+ - Gamma (Discount Factor): 0.9
87
+ - Epsilon (Exploration Rate): 0.5
88
+ - Epsilon Decay: 0.995
89
+ - Epsilon Minimum: 0.01
90
+ - **Training Phase**:
91
+ - Total Rewards per Episode
92
+ - Epsilon Decay Curve
93
+ - **Evaluation Phase**:
94
+ - Evaluation Rewards per Episode
95
+
96
+ ### Double Q-Learning (DQL)
97
+
98
+ Implemented Double Q-Learning to mitigate overestimation biases.
99
+
100
+ - **Settings**: Same as Q-Learning.
101
+ - **Hyperparameters**: Same as Q-Learning.
102
+ - **Training Phase**:
103
+ - Total Rewards per Episode
104
+ - **Evaluation Phase**:
105
+ - Evaluation Rewards
106
+
107
+ ### Deep Q Network (DQN)
108
+
109
+ Utilized a neural network to approximate the Q-values.
110
+
111
+ - **Settings**:
112
+ - Neural Network Architecture: Two hidden layers with ReLU activation.
113
+ - Replay Memory: Stores past experiences for experience replay.
114
+ - Select Action Function: Epsilon-greedy strategy.
115
+ - Optimize Model Function: Gradient descent on the Q-network.
116
+ - **Hyperparameters**:
117
+ - Number of Episodes: 1000
118
+ - Target Network Update Frequency: 10 episodes
119
+ - Batch Size: 256
120
+ - Discount Factor: 0.9
121
+ - Learning Rate: 0.001
122
+ - Epsilon Initial Value: 1
123
+ - Epsilon Final Value: 0.05
124
+ - Epsilon Decay: 10000
125
+ - Maximum Timestamps per Episode: 30
126
+ - **Training Phase**:
127
+ - Total Rewards per Episode
128
+ - Epsilon Decay Curve
129
+ - **Evaluation Phase**:
130
+ - Evaluation Rewards
131
+
132
+ ### Double Deep Q Network (DDQN)
133
+
134
+ Addresses overestimation bias by decoupling selection and evaluation of the action.
135
+
136
+ - **Settings**:
137
+ - Optimize Model Function: Gradient descent with gradients clipping.
138
+ - **Hyperparameters**:
139
+ - Batch Size: 256
140
+ - Gamma: 0.9
141
+ - Learning Rate: 0.001
142
+ - Epsilon Start: 1
143
+ - Epsilon End: 0.05
144
+ - Epsilon Decay: 10,000
145
+ - Number of Episodes: 1000
146
+ - Target Update: Every 10 episodes
147
+ - **Training Phase**:
148
+ - Total Rewards per Episode
149
+ - Epsilon Decay Curve
150
+ - **Evaluation Phase**:
151
+ - Evaluation Rewards
152
+
153
+ ### Proximal Policy Optimization (PPO)
154
+
155
+ A robust and efficient algorithm developed by OpenAI.
156
+
157
+ - **Settings**:
158
+ - Neural Network Architecture: Actor and critic heads.
159
+ - Training Loop: States, actions, rewards, values, and log-probs collected in replay buffer.
160
+ - Loss Functions: Policy and value losses.
161
+ - **Hyperparameters**:
162
+ - Total Timesteps: 50,000
163
+ - Gamma: 0.99
164
+ - Lambda: 0.95
165
+ - Epsilon: 0.2
166
+ - Epochs: 3
167
+ - Batch Size: 64
168
+ - Learning Rate: 0.001
169
+ - **Training Phase**:
170
+ - Total Rewards per Episode
171
+ - **Evaluation Phase**:
172
+ - Evaluation Rewards
173
+
174
+ ### Actor Critic (A2C)
175
+
176
+ Combines elements of both policy-based methods (Actor) and value-based methods (Critic).
177
+
178
+ - **Settings**:
179
+ - Actor Network: Learns a policy π(s).
180
+ - Critic Network: Learns the value function V(s).
181
+ - Advantage: Measures action quality.
182
+ - Policy and Value Function Updates: Gradient descent.
183
+ - **Training Phase**:
184
+ - Total Rewards per Episode
185
+ - **Evaluation Phase**:
186
+ - Evaluation Rewards
187
+
188
+ **Total Rewards Plot during Training:**
189
+
190
+ ![Plots](images/all_algos.png)
191
+
192
+ **Evaluation for 10 Timestamps:**
193
+
194
+ ![Evaluation Plots](images/all_algos_testing.png)
195
+
196
+ ## Comparison
197
+
198
+ Comparison of different reinforcement learning algorithms and their performance:
199
+
200
+ | Aspect | Q-Learning | Double Q-Learning | DQN | DDQN | PPO | A2C |
201
+ | ------------------------- | ----------------------- | ----------------------- | -------------------- | ------------------------------ | ----------------------------- | ----------------------------- |
202
+ | Algorithm Type | Model-free, Value-based | Model-free, Value-based | Value-based, Deep NN | Value-based, Deep NN | Actor-Critic, Policy Gradient | Actor-Critic, Policy Gradient |
203
+ | Exploration-Exploitation | Epsilon-greedy | Epsilon-greedy | Epsilon-greedy | Epsilon-greedy | Continuous | Continuous |
204
+ | Stability & Learning Rate | No target network | No target network | Target network | Double updates, target network | Adaptive, Gradient Clipping | Adaptive, Entropy Bonus |
205
+ | Convergence | Slow | Moderate | Fast | Faster | Fast | Moderate |
206
+ | Memory Requirements | Low | Low | High | High | Moderate | Moderate |
207
+ | Adaptability | Moderate | Moderate | Low to Moderate | Low to Moderate | High | Moderate |
208
+ | Generalization | Low to Moderate | Low to Moderate | Low | Low | High | Moderate |
209
+
210
+ ## Challenges Addressed
211
+
212
+ 1. **Navigating a Dynamic World**: Rovers learned to dodge moving obstacles using dynamic obstacle handling techniques.
213
+ 2. **Custom Paths for Diverse Layouts**: Environment adaptable to various hospital settings.
214
+ 3. **Learning Optimal Path**: Rovers trained to take the shortest route to delivery rooms, avoiding obstacles.
215
+
216
+ ## Bonus: Real-World Application
217
+
218
+ 1. **Healthcare System Integration**: Enhances delivery system within hospital settings, benefiting patient care and outcomes.
219
+ 2. **Simulating Hospital Layouts**: Reflects real-world scenarios with hospital corridors, obstacles, and RxRovers.
220
+ 3. **Complex Navigation Challenges**: Optimizes path planning and avoids dynamic/static obstacles.
221
+ 4. **Adaptable Environment**: Can be modified to different healthcare settings.
222
+
223
+ ## References
224
+
225
+ 1. [Deep Reinforcement Learning DQN for Multi-Agent Environment](https://medium.com/yellowme/deep-reinforcement-learning-dqn-for-multi-agent-environment-5f4fae1a9ff5)
226
+ 2. [Warehouse Robot Path Planning](https://github.com/LyapunovJingci/Warehouse_Robot_Path_Planning) for path planning and optimization with obstacle avoidance.
227
+ 3. [Reinforcement Learning (DQN) Tutorial | PyTorch Tutorials](https://pytorch.org/tutorials/intermediate/reinforcement_q_learning.html)
228
+ 4. [Deep Q Learning (DQN) using PyTorch](https://medium.com/@vignesh.g1609/deep-q-learning-dqn-using-pytorch-a31f02a910ac)
229
+
230
+ ---
Report.pdf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a28e62aa4cfc411d65a8b745759d810d14b0674110e044bfbd672f0f991f26ea
3
+ size 3280745
images/all_algos.png ADDED

Git LFS Details

  • SHA256: 53e0b0a16f06d882fb595c8f0c2b39e3b774ba814e0a6cc058642491fcff0bc0
  • Pointer size: 131 Bytes
  • Size of remote file: 275 kB
images/all_algos_testing.png ADDED

Git LFS Details

  • SHA256: ba14f095759a0cb693ec4d5a9a6c59b0b19d1a8d6671a80591d0d925cf7fc9b0
  • Pointer size: 131 Bytes
  • Size of remote file: 188 kB
images/desk.png ADDED
images/enhanced-environment.png ADDED
images/human.png ADDED
images/room.png ADDED
images/rover_desk_collision.png ADDED
images/rover_dest.png ADDED
images/rover_human_collision.png ADDED
images/rover_moving.png ADDED
images/rover_room_collision - Copy.png ADDED
images/rover_rover_collision.png ADDED
images/rover_start.png ADDED
images/target.png ADDED
images/trained_agents.gif ADDED
rx_rover_A2C.ipynb ADDED
The diff for this file is too large to render. See raw diff
 
rx_rover_DDQN.ipynb ADDED
The diff for this file is too large to render. See raw diff
 
rx_rover_DQN.ipynb ADDED
The diff for this file is too large to render. See raw diff
 
rx_rover_Double_Q_Learning.ipynb ADDED
The diff for this file is too large to render. See raw diff
 
rx_rover_Enhanced_Environment.ipynb ADDED
@@ -0,0 +1,278 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "cells": [
3
+ {
4
+ "cell_type": "markdown",
5
+ "metadata": {},
6
+ "source": [
7
+ "# Environment - Enhanced"
8
+ ]
9
+ },
10
+ {
11
+ "cell_type": "code",
12
+ "execution_count": 2,
13
+ "metadata": {},
14
+ "outputs": [],
15
+ "source": [
16
+ "import gym\n",
17
+ "from gym import spaces\n",
18
+ "import numpy as np\n",
19
+ "import matplotlib.pyplot as plt\n",
20
+ "\n",
21
+ "import os\n",
22
+ "import random\n",
23
+ "import imageio\n",
24
+ "from tqdm import tqdm \n",
25
+ "from itertools import count\n",
26
+ "from collections import namedtuple, deque\n",
27
+ "\n",
28
+ "import torch\n",
29
+ "import torch.nn as nn\n",
30
+ "import torch.optim as optim\n",
31
+ "import random\n",
32
+ "import matplotlib.image as mpimg"
33
+ ]
34
+ },
35
+ {
36
+ "cell_type": "code",
37
+ "execution_count": 3,
38
+ "metadata": {},
39
+ "outputs": [],
40
+ "source": [
41
+ "class RoverGridEnv(gym.Env):\n",
42
+ " metadata={'render.modes': ['human']} \n",
43
+ " def __init__(self, max_ts=20): \n",
44
+ " super(RoverGridEnv,self).__init__()\n",
45
+ " self.max_ts=max_ts # The Max_Timestamps is set to 20 by default.\n",
46
+ " self.grid_size=(15,15) \n",
47
+ " self.action_space=spaces.Discrete(5) \n",
48
+ " self.observation_space=spaces.MultiDiscrete([15,15,15,15,15,15])\n",
49
+ " self.rover_positions=np.array([[6,4],[10,4]])\n",
50
+ " self.operation_desks=np.array([[6,3],[10,3]])\n",
51
+ " self.rooms=np.array([[4,7],[4,10],[4,13],[8,7],[8,10],[8,13],[12,7],[12,10],[12,13]])\n",
52
+ " self.human_position=np.array([8,9])\n",
53
+ " self.targets=np.array([[5,10],[9,13]])\n",
54
+ " self.actions=[(0,-1),(0,1),(-1,0),(1,0),(0,0)] # Down,Up,Left,Right,Wait\n",
55
+ " self.rover_done=[False,False] \n",
56
+ " self.reset()\n",
57
+ " \n",
58
+ " def seed(self,seed=None):\n",
59
+ " np.random.seed(seed)\n",
60
+ " random.seed(seed)\n",
61
+ " \n",
62
+ " def reset(self):\n",
63
+ " self.current_step=0\n",
64
+ " self.rover_positions=np.array([[6,4],[10,4]])\n",
65
+ " self.rover_done=[False,False]\n",
66
+ " self.human_position=np.array([7,8])\n",
67
+ " self.current_step=0\n",
68
+ " return self._get_obs()\n",
69
+ " \n",
70
+ " def _get_obs(self):\n",
71
+ " return np.concatenate((self.rover_positions.flatten(),self.human_position))\n",
72
+ " \n",
73
+ " def step(self,actions):\n",
74
+ " rewards=np.zeros(2)\n",
75
+ " done=[False,False]\n",
76
+ " info={'message': ''} \n",
77
+ " for i,action in enumerate(actions):\n",
78
+ " if self.rover_done[i]:\n",
79
+ " done[i]=True \n",
80
+ " continue\n",
81
+ " prev_distance=np.linalg.norm(self.targets[i]-self.rover_positions[i])\n",
82
+ " if self._is_human_adjacent(self.rover_positions[i]):\n",
83
+ " rewards[i] -= 5\n",
84
+ " else:\n",
85
+ " delta=np.array(self.actions[action])\n",
86
+ " new_position=self.rover_positions[i]+delta\n",
87
+ " if self._out_of_bounds(new_position):\n",
88
+ " rewards[i] -= 15\n",
89
+ " continue\n",
90
+ " if self._collision(new_position,i):\n",
91
+ " rewards[i] -= 15\n",
92
+ " continue\n",
93
+ " self.rover_positions[i]=new_position\n",
94
+ " new_distance=np.linalg.norm(self.targets[i]-new_position)\n",
95
+ " if new_distance < prev_distance:\n",
96
+ " rewards[i]+=30 \n",
97
+ " else:\n",
98
+ " rewards[i] -= 20 \n",
99
+ " if np.array_equal(new_position,self.targets[i]):\n",
100
+ " rewards[i]+=100\n",
101
+ " self.rover_done[i]=True \n",
102
+ " done[i]=True\n",
103
+ "\n",
104
+ " # move human randomly\n",
105
+ " self._move_human()\n",
106
+ " self.current_step+=1\n",
107
+ " all_done=all(done) or self.current_step >= self.max_ts\n",
108
+ " if all_done and not all(done): # if the maximum number of steps is reached but not all targets were reached\n",
109
+ " info['message']='Maximum number of timestamps reached'\n",
110
+ " return self._get_obs(),rewards,all_done,info\n",
111
+ "\n",
112
+ " def _is_human_adjacent(self,position):\n",
113
+ " for delta in [(1,1),(1,-1),(-1,1),(-1,-1)]:\n",
114
+ " adjacent_position=position+np.array(delta)\n",
115
+ " if np.array_equal(adjacent_position,self.human_position):\n",
116
+ " return True\n",
117
+ " return False\n",
118
+ "\n",
119
+ " def _out_of_bounds(self,position):\n",
120
+ " return not (0 <= position[0] < self.grid_size[0] and 0 <= position[1] < self.grid_size[1])\n",
121
+ " \n",
122
+ " def _collision(self,new_position,rover_index):\n",
123
+ " if any(np.array_equal(new_position,pos) for pos in np.delete(self.rover_positions,rover_index,axis=0)):\n",
124
+ " return True # Collision with the other rover\n",
125
+ " if any(np.array_equal(new_position,pos) for pos in self.rooms):\n",
126
+ " return True # Collision with a room\n",
127
+ " if any(np.array_equal(new_position,pos) for pos in self.operation_desks):\n",
128
+ " return True # Collision with an operation desk\n",
129
+ " if np.array_equal(new_position,self.human_position):\n",
130
+ " return True # Collision with the human\n",
131
+ " return False\n",
132
+ " \n",
133
+ " def _move_human(self):\n",
134
+ " valid_moves=[move for move in self.actions if not self._out_of_bounds(self.human_position+np.array(move))]\n",
135
+ " self.human_position+=np.array(valid_moves[np.random.choice(len(valid_moves))])\n",
136
+ " \n",
137
+ " # def render(self,mode='human',save_path=None):\n",
138
+ " # fig,ax=plt.subplots(figsize=(7,7))\n",
139
+ " # ax.set_xlim(0,self.grid_size[0])\n",
140
+ " # ax.set_ylim(0,self.grid_size[1])\n",
141
+ " # ax.set_xticks(np.arange(0,15,1))\n",
142
+ " # ax.set_yticks(np.arange(0,15,1))\n",
143
+ " # ax.grid(which='both')\n",
144
+ "\n",
145
+ " # # draw elements\n",
146
+ " # for pos in self.rover_positions:\n",
147
+ " # ax.add_patch(Rectangle((pos[0]-0.5,pos[1]-0.5),1,1,color='blue'))\n",
148
+ " # for pos in self.operation_desks:\n",
149
+ " # ax.add_patch(Rectangle((pos[0]-0.5,pos[1]-0.5),1,1,color='darkgreen'))\n",
150
+ " # for pos in self.rooms:\n",
151
+ " # ax.add_patch(Rectangle((pos[0]-0.5,pos[1]-0.5),1,1,color='black'))\n",
152
+ " # ax.add_patch(Rectangle((self.human_position[0]-0.5,self.human_position[1]-0.5),1,1,color='purple'))\n",
153
+ " # for pos in self.targets:\n",
154
+ " # ax.add_patch(Rectangle((pos[0]-0.5,pos[1]-0.5),1,1,color='yellow',alpha=0.5))\n",
155
+ "\n",
156
+ " # if save_path is not None:\n",
157
+ " # plt.savefig(save_path)\n",
158
+ " # plt.close()\n",
159
+ " \n",
160
+ " # def close(self):\n",
161
+ " # plt.close()\n",
162
+ "\n",
163
+ " def render(self, mode='human', save_path=None):\n",
164
+ " fig, ax=plt.subplots(figsize=(7,7))\n",
165
+ " ax.set_xlim(0, self.grid_size[0])\n",
166
+ " ax.set_ylim(0, self.grid_size[1])\n",
167
+ "\n",
168
+ " rover_start_img_path='images/rover_moving.png' \n",
169
+ " rover_moving_img_path='images/rover_moving.png' \n",
170
+ " rover_dest_img_path='images/rover_dest.png' \n",
171
+ " rover_human_collision_path='images/rover_human_collision.png' \n",
172
+ " rover_room_collision_path='images/rover_room_collision.png' \n",
173
+ " rover_desk_collision_path='images/rover_desk_collision.png' \n",
174
+ " rover_rover_collision_path='images/rover_rover_collision.png' \n",
175
+ " desk_img_path='images/desk.png'\n",
176
+ " room_img_path='images/room.png' \n",
177
+ " human_img_path='images/human.png' \n",
178
+ " target_img_path='images/target.png' \n",
179
+ "\n",
180
+ "\n",
181
+ " for i, pos in enumerate(self.rover_positions):\n",
182
+ " \n",
183
+ " if self.rover_done[i]:\n",
184
+ " rover_img=mpimg.imread(rover_dest_img_path) \n",
185
+ " elif np.array_equal(pos, self.rover_positions[i]):\n",
186
+ " rover_img=mpimg.imread(rover_start_img_path) \n",
187
+ " elif self._is_human_adjacent(pos):\n",
188
+ " rover_img=mpimg.imread(rover_human_collision_path) \n",
189
+ " elif self._collision(pos, i):\n",
190
+ " \n",
191
+ " if any(np.array_equal(pos, pos) for pos in self.rooms):\n",
192
+ " rover_img=mpimg.imread(rover_room_collision_path)\n",
193
+ " elif any(np.array_equal(pos, pos) for pos in self.operation_desks):\n",
194
+ " rover_img=mpimg.imread(rover_desk_collision_path)\n",
195
+ " elif any(np.array_equal(pos, self.rover_positions[i]) for i in range(len(self.rover_positions)) if i != 0):\n",
196
+ " rover_img=mpimg.imread(rover_rover_collision_path)\n",
197
+ " else:\n",
198
+ " rover_img=mpimg.imread(rover_moving_img_path) \n",
199
+ " ax.imshow(rover_img, extent=(pos[0]-0.5, pos[0]+0.5, pos[1]-0.5, pos[1]+0.5))\n",
200
+ "\n",
201
+ " \n",
202
+ " for pos in self.operation_desks:\n",
203
+ " desk_img=mpimg.imread(desk_img_path)\n",
204
+ " ax.imshow(desk_img, extent=(pos[0]-0.5, pos[0]+0.5, pos[1]-0.5, pos[1]+0.5))\n",
205
+ "\n",
206
+ " for pos in self.rooms:\n",
207
+ " room_img=mpimg.imread(room_img_path)\n",
208
+ " ax.imshow(room_img, extent=(pos[0]-0.5, pos[0]+0.5, pos[1]-0.5, pos[1]+0.5))\n",
209
+ "\n",
210
+ " human_img=mpimg.imread(human_img_path)\n",
211
+ " ax.imshow(human_img, extent=(self.human_position[0]-0.5, self.human_position[0]+0.5, self.human_position[1]-0.5, self.human_position[1]+0.5))\n",
212
+ "\n",
213
+ " for pos in self.targets:\n",
214
+ " target_img=mpimg.imread(target_img_path)\n",
215
+ " ax.imshow(target_img, extent=(pos[0]-0.5, pos[0]+0.5, pos[1]-0.5, pos[1]+0.5))\n",
216
+ "\n",
217
+ " if save_path is not None:\n",
218
+ " plt.savefig(save_path)\n",
219
+ " plt.close()\n",
220
+ "\n",
221
+ " def close(self):\n",
222
+ " plt.close()\n",
223
+ "\n"
224
+ ]
225
+ },
226
+ {
227
+ "cell_type": "code",
228
+ "execution_count": 4,
229
+ "metadata": {},
230
+ "outputs": [
231
+ {
232
+ "name": "stdout",
233
+ "output_type": "stream",
234
+ "text": [
235
+ "Initial Setup\n"
236
+ ]
237
+ },
238
+ {
239
+ "data": {
240
+ "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkoAAAJGCAYAAACk4ariAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8g+/7EAAAACXBIWXMAAA9hAAAPYQGoP6dpAABD2ElEQVR4nO3deXhU9d3//9eZJZM9JIEkRBMISF0AQUUUcYFKpdbbhd5urVKq9693F1zpbdW7X2ytC7WLpbYWb717W3vfdekiaLXVUkRQlDWCIoqA7CEhLMkkk8x6Pr8/IpEIJ2SZyUyS5+O65uKaM2fOec9cb8555ZzPnGMZY4wAAABwBFeyCwAAAEhVBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHnmQX8Fm2bauqqko5OTmyLCvZ5QAAgD7GGKOGhgaVlpbK5Wr/mFHKBaWqqiqVlZUluwwAANDH7dy5U8cff3y786RcUMrJyZHUUnxubm6SqwEAAH2N3+9XWVlZa+ZoT8oFpUOn23JzcwlKAAAgYToyxIfB3AAAAA4ISgAAAA4ISgAAAA4ISgAAAA4ISgAAAA4ISgAAAA4ISgAAAA4ISgAAAA4ISgAAAA4ISgAAAA4ISgAAAA4ISgAAAA4ISgAAAA4ISgAAAA4ISgAAAA4ISgAAAA4ISgAAAA4ISgAAAA4ISgAAAA4ISgAAAA4ISgAAAA4ISgAAAA4ISgAAAA4ISgAAAA4ISgAAAA4ISgAAAA4ISgAAAA4ISgAAAA4ISgAAAA46HZSWLl2qSy+9VKWlpbIsSwsWLHCc91vf+pYsy9LcuXO7USIAAEBydDooBQIBjRkzRo8++mi7882fP1/Lly9XaWlpl4sDAABIJk9n33DxxRfr4osvbnee3bt36+abb9arr76qSy65pMvFAQAAJFOng9Kx2Lat6dOn64477tDIkSOPOX8oFFIoFGp97vf7410SAABAl8R9MPdDDz0kj8ejW265pUPzz5kzR3l5ea2PsrKyeJcEAADQJXENSmvWrNEvf/lL/e53v5NlWR16z9133636+vrWx86dO+NZEgAAQJfFNSi98cYb2rt3r8rLy+XxeOTxeLR9+3Z997vf1dChQ4/6Hp/Pp9zc3DYPAACAVBDXMUrTp0/XlClT2kybOnWqpk+frhtuuCGeqwIAAEi4TgelxsZGbd68ufX51q1btXbtWhUUFKi8vFyFhYVt5vd6vSopKdGJJ57Y/WoBAAB6UKeD0urVqzV58uTW57NmzZIkzZgxQ7/73e/iVhgAAECydTooTZo0ScaYDs+/bdu2zq4CAAAgJXCvNwAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAeeZBcAdEYkElE0Gu3WMjwej7xeb5wqQm/S0/1jG6O/vbO3W+vrri+dViSXZSW1hr6C7U//RFBCr9DY2Kjnn39BK5YsU2D/QRnb7tJyLJdLmYX5Gn/eBF155TRlZ2fHuVKkomT1j20bvfBOja6ZOLRL6+uuZ97Yqi+OGSSXm6DUHWx/+jeCElJeY2Oj7r3nPlVUNWpW+XDlD85QV/9ANsboYDioV155Wz96b4Puuff7bKz6uGT3T0aaWycMzunaCrspPc2dlPX2JcnuHyQfQQkp7/nnX1BFVaOuGzFKtqSGcEhul0sul6U0r1fNoZDS09IUjkQVibV/WNySpQJfpq4fMVr/u+ldLZj/oq6f/tWe+SBIilTpn5fX7NY/1lZJ6rmjO+ForMfW1VelSv8geQhKSGmRSEQrlizTrPLhsiW91lyrzKHFysvJktfjUVF+vnZU16hkYIEO+BtU3xBod3nRWFRN22p0YUaRvlg2XL94fZmuufYqxgz0UanUP7GYrQvGVuik8sI4fbpj+9PrG3psXX1RKvUPkoeghJQWjUYV2F+n/MEZagiHlDm0WOedfbqsw459Dyxu2fGUHldyzOUZY7TUrFHj7qDyfRkK7D+oWCzGhqqPSsX+sXpwYDUjk7onFfsHPY/LAyDlGTsmy5KMjNxud7d2NJZlye1xy5iWw+DG5tREX0f/oDvoHxCUAKCHcIQH6H0ISgCQNKYH5wHQFYxRAoAe0hA22hf4bKhJXFiKdO1yPwAOQ1ACgB4SikpNkZ5bXxeviwjgMJx6AwAAcMARJQBIAGOMvvrrSgXCMd3229WK2baMLL27cVuP1RCLGV3/m0pJ0rM3j+ux9QJ9CUEJABIkK92t5245I9llaNovViW7BKDXIigBQIKkuS01R4y8SbwpbcwYeVxcmADoKoIS+h2XZXE9G3RZZ/pn3LAB+seGgxpbUdClde0+0CRLUmlBZpfeL0l7DjZraFFWl9+P+GL70/swmBv9zugThiknq+s7HvRvnemfy8cN1potBxSKdO0KzJuqGrRpT0OX3nvIlj0Nmjp6ULeWgfhh+9P7EJTQ72RnZsjjdie7DPRSHe0fy7KUn+XVrv1N2rbX6Wappt1LJE0aVawLRhZ14DJKzsvZVhvQmPLcY9aLnsH2p/chKAFAgliSDjSEtH5HncJR+yhZxjrivibxnCdmG1UdCKggJ62TlQM4hKCEfmdXTa2aQ6Fkl4FeqrP9k+Z1KRy1tXNfoENjU+I5z/6GkI7LT1cSx5LjM9j+9D4EJaQ8y+WWMUaWLEVjURnT9ftaGWP08a4qBcNhGRlZLg6B93XJ7p+CbJ8G5vr0cU2jbLtn78m2szags0/o2kBytEh2/yD5+NUbUprH41Fm4QAdDAdV4MtU07YaLTVr5PZ0bQMTi8bUtK1a2ZnF2hcMKKswX27GC/RZqdA/Pq9LQwZlacPOOu2tD6poQHqX1t0VW2oa9NWzS3psfX1NKvQPko+ghJTm9Xo1YdJEvfLyMl0/YrQuzChS4+6guvpHnWVJ2ZnFckl6ZccWTbj0fHm93rjWjNSRKv1TWpCpyo8P6KH573dtxV0UisR085TyHl1nX5Iq/YPkIigh5U2bdrnuW/e+fr/pXV1cNlz5vgx19UokRkb7ggG9smOLqoYU6MZpl8W5WqSaVOif7HSPrj13qK49d2iX1ttVP13wvrxuS5bFIKWuSoX+QXJZpjsnXBPA7/crLy9P9fX1ys3lJ61oEQgEtOCFv+qtRW8osP+gjN2169JYLreyCvM1YfK5mjbtMmVlcSG+/iBZ/RON2frWk+t11TlDurS+7npu2TY9fuNoedwMR+0Otj99T2eyBkEJvUokElEs1rWN1CFut5vD3f1UT/ePbYwWvlfbrfV11xdGD5KLI0pxwfan7+hM1uDUG3oVr9fLRgZd1tP947IsTT21qMfWh8Ri+9M/cTwWAADAAUEJAADAAUEJAADAAUEJAADAAUEJAADAAUEJAADAAUEJAADAAUEJAADAAUEJAADAAUEJAADAAUEJAADAAUEJAADAAUEJAADAAUEJAADAAUEJAADAAUEJAADAAUEJAADAAUEJAADAQaeD0tKlS3XppZeqtLRUlmVpwYIFra9FIhHdeeedGj16tLKyslRaWqqvfe1rqqqqimfNAAAAPaLTQSkQCGjMmDF69NFHj3itqalJlZWVmj17tiorK/X8889r48aNuuyyy+JSLAAAQE+yjDGmy2+2LM2fP19XXHGF4zyrVq3S+PHjtX37dpWXlx9zmX6/X3l5eaqvr1dubm5XSwMAADiqzmQNT6KLqa+vl2VZGjBgwFFfD4VCCoVCrc/9fn+iSwIAAOiQhA7mDgaDuvPOO/WVr3zFMbHNmTNHeXl5rY+ysrJElgQAANBhCQtKkUhEV199tYwxmjdvnuN8d999t+rr61sfO3fuTFRJAAAAnZKQU2+HQtL27dv12muvtXv+z+fzyefzJaIMAACAbol7UDoUkjZt2qTFixersLAw3qsAAADoEZ0OSo2Njdq8eXPr861bt2rt2rUqKCjQ4MGDdeWVV6qyslIvvfSSYrGYqqurJUkFBQVKS0uLX+UAAAAJ1unLA7z++uuaPHnyEdNnzJihH/7wh6qoqDjq+xYvXqxJkyYdc/lcHgAAACRSQi8PMGnSJLWXrbpxWSYAAICUwr3eAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHHiSXQDQGZFIRNFotFvL8Hg88nq9HZrXNkZ/e2dvt9bXXV86rUguy0pqDX1FT/cP+hb6p38iKKFXaGxs1PPPv6AVS5YpsP+gjG13aTmWy6XMwnyNP2+CrrxymrKzs9ud37aNXninRtdMHNql9XXXM29s1RfHDJLLTVDqjmT1D/oG+qd/Iygh5TU2Nuree+5TRVWjZpUPV/7gDHX1AIsxRgfDQb3yytv60XsbdM+93z/mxiojza0TBud0bYXdlJ7mTsp6+5Jk9w96N/oHBCWkvOeff0EVVY26bsQo2ZIawiG5XS65XJbSvF41h0JKT0tTOBJVJNb+YXFLlgp8mbp+xGj976Z3tWD+i7p++lc7VMfLa3brH2urJPXc0Z1wNNZj6+qrUqV/0DvRPyAoIaVFIhGtWLJMs8qHy5b0WnOtMocWKy8nS16PR0X5+dpRXaOSgQU64G9QfUOg3eVFY1E1bavRhRlF+mLZcP3i9WW65tqrOjRmIBazdcHYCp1UXhinT3dsf3p9Q4+tqy9Kpf5B70P/QCIoIcVFo1EF9tcpf3CGGsIhZQ4t1nlnny7rsGPfA4tbgkvpcSXHXJ4xRkvNGjXuDirfl6HA/oOKxWKd2lBZPTiwmpFJ3ZOK/YPeg/6BxOUB0AsYOybLkoyM3G53t4KKZVlye9wypuUwuLE5tdXX0T/oDvoHBCWgEzjCAwD9C0EJ6BbTg/MAAHpap4PS0qVLdemll6q0tFSWZWnBggVtXjfG6J577tHgwYOVkZGhKVOmaNOmTfGqF0iqhrDRvsDhD33m+dEeXZ8n0rXLtQAA4qTTQSkQCGjMmDF69NFHj/r6T37yEz3yyCN67LHHtGLFCmVlZWnq1KkKBoPdLhZItlBUaor03KOL17UDAMRJp3/1dvHFF+viiy8+6mvGGM2dO1f/7//9P11++eWSpN///vcqLi7WggULdO2113avWgAAgB4U18sDbN26VdXV1ZoyZUrrtLy8PJ111ll6++23jxqUQqGQQqFQ63O/3x/PkoAuM8boq7+uVCAc022/Xa2YbcvI0rsbt/VYDbGY0fW/qZQkPXvzuB5bLwCgRVyDUnV1tSSpuLi4zfTi4uLW1z5rzpw5uvfee+NZBhA3WeluPXfLGckuQ9N+sSrZJQBAv5T0X73dfffdqq+vb33s3Lkz2SUBrdLclpojRlFbSXuEYkYeFxcmAIBkiOsRpZKSliuT1tTUaPDgwa3Ta2pqNHbs2KO+x+fzyefzxbMMoF0uy+rw9ZDGDRugf2w4qLEVBV1a1+4DTbIklRZkdun9krTnYLOGFmV1+f2Ir870D/BZ9E/vE9cjShUVFSopKdGiRYtap/n9fq1YsUITJkyI56qALht9wjDlZHUsuFw+brDWbDmgUKRrV9DdVNWgTXsauvTeQ7bsadDU0YO6tQzET2f6B/gs+qf36fQRpcbGRm3evLn1+datW7V27VoVFBSovLxct912m+6//36NGDFCFRUVmj17tkpLS3XFFVfEs26gy7IzMyS3W2Ed407flqX8LK927W/Str0BnXhc7lHmMpKxHC/ZPWlU8Sfz6BiX9XZezrbagC465fh2a0XP6Wj/AEdD//Q+nQ5Kq1ev1uTJk1ufz5o1S5I0Y8YM/e53v9P3vvc9BQIB/fu//7vq6up07rnn6pVXXlF6enr8qgZ6iCXpQENI63fUqaI4W16P6zNZ5shwc2Qm6vo8tm1UdSCggpy0Ln8GAEDXdTooTZo0ScY4327Bsiz96Ec/0o9+9KNuFQYkyq6aWhWGQnJ3cP40r0vhqK2d+wIaXpJzzPk7Mv6go/PsbwjpuPx0uRnUkDI62z/A4eif3ifpv3oDjsVyuWWMkSVL0Vi03aB+LMYYfbyrSsFwWEZGluvYm6uCbJ8G5vr0cU2jbLtn78m2szags0/o2kBytEh2/6B3o38Q11+9AfHm8XiUWThAB8NBFfgy1bStRkvNGrk9XdvAxKIxNW2rVnZmsfYFA8oqzJfb3f6yfF6XhgzK0oadddpbH1TRgJ47jbylpkFfPbukx9bX16RC/6D3on8gEZSQ4rxeryZMmqhXXl6m60eM1oUZRWrcHVRX/6izLCk7s1guSa/s2KIJl54vr9d7zPeVFmSq8uMDemj++11bcReFIjHdPKW8R9fZl6RK/6B3on8gEZTQC0ybdrnuW/e+fr/pXV1cNlz5vgx19UokRkb7ggG9smOLqoYU6MZpl3XofdnpHl177lBde+7QLq23q3664H153ZYsi0FKXZUK/YPei/6BZbpzwjUB/H6/8vLyVF9fr9zco/0cG/1RIBDQghf+qrcWvaHA/oMydteua2S53MoqzNeEyedq2rTLlJXV/oUcozFb33pyva46Z0iX1tddzy3bpsdvHC2Pm+GE3ZGs/kHfQP/0PZ3JGgQl9CqRSESxWNc2Uoe43e4OH+62jdHC92q7tb7u+sLoQXJxRCkuerp/0LfQP31HZ7IGp97Qq3i93h7dyLgsS1NPLeqx9SGxerp/0LfQP/0Tx/MBAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAceJJdAID+wxijWCwm27ZljGnzmmVZcrlccrvdsiwrSRUCQFsEJQAJZ4zR5qo9eito9Pq+Or17MKDqUEgNkahiRsryuDUwzauRA7L0+UEFmprj0bDBJckuGwAISgASr7ohqK9/tFcrDzQoqsOOJFluyZKabKk2GNEH1XWaX1Oni/Kz9eKggfJ42EQBSC7GKAFIuA3792nVAX/bkOQgZqQlBxtVHwz1QGUA0D6CEoCEam5u1hsbPlDMHDskHRKVtG7P3sQVBQAdRFACkDChUEgrFi/Uyo82duBY0qdiMVv/XLRQOzZ9lLDaAKAjCEoAEmbHpg+18uW/aKft6lRQsiVV7tmjN+Y/q5rqPYkqDwCOiaAEICECjY1a/Oc/qG5frRrTszv3Zkuq86Zrw5rl2vD2Utm2nZgiAeAYCEoAEuLDtWu048MNislSxO3t1HuNLIW9PhljtGrRK2rw1yeoSgBoH7+9BRB3kUhE6958TbYdU8ztVUa4ScOqt6qgYb9ym/zKaWqQNxqRzw4rP1M69QSPsgd49WEoT6/tzdPO9IFymZajSP59tfpwXaXOuuDCJH8qAP0RQQlA3G36YIP279ohS5a8sYi+vGy+JMmSkWWMXJaUl2E06nij8080yvS1vC/mlaaGXfrHO24Fwy2jmmzb1voVb2n8+Z/nit0AehxBCUDcbfnoQzU11MvIyJLkNp+OMUpzG4083uj0oUZDCiWP+9P3uVzS6eW26hqNln1kyciSJSlcX6fGhgbl5Ob2+GcB0L/FfYxSLBbT7NmzVVFRoYyMDA0fPlz33XffEfd1AtA3hUIhhev2KxwMHjbVyGUZHZdv9K/jbX1pjNGworYh6ZB0r3TeiS1HnVreadTc6Ff17l09Uj8AHC7uR5QeeughzZs3T0899ZRGjhyp1atX64YbblBeXp5uueWWeK8OQIpp8PvVXLe/zR9HliV9YaTRuGFGmWmfTGtnGeneT2b4ZBHB5oBq99ZoxMmnJKpsADiquAelt956S5dffrkuueQSSdLQoUP1zDPPaOXKlfFeFYAUFAgE1HBgf5tpbkua+Dkjj1v6YIdflmXppLKcNvM4TZekaDgsf31dIssGgKOK+6m3c845R4sWLdJHH7VcUXfdunV68803dfHFFx91/lAoJL/f3+YBoPcKBpvV1Njg+Prm3Y3avLuxw9MlKRqJKNDgvEwASJS4H1G666675Pf7ddJJJ8ntdisWi+mBBx7Qddddd9T558yZo3vvvTfeZQBIkmAwqObA0QOPJH3prMFHPe3mNF1q+eVbJBKJS30A0BlxP6L0xz/+UX/4wx/09NNPq7KyUk899ZR+9rOf6amnnjrq/Hfffbfq6+tbHzt37ox3SQB6UDQSUTQUcnzd7bLkch0ZiZymS5IxRtFoNG41AkBHxf2I0h133KG77rpL1157rSRp9OjR2r59u+bMmaMZM2YcMb/P55PP54t3GQCSxLZtRWOxttOMtOtAVEMGuo95LSRjjHYfiMmYtMMnyhhuYwKg58X9iFJTU5NcrraLdbvd3KsJ6Cd86enKzM5ucxotZku/W9yovfV2u5cKMcZorz+mJxc3Kta6ybA0ePjnNG7CeYksGwCOKu5B6dJLL9UDDzygl19+Wdu2bdP8+fP18MMPa9q0afFeFYAUdMKIz2nytTdocMWIlusCSDJGen9XWI+/3qy99aGjhiVjjKrrw/rN62Ft2BU+dGUAlQ47QRd+9UadPGpUD34KAGgR91Nvv/rVrzR79mx95zvf0d69e1VaWqpvfvObuueee+K9KgApKDMzU+MnXajjR5ykJX/+P21c9baMiSpkW1q826fLqjdpkOWV0gdKnqyWN0UDMsF9qtoT1dLdn1OO3STLslQ6bITOu3qGTh41OrkfCkC/FfeglJOTo7lz52ru3LnxXjSAXsLtdqusvFxXfONmrR5xklYvfFmRmkY1+XJlR5tlGnfLNG7/zLuMTLRQwbQspcdCKj3hpE+OJBGSACQP93oDkDDZOTkaO+kLqnRlao9/gayGgLZllGmUDsiltqffbEnbMo5XKCtH1ceXybrgEpVVDE9O4QDwibiPUQKAQzYdOKibVm7QQ9FsBdOy1FQwUD8b+DX5rawj5q13ZevhwulqzstXQ2au7g9laOaK97TlABehBZA8BCUAcWfbtt7eVaMvr96iZw4GtS8UUXrdfoWzc7Uqe5QezrlWocMOaIfk0c9zrtWa7JEKZ+cq/cB+1QbD+r8DQV2+eqOWV9Xwy1kASUFQAhB37+w/oJs27tT7DQHZliUjI3coqFhamiLuNP0h6wt62zdKRi33vX3LN1pPZ16kiDtN0bR0eZubZCTZlqUN/mbdsqFK67ZxMVoAPY+gBCCuDtTX6zd76vVuXZOMLCkWU/aH76l5X620/WN5/HXa6S7SE9mXqtY1QHtd+Xoi+1Ltcg9U5oFaZdfukTfUrKIN6+QKh2RktLa+QfN21qqOe0EC6GEEJQBxVdkU1F+27tGhG45kbflQRa/9TccXDVRJ7W4NWvSSbCPNzzhff8m8QH/OnKQFGefL6/drxD9fkre5SU0FA3XcO8s1+N3VsmIxRST9qS6ktXX8/gRAzyIoAYibWCymV2v8qo9+Op4o7cA+pYVCmvG16zX21FOVvnunZIzCVpp+mvNV/SznKwpbXg3c/IEkafuESdr4xS+r/vihyt/+sdICDZKkumhMrxzYzVglAD2KP88AxI1t21pWW99mWrhwkEK+dD351O9V729Q8PghrVfs3uEpaZ0v7+A+2bl58g8uk+316uCwz6nojYXyhsM6dIvdZbV1sm37iNskAUCiEJQAxI1t29rS2NRmWmD4ido75V/U/ME6RUqHqW7cOZJ1ZNC5/d++LtsYfXP7AUnSkJNP1iNfnKj/b/sBrWuOSJI2Nza3e684AIg3ghKAuPF4PLrd1CtsDjs9Zkk6qazl0apOn7nepLKrGyVJ96olFGUHLVXvqNP0YERf/iQc+eTmaBKAHmWZFPvzzO/3Ky8vT/X19crNzU12OQAAoI/pTNbgTzMAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHXB4AvUokElE0Gj32jO3weDzyer1xqgi9Cf2D7qB/+ieCEnqFxsZGPf/8C1qxZJkC+w/KdPE2FpbLpczCfI0/b4KuvHKasrOz41wpUhH9g+6gf/o3rqOElNfY2Kh777lPFVWNmlo+TPlpGYfugNFpxhgdDAf1yo4t2n5cnu659/tsrPo4+gfdQf/0TVxHCX3K88+/oIqqRl03YpQKfJlqjITVFI0qaMdku10KRCOKuSw1x2Lyh0PtPhojERX4MnX9iNEq312nBfNfTPbHQ4LRP+gO+gecekNKi0QiWrFkmWaVD5ct6bXmWmUOLVZeTpa8Ho+K8vO1o7pGJQMLdMDfoPqGQLvLi8aiatpWowszivTFsuH6xevLdM21VzFmoI+if9Ad9A8kghJSXDQaVWB/nfIHZ6ghHFLm0GKdd/bpsg479j2wuFCSVHpcidNiWhljtNSsUePuoPJ9GQrsP6hYLMaGqo+if9Ad9A8kTr2hFzB2TJYlGRm53e42G6nOsixLbo9bxkiWLBk7FsdKkYroH3QH/QOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEvodl2Wp65eMQ39H/6A76J/eh6CEfmf0CcOUk5WZ7DLQS9E/6A76p/chKKHfyc7MkMftTnYZ6KXoH3QH/dP7EJQAAAAcEJTQ7+yqqVVzKJTsMtBL0T/oDvqn9yEoIeVZLreMMbJkKRqLyhjT5WUZY/TxrioFw2EZGVkuDoH3dfQPuoP+gSfZBQDt8Xg8yiwcoIPhoAp8mWraVqOlZo3cnq5tYGLRmJq2VSs7s1j7ggFlFebLzXiBPov+QXfQP5AISkhxXq9XEyZN1CsvL9P1I0brwowiNe4Oqqt/1FmWlJ1ZLJekV3Zs0YRLz5fX641rzUgd9A+6g/6BRFBCLzBt2uW6b937+v2md3Vx2XDl+zLU1SuRGBntCwb0yo4tqhpSoBunXRbnapFq6B90B/0Dy3TnhGsC+P1+5eXlqb6+Xrm5uckuBykiEAhowQt/1VuL3lBg/0EZO9al5Vgut7IK8zVh8rmaNu0yZWVlxblSpCL6B91B//Q9nckaBCX0KpFIRLFY1zZSh7jdbg5391P0D7qD/uk7OpM1OPWGXsXr9bKRQZfRP+gO+qd/4vIAAAAADghKAAAADghKAAAADghKAAAADghKAAAADghKAAAADghKAAAADghKAAAADghKAAAADghKAAAADghKAAAADghKAAAADghKAAAADghKAAAADghKAAAADghKAAAADghKAAAADghKAAAADhISlHbv3q3rr79ehYWFysjI0OjRo7V69epErAoAACBhPPFe4MGDBzVx4kRNnjxZf//73zVo0CBt2rRJ+fn58V4VAABAQsU9KD300EMqKyvTk08+2TqtoqIi3qsBAABIuLifenvxxRc1btw4XXXVVSoqKtJpp52mJ554wnH+UCgkv9/f5gEAAJAK4h6UPv74Y82bN08jRozQq6++qm9/+9u65ZZb9NRTTx11/jlz5igvL6/1UVZWFu+SAAAAusQyxph4LjAtLU3jxo3TW2+91Trtlltu0apVq/T2228fMX8oFFIoFGp97vf7VVZWpvr6euXm5sazNAAAAPn9fuXl5XUoa8T9iNLgwYN1yimntJl28skna8eOHUed3+fzKTc3t80DAAAgFcQ9KE2cOFEbN25sM+2jjz7SkCFD4r0qAACAhIp7ULr99tu1fPlyPfjgg9q8ebOefvppPf7445o5c2a8VwUAAJBQcQ9KZ555pubPn69nnnlGo0aN0n333ae5c+fquuuui/eqAAAAEirug7m7qzMDrAAAADorqYO5AQAA+gqCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgANPsgsA0D3BYEiv/3OlHvv1H7X8rXUKhyLdWl6aL03nnDtW37zpak36/JnypafFqVKkIvoHaJ9ljDHJLuJwfr9feXl5qq+vV25ubrLLAVKabRst/Psyfe+2n6uopFDnnDtWmVkZ3VpmU6BZy5ZWal9tnX76yH9oytQJsiwrThUjldA/6K86kzU4ogT0YrFoVE/M+7OKigv044dv16ljT5TX273/1pFwRGvf2ai7Zz2s/573F02+cLw83VwmUhP9AxwbY5SAXqSmep/Wv7tJtt1yIDhmG61c/p4mnn9aXHZykuRN82rsaSfqnPNO04q317Wuy7ZtrX93k2qq93d7HUgO+gfoPIIS0Iv8+dl/aMa1/6loNPrJFKNwKKKMzIw2OznTFJS9aKViz74qe9FKmcamI5bV3jzeNK8yM9MVCkZk1LKji0Si+to1d+n5Py1M6GdE4tA/QOdxPBToRRr8AVXt2qv2hhaapqDs3y6Q3lgr0xSUMtNlNnws179dISszvcPzHLFcY1S1q1b++kACPhl6Av0DdB5BCehlbNvW8mXr5PF6FAlHZNt2m9fN2++27MACzbKkln/fXCcz6gRZk8d1eJ6OrAu9D/0DdA5BCehlIpGopl18i+PrpvagTFNQh35nZKnlCIC1r65T83RkXeh96B+gcwhKQC/j9Xr0l5fnfvJXelT/esmtbV63BuW3nAo59Ne+1HI6ZOCATs3TkXWh96F/gM4hKAG9jMvl0plnj5bPl6ZgMCSXq+1vMqyzRsls+LjlVEhTs6zMDOm8sbLGj+zUPB1ZF3of+gfoHIIS0Ivk5mXr+LLidi/gZ2VnyvVvV7SMF9lXJw0cIGv8SFmHXUiwI/McsVzL0nFlxcrLy47jJ0JPon+AziMoAb3IlddO1eQpZ8njOfRf11JauldNgWZFwhF507wtUzPT2wyqPZr25gmHIwoEmpWe4ZP1yUgUr9ej//3TjzVoUH7cPg96Fv0DdB7HQYFepKi4QCePHCaXq2Xn43a7dPaEMXpzaaXWvvOhIuHu3adLatnJvbPmAy1b+o7OPufU1nW5XC6dMnK4BhUVdHsdSA76B+g87vUG9GK2bfTPV9/S9279uQYOGqAJ556mrKyjX8emowKBZi1b+o4OHKjXzx65QxdedDb36uqj6B/0V53JGgkPSj/+8Y91991369Zbb9XcuXOPOT9BCeicUDCspa+v1n/P+4uWv71O4WD3jgr40r2aMHGs/u1b/6rzJp0hn4+7v/dl9A/6o5QJSqtWrdLVV1+t3NxcTZ48maAEJIgxRrFoTLZtWm8ZcYi956BiO/fJVZAt95Aiyetud1mWLLlcltweN0cC+gn6B/1NZ7JGwgZzNzY26rrrrtMTTzyh+++/P1GrAaCWXxR99g7tsV37FHxptUwoIntvnVyjhyi2crM8pxyvtLNOTFKlSEX0D+AsYYO5Z86cqUsuuURTpkxpd75QKCS/39/mAaD7Yrv3K/DT+Qo+vVSRNVtk9jfINThfjT94Ws3PvSETjLR7zy/0b/QP0CIhR5SeffZZVVZWatWqVcecd86cObr33nsTUQaAaEx29UGp+qACG3dLPq9yfvw1+Tfv0aYbfqnAr7+pETkZGpjGlUJwFPQPEP8xSjt37tS4ceO0cOFCnXrqqZKkSZMmaezYsUcdoxQKhRQKhVqf+/1+lZWVMUYJ6KbYrv0K/mmZmv/4puyqAy3TLEt/vGC43vv8WFlnjlK+16Mst0tfHzxA5elpYkgJDqF/0JcldTD3ggULNG3aNLndnw74i8VisixLLpdLoVCozWufxWBuoGuMMTJGkjFyuV2HJiry/g41/vAZNa/dqt9eeLJ+dv5I+fMLlJ+Vqe8OGagzczL0v9V1+s5xBTorJ12yLFmWGIjbz9A/6E+SGpQaGhq0ffv2NtNuuOEGnXTSSbrzzjs1atSodt9PUAK6xraN3nh9tRoamvQvl1/QOt0YI0VjemrdBj0ZkN6IuGQ+2Ym5JJ2QmaafnlCi3+zar5nbt2nAgByde8Hp7Oj6GfoH/UlnskbcB3Pn5ORo1KhRbR5ZWVkqLCw8ZkgC0HXRaFRzfvSE3n9vc5vplmXJ8nr09XGnauF5o3Rj6ae3kLAlfdQU1u/31Kks3av3392kOT96QtForIerR7LRP8DRcQsToI+wbVvvrdvU7jxey9LPP1eiN86o0EMnFCvLZem0bJ+uLcnTPRVFkqT31n4kY/Nrpv6G/gGOrkd+qvD666/3xGqAfi92jL/kLctSnsetcwdkakJehm4tK5AlS97DxpREo7EjLjqI/oH+AY7EbzqBXqapKahlSyr10Ufb1egPtF7LJhqNKRaL6c0la2Rsu0vLfnNppaLRmH724JNyfzKg17IsZedm6XMnDtG555+ujMzu3QsMyUX/AJ3DTXGBXsIYo61bdunH9/233npjrfz+RkUiUSX8j3dL8no9ysvL1sQLztCd379RQ4cdx2DdXob+AT6VErcwARBf27dV6T9u+ak2bdyub3z7Sl1+5edVUjJQliuxOxxjG+3ZU6sFf16k3z72vPbVHtTDv/6eyocMTuh6EV/0D9A1BCWgl1jwl9f0/nub9cMHb9I1131RLlfP/RajYtjxuvU/pmtQUYHumz1PLz6/WDfd/tUeWz+6j/4BuoZfvQG9xPw/LtSEiWP1hS9OaN3J2batv/xxoa771zu14M+LZH9mbMnO7Xv0Hzf/VHfc+nPt3FHd5rXOvtflcmnqlyZq/IRT9ZfnXk3sh0Xc0T9A13BECeglNn20XVOmnqP8/E/Pp0ciUf3w7ke1e1eN1r+7SV+67HylpX36989fFyzRH556SbKk4SeU6Vs3X92t9+YX5OmEEeV6beHynvnQiBv6B+gaghLQS+TkZKmxsUnhSFQZnpbbALlcLp159mj5X23U+AmjjzidMnxEmQYfVySXZWn4Cce3ea0r7w2HI2psDCgnNyuBnxSJQP8AXcOv3oBe4uvX/qe2b6vS47+/VyeMKJdlWTLGqKZ6v6p21+q44wepqLiwza+Jgs0hfbxllyzLUsWw45Se4Wt9rbPv9aWn6aON2/XNGT/Q8BPK9Ns/3N+jnx/dQ/8An0rqvd66i6AEHN2bSyr1za//UOPOGqnv3/tNlZWVKD3Dl/CfWRtjFGwOaceOat0/e57eWfOhnvj9vZpw7tiErhfxRf8AnyIoAX1QU1NQz/7f3/TzOb9TUXGBzjxrlAYU5MqV4B2dbYwOHvBr1fL3tK+2Tv/xnzfomuu+qIwMLhzYm9A/wKcISkAf1dwU1DtrPtATv/mT3lq2Vvtq6xJ+Xy2Xy9LAQfk659yx+sZ3rtLY00/i6sq9FP0DtCAoAQAAOODK3ACOEA6HtXv7drksS6VDhsjr9Sa7JPQi9A/6K4IS0MfFYjHV7dwlLVulIRu3ylTV6MBJFdIVX1RBxVC53e5kl4gURv+gvyMoAX2UMUYH9taq+vU3NWLzTnkaGiU7JquuXgV/XaTwinVqmHKO7Cu+pPyiQdykFG3QP0ALxigBfVBDXb0OrFyjAes+UE5Ts6ymoIykgFwyG7coa/9+WZKMx6P68uPkv3yKBk25QFn8n4PoH/R9DOYG+qnmpib5129U5spKZcmSNf40aWChmuf/XetzC7V4zDjFwhGd/49XNGblCmUGGiXLkklLU+DkE9T85YuUN36cMrK4cnJ/RP+gvyAoAf1MKBRS1Qcfylq2SuXBqKzTRklnnCo7M1O79jfob7v9+mBgiSKfDMB1RyIa/uEHuvDlF3XChg3yhYKyJNkZ6Yqcdor2XvUvKhk9Uj6fr/0Vo0+gf9DfEJSAfiIWi2nf9h3SslUq3LJD7tEnSeedLVOYrwP1TXq90dbbVoaaXG7ps2NIjJE3HNaoytW68OW/qmzrx/JEozKWJTs7U/snnSXXtEtUOIwBu30V/YP+iqAE9AP11dUyK95V5rr18gSaZPnSpG9+TY25uXqrLqwlJl11bq/MsQbZGqPMxkaNe+tNTfr7yyraU9WyU7QsRQvzFbxkivSlC5U3uKhnPhh6BP2D/oygBPRxu9a8o8EvL5LVHJJlGxlJ/swsrRl/tt48ZYz2etJkW65jLudwlm2roLZWExf9Q+PfWKoBBz4ZsOt2y5QPVc2t03XcGWMT8XHQw+gf9HdccBLo4/a8+ppK39kgFQ9SY/4ArT/hRL1+2njtKiqR7XLYwRmjdL9fFWtWamjlKqX7/WoYWKQtZ5+jXaPHKJKRof3FxfrrNV9V5Tnn6vxX/6Yxq1Yqq6FB1vatqlq4mB1dH0H/AB3HESWgF9q1fJU8j/2vdg88Tksu/pK2jDhRtvso40gOMUYD9lTpgid+o/J31shl260vRb1p+uDCL+it67+uYG5e6/yuWEzDN36gC/7+skr37VH0O19T2dln9sCnQ6LRP+jvOPUG9HHGGPn9fv1wW6OCA0ucd3AtMyuveo++8Mufq3TDe/JalnLcHnktl0J2TA2xqKIul1Zf9RW9/ZXp0uFHFIxRRm2VfjgsTzk5OVxUsI+gf9DfdSZrdO4kNICUYFmW0tLSZLusY+7kcmuqNeVXD6v0g/VKsyyVeH0a6EnTAI9Xg7w+DfT65LZtfW7pYg2o3vPZFSnqcistLY2dXB9C/wAdR1AC+ihjjJoPHtS4/3pUx73/nnyyVOJNV4bL3brTcn1ydMBjWcreV6ucvTVJrhqpgv4BWhCUgD7IGCN/Xb0WLlqsecaj2owsFXl9Sne5jvjL3pJkyZLLtuWyY8kpGCmF/gE+RVAC+hhjjPz19Vq8cJGqdldpcWGRHjx9omqzc444zWKMUdC2FTW2mnNy1TQgP0lVI1XQP0BbBCWgDzHGqLGhQUv++Zqqq6okt1vKGaB/lJbrO6eOV1V6hsxh84aNrdpISDFJVaNO1YHjy9ssKxYOK9rUmJTPgp5H/wBHIigBfYht21r51tuq2rW7dScnT8vVlf9RNFjfPnW8dmRkSpJCxlZ1OKSgsbWvYpiWX3u9Ymlpkj7ZyQWb1VRTpdDB/Un8ROhJ9A9wJIIS0IfU7t2rHdt2tJwiyc6TPN7W0yW2ZelvxaX69pjx2pieob2H7eT+efMsHSgrb7OsUP1B2ZFwMj4GkoT+AY5EUAL6kIP7DygcCrXs4LxpR44pkfRqwSB9e/iJ2pKRqX0Vw/WPW+/Q3uEjjpiXnVz/Q/8AR+IWJkAfYh+6YvLRrlljjBSLSg31Wpqbr59NnqozLpqi/UOHOcyf2FqReugf4EgEJaAPyc3Lk8frVSQaadmpuT0tO7HDdnKKRpQ/cKCyvnCh9hUN4kKAaEX/AEfi1BvQh5QMHqziwSWSbbfs1MJBKRKWQs2Sv06KRlQ4cKAmfeFCDWQnh8+gf4AjcUQJ6EM8Xo/GnzNBzU1N2l+7r2Xn9skRAcuyNKikWOd/frIKBw1kJ4cj0D/AkQhKQB9iWZYGFQ3SF750sTa8+5527tihYHOzMrOyNKRiqE4ePUrZ2dnt7uSMMTK2LcMgk36H/gGORFACeinLsmTpyB2WZVnKG5Cns8+dqHGxs2SMkcuy5PZ4Wt7zyU7OGFuxYFCxcFifHXkbC4Vkoi23o3C5LI4e9EH0D9AxBCWgl3K73SrL8GjzUV6zLEuW21KaO+2o7zXGVujgAYX8dS3jUdoxJCdbLhfDGfsa+gfoGLoX6KW8Xq8m236VRJvlNqbll0kdFAuFFfbXO+/kTMvGodQjTR3gkdfrjU/RSBn0D9AxHFECerHTPzdcg6prtGHbBgVDoQ6/zxgjE4u2O096mk+nlA3VkONKu1smUhT9AxwbQQno5cpKilVWUpzsMtBL0T9A+whKQB8Ti8UUDocViUQUiUQUi8Vk27aMMa1XXna5XLIsSy6XS263W16vV16vV2lpaXK73Un+BEgm+gdoi6AE9AFNTU2qqanR/v371dTUpGg0Ktu2W3dy0ienSz4Zh3L4r5cO7excLpc8Ho8yMzNVWFiokpISZWRkJO0zoefQP4AzghLQB9TW1urjjz9WOBxu/eu/PYe/HovFFIlEJLXsAAOBgOrr6+V2u1VeXu60CPQh9A/gjKAE9AHl5eUaOHCg/H6/Ghsb1dzcrGAwqGAwqEgkcsTRgcOPAni9XqWnpys9PV0ZGRnKzs5Wbm6uMjMzk/yp0FPoH8AZQQnoAyzLUlZWlrKyspJdCnoh+gdwxnWUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHMQ9KM2ZM0dnnnmmcnJyVFRUpCuuuEIbN26M92oAAAASLu5BacmSJZo5c6aWL1+uhQsXKhKJ6KKLLlIgEIj3qgAAABLKMsaYRK6gtrZWRUVFWrJkic4///wjXg+FQgqFQq3P/X6/ysrKVF9fr9zc3ESWBgAA+iG/36+8vLwOZY2Ej1Gqr6+XJBUUFBz19Tlz5igvL6/1UVZWluiSAAAAOiShR5Rs29Zll12muro6vfnmm0edhyNKAACgJ3XmiJInkYXMnDlT69evdwxJkuTz+eTz+RJZBgAAQJckLCjddNNNeumll7R06VIdf/zxiVoNAABAwsQ9KBljdPPNN2v+/Pl6/fXXVVFREe9VAAAA9Ii4B6WZM2fq6aef1gsvvKCcnBxVV1dLkvLy8pSRkRHv1QEAACRM3AdzW5Z11OlPPvmkvv71rx/z/Z0ZYAUAANBZSR3MneDLMgEAAPQY7vUGAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADgIGFB6dFHH9XQoUOVnp6us846SytXrkzUqgAAABIiIUHpueee06xZs/SDH/xAlZWVGjNmjKZOnaq9e/cmYnUAAAAJkZCg9PDDD+sb3/iGbrjhBp1yyil67LHHlJmZqf/5n/9JxOoAAAASwhPvBYbDYa1Zs0Z333136zSXy6UpU6bo7bffPmL+UCikUCjU+ry+vl6S5Pf7410aAABAa8Ywxhxz3rgHpX379ikWi6m4uLjN9OLiYn344YdHzD9nzhzde++9R0wvKyuLd2kAAACtGhoalJeX1+48cQ9KnXX33Xdr1qxZrc/r6uo0ZMgQ7dix45jF90d+v19lZWXauXOncnNzk11OyuH7aR/fT/v4ftrH99M+vh9nqfbdGGPU0NCg0tLSY84b96A0cOBAud1u1dTUtJleU1OjkpKSI+b3+Xzy+XxHTM/Ly0uJLzNV5ebm8v20g++nfXw/7eP7aR/fT/v4fpyl0nfT0YMxcR/MnZaWpjPOOEOLFi1qnWbbthYtWqQJEybEe3UAAAAJk5BTb7NmzdKMGTM0btw4jR8/XnPnzlUgENANN9yQiNUBAAAkREKC0jXXXKPa2lrdc889qq6u1tixY/XKK68cMcD7aHw+n37wgx8c9XQc+H6Ohe+nfXw/7eP7aR/fT/v4fpz15u/GMh35bRwAAEA/xL3eAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHKRcUHr00Uc1dOhQpaen66yzztLKlSuTXVJKmDNnjs4880zl5OSoqKhIV1xxhTZu3JjsslLSj3/8Y1mWpdtuuy3ZpaSU3bt36/rrr1dhYaEyMjI0evRorV69OtllJV0sFtPs2bNVUVGhjIwMDR8+XPfdd1+HbpbZFy1dulSXXnqpSktLZVmWFixY0OZ1Y4zuueceDR48WBkZGZoyZYo2bdqUnGKToL3vJxKJ6M4779To0aOVlZWl0tJSfe1rX1NVVVXyCu5hx+qfw33rW9+SZVmaO3duj9XXFSkVlJ577jnNmjVLP/jBD1RZWakxY8Zo6tSp2rt3b7JLS7olS5Zo5syZWr58uRYuXKhIJKKLLrpIgUAg2aWllFWrVum//uu/dOqppya7lJRy8OBBTZw4UV6vV3//+9+1YcMG/fznP1d+fn6yS0u6hx56SPPmzdOvf/1rffDBB3rooYf0k5/8RL/61a+SXVpSBAIBjRkzRo8++uhRX//JT36iRx55RI899phWrFihrKwsTZ06VcFgsIcrTY72vp+mpiZVVlZq9uzZqqys1PPPP6+NGzfqsssuS0KlyXGs/jlk/vz5Wr58eYfutZZ0JoWMHz/ezJw5s/V5LBYzpaWlZs6cOUmsKjXt3bvXSDJLlixJdikpo6GhwYwYMcIsXLjQXHDBBebWW29Ndkkp48477zTnnntusstISZdccom58cYb20z78pe/bK677rokVZQ6JJn58+e3Prdt25SUlJif/vSnrdPq6uqMz+czzzzzTBIqTK7Pfj9Hs3LlSiPJbN++vWeKSiFO38+uXbvMcccdZ9avX2+GDBlifvGLX/R4bZ2RMkeUwuGw1qxZoylTprROc7lcmjJlit5+++0kVpaa6uvrJUkFBQVJriR1zJw5U5dcckmbHkKLF198UePGjdNVV12loqIinXbaaXriiSeSXVZKOOecc7Ro0SJ99NFHkqR169bpzTff1MUXX5zkylLP1q1bVV1d3eb/WF5ens466yy20w7q6+tlWZYGDBiQ7FJSgm3bmj59uu644w6NHDky2eV0SEJuYdIV+/btUywWO+I2J8XFxfrwww+TVFVqsm1bt912myZOnKhRo0Ylu5yU8Oyzz6qyslKrVq1Kdikp6eOPP9a8efM0a9Ys/ed//qdWrVqlW265RWlpaZoxY0ayy0uqu+66S36/XyeddJLcbrdisZgeeOABXXfddckuLeVUV1dL0lG304dew6eCwaDuvPNOfeUrX1Fubm6yy0kJDz30kDwej2655ZZkl9JhKROU0HEzZ87U+vXr9eabbya7lJSwc+dO3XrrrVq4cKHS09OTXU5Ksm1b48aN04MPPihJOu2007R+/Xo99thj/T4o/fGPf9Qf/vAHPf300xo5cqTWrl2r2267TaWlpf3+u0HXRSIRXX311TLGaN68eckuJyWsWbNGv/zlL1VZWSnLspJdToelzKm3gQMHyu12q6amps30mpoalZSUJKmq1HPTTTfppZde0uLFi3X88ccnu5yUsGbNGu3du1enn366PB6PPB6PlixZokceeUQej0exWCzZJSbd4MGDdcopp7SZdvLJJ2vHjh1Jqih13HHHHbrrrrt07bXXavTo0Zo+fbpuv/12zZkzJ9mlpZxD22K20+07FJK2b9+uhQsXcjTpE2+88Yb27t2r8vLy1m319u3b9d3vfldDhw5NdnmOUiYopaWl6YwzztCiRYtap9m2rUWLFmnChAlJrCw1GGN00003af78+XrttddUUVGR7JJSxoUXXqj33ntPa9eubX2MGzdO1113ndauXSu3253sEpNu4sSJR1xO4qOPPtKQIUOSVFHqaGpqksvVdlPodrtl23aSKkpdFRUVKikpabOd9vv9WrFiBdvpTxwKSZs2bdI///lPFRYWJruklDF9+nS9++67bbbVpaWluuOOO/Tqq68muzxHKXXqbdasWZoxY4bGjRun8ePHa+7cuQoEArrhhhuSXVrSzZw5U08//bReeOEF5eTktI4HyMvLU0ZGRpKrS66cnJwjxmplZWWpsLCQMVyfuP3223XOOefowQcf1NVXX62VK1fq8ccf1+OPP57s0pLu0ksv1QMPPKDy8nKNHDlS77zzjh5++GHdeOONyS4tKRobG7V58+bW51u3btXatWtVUFCg8vJy3Xbbbbr//vs1YsQIVVRUaPbs2SotLdUVV1yRvKJ7UHvfz+DBg3XllVeqsrJSL730kmKxWOu2uqCgQGlpackqu8ccq38+Gxy9Xq9KSkp04okn9nSpHZfsn9191q9+9StTXl5u0tLSzPjx483y5cuTXVJKkHTUx5NPPpns0lISlwc40l//+lczatQo4/P5zEknnWQef/zxZJeUEvx+v7n11ltNeXm5SU9PN8OGDTPf//73TSgUSnZpSbF48eKjbmtmzJhhjGm5RMDs2bNNcXGx8fl85sILLzQbN25MbtE9qL3vZ+vWrY7b6sWLFye79B5xrP75rN5weQDLmH56+VkAAIBjSJkxSgAAAKmGoAQAAOCAoAQAAOCAoAQAAOCAoAQAAOCAoAQAAOCAoAQAAOCAoAQAAOCAoAQAAOCAoAQAAOCAoAQAAODg/wfmcbw/SwqIXQAAAABJRU5ErkJggg==",
241
+ "text/plain": [
242
+ "<Figure size 700x700 with 1 Axes>"
243
+ ]
244
+ },
245
+ "metadata": {},
246
+ "output_type": "display_data"
247
+ }
248
+ ],
249
+ "source": [
250
+ "env=RoverGridEnv()\n",
251
+ "print(\"Initial Setup\")\n",
252
+ "observation=env.reset()\n",
253
+ "env.render()"
254
+ ]
255
+ }
256
+ ],
257
+ "metadata": {
258
+ "kernelspec": {
259
+ "display_name": "Python 3",
260
+ "language": "python",
261
+ "name": "python3"
262
+ },
263
+ "language_info": {
264
+ "codemirror_mode": {
265
+ "name": "ipython",
266
+ "version": 3
267
+ },
268
+ "file_extension": ".py",
269
+ "mimetype": "text/x-python",
270
+ "name": "python",
271
+ "nbconvert_exporter": "python",
272
+ "pygments_lexer": "ipython3",
273
+ "version": "3.8.10"
274
+ }
275
+ },
276
+ "nbformat": 4,
277
+ "nbformat_minor": 2
278
+ }
rx_rover_PPO.ipynb ADDED
The diff for this file is too large to render. See raw diff
 
rx_rover_Q_Learning.ipynb ADDED
The diff for this file is too large to render. See raw diff