Upload 25 files
Browse files- .gitattributes +3 -0
- Enhanced Environemnt Simulation.gif +0 -0
- README.md +230 -3
- Report.pdf +3 -0
- images/all_algos.png +3 -0
- images/all_algos_testing.png +3 -0
- images/desk.png +0 -0
- images/enhanced-environment.png +0 -0
- images/human.png +0 -0
- images/room.png +0 -0
- images/rover_desk_collision.png +0 -0
- images/rover_dest.png +0 -0
- images/rover_human_collision.png +0 -0
- images/rover_moving.png +0 -0
- images/rover_room_collision - Copy.png +0 -0
- images/rover_rover_collision.png +0 -0
- images/rover_start.png +0 -0
- images/target.png +0 -0
- images/trained_agents.gif +0 -0
- rx_rover_A2C.ipynb +0 -0
- rx_rover_DDQN.ipynb +0 -0
- rx_rover_DQN.ipynb +0 -0
- rx_rover_Double_Q_Learning.ipynb +0 -0
- rx_rover_Enhanced_Environment.ipynb +278 -0
- rx_rover_PPO.ipynb +0 -0
- rx_rover_Q_Learning.ipynb +0 -0
.gitattributes
CHANGED
@@ -33,3 +33,6 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
36 |
+
images/all_algos_testing.png filter=lfs diff=lfs merge=lfs -text
|
37 |
+
images/all_algos.png filter=lfs diff=lfs merge=lfs -text
|
38 |
+
Report.pdf filter=lfs diff=lfs merge=lfs -text
|
Enhanced Environemnt Simulation.gif
ADDED
![]() |
README.md
CHANGED
@@ -1,3 +1,230 @@
|
|
1 |
-
|
2 |
-
|
3 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
# RxRovers: Roaming for Rapid Relief
|
2 |
+
|
3 |
+
<span style="color: blue;">Dynamic Obstacles and Path Optimization
|
4 |
+
|
5 |
+

|
6 |
+
|
7 |
+
## Project Overview
|
8 |
+
|
9 |
+
"RxRovers: Roaming for Rapid Relief" aims to integrate advanced reinforcement learning (RL) into the healthcare domain, specifically focusing on optimizing medical supply delivery within hospital settings. This project seeks to deploy autonomous agents, RxRovers, which are programmed to navigate through hospital corridors efficiently, dodging any potential obstacles (dynamic and static) to ensure the timely distribution of medicines. This innovation can significantly enhance patient care and outcomes.
|
10 |
+
|
11 |
+
## Team Members
|
12 |
+
|
13 |
+
- Charvi Kusuma [GitHub](https://github.com/kcharvi)
|
14 |
+
- Tarun Reddi [GitHub](https://github.com/REDDITARUN)
|
15 |
+
|
16 |
+
### Real-World Application
|
17 |
+
|
18 |
+
The RxRovers project directly addresses a real-world healthcare problem: simulating delivery of medical supplies efficiently within hospital environments. This project integrates RL into the healthcare domain to enhance this delivery system, directly benefiting patient care and outcomes.
|
19 |
+
|
20 |
+
### Complex Navigation Challenges
|
21 |
+
|
22 |
+
The project’s challenges mirror real-world navigation complexities in hospital settings:
|
23 |
+
|
24 |
+
- **Dynamic and Static Obstacles**: Accurate representation of unpredictable hospital environments.
|
25 |
+
- **Path Planning**: Optimization to ensure timely and efficient delivery of medical supplies.
|
26 |
+
|
27 |
+
### Adaptable Environment
|
28 |
+
|
29 |
+
The project's environment can be modified to reflect various real-world hospital layouts, demonstrating its adaptability to different healthcare settings.
|
30 |
+
|
31 |
+
## Objectives
|
32 |
+
|
33 |
+
1. Develop RL agents capable of autonomously navigating hospital environments while delivering medical supplies.
|
34 |
+
2. Optimize path planning strategies to ensure timely and efficient delivery of medicines, while avoiding obstacles such as equipment, humans, and environmental constraints.
|
35 |
+
3. Enhance the visual representation and user experience of the simulated hospital environment to improve engagement and realism.
|
36 |
+
4. Conduct comparative analysis of various reinforcement learning algorithms to identify the optimal approach for medical supply delivery optimization within hospital environments.
|
37 |
+
|
38 |
+
## Environment
|
39 |
+
|
40 |
+
### Simulated Hospital Layouts
|
41 |
+
|
42 |
+
The project creates a hospital environment that mirrors real-world scenarios. The environment includes features such as:
|
43 |
+
|
44 |
+
- **Hospital Corridors**: Multiple corridors, rooms, and operation desks.
|
45 |
+
- **Obstacles**: Dynamic obstacles (humans, other rovers) and static obstacles (rooms, walls).
|
46 |
+
- **RxRovers**: Autonomous agents programmed to navigate these environments, avoid obstacles, and reach designated destinations.
|
47 |
+
|
48 |
+
### Initial Stage
|
49 |
+
|
50 |
+
- **Grid**: 9×9 grid simulating rovers navigating on a grid.
|
51 |
+
- **Setup**: Initialized action and observation spaces, set up grid size, starting points, destinations, and parameters such as rewards and penalties for actions and events.
|
52 |
+
|
53 |
+
### Refined Version
|
54 |
+
|
55 |
+
- **Grid Size**: 15×15 grid representing the hospital environment.
|
56 |
+
- **Rovers**: Two agents represented by blue squares.
|
57 |
+
- **Targets**: Yellow target destinations for delivering medicine.
|
58 |
+
- **Actions**: Move down, up, left, right, or stay still (5 possible actions).
|
59 |
+
- **Operation Desks**: Dark green squares representing starting points.
|
60 |
+
- **Rooms**: Black squares indicating static obstacles.
|
61 |
+
- **Human**: Purple square representing a moving human obstacle.
|
62 |
+
- **Observation Space**: Positions of the Rovers, human, rooms, operation desks, and the grid boundary.
|
63 |
+
- **Rewards**:
|
64 |
+
- +30 for moving closer to targets.
|
65 |
+
- +100 for reaching destinations.
|
66 |
+
- -15 for collisions or moving out of grid bounds.
|
67 |
+
- -5 for waiting near the human obstacle.
|
68 |
+
- -20 for moving away from targets.
|
69 |
+
- **Termination**: Episode ends if both Rovers reach their targets or maximum time steps (20) are reached.
|
70 |
+
|
71 |
+

|
72 |
+
|
73 |
+
## Algorithms
|
74 |
+
|
75 |
+
### Q-Learning (QL)
|
76 |
+
|
77 |
+
Implemented Q-Learning for a rover agent navigating a grid environment.
|
78 |
+
|
79 |
+
- **Settings**:
|
80 |
+
- State Representation: Positions of the two rovers and the human.
|
81 |
+
- Action Space: Move up, down, left, right, or stay in place.
|
82 |
+
- Rewards: Based on interactions (moving closer to target, collisions, reaching target).
|
83 |
+
- Done Flag: Episode terminates when both rovers reach targets or max time steps are reached.
|
84 |
+
- **Hyperparameters**:
|
85 |
+
- Alpha (Learning Rate): 1e-4
|
86 |
+
- Gamma (Discount Factor): 0.9
|
87 |
+
- Epsilon (Exploration Rate): 0.5
|
88 |
+
- Epsilon Decay: 0.995
|
89 |
+
- Epsilon Minimum: 0.01
|
90 |
+
- **Training Phase**:
|
91 |
+
- Total Rewards per Episode
|
92 |
+
- Epsilon Decay Curve
|
93 |
+
- **Evaluation Phase**:
|
94 |
+
- Evaluation Rewards per Episode
|
95 |
+
|
96 |
+
### Double Q-Learning (DQL)
|
97 |
+
|
98 |
+
Implemented Double Q-Learning to mitigate overestimation biases.
|
99 |
+
|
100 |
+
- **Settings**: Same as Q-Learning.
|
101 |
+
- **Hyperparameters**: Same as Q-Learning.
|
102 |
+
- **Training Phase**:
|
103 |
+
- Total Rewards per Episode
|
104 |
+
- **Evaluation Phase**:
|
105 |
+
- Evaluation Rewards
|
106 |
+
|
107 |
+
### Deep Q Network (DQN)
|
108 |
+
|
109 |
+
Utilized a neural network to approximate the Q-values.
|
110 |
+
|
111 |
+
- **Settings**:
|
112 |
+
- Neural Network Architecture: Two hidden layers with ReLU activation.
|
113 |
+
- Replay Memory: Stores past experiences for experience replay.
|
114 |
+
- Select Action Function: Epsilon-greedy strategy.
|
115 |
+
- Optimize Model Function: Gradient descent on the Q-network.
|
116 |
+
- **Hyperparameters**:
|
117 |
+
- Number of Episodes: 1000
|
118 |
+
- Target Network Update Frequency: 10 episodes
|
119 |
+
- Batch Size: 256
|
120 |
+
- Discount Factor: 0.9
|
121 |
+
- Learning Rate: 0.001
|
122 |
+
- Epsilon Initial Value: 1
|
123 |
+
- Epsilon Final Value: 0.05
|
124 |
+
- Epsilon Decay: 10000
|
125 |
+
- Maximum Timestamps per Episode: 30
|
126 |
+
- **Training Phase**:
|
127 |
+
- Total Rewards per Episode
|
128 |
+
- Epsilon Decay Curve
|
129 |
+
- **Evaluation Phase**:
|
130 |
+
- Evaluation Rewards
|
131 |
+
|
132 |
+
### Double Deep Q Network (DDQN)
|
133 |
+
|
134 |
+
Addresses overestimation bias by decoupling selection and evaluation of the action.
|
135 |
+
|
136 |
+
- **Settings**:
|
137 |
+
- Optimize Model Function: Gradient descent with gradients clipping.
|
138 |
+
- **Hyperparameters**:
|
139 |
+
- Batch Size: 256
|
140 |
+
- Gamma: 0.9
|
141 |
+
- Learning Rate: 0.001
|
142 |
+
- Epsilon Start: 1
|
143 |
+
- Epsilon End: 0.05
|
144 |
+
- Epsilon Decay: 10,000
|
145 |
+
- Number of Episodes: 1000
|
146 |
+
- Target Update: Every 10 episodes
|
147 |
+
- **Training Phase**:
|
148 |
+
- Total Rewards per Episode
|
149 |
+
- Epsilon Decay Curve
|
150 |
+
- **Evaluation Phase**:
|
151 |
+
- Evaluation Rewards
|
152 |
+
|
153 |
+
### Proximal Policy Optimization (PPO)
|
154 |
+
|
155 |
+
A robust and efficient algorithm developed by OpenAI.
|
156 |
+
|
157 |
+
- **Settings**:
|
158 |
+
- Neural Network Architecture: Actor and critic heads.
|
159 |
+
- Training Loop: States, actions, rewards, values, and log-probs collected in replay buffer.
|
160 |
+
- Loss Functions: Policy and value losses.
|
161 |
+
- **Hyperparameters**:
|
162 |
+
- Total Timesteps: 50,000
|
163 |
+
- Gamma: 0.99
|
164 |
+
- Lambda: 0.95
|
165 |
+
- Epsilon: 0.2
|
166 |
+
- Epochs: 3
|
167 |
+
- Batch Size: 64
|
168 |
+
- Learning Rate: 0.001
|
169 |
+
- **Training Phase**:
|
170 |
+
- Total Rewards per Episode
|
171 |
+
- **Evaluation Phase**:
|
172 |
+
- Evaluation Rewards
|
173 |
+
|
174 |
+
### Actor Critic (A2C)
|
175 |
+
|
176 |
+
Combines elements of both policy-based methods (Actor) and value-based methods (Critic).
|
177 |
+
|
178 |
+
- **Settings**:
|
179 |
+
- Actor Network: Learns a policy π(s).
|
180 |
+
- Critic Network: Learns the value function V(s).
|
181 |
+
- Advantage: Measures action quality.
|
182 |
+
- Policy and Value Function Updates: Gradient descent.
|
183 |
+
- **Training Phase**:
|
184 |
+
- Total Rewards per Episode
|
185 |
+
- **Evaluation Phase**:
|
186 |
+
- Evaluation Rewards
|
187 |
+
|
188 |
+
**Total Rewards Plot during Training:**
|
189 |
+
|
190 |
+

|
191 |
+
|
192 |
+
**Evaluation for 10 Timestamps:**
|
193 |
+
|
194 |
+

|
195 |
+
|
196 |
+
## Comparison
|
197 |
+
|
198 |
+
Comparison of different reinforcement learning algorithms and their performance:
|
199 |
+
|
200 |
+
| Aspect | Q-Learning | Double Q-Learning | DQN | DDQN | PPO | A2C |
|
201 |
+
| ------------------------- | ----------------------- | ----------------------- | -------------------- | ------------------------------ | ----------------------------- | ----------------------------- |
|
202 |
+
| Algorithm Type | Model-free, Value-based | Model-free, Value-based | Value-based, Deep NN | Value-based, Deep NN | Actor-Critic, Policy Gradient | Actor-Critic, Policy Gradient |
|
203 |
+
| Exploration-Exploitation | Epsilon-greedy | Epsilon-greedy | Epsilon-greedy | Epsilon-greedy | Continuous | Continuous |
|
204 |
+
| Stability & Learning Rate | No target network | No target network | Target network | Double updates, target network | Adaptive, Gradient Clipping | Adaptive, Entropy Bonus |
|
205 |
+
| Convergence | Slow | Moderate | Fast | Faster | Fast | Moderate |
|
206 |
+
| Memory Requirements | Low | Low | High | High | Moderate | Moderate |
|
207 |
+
| Adaptability | Moderate | Moderate | Low to Moderate | Low to Moderate | High | Moderate |
|
208 |
+
| Generalization | Low to Moderate | Low to Moderate | Low | Low | High | Moderate |
|
209 |
+
|
210 |
+
## Challenges Addressed
|
211 |
+
|
212 |
+
1. **Navigating a Dynamic World**: Rovers learned to dodge moving obstacles using dynamic obstacle handling techniques.
|
213 |
+
2. **Custom Paths for Diverse Layouts**: Environment adaptable to various hospital settings.
|
214 |
+
3. **Learning Optimal Path**: Rovers trained to take the shortest route to delivery rooms, avoiding obstacles.
|
215 |
+
|
216 |
+
## Bonus: Real-World Application
|
217 |
+
|
218 |
+
1. **Healthcare System Integration**: Enhances delivery system within hospital settings, benefiting patient care and outcomes.
|
219 |
+
2. **Simulating Hospital Layouts**: Reflects real-world scenarios with hospital corridors, obstacles, and RxRovers.
|
220 |
+
3. **Complex Navigation Challenges**: Optimizes path planning and avoids dynamic/static obstacles.
|
221 |
+
4. **Adaptable Environment**: Can be modified to different healthcare settings.
|
222 |
+
|
223 |
+
## References
|
224 |
+
|
225 |
+
1. [Deep Reinforcement Learning DQN for Multi-Agent Environment](https://medium.com/yellowme/deep-reinforcement-learning-dqn-for-multi-agent-environment-5f4fae1a9ff5)
|
226 |
+
2. [Warehouse Robot Path Planning](https://github.com/LyapunovJingci/Warehouse_Robot_Path_Planning) for path planning and optimization with obstacle avoidance.
|
227 |
+
3. [Reinforcement Learning (DQN) Tutorial | PyTorch Tutorials](https://pytorch.org/tutorials/intermediate/reinforcement_q_learning.html)
|
228 |
+
4. [Deep Q Learning (DQN) using PyTorch](https://medium.com/@vignesh.g1609/deep-q-learning-dqn-using-pytorch-a31f02a910ac)
|
229 |
+
|
230 |
+
---
|
Report.pdf
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:a28e62aa4cfc411d65a8b745759d810d14b0674110e044bfbd672f0f991f26ea
|
3 |
+
size 3280745
|
images/all_algos.png
ADDED
![]() |
Git LFS Details
|
images/all_algos_testing.png
ADDED
![]() |
Git LFS Details
|
images/desk.png
ADDED
![]() |
images/enhanced-environment.png
ADDED
![]() |
images/human.png
ADDED
![]() |
images/room.png
ADDED
![]() |
images/rover_desk_collision.png
ADDED
![]() |
images/rover_dest.png
ADDED
![]() |
images/rover_human_collision.png
ADDED
![]() |
images/rover_moving.png
ADDED
![]() |
images/rover_room_collision - Copy.png
ADDED
![]() |
images/rover_rover_collision.png
ADDED
![]() |
images/rover_start.png
ADDED
![]() |
images/target.png
ADDED
![]() |
images/trained_agents.gif
ADDED
![]() |
rx_rover_A2C.ipynb
ADDED
The diff for this file is too large to render.
See raw diff
|
|
rx_rover_DDQN.ipynb
ADDED
The diff for this file is too large to render.
See raw diff
|
|
rx_rover_DQN.ipynb
ADDED
The diff for this file is too large to render.
See raw diff
|
|
rx_rover_Double_Q_Learning.ipynb
ADDED
The diff for this file is too large to render.
See raw diff
|
|
rx_rover_Enhanced_Environment.ipynb
ADDED
@@ -0,0 +1,278 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
{
|
2 |
+
"cells": [
|
3 |
+
{
|
4 |
+
"cell_type": "markdown",
|
5 |
+
"metadata": {},
|
6 |
+
"source": [
|
7 |
+
"# Environment - Enhanced"
|
8 |
+
]
|
9 |
+
},
|
10 |
+
{
|
11 |
+
"cell_type": "code",
|
12 |
+
"execution_count": 2,
|
13 |
+
"metadata": {},
|
14 |
+
"outputs": [],
|
15 |
+
"source": [
|
16 |
+
"import gym\n",
|
17 |
+
"from gym import spaces\n",
|
18 |
+
"import numpy as np\n",
|
19 |
+
"import matplotlib.pyplot as plt\n",
|
20 |
+
"\n",
|
21 |
+
"import os\n",
|
22 |
+
"import random\n",
|
23 |
+
"import imageio\n",
|
24 |
+
"from tqdm import tqdm \n",
|
25 |
+
"from itertools import count\n",
|
26 |
+
"from collections import namedtuple, deque\n",
|
27 |
+
"\n",
|
28 |
+
"import torch\n",
|
29 |
+
"import torch.nn as nn\n",
|
30 |
+
"import torch.optim as optim\n",
|
31 |
+
"import random\n",
|
32 |
+
"import matplotlib.image as mpimg"
|
33 |
+
]
|
34 |
+
},
|
35 |
+
{
|
36 |
+
"cell_type": "code",
|
37 |
+
"execution_count": 3,
|
38 |
+
"metadata": {},
|
39 |
+
"outputs": [],
|
40 |
+
"source": [
|
41 |
+
"class RoverGridEnv(gym.Env):\n",
|
42 |
+
" metadata={'render.modes': ['human']} \n",
|
43 |
+
" def __init__(self, max_ts=20): \n",
|
44 |
+
" super(RoverGridEnv,self).__init__()\n",
|
45 |
+
" self.max_ts=max_ts # The Max_Timestamps is set to 20 by default.\n",
|
46 |
+
" self.grid_size=(15,15) \n",
|
47 |
+
" self.action_space=spaces.Discrete(5) \n",
|
48 |
+
" self.observation_space=spaces.MultiDiscrete([15,15,15,15,15,15])\n",
|
49 |
+
" self.rover_positions=np.array([[6,4],[10,4]])\n",
|
50 |
+
" self.operation_desks=np.array([[6,3],[10,3]])\n",
|
51 |
+
" self.rooms=np.array([[4,7],[4,10],[4,13],[8,7],[8,10],[8,13],[12,7],[12,10],[12,13]])\n",
|
52 |
+
" self.human_position=np.array([8,9])\n",
|
53 |
+
" self.targets=np.array([[5,10],[9,13]])\n",
|
54 |
+
" self.actions=[(0,-1),(0,1),(-1,0),(1,0),(0,0)] # Down,Up,Left,Right,Wait\n",
|
55 |
+
" self.rover_done=[False,False] \n",
|
56 |
+
" self.reset()\n",
|
57 |
+
" \n",
|
58 |
+
" def seed(self,seed=None):\n",
|
59 |
+
" np.random.seed(seed)\n",
|
60 |
+
" random.seed(seed)\n",
|
61 |
+
" \n",
|
62 |
+
" def reset(self):\n",
|
63 |
+
" self.current_step=0\n",
|
64 |
+
" self.rover_positions=np.array([[6,4],[10,4]])\n",
|
65 |
+
" self.rover_done=[False,False]\n",
|
66 |
+
" self.human_position=np.array([7,8])\n",
|
67 |
+
" self.current_step=0\n",
|
68 |
+
" return self._get_obs()\n",
|
69 |
+
" \n",
|
70 |
+
" def _get_obs(self):\n",
|
71 |
+
" return np.concatenate((self.rover_positions.flatten(),self.human_position))\n",
|
72 |
+
" \n",
|
73 |
+
" def step(self,actions):\n",
|
74 |
+
" rewards=np.zeros(2)\n",
|
75 |
+
" done=[False,False]\n",
|
76 |
+
" info={'message': ''} \n",
|
77 |
+
" for i,action in enumerate(actions):\n",
|
78 |
+
" if self.rover_done[i]:\n",
|
79 |
+
" done[i]=True \n",
|
80 |
+
" continue\n",
|
81 |
+
" prev_distance=np.linalg.norm(self.targets[i]-self.rover_positions[i])\n",
|
82 |
+
" if self._is_human_adjacent(self.rover_positions[i]):\n",
|
83 |
+
" rewards[i] -= 5\n",
|
84 |
+
" else:\n",
|
85 |
+
" delta=np.array(self.actions[action])\n",
|
86 |
+
" new_position=self.rover_positions[i]+delta\n",
|
87 |
+
" if self._out_of_bounds(new_position):\n",
|
88 |
+
" rewards[i] -= 15\n",
|
89 |
+
" continue\n",
|
90 |
+
" if self._collision(new_position,i):\n",
|
91 |
+
" rewards[i] -= 15\n",
|
92 |
+
" continue\n",
|
93 |
+
" self.rover_positions[i]=new_position\n",
|
94 |
+
" new_distance=np.linalg.norm(self.targets[i]-new_position)\n",
|
95 |
+
" if new_distance < prev_distance:\n",
|
96 |
+
" rewards[i]+=30 \n",
|
97 |
+
" else:\n",
|
98 |
+
" rewards[i] -= 20 \n",
|
99 |
+
" if np.array_equal(new_position,self.targets[i]):\n",
|
100 |
+
" rewards[i]+=100\n",
|
101 |
+
" self.rover_done[i]=True \n",
|
102 |
+
" done[i]=True\n",
|
103 |
+
"\n",
|
104 |
+
" # move human randomly\n",
|
105 |
+
" self._move_human()\n",
|
106 |
+
" self.current_step+=1\n",
|
107 |
+
" all_done=all(done) or self.current_step >= self.max_ts\n",
|
108 |
+
" if all_done and not all(done): # if the maximum number of steps is reached but not all targets were reached\n",
|
109 |
+
" info['message']='Maximum number of timestamps reached'\n",
|
110 |
+
" return self._get_obs(),rewards,all_done,info\n",
|
111 |
+
"\n",
|
112 |
+
" def _is_human_adjacent(self,position):\n",
|
113 |
+
" for delta in [(1,1),(1,-1),(-1,1),(-1,-1)]:\n",
|
114 |
+
" adjacent_position=position+np.array(delta)\n",
|
115 |
+
" if np.array_equal(adjacent_position,self.human_position):\n",
|
116 |
+
" return True\n",
|
117 |
+
" return False\n",
|
118 |
+
"\n",
|
119 |
+
" def _out_of_bounds(self,position):\n",
|
120 |
+
" return not (0 <= position[0] < self.grid_size[0] and 0 <= position[1] < self.grid_size[1])\n",
|
121 |
+
" \n",
|
122 |
+
" def _collision(self,new_position,rover_index):\n",
|
123 |
+
" if any(np.array_equal(new_position,pos) for pos in np.delete(self.rover_positions,rover_index,axis=0)):\n",
|
124 |
+
" return True # Collision with the other rover\n",
|
125 |
+
" if any(np.array_equal(new_position,pos) for pos in self.rooms):\n",
|
126 |
+
" return True # Collision with a room\n",
|
127 |
+
" if any(np.array_equal(new_position,pos) for pos in self.operation_desks):\n",
|
128 |
+
" return True # Collision with an operation desk\n",
|
129 |
+
" if np.array_equal(new_position,self.human_position):\n",
|
130 |
+
" return True # Collision with the human\n",
|
131 |
+
" return False\n",
|
132 |
+
" \n",
|
133 |
+
" def _move_human(self):\n",
|
134 |
+
" valid_moves=[move for move in self.actions if not self._out_of_bounds(self.human_position+np.array(move))]\n",
|
135 |
+
" self.human_position+=np.array(valid_moves[np.random.choice(len(valid_moves))])\n",
|
136 |
+
" \n",
|
137 |
+
" # def render(self,mode='human',save_path=None):\n",
|
138 |
+
" # fig,ax=plt.subplots(figsize=(7,7))\n",
|
139 |
+
" # ax.set_xlim(0,self.grid_size[0])\n",
|
140 |
+
" # ax.set_ylim(0,self.grid_size[1])\n",
|
141 |
+
" # ax.set_xticks(np.arange(0,15,1))\n",
|
142 |
+
" # ax.set_yticks(np.arange(0,15,1))\n",
|
143 |
+
" # ax.grid(which='both')\n",
|
144 |
+
"\n",
|
145 |
+
" # # draw elements\n",
|
146 |
+
" # for pos in self.rover_positions:\n",
|
147 |
+
" # ax.add_patch(Rectangle((pos[0]-0.5,pos[1]-0.5),1,1,color='blue'))\n",
|
148 |
+
" # for pos in self.operation_desks:\n",
|
149 |
+
" # ax.add_patch(Rectangle((pos[0]-0.5,pos[1]-0.5),1,1,color='darkgreen'))\n",
|
150 |
+
" # for pos in self.rooms:\n",
|
151 |
+
" # ax.add_patch(Rectangle((pos[0]-0.5,pos[1]-0.5),1,1,color='black'))\n",
|
152 |
+
" # ax.add_patch(Rectangle((self.human_position[0]-0.5,self.human_position[1]-0.5),1,1,color='purple'))\n",
|
153 |
+
" # for pos in self.targets:\n",
|
154 |
+
" # ax.add_patch(Rectangle((pos[0]-0.5,pos[1]-0.5),1,1,color='yellow',alpha=0.5))\n",
|
155 |
+
"\n",
|
156 |
+
" # if save_path is not None:\n",
|
157 |
+
" # plt.savefig(save_path)\n",
|
158 |
+
" # plt.close()\n",
|
159 |
+
" \n",
|
160 |
+
" # def close(self):\n",
|
161 |
+
" # plt.close()\n",
|
162 |
+
"\n",
|
163 |
+
" def render(self, mode='human', save_path=None):\n",
|
164 |
+
" fig, ax=plt.subplots(figsize=(7,7))\n",
|
165 |
+
" ax.set_xlim(0, self.grid_size[0])\n",
|
166 |
+
" ax.set_ylim(0, self.grid_size[1])\n",
|
167 |
+
"\n",
|
168 |
+
" rover_start_img_path='images/rover_moving.png' \n",
|
169 |
+
" rover_moving_img_path='images/rover_moving.png' \n",
|
170 |
+
" rover_dest_img_path='images/rover_dest.png' \n",
|
171 |
+
" rover_human_collision_path='images/rover_human_collision.png' \n",
|
172 |
+
" rover_room_collision_path='images/rover_room_collision.png' \n",
|
173 |
+
" rover_desk_collision_path='images/rover_desk_collision.png' \n",
|
174 |
+
" rover_rover_collision_path='images/rover_rover_collision.png' \n",
|
175 |
+
" desk_img_path='images/desk.png'\n",
|
176 |
+
" room_img_path='images/room.png' \n",
|
177 |
+
" human_img_path='images/human.png' \n",
|
178 |
+
" target_img_path='images/target.png' \n",
|
179 |
+
"\n",
|
180 |
+
"\n",
|
181 |
+
" for i, pos in enumerate(self.rover_positions):\n",
|
182 |
+
" \n",
|
183 |
+
" if self.rover_done[i]:\n",
|
184 |
+
" rover_img=mpimg.imread(rover_dest_img_path) \n",
|
185 |
+
" elif np.array_equal(pos, self.rover_positions[i]):\n",
|
186 |
+
" rover_img=mpimg.imread(rover_start_img_path) \n",
|
187 |
+
" elif self._is_human_adjacent(pos):\n",
|
188 |
+
" rover_img=mpimg.imread(rover_human_collision_path) \n",
|
189 |
+
" elif self._collision(pos, i):\n",
|
190 |
+
" \n",
|
191 |
+
" if any(np.array_equal(pos, pos) for pos in self.rooms):\n",
|
192 |
+
" rover_img=mpimg.imread(rover_room_collision_path)\n",
|
193 |
+
" elif any(np.array_equal(pos, pos) for pos in self.operation_desks):\n",
|
194 |
+
" rover_img=mpimg.imread(rover_desk_collision_path)\n",
|
195 |
+
" elif any(np.array_equal(pos, self.rover_positions[i]) for i in range(len(self.rover_positions)) if i != 0):\n",
|
196 |
+
" rover_img=mpimg.imread(rover_rover_collision_path)\n",
|
197 |
+
" else:\n",
|
198 |
+
" rover_img=mpimg.imread(rover_moving_img_path) \n",
|
199 |
+
" ax.imshow(rover_img, extent=(pos[0]-0.5, pos[0]+0.5, pos[1]-0.5, pos[1]+0.5))\n",
|
200 |
+
"\n",
|
201 |
+
" \n",
|
202 |
+
" for pos in self.operation_desks:\n",
|
203 |
+
" desk_img=mpimg.imread(desk_img_path)\n",
|
204 |
+
" ax.imshow(desk_img, extent=(pos[0]-0.5, pos[0]+0.5, pos[1]-0.5, pos[1]+0.5))\n",
|
205 |
+
"\n",
|
206 |
+
" for pos in self.rooms:\n",
|
207 |
+
" room_img=mpimg.imread(room_img_path)\n",
|
208 |
+
" ax.imshow(room_img, extent=(pos[0]-0.5, pos[0]+0.5, pos[1]-0.5, pos[1]+0.5))\n",
|
209 |
+
"\n",
|
210 |
+
" human_img=mpimg.imread(human_img_path)\n",
|
211 |
+
" ax.imshow(human_img, extent=(self.human_position[0]-0.5, self.human_position[0]+0.5, self.human_position[1]-0.5, self.human_position[1]+0.5))\n",
|
212 |
+
"\n",
|
213 |
+
" for pos in self.targets:\n",
|
214 |
+
" target_img=mpimg.imread(target_img_path)\n",
|
215 |
+
" ax.imshow(target_img, extent=(pos[0]-0.5, pos[0]+0.5, pos[1]-0.5, pos[1]+0.5))\n",
|
216 |
+
"\n",
|
217 |
+
" if save_path is not None:\n",
|
218 |
+
" plt.savefig(save_path)\n",
|
219 |
+
" plt.close()\n",
|
220 |
+
"\n",
|
221 |
+
" def close(self):\n",
|
222 |
+
" plt.close()\n",
|
223 |
+
"\n"
|
224 |
+
]
|
225 |
+
},
|
226 |
+
{
|
227 |
+
"cell_type": "code",
|
228 |
+
"execution_count": 4,
|
229 |
+
"metadata": {},
|
230 |
+
"outputs": [
|
231 |
+
{
|
232 |
+
"name": "stdout",
|
233 |
+
"output_type": "stream",
|
234 |
+
"text": [
|
235 |
+
"Initial Setup\n"
|
236 |
+
]
|
237 |
+
},
|
238 |
+
{
|
239 |
+
"data": {
|
240 |
+
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAkoAAAJGCAYAAACk4ariAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8g+/7EAAAACXBIWXMAAA9hAAAPYQGoP6dpAABD2ElEQVR4nO3deXhU9d3//9eZJZM9JIEkRBMISF0AQUUUcYFKpdbbhd5urVKq9693F1zpbdW7X2ytC7WLpbYWb717W3vfdekiaLXVUkRQlDWCIoqA7CEhLMkkk8x6Pr8/IpEIJ2SZyUyS5+O65uKaM2fOec9cb8555ZzPnGMZY4wAAABwBFeyCwAAAEhVBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHnmQX8Fm2bauqqko5OTmyLCvZ5QAAgD7GGKOGhgaVlpbK5Wr/mFHKBaWqqiqVlZUluwwAANDH7dy5U8cff3y786RcUMrJyZHUUnxubm6SqwEAAH2N3+9XWVlZa+ZoT8oFpUOn23JzcwlKAAAgYToyxIfB3AAAAA4ISgAAAA4ISgAAAA4ISgAAAA4ISgAAAA4ISgAAAA4ISgAAAA4ISgAAAA4ISgAAAA4ISgAAAA4ISgAAAA4ISgAAAA4ISgAAAA4ISgAAAA4ISgAAAA4ISgAAAA4ISgAAAA4ISgAAAA4ISgAAAA4ISgAAAA4ISgAAAA4ISgAAAA4ISgAAAA4ISgAAAA4ISgAAAA4ISgAAAA4ISgAAAA4ISgAAAA4ISgAAAA46HZSWLl2qSy+9VKWlpbIsSwsWLHCc91vf+pYsy9LcuXO7USIAAEBydDooBQIBjRkzRo8++mi7882fP1/Lly9XaWlpl4sDAABIJk9n33DxxRfr4osvbnee3bt36+abb9arr76qSy65pMvFAQAAJFOng9Kx2Lat6dOn64477tDIkSOPOX8oFFIoFGp97vf7410SAABAl8R9MPdDDz0kj8ejW265pUPzz5kzR3l5ea2PsrKyeJcEAADQJXENSmvWrNEvf/lL/e53v5NlWR16z9133636+vrWx86dO+NZEgAAQJfFNSi98cYb2rt3r8rLy+XxeOTxeLR9+3Z997vf1dChQ4/6Hp/Pp9zc3DYPAACAVBDXMUrTp0/XlClT2kybOnWqpk+frhtuuCGeqwIAAEi4TgelxsZGbd68ufX51q1btXbtWhUUFKi8vFyFhYVt5vd6vSopKdGJJ57Y/WoBAAB6UKeD0urVqzV58uTW57NmzZIkzZgxQ7/73e/iVhgAAECydTooTZo0ScaYDs+/bdu2zq4CAAAgJXCvNwAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAeeZBcAdEYkElE0Gu3WMjwej7xeb5wqQm/S0/1jG6O/vbO3W+vrri+dViSXZSW1hr6C7U//RFBCr9DY2Kjnn39BK5YsU2D/QRnb7tJyLJdLmYX5Gn/eBF155TRlZ2fHuVKkomT1j20bvfBOja6ZOLRL6+uuZ97Yqi+OGSSXm6DUHWx/+jeCElJeY2Oj7r3nPlVUNWpW+XDlD85QV/9ANsboYDioV155Wz96b4Puuff7bKz6uGT3T0aaWycMzunaCrspPc2dlPX2JcnuHyQfQQkp7/nnX1BFVaOuGzFKtqSGcEhul0sul6U0r1fNoZDS09IUjkQVibV/WNySpQJfpq4fMVr/u+ldLZj/oq6f/tWe+SBIilTpn5fX7NY/1lZJ6rmjO+ForMfW1VelSv8geQhKSGmRSEQrlizTrPLhsiW91lyrzKHFysvJktfjUVF+vnZU16hkYIEO+BtU3xBod3nRWFRN22p0YUaRvlg2XL94fZmuufYqxgz0UanUP7GYrQvGVuik8sI4fbpj+9PrG3psXX1RKvUPkoeghJQWjUYV2F+n/MEZagiHlDm0WOedfbqsw459Dyxu2fGUHldyzOUZY7TUrFHj7qDyfRkK7D+oWCzGhqqPSsX+sXpwYDUjk7onFfsHPY/LAyDlGTsmy5KMjNxud7d2NJZlye1xy5iWw+DG5tREX0f/oDvoHxCUAKCHcIQH6H0ISgCQNKYH5wHQFYxRAoAe0hA22hf4bKhJXFiKdO1yPwAOQ1ACgB4SikpNkZ5bXxeviwjgMJx6AwAAcMARJQBIAGOMvvrrSgXCMd3229WK2baMLL27cVuP1RCLGV3/m0pJ0rM3j+ux9QJ9CUEJABIkK92t5245I9llaNovViW7BKDXIigBQIKkuS01R4y8SbwpbcwYeVxcmADoKoIS+h2XZXE9G3RZZ/pn3LAB+seGgxpbUdClde0+0CRLUmlBZpfeL0l7DjZraFFWl9+P+GL70/swmBv9zugThiknq+s7HvRvnemfy8cN1potBxSKdO0KzJuqGrRpT0OX3nvIlj0Nmjp6ULeWgfhh+9P7EJTQ72RnZsjjdie7DPRSHe0fy7KUn+XVrv1N2rbX6Wappt1LJE0aVawLRhZ14DJKzsvZVhvQmPLcY9aLnsH2p/chKAFAgliSDjSEtH5HncJR+yhZxjrivibxnCdmG1UdCKggJ62TlQM4hKCEfmdXTa2aQ6Fkl4FeqrP9k+Z1KRy1tXNfoENjU+I5z/6GkI7LT1cSx5LjM9j+9D4EJaQ8y+WWMUaWLEVjURnT9ftaGWP08a4qBcNhGRlZLg6B93XJ7p+CbJ8G5vr0cU2jbLtn78m2szags0/o2kBytEh2/yD5+NUbUprH41Fm4QAdDAdV4MtU07YaLTVr5PZ0bQMTi8bUtK1a2ZnF2hcMKKswX27GC/RZqdA/Pq9LQwZlacPOOu2tD6poQHqX1t0VW2oa9NWzS3psfX1NKvQPko+ghJTm9Xo1YdJEvfLyMl0/YrQuzChS4+6guvpHnWVJ2ZnFckl6ZccWTbj0fHm93rjWjNSRKv1TWpCpyo8P6KH573dtxV0UisR085TyHl1nX5Iq/YPkIigh5U2bdrnuW/e+fr/pXV1cNlz5vgx19UokRkb7ggG9smOLqoYU6MZpl8W5WqSaVOif7HSPrj13qK49d2iX1ttVP13wvrxuS5bFIKWuSoX+QXJZpjsnXBPA7/crLy9P9fX1ys3lJ61oEQgEtOCFv+qtRW8osP+gjN2169JYLreyCvM1YfK5mjbtMmVlcSG+/iBZ/RON2frWk+t11TlDurS+7npu2TY9fuNoedwMR+0Otj99T2eyBkEJvUokElEs1rWN1CFut5vD3f1UT/ePbYwWvlfbrfV11xdGD5KLI0pxwfan7+hM1uDUG3oVr9fLRgZd1tP947IsTT21qMfWh8Ri+9M/cTwWAADAAUEJAADAAUEJAADAAUEJAADAAUEJAADAAUEJAADAAUEJAADAAUEJAADAAUEJAADAAUEJAADAAUEJAADAAUEJAADAAUEJAADAAUEJAADAAUEJAADAAUEJAADAAUEJAADAAUEJAADAQaeD0tKlS3XppZeqtLRUlmVpwYIFra9FIhHdeeedGj16tLKyslRaWqqvfe1rqqqqimfNAAAAPaLTQSkQCGjMmDF69NFHj3itqalJlZWVmj17tiorK/X8889r48aNuuyyy+JSLAAAQE+yjDGmy2+2LM2fP19XXHGF4zyrVq3S+PHjtX37dpWXlx9zmX6/X3l5eaqvr1dubm5XSwMAADiqzmQNT6KLqa+vl2VZGjBgwFFfD4VCCoVCrc/9fn+iSwIAAOiQhA7mDgaDuvPOO/WVr3zFMbHNmTNHeXl5rY+ysrJElgQAANBhCQtKkUhEV199tYwxmjdvnuN8d999t+rr61sfO3fuTFRJAAAAnZKQU2+HQtL27dv12muvtXv+z+fzyefzJaIMAACAbol7UDoUkjZt2qTFixersLAw3qsAAADoEZ0OSo2Njdq8eXPr861bt2rt2rUqKCjQ4MGDdeWVV6qyslIvvfSSYrGYqqurJUkFBQVKS0uLX+UAAAAJ1unLA7z++uuaPHnyEdNnzJihH/7wh6qoqDjq+xYvXqxJkyYdc/lcHgAAACRSQi8PMGnSJLWXrbpxWSYAAICUwr3eAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHHiSXQDQGZFIRNFotFvL8Hg88nq9HZrXNkZ/e2dvt9bXXV86rUguy0pqDX1FT/cP+hb6p38iKKFXaGxs1PPPv6AVS5YpsP+gjG13aTmWy6XMwnyNP2+CrrxymrKzs9ud37aNXninRtdMHNql9XXXM29s1RfHDJLLTVDqjmT1D/oG+qd/Iygh5TU2Nuree+5TRVWjZpUPV/7gDHX1AIsxRgfDQb3yytv60XsbdM+93z/mxiojza0TBud0bYXdlJ7mTsp6+5Jk9w96N/oHBCWkvOeff0EVVY26bsQo2ZIawiG5XS65XJbSvF41h0JKT0tTOBJVJNb+YXFLlgp8mbp+xGj976Z3tWD+i7p++lc7VMfLa3brH2urJPXc0Z1wNNZj6+qrUqV/0DvRPyAoIaVFIhGtWLJMs8qHy5b0WnOtMocWKy8nS16PR0X5+dpRXaOSgQU64G9QfUOg3eVFY1E1bavRhRlF+mLZcP3i9WW65tqrOjRmIBazdcHYCp1UXhinT3dsf3p9Q4+tqy9Kpf5B70P/QCIoIcVFo1EF9tcpf3CGGsIhZQ4t1nlnny7rsGPfA4tbgkvpcSXHXJ4xRkvNGjXuDirfl6HA/oOKxWKd2lBZPTiwmpFJ3ZOK/YPeg/6BxOUB0AsYOybLkoyM3G53t4KKZVlye9wypuUwuLE5tdXX0T/oDvoHBCWgEzjCAwD9C0EJ6BbTg/MAAHpap4PS0qVLdemll6q0tFSWZWnBggVtXjfG6J577tHgwYOVkZGhKVOmaNOmTfGqF0iqhrDRvsDhD33m+dEeXZ8n0rXLtQAA4qTTQSkQCGjMmDF69NFHj/r6T37yEz3yyCN67LHHtGLFCmVlZWnq1KkKBoPdLhZItlBUaor03KOL17UDAMRJp3/1dvHFF+viiy8+6mvGGM2dO1f/7//9P11++eWSpN///vcqLi7WggULdO2113avWgAAgB4U18sDbN26VdXV1ZoyZUrrtLy8PJ111ll6++23jxqUQqGQQqFQ63O/3x/PkoAuM8boq7+uVCAc022/Xa2YbcvI0rsbt/VYDbGY0fW/qZQkPXvzuB5bLwCgRVyDUnV1tSSpuLi4zfTi4uLW1z5rzpw5uvfee+NZBhA3WeluPXfLGckuQ9N+sSrZJQBAv5T0X73dfffdqq+vb33s3Lkz2SUBrdLclpojRlFbSXuEYkYeFxcmAIBkiOsRpZKSliuT1tTUaPDgwa3Ta2pqNHbs2KO+x+fzyefzxbMMoF0uy+rw9ZDGDRugf2w4qLEVBV1a1+4DTbIklRZkdun9krTnYLOGFmV1+f2Ir870D/BZ9E/vE9cjShUVFSopKdGiRYtap/n9fq1YsUITJkyI56qALht9wjDlZHUsuFw+brDWbDmgUKRrV9DdVNWgTXsauvTeQ7bsadDU0YO6tQzET2f6B/gs+qf36fQRpcbGRm3evLn1+datW7V27VoVFBSovLxct912m+6//36NGDFCFRUVmj17tkpLS3XFFVfEs26gy7IzMyS3W2Ed407flqX8LK927W/Str0BnXhc7lHmMpKxHC/ZPWlU8Sfz6BiX9XZezrbagC465fh2a0XP6Wj/AEdD//Q+nQ5Kq1ev1uTJk1ufz5o1S5I0Y8YM/e53v9P3vvc9BQIB/fu//7vq6up07rnn6pVXXlF6enr8qgZ6iCXpQENI63fUqaI4W16P6zNZ5shwc2Qm6vo8tm1UdSCggpy0Ln8GAEDXdTooTZo0ScY4327Bsiz96Ec/0o9+9KNuFQYkyq6aWhWGQnJ3cP40r0vhqK2d+wIaXpJzzPk7Mv6go/PsbwjpuPx0uRnUkDI62z/A4eif3ifpv3oDjsVyuWWMkSVL0Vi03aB+LMYYfbyrSsFwWEZGluvYm6uCbJ8G5vr0cU2jbLtn78m2szags0/o2kBytEh2/6B3o38Q11+9AfHm8XiUWThAB8NBFfgy1bStRkvNGrk9XdvAxKIxNW2rVnZmsfYFA8oqzJfb3f6yfF6XhgzK0oadddpbH1TRgJ47jbylpkFfPbukx9bX16RC/6D3on8gEZSQ4rxeryZMmqhXXl6m60eM1oUZRWrcHVRX/6izLCk7s1guSa/s2KIJl54vr9d7zPeVFmSq8uMDemj++11bcReFIjHdPKW8R9fZl6RK/6B3on8gEZTQC0ybdrnuW/e+fr/pXV1cNlz5vgx19UokRkb7ggG9smOLqoYU6MZpl3XofdnpHl177lBde+7QLq23q3664H153ZYsi0FKXZUK/YPei/6BZbpzwjUB/H6/8vLyVF9fr9zco/0cG/1RIBDQghf+qrcWvaHA/oMydteua2S53MoqzNeEyedq2rTLlJXV/oUcozFb33pyva46Z0iX1tddzy3bpsdvHC2Pm+GE3ZGs/kHfQP/0PZ3JGgQl9CqRSESxWNc2Uoe43e4OH+62jdHC92q7tb7u+sLoQXJxRCkuerp/0LfQP31HZ7IGp97Qq3i93h7dyLgsS1NPLeqx9SGxerp/0LfQP/0Tx/MBAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAcEJQAAAAceJJdAID+wxijWCwm27ZljGnzmmVZcrlccrvdsiwrSRUCQFsEJQAJZ4zR5qo9eito9Pq+Or17MKDqUEgNkahiRsryuDUwzauRA7L0+UEFmprj0bDBJckuGwAISgASr7ohqK9/tFcrDzQoqsOOJFluyZKabKk2GNEH1XWaX1Oni/Kz9eKggfJ42EQBSC7GKAFIuA3792nVAX/bkOQgZqQlBxtVHwz1QGUA0D6CEoCEam5u1hsbPlDMHDskHRKVtG7P3sQVBQAdRFACkDChUEgrFi/Uyo82duBY0qdiMVv/XLRQOzZ9lLDaAKAjCEoAEmbHpg+18uW/aKft6lRQsiVV7tmjN+Y/q5rqPYkqDwCOiaAEICECjY1a/Oc/qG5frRrTszv3Zkuq86Zrw5rl2vD2Utm2nZgiAeAYCEoAEuLDtWu048MNislSxO3t1HuNLIW9PhljtGrRK2rw1yeoSgBoH7+9BRB3kUhE6958TbYdU8ztVUa4ScOqt6qgYb9ym/zKaWqQNxqRzw4rP1M69QSPsgd49WEoT6/tzdPO9IFymZajSP59tfpwXaXOuuDCJH8qAP0RQQlA3G36YIP279ohS5a8sYi+vGy+JMmSkWWMXJaUl2E06nij8080yvS1vC/mlaaGXfrHO24Fwy2jmmzb1voVb2n8+Z/nit0AehxBCUDcbfnoQzU11MvIyJLkNp+OMUpzG4083uj0oUZDCiWP+9P3uVzS6eW26hqNln1kyciSJSlcX6fGhgbl5Ob2+GcB0L/FfYxSLBbT7NmzVVFRoYyMDA0fPlz33XffEfd1AtA3hUIhhev2KxwMHjbVyGUZHZdv9K/jbX1pjNGworYh6ZB0r3TeiS1HnVreadTc6Ff17l09Uj8AHC7uR5QeeughzZs3T0899ZRGjhyp1atX64YbblBeXp5uueWWeK8OQIpp8PvVXLe/zR9HliV9YaTRuGFGmWmfTGtnGeneT2b4ZBHB5oBq99ZoxMmnJKpsADiquAelt956S5dffrkuueQSSdLQoUP1zDPPaOXKlfFeFYAUFAgE1HBgf5tpbkua+Dkjj1v6YIdflmXppLKcNvM4TZekaDgsf31dIssGgKOK+6m3c845R4sWLdJHH7VcUXfdunV68803dfHFFx91/lAoJL/f3+YBoPcKBpvV1Njg+Prm3Y3avLuxw9MlKRqJKNDgvEwASJS4H1G666675Pf7ddJJJ8ntdisWi+mBBx7Qddddd9T558yZo3vvvTfeZQBIkmAwqObA0QOPJH3prMFHPe3mNF1q+eVbJBKJS30A0BlxP6L0xz/+UX/4wx/09NNPq7KyUk899ZR+9rOf6amnnjrq/Hfffbfq6+tbHzt37ox3SQB6UDQSUTQUcnzd7bLkch0ZiZymS5IxRtFoNG41AkBHxf2I0h133KG77rpL1157rSRp9OjR2r59u+bMmaMZM2YcMb/P55PP54t3GQCSxLZtRWOxttOMtOtAVEMGuo95LSRjjHYfiMmYtMMnyhhuYwKg58X9iFJTU5NcrraLdbvd3KsJ6Cd86enKzM5ucxotZku/W9yovfV2u5cKMcZorz+mJxc3Kta6ybA0ePjnNG7CeYksGwCOKu5B6dJLL9UDDzygl19+Wdu2bdP8+fP18MMPa9q0afFeFYAUdMKIz2nytTdocMWIlusCSDJGen9XWI+/3qy99aGjhiVjjKrrw/rN62Ft2BU+dGUAlQ47QRd+9UadPGpUD34KAGgR91Nvv/rVrzR79mx95zvf0d69e1VaWqpvfvObuueee+K9KgApKDMzU+MnXajjR5ykJX/+P21c9baMiSpkW1q826fLqjdpkOWV0gdKnqyWN0UDMsF9qtoT1dLdn1OO3STLslQ6bITOu3qGTh41OrkfCkC/FfeglJOTo7lz52ru3LnxXjSAXsLtdqusvFxXfONmrR5xklYvfFmRmkY1+XJlR5tlGnfLNG7/zLuMTLRQwbQspcdCKj3hpE+OJBGSACQP93oDkDDZOTkaO+kLqnRlao9/gayGgLZllGmUDsiltqffbEnbMo5XKCtH1ceXybrgEpVVDE9O4QDwibiPUQKAQzYdOKibVm7QQ9FsBdOy1FQwUD8b+DX5rawj5q13ZevhwulqzstXQ2au7g9laOaK97TlABehBZA8BCUAcWfbtt7eVaMvr96iZw4GtS8UUXrdfoWzc7Uqe5QezrlWocMOaIfk0c9zrtWa7JEKZ+cq/cB+1QbD+r8DQV2+eqOWV9Xwy1kASUFQAhB37+w/oJs27tT7DQHZliUjI3coqFhamiLuNP0h6wt62zdKRi33vX3LN1pPZ16kiDtN0bR0eZubZCTZlqUN/mbdsqFK67ZxMVoAPY+gBCCuDtTX6zd76vVuXZOMLCkWU/aH76l5X620/WN5/HXa6S7SE9mXqtY1QHtd+Xoi+1Ltcg9U5oFaZdfukTfUrKIN6+QKh2RktLa+QfN21qqOe0EC6GEEJQBxVdkU1F+27tGhG45kbflQRa/9TccXDVRJ7W4NWvSSbCPNzzhff8m8QH/OnKQFGefL6/drxD9fkre5SU0FA3XcO8s1+N3VsmIxRST9qS6ktXX8/gRAzyIoAYibWCymV2v8qo9+Op4o7cA+pYVCmvG16zX21FOVvnunZIzCVpp+mvNV/SznKwpbXg3c/IEkafuESdr4xS+r/vihyt/+sdICDZKkumhMrxzYzVglAD2KP88AxI1t21pWW99mWrhwkEK+dD351O9V729Q8PghrVfs3uEpaZ0v7+A+2bl58g8uk+316uCwz6nojYXyhsM6dIvdZbV1sm37iNskAUCiEJQAxI1t29rS2NRmWmD4ido75V/U/ME6RUqHqW7cOZJ1ZNC5/d++LtsYfXP7AUnSkJNP1iNfnKj/b/sBrWuOSJI2Nza3e684AIg3ghKAuPF4PLrd1CtsDjs9Zkk6qazl0apOn7nepLKrGyVJ96olFGUHLVXvqNP0YERf/iQc+eTmaBKAHmWZFPvzzO/3Ky8vT/X19crNzU12OQAAoI/pTNbgTzMAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHXB4AvUokElE0Gj32jO3weDzyer1xqgi9Cf2D7qB/+ieCEnqFxsZGPf/8C1qxZJkC+w/KdPE2FpbLpczCfI0/b4KuvHKasrOz41wpUhH9g+6gf/o3rqOElNfY2Kh777lPFVWNmlo+TPlpGYfugNFpxhgdDAf1yo4t2n5cnu659/tsrPo4+gfdQf/0TVxHCX3K88+/oIqqRl03YpQKfJlqjITVFI0qaMdku10KRCOKuSw1x2Lyh0PtPhojERX4MnX9iNEq312nBfNfTPbHQ4LRP+gO+gecekNKi0QiWrFkmWaVD5ct6bXmWmUOLVZeTpa8Ho+K8vO1o7pGJQMLdMDfoPqGQLvLi8aiatpWowszivTFsuH6xevLdM21VzFmoI+if9Ad9A8kghJSXDQaVWB/nfIHZ6ghHFLm0GKdd/bpsg479j2wuFCSVHpcidNiWhljtNSsUePuoPJ9GQrsP6hYLMaGqo+if9Ad9A8kTr2hFzB2TJYlGRm53e42G6nOsixLbo9bxkiWLBk7FsdKkYroH3QH/QOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEvodl2Wp65eMQ39H/6A76J/eh6CEfmf0CcOUk5WZ7DLQS9E/6A76p/chKKHfyc7MkMftTnYZ6KXoH3QH/dP7EJQAAAAcEJTQ7+yqqVVzKJTsMtBL0T/oDvqn9yEoIeVZLreMMbJkKRqLyhjT5WUZY/TxrioFw2EZGVkuDoH3dfQPuoP+gSfZBQDt8Xg8yiwcoIPhoAp8mWraVqOlZo3cnq5tYGLRmJq2VSs7s1j7ggFlFebLzXiBPov+QXfQP5AISkhxXq9XEyZN1CsvL9P1I0brwowiNe4Oqqt/1FmWlJ1ZLJekV3Zs0YRLz5fX641rzUgd9A+6g/6BRFBCLzBt2uW6b937+v2md3Vx2XDl+zLU1SuRGBntCwb0yo4tqhpSoBunXRbnapFq6B90B/0Dy3TnhGsC+P1+5eXlqb6+Xrm5uckuBykiEAhowQt/1VuL3lBg/0EZO9al5Vgut7IK8zVh8rmaNu0yZWVlxblSpCL6B91B//Q9nckaBCX0KpFIRLFY1zZSh7jdbg5391P0D7qD/uk7OpM1OPWGXsXr9bKRQZfRP+gO+qd/4vIAAAAADghKAAAADghKAAAADghKAAAADghKAAAADghKAAAADghKAAAADghKAAAADghKAAAADghKAAAADghKAAAADghKAAAADghKAAAADghKAAAADghKAAAADghKAAAADghKAAAADghKAAAADhISlHbv3q3rr79ehYWFysjI0OjRo7V69epErAoAACBhPPFe4MGDBzVx4kRNnjxZf//73zVo0CBt2rRJ+fn58V4VAABAQsU9KD300EMqKyvTk08+2TqtoqIi3qsBAABIuLifenvxxRc1btw4XXXVVSoqKtJpp52mJ554wnH+UCgkv9/f5gEAAJAK4h6UPv74Y82bN08jRozQq6++qm9/+9u65ZZb9NRTTx11/jlz5igvL6/1UVZWFu+SAAAAusQyxph4LjAtLU3jxo3TW2+91Trtlltu0apVq/T2228fMX8oFFIoFGp97vf7VVZWpvr6euXm5sazNAAAAPn9fuXl5XUoa8T9iNLgwYN1yimntJl28skna8eOHUed3+fzKTc3t80DAAAgFcQ9KE2cOFEbN25sM+2jjz7SkCFD4r0qAACAhIp7ULr99tu1fPlyPfjgg9q8ebOefvppPf7445o5c2a8VwUAAJBQcQ9KZ555pubPn69nnnlGo0aN0n333ae5c+fquuuui/eqAAAAEirug7m7qzMDrAAAADorqYO5AQAA+gqCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgAOCEgAAgANPsgsA0D3BYEiv/3OlHvv1H7X8rXUKhyLdWl6aL03nnDtW37zpak36/JnypafFqVKkIvoHaJ9ljDHJLuJwfr9feXl5qq+vV25ubrLLAVKabRst/Psyfe+2n6uopFDnnDtWmVkZ3VpmU6BZy5ZWal9tnX76yH9oytQJsiwrThUjldA/6K86kzU4ogT0YrFoVE/M+7OKigv044dv16ljT5TX273/1pFwRGvf2ai7Zz2s/573F02+cLw83VwmUhP9AxwbY5SAXqSmep/Wv7tJtt1yIDhmG61c/p4mnn9aXHZykuRN82rsaSfqnPNO04q317Wuy7ZtrX93k2qq93d7HUgO+gfoPIIS0Iv8+dl/aMa1/6loNPrJFKNwKKKMzIw2OznTFJS9aKViz74qe9FKmcamI5bV3jzeNK8yM9MVCkZk1LKji0Si+to1d+n5Py1M6GdE4tA/QOdxPBToRRr8AVXt2qv2hhaapqDs3y6Q3lgr0xSUMtNlNnws179dISszvcPzHLFcY1S1q1b++kACPhl6Av0DdB5BCehlbNvW8mXr5PF6FAlHZNt2m9fN2++27MACzbKkln/fXCcz6gRZk8d1eJ6OrAu9D/0DdA5BCehlIpGopl18i+PrpvagTFNQh35nZKnlCIC1r65T83RkXeh96B+gcwhKQC/j9Xr0l5fnfvJXelT/esmtbV63BuW3nAo59Ne+1HI6ZOCATs3TkXWh96F/gM4hKAG9jMvl0plnj5bPl6ZgMCSXq+1vMqyzRsls+LjlVEhTs6zMDOm8sbLGj+zUPB1ZF3of+gfoHIIS0Ivk5mXr+LLidi/gZ2VnyvVvV7SMF9lXJw0cIGv8SFmHXUiwI/McsVzL0nFlxcrLy47jJ0JPon+AziMoAb3IlddO1eQpZ8njOfRf11JauldNgWZFwhF507wtUzPT2wyqPZr25gmHIwoEmpWe4ZP1yUgUr9ej//3TjzVoUH7cPg96Fv0DdB7HQYFepKi4QCePHCaXq2Xn43a7dPaEMXpzaaXWvvOhIuHu3adLatnJvbPmAy1b+o7OPufU1nW5XC6dMnK4BhUVdHsdSA76B+g87vUG9GK2bfTPV9/S9279uQYOGqAJ556mrKyjX8emowKBZi1b+o4OHKjXzx65QxdedDb36uqj6B/0V53JGgkPSj/+8Y91991369Zbb9XcuXOPOT9BCeicUDCspa+v1n/P+4uWv71O4WD3jgr40r2aMHGs/u1b/6rzJp0hn4+7v/dl9A/6o5QJSqtWrdLVV1+t3NxcTZ48maAEJIgxRrFoTLZtWm8ZcYi956BiO/fJVZAt95Aiyetud1mWLLlcltweN0cC+gn6B/1NZ7JGwgZzNzY26rrrrtMTTzyh+++/P1GrAaCWXxR99g7tsV37FHxptUwoIntvnVyjhyi2crM8pxyvtLNOTFKlSEX0D+AsYYO5Z86cqUsuuURTpkxpd75QKCS/39/mAaD7Yrv3K/DT+Qo+vVSRNVtk9jfINThfjT94Ws3PvSETjLR7zy/0b/QP0CIhR5SeffZZVVZWatWqVcecd86cObr33nsTUQaAaEx29UGp+qACG3dLPq9yfvw1+Tfv0aYbfqnAr7+pETkZGpjGlUJwFPQPEP8xSjt37tS4ceO0cOFCnXrqqZKkSZMmaezYsUcdoxQKhRQKhVqf+/1+lZWVMUYJ6KbYrv0K/mmZmv/4puyqAy3TLEt/vGC43vv8WFlnjlK+16Mst0tfHzxA5elpYkgJDqF/0JcldTD3ggULNG3aNLndnw74i8VisixLLpdLoVCozWufxWBuoGuMMTJGkjFyuV2HJiry/g41/vAZNa/dqt9eeLJ+dv5I+fMLlJ+Vqe8OGagzczL0v9V1+s5xBTorJ12yLFmWGIjbz9A/6E+SGpQaGhq0ffv2NtNuuOEGnXTSSbrzzjs1atSodt9PUAK6xraN3nh9tRoamvQvl1/QOt0YI0VjemrdBj0ZkN6IuGQ+2Ym5JJ2QmaafnlCi3+zar5nbt2nAgByde8Hp7Oj6GfoH/UlnskbcB3Pn5ORo1KhRbR5ZWVkqLCw8ZkgC0HXRaFRzfvSE3n9vc5vplmXJ8nr09XGnauF5o3Rj6ae3kLAlfdQU1u/31Kks3av3392kOT96QtForIerR7LRP8DRcQsToI+wbVvvrdvU7jxey9LPP1eiN86o0EMnFCvLZem0bJ+uLcnTPRVFkqT31n4kY/Nrpv6G/gGOrkd+qvD666/3xGqAfi92jL/kLctSnsetcwdkakJehm4tK5AlS97DxpREo7EjLjqI/oH+AY7EbzqBXqapKahlSyr10Ufb1egPtF7LJhqNKRaL6c0la2Rsu0vLfnNppaLRmH724JNyfzKg17IsZedm6XMnDtG555+ujMzu3QsMyUX/AJ3DTXGBXsIYo61bdunH9/233npjrfz+RkUiUSX8j3dL8no9ysvL1sQLztCd379RQ4cdx2DdXob+AT6VErcwARBf27dV6T9u+ak2bdyub3z7Sl1+5edVUjJQliuxOxxjG+3ZU6sFf16k3z72vPbVHtTDv/6eyocMTuh6EV/0D9A1BCWgl1jwl9f0/nub9cMHb9I1131RLlfP/RajYtjxuvU/pmtQUYHumz1PLz6/WDfd/tUeWz+6j/4BuoZfvQG9xPw/LtSEiWP1hS9OaN3J2batv/xxoa771zu14M+LZH9mbMnO7Xv0Hzf/VHfc+nPt3FHd5rXOvtflcmnqlyZq/IRT9ZfnXk3sh0Xc0T9A13BECeglNn20XVOmnqP8/E/Pp0ciUf3w7ke1e1eN1r+7SV+67HylpX36989fFyzRH556SbKk4SeU6Vs3X92t9+YX5OmEEeV6beHynvnQiBv6B+gaghLQS+TkZKmxsUnhSFQZnpbbALlcLp159mj5X23U+AmjjzidMnxEmQYfVySXZWn4Cce3ea0r7w2HI2psDCgnNyuBnxSJQP8AXcOv3oBe4uvX/qe2b6vS47+/VyeMKJdlWTLGqKZ6v6p21+q44wepqLiwza+Jgs0hfbxllyzLUsWw45Se4Wt9rbPv9aWn6aON2/XNGT/Q8BPK9Ns/3N+jnx/dQ/8An0rqvd66i6AEHN2bSyr1za//UOPOGqnv3/tNlZWVKD3Dl/CfWRtjFGwOaceOat0/e57eWfOhnvj9vZpw7tiErhfxRf8AnyIoAX1QU1NQz/7f3/TzOb9TUXGBzjxrlAYU5MqV4B2dbYwOHvBr1fL3tK+2Tv/xnzfomuu+qIwMLhzYm9A/wKcISkAf1dwU1DtrPtATv/mT3lq2Vvtq6xJ+Xy2Xy9LAQfk659yx+sZ3rtLY00/i6sq9FP0DtCAoAQAAOODK3ACOEA6HtXv7drksS6VDhsjr9Sa7JPQi9A/6K4IS0MfFYjHV7dwlLVulIRu3ylTV6MBJFdIVX1RBxVC53e5kl4gURv+gvyMoAX2UMUYH9taq+vU3NWLzTnkaGiU7JquuXgV/XaTwinVqmHKO7Cu+pPyiQdykFG3QP0ALxigBfVBDXb0OrFyjAes+UE5Ts6ymoIykgFwyG7coa/9+WZKMx6P68uPkv3yKBk25QFn8n4PoH/R9DOYG+qnmpib5129U5spKZcmSNf40aWChmuf/XetzC7V4zDjFwhGd/49XNGblCmUGGiXLkklLU+DkE9T85YuUN36cMrK4cnJ/RP+gvyAoAf1MKBRS1Qcfylq2SuXBqKzTRklnnCo7M1O79jfob7v9+mBgiSKfDMB1RyIa/uEHuvDlF3XChg3yhYKyJNkZ6Yqcdor2XvUvKhk9Uj6fr/0Vo0+gf9DfEJSAfiIWi2nf9h3SslUq3LJD7tEnSeedLVOYrwP1TXq90dbbVoaaXG7ps2NIjJE3HNaoytW68OW/qmzrx/JEozKWJTs7U/snnSXXtEtUOIwBu30V/YP+iqAE9AP11dUyK95V5rr18gSaZPnSpG9+TY25uXqrLqwlJl11bq/MsQbZGqPMxkaNe+tNTfr7yyraU9WyU7QsRQvzFbxkivSlC5U3uKhnPhh6BP2D/oygBPRxu9a8o8EvL5LVHJJlGxlJ/swsrRl/tt48ZYz2etJkW65jLudwlm2roLZWExf9Q+PfWKoBBz4ZsOt2y5QPVc2t03XcGWMT8XHQw+gf9HdccBLo4/a8+ppK39kgFQ9SY/4ArT/hRL1+2njtKiqR7XLYwRmjdL9fFWtWamjlKqX7/WoYWKQtZ5+jXaPHKJKRof3FxfrrNV9V5Tnn6vxX/6Yxq1Yqq6FB1vatqlq4mB1dH0H/AB3HESWgF9q1fJU8j/2vdg88Tksu/pK2jDhRtvso40gOMUYD9lTpgid+o/J31shl260vRb1p+uDCL+it67+uYG5e6/yuWEzDN36gC/7+skr37VH0O19T2dln9sCnQ6LRP+jvOPUG9HHGGPn9fv1wW6OCA0ucd3AtMyuveo++8Mufq3TDe/JalnLcHnktl0J2TA2xqKIul1Zf9RW9/ZXp0uFHFIxRRm2VfjgsTzk5OVxUsI+gf9DfdSZrdO4kNICUYFmW0tLSZLusY+7kcmuqNeVXD6v0g/VKsyyVeH0a6EnTAI9Xg7w+DfT65LZtfW7pYg2o3vPZFSnqcistLY2dXB9C/wAdR1AC+ihjjJoPHtS4/3pUx73/nnyyVOJNV4bL3brTcn1ydMBjWcreV6ucvTVJrhqpgv4BWhCUgD7IGCN/Xb0WLlqsecaj2owsFXl9Sne5jvjL3pJkyZLLtuWyY8kpGCmF/gE+RVAC+hhjjPz19Vq8cJGqdldpcWGRHjx9omqzc444zWKMUdC2FTW2mnNy1TQgP0lVI1XQP0BbBCWgDzHGqLGhQUv++Zqqq6okt1vKGaB/lJbrO6eOV1V6hsxh84aNrdpISDFJVaNO1YHjy9ssKxYOK9rUmJTPgp5H/wBHIigBfYht21r51tuq2rW7dScnT8vVlf9RNFjfPnW8dmRkSpJCxlZ1OKSgsbWvYpiWX3u9Ymlpkj7ZyQWb1VRTpdDB/Un8ROhJ9A9wJIIS0IfU7t2rHdt2tJwiyc6TPN7W0yW2ZelvxaX69pjx2pieob2H7eT+efMsHSgrb7OsUP1B2ZFwMj4GkoT+AY5EUAL6kIP7DygcCrXs4LxpR44pkfRqwSB9e/iJ2pKRqX0Vw/WPW+/Q3uEjjpiXnVz/Q/8AR+IWJkAfYh+6YvLRrlljjBSLSg31Wpqbr59NnqozLpqi/UOHOcyf2FqReugf4EgEJaAPyc3Lk8frVSQaadmpuT0tO7HDdnKKRpQ/cKCyvnCh9hUN4kKAaEX/AEfi1BvQh5QMHqziwSWSbbfs1MJBKRKWQs2Sv06KRlQ4cKAmfeFCDWQnh8+gf4AjcUQJ6EM8Xo/GnzNBzU1N2l+7r2Xn9skRAcuyNKikWOd/frIKBw1kJ4cj0D/AkQhKQB9iWZYGFQ3SF750sTa8+5527tihYHOzMrOyNKRiqE4ePUrZ2dnt7uSMMTK2LcMgk36H/gGORFACeinLsmTpyB2WZVnKG5Cns8+dqHGxs2SMkcuy5PZ4Wt7zyU7OGFuxYFCxcFifHXkbC4Vkoi23o3C5LI4e9EH0D9AxBCWgl3K73SrL8GjzUV6zLEuW21KaO+2o7zXGVujgAYX8dS3jUdoxJCdbLhfDGfsa+gfoGLoX6KW8Xq8m236VRJvlNqbll0kdFAuFFfbXO+/kTMvGodQjTR3gkdfrjU/RSBn0D9AxHFECerHTPzdcg6prtGHbBgVDoQ6/zxgjE4u2O096mk+nlA3VkONKu1smUhT9AxwbQQno5cpKilVWUpzsMtBL0T9A+whKQB8Ti8UUDocViUQUiUQUi8Vk27aMMa1XXna5XLIsSy6XS263W16vV16vV2lpaXK73Un+BEgm+gdoi6AE9AFNTU2qqanR/v371dTUpGg0Ktu2W3dy0ienSz4Zh3L4r5cO7excLpc8Ho8yMzNVWFiokpISZWRkJO0zoefQP4AzghLQB9TW1urjjz9WOBxu/eu/PYe/HovFFIlEJLXsAAOBgOrr6+V2u1VeXu60CPQh9A/gjKAE9AHl5eUaOHCg/H6/Ghsb1dzcrGAwqGAwqEgkcsTRgcOPAni9XqWnpys9PV0ZGRnKzs5Wbm6uMjMzk/yp0FPoH8AZQQnoAyzLUlZWlrKyspJdCnoh+gdwxnWUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHMQ9KM2ZM0dnnnmmcnJyVFRUpCuuuEIbN26M92oAAAASLu5BacmSJZo5c6aWL1+uhQsXKhKJ6KKLLlIgEIj3qgAAABLKMsaYRK6gtrZWRUVFWrJkic4///wjXg+FQgqFQq3P/X6/ysrKVF9fr9zc3ESWBgAA+iG/36+8vLwOZY2Ej1Gqr6+XJBUUFBz19Tlz5igvL6/1UVZWluiSAAAAOiShR5Rs29Zll12muro6vfnmm0edhyNKAACgJ3XmiJInkYXMnDlT69evdwxJkuTz+eTz+RJZBgAAQJckLCjddNNNeumll7R06VIdf/zxiVoNAABAwsQ9KBljdPPNN2v+/Pl6/fXXVVFREe9VAAAA9Ii4B6WZM2fq6aef1gsvvKCcnBxVV1dLkvLy8pSRkRHv1QEAACRM3AdzW5Z11OlPPvmkvv71rx/z/Z0ZYAUAANBZSR3MneDLMgEAAPQY7vUGAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADggKAEAADgIGFB6dFHH9XQoUOVnp6us846SytXrkzUqgAAABIiIUHpueee06xZs/SDH/xAlZWVGjNmjKZOnaq9e/cmYnUAAAAJkZCg9PDDD+sb3/iGbrjhBp1yyil67LHHlJmZqf/5n/9JxOoAAAASwhPvBYbDYa1Zs0Z333136zSXy6UpU6bo7bffPmL+UCikUCjU+ry+vl6S5Pf7410aAABAa8Ywxhxz3rgHpX379ikWi6m4uLjN9OLiYn344YdHzD9nzhzde++9R0wvKyuLd2kAAACtGhoalJeX1+48cQ9KnXX33Xdr1qxZrc/r6uo0ZMgQ7dix45jF90d+v19lZWXauXOncnNzk11OyuH7aR/fT/v4ftrH99M+vh9nqfbdGGPU0NCg0tLSY84b96A0cOBAud1u1dTUtJleU1OjkpKSI+b3+Xzy+XxHTM/Ly0uJLzNV5ebm8v20g++nfXw/7eP7aR/fT/v4fpyl0nfT0YMxcR/MnZaWpjPOOEOLFi1qnWbbthYtWqQJEybEe3UAAAAJk5BTb7NmzdKMGTM0btw4jR8/XnPnzlUgENANN9yQiNUBAAAkREKC0jXXXKPa2lrdc889qq6u1tixY/XKK68cMcD7aHw+n37wgx8c9XQc+H6Ohe+nfXw/7eP7aR/fT/v4fpz15u/GMh35bRwAAEA/xL3eAAAAHBCUAAAAHBCUAAAAHBCUAAAAHBCUAAAAHKRcUHr00Uc1dOhQpaen66yzztLKlSuTXVJKmDNnjs4880zl5OSoqKhIV1xxhTZu3JjsslLSj3/8Y1mWpdtuuy3ZpaSU3bt36/rrr1dhYaEyMjI0evRorV69OtllJV0sFtPs2bNVUVGhjIwMDR8+XPfdd1+HbpbZFy1dulSXXnqpSktLZVmWFixY0OZ1Y4zuueceDR48WBkZGZoyZYo2bdqUnGKToL3vJxKJ6M4779To0aOVlZWl0tJSfe1rX1NVVVXyCu5hx+qfw33rW9+SZVmaO3duj9XXFSkVlJ577jnNmjVLP/jBD1RZWakxY8Zo6tSp2rt3b7JLS7olS5Zo5syZWr58uRYuXKhIJKKLLrpIgUAg2aWllFWrVum//uu/dOqppya7lJRy8OBBTZw4UV6vV3//+9+1YcMG/fznP1d+fn6yS0u6hx56SPPmzdOvf/1rffDBB3rooYf0k5/8RL/61a+SXVpSBAIBjRkzRo8++uhRX//JT36iRx55RI899phWrFihrKwsTZ06VcFgsIcrTY72vp+mpiZVVlZq9uzZqqys1PPPP6+NGzfqsssuS0KlyXGs/jlk/vz5Wr58eYfutZZ0JoWMHz/ezJw5s/V5LBYzpaWlZs6cOUmsKjXt3bvXSDJLlixJdikpo6GhwYwYMcIsXLjQXHDBBebWW29Ndkkp48477zTnnntusstISZdccom58cYb20z78pe/bK677rokVZQ6JJn58+e3Prdt25SUlJif/vSnrdPq6uqMz+czzzzzTBIqTK7Pfj9Hs3LlSiPJbN++vWeKSiFO38+uXbvMcccdZ9avX2+GDBlifvGLX/R4bZ2RMkeUwuGw1qxZoylTprROc7lcmjJlit5+++0kVpaa6uvrJUkFBQVJriR1zJw5U5dcckmbHkKLF198UePGjdNVV12loqIinXbaaXriiSeSXVZKOOecc7Ro0SJ99NFHkqR169bpzTff1MUXX5zkylLP1q1bVV1d3eb/WF5ens466yy20w7q6+tlWZYGDBiQ7FJSgm3bmj59uu644w6NHDky2eV0SEJuYdIV+/btUywWO+I2J8XFxfrwww+TVFVqsm1bt912myZOnKhRo0Ylu5yU8Oyzz6qyslKrVq1Kdikp6eOPP9a8efM0a9Ys/ed//qdWrVqlW265RWlpaZoxY0ayy0uqu+66S36/XyeddJLcbrdisZgeeOABXXfddckuLeVUV1dL0lG304dew6eCwaDuvPNOfeUrX1Fubm6yy0kJDz30kDwej2655ZZkl9JhKROU0HEzZ87U+vXr9eabbya7lJSwc+dO3XrrrVq4cKHS09OTXU5Ksm1b48aN04MPPihJOu2007R+/Xo99thj/T4o/fGPf9Qf/vAHPf300xo5cqTWrl2r2267TaWlpf3+u0HXRSIRXX311TLGaN68eckuJyWsWbNGv/zlL1VZWSnLspJdToelzKm3gQMHyu12q6amps30mpoalZSUJKmq1HPTTTfppZde0uLFi3X88ccnu5yUsGbNGu3du1enn366PB6PPB6PlixZokceeUQej0exWCzZJSbd4MGDdcopp7SZdvLJJ2vHjh1Jqih13HHHHbrrrrt07bXXavTo0Zo+fbpuv/12zZkzJ9mlpZxD22K20+07FJK2b9+uhQsXcjTpE2+88Yb27t2r8vLy1m319u3b9d3vfldDhw5NdnmOUiYopaWl6YwzztCiRYtap9m2rUWLFmnChAlJrCw1GGN00003af78+XrttddUUVGR7JJSxoUXXqj33ntPa9eubX2MGzdO1113ndauXSu3253sEpNu4sSJR1xO4qOPPtKQIUOSVFHqaGpqksvVdlPodrtl23aSKkpdFRUVKikpabOd9vv9WrFiBdvpTxwKSZs2bdI///lPFRYWJruklDF9+nS9++67bbbVpaWluuOOO/Tqq68muzxHKXXqbdasWZoxY4bGjRun8ePHa+7cuQoEArrhhhuSXVrSzZw5U08//bReeOEF5eTktI4HyMvLU0ZGRpKrS66cnJwjxmplZWWpsLCQMVyfuP3223XOOefowQcf1NVXX62VK1fq8ccf1+OPP57s0pLu0ksv1QMPPKDy8nKNHDlS77zzjh5++GHdeOONyS4tKRobG7V58+bW51u3btXatWtVUFCg8vJy3Xbbbbr//vs1YsQIVVRUaPbs2SotLdUVV1yRvKJ7UHvfz+DBg3XllVeqsrJSL730kmKxWOu2uqCgQGlpackqu8ccq38+Gxy9Xq9KSkp04okn9nSpHZfsn9191q9+9StTXl5u0tLSzPjx483y5cuTXVJKkHTUx5NPPpns0lISlwc40l//+lczatQo4/P5zEknnWQef/zxZJeUEvx+v7n11ltNeXm5SU9PN8OGDTPf//73TSgUSnZpSbF48eKjbmtmzJhhjGm5RMDs2bNNcXGx8fl85sILLzQbN25MbtE9qL3vZ+vWrY7b6sWLFye79B5xrP75rN5weQDLmH56+VkAAIBjSJkxSgAAAKmGoAQAAOCAoAQAAOCAoAQAAOCAoAQAAOCAoAQAAOCAoAQAAOCAoAQAAOCAoAQAAOCAoAQAAOCAoAQAAODg/wfmcbw/SwqIXQAAAABJRU5ErkJggg==",
|
241 |
+
"text/plain": [
|
242 |
+
"<Figure size 700x700 with 1 Axes>"
|
243 |
+
]
|
244 |
+
},
|
245 |
+
"metadata": {},
|
246 |
+
"output_type": "display_data"
|
247 |
+
}
|
248 |
+
],
|
249 |
+
"source": [
|
250 |
+
"env=RoverGridEnv()\n",
|
251 |
+
"print(\"Initial Setup\")\n",
|
252 |
+
"observation=env.reset()\n",
|
253 |
+
"env.render()"
|
254 |
+
]
|
255 |
+
}
|
256 |
+
],
|
257 |
+
"metadata": {
|
258 |
+
"kernelspec": {
|
259 |
+
"display_name": "Python 3",
|
260 |
+
"language": "python",
|
261 |
+
"name": "python3"
|
262 |
+
},
|
263 |
+
"language_info": {
|
264 |
+
"codemirror_mode": {
|
265 |
+
"name": "ipython",
|
266 |
+
"version": 3
|
267 |
+
},
|
268 |
+
"file_extension": ".py",
|
269 |
+
"mimetype": "text/x-python",
|
270 |
+
"name": "python",
|
271 |
+
"nbconvert_exporter": "python",
|
272 |
+
"pygments_lexer": "ipython3",
|
273 |
+
"version": "3.8.10"
|
274 |
+
}
|
275 |
+
},
|
276 |
+
"nbformat": 4,
|
277 |
+
"nbformat_minor": 2
|
278 |
+
}
|
rx_rover_PPO.ipynb
ADDED
The diff for this file is too large to render.
See raw diff
|
|
rx_rover_Q_Learning.ipynb
ADDED
The diff for this file is too large to render.
See raw diff
|
|