fix window bounds
[talweg.git] / reports / report.ipynb
1 {
2 "cells": [
3 {
4 "cell_type": "markdown",
5 "metadata": {
6 "deletable": true,
7 "editable": true
8 },
9 "source": [
10 "\n",
11 "\n",
12 "<h2>Introduction</h2>\n",
13 "\n",
14 "J'ai fait quelques essais dans différentes configurations pour la méthode \"Neighbors\"\n",
15 "(la seule dont on a parlé) et sa variante récente appelée pour l'instant \"Neighbors2\",\n",
16 "avec simtype=\"mix\" : deux types de similarités prises en compte, puis multiplication des poids.\n",
17 "Pour Neighbors on prédit le saut (par la moyenne pondérée des sauts passés), et pour Neighbors2\n",
18 "on n'effectue aucun raccordement (prévision directe).\n",
19 "\n",
20 "J'ai systématiquement comparé à une approche naïve : la moyenne des lendemains des jours\n",
21 "\"similaires\" dans tout le passé, ainsi qu'à la persistence -- reproduisant le jour courant ou\n",
22 "allant chercher le futur similaire une semaine avant.\n",
23 "\n",
24 "Ensuite j'affiche les erreurs, quelques courbes prévues/mesurées, quelques filaments puis les\n",
25 "histogrammes de quelques poids. Concernant les graphes de filaments, la moitié gauche du graphe\n",
26 "correspond aux jours similaires au jour courant, tandis que la moitié droite affiche les\n",
27 "lendemains : ce sont donc les voisinages tels qu'utilisés dans l'algorithme.\n",
28 "\n"
29 ]
30 },
31 {
32 "cell_type": "code",
33 "execution_count": null,
34 "metadata": {
35 "collapsed": false,
36 "deletable": true,
37 "editable": true
38 },
39 "outputs": [],
40 "source": [
41 "library(talweg)\n",
42 "\n",
43 "P = 7 #instant de prévision\n",
44 "H = 17 #horizon (en heures)\n",
45 "\n",
46 "ts_data = read.csv(system.file(\"extdata\",\"pm10_mesures_H_loc_report.csv\",package=\"talweg\"))\n",
47 "exo_data = read.csv(system.file(\"extdata\",\"meteo_extra_noNAs.csv\",package=\"talweg\"))\n",
48 "# NOTE: 'GMT' because DST gaps are filled and multiple values merged in above dataset.\n",
49 "# Prediction from P+1 to P+H included.\n",
50 "data = getData(ts_data, exo_data, input_tz = \"GMT\", working_tz=\"GMT\", predict_at=P)\n",
51 "\n",
52 "indices_ch = seq(as.Date(\"2015-01-18\"),as.Date(\"2015-01-24\"),\"days\")\n",
53 "indices_ep = seq(as.Date(\"2015-03-15\"),as.Date(\"2015-03-21\"),\"days\")\n",
54 "indices_np = seq(as.Date(\"2015-04-26\"),as.Date(\"2015-05-02\"),\"days\")\n"
55 ]
56 },
57 {
58 "cell_type": "markdown",
59 "metadata": {
60 "deletable": true,
61 "editable": true
62 },
63 "source": [
64 "\n",
65 "\n",
66 "<h2 style=\"color:blue;font-size:2em\">Pollution par chauffage</h2>"
67 ]
68 },
69 {
70 "cell_type": "code",
71 "execution_count": null,
72 "metadata": {
73 "collapsed": false,
74 "deletable": true,
75 "editable": true
76 },
77 "outputs": [],
78 "source": [
79 "reload(\"../pkg\")\n",
80 "p_nn = computeForecast(data, indices_ch, \"Neighbors\", \"Neighbors\", horizon=H)\n",
81 "p_nn2 = computeForecast(data, indices_ch, \"Neighbors2\", \"Neighbors\", horizon=H)\n",
82 "p_az = computeForecast(data, indices_ch, \"Average\", \"Zero\", horizon=H)\n",
83 "p_pz = computeForecast(data, indices_ch, \"Persistence\", \"Zero\", horizon=H, same_day=TRUE)"
84 ]
85 },
86 {
87 "cell_type": "code",
88 "execution_count": null,
89 "metadata": {
90 "collapsed": false,
91 "deletable": true,
92 "editable": true
93 },
94 "outputs": [],
95 "source": [
96 "p_nn2$getParams(5)$weights"
97 ]
98 },
99 {
100 "cell_type": "code",
101 "execution_count": null,
102 "metadata": {
103 "collapsed": false,
104 "deletable": true,
105 "editable": true
106 },
107 "outputs": [],
108 "source": [
109 "e_nn = computeError(data, p_nn, H)\n",
110 "e_nn2 = computeError(data, p_nn2, H)\n",
111 "e_az = computeError(data, p_az, H)\n",
112 "e_pz = computeError(data, p_pz, H)\n",
113 "options(repr.plot.width=9, repr.plot.height=7)\n",
114 "plotError(list(e_nn, e_pz, e_az, e_nn2), cols=c(1,2,colors()[258], 4))\n",
115 "\n",
116 "# Noir: Neighbors, bleu: Neighbors2, vert: moyenne, rouge: persistence\n",
117 "\n",
118 "i_np = which.min(e_nn$abs$indices)\n",
119 "i_p = which.max(e_nn$abs$indices)"
120 ]
121 },
122 {
123 "cell_type": "code",
124 "execution_count": null,
125 "metadata": {
126 "collapsed": false,
127 "deletable": true,
128 "editable": true
129 },
130 "outputs": [],
131 "source": [
132 "options(repr.plot.width=9, repr.plot.height=4)\n",
133 "par(mfrow=c(1,2))\n",
134 "\n",
135 "plotPredReal(data, p_nn, i_np); title(paste(\"PredReal nn day\",i_np))\n",
136 "plotPredReal(data, p_nn2, i_p); title(paste(\"PredReal nn day\",i_p))\n",
137 "\n",
138 "plotPredReal(data, p_nn2, i_np); title(paste(\"PredReal nn2 day\",i_np))\n",
139 "plotPredReal(data, p_nn2, i_p); title(paste(\"PredReal nn2 day\",i_p))\n",
140 "\n",
141 "plotPredReal(data, p_az, i_np); title(paste(\"PredReal az day\",i_np))\n",
142 "plotPredReal(data, p_az, i_p); title(paste(\"PredReal az day\",i_p))\n",
143 "\n",
144 "# Bleu: prévue, noir: réalisée"
145 ]
146 },
147 {
148 "cell_type": "code",
149 "execution_count": null,
150 "metadata": {
151 "collapsed": false,
152 "deletable": true,
153 "editable": true
154 },
155 "outputs": [],
156 "source": [
157 "par(mfrow=c(1,2))\n",
158 "f_np = computeFilaments(data, p_nn, i_np, plot=TRUE); title(paste(\"Filaments nn day\",i_np))\n",
159 "f_p = computeFilaments(data, p_nn, i_p, plot=TRUE); title(paste(\"Filaments nn day\",i_p))\n",
160 "\n",
161 "f_np2 = computeFilaments(data, p_nn2, i_np, plot=TRUE); title(paste(\"Filaments nn2 day\",i_np))\n",
162 "f_p2 = computeFilaments(data, p_nn2, i_p, plot=TRUE); title(paste(\"Filaments nn2 day\",i_p))"
163 ]
164 },
165 {
166 "cell_type": "code",
167 "execution_count": null,
168 "metadata": {
169 "collapsed": false,
170 "deletable": true,
171 "editable": true
172 },
173 "outputs": [],
174 "source": [
175 "par(mfrow=c(1,2))\n",
176 "#plotFilamentsBox(data, f_np); title(paste(\"FilBox nn day\",i_np))\n",
177 "#plotFilamentsBox(data, f_p); title(paste(\"FilBox nn day\",i_p))\n",
178 "\n",
179 "plotFilamentsBox(data, f_np2); title(paste(\"FilBox nn2 day\",i_np))\n",
180 "#plotFilamentsBox(data, f_p2); title(paste(\"FilBox nn2 day\",i_p))"
181 ]
182 },
183 {
184 "cell_type": "code",
185 "execution_count": null,
186 "metadata": {
187 "collapsed": false
188 },
189 "outputs": [],
190 "source": [
191 "f_np2"
192 ]
193 },
194 {
195 "cell_type": "code",
196 "execution_count": null,
197 "metadata": {
198 "collapsed": false,
199 "deletable": true,
200 "editable": true
201 },
202 "outputs": [],
203 "source": [
204 "par(mfrow=c(1,2))\n",
205 "plotRelVar(data, f_np); title(paste(\"StdDev nn day\",i_np))\n",
206 "plotRelVar(data, f_p); title(paste(\"StdDev nn day\",i_p))\n",
207 "\n",
208 "plotRelVar(data, f_np2); title(paste(\"StdDev nn2 day\",i_np))\n",
209 "plotRelVar(data, f_p2); title(paste(\"StdDev nn2 day\",i_p))\n",
210 "\n",
211 "# Variabilité globale en rouge ; sur les 60 voisins (+ lendemains) en noir"
212 ]
213 },
214 {
215 "cell_type": "code",
216 "execution_count": null,
217 "metadata": {
218 "collapsed": false,
219 "deletable": true,
220 "editable": true
221 },
222 "outputs": [],
223 "source": [
224 "par(mfrow=c(1,2))\n",
225 "plotSimils(p_nn, i_np); title(paste(\"Weights nn day\",i_np))\n",
226 "plotSimils(p_nn, i_p); title(paste(\"Weights nn day\",i_p))\n",
227 "\n",
228 "plotSimils(p_nn2, i_np); title(paste(\"Weights nn2 day\",i_np))\n",
229 "plotSimils(p_nn2, i_p); title(paste(\"Weights nn2 day\",i_p))\n",
230 "\n",
231 "# - pollué à gauche, + pollué à droite"
232 ]
233 },
234 {
235 "cell_type": "code",
236 "execution_count": null,
237 "metadata": {
238 "collapsed": false,
239 "deletable": true,
240 "editable": true
241 },
242 "outputs": [],
243 "source": [
244 "# Fenêtres sélectionnées dans ]0,7] / nn à gauche, nn2 à droite\n",
245 "p_nn$getParams(i_np)$window\n",
246 "p_nn$getParams(i_p)$window\n",
247 "\n",
248 "p_nn2$getParams(i_np)$window\n",
249 "p_nn2$getParams(i_p)$window"
250 ]
251 },
252 {
253 "cell_type": "markdown",
254 "metadata": {
255 "deletable": true,
256 "editable": true
257 },
258 "source": [
259 "\n",
260 "\n",
261 "<h2 style=\"color:blue;font-size:2em\">Pollution par épandage</h2>"
262 ]
263 },
264 {
265 "cell_type": "code",
266 "execution_count": null,
267 "metadata": {
268 "collapsed": false,
269 "deletable": true,
270 "editable": true
271 },
272 "outputs": [],
273 "source": [
274 "p_nn = computeForecast(data, indices_ep, \"Neighbors\", \"Neighbors\", horizon=H)\n",
275 "p_nn2 = computeForecast(data, indices_ep, \"Neighbors2\", \"Zero\", horizon=H)\n",
276 "p_az = computeForecast(data, indices_ep, \"Average\", \"Zero\", horizon=H)\n",
277 "p_pz = computeForecast(data, indices_ep, \"Persistence\", \"Zero\", horizon=H, same_day=TRUE)"
278 ]
279 },
280 {
281 "cell_type": "code",
282 "execution_count": null,
283 "metadata": {
284 "collapsed": false,
285 "deletable": true,
286 "editable": true
287 },
288 "outputs": [],
289 "source": [
290 "e_nn = computeError(data, p_nn, H)\n",
291 "e_nn2 = computeError(data, p_nn2, H)\n",
292 "e_az = computeError(data, p_az, H)\n",
293 "e_pz = computeError(data, p_pz, H)\n",
294 "options(repr.plot.width=9, repr.plot.height=7)\n",
295 "plotError(list(e_nn, e_pz, e_az, e_nn2), cols=c(1,2,colors()[258], 4))\n",
296 "\n",
297 "# Noir: Neighbors, bleu: Neighbors2, vert: moyenne, rouge: persistence\n",
298 "\n",
299 "i_np = which.min(e_nn$abs$indices)\n",
300 "i_p = which.max(e_nn$abs$indices)"
301 ]
302 },
303 {
304 "cell_type": "code",
305 "execution_count": null,
306 "metadata": {
307 "collapsed": false,
308 "deletable": true,
309 "editable": true
310 },
311 "outputs": [],
312 "source": [
313 "options(repr.plot.width=9, repr.plot.height=4)\n",
314 "par(mfrow=c(1,2))\n",
315 "\n",
316 "plotPredReal(data, p_nn, i_np); title(paste(\"PredReal nn day\",i_np))\n",
317 "plotPredReal(data, p_nn2, i_p); title(paste(\"PredReal nn day\",i_p))\n",
318 "\n",
319 "plotPredReal(data, p_nn2, i_np); title(paste(\"PredReal nn2 day\",i_np))\n",
320 "plotPredReal(data, p_nn2, i_p); title(paste(\"PredReal nn2 day\",i_p))\n",
321 "\n",
322 "plotPredReal(data, p_az, i_np); title(paste(\"PredReal az day\",i_np))\n",
323 "plotPredReal(data, p_az, i_p); title(paste(\"PredReal az day\",i_p))\n",
324 "\n",
325 "# Bleu: prévue, noir: réalisée"
326 ]
327 },
328 {
329 "cell_type": "code",
330 "execution_count": null,
331 "metadata": {
332 "collapsed": false,
333 "deletable": true,
334 "editable": true
335 },
336 "outputs": [],
337 "source": [
338 "par(mfrow=c(1,2))\n",
339 "f_np = computeFilaments(data, p_nn, i_np, plot=TRUE); title(paste(\"Filaments nn day\",i_np))\n",
340 "f_p = computeFilaments(data, p_nn, i_p, plot=TRUE); title(paste(\"Filaments nn day\",i_p))\n",
341 "\n",
342 "f_np2 = computeFilaments(data, p_nn2, i_np, plot=TRUE); title(paste(\"Filaments nn2 day\",i_np))\n",
343 "f_p2 = computeFilaments(data, p_nn2, i_p, plot=TRUE); title(paste(\"Filaments nn2 day\",i_p))"
344 ]
345 },
346 {
347 "cell_type": "code",
348 "execution_count": null,
349 "metadata": {
350 "collapsed": false,
351 "deletable": true,
352 "editable": true
353 },
354 "outputs": [],
355 "source": [
356 "par(mfrow=c(1,2))\n",
357 "plotFilamentsBox(data, f_np); title(paste(\"FilBox nn day\",i_np))\n",
358 "plotFilamentsBox(data, f_p); title(paste(\"FilBox nn day\",i_p))\n",
359 "\n",
360 "plotFilamentsBox(data, f_np2); title(paste(\"FilBox nn2 day\",i_np))\n",
361 "plotFilamentsBox(data, f_p2); title(paste(\"FilBox nn2 day\",i_p))"
362 ]
363 },
364 {
365 "cell_type": "code",
366 "execution_count": null,
367 "metadata": {
368 "collapsed": false,
369 "deletable": true,
370 "editable": true
371 },
372 "outputs": [],
373 "source": [
374 "par(mfrow=c(1,2))\n",
375 "plotRelVar(data, f_np); title(paste(\"StdDev nn day\",i_np))\n",
376 "plotRelVar(data, f_p); title(paste(\"StdDev nn day\",i_p))\n",
377 "\n",
378 "plotRelVar(data, f_np2); title(paste(\"StdDev nn2 day\",i_np))\n",
379 "plotRelVar(data, f_p2); title(paste(\"StdDev nn2 day\",i_p))\n",
380 "\n",
381 "# Variabilité globale en rouge ; sur les 60 voisins (+ lendemains) en noir"
382 ]
383 },
384 {
385 "cell_type": "code",
386 "execution_count": null,
387 "metadata": {
388 "collapsed": false,
389 "deletable": true,
390 "editable": true
391 },
392 "outputs": [],
393 "source": [
394 "par(mfrow=c(1,2))\n",
395 "plotSimils(p_nn, i_np); title(paste(\"Weights nn day\",i_np))\n",
396 "plotSimils(p_nn, i_p); title(paste(\"Weights nn day\",i_p))\n",
397 "\n",
398 "plotSimils(p_nn2, i_np); title(paste(\"Weights nn2 day\",i_np))\n",
399 "plotSimils(p_nn2, i_p); title(paste(\"Weights nn2 day\",i_p))\n",
400 "\n",
401 "# - pollué à gauche, + pollué à droite"
402 ]
403 },
404 {
405 "cell_type": "code",
406 "execution_count": null,
407 "metadata": {
408 "collapsed": false,
409 "deletable": true,
410 "editable": true
411 },
412 "outputs": [],
413 "source": [
414 "# Fenêtres sélectionnées dans ]0,7] / nn à gauche, nn2 à droite\n",
415 "p_nn$getParams(i_np)$window\n",
416 "p_nn$getParams(i_p)$window\n",
417 "\n",
418 "p_nn2$getParams(i_np)$window\n",
419 "p_nn2$getParams(i_p)$window"
420 ]
421 },
422 {
423 "cell_type": "markdown",
424 "metadata": {
425 "deletable": true,
426 "editable": true
427 },
428 "source": [
429 "\n",
430 "\n",
431 "<h2 style=\"color:blue;font-size:2em\">Semaine non polluée</h2>"
432 ]
433 },
434 {
435 "cell_type": "code",
436 "execution_count": null,
437 "metadata": {
438 "collapsed": false,
439 "deletable": true,
440 "editable": true
441 },
442 "outputs": [],
443 "source": [
444 "p_nn = computeForecast(data, indices_np, \"Neighbors\", \"Neighbors\", horizon=H)\n",
445 "p_nn2 = computeForecast(data, indices_np, \"Neighbors2\", \"Zero\", horizon=H)\n",
446 "p_az = computeForecast(data, indices_np, \"Average\", \"Zero\", horizon=H)\n",
447 "p_pz = computeForecast(data, indices_np, \"Persistence\", \"Zero\", horizon=H, same_day=FALSE)"
448 ]
449 },
450 {
451 "cell_type": "code",
452 "execution_count": null,
453 "metadata": {
454 "collapsed": false,
455 "deletable": true,
456 "editable": true
457 },
458 "outputs": [],
459 "source": [
460 "e_nn = computeError(data, p_nn, H)\n",
461 "e_nn2 = computeError(data, p_nn2, H)\n",
462 "e_az = computeError(data, p_az, H)\n",
463 "e_pz = computeError(data, p_pz, H)\n",
464 "options(repr.plot.width=9, repr.plot.height=7)\n",
465 "plotError(list(e_nn, e_pz, e_az, e_nn2), cols=c(1,2,colors()[258], 4))\n",
466 "\n",
467 "# Noir: Neighbors, bleu: Neighbors2, vert: moyenne, rouge: persistence\n",
468 "\n",
469 "i_np = which.min(e_nn$abs$indices)\n",
470 "i_p = which.max(e_nn$abs$indices)"
471 ]
472 },
473 {
474 "cell_type": "code",
475 "execution_count": null,
476 "metadata": {
477 "collapsed": false,
478 "deletable": true,
479 "editable": true
480 },
481 "outputs": [],
482 "source": [
483 "options(repr.plot.width=9, repr.plot.height=4)\n",
484 "par(mfrow=c(1,2))\n",
485 "\n",
486 "plotPredReal(data, p_nn, i_np); title(paste(\"PredReal nn day\",i_np))\n",
487 "plotPredReal(data, p_nn2, i_p); title(paste(\"PredReal nn day\",i_p))\n",
488 "\n",
489 "plotPredReal(data, p_nn2, i_np); title(paste(\"PredReal nn2 day\",i_np))\n",
490 "plotPredReal(data, p_nn2, i_p); title(paste(\"PredReal nn2 day\",i_p))\n",
491 "\n",
492 "plotPredReal(data, p_az, i_np); title(paste(\"PredReal az day\",i_np))\n",
493 "plotPredReal(data, p_az, i_p); title(paste(\"PredReal az day\",i_p))\n",
494 "\n",
495 "# Bleu: prévue, noir: réalisée"
496 ]
497 },
498 {
499 "cell_type": "code",
500 "execution_count": null,
501 "metadata": {
502 "collapsed": false,
503 "deletable": true,
504 "editable": true
505 },
506 "outputs": [],
507 "source": [
508 "par(mfrow=c(1,2))\n",
509 "f_np = computeFilaments(data, p_nn, i_np, plot=TRUE); title(paste(\"Filaments nn day\",i_np))\n",
510 "f_p = computeFilaments(data, p_nn, i_p, plot=TRUE); title(paste(\"Filaments nn day\",i_p))\n",
511 "\n",
512 "f_np2 = computeFilaments(data, p_nn2, i_np, plot=TRUE); title(paste(\"Filaments nn2 day\",i_np))\n",
513 "f_p2 = computeFilaments(data, p_nn2, i_p, plot=TRUE); title(paste(\"Filaments nn2 day\",i_p))"
514 ]
515 },
516 {
517 "cell_type": "code",
518 "execution_count": null,
519 "metadata": {
520 "collapsed": false,
521 "deletable": true,
522 "editable": true
523 },
524 "outputs": [],
525 "source": [
526 "par(mfrow=c(1,2))\n",
527 "plotFilamentsBox(data, f_np); title(paste(\"FilBox nn day\",i_np))\n",
528 "plotFilamentsBox(data, f_p); title(paste(\"FilBox nn day\",i_p))\n",
529 "\n",
530 "plotFilamentsBox(data, f_np2); title(paste(\"FilBox nn2 day\",i_np))\n",
531 "plotFilamentsBox(data, f_p2); title(paste(\"FilBox nn2 day\",i_p))"
532 ]
533 },
534 {
535 "cell_type": "code",
536 "execution_count": null,
537 "metadata": {
538 "collapsed": false,
539 "deletable": true,
540 "editable": true
541 },
542 "outputs": [],
543 "source": [
544 "par(mfrow=c(1,2))\n",
545 "plotRelVar(data, f_np); title(paste(\"StdDev nn day\",i_np))\n",
546 "plotRelVar(data, f_p); title(paste(\"StdDev nn day\",i_p))\n",
547 "\n",
548 "plotRelVar(data, f_np2); title(paste(\"StdDev nn2 day\",i_np))\n",
549 "plotRelVar(data, f_p2); title(paste(\"StdDev nn2 day\",i_p))\n",
550 "\n",
551 "# Variabilité globale en rouge ; sur les 60 voisins (+ lendemains) en noir"
552 ]
553 },
554 {
555 "cell_type": "code",
556 "execution_count": null,
557 "metadata": {
558 "collapsed": false,
559 "deletable": true,
560 "editable": true
561 },
562 "outputs": [],
563 "source": [
564 "par(mfrow=c(1,2))\n",
565 "plotSimils(p_nn, i_np); title(paste(\"Weights nn day\",i_np))\n",
566 "plotSimils(p_nn, i_p); title(paste(\"Weights nn day\",i_p))\n",
567 "\n",
568 "plotSimils(p_nn2, i_np); title(paste(\"Weights nn2 day\",i_np))\n",
569 "plotSimils(p_nn2, i_p); title(paste(\"Weights nn2 day\",i_p))\n",
570 "\n",
571 "# - pollué à gauche, + pollué à droite"
572 ]
573 },
574 {
575 "cell_type": "code",
576 "execution_count": null,
577 "metadata": {
578 "collapsed": false,
579 "deletable": true,
580 "editable": true
581 },
582 "outputs": [],
583 "source": [
584 "# Fenêtres sélectionnées dans ]0,7] / nn à gauche, nn2 à droite\n",
585 "p_nn$getParams(i_np)$window\n",
586 "p_nn$getParams(i_p)$window\n",
587 "\n",
588 "p_nn2$getParams(i_np)$window\n",
589 "p_nn2$getParams(i_p)$window"
590 ]
591 }
592 ],
593 "metadata": {
594 "kernelspec": {
595 "display_name": "R",
596 "language": "R",
597 "name": "ir"
598 },
599 "language_info": {
600 "codemirror_mode": "r",
601 "file_extension": ".r",
602 "mimetype": "text/x-r-source",
603 "name": "R",
604 "pygments_lexer": "r",
605 "version": "3.3.3"
606 }
607 },
608 "nbformat": 4,
609 "nbformat_minor": 2
610 }