8af4acd6dd8794626d526f7aa854e84eab6e7166
[talweg.git] / reports / report.ipynb
1 {
2 "cells": [
3 {
4 "cell_type": "markdown",
5 "metadata": {
6 "deletable": true,
7 "editable": true
8 },
9 "source": [
10 "\n",
11 "\n",
12 "<h2>Introduction</h2>\n",
13 "\n",
14 "J'ai fait quelques essais dans différentes configurations pour la méthode \"Neighbors\"\n",
15 "(la seule dont on a parlé) et sa variante récente appelée pour l'instant \"Neighbors2\",\n",
16 "avec simtype=\"mix\" : deux types de similarités prises en compte, puis multiplication des poids.\n",
17 "Pour Neighbors on prédit le saut (par la moyenne pondérée des sauts passés), et pour Neighbors2\n",
18 "on n'effectue aucun raccordement (prévision directe).\n",
19 "\n",
20 "J'ai systématiquement comparé à une approche naïve : la moyenne des lendemains des jours\n",
21 "\"similaires\" dans tout le passé, ainsi qu'à la persistence -- reproduisant le jour courant ou\n",
22 "allant chercher le futur similaire une semaine avant.\n",
23 "\n",
24 "Ensuite j'affiche les erreurs, quelques courbes prévues/mesurées, quelques filaments puis les\n",
25 "histogrammes de quelques poids. Concernant les graphes de filaments, la moitié gauche du graphe\n",
26 "correspond aux jours similaires au jour courant, tandis que la moitié droite affiche les\n",
27 "lendemains : ce sont donc les voisinages tels qu'utilisés dans l'algorithme.\n",
28 "\n"
29 ]
30 },
31 {
32 "cell_type": "code",
33 "execution_count": null,
34 "metadata": {
35 "collapsed": false,
36 "deletable": true,
37 "editable": true
38 },
39 "outputs": [],
40 "source": [
41 "library(talweg)\n",
42 "\n",
43 "P = 7 #instant de prévision\n",
44 "H = 17 #horizon (en heures)\n",
45 "\n",
46 "ts_data = read.csv(system.file(\"extdata\",\"pm10_mesures_H_loc_report.csv\",package=\"talweg\"))\n",
47 "exo_data = read.csv(system.file(\"extdata\",\"meteo_extra_noNAs.csv\",package=\"talweg\"))\n",
48 "# NOTE: 'GMT' because DST gaps are filled and multiple values merged in above dataset.\n",
49 "# Prediction from P+1 to P+H included.\n",
50 "data = getData(ts_data, exo_data, input_tz = \"GMT\", working_tz=\"GMT\", predict_at=P)\n",
51 "\n",
52 "indices_ch = seq(as.Date(\"2015-01-18\"),as.Date(\"2015-01-24\"),\"days\")\n",
53 "indices_ep = seq(as.Date(\"2015-03-15\"),as.Date(\"2015-03-21\"),\"days\")\n",
54 "indices_np = seq(as.Date(\"2015-04-26\"),as.Date(\"2015-05-02\"),\"days\")\n"
55 ]
56 },
57 {
58 "cell_type": "markdown",
59 "metadata": {
60 "deletable": true,
61 "editable": true
62 },
63 "source": [
64 "\n",
65 "\n",
66 "<h2 style=\"color:blue;font-size:2em\">Pollution par chauffage</h2>"
67 ]
68 },
69 {
70 "cell_type": "code",
71 "execution_count": null,
72 "metadata": {
73 "collapsed": false,
74 "deletable": true,
75 "editable": true
76 },
77 "outputs": [],
78 "source": [
79 "reload(\"../pkg\")\n",
80 "p_nn = computeForecast(data, indices_ch, \"Neighbors\", \"Neighbors\", horizon=H)\n",
81 "p_nn2 = computeForecast(data, indices_ch, \"Neighbors2\", \"Neighbors\", horizon=H)\n",
82 "p_az = computeForecast(data, indices_ch, \"Average\", \"Zero\", horizon=H)\n",
83 "p_pz = computeForecast(data, indices_ch, \"Persistence\", \"Zero\", horizon=H, same_day=TRUE)"
84 ]
85 },
86 {
87 "cell_type": "code",
88 "execution_count": null,
89 "metadata": {
90 "collapsed": false
91 },
92 "outputs": [],
93 "source": [
94 "p_nn2$getParams(5)$weights"
95 ]
96 },
97 {
98 "cell_type": "code",
99 "execution_count": null,
100 "metadata": {
101 "collapsed": false,
102 "deletable": true,
103 "editable": true
104 },
105 "outputs": [],
106 "source": [
107 "e_nn = computeError(data, p_nn, H)\n",
108 "e_nn2 = computeError(data, p_nn2, H)\n",
109 "e_az = computeError(data, p_az, H)\n",
110 "e_pz = computeError(data, p_pz, H)\n",
111 "options(repr.plot.width=9, repr.plot.height=7)\n",
112 "plotError(list(e_nn, e_pz, e_az, e_nn2), cols=c(1,2,colors()[258], 4))\n",
113 "\n",
114 "# Noir: Neighbors, bleu: Neighbors2, vert: moyenne, rouge: persistence\n",
115 "\n",
116 "i_np = which.min(e_nn$abs$indices)\n",
117 "i_p = which.max(e_nn$abs$indices)"
118 ]
119 },
120 {
121 "cell_type": "code",
122 "execution_count": null,
123 "metadata": {
124 "collapsed": false,
125 "deletable": true,
126 "editable": true
127 },
128 "outputs": [],
129 "source": [
130 "options(repr.plot.width=9, repr.plot.height=4)\n",
131 "par(mfrow=c(1,2))\n",
132 "\n",
133 "plotPredReal(data, p_nn, i_np); title(paste(\"PredReal nn day\",i_np))\n",
134 "plotPredReal(data, p_nn2, i_p); title(paste(\"PredReal nn day\",i_p))\n",
135 "\n",
136 "plotPredReal(data, p_nn2, i_np); title(paste(\"PredReal nn2 day\",i_np))\n",
137 "plotPredReal(data, p_nn2, i_p); title(paste(\"PredReal nn2 day\",i_p))\n",
138 "\n",
139 "plotPredReal(data, p_az, i_np); title(paste(\"PredReal az day\",i_np))\n",
140 "plotPredReal(data, p_az, i_p); title(paste(\"PredReal az day\",i_p))\n",
141 "\n",
142 "# Bleu: prévue, noir: réalisée"
143 ]
144 },
145 {
146 "cell_type": "code",
147 "execution_count": null,
148 "metadata": {
149 "collapsed": false,
150 "deletable": true,
151 "editable": true
152 },
153 "outputs": [],
154 "source": [
155 "par(mfrow=c(1,2))\n",
156 "f_np = computeFilaments(data, p_nn, i_np, plot=TRUE); title(paste(\"Filaments nn day\",i_np))\n",
157 "f_p = computeFilaments(data, p_nn, i_p, plot=TRUE); title(paste(\"Filaments nn day\",i_p))\n",
158 "\n",
159 "f_np2 = computeFilaments(data, p_nn2, i_np, plot=TRUE); title(paste(\"Filaments nn2 day\",i_np))\n",
160 "f_p2 = computeFilaments(data, p_nn2, i_p, plot=TRUE); title(paste(\"Filaments nn2 day\",i_p))"
161 ]
162 },
163 {
164 "cell_type": "code",
165 "execution_count": null,
166 "metadata": {
167 "collapsed": false,
168 "deletable": true,
169 "editable": true
170 },
171 "outputs": [],
172 "source": [
173 "par(mfrow=c(1,2))\n",
174 "plotFilamentsBox(data, f_np); title(paste(\"FilBox nn day\",i_np))\n",
175 "plotFilamentsBox(data, f_p); title(paste(\"FilBox nn day\",i_p))\n",
176 "\n",
177 "plotFilamentsBox(data, f_np2); title(paste(\"FilBox nn2 day\",i_np))\n",
178 "plotFilamentsBox(data, f_p2); title(paste(\"FilBox nn2 day\",i_p))"
179 ]
180 },
181 {
182 "cell_type": "code",
183 "execution_count": null,
184 "metadata": {
185 "collapsed": false,
186 "deletable": true,
187 "editable": true
188 },
189 "outputs": [],
190 "source": [
191 "par(mfrow=c(1,2))\n",
192 "plotRelVar(data, f_np); title(paste(\"StdDev nn day\",i_np))\n",
193 "plotRelVar(data, f_p); title(paste(\"StdDev nn day\",i_p))\n",
194 "\n",
195 "plotRelVar(data, f_np2); title(paste(\"StdDev nn2 day\",i_np))\n",
196 "plotRelVar(data, f_p2); title(paste(\"StdDev nn2 day\",i_p))\n",
197 "\n",
198 "# Variabilité globale en rouge ; sur les 60 voisins (+ lendemains) en noir"
199 ]
200 },
201 {
202 "cell_type": "code",
203 "execution_count": null,
204 "metadata": {
205 "collapsed": false,
206 "deletable": true,
207 "editable": true
208 },
209 "outputs": [],
210 "source": [
211 "par(mfrow=c(1,2))\n",
212 "plotSimils(p_nn, i_np); title(paste(\"Weights nn day\",i_np))\n",
213 "plotSimils(p_nn, i_p); title(paste(\"Weights nn day\",i_p))\n",
214 "\n",
215 "plotSimils(p_nn2, i_np); title(paste(\"Weights nn2 day\",i_np))\n",
216 "plotSimils(p_nn2, i_p); title(paste(\"Weights nn2 day\",i_p))\n",
217 "\n",
218 "# - pollué à gauche, + pollué à droite"
219 ]
220 },
221 {
222 "cell_type": "code",
223 "execution_count": null,
224 "metadata": {
225 "collapsed": false,
226 "deletable": true,
227 "editable": true
228 },
229 "outputs": [],
230 "source": [
231 "# Fenêtres sélectionnées dans ]0,7] / nn à gauche, nn2 à droite\n",
232 "p_nn$getParams(i_np)$window\n",
233 "p_nn$getParams(i_p)$window\n",
234 "\n",
235 "p_nn2$getParams(i_np)$window\n",
236 "p_nn2$getParams(i_p)$window"
237 ]
238 },
239 {
240 "cell_type": "markdown",
241 "metadata": {
242 "deletable": true,
243 "editable": true
244 },
245 "source": [
246 "\n",
247 "\n",
248 "<h2 style=\"color:blue;font-size:2em\">Pollution par épandage</h2>"
249 ]
250 },
251 {
252 "cell_type": "code",
253 "execution_count": null,
254 "metadata": {
255 "collapsed": false,
256 "deletable": true,
257 "editable": true
258 },
259 "outputs": [],
260 "source": [
261 "p_nn = computeForecast(data, indices_ep, \"Neighbors\", \"Neighbors\", horizon=H)\n",
262 "p_nn2 = computeForecast(data, indices_ep, \"Neighbors2\", \"Zero\", horizon=H)\n",
263 "p_az = computeForecast(data, indices_ep, \"Average\", \"Zero\", horizon=H)\n",
264 "p_pz = computeForecast(data, indices_ep, \"Persistence\", \"Zero\", horizon=H, same_day=TRUE)"
265 ]
266 },
267 {
268 "cell_type": "code",
269 "execution_count": null,
270 "metadata": {
271 "collapsed": false,
272 "deletable": true,
273 "editable": true
274 },
275 "outputs": [],
276 "source": [
277 "e_nn = computeError(data, p_nn, H)\n",
278 "e_nn2 = computeError(data, p_nn2, H)\n",
279 "e_az = computeError(data, p_az, H)\n",
280 "e_pz = computeError(data, p_pz, H)\n",
281 "options(repr.plot.width=9, repr.plot.height=7)\n",
282 "plotError(list(e_nn, e_pz, e_az, e_nn2), cols=c(1,2,colors()[258], 4))\n",
283 "\n",
284 "# Noir: Neighbors, bleu: Neighbors2, vert: moyenne, rouge: persistence\n",
285 "\n",
286 "i_np = which.min(e_nn$abs$indices)\n",
287 "i_p = which.max(e_nn$abs$indices)"
288 ]
289 },
290 {
291 "cell_type": "code",
292 "execution_count": null,
293 "metadata": {
294 "collapsed": false,
295 "deletable": true,
296 "editable": true
297 },
298 "outputs": [],
299 "source": [
300 "options(repr.plot.width=9, repr.plot.height=4)\n",
301 "par(mfrow=c(1,2))\n",
302 "\n",
303 "plotPredReal(data, p_nn, i_np); title(paste(\"PredReal nn day\",i_np))\n",
304 "plotPredReal(data, p_nn2, i_p); title(paste(\"PredReal nn day\",i_p))\n",
305 "\n",
306 "plotPredReal(data, p_nn2, i_np); title(paste(\"PredReal nn2 day\",i_np))\n",
307 "plotPredReal(data, p_nn2, i_p); title(paste(\"PredReal nn2 day\",i_p))\n",
308 "\n",
309 "plotPredReal(data, p_az, i_np); title(paste(\"PredReal az day\",i_np))\n",
310 "plotPredReal(data, p_az, i_p); title(paste(\"PredReal az day\",i_p))\n",
311 "\n",
312 "# Bleu: prévue, noir: réalisée"
313 ]
314 },
315 {
316 "cell_type": "code",
317 "execution_count": null,
318 "metadata": {
319 "collapsed": false,
320 "deletable": true,
321 "editable": true
322 },
323 "outputs": [],
324 "source": [
325 "par(mfrow=c(1,2))\n",
326 "f_np = computeFilaments(data, p_nn, i_np, plot=TRUE); title(paste(\"Filaments nn day\",i_np))\n",
327 "f_p = computeFilaments(data, p_nn, i_p, plot=TRUE); title(paste(\"Filaments nn day\",i_p))\n",
328 "\n",
329 "f_np2 = computeFilaments(data, p_nn2, i_np, plot=TRUE); title(paste(\"Filaments nn2 day\",i_np))\n",
330 "f_p2 = computeFilaments(data, p_nn2, i_p, plot=TRUE); title(paste(\"Filaments nn2 day\",i_p))"
331 ]
332 },
333 {
334 "cell_type": "code",
335 "execution_count": null,
336 "metadata": {
337 "collapsed": false,
338 "deletable": true,
339 "editable": true
340 },
341 "outputs": [],
342 "source": [
343 "par(mfrow=c(1,2))\n",
344 "plotFilamentsBox(data, f_np); title(paste(\"FilBox nn day\",i_np))\n",
345 "plotFilamentsBox(data, f_p); title(paste(\"FilBox nn day\",i_p))\n",
346 "\n",
347 "plotFilamentsBox(data, f_np2); title(paste(\"FilBox nn2 day\",i_np))\n",
348 "plotFilamentsBox(data, f_p2); title(paste(\"FilBox nn2 day\",i_p))"
349 ]
350 },
351 {
352 "cell_type": "code",
353 "execution_count": null,
354 "metadata": {
355 "collapsed": false,
356 "deletable": true,
357 "editable": true
358 },
359 "outputs": [],
360 "source": [
361 "par(mfrow=c(1,2))\n",
362 "plotRelVar(data, f_np); title(paste(\"StdDev nn day\",i_np))\n",
363 "plotRelVar(data, f_p); title(paste(\"StdDev nn day\",i_p))\n",
364 "\n",
365 "plotRelVar(data, f_np2); title(paste(\"StdDev nn2 day\",i_np))\n",
366 "plotRelVar(data, f_p2); title(paste(\"StdDev nn2 day\",i_p))\n",
367 "\n",
368 "# Variabilité globale en rouge ; sur les 60 voisins (+ lendemains) en noir"
369 ]
370 },
371 {
372 "cell_type": "code",
373 "execution_count": null,
374 "metadata": {
375 "collapsed": false,
376 "deletable": true,
377 "editable": true
378 },
379 "outputs": [],
380 "source": [
381 "par(mfrow=c(1,2))\n",
382 "plotSimils(p_nn, i_np); title(paste(\"Weights nn day\",i_np))\n",
383 "plotSimils(p_nn, i_p); title(paste(\"Weights nn day\",i_p))\n",
384 "\n",
385 "plotSimils(p_nn2, i_np); title(paste(\"Weights nn2 day\",i_np))\n",
386 "plotSimils(p_nn2, i_p); title(paste(\"Weights nn2 day\",i_p))\n",
387 "\n",
388 "# - pollué à gauche, + pollué à droite"
389 ]
390 },
391 {
392 "cell_type": "code",
393 "execution_count": null,
394 "metadata": {
395 "collapsed": false,
396 "deletable": true,
397 "editable": true
398 },
399 "outputs": [],
400 "source": [
401 "# Fenêtres sélectionnées dans ]0,7] / nn à gauche, nn2 à droite\n",
402 "p_nn$getParams(i_np)$window\n",
403 "p_nn$getParams(i_p)$window\n",
404 "\n",
405 "p_nn2$getParams(i_np)$window\n",
406 "p_nn2$getParams(i_p)$window"
407 ]
408 },
409 {
410 "cell_type": "markdown",
411 "metadata": {
412 "deletable": true,
413 "editable": true
414 },
415 "source": [
416 "\n",
417 "\n",
418 "<h2 style=\"color:blue;font-size:2em\">Semaine non polluée</h2>"
419 ]
420 },
421 {
422 "cell_type": "code",
423 "execution_count": null,
424 "metadata": {
425 "collapsed": false,
426 "deletable": true,
427 "editable": true
428 },
429 "outputs": [],
430 "source": [
431 "p_nn = computeForecast(data, indices_np, \"Neighbors\", \"Neighbors\", horizon=H)\n",
432 "p_nn2 = computeForecast(data, indices_np, \"Neighbors2\", \"Zero\", horizon=H)\n",
433 "p_az = computeForecast(data, indices_np, \"Average\", \"Zero\", horizon=H)\n",
434 "p_pz = computeForecast(data, indices_np, \"Persistence\", \"Zero\", horizon=H, same_day=FALSE)"
435 ]
436 },
437 {
438 "cell_type": "code",
439 "execution_count": null,
440 "metadata": {
441 "collapsed": false,
442 "deletable": true,
443 "editable": true
444 },
445 "outputs": [],
446 "source": [
447 "e_nn = computeError(data, p_nn, H)\n",
448 "e_nn2 = computeError(data, p_nn2, H)\n",
449 "e_az = computeError(data, p_az, H)\n",
450 "e_pz = computeError(data, p_pz, H)\n",
451 "options(repr.plot.width=9, repr.plot.height=7)\n",
452 "plotError(list(e_nn, e_pz, e_az, e_nn2), cols=c(1,2,colors()[258], 4))\n",
453 "\n",
454 "# Noir: Neighbors, bleu: Neighbors2, vert: moyenne, rouge: persistence\n",
455 "\n",
456 "i_np = which.min(e_nn$abs$indices)\n",
457 "i_p = which.max(e_nn$abs$indices)"
458 ]
459 },
460 {
461 "cell_type": "code",
462 "execution_count": null,
463 "metadata": {
464 "collapsed": false,
465 "deletable": true,
466 "editable": true
467 },
468 "outputs": [],
469 "source": [
470 "options(repr.plot.width=9, repr.plot.height=4)\n",
471 "par(mfrow=c(1,2))\n",
472 "\n",
473 "plotPredReal(data, p_nn, i_np); title(paste(\"PredReal nn day\",i_np))\n",
474 "plotPredReal(data, p_nn2, i_p); title(paste(\"PredReal nn day\",i_p))\n",
475 "\n",
476 "plotPredReal(data, p_nn2, i_np); title(paste(\"PredReal nn2 day\",i_np))\n",
477 "plotPredReal(data, p_nn2, i_p); title(paste(\"PredReal nn2 day\",i_p))\n",
478 "\n",
479 "plotPredReal(data, p_az, i_np); title(paste(\"PredReal az day\",i_np))\n",
480 "plotPredReal(data, p_az, i_p); title(paste(\"PredReal az day\",i_p))\n",
481 "\n",
482 "# Bleu: prévue, noir: réalisée"
483 ]
484 },
485 {
486 "cell_type": "code",
487 "execution_count": null,
488 "metadata": {
489 "collapsed": false,
490 "deletable": true,
491 "editable": true
492 },
493 "outputs": [],
494 "source": [
495 "par(mfrow=c(1,2))\n",
496 "f_np = computeFilaments(data, p_nn, i_np, plot=TRUE); title(paste(\"Filaments nn day\",i_np))\n",
497 "f_p = computeFilaments(data, p_nn, i_p, plot=TRUE); title(paste(\"Filaments nn day\",i_p))\n",
498 "\n",
499 "f_np2 = computeFilaments(data, p_nn2, i_np, plot=TRUE); title(paste(\"Filaments nn2 day\",i_np))\n",
500 "f_p2 = computeFilaments(data, p_nn2, i_p, plot=TRUE); title(paste(\"Filaments nn2 day\",i_p))"
501 ]
502 },
503 {
504 "cell_type": "code",
505 "execution_count": null,
506 "metadata": {
507 "collapsed": false,
508 "deletable": true,
509 "editable": true
510 },
511 "outputs": [],
512 "source": [
513 "par(mfrow=c(1,2))\n",
514 "plotFilamentsBox(data, f_np); title(paste(\"FilBox nn day\",i_np))\n",
515 "plotFilamentsBox(data, f_p); title(paste(\"FilBox nn day\",i_p))\n",
516 "\n",
517 "plotFilamentsBox(data, f_np2); title(paste(\"FilBox nn2 day\",i_np))\n",
518 "plotFilamentsBox(data, f_p2); title(paste(\"FilBox nn2 day\",i_p))"
519 ]
520 },
521 {
522 "cell_type": "code",
523 "execution_count": null,
524 "metadata": {
525 "collapsed": false,
526 "deletable": true,
527 "editable": true
528 },
529 "outputs": [],
530 "source": [
531 "par(mfrow=c(1,2))\n",
532 "plotRelVar(data, f_np); title(paste(\"StdDev nn day\",i_np))\n",
533 "plotRelVar(data, f_p); title(paste(\"StdDev nn day\",i_p))\n",
534 "\n",
535 "plotRelVar(data, f_np2); title(paste(\"StdDev nn2 day\",i_np))\n",
536 "plotRelVar(data, f_p2); title(paste(\"StdDev nn2 day\",i_p))\n",
537 "\n",
538 "# Variabilité globale en rouge ; sur les 60 voisins (+ lendemains) en noir"
539 ]
540 },
541 {
542 "cell_type": "code",
543 "execution_count": null,
544 "metadata": {
545 "collapsed": false,
546 "deletable": true,
547 "editable": true
548 },
549 "outputs": [],
550 "source": [
551 "par(mfrow=c(1,2))\n",
552 "plotSimils(p_nn, i_np); title(paste(\"Weights nn day\",i_np))\n",
553 "plotSimils(p_nn, i_p); title(paste(\"Weights nn day\",i_p))\n",
554 "\n",
555 "plotSimils(p_nn2, i_np); title(paste(\"Weights nn2 day\",i_np))\n",
556 "plotSimils(p_nn2, i_p); title(paste(\"Weights nn2 day\",i_p))\n",
557 "\n",
558 "# - pollué à gauche, + pollué à droite"
559 ]
560 },
561 {
562 "cell_type": "code",
563 "execution_count": null,
564 "metadata": {
565 "collapsed": false,
566 "deletable": true,
567 "editable": true
568 },
569 "outputs": [],
570 "source": [
571 "# Fenêtres sélectionnées dans ]0,7] / nn à gauche, nn2 à droite\n",
572 "p_nn$getParams(i_np)$window\n",
573 "p_nn$getParams(i_p)$window\n",
574 "\n",
575 "p_nn2$getParams(i_np)$window\n",
576 "p_nn2$getParams(i_p)$window"
577 ]
578 }
579 ],
580 "metadata": {
581 "kernelspec": {
582 "display_name": "R",
583 "language": "R",
584 "name": "ir"
585 },
586 "language_info": {
587 "codemirror_mode": "r",
588 "file_extension": ".r",
589 "mimetype": "text/x-r-source",
590 "name": "R",
591 "pygments_lexer": "r",
592 "version": "3.3.3"
593 }
594 },
595 "nbformat": 4,
596 "nbformat_minor": 2
597 }