},
"outputs": [],
"source": [
- "data = getData(ts_data=\"../pkg/data/pm10_mesures_H_loc.csv\", exo_data=\"../pkg/data/meteo_extra_noNAs.csv\",\n",
- " input_tz = \"Europe/Paris\", working_tz=\"Europe/Paris\", predict_at=7)"
+ "ts_data = read.csv(system.file(\"extdata\",\"pm10_mesures_H_loc.csv\",package=\"talweg\"))\n",
+ "exo_data = read.csv(system.file(\"extdata\",\"meteo_extra_noNAs.csv\",package=\"talweg\"))\n",
+ "data = getData(ts_data, exo_data, input_tz = \"Europe/Paris\", working_tz=\"Europe/Paris\", predict_at=7)"
]
},
{
" * same_season=FALSE : les indices pour la validation croisée ne tiennent pas compte des saisons\n",
" * mix_strategy=\"mult\" : on multiplie les poids (au lieu d'en éteindre)\n",
"\n",
+ "(valeurs par défaut).\n",
+ "\n",
"J'ai systématiquement comparé à deux autres approches : la persistence et la moyennes des lendemains des jours \"similaires\" dans tout le passé ; à chaque fois sans prédiction du saut (sauf pour Neighbors : prédiction basée sur les poids calculés).\n",
"\n",
"Ensuite j'affiche les erreurs, quelques courbes prévues/mesurées, quelques filaments puis les histogrammes de quelques poids. Concernant les graphes de filaments, la moitié gauche du graphe correspond aux jours similaires au jour courant, tandis que la moitié droite affiche les lendemains : ce sont donc les voisinages tels qu'utilisés dans l'algorithme.\n",
},
"outputs": [],
"source": [
- "indices = seq(as.Date(\"2015-01-18\"),as.Date(\"2015-01-24\"),\"days\")\n",
- "p_ch_nn = getForecast(data,indices,\"Neighbors\",\"Neighbors\",simtype=\"mix\",same_season=FALSE,mix_strategy=\"mult\")\n",
- "p_ch_pz = getForecast(data, indices, \"Persistence\", \"Zero\", same_day=TRUE)\n",
- "p_ch_az = getForecast(data, indices, \"Average\", \"Zero\") #, memory=183)\n",
- "#p_ch_zz = getForecast(data, indices, \"Zero\", \"Zero\")"
+ "indices_ch = seq(as.Date(\"2015-01-18\"),as.Date(\"2015-01-24\"),\"days\")\n",
+ "p_ch_nn = computeForecast(data,indices_ch, \"Neighbors\", \"Neighbors\", simtype=\"mix\")\n",
+ "p_ch_pz = computeForecast(data, indices_ch, \"Persistence\", \"Zero\", same_day=TRUE)\n",
+ "p_ch_az = computeForecast(data, indices_ch, \"Average\", \"Zero\") #, memory=183)\n",
+ "#p_ch_zz = computeForecast(data, indices_ch, \"Zero\", \"Zero\")"
]
},
{
},
"outputs": [],
"source": [
- "e_ch_nn = getError(data, p_ch_nn)\n",
- "e_ch_pz = getError(data, p_ch_pz)\n",
- "e_ch_az = getError(data, p_ch_az)\n",
- "#e_ch_zz = getError(data, p_ch_zz)\n",
+ "e_ch_nn = computeError(data, p_ch_nn)\n",
+ "e_ch_pz = computeError(data, p_ch_pz)\n",
+ "e_ch_az = computeError(data, p_ch_az)\n",
+ "#e_ch_zz = computeError(data, p_ch_zz)\n",
"options(repr.plot.width=9, repr.plot.height=7)\n",
"plotError(list(e_ch_nn, e_ch_pz, e_ch_az), cols=c(1,2,colors()[258]))\n",
"\n",
"#Noir: neighbors, rouge: persistence, vert: moyenne"
]
},
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "La méthode Neighbors fait assez nettement mieux que les autres dans ce cas."
- ]
- },
{
"cell_type": "code",
"execution_count": null,
"plotPredReal(data, p_ch_az, 4)"
]
},
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "Les erreurs sont proches, mais les courbes prédites très différentes : avantage à \"Neighbors\""
- ]
- },
{
"cell_type": "code",
"execution_count": null,
"outputs": [],
"source": [
"par(mfrow=c(1,2))\n",
- "f3 = computeFilaments(data, p_ch_nn$getIndexInData(3), plot=TRUE)\n",
- "f4 = computeFilaments(data, p_ch_nn$getIndexInData(4), plot=TRUE)"
+ "f3_ch = computeFilaments(data, p_ch_nn$getIndexInData(3), plot=TRUE)\n",
+ "f4_ch = computeFilaments(data, p_ch_nn$getIndexInData(4), plot=TRUE)"
]
},
{
"source": [
"par(mfrow=c(2,2))\n",
"options(repr.plot.width=9, repr.plot.height=7)\n",
- "plotFilamentsBox(data, f3$indices)\n",
- "plotFilamentsBox(data, f3$indices+1)\n",
- "plotFilamentsBox(data, f4$indices)\n",
- "plotFilamentsBox(data, f4$indices+1)\n",
+ "plotFilamentsBox(data, f3_ch$indices)\n",
+ "plotFilamentsBox(data, f3_ch$indices+1)\n",
+ "plotFilamentsBox(data, f4_ch$indices)\n",
+ "plotFilamentsBox(data, f4_ch$indices+1)\n",
"\n",
"#En haut : jour 3 + lendemain (4) ; en bas : jour 4 + lendemain (5)\n",
"#À gauche : premières 24h ; à droite : 24h suivantes"
"source": [
"par(mfrow=c(1,2))\n",
"options(repr.plot.width=9, repr.plot.height=4)\n",
- "plotRelativeVariability(data, f3$indices)\n",
- "plotRelativeVariability(data, f4$indices)\n",
+ "plotRelativeVariability(data, f3_ch$indices)\n",
+ "plotRelativeVariability(data, f4_ch$indices)\n",
"\n",
"#Variabilité sur 60 courbes au hasard en rouge ; sur nos 60 voisins (+ lendemains) en noir"
]
"cell_type": "markdown",
"metadata": {},
"source": [
- "Dans la situation idéale, il faudrait que la courbe noire soit nettement plus basse que la courbe rouge. Ce n'est pas très clair à J+1, surtout en soirée (de 21h à 0h)."
+ "Il faudrait que la courbe noire soit nettement plus basse que la courbe rouge."
]
},
{
"ignored <- computeFilaments(data, p_ch_nn$getIndexInData(5), plot=TRUE)"
]
},
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [],
+ "source": [
+ "#Fenêtres sélectionnées dans ]0,10] / endo à gauche, exo à droite\n",
+ "p_ch_nn$getParams(3)$window\n",
+ "p_ch_nn$getParams(4)$window"
+ ]
+ },
{
"cell_type": "markdown",
"metadata": {},
},
"outputs": [],
"source": [
- "indices = seq(as.Date(\"2015-03-15\"),as.Date(\"2015-03-21\"),\"days\")\n",
- "p_ep_nn = getForecast(data,indices,\"Neighbors\",\"Neighbors\",simtype=\"mix\",same_season=FALSE,mix_strategy=\"mult\")\n",
- "p_ep_pz = getForecast(data, indices, \"Persistence\", \"Zero\", same_day=TRUE)\n",
- "p_ep_az = getForecast(data, indices, \"Average\", \"Zero\") #, memory=183)\n",
- "#p_ep_zz = getForecast(data, indices, \"Zero\", \"Zero\")"
+ "indices_ep = seq(as.Date(\"2015-03-15\"),as.Date(\"2015-03-21\"),\"days\")\n",
+ "p_ep_nn = computeForecast(data,indices_ep, \"Neighbors\", \"Neighbors\", simtype=\"mix\")\n",
+ "p_ep_pz = computeForecast(data, indices_ep, \"Persistence\", \"Zero\", same_day=TRUE)\n",
+ "p_ep_az = computeForecast(data, indices_ep, \"Average\", \"Zero\") #, memory=183)\n",
+ "#p_ep_zz = computeForecast(data, indices_ep, \"Zero\", \"Zero\")"
]
},
{
},
"outputs": [],
"source": [
- "e_ep_nn = getError(data, p_ep_nn)\n",
- "e_ep_pz = getError(data, p_ep_pz)\n",
- "e_ep_az = getError(data, p_ep_az)\n",
- "#e_ep_zz = getError(data, p_ep_zz)\n",
- "#e_ep_l = getError(data, p_ep_l)\n",
+ "e_ep_nn = computeError(data, p_ep_nn)\n",
+ "e_ep_pz = computeError(data, p_ep_pz)\n",
+ "e_ep_az = computeError(data, p_ep_az)\n",
+ "#e_ep_zz = computeError(data, p_ep_zz)\n",
"options(repr.plot.width=9, repr.plot.height=7)\n",
"plotError(list(e_ep_nn, e_ep_pz, e_ep_az), cols=c(1,2,colors()[258]))\n",
"\n",
"#Noir: neighbors, rouge: persistence, vert: moyenne"
]
},
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "Cette fois les deux méthodes naïves font en moyenne moins d'erreurs que Neighbors. Prédiction trop difficile ?"
- ]
- },
{
"cell_type": "code",
"execution_count": null,
"source": [
"par(mfrow=c(1,2))\n",
"options(repr.plot.width=9, repr.plot.height=4)\n",
- "plotPredReal(data, p_ep_nn, 5)\n",
- "plotPredReal(data, p_ep_nn, 2)\n",
+ "plotPredReal(data, p_ep_nn, 4)\n",
+ "plotPredReal(data, p_ep_nn, 6)\n",
"\n",
"#Bleu: prévue, noir: réalisée"
]
"outputs": [],
"source": [
"par(mfrow=c(1,2))\n",
- "plotPredReal(data, p_ep_pz, 4)\n",
- "plotPredReal(data, p_ep_pz, 6)"
+ "plotPredReal(data, p_ep_az, 4)\n",
+ "plotPredReal(data, p_ep_az, 6)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [],
+ "source": [
+ "par(mfrow=c(1,2))\n",
+ "f4_ep = computeFilaments(data, p_ep_nn$getIndexInData(4), plot=TRUE)\n",
+ "f6_ep = computeFilaments(data, p_ep_nn$getIndexInData(6), plot=TRUE)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [],
+ "source": [
+ "par(mfrow=c(2,2))\n",
+ "options(repr.plot.width=9, repr.plot.height=7)\n",
+ "plotFilamentsBox(data, f4_ep$indices)\n",
+ "plotFilamentsBox(data, f4_ep$indices+1)\n",
+ "plotFilamentsBox(data, f6_ep$indices)\n",
+ "plotFilamentsBox(data, f6_ep$indices+1)\n",
+ "\n",
+ "#En haut : jour 4 + lendemain (5) ; en bas : jour 6 + lendemain (7)\n",
+ "#À gauche : premières 24h ; à droite : 24h suivantes"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
- "Bonnes performances de la persistence (par chance...)"
+ "\"Voisinages\" catastrophiques : les jours 4 et 6 sont trop atypiques."
]
},
{
"outputs": [],
"source": [
"par(mfrow=c(1,2))\n",
- "plotFilaments(data, p_ep_nn$getIndexInData(5))\n",
- "plotFilaments(data, p_ep_nn$getIndexInData(2))"
+ "options(repr.plot.width=9, repr.plot.height=4)\n",
+ "plotRelativeVariability(data, f4_ep$indices)\n",
+ "plotRelativeVariability(data, f6_ep$indices)\n",
+ "\n",
+ "#Variabilité sur 60 courbes au hasard en rouge ; sur nos 60 voisins (+ lendemains) en noir"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Il faudrait que la courbe noire soit nettement plus basse que la courbe rouge."
]
},
{
"outputs": [],
"source": [
"par(mfrow=c(1,2))\n",
- "plotSimils(p_ep_nn, 5)\n",
- "plotSimils(p_ep_nn, 2)"
+ "plotSimils(p_ep_nn, 4)\n",
+ "plotSimils(p_ep_nn, 6)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
- "Même observation concernant les poids : concentrés près de zéro pour les prédictions avec peu de voisins."
+ "Poids très concentrés près de zéro pour les prédictions avec peu de voisins."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [],
+ "source": [
+ "#Fenêtres sélectionnées dans ]0,10] / endo à gauche, exo à droite\n",
+ "p_ep_nn$getParams(4)$window\n",
+ "p_ep_nn$getParams(6)$window"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
- "## Semaine non polluée"
+ "<h2 style=\"color:blue;font-size:2em\">Semaine non polluée</h2>"
]
},
{
},
"outputs": [],
"source": [
- "indices = seq(as.Date(\"2015-04-26\"),as.Date(\"2015-05-02\"),\"days\")\n",
- "p_np_nn = getForecast(data,indices,\"Neighbors\",\"Neighbors\",simtype=\"mix\",same_season=FALSE,mix_strategy=\"mult\")\n",
- "p_np_pz = getForecast(data, indices, \"Persistence\", \"Zero\", same_day=FALSE)\n",
- "p_np_az = getForecast(data, indices, \"Average\", \"Zero\") #, memory=183)\n",
- "#p_np_zz = getForecast(data, indices, \"Zero\", \"Zero\")\n",
- "#p_np_l = getForecast(data, indices, \"Level\", same_day=FALSE)"
+ "indices_np = seq(as.Date(\"2015-04-26\"),as.Date(\"2015-05-02\"),\"days\")\n",
+ "p_np_nn = computeForecast(data,indices_np, \"Neighbors\", \"Neighbors\", simtype=\"mix\")\n",
+ "p_np_pz = computeForecast(data, indices_np, \"Persistence\", \"Zero\", same_day=FALSE)\n",
+ "p_np_az = computeForecast(data, indices_np, \"Average\", \"Zero\") #, memory=183)\n",
+ "#p_np_zz = computeForecast(data, indices_np, \"Zero\", \"Zero\")"
]
},
{
},
"outputs": [],
"source": [
- "e_np_nn = getError(data, p_np_nn)\n",
- "e_np_pz = getError(data, p_np_pz)\n",
- "e_np_az = getError(data, p_np_az)\n",
- "#e_np_zz = getError(data, p_np_zz)\n",
- "#e_np_l = getError(data, p_np_l)\n",
- "options(repr.plot.width=9, repr.plot.height=6)\n",
+ "e_np_nn = computeError(data, p_np_nn)\n",
+ "e_np_pz = computeError(data, p_np_pz)\n",
+ "e_np_az = computeError(data, p_np_az)\n",
+ "#e_np_zz = computeError(data, p_np_zz)\n",
+ "options(repr.plot.width=9, repr.plot.height=7)\n",
"plotError(list(e_np_nn, e_np_pz, e_np_az), cols=c(1,2,colors()[258]))\n",
"\n",
"#Noir: neighbors, rouge: persistence, vert: moyenne"
"source": [
"par(mfrow=c(1,2))\n",
"options(repr.plot.width=9, repr.plot.height=4)\n",
- "plotPredReal(data, p_np_nn, 3)\n",
+ "plotPredReal(data, p_np_nn, 5)\n",
"plotPredReal(data, p_np_nn, 6)\n",
"\n",
"#Bleu: prévue, noir: réalisée"
]
},
{
- "cell_type": "markdown",
- "metadata": {},
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [],
"source": [
- "Les \"bonnes\" prédictions (à gauche) sont tout de même trop lissées."
+ "par(mfrow=c(1,2))\n",
+ "plotPredReal(data, p_np_az, 5)\n",
+ "plotPredReal(data, p_np_az, 6)"
]
},
{
"outputs": [],
"source": [
"par(mfrow=c(1,2))\n",
- "plotPredReal(data, p_np_az, 3)\n",
- "plotPredReal(data, p_np_az, 6)"
+ "f5_np = computeFilaments(data, p_np_nn$getIndexInData(5), plot=TRUE)\n",
+ "f6_np = computeFilaments(data, p_np_nn$getIndexInData(6), plot=TRUE)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [],
+ "source": [
+ "par(mfrow=c(2,2))\n",
+ "options(repr.plot.width=9, repr.plot.height=7)\n",
+ "plotFilamentsBox(data, f5_np$indices)\n",
+ "plotFilamentsBox(data, f5_np$indices+1)\n",
+ "plotFilamentsBox(data, f6_np$indices)\n",
+ "plotFilamentsBox(data, f6_np$indices+1)\n",
+ "\n",
+ "#En haut : jour 3 + lendemain (4) ; en bas : jour 6 + lendemain (7)\n",
+ "#À gauche : premières 24h ; à droite : 24h suivantes"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
- "Légèrement meilleur ajustement par la méthode \"Average\" ; très net à droite."
+ "Jours \"typiques\", donc beaucoup de voisins. En revanche les lendemains des jours similaires sont très variables."
]
},
{
"outputs": [],
"source": [
"par(mfrow=c(1,2))\n",
- "plotFilaments(data, p_np_nn$getIndexInData(3))\n",
- "plotFilaments(data, p_np_nn$getIndexInData(6))"
+ "options(repr.plot.width=9, repr.plot.height=4)\n",
+ "plotRelativeVariability(data, f5_np$indices)\n",
+ "plotRelativeVariability(data, f6_np$indices)\n",
+ "\n",
+ "#Variabilité sur 60 courbes au hasard en rouge ; sur nos 60 voisins (+ lendemains) en noir"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
- "Jours \"typiques\", donc beaucoup de voisins."
+ "Bonne situation : la courbe noire est toujours assez nettement en dessous."
]
},
{
},
"outputs": [],
"source": [
- "par(mfrow=c(1,3))\n",
- "plotSimils(p_np_nn, 3)\n",
- "plotSimils(p_np_nn, 4)\n",
+ "par(mfrow=c(1,2))\n",
+ "plotSimils(p_np_nn, 5)\n",
"plotSimils(p_np_nn, 6)"
]
},
"Répartition idéale des poids : quelques uns au-delà de 0.3-0.4, le reste très proche de zéro."
]
},
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [],
+ "source": [
+ "#Fenêtres sélectionnées dans ]0,10] / endo à gauche, exo à droite\n",
+ "p_np_nn$getParams(5)$window\n",
+ "p_np_nn$getParams(6)$window"
+ ]
+ },
{
"cell_type": "markdown",
"metadata": {},
"mimetype": "text/x-r-source",
"name": "R",
"pygments_lexer": "r",
- "version": "3.3.2"
+ "version": "3.2.3"
}
},
"nbformat": 4,
projects / talweg.git / blobdiff