Detailed view behaviour with missing data

Talks about metering
segaura
Posts: 9
Joined: Sat Feb 03, 2018 1:22 am

Detailed view behaviour with missing data

Post by segaura » Sat Mar 10, 2018 9:41 am

Hallo, I am using meterN with great pleasure, so thank you all, and jm especially.

I configured two meters, production (1) and consumption (2) but my pvi is still disconnected and doesn't log production data so every csv line results in something like this

23:10,,1047844

the issue wih the detailed.php page is that the color intended for the first meter is used for the second one (the color legend shows to colors and the chart for meter 2 use color intended for meter 1).
The same happens with three meter, if a "column" of data is missing, the next meter gets its previous meter's color.

Not a big problem for me and not statistically so common, but I thought of reporting it if someone is interested in knowing or fixing it.

jeanmarc
Posts: 1764
Joined: Thu Aug 29, 2013 7:16 am

Re: Detailed view behaviour with missing data

Post by jeanmarc » Sat Mar 10, 2018 10:01 am

Hi,
Did you use latest version ? i don't get the issue
Thanks

segaura
Posts: 9
Joined: Sat Feb 03, 2018 1:22 am

Re: Detailed view behaviour with missing data

Post by segaura » Sat Mar 10, 2018 11:51 am

Yes, I use 0.8.5.

Did you tried with a csv file like this

00:00,,1027553
00:05,,1027574
00:10,,1027584
00:15,,1027594
00:20,,1027614
00:25,,1027624
00:30,,1027634
00:35,,1027654
00:40,,1027663

completely laking the values of the production meter?

jeanmarc
Posts: 1764
Joined: Thu Aug 29, 2013 7:16 am

Re: Detailed view behaviour with missing data

Post by jeanmarc » Sat Mar 10, 2018 12:18 pm

yes, it's ok here. Can you PM me the url of your logger ?

segaura
Posts: 9
Joined: Sat Feb 03, 2018 1:22 am

Re: Detailed view behaviour with missing data

Post by segaura » Sat Mar 10, 2018 12:28 pm

You mean a url you can connect to?
...my raspi is not exposed out of the house lan, at least without my ssh tunnel

Anyway if this is not a common problem, I can live with it until I will connect my pvi, thank you

jeanmarc
Posts: 1764
Joined: Thu Aug 29, 2013 7:16 am

Re: Detailed view behaviour with missing data

Post by jeanmarc » Sat Mar 10, 2018 12:54 pm

Yes, maybe try this :
In chrome : Press F12, then check "network" tab go into detailled.
You see "programdetailed.php?date1=20180310.csv&check1=on&check2=off..." then check "response": You may see if color is right for the serie eg: "color":"#962629"

segaura
Posts: 9
Joined: Sat Feb 03, 2018 1:22 am

Re: Detailed view behaviour with missing data

Post by segaura » Sat Mar 10, 2018 1:04 pm

I did, the colors are right, but I find the data of the meter number 2 (Consumi) in the first meter's part of the JSON

{"data":[{"name":"Produzione","data":[[1520640300000,30],[1520640600000,30],[1520640900000,9],[1520641200000,0],[1520641500000,20],[1520641800000,11],[1520642100000,20],[1520642400000,10],[1520642700000,20],[1520643000000,20],[1520643300000,10],[1520643600000,19],
(many more rows...)
[1520685300000,170],[1520685600000,140],[1520685900000,130],[1520686200000,140],[1520686500000,60]],"yAxis":0,"type":"areaspline","color":"#279621","dashStyle":"Solid","val24":"(0 Wh)"},{"name":"Consumi","data":null,"yAxis":0,"type":"areaspline","color":"#962629","dashStyle":"Solid","val24":"(10.300 Wh)"}],"title":"Dettagli del 10\/03\/2018 "}

In the event of a possible misunderstanding, the symptom with two meter enabled in the chart is this: https://ibb.co/joPf3S ("Consumi" data points with "Production" color, green #279621)
While the correct chart is this: https://ibb.co/hCMhHn ("Consumi" data points with correct color, green #962629)

jeanmarc
Posts: 1764
Joined: Thu Aug 29, 2013 7:16 am

Re: Detailed view behaviour with missing data

Post by jeanmarc » Sat Mar 10, 2018 5:08 pm

ok, now i see it :) . try this as programs/programdetailed.php

Code: Select all

<?php
define('checkaccess', TRUE);
include('../config/config_main.php');
date_default_timezone_set('UTC');

if (isset($_COOKIE['user_lang'])) {
    $user_lang = $_COOKIE['user_lang'];
} else {
    $user_lang = 'English';
}
include('../languages/' . $user_lang . '.php');

if (!empty($_GET['date1'])) {
    $date1 = htmlspecialchars($_GET['date1'], ENT_QUOTES, 'UTF-8');
} else {
    $date1 = FALSE;
}
$grid = array();

for ($i = 1; $i <= $NUMMETER; $i++) {
    include("../config/config_met$i.php");
    $check[$i] = htmlspecialchars($_GET["check$i"], ENT_QUOTES, 'UTF-8');
}

if (file_exists("../data/csv/$date1")) {
    $file       = file("../data/csv/$date1");
    $contalines = count($file);
    
    $year  = substr($date1, 0, 4);
    $month = substr($date1, 4, 2);
    $day   = substr($date1, 6, 2);
    
    $unitlist = array();
    $metlist  = array();
    $nbrstack = 0;
    
    for ($i = 1; $i <= $NUMMETER; $i++) {
        $val_first24[$i] = null;
        $val_last24[$i]  = null;
        if ($check[$i] == 'on') {
            array_push($metlist, $i);
            $stackname[$nbrstack] = "${'METNAME'.$i}";
            $color[$nbrstack]     = "#${'COLOR'.$i}";
            if (!in_array(${'UNIT' . $i}, $unitlist)) {
                array_push($unitlist, ${'UNIT' . $i});
            }
            $nbrstack++;
        }
    }
    
    $stack = array(
        array()
    );
    
    for ($line_num = 2; $line_num < $contalines; $line_num++) {
        $pastline_num = $line_num - 1;
        $prevarray    = preg_split("/,/", $file[$pastline_num]);
        $array        = preg_split("/,/", $file[$line_num]);
        
        $prevhour    = substr($prevarray[0], 0, 2);
        $prevmin     = substr($prevarray[0], 3, 2);
        $prevUTCdate = strtotime($year . '-' . $month . '-' . $day . ' ' . $prevhour . ':' . $prevmin);
        $prevUTCdate *= 1000;
        
        $SDTE   = $array[0];
        $hour   = substr($SDTE, 0, 2);
        $minute = substr($SDTE, 3, 2);
        
        if ($line_num == ($contalines - 1) && $minute == '00' && $hour == '00') {
            $epochdate = strtotime(date("$year$month$day")) + 86400;
        } else {
            $epochdate = strtotime($year . '-' . $month . '-' . $day . ' ' . $hour . ':' . $minute);
        }
        $epochdate *= 1000;
        
        $nbrstack = 0;
        for ($i = 1; $i <= $NUMMETER; $i++) {
            if ($check[$i] == 'on') {
                $val_first = null;
                $val_last  = null;
                
                if (isset($array[$i])) {
                    $val_first = trim($prevarray[$i]);
                }
                if (isset($prevarray[$i])) {
                    $val_last = trim($array[$i]);
                }
                if (${'TYPE' . $i} != 'Sensor') { // Meter
                    if (empty($val_first24[$i]) && !empty($val_first)) {
                        $val_first24[$i] = $val_first;
                    }
                    if (!empty($val_last)) {
                        $val_last24[$i] = $val_last;
                    }
                }
                if (${'TYPE' . $i} != 'Sensor' && $line_num > 1 && $epochdate - $prevUTCdate == 300000) { // meter
                    if (!empty($val_first) && !empty($val_last)) {
                        settype($val_first, 'float');
                        settype($val_last, 'float');
                        if ($val_first <= $val_last) {
                            $val_last -= $val_first;
                        } else { // counter pass over
                            $val_last += ${'PASSO' . $i} - $val_first;
                        }
                        
                        $val_last = round($val_last, ${'PRECI' . $i});
                    } else {
                        $val_last = null;
                    }
                    $stack[$nbrstack][$line_num] = array(
                        $epochdate,
                        $val_last
                    );
                    $nbrstack++;
                } elseif (${'TYPE' . $i} == 'Sensor' && $line_num > 1 && !empty($val_last)) { // sensor
                    settype($val_last, 'float');
                    $val_last                    = round($val_last, ${'PRECI' . $i});
                    $stack[$nbrstack][$line_num] = array(
                        $epochdate,
                        $val_last
                    );
                    $nbrstack++;
                }
                
            }
        } // each meters
    } // End of foreach
    
    $dday = date($DATEFORMAT, mktime(0, 0, 0, $month, $day, $year));
    
    settype($titledate, "string");
    $title = "$lgDETAILEDOFTITLE $dday $titledate";
    
    $data    = array();
    $cntunit = count($unitlist);
    
    # // Return datas via json
    $j = 0;
    for ($i = 1; $i <= $NUMMETER; $i++) {
        if ($check[$i] == 'on') {
            settype($val_first24[$i], 'float');
            settype($val_last24[$i], 'float');
            
            if (${'TYPE' . $i} != 'Sensor') { // Meter
                if ($val_first24[$i] <= $val_last24[$i]) {
                    $val_last24[$i] -= $val_first24[$i];
                } else { // counter pass over
                    $val_last24[$i] += ${'PASSO' . $i} - $val_first24[$i];
                }
                $val_last24[$i] = number_format($val_last24[$i], ${'PRECI' . $i}, $DPOINT, $THSEP);
            }
            
            sort($stack[$j]);
            for ($h = 0; $h < $cntunit; $h++) {
                if ($unitlist[$h] == ${'UNIT' . $i}) {
                    $thisyaxis = $h;
                }
            }
            $data[$j] = array(
                'name' => $stackname[$j],
                'data' => $stack[$j],
                'yAxis' => $thisyaxis,
                'type' => 'areaspline',
                'color' => $color[$j],
                'dashStyle' => 'Solid',
                'val24' => "($val_last24[$i] ${'UNIT' . $i})"
            );
            $j++;
        }
    }
    
    $jsonreturn = array(
        'data' => $data,
        'title' => $title
    );
} else {
    $jsonreturn = array(
        'data' => null,
        'title' => 'No Data'
    );
}
header("Content-type: text/json");
echo json_encode($jsonreturn);
?>

segaura
Posts: 9
Joined: Sat Feb 03, 2018 1:22 am

Re: Detailed view behaviour with missing data

Post by segaura » Sat Mar 10, 2018 5:38 pm

Yeah, what a service!! It worked, thank you.
I've started looking at the code but you solved the issue faster :D
(17 years after last php job didn't helped)

By the way, I have two questions:
- do you have a github repository?
- do you ever considered creating a docker image to setup a raspberry quickly and in a manageable way?

jeanmarc
Posts: 1764
Joined: Thu Aug 29, 2013 7:16 am

Re: Detailed view behaviour with missing data

Post by jeanmarc » Sat Mar 10, 2018 6:22 pm

No github, i am rather skeptical of forks with small projects and i prefer to keep the hand on it.
I haven't time nor the will for rasp image. There is already one around search about flanesi (italian)

Post Reply

Who is online

Users browsing this forum: No registered users and 2 guests