Peaks in power inputs when production sample is missing

[polvere79]

123s get datas directly from your inverter and write 123s's CSV and JSON files in memory.
mN get datas using various com apps. To read 123s there is pool123s.php that read 123s JSON file, simple as that
It also mean that if you reboot your PC on night the 123s JSON dosen't exist.

Ok, so metern uses pool123s.php (which is a comapp, right?) to get data from 123s JSON file, but then its metern.php writing data in metern csv files?

Actually I don't restart raspy at night...
I noticed that the same happens if I stop 123solar, while metern is still active. I got empty production in matern csv, and peaks of exported power...

Sorry for my many questions, just trying to figure out how the whole thing works (reverse engineering on code it's quite tricky )

[jeanmarc]

Ok, so metern uses pool123s.php (which is a comapp, right?) to get data from 123s JSON file, but then its metern.php writing data in metern csv files?

Yup and eflow read mN's CSV to make calculation. I don't know why i write that crap around 5 years ago

[polvere79]

Ok, so metern uses pool123s.php (which is a comapp, right?) to get data from 123s JSON file, but then its metern.php writing data in metern csv files?

Yup and eflow read mN's CSV to make calculation. I don't know why i write that crap around 5 years ago

So it's also metern.php making calculation for writing the increase in values in the csv?

Now I even have an additional issue:

PHP Notice: A non well formed numeric value encountered in /var/www/comapps/pool123s.php on line 59

which is the following:
$KWHT = round(($array_last[27] * ${'CORRECTFACTOR' . $invtnum} * 1000), 0); //in Wh

This produces a similar notice
PHP Notice: A non well formed numeric value encountered in /var/www/comapps/eflow.php on line 159
and it prevents self consumption not be shown in graphics...

I also have a lot of the following:

Code: Select all

19/11/2020 17:25:09	Connection with #4 Immissioni restored

19/11/2020 17:25:09	Connection with #3 Prelievi restored

19/11/2020 17:25:09	Connection with #5 Autoconsumo restored

19/11/2020 17:25:09	Connection lost with #5 Autoconsumo, missing 5' sample

19/11/2020 17:25:06	Connection lost with #4 Immissioni, missing 5' sample

19/11/2020 17:25:03	Connection lost with #3 Prelievi, missing 5' sample

19/11/2020 17:20:10	Connection with #5 Autoconsumo restored

19/11/2020 17:20:10	Connection with #4 Immissioni restored

19/11/2020 17:20:10	Connection with #3 Prelievi restored

19/11/2020 17:20:10	Connection lost with #5 Autoconsumo, missing 5' sample

19/11/2020 17:20:07	Connection lost with #4 Immissioni, missing 5' sample

19/11/2020 17:20:03	Connection lost with #3 Prelievi, missing 5' sample

I'm getting crazy here...

[jeanmarc]

No, mN just read data from your com. apps, it dosen't make any calculation.
You don't use latest pool123s.php, i guess a value is empty or not numeric (maybe thought it's text).

You could help your debug by outputting values in a file or with echo

Code: Select all

..
$log       = 'mylog.txt';
$stamp = date('d/m/Y H:i:s');
$data = "$stamp : $KWHT $whatever";
echo $data;
file_put_contents($log, $data, FILE_APPEND);

have fun while coding

[polvere79]

No, mN just read data from your com. apps, it dosen't make any calculation.
You don't use latest pool123s.php, i guess a value is empty or not numeric (maybe thought it's text).

You could help your debug by outputting values in a file or with echo

Code: Select all

..
$log       = 'mylog.txt';
$stamp = date('d/m/Y H:i:s');
$data = "$stamp : $KWHT $whatever";
echo $data;
file_put_contents($log, $data, FILE_APPEND);

have fun while coding

So...metern-php uses pool123s gets data from 123solar and writes 123s_LIVEMEMORY (where there are live values from the inverter) and 123s_MEMORY (where you have total energy?? what's 123s_MEMORY for?). These files are used by comapps (like eflow as of now for me) to make calculations. Then, such calculations are written in mN_LIVEMEMROY and mN_MEMORY for metern to get them and write them in CSV, correct? So if there is any strange value in metern csv it's comapp fault, right?
And dashboard page retrieves data from CSV or Json for daily and total values? As of now, all graphs are ok, but dashboard show me -- as total exported and self consumption.

I guess it's because of "First" value in arrays, which are 0 for those 2 meters:

Code: Select all

Memory (/dev/shm/mN_MEMORY.json) :
Array ( [msgflag1] => [First1] => 2523940 [Last1] => 2527200 [msgflag2] => [First2] => 309836 [Last2] => 320764 [msgflag3] => [First3] => 5009 [Last3] => 12592 [msgflag4] => [First4] => 0 [Last4] => 17667495 [msgflag5] => [First5] => 0 [Last5] => 3345 [5minflag] => 1 )
Live (/dev/shm/mN_LIVEMEMORY.json) :
Array ( [UTC] => 1605885912 [Produzione1] => 70 [Consumi2] => 441 [Prelievi3] => 371 [Immissioni4] => 0 [Autoconsumo5] => 70 )
Indicators (/dev/shm/mN_ILIVEMEMORY.json) :
Array ( [Tensione1] => 237.20 [Corrente2] => 2.17 [Cos φ3] => 0.91 [Temp. CPU4] => 59.4 ) 

These values in mN_MEMORY are writtem by comapps (eflow), but I can't understand why they are 0...
metern.err shows this:

Code: Select all

PHP Notice:  A non well formed numeric value encountered in /var/www/comapps/eflow.php on line 159
PHP Notice:  Trying to access array offset on value of type null in /var/www/comapps/pool123s.php on line 36
PHP Notice:  A non well formed numeric value encountered in /var/www/comapps/eflow.php on line 159

I'll try and add the log you suggested...pool123s is a version modified by Flane...

I'm loosing my eyes on this...but now it's a matter of principle

[jeanmarc]

123s write 123s_LIVEMEMORY and mN use pool123s to read it.

You can see value in MEMORY via admin/help.php page. Both mN and 123s LIVEMEMORY are tracking live things (latests samples), _MEMORY is used for various needs i don't have the time to explain the "nuts and bolts". I believe my code is simple to understand

One interesting thing is that mN_MEMORY contains the firsts and lasts meters values of the day during the 5" sample. Anyone can do calculation on them to make a virtual meter, making differences or sums.

Dashboard show values from mN_MEMORY. Graphs are rendered reading CSV. Hope it's clear..

[polvere79]

123s write 123s_LIVEMEMORY and mN use pool123s to read it.

Ok, here is what I discovered...peaks in graps are happens systematically if I stop 123solar. This way, 123s_LIVEMEMORY.json is not created, and the first IF clause in pooler123

Code: Select all

if (file_exists($LIVEMEMORY))

goes directly to

Code: Select all

die("Abording: Empty SHM\n");

This way, when meterN calls pooler123 for production value, it gets an invalid return. It retries 3 times, but eventually it sets production to null. This null value goes to meterN csv, and then it's used by eflow to make the math, leading to a 0 value as prod_val_first, instead of the actual previous production value, and therefore all the math is screwed up for exported power.

123s_LIVEMEMORY happens not to be present also at inverter boot in the morning, and sometimes (it didn't happen in the last 2 days) at inverter stop when the sun sets.

So...how do I fix this? It doesn't look like it's an issue with eflow, looks more like an issue with 123solar and the communication with the inverter (ABB UNO).
If I can't fix the communication, pool123s should go and get the value the latest value of production in csv, intea of aborting directly...

What do you think?

Thanks

[jeanmarc]

Yup that should be your problem. pool123s return Empty SHM but that's ain't a problem with mN, it try 3 times hence the "Missing sample" and it leave an empty row in the csv.
There won't be any graphical glicth in your production with a missing data.

The real problem is eflow mess up with that. You can either change pool123 to return latest prod using csv or better get ride of this bad script and rewrite it
Again, take a look on houseenergy and how it deal with that

Code: Select all

..
            if (isset($production)) {
..
            } else { // no production case
                $diffPROD = 0;
..
            }
..
            if ($difference < 0) { // Might happen if the production return no difference while the export did !
                $difference = 0;
            }
..            

houseenergy read latest data in SHM memory not in CSV. It just takes the latest CSV in case the virtual meters have no previous data yet.

Maybe look on misc examples pooltot.php, it's the same principle but easier to understand. It make one virtual meter from several production or consumption meters.
Set verbose mode and try in a terminal, you'll see how it handle data. But it check if it's a production or consumption with meter type ($WHICHTYPE)

[polvere79]

Ok, while I start having a look to houseenergy, i made a quick fix:

Code: Select all

if ($prod_val_first <= $prod_val_last) 
				{
                    if (!is_null($prod_val_first) && !empty($prod_val_first) && $prod_val_first!=0) 
					{
						$prod_val_last -= $prod_val_first;
						//debug
						if ($DEBUG_davide)
						{		
							$log   = '/var/www/metern/eflow.log';
							$stamp = date('d/m/Y H:i:s');
							$data = "$stamp : (111) prod_val_last -= prod_val_first $prod_val_last\n";
							file_put_contents($log, $data, FILE_APPEND);
						}
						//
					}
					else 
					{
						$prod_val_last = 0;
					}
                } else { // counter pass over
                    $prod_val_last += ${'PASSO' . $prodmet} - $prod_val_first;
                }
                settype($housep, 'float');
                $housep += $prod_val_last; //PRODUZIONE

This way, if in csv there is 0 in production, daily production is not the total production value given by the inverter at the next 5 minute sample.
It seems to be working...

Only thing, which I don't know whether I broke myself with teh above fix, is that now the last 2 meters (exported power and self consumption) value in dashbord show --Wh for daily and total. Maybe this is linked to the fact that I set to 0 all values of previous days using refine index?

I noticed that now "first" value in mN_MEMORY is 0 for those values, but I'm not sure if they are going to be correctly set later (next day?)

[jeanmarc]

I belive it's because "First" value are made at midnight, if there is no value you'll get '--'.
Maybe i could change that behavior that would make sense