odysseaskr / neural-disaggregator Goto Github PK

Hi @OdysseasKr ,
To pickle the model, should I pickle the all disaggregator? Thanks

Hello!

Thank you sir for your great work. could you tell me please how can i call the nilmtk and RNNDisaggregator.

Hello!
I am attempting to use your implementation of the WindowGRU algorithm to train across multiple houses.
The following is how the code is structured,

I am having issues with the test section and the disaggregate method, and am not sure how to proceed.
I am working on python 3.7 on Mac OS X.

Thank you!

Hello：
I‘m confused about the sample_period, I set sample_period=1 for the dish washer but get bad results, the predict result can hardly match the actual data.May I ask about the sample_period for dish washer and microwave and how you choose the value of them.Thanks a lot!

when i run the command:
dae = DAEDisaggregator(256)
i get the error:
OSError: pydot failed to call GraphViz.Please install GraphViz (https://www.graphviz.org/) and ensure that its executables are in the $PATH
but i already have pydot and graphviz installed. please help, thanks in advance!

@OdysseasKr @ChristoferNal @Spatzi

I am trying to execute your code ukdale-test.py. I also tried to execute ipynb file "RNN-example.ipynb". All went well but when I try to call the function "metrics.recall_precision_accuracy_f1(predicted, ground_truth)" to check the performance of the model, it gives me type error like this: " TypeError: Only valid with DatetimeIndex, TimedeltaIndex or PeriodIndex, but got an instance of 'Index'
". I am unable to resolve this error. Something is wrong with "predicted" file (which holds the data of building-2 [dish washer]that was generated after testing the model in HDF5 format.

Please help me out in this matter.

hello !
i don't know how to install the requirements !
PS : I already have a working nilmtk environment

Why am I wrong here?

in DAE-example.ipynb the F1 score should be called with rpaf[3]? It should be .45701927 for the example given.

import metrics
rpaf = metrics.recall_precision_accuracy_f1(predicted, ground_truth)
print("============ Recall: {}".format(rpaf[0]))
print("============ Precision: {}".format(rpaf[1]))
print("============ Accuracy: {}".format(rpaf[2]))
print("============ F1 Score: {}".format(rpaf[2]))

print("============ Relative error in total energy: {}".format(metrics.relative_error_total_energy(predicted, ground_truth)))
print("============ Mean absolute error(in Watts): {}".format(metrics.mean_absolute_error(predicted, ground_truth)))

Hi, Sir
Thanks for your help.
I am so confused for the following,

input(for ukdale):
train_meter = train_elec.submeters()['kettle']
train_mains = train_elec.mains()

output:
ElecMeter(instance=54, building=1, dataset='UK-DALE', site_meter, appliances=[Appliance(type='immersion heater', instance=1), Appliance(type='water pump', instance=1), Appliance(type='security alarm', instance=1), Appliance(type='fan', instance=2), Appliance(type='drill', instance=1), Appliance(type='laptop computer', instance=2)])
)

What is the meaning for this elecmeter ? is it site_meter for all application in this building ? but why there are part of application likes water pump, security alarm etc in the same ElecMeter.

Thanks,

Hi,
I found your project very interesting. Just a quick question.Is it possible to train and test the whole number of appliances (not only a single one)?. If it is possible, which command is used for doing and plot "ground_truth" and all the predicted appliances together?
Thank you in advance
Cheers

Thanks for your work, but I have a question about it.
in case of redd dataset there are two meters in site meter so only one meter of site meter is loaded to dis aggregate as like this
train_mains = train_elec.mains().all_meters()[0]

but i use synd dataset for this i have only one meter so i modified this as
train_mains = train_elec.mains()
with this code i got the Attribute error. as shown below

could you please suggest me the solution

Thanking you

hello!
I'm using Python 3.7 with Anaconda in Windows 7. But I can't run the code.
so could you a list of the packages and their versions?

Hello Odyssea,

When I run the GRUWithWindow code (python3 experiment.py kettle) from the https://github.com/OdysseasKr/online-nilm repository, I get the following numbers for metrics after training the class kettle for 4 houses and testing in the last house.

Epoch 00001: saving model to experiments/kettle/CHECKPOINT-kettle-1epochs.hdf5
Epoch 2/3
800000/800000 [==============================] - 722s 902us/step - loss: 0.0039

Epoch 00002: saving model to experiments/kettle/CHECKPOINT-kettle-2epochs.hdf5
Epoch 3/3
800000/800000 [==============================] - 698s 873us/step - loss: 0.0039

Epoch 00003: saving model to experiments/kettle/CHECKPOINT-kettle-3epochs.hdf5
/home/anasvaf/Desktop/MultiLabel_Energy/GRUWithWindow/metrics.py:40: RuntimeWarning: invalid value encountered in true_divide
  return tp/float(tp+fp)
============ Recall: 0.0
============ Precision: nan
============ Accuracy: 0.9962941517081645
============ F1 Score: nan
============ Relative error in total energy: 0.012862401778851902
============ Mean absolute error(in Watts): 21.46095589801764

I can confirm also the same exact numbers when I run the RNN code using the https://github.com/OdysseasKr/neural-disaggregator repository.

I am using Python 3.5 and the following library versions:
numpy: 1.15.2
matplotlib: 3.0.0
pandas: 0.23.4
pytables: 3.4.4
nilmtk: 0.2
keras: 2.2.4

Could you help me with that?

Thanks in advance!

-Tasos

Thanks for your awesome work, but I have a question about it. It looks like you just load
only one meter of site meter, as shown in your code:
train_mains = train_elec.mains().all_meters()[0] # The aggregated meter that provides the input

but there are two meters for recording site meter.
The aggregate powers summed together by elec.mains().power_series_all_data().

This is probably more of an NILMTK question, but the disaggregate function in the shortseq2pointdisaggregator.py takes an Elecmeter object as a parameter. However, I am interested in getting the disaggregation of a single monthly aggregate. I tried to circumvent this by using something similar to the dataset converter code, by writing and reading from a hdf5 store, but I didn't manage to get anywhere with it. Is there any other way to achieve this?

When I run rnn = RNNDisaggregator() , I get the error: ImportError: Failed to import pydot. Please install pydot. For example with pip install pydot. Pydot is already installed in my python.

Hi, first of all, thank you for your work. I have a question that I do n’t quite understand. I hope to get your answers. What is the use of 'train_across_buildings_chunk'? I do n’t know what this training is different from the previous 'train on chunk' It seems that 'train_across_buildings_chunk' is not used in the program. What is the purpose of this code?

Hi im trying to run the example but getting this error. I have already installed the NILMTK package

Hi there,

We are a class group of graduate students at USC. We are interested in this project and want to import it to implement energy disaggregation.
According to the official document and class requirement, MIT licence should be inlcuded.
Could you kindly add this license, which also makes the project more official?

Here is the reference
https://docs.github.com/en/communities/setting-up-your-project-for-healthy-contributions/adding-a-license-to-a-repository

Hi, I have no idea why so big different results when I run exactly the same code you posted.

Hello, can you give a detailed environment configuration? Specifically, it's about the version of tensorflow, and other Python wheel versions, because I always make mistakes when running these sources. I will be grateful.

I get this pydot and graphviz issue on my windows conda environment.

OSError: pydot failed to call GraphViz.Please install GraphViz (https://www.graphviz.org/) and ensure that its executables are in the $PATH.

Hi, I have some question as described as following,

after 1D Conv, the shape change from (1,1) to (1,16), is that means that each input value changed to (1,16)?
besides, Could you please give me more detail about how the first LSTM work? ex: How many step numbers here etc.

I am so confused, Thanks!

Hi,
I've been playing around with your code according to your IPython Notebooks and came across an error when calling the metrics function. Down the line, this function calls nlimtks goodsections.py function _process_chunk (https://github.com/nilmtk/nilmtk/blob/412be54eefd81ae78d0f1ed3053bf8977063e61d/nilmtk/stats/goodsections.py#L36) which expects a DataFrame with parameter timeframe, which did not exist and therefore threw an error.

While trying to fix my error, I came across the documentation of align_two_meters which says it would take the timeframe from meter 'master', which is the first argument of that function. In your call you provide the function with (pred, ground), which seems off to me. After changing the order to (ground, pred) I no longer encountered the problem.

Can you assure me, that the order of your arguments is indeed correct?

This is the section I'm referencing, but the call is them same for all metrics files:

Hello everybody ,
thank you OdysseasKr for the great work
i have tried the Recurrent network with LSTM , i have trained the model then exported it and it's working very well.
but i have an enquiry , after loading the model and when we use the method disaggregate of ### RNNDisaggregator() , why we need the aggregated meter that provides the input (train_meter) for disaggregation ? i'm confused

after calling " rnn.disaggregate(test_mains, output, train_meter, sample_period=1) "
there should be a output.close()
or error will be raised.

Hello!
Thank you for your contribution. I would like to know how to add loss visualization function in RNN network. I would like to observe its fitting

Hi,there! I'm wondering how to get activations like the paper you sited in the "ReadME.txt", I download your code but I can't find where did you use NILMTK to get activations of the target electrical equipment. Hope to get an answer of how to use NILMTK to solve this problem.
Meanwhile, the title of your project is neural-disaggregator, but the coda is for predicting. I'm wondering the contact between them.
Thx for your answering!

In the Neural NILM paper it says
https://arxiv.org/pdf/1507.06594.pdf

The
window width is decided on an appliance-by-appliance basis
and varies from 128 samples (13 minutes) for the kettle to
1536 samples (2.5 hours) for the dish washer

Where in your code can I find the window width ? Or if you could point me to the code where we can play with window width

Hello Odyssea,
when applying the method export_model after training my model:

disaggregator.export_model("model-redd5.h5")

I try to import this model by doing :

cls = disaggregator.import_model("model-redd5.h5")
predictions = cls.disaggregate_chunk(A_batch)

but by doing that, it returns that cls has no method disaggregate_chunk. To predict test values, i have to train every time my model. So how can i use the method import_model for the disaggregation, without doing the training every time I need to predict test values.

Thanks

I can not find this method in your code

Hi, @OdysseasKr
How do you extract activations and select window width in your code as written in Neural NILM?

1. extract activations
   
2. select window width
   

THANKS!

Hello,

I am confused with Batch Size and Window Size. Can you please explain with an example along with how do choose these numbers?

hi, i have run RNN-example.ipynb,but the data is not correct.
my result is
============ Recall: 0.06080812748658777
============ Precision: 0.7873688147161255
============ Accuracy: 0.29049305213046556
============ F1 Score: 0.11289725264136873
============ Relative error in total energy: 0.7858544224150322
============ Mean absolute error(in Watts): 19.859596349905722

but your result is
============ Recall: 0.997835349341
============ Precision: 0.742378777703
============ Accuracy: 0.741308963402
============ F1 Score: 0.851357054837
============ Relative error in total energy: 0.871686427835
============ Mean absolute error(in Watts): 32.2338755931

I don't know what I did wrong.I ran it all according to your code.

NILMTK seems to have moved to Pandas 0.22. Compatibility should be tested to make sure that everything runs smoothly.

Hello Odyssea,

I am trying to replicate the results based on the Kelly et al. paper.
Using your code I changed the train building to the following:

train.set_window(start="13-4-2013", end="1-1-2014")
test.set_window(start="1-1-2014", end="30-3-2014")

train_elec = []
test_building = 5
sample_period = 6
meter_key = 'kettle'

# Lists for train_elec, train_meter and train_mains for multiple buildings
train_elec = [train.buildings[i].elec for i in range(1,5)]
train_meter = [train_elec[j].submeters()[meter_key] for j in range(len(train_elec))]
train_mains = [train_elec[k].mains() for k in range(len(train_elec))]

# Test only in one house
test_elec = test.buildings[test_building].elec
test_mains = test_elec.mains()
rnn = RNNLSTMDissaggregate()

and then I call the train_across_buildings

start = time.time()
print("========== TRAIN ============")
epochs = 0
for i in range(3):
    print("CHECKPOINT {}".format(epochs))
    rnn.train_across_buildings(train_mains, train_meter, epochs=5, sample_period=sample_period)
    epochs += 5
    rnn.export_model("UKDALE-RNN-h{}-{}-{}epochs.h5".format(train_building,
                                                        meter_key,

I am creating lists for the 4 houses that I need during the training but I get the following error when I run the RNN for the function "train_across_buildings"

Could you give me some hint on how to change the code?

Best,
Tasos