Comments (2)
Playing around with ideas for an Environment class:
class Enviroment(object):
"""Store and process environmental data from multiple SensorStations.
Attributes
----------
sensor_stations : dict of SensorStations
Local store of environmental sensor data.
Each key is a (latitude, longitude) tuple representing the location of
the sensor station; where a `sensor station` is a set of sensors all
installed at the same physical location; either by a national
meteorological office or by interested individuals.
Each value is a SensorStation storing environmental data using
standard physical units and standard column names. e.g.
'temperature' | 'rainfall' | 'sunshine'
targets : list
A list of tuples representing target locations and measurements in the
form (<latitude>, <longitude>, <measurements>).
Usage
-----
> london = (51.51125, -0.10849) # latitude, longitude
> enviro = Environment()
> enviro.register_target(london, ['temperature'])
enviro will now asynchronously find suitable data. We can help by
loading some data from disk:
> enviro.load_sensor_station_from_disk('heathrow_metoffice.h5')
Now we can get data:
> data, distance = enviro.estimate_measurement_at(london)
"""
def register_target(self, target_location, measurements,
start=None, end=None, k=3):
"""Register a target of interest.
This function returns immediately. Asynchronously, `Environment` will
search thingful.net for sensors near the target's geo location.
Arguments
---------
target_location : (latitude, longitude) pair
measurements : list of strings
'temperature' | 'rainfall' | 'sunshine' | etc...
start, end : DateTime, optional
Specify the start and end time of interest
k : int, optional
The maximum number of nearest `sensor_stations` to
use in the interpolation.
"""
return
def get_estimate_measurement_at(self, target_location, measurement=None,
start=None, end=None):
"""Get a timeseries of sensor data nearest target_location.
This function finds the `k` nearest `sensor_stations` to
`target_location` and interpolates readings for a
specific `measurement` and a specific time `period`. It searches local
data stored in `sensors_stations` first; if it can't find suitable data
then it queries known sources of data from the network and tries to
find new sources using thingful.net (an index of public IoT sensors).
Arguments
---------
target_location: (latitude, longitude) pair
measurement : string
'temperature' | 'rainfall' | 'sunshine' | etc...
start, end : DateTime, optional
Specify the start and end time of interest
Returns
-------
data, distance
data : pandas.Series
The interpolated data.
distance : float
The distance (in km) from `target_location` to the
nearest `sensor_station`.
"""
# Could use geopy to measure distances
# https://github.com/geopy/geopy
# To start with, let's just use simple linear interpolation.
# Later down the line (perhaps as an MSc project?) we could
# explore more sophisticated interpolation, e.g.:
#
# Osborne, Michael A., Roberts, Stephen J., Rogers, Alex and
# Jennings, Nicholas R. (2012) Real-time information
# processing of environmental sensor network data using
# Bayesian Gaussian processes. ACM Transactions on Sensor
# Networks, 9, (1), 1:1-1:32. (doi:10.1145/2379799.2379800)
#
# See this twitter conversation:
# https://twitter.com/acr_ecs/status/410891244983058433
return
def load_sensor_station_from_disk(self, filename):
return
def plot_sensor_stations_on_map(self):
# Could use basemap:
# http://matplotlib.org/basemap/
return
def update_sensor_data_from_network(self):
return
class SensorStation(object):
"""Store and process environmental data from a sensor station
Attributes
----------
data : DataFrame
Store environmental data using
standard physical units and standard column names. e.g.
'temperature' | 'rainfall' | 'sunshine'
source_url : string
Local or network uniform resource locator for source of data
last_updated: DateTime
The date and time we last pulled data from `source_url`
measurements_available_from_source : list of strings
"""
def update_from_source(self):
return
def save_to_disk(self):
return
def load_from_disk(self, filename):
return
class UKMetOffice(SensorStation):
def load_native_from_disk(self, filename):
"""Loading and convert UK Met Office CSV files."""
For reference, here's my (ugly) code from PDA for importing data from a UK metoffice .xls file.
For converting between (latitude, longitude)
and human-readable addresses; and for calculating distances between two points, we could use geopy: "geopy makes it easy for Python developers to locate the coordinates of addresses, cities, countries, and landmarks across the globe using third-party geocoders and other data sources." and can do distance calcs. MIT Licensed and is on PyPI.
Thoughts?
from nilmtk.
I've just updated the code sketch above. Some new features:
- usage example!
- users can now register targets of interest and system with asynchronously find nearby sensor stations from thingful.net
- new SensorStation class
I've spoken to some folks about the idea of having a web service which aggregates live, public, environmental sensor data (e.g. from Xively, national metoffices, smart phones etc). Users would be able to query to service to ask “give me for over at ”. e.g. “give me temperature for SE15 over the last month at hourly resolution” The service would find the K nearest measurement stations (using thingful.net) to the target location and then interpolate spatially and temporarily to produce the output the user wants. The service would take data from as many heterogeneous sources as possible and handle dodgy input data. It might also use simple models to make the spatial and temporal interpolation vaguely smart.
I might propose this project as an MSc individual project over summer.
from nilmtk.
Related Issues (20)
- Can I use it to know if the household has electric vehicle?
- Unable to import FHMM_Exact using experimentation API HOT 5
- Redd to Aws converter
- Error: failed to send plot to http://127.0.0.1:63342 when trying to plot any NILMTK plot on Pycharm! HOT 1
- AssertionError: when trying to load iawe-dataset (or any other) with nilmtk DataSet HOT 5
- What units are used for power and energy ?
- REDD dataset link not working HOT 14
- Error training with FHMM (unable to allocate 26.0GB for an array) HOT 3
- How to run nilmtk in the cloud? HOT 4
- I am trying to convert the AMPDS2 dataset to work with the REDD dataset
- Installation Help and clarification! Support! HOT 4
- Data loss when converting e.g. iAWE csv to h5 HOT 1
- issue about "ukdale.h5"
- Conda install not working HOT 11
- Need Help: Creating Artificial Dataset
- Error for train_elec.plot() whcih is :DatetimeArray._generate_range() got an unexpected keyword argument 'closed' HOT 4
- Why the need for sort_learnt_parameters()?
- https://github.com/nilmtk/nilmtk/blob/master/docs/manual/user_guide/install_user.md HOT 1
- DISSAGREGATION WITH A MODEL GENERATED AND A NEW DATASET WITH ONLY THE AGREGATE(MAIN) HOT 1
- Modifying AMPDS2 Dataset for Appliance-Level Analysis HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from nilmtk.