alanmitchell / bmon Goto Github PK
View Code? Open in Web Editor NEWWeb-based Building Monitoring Database and Analysis Software
License: Other
Web-based Building Monitoring Database and Analysis Software
License: Other
Call this every 5 minutes (whatever the fastest needs are, probably for Alert checking), and then dispatch to the other various scripts at the correct time.
AHFC Description: Link ARIS cost use with BMS: Work with Shawn Butler at RDI (770-4091) to develop a means to export the ARIS cost use data with the data from the BMS would enable users to see costs associated with various energy uses and building operations, and allow AMP’s and management options for cost reductions. Coordinating with RDI, obtain a data link from ARIS to the building monitoring system and display costs on gas and electric meters graphs, and elsewhere as appropriate.
The schedule day range Sa-Su
does not work because it goes from day number 6 to day number 0.
See the testing.ipynb
IPython notebook that was added to the repo to facilitate testing. The graph at the end of the notebook shows the occupied schedule across a full week so helps in spotting problems.
Then, a Version Change Log and Installation Notes document can be created. Sometimes changes to the settings.py
file will be required by a new version, and these should be noted somewhere for users that upgrade their installations.
AHFC Description: Small maintenance comes up with the web server that hosts the data viewing application. Detail maintenance requirements and an annual budget for server maintenance.
Functions are already coded but need to be assigned to the various sensors on the Live AHFC site..
Right now the extent of of the Map is locked to all of Alaska. Probably should have both the size and center of the Map be based on the extent of the plotted buildings.
Branch: cur_values_multi
, which has already been created
Billing: Task 7
This report will look similar to the existing Current Values report, except that it will allow selection of sensors in multiple buildings. Not all sensors will be shown; only those sensors selected for the report in the Admin interface.
The layout will differ from the existing current values report in the following ways:
reports/<building ID>/
will do that, similar to how it works in the Map interface).**
, and the sensor with the largest value will have a bar length of 20, ********************
(or adjust as you see fit).bmsapp/reports/currentvalues.py will be useful, but bmsapp/reports/normalizedbyft2.py will also be useful to show how a multi-building report is created. As you'll see in that file, the report must filter out buildings that don't fit in the User's selected building group.
The new report will go in the bmsapp/reports/currentvalues_multi.py
file, which has already been set up.
In the bmon.sqlite
Django database I sent you, there is already a sample Multi Building Chart set up that has selected this new report type and has selected some sensors across a couple different buildings.
Currently, the Histogram graph has a separate series for the Raw data, the 1 Hour averaged data, the 2 hour averaged data, etc. For a building that has an occupied Schedule, it is probably more valuable to have the different series be related to the schedule. So, for a building with a schedule, the different series should be:
I think that initially, we should show the Occupied
and Unoccupied
series and have the All Data
series toggled off.
To still allow for averaging, I will enable the Averaging select control in the user interface so the user can choose what level of time averaging they want. The chart code will now need to process that input. (see the timeseries.py
graph file as an example of how to acquire that input). Also, the classification of a data point should depend on the averaging selection. Stay consistent with timeseries.py
: the BaseChart.occupied_resolution()
method determines how much resolution should apply to classification. If the function returns None, only the All Data
series should be built, as occupied/unoccupied classification does not make sense.
Because I am now exposing the averaging selector, I think a building without a schedule can only have one series, the All Data
series.
The relevant file to change is the bmon\bmsapp\reports\histogram.py
file.
The only building that currently has a schedule defined is AHFC Headquarters
. So, you'll need to use that for your testing.
AHFC Description: Develop monitoring protocol and hardware standard for forced air heating systems. While most AHFC owned buildings are heated with boilers, there are a few with forced air systems. Knowing the system is operating, run times and estimates on fuel usage are useful to operations staff. Develop a means and method for monitoring these systems.
SUNDAY
and su
should work for Sunday.
This comes from Weather Underground I believe, so may be some missing values returned that cause this.
To replicate: refresh the site, select a building, select Histogram, change the sensor.
Parameter skeleton was added but not implement in commit aca73c6. At the moment, no need to collapse contiguous occupied days into one occupied tuple; individual tuples for each day are OK.
The Burt158_boiler_gas calculated sensor is a linear calculation off of the Burt158_firing_rate sensor. There are periods (e.g. Nov 5 - 9, 2014) where the source sensor readings (firing rate) were present, but the calculated values are not showing in the database. Need to check to see if this occurs for other calculated values as well.
AHFC Description: The Monnit wireless sensor online portal has the option to send alarm notices to users. A second online system is necessary to set up the alarms. Investigate and estimate costs to set up an alarm notification system, and work with AHFC staff to designate recipients of alarms for each building.
AHFC Description:
Different users of the system could be turned On and Off without affecting other users.
AHFC Description: Perform data analysis on the data from the installed systems. Analyze the data from the phase I buildings and newly selected buildings to be monitored and provide recommendations for:
Simulate the data flow that will occur from a large system involving 100+ buildings, to help determine how well it will be handled by the existing web server and system architecture. Report should include recommendations for changes or upgrades to handle such a load.
bmsapp/data
directory into the bmsapp\readingdb
directory.bmsdata.py
module that gives the absolute path to the default database.bmsapp/readingdb/scripts
directory to hold scripts related to the reading database:
backup.py
for backing up database.daily_status.py
to record number of readings insertedimport_readings.py
for importing readings from a text filecalc_readings.py
script a runscript
script for cleaner access to the main Django database.add_sensors.py
script to the bmsapp/scripts
directory instead of the top level directory.DATA_DB_FILENAME
and PROJ_DB_FILENAME
in global_vars.py
. This module then may only be needed for setting up logging, so should probably be renamed. Make sure it runs when needed (because many imports to global_vars
were removed when the database path was made optional).bms_data.sqlite
from bmsapp/data
to bmsapp/readingdb/data
. Remove the remaining empty bmsapp/data
directory.import logging
and LOG_LEVEL
to settings.py file.manage.py runscript
when necessary.It uses the Linux command gzip
, which will not be present on a Windows Server. Could replace this with the use of a Python compression library.
AHFC Description: The mini-monitor is able to read a large number of alarm conditions from a boiler (e.g. low water level, high temperature, low air flow, flame safeguard lockout, etc.). The online software does not have a means to display these. Create a means and methodology to display alarm conditions. Outline and document the methodology and include in the source code provided to AHFC.
There are no occupied periods shown in the graph for Sunday. A cell that tests 1 pm on Sunday shows it to be unoccupied, but the entered schedule shows 12p-4p occupied hours for Sunday.
Should use the value in the Django settings file, TIME_ZONE, or the value set for a particular building.
The HighStock control can handle very large datasets. We could only use HighStock with large datasets, if it is not as user friendly as the HighChart control. But that might be more confusing to have the chart type be different depending on the time period of data selected.
Tooltips often annoyingly obscure parts of the graph. This looks like a better technique, which reports the values where the mouse is hovering in a
Although, the logging_setup.py may be imported by non-Django scripts, and so will easy access to settings.py be possible?
Billing: Task 16
Can the Server and Client be configured, through MIME type perhaps, to do this automatically? This is part of Task 16.
The footer should reference an https image, as secure sites will not load from an http unsecure source.
There is no logging of the execution of the BMON crontab jobs. See http://docs.webfaction.com/software/general.html for an approach for doing this on the Webfaction Host. This will help debugging Issue #1.
AHFC Description: Create the external documentation describing structure, tools needed, and basic instruction for someone proficient in Python and Django programming language to be able to install the software and make it operational and the advantages or disadvantages of using this system or setting up an independent system using this software.
AHFC Description: Presently graphing more than a few months of raw (unaveraged) data, the system slows way down. The problem is the graphing tool currently is in use is not good at rendering 20,000+ points of data. Obtain, install and integrate the BMS with a different graphing tool that is more suited to graphing very large data sets; the enhanced graphing tool costs about $700. Retain the existing graphing tool for small data sets, but automatically switch to the new one for large data sets (it operates slightly different from the old). Working with RDI, CCHRC and AHFC, investigate and outline methods and means to display, store and archive data for future research, maintenance and operation records, or other analysis.
AHFC Description: Outline and provide a reasonable budget for unforeseen/contingency needs may arise with unforeseen upgrades. Provide documentation and source code on the upgraded system to AHFC.
This may be able to cover a portion of the Client-Side Restructure.
It is useful to see the version of software running on the server without having to SSH to the server.
For approaches to retrieving the info:
Probably should not do this by supplying a sample database, since the database contains the Super User info including password. Maybe a simple Django runscript that loads table through use of the data model.
It's my API Key. Other users should enter their own API Key if they want to use Weather Underground data.
This give flexibility to enter arrays as parameters as well.
yaml.load()
for makeKewyordArgs()
.**params
call in the Transformer.transform_value()
method as the parameters object may be None, a List or a Dictionary. Also true in CalculateReadings.processCalc()
.:
separating a YAML keyword from the value. Can also be a space before. a: 23.3
and a : 23.3
are valid. a:23
is not (no space)._sync
suffix to identify the sensor to synchronize on; otherwise code will just use the sensor ID that appears first in a Python dictionary.AHFC Description: Currently, the web site is accessed through "http" as opposed to the more secure "https". That means that it is possible for a snooper to intercept passwords that are used to access the "administrative" part of the application where sensor and building descriptions are set up. Also, every time sensor data is posted to the site, a "storage key" (like a password) is used to ensure that the data is coming from a trusted source. Right now, that storage key can be intercepted by a hacker. Moving the web site to "https" access will solve these problems. The work involves purchasing and installing an SSL security certificate on the web server and then changing the URL string for each sensor on the system from "http" to "https". Provide AHFC with all passwords and credentials to access all aspects of the system.
AHFC Description: The Hourly Profile graph currently allows the data to be separated by days of the week or groups of days of the week. Functionality for the graphs to group the data and show the difference between occupied and unoccupied periods. Document the methodology and include in the source code provided to AHFC.
This will show up in the browser tab. See http://www.w3.org/2005/10/howto-favicon.
May not be able to use the 4.x series of Highcharts since maintenance was not bought. Highcharts was bought on 5/9/2013. However, looks like the BMS site could be eligible for Single Website license.
Important for Security.
Billing: Task 13
In the module https://github.com/alanmitchell/bmon/blob/task13/bmsapp/formatters.py, add a comment to each formatter function to indicate which field(s) it can format, e.g.:
"""This formats the 'demand_source' field of the mini-monitor 'sage_boiler' reader.
"""
Pick some keep time (maybe two weeks?) and delete all backup files older than that. Could do this right after completing the current backup.
AHFC Description: Generating alarm reports across multiple buildings would be useful. These would be in report format, not a graph. Some multiple building energy reports would be useful for maintenance staff. Document the methodology and include in the source code provided to AHFC.
AHFC Description: Prepare training guidelines for various levels of users to optimize their use of the system. Identify levels of training necessary for various users. Analyst level users (AHFC Staff) should be trained to spot anomalies in the data, analyze trends and combine that knowledge with building operations to maximize efficiency. AMPS should be able to obtain reports of energy costs and use for each building and see periodic snapshots of building operations. Maintenance and operations staff should be able to use the data to check for proper system operations on a daily basis. Install dashboards similar to the Glacier view building where appropriate. Consumers, residents and Management may want to view what is going on, but at a very basic level.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.