azure / connectthedots Goto Github PK

Update images in documentation to reflect updated website, JSON format, logfiles, etc

Hi, today I try this project sample, it work fine.

But I notice WebPage won't receive alerts from EventHubs,

Aggregates always have output envets to EHAlerts

LightSensor and Alerts receive data from EHDevices but won't output anythings to EHAlerts

but ehalerts still recive some data?... but won't show on webpage.

here is ehdvices

i also adjust some query string to fit my sensors.(either alerts)

I check every step, but can't find why..

and when i create StreamingAnalytics, add output, it alias can't change, it is gray.

So alerts, lightsensor, aggregates these 3 stream analytics output name all call "output"

I've run the ConnectTheDots demo app a couple of times now, once for a user group, and again for an online conference.

While it's a great demo for showing end-to-end IoT communication, there's a lot involved in cleaning up the Azure resources used by the demo, and it would be great if this could be automated to some degree.
Perhaps it would be possible to have the app that performs a substantial portion of the setup also create a powershell script that could use the Azure command line APIs to remove the Service Bus and Storage resources it creates.

If I can find the time to go through the code and try my hand at hacking it together, and if I manage to make something that works, I'll submit it as a pull request.

For v2 of the website.

Would be useful to show data coming out of the Raspberry Pi, so user can see that JSON format has not been changed from when it was sent by the Arduino.

After a task is popped from the gateway queue, we discard it if we fail to process it

// process all messages that have not been processed yet
while (--count >= 0)
{
var t = _DataSource.TryPop(); <<<<<<<<<<<<<<< HERE

                        Debug.Assert( _outstandingTasks >= 0 ); 

                        bool added = false;
                        try
                        {
                            // increment outstanding task count but be ready to decrement if we fail to add to the queue
                            Interlocked.Increment( ref _outstandingTasks );

                            Debug.Assert( tasks != null );

                            tasks.Add(
                                t.ContinueWith<Task>( popped =>
                                {
                                    Interlocked.Decrement( ref _outstandingTasks );

                                    // because the outstanding task counter is incremented before 
                                    // adding, we should never incur a negative count 
                                    Debug.Assert( _outstandingTasks >= 0 );

                                    if( popped.Result.IsSuccess )
                                    {
                                        if( _DataTransform != null )
                                        {
                                            return _DataTarget.SendMessage( _DataTransform( popped.Result.Result ) );
                                        }
                                        if( _SerializedData != null )
                                        {
                                            return _DataTarget.SendSerialized( _SerializedData( popped.Result.Result ) );
                                        }
                                    }
                                    else
                                    {
                                        throw new Exception( );
                                    }
                                    return popped;
                                } )
                            );

                            // the only case when we do not get here is if List.Add fails (tasks.Add) or
                            // if the task engine fails Task.ContinueWith (t.ContinueWith)
                            added = true;
                        }
                        catch
                        {
                            if( !added )
                            {
                                Interlocked.Decrement( ref _outstandingTasks );   
                            }

                            //
                            // TODO: Issue 
                            //
                            throw;
                        }
                    }

Hi there
Is someone is able to show a example of sending data to the hub with a more "generic" way; IE not using mono or any other layer.

For example posting data to the hub just using curl or a browser URL ?

Thanks for your help

Current endpoint abstraction for gateway

    public bool SetEndpoint(ISensorEndpoint endpoint)

does not work very well across devices

find better abstraction or cut!

autorun_once.sh not being launched by rc.local

BatchSenderThread would not send messages if new event popped before we WaitOne it and after encounting number of items to queue.

Currently ConnectTheDotsCloudDeploy looks for the publish settings in the same directory as the .exe. This means that if the developer does a clean build the file will be deleted.

We need to look in a more sensible location.

Legacy device name should be changed to be consistent with current names

The only reason why we need SignalDataArrival is to invoke the event,
find another way

we need to validate incoming JSON sooner rather than later for efficiency and ability to log error where they make sense

There is nothing specific, other than startup scripts, to the RaspberryPi within the Gateway. In fact during testing it is easier to use a Windows machine. Some folks might even prefer to use a Windows machine as the gateway.

For example, I have a Dell Windows 8 Tablet as a controller for my Hue lights in some of my rooms. Rather than install a Pi in these rooms I'd rather use the existing tablet as the Gateway.

It therefore seems to make sense to separate out the startup scripts from the Gateway code. THis would give us both a "RaspberryPiGateway" and a "Gateway" artifact.

Thoughts?

i.e.: for serial port, there are many lines for printing on serial.
that is error prone.
build the string and dump it only once.

I tried to create azure resources as
cd ConnectTheDots\Azure\CloudDeploy\ConnecTheDotsCloudDeploy\bin\debug
ConnectTheDotsCloudDeploy.exe –n -ps

but when I run this command I am getting this error

Error:unrecognized argument: -n
Usage :ConnecttheDotsCloudDeploy -publishSettingFile [-NamePrefix ] [-Location ] [-website ]

I tried to deploy the sample application given for Intel Galileo but it is giving runtime errors. Has someone worked with Galileo?
I am getting the following errors

'ConnectTheDotsGalileo.exe' (Win32): Loaded 'C:\test\ConnectTheDotsGalileo.exe'. Symbols loaded.
'ConnectTheDotsGalileo.exe' (Win32): Loaded 'C:\Windows\System32\ntdll.dll'. Cannot find or open the PDB file.
'ConnectTheDotsGalileo.exe' (Win32): Loaded 'C:\Windows\System32\kernel32.dll'. Cannot find or open the PDB file.
'ConnectTheDotsGalileo.exe' (Win32): Loaded 'C:\Windows\System32\kernel32legacy.dll'. Cannot find or open the PDB file.
'ConnectTheDotsGalileo.exe' (Win32): Loaded 'C:\Windows\System32\KernelBase.dll'. Cannot find or open the PDB file.
The program '[1924] ConnectTheDotsGalileo.exe' has exited with code -1073741515 (0xc0000135) 'A dependent dll was not found'.

To allow for additional devices, Cloud services and scenarios to be added, we need to break the main solution into several ones. One for the Raspberry Pi + Arduino code, one for the Cloud Deploy tool, one for the Web site.
The Wiki will have to be adapted as well and the Step-by-Step document broke into several pages.
Once we have this in place, it will be easier to add devices, and then add services and scenarios.

ConnectTheDotsCloudDeploy requires that a name be submitted. This name has fairly stringent constraingts. Test for them before starting to create the service otherwise we get a partially complete service:

We need to check for (at least):

• only letters and numbers – no spaces, dashes, underlines, etc
• less than 17 characters (storage account names must be < 24 characters and we add "storage" to the end)

Verify that CoreTest test actually work when working with MockSender and Actual AMQPSender

Needs updating. Also, splitting MD into separate folders will make it easier to add devices and gateways

Has dependency on BT interface on gateway being enabled

will make it easier to debut, and to e.g. just publish website with existing EH

The user is currently instructed to manually configure autorun.sh which means the file cannot be reliably shared in version control without breaking other peoples builds.

This configuration needs to be moved to a properties file.

Hello!

Today I started to follow the step by step guide to build the connect the dots project. While publishing the Connect The Dots website to my azure website the server returned an error. I enabled websockets in the Azure portal and already spent hours on this problem. I do not know what's wrong or how to fix this. If you need more information, please let me know.

Please change to:
EventHubNameDevices = "ehdevices";
and
EventHubNameAlerts = "ehalerts";

Otherwise deploying the web site project fails.

Running the NETMF/Gadgeteer code (no changes) and I get this compilation error on the following line of code:
message.Body = new Data() { Binary = Encoding.UTF8.GetBytes(payload) };

Error:
Property or indexer 'Amqp.Message.Body' cannot be assigned to -- it is read only...

Here is the code block:
void SendAMQPMessage(string payload)
{
try
{
// 0 indicates the method is not in use
if (0 == Interlocked.Exchange(ref IsSendingMessage, 1))
{
// Create the AMQP message
var message = new Message();
message.Properties = new Properties()
{
Subject = "wthr",
CreationTime = DateTime.UtcNow,
ContentType = "text/json",
};

                message.MessageAnnotations = new MessageAnnotations();
                message.MessageAnnotations[new Symbol("x-opt-partition-key")] = DeviceID;
                message.ApplicationProperties = new ApplicationProperties();
                message.ApplicationProperties["time"] = message.Properties.CreationTime;
                message.ApplicationProperties["from"] = DeviceID;
                message.ApplicationProperties["dspl"] = DeviceName;
                //message.Properties.ContentType = "text/json";
                message.Body = new Data() { Binary = Encoding.UTF8.GetBytes(payload) };

                sender.Send(message);

                // release lock
                Interlocked.Exchange(ref IsSendingMessage, 0);
            }
        }
        catch (Exception e)
        {
            Debug.Print("Exception caught:" + e.Message);
        }
    }

need to separate interfaces from Gateway assembly, so we reduce dependencies

Remove raw data portion from CTD web

It seems that when deploying ConnectTheDotsWebsite locally, I can view independent device streams as pictured below:

However, when deploying ConnectTheDotsWebsite to Azure and enabling Web Sockets, the user only sees aggregate data from the ehalerts event hub. Selecting an independent device yields no data in the Temp / Humidity Graphs.

To reproduce, simply visit http://connectthedots.azurewebsites.net and select any of the available devices. Note that the only data displayed is the Average data from ehalerts.

for example, this code:

Assembly ass = Assembly.LoadFrom( e.AssemblyPath );
Type handlerType = ass.GetType( e.TypeName );

                    IDataIntake dataIntake = ( IDataIntake )Activator.CreateInstance( handlerType );

should check for null on the return types

(at the time I opened this bug, MSDN was down, so check may not be needed)

current entries are inconsistent, please make all lower case and no under score

public class SensorDataContract
{
[DataMember(Name = "Value")]
public double Value { get; set; }

    [DataMember(Name = "GUID")]
    public int Guid { get; set; }

    [DataMember(Name = "Organization")]
    public string Organization { get; set; }

    [DataMember(Name = "DisplayName")]
    public string DisplayName { get; set; }

    [DataMember(Name = "UnitOfMeasure")]
    public string UnitOfMeasure { get; set; }

    [DataMember(Name = "MeasureName")]
    public string MeasureName { get; set; }

    [DataMember(Name = "Location")]
    public string Location { get; set; }

    [DataMember(Name = "time_created")]
    public DateTime Location { get; set; }

    [DataMember(Name = "time_arrived")]
    public DateTime Location { get; set; }

}

Similar to stock charts? 1 hour/1 day/1 mo/3 mo?

It seems that the code to display alerts on the web page is missing. I could only see the table headers but nothing else.

        </div>

    </div>

Hello,
Am getting the following error after publishing this website in azure. Kindly help me to proceed further.

Thank you!

Azure\ConnectTheDotsWebSite\web.config is modified by ConnectTheDotsCloudDeploy.exe Since this is a source file it should not be modified by the application itself.

We should move to having a template in source and saving the modified version in our local config for the application.

I'm only just starting to look at the gateway application and I'm not a C# developer so I may be missing something fundamental, but...

Program.cs generates Unique identifier for the device:

string deviceId = Guid.NewGuid().ToString ("N"); // Unique identifier for the device

However, this gets overwritten in autorun.sh:
-deviceid C8CA5B13-A550-4FF0-B823-46D8A2640880

It seems to me that autorun.sh should not set the device ID, otherwise every device in the network will have the same ID. I've verified that running without a DeviceID appears to work (on Windows and in a very limited test environment).

What are the implications of removing that switch from aytorun.sh, that is what is deviceID used for?

I created the website by following the same steps and published it but when running it I am getting the following error(The name I have given is "weatherstation")

Server Error in '/' Application.
EventHub names must conform to the following rules to be able to use it with EventProcessorHost: Must start with a letter or number, and can contain only letters, numbers, and the dash (-) character. Every dash (-) character must be immediately preceded and followed by a letter or number; consecutive dashes are not permitted in container names. All letters in a container name must be lowercase. Must be from 3 to 63 characters long.
Parameter name: leaseContainerName

[ArgumentException: EventHub names must conform to the following rules to be able to use it with EventProcessorHost: Must start with a letter or number, and can contain only letters, numbers, and the dash (-) character. Every dash (-) character must be immediately preceded and followed by a letter or number; consecutive dashes are not permitted in container names. All letters in a container name must be lowercase. Must be from 3 to 63 characters long.
Parameter name: leaseContainerName]

Currently trace info sent to console, piped to a log file. If the Pi left on for a long time, this log file eventually uses us all available space on the Pi, causing issues. Need to implement better log process or remove from sample code.