Support Azure Storage Emulator about azure-webjobs-sdk HOT 88 CLOSED

This isn't just a cost issue. Not all development (or testing) should happen with network connectivity required. Other platforms allow you to test or dev locally or even in in-memory mode. It is an insight to the culture of the team releasing this software that this wasn't in by default and still hasn't been actioned in 20months.

from azure-webjobs-sdk.

You remove the posts because you are UNPROFESSIONAL, don't like being
called out into the light of truth and you can't take criticism. I don't
think you are doing the right job.
On Apr 30, 2016 12:16 PM, "Christopher Anderson" [email protected]
wrote:

Let me blunt since my previous hint was ignored. Any post that isn't about
how to make this feature work instead of why we aren't doing, will be
removed. You can email me personally if you'd like to continue to accuse
us of being unprofessional or provide a feedback that this lack of feature
prevented you from using WebJobs SDK. This conversation is distracting to
our team and our community members which might actually want to discuss the
issue itself.

My email is chrande (at) microsoft (dot) com. Please let us get back on
track.

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#53 (comment)

from azure-webjobs-sdk.

ping is this fixed yet?

from azure-webjobs-sdk.

I'm pleased to announce that I've merged @xt0rted's PR for this. Our latest prerelease build version 1.2.0-alpha-10307 on myget (instructions here) has these changes.

Please give the package a try and report any additional issues as new issues. As we've said above - there may be some issues you run into, and not all things may work in the emulator (YMMV).

We'll also be getting a prerelease version up on Nuget in the near future for this release (we haven't done that yet).

from azure-webjobs-sdk.

This is my full test. With this running I then used the Azure Storage Explorer to add items to the blob container & queue and saw the output in the console. As per the wiki I also have an environment variable of AzureWebJobsEnv set to Development.

class Program
{
    static void Main(string[] args)
    {
        var config = new JobHostConfiguration();

        if (config.IsDevelopment)
        {
            config.UseDevelopmentSettings();
        }

        var host = new JobHost(config);
        host.RunAndBlock();
    }
}

public class Functions
{
    public static void QueueTest([QueueTrigger("testqueue")] string message)
    {
        Console.WriteLine(message);
    }

    public static void BlobTest([BlobTrigger("testblob")] string message)
    {
        Console.WriteLine(message);
    }
}

<configuration>
    <connectionStrings>
        <add name="AzureWebJobsDashboard" connectionString="UseDevelopmentStorage=true;" />
        <add name="AzureWebJobsStorage" connectionString="UseDevelopmentStorage=true;" />
    </connectionStrings>

    <startup>
        <supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.6.1" />
    </startup>
</configuration>

from azure-webjobs-sdk.

I have a paid account on azure :) but I'd rather test locally before
putting it up. Thanks for the info though.

On Tuesday, 23 February 2016, David Kreth Allen [email protected]
wrote:

For those who are developing locally and don't want to pay for an Azure
account to develop, I just learned of the free Developer program that
offers $25/month in free Azure services.
https://www.visualstudio.com/en-us/products/visual-studio-dev-essentials-vs.aspx.
That doesn't sound like much, but a Queue or Blob does not cost that much
to operate with amounts of data and transaction volumes typical of
development activities. So it may be a good option for many people.

—
Reply to this email directly or view it on GitHub
#53 (comment)
.

https://www.profiles.google.com/fwd079

from azure-webjobs-sdk.

For those who are developing locally and don't want to pay for an Azure account to develop, I just learned of the free Developer program that offers $25/month in free Azure services. https://www.visualstudio.com/en-us/products/visual-studio-dev-essentials-vs.aspx. That doesn't sound like much, but a Queue or Blob does not cost that much to operate with amounts of data and transaction volumes typical of development activities. So it may be a good option for many people.

from azure-webjobs-sdk.

@RickRockhold using a test storage account is what everyone is doing, and it's pretty painless. Everyone can use their own (they're basically free if you don't have huge amount of data), so there is no need to prefix data with names, nor to undo anything. So I really wouldn't call that awkward.

That being said, I'm all for getting emulator working if enough users put their head together and figure out how to make it work (see my entry above for the initial step to take).

from azure-webjobs-sdk.

@RickRockhold I don't mind people wanting a local solution. But I'm
confused about the security/safety concern you express. There should be no
security or safety concerns when working with Azure. When I work in Azure
as a developer, I use development environments in Azure. Those storage
accounts are different from the production storage accounts. You can
control access to production storage accounts. You can all be in Azure but
still protect production software assets. It is common practice to set
storage keys in settings and override them for environments.

On Fri, Apr 29, 2016 at 10:35 AM, RickRockhold [email protected]
wrote:

I've never worked in a development environment where the official testing
procedure is "just throw some test data in your production environment
[storage] so individual developers can debug code locally". This isn't a
cost-of-storage issue, this is an issue of having multiple developers throw
test data into live Azure Storage is not safe, secure or logical:

1. It's a security issue. Not all developers need/should have access to
   live storage; sometimes clients store sensitive information in our storage.
2. It's a safety issue. What if a developer accidentally overwrites
   critical live data? Or after debugging, deletes the wrong data?

Sure, there are workarounds, awkward as they may be--we could create a
'developer-only' storage account, and ask developers to please prefix their
data with their name, and please don't forget to undo some of that before
submitting for production--but these are half-measures with their own
problems.

I understand that the emulator is lacking some features--but that's a
technical reason, not an excuse for forcing a poor process. Once upon a
time, there wasn't a storage emulator at all--but it was recognized that
asking developers to always connect to a live storage account wasn't a good
idea, so the emulator was created. Now we're in a similar
situation--connecting to live storage still isn't a good idea, so let's
create a solution, not push a flawed "workaround".

—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub
#53 (comment)

David Kreth Allen
612-374-1119

from azure-webjobs-sdk.

I've been following this issue for a while, hoping it would eventually be resolved. At this point the official response from Microsoft seems to be that they have no intention of addressing it outside of pushing the community to fix it. I'm all for community driven solutions, but I find this response disappointing for the following reasons:

1. This issue has been open for almost two years.
2. WebJobs is a supposedly supported and highly integrated feature of Microsoft Azure (a paid service), that has little to no utility outside of Microsoft Azure (a paid service). As such I think Microsoft has an internal responsibility to address any fundamental shortcomings of WebJobs.
3. Yes, you can work around this issue by using a live storage account. At best this is an annoyance, and at worst this is a barrier to entry for some teams who are unwilling to deal with it. Either way, I would think Microsoft would strive to create a seamless development experience.
4. As you begin to create your first WebJob this is almost certainly the first issue you will run into. It's not a great first impression and likely leaves some developers thinking that the feature set in general is not fully thought through or supported. Again, I would think this is something Microsoft would want to fix.

The lack of action on this issue leaves me wondering what exactly Microsoft's commitment to WebJobs is, if any. I've been using Azure for a long time and always used Worker Roles for background tasks, but when starting my latest project all of the documentation and marketing material (coupled with cloud services being marked "classic" in the portal) really make it seem like WebJobs are the preferred path so that's the path I took. However, the apparent unwillingness of Microsoft to create a fully integrated development experience makes me question what the long term strategy is here. Are WebJobs not fully supported? Should we be using something else for background tasks? Is Microsoft planning to do away with the storage emulator? Why has there been no movement on this issue?

from azure-webjobs-sdk.

Let me blunt since my previous hint was ignored. Any post that isn't about how to make this feature work instead of why we aren't doing it, will be removed. You can email me personally if you'd like to continue to accuse us of being unprofessional or provide feedback that this lack of feature prevented you from using WebJobs SDK. This conversation is distracting to our team and our community members which might actually want to discuss the issue itself.

My email is chrande (at) microsoft (dot) com. Please let us get back on track.

from azure-webjobs-sdk.

I understand that there isn't a security problem if we create separate live accounts for each developer. One final attempt to hopefully increase the priority of this feature:

At the end of the day, requiring constant internet isn't a practical workflow for all developers. Sometimes we're working at client offices, sometimes we're traveling, and trying to figure out why a 20GB blob isn't triggering isn't much fun on a slow or spotty connection.

We CAN sometimes work online, but can't rely on it. Since it looks like this feature is a low priority, we're going to stop using the webjobs SDK, create a standard console app, and partially recreate the SDK triggering in a way that works for us.

Please keep in mind that "local isn't needed, you can just use a live account" doesn't work for everyone, and isn't a great reason to not provide an emulator solution. By the same logic, we don't need a storage emulator at all, or a local SQL database or even IIS Express--we don't need local anything, because we can just test everything on a live site. But this isn't how we work, because local testing is at minimum a huge convenience, and at most necessary for some workflows.

Thanks for the consideration, but we're moving on.

from azure-webjobs-sdk.

Any news on a v2 RTM?

from azure-webjobs-sdk.

@zeeshanejaz : This is resolved with 8.x of the WindowsAzure.Storage library if you grab the latest emulator (v4.6 at time of writing) https://go.microsoft.com/fwlink/?linkid=717179&clcid=0x409

from azure-webjobs-sdk.

Any update on when this issue will be resolved? We are starting a new project, and will be relying heavily on Azure storage with WebJobs - not being able to run them locally is a HUGE issue for us, as we have times where we need to develop/demo our product without internet connections being available.

How has this been able to happen for so long? Many people rely on Azure for large scale apps, and issues like this really slow down development velocity!

from azure-webjobs-sdk.

Should this discussion be flipped around? Instead of getting WebJobs SDK working on the emulator, should the emulator be improved to close the gap between itself and "real" Azure storage?

Doesn't look like the emulator bits are on github however.

from azure-webjobs-sdk.

1. WebJobs SDK is something we're fully committed to supporting. We're not fully committed to supporting every feature that's requested. We are fully committed to helping users and the community add support that we're not willing to take on. I can't speak on the Storage Emulator and why it isn't at full parity, which is the real issue here - not whether the WebJobs SDK team will build a bunch of hacks to support it.
2. What progress would you like to see, beyond us implementing this feature which we've made quite clear we don't think is in a good state to support? We've made it quite clear (at least recently :)) that it's open for others to give a whack at. We have to be reasonable about what work we agree to take on. This is going to be expensive both short term to implement and long term to maintain. We aren't going invest in adding this support anytime soon, regardless of how long the issue is open.
3. You're right that we have a lack of a fully offline development story. Offline development is, unfortunately for the folks in this thread, a non-goal for us, given much of our utility is talking to Cloud based services. Beyond Storage, there is Service Bus, Event Hub, Mobile Apps, Notification Hubs, DocDB, and the list will continue to grow. So you see, even if we built hacks around the Storage Emulator, we couldn't be fully offline. It just doesn't make sense for us to invest it because we can't possibly provide a end to end story for, on top of the fact that adding hacks to support the gaps in the emulator is likely to decrease stability. A lot of red flags go off for us as we thought through how we might add this support.

To be clear, I'm leaving this issue open for two reasons. 1) Storage might patch the gaps in the emulator, then we can add support. 2) The community might invest in finding clever hacks around those gaps. It's not a priority for us, for all the reasons I stated in 3 above.

It'd be great if this thread could move on from "But really, why aren't you supporting this feature" to "Well, I took a stab at this, and found X issues...". I don't imagine any of us will take the time to respond to any more of the former for a while. :)

from azure-webjobs-sdk.

This is one of the many reasons why Microsoft will continue to lose market
share in my opinion. I was chastised for this earlier but I'm going to
repeat myself, it's very unprofessional. If you don't like that opinion
tough. I've been a software professional for 30 years and I've never heard
the types of lame excuses to not fix something that's obviously a problem.
I understand the problem isn't with the web jobs SDK but don't you have
pride in the overall image that Azure has? You keep on talking about
working on a workaround. I work 5 days a week I have a good work-life
balance. I don't have time to look into this on my own and my company is
not going to allow me to Bill that time to my client. To be quite honest
with you because of the limitation, we decided not to use it. I'm also
going to be pretty clear about something else and that is that this
dialogue didn't start until after I put a fire under your butt with my
post.

In my daily work it usually doesn't take humiliation for somebody to do
their job. We work as a cohesive team and we act like adults. No such thing
as pointing fingers no such thing as it wasn't my job no such thing as it
wasn't my code. We all strive to provide value to the client no matter what
that takes within the limits of statement of work, team responsibility and
client needs & wants. As a team we don't tolerate excuses.

The epilogue to this is that I'm not known as a soft-spoken man however I
speak the truth as blunt as it is it is the truth.

Now Let the Flames Begin!
On Apr 30, 2016 10:36 AM, "Christopher Anderson" [email protected]
wrote:

WebJobs SDK is something we're fully committed to supporting. We're
not fully committed to supporting every feature that's requested. We are
fully committed to helping users and the community add support that we're
not willing to take on. I can't speak on the Storage Emulator and why it
isn't at full parity, which is the real issue here - not whether the
WebJobs SDK team will build a bunch of hacks to support it.
2.

What progress would you like to see, beyond us implementing this
feature which we've made quite clear we don't think is in a good state to
support? We've made it quite clear (at least recently :)) that it's open
for others to give a whack at. We have to be reasonable about what work we
agree to take on. This is going to be expensive both short term to
implement and long term to maintain. We aren't going invest in adding this
support anytime soon, regardless of how long the issue is open.
3.

You're right that we have a lack of a fully offline development
story. Offline development is, unfortunately for the folks in this thread,
a non-goal for us, given much of our utility is talking to Cloud based
services. Beyond Storage, there is Service Bus, Event Hub, Mobile Apps,
Notification Hubs, DocDB, and the list will continue to grow. So you see,
even if we built hacks around the Storage Emulator, we couldn't be fully
offline. It just doesn't make sense for us to invest it because we can't
possibly provide a end to end story for, on top of the fact that adding
hacks to support the gaps in the emulator is likely to decrease stability.
A lot of red flags go off for us as we thought through how we might add
this support.

To be clear, I'm leaving this issue open for two reasons. 1) Storage might
patch the gaps in the emulator, then we can add support. 2) The community
might invest in finding clever hacks around those gaps. It's not a priority
for us, for all the reasons I stated in 3 above.

It'd be great if this thread could move on from "But really, why aren't
you supporting this feature" to "Well, I took a stab at this, and found X
issues...". I don't imagine any of us will take the time to respond to any
more of the former for a while. :)

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#53 (comment)

from azure-webjobs-sdk.

I haven't deleted any of your other posts. I just want to get this discussion on track. I'm sorry if you disagree with us, but this is unhealthy and sometimes we have to be aggressive to be back on track. Last warning.

from azure-webjobs-sdk.

I haven't done anything more than a basic test, but removing the check that @davidebbo mentioned allows me to use blob and queue triggers with the emulator. So there's some good news there.

I'm not sure what I'm doing wrong but my test was to just add a console app to the existing solution and then reference WebJobs and WebJobs.Host and upon running my app I kept getting a strong naming error. After unchecking sign the assembly on those two projects the app ran fine.

from azure-webjobs-sdk.

I had made the same changes @xt0rted did, and tried running some of our E2E tests. I was getting some test failures, so I shelved it temporarily until I had more time to dig deeper.

However, I just tried @xt0rted's PR locally and basic Queue/Blob scenarios do indeed work. So I think what we can do is take the PR and let people start playing with this. Some things may not work, and we can address those issues one by one as they arise. The emulator may not cover all bases, but that's OK. I'll work on merging the PR in tomorrow :)

from azure-webjobs-sdk.

Could this feature be added to a 1.x release rather than waiting for 2.0?

This has been a known issue for a long time and has put a burden on developers wanting to use queues locally! Pre-Release packages are fine when checking out new features or getting to know a new SDK, but developing against them for production apps is far from ideal!

If 2.0 is not right around the corner from being release (judging from the milestone, it seems far from it!), please release a patch to fix this issue for the current 1.x.

from azure-webjobs-sdk.

from azure-webjobs-sdk.

I personally think it is ridiculous and unprofessional for this pretty major issue to go 2 years without being resolved or at least seriously looked at, I have plenty of evidence to support this claim:

1. This issue is not even assigned to anyone
2. The milestone hasn't moved
3. No labels have been applied

I mean, what gives??? If you are NOT going to fix the issue then say so and then allow someone from the community to come up with a work around solution.

from azure-webjobs-sdk.

I've never worked in a development environment where the official testing procedure is "just throw some test data in your production environment [storage] so individual developers can debug code locally". This isn't a cost-of-storage issue, this is an issue of having multiple developers pollute live data:

1. It's a security issue. Not all developers need/should have access to live storage, which might contain sensitive client information.
2. It's a safety issue. What if a developer accidentally overwrites critical live data? Or after debugging, deletes the wrong data?

Sure, there are workarounds, awkward as they may be--we could create a 'developer-only' storage account, and ask developers to please prefix their data with their name, and please don't forget to undo some of that before submitting for production--but these are half-measures with their own problems.

I understand that the emulator is lacking some features--but that's a technical reason, not an excuse for forcing a poor process. Once upon a time, there wasn't a storage emulator at all--but it was recognized that asking developers to always connect to a live storage account wasn't a good idea, so the emulator was created. Now we're in a similar situation--connecting to live storage still isn't a good idea, so let's create a solution, not push a flawed "workaround".

from azure-webjobs-sdk.

Thanks @xt0rted for bringing this thread back on track! This is exactly what helps issues move forward.

Recently, when @mathewc tried it, he ran into some issues, though I'm not sure what they were. @mathewc do you remember?

My view here is that even if it doesn't work 100%, we should remove the check, as @xt0rted established that some scenarios do work. For scenarios that don't work, we can open specific issues to track them , and take it from there.

from azure-webjobs-sdk.

@RickRockhold, please read the latest comment from @mathewc. At this point, the discussion is strictly on the specific steps that will make this go forward, and no longer on the meta discussion about whether it's a worthy thing to do. Thanks!

from azure-webjobs-sdk.

The 2.0.0-beta1 packages have now been uploaded to nuget here. That includes the changes for this issue. Please give it a try, thanks.

from azure-webjobs-sdk.

Quick question @mathewc - what version will this PR be included in? As in ... the official release, not a pre-release?

Also: maybe the label/milestones (for this issue) could be updated to help clarify my question, too?

from azure-webjobs-sdk.

Even a downgraded experience would be extremely useful.

from azure-webjobs-sdk.

I've been able to work around this by including an #if DEBUG statement to manually pull from a queue instead of the lack of support for the JobHost class (and corresponding QueueTrigger) in development.

https://gist.github.com/gweinhold/58f4e928378ab6329a3f

from azure-webjobs-sdk.

My problem is that I am working in a team currently working with the Storage Emulator was easy for us now I need to setup all members of my team with proper Azure Subscriptions or find a way to all work on the same Subscription but different Storage Account or Queues. Storage Emulator would be easier than configuring all our web apps and tooling to work in Azure in separated environments.

from azure-webjobs-sdk.

Sorry for the lack of a clear answer here. It's a tricky issue because, for one reason or another, the SDK was never designed with the emulator in mind and there are a lot of gaps between the emulator and the actual Storage products which would impact the SDK.

I don't think that this is a good time for the Azure WebJobs team to investigate this any further, thus I'm moving this into backlog for the time being.

With that said, we're always open for community requests on issues we haven't closed. If you're interested in investigating the gaps and finding work arounds for them, we're happy to work with you to do that in the best way possible.

You can always reach out to me at chrande (at) microsoft (dot) com if you want to discuss this or any other feature and its relative priority.

from azure-webjobs-sdk.

I will not apologize for speaking up about a dated issue that was clearly neglected. I will say that my response elicited a useful response for the community so for that I am thankful.

from azure-webjobs-sdk.

True, but it's yet one more thing to configure, requires internet, etc. Of course, the same argument could be made for storage in general--why is there a storage emulator at all, if working with live, online test accounts is so painless?

from azure-webjobs-sdk.

I started out using the emulator, but as soon as I hit this issue (unable to use storage triggers when testing my webjobs) I abandoned it for a test account on azure. At this point the only thing the storage emulator is used for is some basic integration tests on the build server.

from azure-webjobs-sdk.

@PureKrome Yes, it'll be the 2.0.0 release. Note that we've released 2.0.0-beta1 to nuget.org as well. We'll push additional builds there (as well as nuget.org) as we move forward.

from azure-webjobs-sdk.

I vote that we leave this issue and open new issues as needed for new emulator problems. This issue was all about removing some explicit restrictions we had in place, which we've done. This will make it easier to track things for us, thanks.

from azure-webjobs-sdk.

We hear you guys, and we'll try to take a look at this soon. While there are some features of storage the SDK leverages (e.g. Blob logs) that don't work in the emulator, we may be able to provide a downgraded experience when running locally (you can probably live w/o having BlobTrigger trigger immediately for new blobs, which requires Blob logs).

from azure-webjobs-sdk.

I've just moved this from backlog to the v1.2.0 milestone for "soonish" investigation.

from azure-webjobs-sdk.

@fwd079 try going to a Linux issue and call people unprofessional because you think it's not getting fixed fast enough. Let's see how that works for you :)

Joking aside, I know this OSS thing is new to some of you, but let's try to make this thread productive. As outlined in my previous entry, there are concrete actionable steps that anyone could take to make this go forward and get a sense of where the Emulator stands. Anyone wants to take that on?

from azure-webjobs-sdk.

Brilliant to see some activity in this thread and some great discussion,
but it's obviously a left behind project and needs serious work that no one
company would like to pay for...
Crowd funding could be a good route to accommodate different financial
contributions/budgets and feature needs?
Very successful in some OSS projects I've utilised.

Just my 2c.

Tyeth.
On 29 Apr 2016 16:35, "RickRockhold" [email protected] wrote:

I've never worked in a development environment where the official testing
procedure is "just throw some test data in your production environment
[storage] so individual developers can debug code locally". This isn't a
cost-of-storage issue, this is an issue of having multiple developers throw
test data into live Azure Storage is not safe, secure or logical:

1. It's a security issue. Not all developers need/should have access to
   live storage; sometimes clients store sensitive information in our storage.
2. It's a safety issue. What if a developer accidentally overwrites
   critical live data? Or after debugging, deletes the wrong data?

Sure, there are workarounds, awkward as they may be--we could create a
'developer-only' storage account, and ask developers to please prefix their
data with their name, and please don't forget to undo some of that before
submitting for production--but these are half-measures with their own
problems.

I understand that the emulator is lacking some features--but that's a
technical reason, not an excuse for forcing a poor process. Once upon a
time, there wasn't a storage emulator at all--but it was recognized that
asking developers to always connect to a live storage account wasn't a good
idea, so the emulator was created. Now we're in a similar
situation--connecting to live storage still isn't a good idea, so let's
create a solution, not push a flawed "workaround".

—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub
#53 (comment)

from azure-webjobs-sdk.

Thanks for fixing this. Is there an ETA when the stable package will be out? Last release was in April, just before this fix.

from azure-webjobs-sdk.

Ho hum... scratch that, it supports version 7+ :))

A new lesson, go see and be pleased ;-)
Time to break off the rust and take a test drive :)

from azure-webjobs-sdk.

Yes, I confirmed that a default package retention policy that we never created was in effect, and it was configured to retain only 20 versions. I've removed that restriction so now all versions will be kept going forward. Thanks for reporting this :)

from azure-webjobs-sdk.

@zeeshanejaz sadly it never made it into v1. It's in v2 but there hasn't been a stable release yet. You can get it on NuGet in one of the the 2.0.0-beta2 builds. I've been using the preview builds ever since this was merged and haven't had any issues with them.

It would be nice if this could be pulled into v1 since we've been waiting 6 months for a release and there hasn't been one.

from azure-webjobs-sdk.

Any update on release date for V2 - This fix is much needed by many people!

from azure-webjobs-sdk.

Yes, I want this issue to die so we can start fresh, and track any new issues people have in new issues :) However, note that if you're running into bugs in the emulator or limitations with the emulator, we can't fix those - you should instead log bugs against the storage SDK / emulator itself. Our statement on emulator support as resolved for this issue is that we no longer block it as we were before, however we make no strong guarantees on how well it will work for you. E.g. we know there are features of the WebJobs SDK that use storage features that the emulator doesn't support (e.g. blob leases, at least in the past that was an issue) so YMMV. I view the emulator as a way to try out simple scenarios, however I always use a full test account for my work.

from azure-webjobs-sdk.

I found the correct way to connect, I have not seen any similar comments so I share it.

<add name="AzureWebJobsDashboard" connectionString="DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;BlobEndpoint=http://127.0.0.1:10000/devstoreaccount1;TableEndpoint=http://127.0.0.1:10002/devstoreaccount1;QueueEndpoint=http://127.0.0.1:10001/devstoreaccount1;" /> <add name="AzureWebJobsStorage" connectionString="DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;BlobEndpoint=http://127.0.0.1:10000/devstoreaccount1;TableEndpoint=http://127.0.0.1:10002/devstoreaccount1;QueueEndpoint=http://127.0.0.1:10001/devstoreaccount1;" />

Note: Not work with UseDevelopmentStorage=true

https://docs.microsoft.com/en-us/azure/storage/common/storage-configure-connection-string?toc=%2fazure%2fstorage%2fqueues%2ftoc.json

from azure-webjobs-sdk.

I seem to be experiencing this with Emulator 4.0.0.0. Has this issue been resolved yet?

from azure-webjobs-sdk.

Not yet. The SDK only works with Azure Storage

from azure-webjobs-sdk.

Are there any known workarounds for this?

from azure-webjobs-sdk.

Unfortunately no. Local emulator is missing features that the SDK requires to function (e.g. blob logs).

from azure-webjobs-sdk.

+1

from azure-webjobs-sdk.

+1 We need this ASAP.

Forcing developers to pay for azure storage during their development is not cool when we used to be able to do it locally on our dev machines.

from azure-webjobs-sdk.

Thanks gweinhold, this is a great workaround

from azure-webjobs-sdk.

@mathewc Any news on this?

from azure-webjobs-sdk.

I would use this. But it's not essential as I can use Azure accounts in the meantime.

from azure-webjobs-sdk.

👍 Cool, it will be true dev experience if supported locally. Thanks

from azure-webjobs-sdk.

I can't use Azure to test due to multiple MS accounts that are f-d and still linked to my card, so the local emulator is my only hope. Shame it's lacking... Maybe time to consider Azure/C# alternatives, everyone knows and loves JS

from azure-webjobs-sdk.

I tried this link:
https://github.com/Azure/azure-webjobs-sdk/wiki/Running-Locally

And my code is:

Environment.SetEnvironmentVariable("AzureWebJobsEnv", "Development");
            if (!VerifyConfiguration())
            {
                Console.ReadLine();
                return;
            }
            CreateDemoData();
            var config = new JobHostConfiguration();

            if (config.IsDevelopment)
            {
                config.UseDevelopmentSettings();
            }

            JobHost host = new JobHost(config);
            host.RunAndBlock();

But it failed with this :

Exception thrown: 'System.InvalidOperationException' in mscorlib.dll

Additional information: Failed to validate Microsoft Azure WebJobs SDK Dashboard account. The Microsoft Azure Storage Emulator is not supported, please use a Microsoft Azure Storage account hosted in Microsoft Azure.

If I stop at: if (config.IsDevelopment) the value is true but I still get the exception.

Please can you help? Thanks.
.

from azure-webjobs-sdk.

When developing my WebJob with a local development queue, mscorlib.dll is throwing an InvalidOperationException with message "Failed to validate Microsoft Azure WebJobs SDK Dashboard account. The Microsoft Azure Storage Emulator is not supported, please use a Microsoft Azure Storage account hosted in Microsoft Azure".

So forcing me to use an Azure hosted Queue when developing my Web Job seems a rather awkward step I must take and goes against the tenant of the Azure SDK's capabilities of emulating Azure Storage capabilities locally.

Please can you provide a time frame when I can develop my WebJob without a dependency on a live Azure Storage account?

from azure-webjobs-sdk.

First, I do want to comment on this statement:

I mean, what gives??? If you are NOT going to fix the issue then say so and then allow someone from the community to come up with a work around solution.

It's really quite the other way around. Please note that this is an OSS project. Generally, any issue that is open and clearly not being worked on is up for grab by the community. It all comes down to someone having the cycles and the desire to do it. And @ScottyMac52 I will suggest that you use a different tone, as this kind of comments is not the best way to make things move forward productively in an open source project.

If someone wants to play around with this, the first this you should try to do is remove this check, which currently blocks the use of the emulator. Frankly, at this point no one seems sure what's wrong with the emulator. The SDK has evolved, and the emulator might have improved as well. So let's see how it fares without the check, and take it from there.

from azure-webjobs-sdk.

Not a problem with speaking up, Scotty, but we did perceive your post as being overly aggressive. It's fine to keep the pressure on us, but we'd prefer to keep a friendly tone inside our communities so that everyone feels welcome to contribute and discuss the issues.

You can always send the aggressive comments to me directly if that's more your style. ;-)

from azure-webjobs-sdk.

Well Scotty said nothing wrong.
Its surprising to know that SDK was never designed for local environment, but if you perceived Scotty's remarks to be "overly aggressive" then you have a lot to learn from open source environment.

The open source community doesn't hang about. they move on and get the job done, that is my observation. Or the project is ditched/picked up by other Open Source community. Ubuntu, and other *nix communities and platforms aren't that slow. Unlike corporate networks.

So if someone has vented some frustration and that frustration has pushed someone to come here and at least put a tag (albeit an undesirable one) the I say good on them, for at least being successful in generating some response.

I'll patiently await for a workaround/solution to this.

from azure-webjobs-sdk.

Its not new to me David. I have both Win 10 and Ubuntu, so perhaps its new to folks used to only one tech and platform lol. The reasons if there are any, as I said, baffles anyone who reads this.

But yes agreed, I'll await solution by someone or a workaround.

from azure-webjobs-sdk.

Were you actually seeing the WebJobs SDK trigger on new Blobs? (I ask this because we don't expect it to work)

from azure-webjobs-sdk.

@mathewc Thank you very much for dierctly looking into it, much appreciated.

from azure-webjobs-sdk.

Just to jump on the activity, is there an eta and link for the prerelease,
having trouble using nuget search...
Thanks in advance for your assistance.

On 14 July 2016 at 06:48, Zachary Sims [email protected] wrote:

Thanks for fixing this. Is there an ETA when the stable package will be
out? Last release was in April, just before this fix.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
#53 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/AGYc8_Ogkr2XjmJWHcPJMfYZESicym2zks5qVc2-gaJpZM4CVjrK
.

from azure-webjobs-sdk.

Hi guys, I agree that some flames need to be consigned to a black hole somewhere whilst others should serve as an example of what is unacceptable, as, diplomatic coercion can be very effective. In my 30yrs I've seen lots of forums, complained in quite a few as well and have observed constant, the worse the flaming, the slower the progress. Quite often I've had to solve stuff for myself, e.g., using the MySQL to .net 1.1 connector and getting it to consistently work with datasets. In an open source world such as we see here, I might have shared that. So, back to the business of the thread...

Thanks for the info Mathew and I know it's Friday but I'm wondering if this is the anticipated fix that removes the requirement for azure storage 4.3 to 5.x that was in v1.1? I've just updated to SDK v1.2 via nuget and I don't see any version limitations on WindowsAzure.Storage so being the deeply trusting person that I sometimes am not, I thought I had better hunt for some clarification.

from azure-webjobs-sdk.

I've been using 2.0.0-alpha-10313 which is one of the first builds to include this change. I haven't tried anything newer since I don't want to risk running a buggy build.

from azure-webjobs-sdk.

Based upon Matthew's comment I'm guessing it will be 2.0.0 but I just wanted to hear it from the owners.

Also @xt0rted - nice work on the PR and helping ... and get into the JabbR slack room more often :)

from azure-webjobs-sdk.

Thanks kindly @mathewc :) Looking forward to the 2.0 RTM/RTW.

Keep up the good work, MS / webjobs-team. 🍰

from azure-webjobs-sdk.

Watch out if you're using the myget feed to get this update. My builds just started failing because older packages were removed from the feed.

from azure-webjobs-sdk.

@xt0rted What myget package are you saying doesn't work? Both the nuget and myget packages should work just fine. I'm not aware of any issues :)

from azure-webjobs-sdk.

@mathewc I was using version 2.0.0-alpha-10313 of the Microsoft.Azure.WebJobs package. My build server started failing on the 19th so this package was probably removed earlier that day or later on the 18th when 2.0.0-beta1-10381 was published.

from azure-webjobs-sdk.

I think myget is doing auto-purge on pre-release versions - we haven't been deleting those ourselves. We need to look into this more - might be something we can configure.

from azure-webjobs-sdk.

How come this issue is closed? I don't see the emulator support in the SDK :-/ Am I missing something?

from azure-webjobs-sdk.

I'm not getting this to work at all using 2.0.0-beta2, Emulator v4.5. Should I open a new issue?

error: System.InvalidOperationException: 'Invalid storage account 'devstoreaccount1'. Please make sure your credentials are correct.'

string: "UseDevelopmentStorage=true;"

Total loss here, reading this thread it seems this should totally work.

from azure-webjobs-sdk.

Yes, if you're having issues with the emulator in the beta builds, please create a new issue.

from azure-webjobs-sdk.

In my experience WebJobs SDK 2.0.0-beta2 works with Emulator if you are using WindowsAzure.Storage 7.2.1 but doesn't work if you are using WindowsAzure.Storage 8.0.1. In the latter case, it throws up an exception "Invalid storage account: 'devstoreaccount1'".

I hope this helps someone

from azure-webjobs-sdk.

I'm running emulator v 4.6. Still does not work at least from VS 2015 ASP.NET MVC 4.6.2 project.
WindowsAzure.Storage v 8.1.1

Same error "Failed to validate Microsoft Azure WebJobs SDK Dashboard account. The Microsoft Azure Storage Emulator is not supported, please use a Microsoft Azure Storage account hosted in Microsoft Azure." from my WebJob

connection strings
<connectionStrings> <add name="AzureWebJobsDashboard" connectionString="UseDevelopmentStorage=true" /> <add name="AzureWebJobsStorage" connectionString="UseDevelopmentStorage=true" /> </connectionStrings>

from azure-webjobs-sdk.

This seems to work reasonably well with 4.6. However, if you give it a reasonable amount of blobs to process in parallel - say, 10, it often just pops with a 500 server error. Not been able to trace it to anything more than that.

from azure-webjobs-sdk.

Still not working for me on .net 4.6 with the latest storage SDK & emulator. I can't believe that this has been an issue for 3 years.

from azure-webjobs-sdk.

The latest emulator seems to have some breaking changes in it as well - at last one has broken the F# Storage Type Provider - that lead to null reference exceptions being thrown.

from azure-webjobs-sdk.

I'm using version 4.6 in VS 2015, and I can connect (no exceptions), however I don't know how I can define to which storage account I want to connect to.

Note that my local storage account is define like: http://127.0.0.1:10000/devstoreaccount1/webjob-test

And my handler method looks like:

public static void RenameFile(
    [BlobTrigger("input/{blobName}.exe")] Stream input,
    [Blob("output/processed_{blobName}_final.exe", FileAccess.Write)] Stream output)
{
    input.CopyTo(output);
}

Can you please help me?

from azure-webjobs-sdk.

@MatthewC do you want me to raise another one then related to many blobs created at once?

from azure-webjobs-sdk.

Regarding the invalid operation exception:

Exception thrown: 'System.InvalidOperationException' in mscorlib.dll

Additional information: Failed to validate Microsoft Azure WebJobs SDK Dashboard account. The Microsoft Azure Storage Emulator is not supported, please use a Microsoft Azure Storage account hosted in Microsoft Azure.

One way to get around this limitation is to create a Console app rather than an Azure WebJob project. You can then add the Microsoft.Azure.WebJobs nuget package manually, add the standard WebJob boilerplate code to Program.cs and everything should just work.

Visual Studio even lets you publish it as an Azure WebJob like normal via the right-click menu (at least VS 2017 does). It also runs just fine in Azure.

from azure-webjobs-sdk.