One of the reasons for using HTTPS is that you can be sure that there is no man-in-the-middle, intercepting and potentially altering your requests.

However, the Azure::Core::Http::HttpRequest contains this alarming code:

          if uri.scheme.downcase == 'https'
            # require 'net/https'
            http.use_ssl = true
            http.verify_mode = OpenSSL::SSL::VERIFY_NONE
          end

It appears that the server's SSL key is not verified, which means that if I can hijack your router, then all your HTTPS traffic is as open as HTTP traffic.

The code base has two versions of SharedKey and SharedKeyLite, but only has unit tests for one version.

c:\dd\git\jcooke\azure-sdk-for-ruby>grep -lr SharedKey * | grep -v doc/
lib/azure/core/auth/authorizer.rb
lib/azure/core/auth/shared_key.rb
lib/azure/core/auth/shared_key_lite.rb
lib/azure/core/signed_service.rb
lib/azure/service/storage_service.rb
lib/azure/table/auth/shared_key.rb
lib/azure/table/auth/shared_key_lite.rb
lib/azure/table/table_service.rb
test/unit/core/auth/shared_key_lite_test.rb
test/unit/core/auth/shared_key_test.rb

We should consolidate these two versions.

There are many places in our code where user strings get passed directly to the XML serializer. For example:

subject { Azure::ServiceBus::ServiceBusService.new }
subject.create_queue queue_name, {
  :dead_lettering_on_message_expiration =>
     "\u2488\u2460\u216B\u3128\u3129".encode("GB18030"),
}

This leads to this console output:

output error : string is not in UTF-8

and the XML output in the request:

<QueueDescription ...>
  <DeadLetteringOnMessageExpiration></DeadLetteringOnMessageExpiration>
</QueueDescription>

Notice that the inner tag is empty, which could lead to confusion to people trying to debug issues.

EXPECT: As a part of our XML serializer, we automatically convert strings to UTF-8 (unless they are some binary format that should remain unchanged.)

• Storage Service Versioning: http://msdn.microsoft.com/en-us/library/windowsazure/dd894041.aspx
• Service Management Versioning: http://msdn.microsoft.com/en-us/library/windowsazure/gg592580.aspx

No public document about service bus.

The Azure REST API doesn’t define properties like 'uri' in the response. See http://msdn.microsoft.com/en-us/library/windowsazure/jj732965.aspx. I see they are defined in the .NET client library. See http://msdn.microsoft.com/en-us/library/windowsazure/jj732965.aspx. And there are a bunch of other properties defined in the .NET client library.

I guess if we want to add this property, we need to generate the uri by ourselves. We can consider this in the next release.

It's a common scenario to convert an object from/to json in Ruby applications. Usually people will expect a to_json method on an object. We should consider add it to the model classes we have.

Following is what you will get as the return value of create_subscription().

author_name: 
description: {}
id: 
name: test-subscription-1
published: 
updated: 
dead_lettering_on_message_expiration: 
default_message_ttl: 
enable_batched_operations: 
lock_duration: 
max_delivery_count: 
message_count: 
ordered_props: ["LockDuration", "RequiresSession", "DefaultMessageTimeToLive", "DeadLetteringOnMessageExpiration", "DeadLetteringOnFilterEvaluationExceptions"]
requires_session: 

However, if you call get_subscription(), it will return

author_name: 
description: {"LockDuration"=>"PT1M", "RequiresSession"=>"false", "DefaultMessageTimeToLive"=>"P10675199DT2H48M5.4775807S", "DeadLetteringOnMessageExpiration"=>"false", "DeadLetteringOnFilterEvaluationExceptions"=>"true", "MessageCount"=>"0", "MaxDeliveryCount"=>"10", "EnableBatchedOperations"=>"true"}
id: sb://rorvmtest.servicebus.windows.net/test-topic-1/subscriptions/test-subscription-1
name: test-subscription-1
published: 2013-01-09 01:33:30 UTC
updated: 2013-01-09 01:33:30 UTC
dead_lettering_on_message_expiration: false
default_message_ttl: P10675199DT2H48M5.4775390625S
enable_batched_operations: true
lock_duration: PT1M
max_delivery_count: 10
message_count: 0
ordered_props: ["LockDuration", "RequiresSession", "DefaultMessageTimeToLive", "DeadLetteringOnMessageExpiration", "DeadLetteringOnFilterEvaluationExceptions"]
requires_session: false

which is the expected result.

Dev Estimate: 2
Test Estimate: 1
Ranking: 200

To be GB-18030 compliant, the Ruby SDK needs to support Hangul characters. For example, "다", a.k.a. Unicode codepoint U+B2E4, represented as 0xEB 0x8B 0xA4 in UTF-8.

When I update Azure::Storage::Table::TableService::#insert_entity with this code:

    let(:entity_properties){ 
      { 
        "CustomStringProperty" => "\xEB\x8B\xA4 U+B2E4",
      }
    }

The XML emitted looks good (at first glance), Ruby is smart enough to encode the character using XML escaping:

    <m:properties>
      <d:PartitionKey m:type="">testingpartition</d:PartitionKey>
      <d:RowKey m:type="">abcd123</d:RowKey>
      <d:CustomStringProperty m:type="Edm.Binary">&#xB2E4; U+B2E4</d:CustomStringProperty>
    </m:properties>

But it is wrong, the type is "BINARY", when it should be treated as a STRING. The Azure service notices this and is not happy about the switch:

HTTP/1.1 400 Bad Request
  <code>InvalidInput</code>
  <message xml:lang="en-US">One of the request inputs is not valid.</message>

It is incorrect to treat this data a binary; that will impact interop with other languages, if multiple SDKs are targeting the same table store.

I'm not sure how to fix this issue, it is more of a design flaw.

Like in #295, without re-encoding property names, they will be embedded in the output XML with their original encoding, so any GB-18030 encoded string will not be interpreted correctly by the server.

        prop = "prop" + [0x3128, 0x3129].pack('U*').encode!("GB18030")
        entity_properties = { 
          "PartitionKey" => "x",
          "RowKey" => k
          prop => "value",
        }
        result = subject.insert_entity table_name, entity_properties

The XML going out across the wire retains is original GB-18030 encoding (0xA8 0xE8 0xA8 0xE9), which causes the server to complain:

HTTP/1.1 400 Bad Request
...
  <code>InvalidInput</code>
  <message xml:lang="en-US">One of the request inputs is not valid.
RequestId:2c079ac8-c761-4227-8fe1-24f3291fa8e3
Time:2013-03-18T17:55:23.8220736Z</message>
</error>

If I upload a GB-18030 encoded string to a blob, then download it, it will be considered different, because the encoding information is lost.

content = "啊阿鼾齄".encode!("GB18030")
subject.create_block_blob container_name, blob_name, content, options
blob, returned_content = subject.get_blob container_name, blob_name
p returned_content.encoding
p returned_content
p content.encoding
p content
returned_content.must_equal content

This fails and outputs:

#<Encoding:ASCII-8BIT>
"\xA2\xB1\xA2\xD9\xA2\xFC\xA8\xE8\xA8\xE9"
#<Encoding:GB18030>
"\x{A2B1}\x{A2D9}\x{A2FC}\x{A8E8}\x{A8E9}"

This could be mitigated by the user by adding code this this:

options = { :content_encoding=>"GB18030" }
...
returned_content.force_encoding(blob.properties[:content_encoding])

But it would be a nice feature if the create_*_blob and get_blob methods could automatically add the encoding info to\from strings going in\out of the blob.

This can be done after the initial release

According to Specifying Conditional Headers for Blob Service Operations, the blob's acquire_lease method should support the conditional headers.

I assembled the following table from that Web page for auditing the PHP and Java blobs; it might be helpful for auditing the Ruby methods too:

For example, I just used it to find that the delete_blob is missing :if_modified_since and :if_unmodified_since.

It is possible for an HTTP call to never return. When that happens, the client needs to know when to give up, using a client-side timeout. See this Java code that adds the client-timeout support for ideas.

Note: PHP does not have this feature, and neither did Java for the first 8 months.

There are a few places where the an entity's keys are placed into the URL, which all appear to go through this:
``ruby
return table_name if table_name.kind_of? ::URI

    path = if partition_key && row_key
      "%s(PartitionKey='%s',RowKey='%s')" % [
        table_name, encodeODataUriValue(partition_key), encodeODataUriValue(row_key)
      ]

However, `encodeODataUriValue` simply URI encodes the value without first converting to UTF-8, which is required by by the server.

To make this concrete, when uploading this value:
```xml
    <m:properties>
      <d:PartitionKey>value⒈①Ⅻㄨㄩ</d:PartitionKey>
      <d:RowKey>value⒈①Ⅻㄨㄩ</d:RowKey>
    </m:properties>

Then when querying for this using get_entity, using the GB-18030 encoding, have this URL:

http://...cscizktpdn(PartitionKey='value%A2%B1%A2%D9%A2%FC%A8%E8%A8%E9',RowKey='value%A2%B1%A2%D9%A2%FC%A8%E8%A8%E9')

which is remains encoded as GB-18030, instead of UTF-8.

Blob test code should use blob clean methods, not table clean methods.

$ grep -r TableNameHelper * | grep -v doc
test/integration/storage/blob/blob_metadata_test.rb:  after { TableNameHelper.clean }
test/integration/storage/blob/blob_pages_test.rb:  after { TableNameHelper.clean }
test/integration/storage/blob/blob_properties_test.rb:  after { TableNameHelper.clean }
test/integration/storage/blob/block_blob_test.rb:  after { TableNameHelper.clean }
test/integration/storage/blob/container/container_acl_test.rb:  after { TableNameHelper.clean }
test/integration/storage/blob/container/container_metadata_test.rb:  after { TableNameHelper.clean }
test/integration/storage/blob/container/create_container_test.rb:  after { TableNameHelper.clean }
test/integration/storage/blob/container/delete_container_test.rb:  after { TableNameHelper.clean }
test/integration/storage/blob/container/get_container_properties_test.rb:  after { TableNameHelper.clean }
test/integration/storage/blob/container/list_containers_test.rb:  after { TableNameHelper.clean }
test/integration/storage/blob/copy_blob_test.rb:  after { TableNameHelper.clean }
test/integration/storage/blob/create_blob_snapshot_test.rb:  after { TableNameHelper.clean }
test/integration/storage/blob/create_page_blob_test.rb:  after { TableNameHelper.clean }
test/integration/storage/blob/delete_blob_test.rb:  after { TableNameHelper.clean }
test/integration/storage/blob/get_blob_test.rb:  after { TableNameHelper.clean }
test/integration/storage/blob/informative_errors_test.rb:    after { TableNameHelper.clean }
test/integration/storage/blob/list_blobs_test.rb:  after { TableNameHelper.clean }
<rest are for table tests>

The Table integration tests run slower and slower as the tests progress. I took a look at what was happening with Fidder, and saw that the same tables were getting deleted over and over again.

It looks like the issue is that there is a single stateful TableNameHelper helper class that stores all table names in an array, which it can then delete. This is fine, but the array doesn't get cleared. Some fixes:

1. Perhaps the code in name_generator.rb should be removing the items from the array, but it fails for some reason (I don't know enough Ruby to be sure about the details of this code):

     @names.reject! do |name|
       @cleanup_proc.call(name)
     end

2. Could just empty the array explicitly at the end of the loop, which is just straightforward:

     @names = []

The Azure::Core::Configuration class only supports the main Azure Service Bus endpoints:

      def acs_host
        "https://#{acs_namespace}-sb.accesscontrol.windows.net"
      end
      def service_bus_host
        "https://#{acs_namespace}.servicebus.windows.net"
      end

However, Service Bus can be hosted in a private cloud, in which case alternate endpoints need to be specified. For example, in addition to just specifying the namespace, like this:

set AZURE_SERVICEBUS_NAMESPACE=jcooketest
set AZURE_SERVICEBUS_ACCESS_KEY=CgX...
set AZURE_SERVICEBUS_ISSUER=owner

we should be able to set the endpoints explicitly, like this:

set AZURE_SERVICEBUS_URI=https://jcooketest.servicebus.windows.net/
set AZURE_SERVICEBUS_WRAP_URI=https://jcooketest-sb.accesscontrol.windows.net/WRAPv0.9
set AZURE_SERVICEBUS_ACCESS_KEY=CgX...
set AZURE_SERVICEBUS_ISSUER=owner

Also, the service_bus_host should include the WRAP0.9 part, instead of hardcoding that into the lib/azure/service_bus/auth/wrap_service.rb

The SqlFilter and SqlRuleAction classes have just two properties, and only one that is commonly set, the SqlExpression property. However, the constructors for those classes only take hashes, and there is no doc comments describing what should be passed. I had to look at the source code to figure out that I need to type:

rule4a.action = Azure::ServiceBus::SqlRuleAction.new(
  {
    "SqlExpression" => 'SET trueRuleA=TRUE; '
  })
rule3.filter = Azure::ServiceBus::SqlFilter.new(
  { 
    "SqlExpression" => 'name LIKE \'%3\' OR user.even = TRUE'
  })

As part of Ruby's design is terseness, I would expect that there would be an "overload" on those constructors like:

      def initialize(hash=nil)
      def initialize(sql_expression, compatibility_level = 20)

Then I could write code like this:

rule4a.action = Azure::ServiceBus::SqlRuleAction.new('SET trueRuleA=TRUE;')
rule3.filter = Azure::ServiceBus::SqlFilter.new('name LIKE \'%3\' OR user.even = TRUE')

The service_code has several public methods that take a single *p parameter, to allow overloading parameters:

• _rule_args takes (topic_name, subscription_name, rule_name) as strings, or (rule) a object with those properties, and an options instance.
• _subscription_args takes (topic_name, subscription_name) as strings, or (subscription) a object with those properties, and an options instance.

This is convenient in some circumstances, but I worry about how it is inconsistent. I would expect one of the following:

1. All methods take a fixed list of parameters, with no overloads.
2. All methods that take parts of existing classes as inputs also take those classes as inputs in place of the individual parts.

Currently, we have a mix, at least 90% in case 1, and just a few in case 2. This makes it hard to reason about how to use the API.

Suggestions, in order of my preference:

1. Remove the overloads for Service Bus methods, for consistency.with the rest of the Blob, Queue, and Table APIs. After shipping, we can figure out how to consistently add overloads.
2. Leave as is.
3. Add overloads everywhere now, before shipping. (But this is crazy.)

Our SDK does not specify which version of the dependencies it requires. So we should make sure that the SDK keeps working with current versions.

Pri 0: just add travis support

After running rdoc, the ...\azure-sdk-for-ruby\doc\Azure.html file's "Class and Module Index" list has duplicate entries for the following items:

• Azure::ServiceBus::ServiceBusService
• Azure::Storage::Blob::BlobService
• Azure::Storage::Queue::QueueService
• Azure::Storage::Table::TableService

Ternary operators are great in limited doses, but nesting them leads to hard-to-maintain code. Two places that I found in the table code are:

lib/azure/storage/table/batch_response.rb:          
  (match_boundary and not match_end)? :boundary : (match_end? :end : nil)
lib/azure/storage/table/serialization.rb:              
  properties[prop.name] = prop.text != ""? cast_property(prop.text, prop["type"]) : prop['null']? nil : ""

These should be refactored.

According to Naming and Referencing Containers, Blobs, and Metadata

Certain characters must be percent-encoded to appear in a URL, using UTF-8 (preferred) or MBCS.

So, if we have strings with the encoding set to something other than UTF-8, theny first need to be converted to the correct encoding. For example,

# Chinese2B1 characters, encoded using GB-18030
test_name = [0x2488, 0x2460, 0x216B, 0x3128, 0x3129].pack('U*').encode!("GB18030")
p test_name
p test_name.encoding 

subject.create_block_blob container_name, test_name, "hi"
blobs = subject.list_blobs container_name
blobs.each { |v|
  v.name.must_equal test_name
}
subject.delete_blob container_name, test_name

The test fails because the blob names don't correctly roundtrip.

The SB tests have descriptions like this:

describe "ServiceBus Queues" do

while the storage tests are like this:

describe Azure::Storage::Blob::BlobService do

A consistent naming hierarchy makes it easier to parse the results of the tests.

Azure::Core::Http::HttpRequest:: default_headers :: when the body is nil
FAIL (0:00:01.015) 0001 leaves the Content-Type, Content-Length, and Content-MD5 headers blank
Expected "" to be nil.
@ (eval):6:in must_be_nil' test/unit/core/http/http_request_test.rb:60:inblock (4 levels) in <top (required)>'

• Cut release branch
• Publish the ruby gem (make Microsoft the owner of the azure rubygem)
• Create WebPI feed (Chain up the ruby environment - ruby runtime, powershell, etc. as of node)
• Smoke test the rubygem and WebPI feed
• Ruby dev center goes live
• Merge and release branch to master and tag the branch
• Make azure-sdk-for-ruby public
• Marketing

It does not appear that the Rule constructor can be used to the Filter and Action, but this is necessary if we want to allow users to create server rules without explictly creating a rule instance locally (set the Filter and Action in the create_rule's options).

According to the HTTP spec and grammar, a header value has this format:

field-value    = *( field-content | LWS )
field-content  = <the OCTETs making up the field-value
                        and consisting of either *TEXT or combinations
                        of token, separators, and quoted-string>
TEXT           = <any OCTET except CTLs,
                        but including LWS>
CTL            = <any US-ASCII control character
                        (octets 0 - 31) and DEL (127)>
LWS            = [CRLF] 1*( SP | HT )

The HTTP library that Ruby uses accepts header values that do not conform to those rules, which leads to unexpected behavior in some edge cases. Consider this specially crafted metadata:

let(:metadata) { {
    'foo2' => "bar2\nX-Ms-Meta-Foo4: baz",
    'foo' =>  'bar',
    'foo3' => 'bar3' } }

This gets sent to the server as

X-Ms-Meta-Foo2: bar2
X-Ms-Meta-Foo4: baz
X-Ms-Meta-Foo: bar
X-Ms-Meta-Foo3: bar3

Then the submit fails due signing errors, (because the client signs something with three headers while the server signs four headers). This type of error is very hard to customers to debug.

One way to give customers a better error experience it to fail early, once an invalid header value starts getting processed. In PHP, the implementation is like this:

public function generateMetadataHeaders($metadata) {
  foreach ($metadata as $key => $value) {
    $headerName = Resources::X_MS_META_HEADER_PREFIX;
    if (strpos($value, "\r") !== false || strpos($value, "\n") !== false) {
      throw new \InvalidArgumentException(Resources::INVALID_META_MSG);

(Not ideal, but better than nothing). Ruby should add something similar.

Looking at the XML produced by some of the Ruby table tests, we see:

  <content type="application/xml">
    <m:properties>
      <d:PartitionKey m:type="">part1</d:PartitionKey>
      <d:RowKey m:type="">entity-3</d:RowKey>
      <d:CustomStringProperty m:type="">CustomPropertyValue</d:CustomStringProperty>
      <d:CustomIntegerProperty m:type="Edm.Int32">37</d:CustomIntegerProperty>
      <d:CustomBooleanProperty m:type="Edm.Boolean">true</d:CustomBooleanProperty>
      <d:CustomDateProperty m:type="Edm.DateTime">2012-12-06T15:41:27-08:00</d:CustomDateProperty>
    </m:properties>
  </content>

The disturbing part is the m:type="" for string valued nodes. The attribute should either be elided or have the value "Edm.String", an empty string is not part of the spec, and could be broken if the server decided that it is an error in the future.

Either:

1. Fill in the value for the attribute, or
2. Remove the attribute for strings.

None of the test ENV variable values are allowed to have beginning or ending whitespace, but it is easy to accidentally enter trailing whitespace from the command line. For example, on my Win8 box:

REM RUBY
set AZURE_STORAGE_ACCOUNT=jcooketest 
set AZURE_STORAGE_ACCESS_KEY=cLA1oIj4L4nElnK6IdPqHppsglpBiVXd95Qzx+i2XXX...
set AZURE_BLOB_HOST=http://jcooketest.blob.core.windows.net
set AZURE_QUEUE_HOST=http://jcooketest.queue.core.windows.net
set AZURE_TABLE_HOST=http://jcooketest.table.core.windows.net
set AZURE_SERVICEBUS_NAMESPACE=jcooketest
set AZURE_SERVICEBUS_ACCESS_KEY=CgXCeMjmTpx740Uw8lxxx...
set AZURE_SERVICEBUS_ISSUER=owner

In particular, I spent too long trying to figure out what was wrong with the scenario tests. Wrong version of Ruby? Are all dependencies up to date? Is my corp-net proxy eating something? No, just a space at the end of the second line :-(

To make the ramp-up as easy as possible for new users, the ENV reading code should just trim the disallowed whitespace from user input.

In the query_entities_tests, there is this code:

    before { 
      subject.create_table table_name
      partitions.each { |p|
        entities[p].each { |e| 
          subject.insert_entity table_name, p, e, entity_properties
        }
      }
    }

It would be somewhat more performant to replace the entity loop with a batch (which would also exercise the batch insert too)

    before { 
      subject.create_table table_name
      partitions.each { |p|
        batch = Azure::Storage::Table::Batch.new table_name, p
        entities[p].each { |e| 
          batch.insert e, entity_properties
        }
        subject.execute_batch batch
      }
    }

This scenario should have good coverage from @jcookems running unit tests on his dev machine (which is Win8).

Moving back to pending because this is not actionable until we have "final" code.

The Azure::Core::Configuration class only supports the main Azure Service Bus endpoints:

      def acs_host
        "https://#{acs_namespace}-sb.accesscontrol.windows.net"
      end
      def service_bus_host
        "https://#{acs_namespace}.servicebus.windows.net"
      end

However, Service Bus can be hosted in a private cloud, in which case alternate endpoints need to be specified. For example, in addition to just specifying the namespace, like this:

set AZURE_SERVICEBUS_NAMESPACE=jcooketest
set AZURE_SERVICEBUS_ACCESS_KEY=CgX...
set AZURE_SERVICEBUS_ISSUER=owner

we should be able to set the endpoints explicitly, like this:

set AZURE_SERVICEBUS_URI=https://jcooketest.servicebus.windows.net/
set AZURE_SERVICEBUS_WRAP_URI=https://jcooketest-sb.accesscontrol.windows.net/WRAPv0.9
set AZURE_SERVICEBUS_ACCESS_KEY=CgX...
set AZURE_SERVICEBUS_ISSUER=owner

Also, the service_bus_host should include the WRAP0.9 part, instead of hardcoding that into the lib/azure/service_bus/auth/wrap_service.rb

This can be done after the initial release

Next release ...

According to the Ruby docs about encoding:

Ruby source files should declare its script encoding by a magic comment even when they only depend on US-ASCII strings or regular expressions.

So, all our files should begin with:

# encoding: UTF-8

(or whatever other encoding we deem appropriate)

Something is going wrong for the signing when running tests against Ruby 2.0.

The string that is getting signed is:

PUT


0








x-ms-date:Wed, 20 Mar 2013 05:58:48 GMT
x-ms-version:2012-02-12
/jcooketest/ntzqevsxwa

While the server is expecting:

PUT


0

application/x-www-form-urlencoded






x-ms-date:Wed, 20 Mar 2013 05:58:48 GMT
x-ms-version:2012-02-12
/jcooketest/ntzqevsxwa

For some reason, the server is getting a application/x-www-form-urlencoded that the SDK is not expecting.

This does not repro for Table (uses different SharedKeyLite) or for Service Bus (uses ACS).

Similarly to the other SDKs

When I run the tests, after installing the Chinese Simplified LIP, and set it as my primary OS language, I get failures. They seem to all come from failures in ...\table\serialization.rb, in particular line 97:

properties[prop.name] = prop.text != "" ?
    Azure::Table::EdmType.unserialize_query_value(prop.text, prop["m:type"]) : 
    prop["null"] ? nil : ""

The problem is that prop["m:type"] is returning nil, so the type information is ignored and all; values are deserialized as strings.

I can fix the issue by removing the "m" namespace, but that is a hack.

Failures:

  1) Failure:
test_0001_creates_an_entity(Azure::Table::TableService::#insert_entity) [C:/dd/git/jcooke/azure-sdk-for-ruby/test/integration/table/insert_entity_test.rb:57]:
Expected: 3.141592
  Actual: "3.141592"

  2) Failure:
test_0001_creates_an_entity(Azure::Table::TableService::#insert_entity_batch) [C:/dd/git/jcooke/azure-sdk-for-ruby/test/integration/table/insert_entity_batch_te
st.rb:61]:
Expected: 3.141592
  Actual: "3.141592"

  3) Failure:
test_0002_updates_an_existing_entity_merging_the_properties(Azure::Table::TableService::#insert_or_merge_entity) [C:/dd/git/jcooke/azure-sdk-for-ruby/test/integ
ration/table/insert_or_merge_entity_test.rb:109]:
Expected: 37
  Actual: "37"

  4) Failure:
test_0001_creates_an_entity_if_it_does_not_already_exist(Azure::Table::TableService::#insert_or_merge_entity) [C:/dd/git/jcooke/azure-sdk-for-ruby/test/integrat
ion/table/insert_or_merge_entity_test.rb:64]:
Expected: 37
  Actual: "37"

  5) Failure:
test_0001_creates_an_entity_if_it_does_not_already_exist(Azure::Table::TableService::#insert_or_merge_entity_batch) [C:/dd/git/jcooke/azure-sdk-for-ruby/test/in
tegration/table/insert_or_merge_entity_batch_test.rb:66]:
Expected: 37
  Actual: "37"

  6) Failure:
test_0002_updates_an_existing_entity_merging_the_properties(Azure::Table::TableService::#insert_or_merge_entity_batch) [C:/dd/git/jcooke/azure-sdk-for-ruby/test
/integration/table/insert_or_merge_entity_batch_test.rb:116]:
Expected: 37
  Actual: "37"

  7) Failure:
test_0001_creates_an_entity_if_it_does_not_already_exist(Azure::Table::TableService::#insert_or_replace_entity) [C:/dd/git/jcooke/azure-sdk-for-ruby/test/integr
ation/table/insert_or_replace_entity_test.rb:64]:
Expected: 37
  Actual: "37"

  8) Failure:
test_0001_creates_an_entity_if_it_does_not_already_exist(Azure::Table::TableService::#insert_or_replace_entity_batch) [C:/dd/git/jcooke/azure-sdk-for-ruby/test/
integration/table/insert_or_replace_entity_batch_test.rb:68]:
Expected: 37
  Actual: "37"

  9) Failure:
test_0001_updates_an_existing_entity_merging_the_properties(Azure::Table::TableService::#merge_entity) [C:/dd/git/jcooke/azure-sdk-for-ruby/test/integration/tab
le/merge_entity_test.rb:73]:
Expected: 37
  Actual: "37"

 10) Failure:
test_0001_updates_an_existing_entity_merging_the_properties(Azure::Table::TableService::#merge_entity_batch) [C:/dd/git/jcooke/azure-sdk-for-ruby/test/integrati
on/table/merge_entity_batch_test.rb:77]:
Expected: 37
  Actual: "37"

 11) Failure:
test_0002_can_constrain_by_partition_and_row_key_returning_zero_or_one_entity(Azure::Table::TableService::#query_entities) [C:/dd/git/jcooke/azure-sdk-for-ruby/
test/integration/table/query_entities_test.rb:88]:
Expected: 37
  Actual: "37"

 12) Failure:
test_0003_can_project_a_subset_of_properties_populating_sparse_properties_with_nil(Azure::Table::TableService::#query_entities) [C:/dd/git/jcooke/azure-sdk-for-
ruby/test/integration/table/query_entities_test.rb:104]:
Expected: 37
  Actual: "37"

 13) Failure:
test_0001_queries_a_table_for_list_of_entities(Azure::Table::TableService::#query_entities) [C:/dd/git/jcooke/azure-sdk-for-ruby/test/integration/table/query_en
tities_test.rb:68]:
Expected: 37
  Actual: "37"

 14) Failure:
test_0007_can_combine_projection_filtering_and_paging_in_the_same_query(Azure::Table::TableService::#query_entities) [C:/dd/git/jcooke/azure-sdk-for-ruby/test/i
ntegration/table/query_entities_test.rb:174]:
Expected: 37
  Actual: "37"

 15) Failure:
test_0007_can_combine_projection_filtering_and_paging_in_the_same_query(Azure::Table::TableService::#query_entities) [C:/dd/git/jcooke/azure-sdk-for-ruby/test/i
ntegration/table/query_test.rb:227]:
Expected: 37
  Actual: "37"

 16) Failure:
test_0002_can_constrain_by_partition_and_row_key_returning_zero_or_one_entity(Azure::Table::TableService::#query_entities) [C:/dd/git/jcooke/azure-sdk-for-ruby/
test/integration/table/query_test.rb:96]:
Expected: 37
  Actual: "37"

 17) Failure:
test_0003_can_project_a_subset_of_properties_populating_sparse_properties_with_nil(Azure::Table::TableService::#query_entities) [C:/dd/git/jcooke/azure-sdk-for-
ruby/test/integration/table/query_test.rb:118]:
Expected: 37
  Actual: "37"

 18) Failure:
test_0001_queries_a_table_for_list_of_entities(Azure::Table::TableService::#query_entities) [C:/dd/git/jcooke/azure-sdk-for-ruby/test/integration/table/query_te
st.rb:71]:
Expected: 37
  Actual: "37"

GB-18030-encoded content cannot be sent to a queue, as this example shows. (It is likely that other encodings will also fail):

content = [0x554A, 0x963F, 0x9F3E, 0x9F44].pack('U*').encode!("GB18030")
subject.create_message queue_name, content
messages = subject.list_messages queue_name, 500
message = messages.first
returned_content = message.message_text
returned_content.must_equal content
subject.delete_message queue_name, message.id, message.pop_receipt

Looking at the network traffic, this is the request that is sent out:

<?xml version="1.0"?>
<QueueMessage>
  <MessageText></MessageText>
</QueueMessage>

By performing an explicit conversion prior to sending out

subject.create_message queue_name, content.encode!("UTF-8")

the message gets sent correctly:

<QueueMessage>
  <MessageText>AB&#x554A;&#x963F;&#x9F3E;&#x9F44;</MessageText>
</QueueMessage>

NOTE: The encode! method is appears to be destructive in-place conversion. To avoid corrupting user data, I suppose that making a copy of the string first would be a better approach.

So is Azure::ServiceBus::Topic.size_in_bytes

So are Azure::ServiceBus::Queue.message_count and Azure::ServiceBus::Subscription.message_count

Like in #293, the query parameters for tables must be percent-encoded UTF-8 representations of the characters. However, if a user passes in a string encoded with a different encoding (like GB-18030), it byte representation will be used in the percent-encoding, which is not abiding by the server's protocol.

For example (and assuming that #297 is fixed):

prop = "prop" + [0x3128, 0x3129].pack('U*').encode!("GB18030")
entity_properties = { 
   "PartitionKey" => "x",
   "RowKey" => k
   prop => "value",
}
result = subject.insert_entity table_name, entity_properties
filter = prop + " eq 'value'"
result = subject.query_entities table_name, { :filter => filter }
result.length.must_equal 1
result.first.properties[prop].must_equal "value"

This should pass, but it fails with:

  1) Error:
test_0001_read_write_and_query_property_names_gb18030(Table GB-18030):
Azure::Core::Http::HTTPError: InvalidInput (400): One of the request inputs is not valid.

This is because the URL is:

http://...ryn()?$filter=prop%A8%E8%A8%E9+eq+%27value%27

Several of the places where the Nokogiri builder is used, it does not specify an encoding. For definiteness, it should always be called like this:

builder = Nokogiri::XML::Builder.new(:encoding=>"utf-8")

I updated one of the tests to use this metadata when creating a blob:

let(:metadata) { { 
    "CustomMetadataProperty"=>"CustomMetadataValue",
    "foo2' => "bar2   ",
    "foo" =>  "bar",
    "foo3" => "bar3" } }

The request strips the spaces from the values, according to the HTTP spec

Such leading or trailing [white space] MAY be removed without changing the semantics of the field value.

X-Ms-Meta-Custommetadataproperty: CustomMetadataValue
X-Ms-Meta-Foo2: bar2
X-Ms-Meta-Foo: bar
X-Ms-Meta-Foo3: bar3

But the server rejects the client's computed signature:

HTTP/1.1 403 Forbidden
The MAC signature found in the HTTP request is not the same as any computed signature.

This is because the client computes the signature on the user input, which is not the same as what the server gets sent. Here is what gets signed:

x-ms-meta-custommetadataproperty:CustomMetadataValue\n
x-ms-meta-foo:bar\n
x-ms-meta-foo2:bar2 \n
x-ms-meta-foo3:bar3\n

Notice the trailing space for foo2's value.