For this requirement in the Exchange MAPI MS-OXCFXICS test scenarios, we need to add checking specifically for 2.2.4.3.23 "recipient Element":

PidTagRowid ([MS-OXPROPS] section 2.935) Required Fixed position None.

When Outlook receives a ROP buffer containing Recipient table, the PidTagRowid property MUST be first in this list or Outlook will ignore this email item.
This is a customer found issue.

in chapter 3, it says:
3 Deploying the test suites
This section describes the deployment of the SharePoint File Sync and WOPI Protocol test suites on the test suite client and the SUT. The SharePoint Server Protocol test suites are packaged in a .zip file which is available on Microsoft Connect. Once you download the test suites, you need to perform the following steps in order to be able to successfully configure the test suites.

If you follow the hyperlink for "Microsoft Connect" it takes you to the redirect site:
https://docs.microsoft.com/en-us/collaborate/connect-redirect

Can we update these documents to point to the github repo instead?

Hi!

According to MS-FSSHTTPB 2.2.2.1.3, bits I and J are not optional.
The referenced Appendix 14 states that SP2010-2016 servers ignore these bits, but does not indicate that they can be omitted.

Can these bits be omitted by the client?

There are important files that Microsoft projects should all have that are not present in this repository. A pull request has been opened to add the missing file(s). When the pr is merged this issue will be closed automatically.

Microsoft teams can learn more about this effort and share feedback within the open source guidance available internally.

Merge this pull request

I am trying to parse a OneNote package using this interop test suite project.

This project is saved directly from SharePoint online thus it is stored in:

MS-ONESTORE
2.8 Alternative Encoding Using the File Synchronization via SOAP Over HTTP Protocol

With a recent #56 had been merged with the alternative packaging support added. So I can actually load this format using the programming sdk:

      byte[] oneNoteBytes = File.ReadAllBytes(@"testOneNoteMultiplePages.one");

      SharedContext.Current.IsMsFsshttpRequirementsCaptured = false;

      AlternativePackaging alternatePackageOneStoreFile = new AlternativePackaging();
      alternatePackageOneStoreFile .DoDeserializeFromByteArray(oneNoteBytes, 0);

My ultimate goal here is to be able to obtain the MS-ONESTORE packaging so that I can extract text from this document.

So I expected this to work:

      MSONESTOREParser onenoteParser = new MSONESTOREParser();
      MSOneStorePackage package = onenoteParser.Parse(alternatePackageOneStoreFile.dataElementPackage);

But this does not work it fails during parsing:

Unhandled exception. System.InvalidOperationException: Unexpected to meet the byte array end.
   at Microsoft.Protocols.TestSuites.Common.BitReader.GetBytes(Int32 needReadlength, Int32 size) in C:\lucidworks\Interop-TestSuites\FileSyncandWOPI\Source\Common\BitField.cs:line 217
   at Microsoft.Protocols.TestSuites.Common.BitReader.ReadUInt32(Int32 readingLength) in C:\lucidworks\Interop-TestSuites\FileSyncandWOPI\Source\Common\BitField.cs:line 121
   at Microsoft.Protocols.TestSuites.MS_ONESTORE.ObjectSpaceObjectStreamHeader.DoDeserializeFromByteArray(Byte[] byteArray, Int32 startIndex) in C:\lucidworks\Interop-TestSuites\FileSyncandWOPI\Source\Common\ONESTORE\OtherStructures\ObjectSpaceObjectStreamHeader.cs:line 50
   at Microsoft.Protocols.TestSuites.MS_ONESTORE.ObjectSpaceObjectStreamOfOSIDs.DoDeserializeFromByteArray(Byte[] byteArray, Int32 startIndex) in C:\lucidworks\Interop-TestSuites\FileSyncandWOPI\Source\Common\ONESTORE\OtherStructures\ObjectSpaceObjectStreamOfOSIDs.cs:line 44
   at Microsoft.Protocols.TestSuites.MS_ONESTORE.ObjectSpaceObjectPropSet.DoDeserializeFromByteArray(Byte[] byteArray, Int32 startIndex) in C:\lucidworks\Interop-TestSuites\FileSyncandWOPI\Source\Common\ONESTORE\OtherStructures\ObjectSpaceObjectPropSet.cs:line 46
   at Microsoft.Protocols.TestSuites.SharedAdapter.HeaderCell.CreateInstance(ObjectGroupDataElementData objectElement) in C:\lucidworks\Interop-TestSuites\FileSyncandWOPI\Source\SharedTestSuite\SharedAdapter\Stack\ONESTORE\MSOneStorePackage.cs:line 125
   at Microsoft.Protocols.TestSuites.SharedAdapter.MSONESTOREParser.ParseHeaderCell(RevisionManifestDataElementData headerCellRevisionManifest) in C:\lucidworks\Interop-TestSuites\FileSyncandWOPI\Source\SharedTestSuite\SharedAdapter\Helper\MSONESTOREParser.cs:line 99
   at Microsoft.Protocols.TestSuites.SharedAdapter.MSONESTOREParser.Parse(DataElementPackage dataElementPackage) in C:\lucidworks\Interop-TestSuites\FileSyncandWOPI\Source\SharedTestSuite\SharedAdapter\Helper\MSONESTOREParser.cs:line 53
   at TestFSSHTTPB.Program.Main(String[] args) in C:\lucidworks\Interop-TestSuites\FileSyncandWOPI\Source\MS-FSSHTTP-FSSHTTPB\TestFSSHTTPB\Program.cs:line 26

Here is the one note file i am testing with:
testOneNoteMultiplePages.zip

Is my understanding of the format flawed here? I assumed that I'd be able to parse the data element package within the alternate packaging format using the MS-OneStore parser.

When I was talking to Tom Jebo, he was mentioning that there may be a need to "combine" some data packaging elements to make this work. Is that the case here?

How do I use this C# library to obtain the MS one store package from the 2.8 alternative packaging format?

Against our exchange server, unit test MSOXNSPI_S02_TC08_GetPropsSuccessWithFlagsContainsfEphID fails when it tries to execute the section “Call NspiGetProps with input parameter proptags not set to null.”

The first set of GetProps calls succeeds, but the second set, requesting PidTagEntryId and PidTagAddressBookX509Certificate, fails. The response from our exchange server is below. Is there something we need to do to additionally configure the server to pass this test, or should we modify the test to request a different value instead of PidTagAddressBookX509Certificate? (The test assumes the call should return a 0 error code, but the server is returning a 263040 (ErrorsReturned))

GetPropsRequest
   Flags:2
   HasState:True
   State
      SortType:0
      ContainerID:0
      CurrentRec:9137
      Delta:0
      NumPos:0
      TotalRecs:0
      CodePage:20261
      TemplateLocale:1033
      SortLocale:1033
   HasPropertyTags:True
   PropertyTags
      PropertyTagCount:2
      PropertyTags
         PropertyTag
            PropertyType:PtypBinary
            PropertyId:PidTagEntryId
         PropertyTag
            PropertyType:PtypMultipleBinary
            PropertyId:PidTagAddressBookX509Certificate
   AuxiliaryBufferSize:0

GetPropsResponse
   MetaTags
      MAPIString
         Value:PROCESSING
      MAPIString
         Value:DONE
   AdditionalHeaders
      MAPIString
         Value:X-StartTime: Tue, 19 Feb 2019 16:21:41 GMT
      MAPIString
         Value:X-ElapsedTime: 3
      MAPIString
         Value:
   StatusCode:0
   ErrorCode:263040
   CodePage:20261
   HasPropertyValues:True
   PropertyValues
      PropertyValueCount:2
      PropertyValues
         AddressBookTaggedPropertyValue
            PropertyType:PtypBinary
            PropertyId:4095
            PropertyValue
               HasValue:True
               PropertyValue
                  Count:32
                  Value:[135,0,0,0,242,251,50,1,245,46,67,64,170,159,166,66,213,91,40,34,1,0,0,0,0,0,0,0,205,255...]
         AddressBookTaggedPropertyValue
            PropertyType:PtypErrorCode
            PropertyId:35946
            PropertyValue
               PropertyValue
                  Value:2147746063
   AuxiliaryBufferSize:0

Thanks in advance

Hi!

During the unpacking of the editor table data, the actual buffer size returned by stream.Read is not taken into account, and as a result the XML load (doc.LoadXml) returns an error (0x00 hexadecimal value, invalid character).

This small modification should eliminate the problem:

Clearly stated in the official documentation: https://docs.microsoft.com/en-us/openspecs/exchange_server_protocols/ms-oxcrpc/4c01d3be-2f25-458a-8ed2-14cb110f9cec
For buffers: they should be de-obfuscated before being decompressed.
But in this project, much of the processing is reversed.

Hi!

@ChangDu2021 In MS-FSSHTTPB - 2.2.2.1.4 at the end of this part ’Expected Storage Index Extended GUID’ I found the following about the ’Imply Null Expected if No Mapping’ flag:
„If the flag specifies one and a mapping exists, the protocol server MUST return a Cell Error failure value of 12 indicating a coherency failure, as specified in section 2.2.3.2.1.”

But at the end of TestCase_S13_TC05_PutChanges_MappingExist_ImplyFlagOne (SharedTestCase/S13_PutChanges.cs) it asserts success instead of coherencyFailure (comment says there: „expect the server responds the Coherency failure error”).
Which errorCode is the correct in this case?

Thank you!

In the last few lines of MSOXCFOLD_S01_TC03_RopCreateFolderSuccess, it validates that pidTagMessageSizeExtended == 0. The description that asserts on failure is:

[In PidTagContentCount Property] The PidTagContentCount property ([MS-OXPROPS] section 2.642) specifies the number of messages in a folder, as computed by the message store.

Since the CaptureRequirementIfAreEqual call doesn't actually refer to PidTagContentCount, either the description or the assertion should be fixed.

In the file ObjectSpaceObjectPropSet.cs of the ONESTORE adapter, we find the following loop:

line 49ff:

if (this.OIDs.Header.OsidStreamNotPresent == 0 && this.OIDs.Header.ExtendedStreamsPresent == 1)
            {
                this.ContextIDs = new ObjectSpaceObjectStreamOfContextIDs();
                len = this.ContextIDs.DoDeserializeFromByteArray(byteArray, index);
                index += len;
            }

Please put on your focus on this.OIDs.Header.ExtendedStreamsPresent == 1 and compare with the specification of ONESTORE

i may quote:

ContextIDs (variable): An optional ObjectSpaceObjectStreamOfContextIDs (section 2.6.4) that specifies the count and list of contexts referenced by this ObjectSpaceObjectPropSet structure. MUST be present if OSIDs is present and the value of the OSIDs.header.ExtendedStreamsPresent field is true; otherwise, the ContextIDs field MUST NOT be present. The count of referenced contexts is calculated as the number of properties specified by the body field with PropertyID equal to 0xC plus the number of referenced contexts specified by properties with PropertyID equal to 0xD, 0x10, and 0x11. This count MUST be equal to the value of ContextIDs.header.Count field. Properties that reference other contexts MUST be matched with the CompactID structures from ContextIDs.body field in the same order as the properties are listed in the body.rgPrids field.

OSIDs is present if the first condition is met (this.OIDs.Header.OsidStreamNotPresent == 0).
However, the specifiction states, that inside the OSIDs' header extendedStreamsPresent must be true. This is different from OIDs' header as currently in the code.

If the specification is strictly correct, the if condition shown above contains a bug (which can be fixed trivially).
Otherwise, the specification would have a typo.

Which location of an error is more plausible to you?

I need some help extracting and asserting on some text from within a OneNote file saved from SharePoint Online.

So I go into SharePoint Online and create a OneNote file with some basic text content inside.

My overall goal is to do assertions on text content inside the OneNote file.

So I copy that file down to .one format and store it locally. Then I'm trying to use this project to create a Test Suite to extract the text from the OneNote file.

I have built the data elements list and that works.

But I'm confused how to extract the MSOneStorePackage from this data.

I tried the following:

      byte[] oneNoteBytes = File.ReadAllBytes(@"C:\test\testOneNoteMultiplePages.one");
      ExGuid exGuid;

      List<DataElement> dataElements = DataElementUtils.BuildDataElements(oneNoteBytes, out exGuid); // this works.

      DataElementPackage dataElementPackage = new DataElementPackage();
      dataElementPackage.DataElements = dataElements;

      MSONESTOREParser parser = new MSONESTOREParser();
      MSOneStorePackage msOneStorePackage = parser.Parse(dataElementPackage); // fails here during this call

      Console.WriteLine("Result: {0}", msOneStorePackage);

But this fails:

Unhandled Exception: System.InvalidOperationException: Unexpected to meet the byte array end.
   at Microsoft.Protocols.TestSuites.Common.BitReader.GetBytes(Int32 needReadlength, Int32 size) in C:\test\Interop-TestSuites\FileSyncandWOPI\Source\Common\BitField.cs:line 217
   at Microsoft.Protocols.TestSuites.Common.BitReader.ReadUInt32(Int32 readingLength) in C:\test\Interop-TestSuites\FileSyncandWOPI\Source\Common\BitField.cs:line 121
   at Microsoft.Protocols.TestSuites.MS_ONESTORE.ObjectSpaceObjectStreamHeader.DoDeserializeFromByteArray(Byte[] byteArray, Int32 startIndex) in C:\test\Interop-TestSuites\FileSyncandWOPI\Source\Common\ONESTORE\OtherStructures\ObjectSpaceObjectStreamHeader.cs:line 50
   at Microsoft.Protocols.TestSuites.MS_ONESTORE.ObjectSpaceObjectStreamOfOSIDs.DoDeserializeFromByteArray(Byte[] byteArray, Int32 startIndex) in C:\test\Interop-TestSuites\FileSyncandWOPI\Source\Common\ONESTORE\OtherStructures\ObjectSpaceObjectStreamOfOSIDs.cs:line 44
   at Microsoft.Protocols.TestSuites.MS_ONESTORE.ObjectSpaceObjectPropSet.DoDeserializeFromByteArray(Byte[] byteArray, Int32 startIndex) in C:\test\Interop-TestSuites\FileSyncandWOPI\Source\Common\ONESTORE\OtherStructures\ObjectSpaceObjectPropSet.cs:line 46
   at Microsoft.Protocols.TestSuites.SharedAdapter.HeaderCell.CreateInstance(ObjectGroupDataElementData objectElement) in C:\test\Interop-TestSuites\FileSyncandWOPI\Source\SharedTestSuite\SharedAdapter\Stack\ONESTORE\MSOneStorePackage.cs:line 125
   at Microsoft.Protocols.TestSuites.SharedAdapter.MSONESTOREParser.ParseHeaderCell(RevisionManifestDataElementData headerCellRevisionManifest) in C:\test\Interop-TestSuites\FileSyncandWOPI\Source\SharedTestSuite\SharedAdapter\Helper\MSONESTOREParser.cs:line 99
   at Microsoft.Protocols.TestSuites.SharedAdapter.MSONESTOREParser.Parse(DataElementPackage dataElementPackage) in C:\test\Interop-TestSuites\FileSyncandWOPI\Source\SharedTestSuite\SharedAdapter\Helper\MSONESTOREParser.cs:line 53
   at TestFSSHTTP_FSSHTTPB.Program.Main(String[] args) in C:\test\Interop-TestSuites\FileSyncandWOPI\Source\TestFSSHTTP-FSSHTTPB\Program.cs:line 24

Looks like the ObjectSpaceObjectStreamHeader is expecting a 32 byte array, but it only got a 16 byte array.

Can someone kick me in the right direction for the full parsing of the content so that I can get to the pages and their text?

Here is the one note file i am testing with:
testOneNoteMultiplePages.zip

Hi!

The compoundTypes in BaseType.cs do not fully correspond to the table in MS-FSSHTTPB in section 2.2.1.5.2. The value of 'Response (FsshttpbResponse)' is missing and 'Target Partition Id' should be a non-compound type according to the table.

Therefore, a header is added at the end of TargetPartitionId, but it is not required by the protocol specification, since it is not a compound type.

Have a nice day!

I was running unit test MSOXCFXICS_S03_SyncFastTransferAttachmentLevel_TestSuite and I think it's setting the display name property on the wrong server handle object. On line 136 it's calling SetProperties on index 3 which is the message object and not the attachment object which is index 4. I know you can set a display name on message and attachments but since the test is referencing attachments it feels like that's where the property should be set.

If it's not a bug and the message object's display name is the intended server handle then we can close this issue. However, if it should be set on the attachment then changing the index to 4 should resolve the issue.

ProtocolTestFramework published new versions which are using .NET 6 (and up) and is available through nuget. This project is stuck on .NetFramework v4-something-old. It's super hard, if not impossible, to build this project. Please upgrade to the modern era.

Hi!

I’m bit confused about putChanges additionalFlags (MS-FSSHTTPB - 2.2.2.1.4.1):

In TestCase_S13_TC20_PutChanges_CheckForIdReuse (SharedTestCase/S13_PutChanges.cs) it seems the ’C – Check for Id Reuse’ flag is connected somehow with ’D – Coherency Check Only Applied Index Entries’ flag (D is mentioned in the method comment (in row 1211) and at the end of the test method between requirements (in row 1263)).
Could you please clarify a bit more how these flags related to each other in usage and where is the D flag set in that test code?

Thank you!

Hi!

At the end of the 'TestCase_S13_TC01_PutChanges_ValidExpectedStorageIndexExtendedGUID' test case, the check expects a 'Referenceddataelementnotfound' error.

However, according to MS-FSSHTTPB 2.2.2.1.4, if the 'Favor Coherency Failure Over Not Found' flag is set to 1, then a 'Coherency Failure' error should be returned. That is, it is not clear because it says: 'A bit that specifies to force a coherency check on the server, if a Referenced Data Element Not Found failure occurred. This might result in a Coherency Failure returned instead of Referenced Data Element Not Found.'

In the current test case, this flag is set to 1:

Can you tell me what exactly you mean by coherency check?

According to MS-FSSHTTPB 2.2.2.1.4, the fields "ContentVersionCoherencyCheck", "Author Logins" and "Reserved (1 byte)" are not optional and cannot be omitted from the request according to Appendix 18 and 19, but are missing from the test-suite request.

Edit: these fields were introduced in the document version issued on 10/5/2021.

After a recent PR was merged, I have now successfully used this test suite to validate One Note files that have been saved from Office 365 / SharePoint Online.

      byte[] oneNoteBytes = File.ReadAllBytes(@"C:\test\saved-from-sharepoint-online.one");

      SharedContext.Current.IsMsFsshttpRequirementsCaptured = false;
      AlternativePackaging alternatePackageOneStoreFile = new AlternativePackaging();
      alternatePackageOneStoreFile.DoDeserializeFromByteArray(oneNoteBytes, 0);

      MSONESTOREParser onenoteParser = new MSONESTOREParser();
      MSOneStorePackage package = onenoteParser.Parse(alternatePackageOneStoreFile.dataElementPackage);

So far so good! Thanks to @xian-rong for all the work on that.

Would it be easy to modify this test suite such that there is a way to use the test suite to parse a file that was saved from Microsoft Office on prem?

Hi!

In SharedTestCase/S12_QueryChanges.cs within TestCase_S12_TC17_QueryChanges_IncludeFilteredOutDataElementsInKnowledge I've found the following:
"These two bytes are magic number which is not documented in the open specification in the current stage, but it will be wrote down in the future."

Is it already possible to know more about these bytes?

I'm interested in MS-FSSHTTP.
Thank you very much!

Hi!
How can I decide, when only clientId needs to be changed and when should we additionally change the user as well? Can the same user use different clientIds simultaneously? For example:

Here only the clientId is changed - TestCase_S04_TC25_ConvertToSchemaJoinCoauth_SharedLock
Here the user is also changed (UserName02) - TestCase_S03_TC30_ReleaseLock_DifferentSchemaLockWithOneClient

I'm interested in MS-FSSHTTP.
Thank you very much!

I used LZ77Decompress to decompress the response buffer returned by Exchange Server.

It causes the program to crash (memory access out of bounds).

I went through the documentation and found one of the variables SizeOfFinalTwoBytes, which should have a value of 3

@XinwLi

We need to add coverage to the MAPI tests in our Interop-TestSuites. Specifically, this passage in MS-OXCFXICS 2.2.2.3:

The set of XIDs MUST be serialized without padding as an array of SizedXid
structures binary-sorted by the value of NamespaceGuid field of the XID structure in the ascending
order.

based on review we have R169 and the following derserialize code, we don't specifically check for sorting:

public override int Deserialize(Stream stream, int size)
        {
            int bytesRead = 0;

            this.sizedXidList = new List<SizedXid>();
            while (((size >= 0) && bytesRead < size)
                || (size < 0 && stream.Position != stream.Length))
            {
                SizedXid xid = new SizedXid();
                bytesRead += xid.Deserialize(stream, -1);
                this.sizedXidList.Add(xid);
            }

            return bytesRead;
        }

When calculating the length, you don't take into account the existence of a shared byte.

This will cause the length of your calculation to be wrong, resulting in overstepping/decompression failure

In the MAPI test suite deployment guide, it says this is required:

Microsoft Spec Explorer 3.6.14230.01 and above

However, in the readme.md for this repo, it contains the link:

Spec Explorer 2010 (https://visualstudiogallery.msdn.microsoft.com/271d0904-f178-4ce9-956b-d9bfa4902745/)

If 3.6 is really required as stated by the deployment guide, we should provide a way to download it externally. Otherwise, fix the requirement in the deployment guide to point to 3.5 release.

Discussing with the dev team here, for the MAPIHTTP tests, the header 'Expect: 100-continue' is added by HttpWebRequest class of .Net when sending a POST request. This is not needed and has no effect on the tests (at least for MAPIHTTP).

We should change the ServicePointManager.Expect100Continue property to false to turn off this header. This will allow the traffic to match more closely with Outlook and possibly other clients.

Hi!
If the server (MS-FSSHTTP) responds an errorCode, when should we return it within the subResponse and when in somewhere else (like in TestCase_S17_TC02_FileOperation_ErrorCode: within the responseVersion)?
Thank you very much!

Where is the GetNumberOfAddressBookObject() function implemented?

Example usage under MSOXNSPI_S02_TC11_SeekEntriesSuccessWithNullTable:

uint objectNum = this.SutControlAdapter.GetNumberOfAddressBookObject();

I found an interface in IMS_OXNSPISUTControlAdapter, but cannot find the actual code that runs to get the count.

MSOXCMSG_S01_TC04_RopSaveChangesMessageWithInvalidSaveFlags():

        RopSaveChangesMessageRequest saveChangesMessageRequest = new RopSaveChangesMessageRequest()
        {
            RopId = (byte)RopId.RopSaveChangesMessage,
            LogonId = CommonLogonId,
            InputHandleIndex = CommonInputHandleIndex,
            ResponseHandleIndex = CommonOutputHandleIndex, // This index specifies the location in the Server object handle table that is referenced in the response.
            **SaveFlags = NotSupportedSaveFlags**
        };
        this.ResponseSOHs = this.MSOXCMSGAdapter.DoRopCall(saveChangesMessageRequest, targetMessageHandle, ref this.response, ref this.rawData, GetPropertiesFlags.None);
        RopSaveChangesMessageResponse saveChangesMessageResponse = (RopSaveChangesMessageResponse)this.response;

        // Add the debug information
        Site.Log.Add(LogEntryKind.Debug, "Verify MS-OXCMSG_R1052");

        // Verify MS-OXCMSG requirement: MS-OXCMSG_R1052
        Site.CaptureRequirementIfAreEqual<uint>(
            **0x80040102,**
            saveChangesMessageResponse.ReturnValue,
            1052,
            @"[In Receiving a RopSaveChangesMessage ROP Request] [ecNotSupported (0x80040102)] The values of the SaveFlags are not a supported combination.");
        #endregion

This uses NotSupportedSaveFlags which is 0x0F. Expecting 0x80040102 or not supported error. The problem is that this is misleading to implementers in that 0x0F implies that 0x08 (not defined in the [MS-OXCMSG] 2.2.3.3.1 table of three flags (0x01, 0x02 and 0x04)) is the problem here. 0x08 is not defined in the specification but should be ignored and not cause an error.

The real reason for the 0x80040102 response is that 0x01 and 0x02 are mutually exclusive. This will be an error for Exchange server and should be caught by implementers. 0x08 should be allowed but ignored.

Better to change the test flags passed to RopSaveChangesMessage to 0x03 to clarify what the problem is.