tian-zhihui / cordysfilecon Goto Github PK

What steps will reproduce the problem?

1. Try the sample in https://wiki.cordys.com/display/dsc/Reading+a+CSV+file 
with the following modifications:
   - Use the following sample data. It has three error records.
     ---------------------------------
     100;CSV Test;CSV Test Adress;
     101;Name-101;Adress-101;
     102;Name-102
     103;
     104;Name-104;Adress-104;
     105;Name-105;
     ---------------------------------

   - Change and add the following SOAP request parameters:
     + validateonly = true
     + usetupleold = false
     + continueonerror = true

What is the expected output? What do you see instead?

- Expected output
  The response should contain error messages for the three error records.

- Actual output is as follows:
<data>
  <ReadFileRecordsResponse xmlns:SOAP="http://schemas.xmlsoap.org/soap/envelope/" xmlns="http://schemas.cordys.com/1.0/ac/FileConnector">
    <endoffset>0</endoffset>
    <recordsread>2</recordsread>
    <endoffile>false</endoffile>
    <errors>
      <item>Unable to the parse the file. * Unable to the parse the file. * Negative position</item>
    </errors>
    <errorcount>1</errorcount>
  </ReadFileRecordsResponse>
</data>

What version of the product are you using? On what operating system?

Version: File Connector 1.2.21 with BOP 4.1
Operation System: Windows Server 2008 R2 Enterprise SP1

Please provide any additional information below.

It seems that an incorrect fileoffset is set in ReadFileRecords.process(). 
(ReadFileRecords.java)
-------------------------------------------------------------------------
359      // Get the actual offset where the parsing stopped.
360      int iValidationEndPosition = rvValidator.getValidationEndPosition();
361
362      long lEndRecordFileOffset = 
w.fcsInputSeq.getFileOffset(iValidationEndPosition < 
363          0 ? (iValidationEndPosition * -1) : iValidationEndPosition);
364                    
365      lCurrentFileOffset = 
w.fcsInputSeq.getFileOffset(rvValidator.getValidationEndPosition());
-------------------------------------------------------------------------

In line 365, rvValidator.getValidationEndPosition() returns a negative value 
when the validator finds an error record. In such a case, 
w.fcsInputSeq.getFileOffset() returns a negative value.

I think that the line should be as follows, like line 362:
-------------------------------------------------------------------------
365      lCurrentFileOffset = w.fcsInputSeq.getFileOffset(iValidationEndPosition < 
             0 ? (iValidationEndPosition * -1) : iValidationEndPosition);
-------------------------------------------------------------------------

The polling interval thread/component should perform scanning as per 
configuration while a separate sub-component tracker should perform tracking 
per file, based on the configuration, instead of the polling thread performing 
both the jobs, as is the current situation in fileconnector.

Please add the configuration samples for the writer-config.xml, 
reader-config.xml and poller-config.xml to the package. They are missing in the 
recent CAP release.

Feature I would like to see:

  WriteFile/ReadFile services to support UTF-8 character set. Currently they support only single byte character set.

Impact if not implemented:

 File Connector cannot be used in projects where Chinese/Japanese language content is used.

Alternatives:

 Write ws-apps code for read/write operations

What steps will reproduce the problem?
1. Configure a trigger soap call with namespaces
2. Run the trigger
3. The soap call is done without the namespaces

What is the expected output? What do you see instead?
The soap request should be with the namespaces

What version of the product are you using? On what operating system?
BOP4 CU18, Linux

Please provide any additional information below.
I have configured the following polling configuration within the XML Store:
  <folder>
    <name>Incoming Retour berichten</name>
    <location>/opt/sw/data/azr/incoming</location>
    <filter>*.txt</filter>
    <track-time>10</track-time>
    <trigger>
      <method>RequestProcess</method>
      <namespace>http://schemas.cordys.com/1.0/coboc</namespace>
      <organization>o=CIBER,cn=cordys,cn=development,o=gen.cms.local</organization>
      <user>cn=ciber,cn=organizational users,o=CIBER,cn=cordys,cn=development,o=gen.cms.local</user>
      <move-file>true</move-file>
      <parameters>
        <type>definition</type>
        <receiver>com/azr/Generic/GeneriekSendMessage</receiver>
        <message>
          <inputGeneriekMessage xmlns="http://schemas.cordys.com/default">
            <md:Metadata xmlns:md="http://www.ciber.nl/azr/CDM/Metadata/v1">
              <md:MessageID />
              <md:MessageName FCDP:element-data="filename" />
              <md:MessageType>AW318</md:MessageType>
              <md:MessageVersion>3.0</md:MessageVersion>
              <md:ConversationID />
              <md:RefMessageID />
              <md:MessageDescription />
              <md:Sender />
              <md:Receiver />
              <md:SentDate />
            </md:Metadata>
            <Data xmlns="http://schemas.cordys.com/default">
              <Encoded>false</Encoded>
              <Payload FCDP:element-data="content-text" />
            </Data>
          </inputGeneriekMessage>
        </message>
        <source>Run from Directory Poller</source>
      </parameters>
    </trigger>
  </folder>

When i put a file witin the folder and look at the PIM, i see the following 
soap call:

<inputGeneriekMessage xmlns="http://schemas.cordys.com/default" > 
 <Metadata> 
   <MessageID/> 
   <MessageName>CIBER_2011-10-01T00:35:21.0.aw318</MessageName> 
   <MessageType>AW318</MessageType> 
   <MessageVersion>3.0</MessageVersion> 
   <ConversationID/> 
   <RefMessageID/> 
   <MessageDescription/> 
   <Sender/> 
   <Receiver/> 
   <SentDate/> 
 </Metadata> 
 <Data> 
  <Payload> Content file </Payload> 
 </Data>
</inputGeneriekMessage> 

This is wrong and when i try to read the content with XPath it does not return 
the correct values.

Feature I would like to see:
Capability to read file records spanning multiple lines.
It should be possible to specify a structure per line in a record.
For example the first line of a two lines record could be defined as fixed 
width format. The second line could be using CSV format

Example of a record structure:
Record1_fixed_width
Element1;This is element 2;and elem 3
Record2_fixed_width
Element1;element 2 is here;and now element3

Alternatives:
None

Priority: Low

Feature I would like to see:
An additional parameter for directory creation which accepts true/false. Upon 
sending true, connector should create the destination directory, mentioned in 
targetPath, if it doesn't exist, before copying or moving the file

Impact if not implemented:
Developer will have to make sure he creates the destination directory before 
moving or copying the file.

Alternatives:
Creating the directory is still an option.

Thank you.

What steps will reproduce the problem?
1. download Fileconnector source
2. create new SVN repository and import downloaded cws data
3. create a new Cordys workspace and point to the SVN repository URL

What is the expected output? 
Project gets created.

What do you see instead?
Error: "Cannot load XML File 
'D:\Cordys.FP1\CU07.1\BOP4\cws\sync\development\FileConnector_2.1.1\FileConnecto
r\com-cordys-coe\fileconnector\runtimereference\standardxmlschemadefinitions\XML
Schema_Binary_Reference_DBConnectorAttributes.xsd#com.cordys.xmlschema.XMLSchema
DefinitionRuntime#.cws'. "

What version of the product are you using? 
2.1.1 with BOP4.1 CU7.1

On what operating system?
Win-7, 64bit

Please provide any additional information below.

What steps will reproduce the problem?
1. load the newest caf file from the Cordys Community (Cordys File Connector 
2.1.1.cap)
2. Try to download the file in the 'Application Registry - Install new package)
3. Browse to the file and select it. Now press 'Download'

What is the expected output? What do you see instead?
An error message instead that the caf file is uploaded.

What version of the product are you using? On what operating system?
bop42cu1 (Version D1.002.001 , Build 23)

Please provide any additional information below.
See the screendump.

Purpose of code changes on this branch: Error is thrown when we test the 
webservice ReadFileRecords. so added xpath for each and every variable declared.

When reviewing my code changes, please focus on: only change in 
\src\java\com\cordys\coe\ac\fileconnector\ApplicationConfiguration.java the 
file.modified variable with xpath while defining.


After the review, I'll merge this branch into:
http://code.google.com/p/cordysfilecon/source/browse/trunk

Feature I would like to see:

It will be a very good value addition if the web services provided by File 
Connector are asynchronous.

Each web service should take the required operation and notification back 
details (like end point and authentication details)

Impact if not implemented:

Most of the file connector web services deal with disk IO operations and all of 
them are synchronous.

In a typical Cordys development/test/production environment all the web 
services are set with a time out value of 30 seconds.

There is every possibility for the File Connector web services to time out in 
case:

1. Dealing with large files.
2. Slow disk IO


Alternatives:

The FileConnector as delivered in version 1.2.15 does have a problem with the 
coelib.jar.
when executing e.g.:
<ReadFileRecords xmlns="http://schemas.cordys.com/1.0/ac/FileConnector">
      <filename>D:\TEMP\eon\new_2.txt</filename>
      <filetype>aw_alif_gk</filetype>
      <numrecords>-1</numrecords>
      <offset>0</offset>
      <validateonly>false</validateonly>
      <usetupleold>false</usetupleold>
    </ReadFileRecords>

An error occurs:
<SOAP:Fault xmlns:SOAP="http://schemas.xmlsoap.org/soap/envelope/"  >
<faultcode>Server.Exception</faultcode>
<faultstring xml:lang="en-US" >java.nio.charset.IllegalCharsetNameException: 
</faultstring>
<faultactor>http://schemas.cordys.com/1.0/ac/FileConnector</faultactor>


Replacing the coelib.jar with the one from version 1.2.14 does work.

What steps will reproduce the problem?
1. define attached read configuration and store in XML store.
2. execute ReadFileRecords with attached file
3. note that not more records are read than until the first that is missing the 
first record of the defined recordsequence.

What is the expected output? 
Expect that the first record of a "recordsequence" can be optional when defined 
with "*".

What do you see instead?
Stops reading records.

What version of the product are you using? 
2.1.1 with BOP4.1 CU7.1

On what operating system?
WIN-7 64bit

SOAP request:
------------
<SOAP:Envelope xmlns:SOAP="http://schemas.xmlsoap.org/soap/envelope/">
  <SOAP:Body>
    <ReadFileRecords xmlns="http://schemas.cordys.com/1.0/ac/FileConnector">
       <filename>D:\Cordys\Kunden\Lufthansa\DEZ_2012\docs\E1\ET000\E1ET000P\ETLIFTS\A4U\G0686V00.hb.test1</filename>
    <filetype>lh_etl_100</filetype>
    <numrecords>12</numrecords>
    <offset>902</offset>
    <validateonly>false</validateonly>
    <usetupleold>false</usetupleold>
    <continueonerror>true</continueonerror>
    </ReadFileRecords>
  </SOAP:Body>
</SOAP:Envelope>

Result:
---------
<ReadFileRecordsResponse xmlns:SOAP="http://schemas.xmlsoap.org/soap/envelope/" 
xmlns="http://schemas.cordys.com/1.0/ac/FileConnector">
    <errorrecords />
    <endoffset>24354</endoffset>
    <recordsread>3</recordsread>
    <endoffile>false</endoffile>
    <errorcount>0</errorcount>
    <data>
      <tuple>
        <cabincls05>
          <R05RecordType>05</R05RecordType>
...

The attached sample file however contains 6 recordsets.

Hi,
 I am unable to checkout the trunk using any SVN client tools. Could you please verify what is the issue?

The sdk folder is causing the issue.

Regards,
Phani

File poller doesnot process the files, if polling folder and processing folder 
is configured to different file systems

Steps to reproduce

1) Configure poller folder to a windows share folder mounted to linux
2) Configure processing folder to a directory in linux.
3) Drop a file in the poller folder.

What is the expected output?

The dropped file should be moved to processing folder and
the configured webservice associated should be triggered

What do you see instead?

Error logs says Moving of file failed.


What version of the product are you using? On what operating system?

BOP-4 CU 13 , Linux


Please provide any additional information below.

The existing code uses File.renameTo() to move the files to a processing folder 
and 'renameTo' API doesnt work 
for moving files between different file systems.

Feature I would like to see:

Support for Cordys Authentication (i.e) SAML 1.1 based authentication

Impact if not implemented:

Cannot be used at most of the client projects. Because Cordys authentication is 
used predominately 

Alternatives:

None

What steps will reproduce the problem?
1. Install connector 1.2.20
2. Make file service container within System
3. Call file connector method with WS-Security within the context of another 
Organization

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" 
xmlns:v1="http://www.zorgregistratie.nl/azr/schema/AZRMessagingService/v1" 
xmlns:v11="http://www.zorgregistratie.nl/azr/CDM/Metadata/v1">
   <soapenv:Header>
      <wsse:Security soapenv:mustUnderstand="0" xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd">
         <wsse:UsernameToken wsu:Id="UsernameToken-2" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">
            <wsse:Username>[email protected]</wsse:Username>
            <wsse:Password Type="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#PasswordText">test</wsse:Password>
         </wsse:UsernameToken>
      </wsse:Security>
   </soapenv:Header>
  <soapenv:Body>
    <WriteFile xmlns="http://schemas.cordys.com/1.0/ac/FileConnector">
      <filename>/opt/</filename>
      <append>PARAMETER</append>
      <encoded>PARAMETER</encoded>
      <charset>PARAMETER</charset>
      <data>PARAMETER</data>
    </WriteFile>
  </soapenv:Body>
</soapenv:Envelope>

What is the expected output? What do you see instead?
I expect that the method is called, instead i get the following error:
   <soapenv:Header>
      <header xmlns="http://schemas.cordys.com/General/1.0/">
         <sender>
            <reply-to>cn=File Connector,cn=CVZ File Connector,cn=soap nodes,o=system,cn=cordys,cn=development,o=local</reply-to>
            <organizationalContext/>
            <component>cn=CVZ File Connector,cn=soap nodes,o=system,cn=cordys,cn=development,o=local</component>
         </sender>
      </header>
   </soapenv:Header>
   <soapenv:Body>
      <soapenv:Fault>
         <faultcode xmlns:ns0="http://schemas.xmlsoap.org/soap/envelope/">ns0:Client</faultcode>
         <faultstring xml:lang="en-US">Access is denied for the method 'WriteFile'.</faultstring>
         <faultactor>http://schemas.cordys.com/1.0/ac/FileConnector</faultactor>
         <detail>
            <cordys:FaultDetails xmlns:cordys="http://schemas.cordys.com/General/1.0/">
               <cordys:LocalizableMessage>
                  <cordys:MessageCode>Cordys.ESBServer.Messages.accessDeniedOnMethod</cordys:MessageCode>
                  <cordys:Insertion>WriteFile</cordys:Insertion>
               </cordys:LocalizableMessage>
            </cordys:FaultDetails>
         </detail>
      </soapenv:Fault>
   </soapenv:Body>
</soapenv:Envelope>
Also when I put the everyone Role of the file connector.

What version of the product are you using? On what operating system?
I use CU18 on Linux

The FileConnector 1.2.15 does not store the value for "use-simple-xpath" in 
CARS.

This is the configuration in CARS:
<configurations autoStartCount="3">
    <cancelReplyInterval>30000</cancelReplyInterval>
    <gracefulCompleteTime>15</gracefulCompleteTime>
    <abortTime>5</abortTime>
    <jreconfig>
        <param value="-Xmx256M"/>
        <param value="-XX:PermSize=5m"/>
    </jreconfig>
    <routing ui_type="loadbalancing" ui_algorithm="failover">
        <preference>1</preference>
    </routing>
    <configuration implementation="com.cordys.coe.ac.fileconnector.FileConnector" htmfile="/cordys/coe/fileconnector/ac/fileconnectorconfig.html">
        <classpath xmlns="http://schemas.cordys.com/1.0/xmlstore">
            <location>coe/fileconnector/fileconnector.jar</location>
            <location>coe/fileconnector/coelib.jar</location>
            <location>coe/fileconnector/stax/stax2-2.1.jar</location>
            <location>coe/fileconnector/stax/wstx-asl-3.2.3.jar</location>
            <location>coe/fileconnector/stax/stax-api-1.0.jar</location>
        </classpath>
        <startupDependency>
            <namespace>http://schemas.cordys.com/1.0/xmlstore</namespace>
        </startupDependency>
        <Configuration>
            <readerconfigfile>/Cordys/coe/fileconnector/config/reader-config.xml</readerconfigfile>
            <writerconfigfile>/Cordys/coe/fileconnector/config/writer-config.xml</writerconfigfile>
            <readercharset></readercharset>
            <writercharset></writercharset>
            <included-directories></included-directories>
            <reload-configuration>false</reload-configuration>
            <drivemappings/>
        </Configuration>
        <component name="Directory Poller">
            <enabled>false</enabled>
            <configuration-file>/Cordys/coe/fileconnector/config/poller-config.xml</configuration-file>
            <processing-folder>poller/processing</processing-folder>
            <app-processing-folder></app-processing-folder>
            <error-folder>poller/error</error-folder>
            <poll-interval>10</poll-interval>
            <minConcurrentWorkers>1</minConcurrentWorkers>
            <maxConcurrentWorkers>10</maxConcurrentWorkers>
            <defaultUserDn></defaultUserDn>
        </component>
    </configuration>
    <loggerconfiguration>
        <systempolicy>true</systempolicy>
        <log4j:configuration xmlns:log4j="http://jakarta.apache.org/log4j">
            <renderer renderedClass="com.eibus.util.logger.internal.LocalizableLogMessage" renderingClass="com.eibus.util.logger.internal.TextRenderer"/>
            <renderer renderedClass="com.eibus.util.logger.internal.LogMessage" renderingClass="com.eibus.util.logger.internal.TextRenderer"/>
            <root>
                <priority value="warn"/>
                <appender-ref ref="DailyRollingFileAppender"/>
                <appender-ref ref="dbappender"/>
            </root>
            <category name="com.eibus.security.acl">
                <priority value="warn"/>
            </category>
            <category name="com.eibus.license">
                <priority value="warn"/>
            </category>
            <category name="com.eibus.directory">
                <priority value="warn"/>
            </category>
            <category name="com.eibus.soap">
                <priority value="warn"/>
            </category>
            <category name="com.eibus.transport.SOAPMessage">
                <priority value="warn"/>
            </category>
            <category name="com.eibus.transport">
                <priority value="warn"/>
            </category>
            <category name="com.eibus.tools">
                <priority value="warn"/>
            </category>
            <category name="com.eibus.util">
                <priority value="warn"/>
            </category>
            <category name="com.cordys.coe.ac.fileconnector">
                <priority value="warn"/>
            </category>
            <appender name="DailyRollingFileAppender" class="org.apache.log4j.DailyRollingFileAppender">
                <param name="File" value="system#fileconnector_group#fileconnector_system.xml"/>
                <param name="DatePattern" value=".yyyy-MM-dd"/>
                <layout class="org.apache.log4j.xml.XMLLayout">
                    <param name="locationInfo" value="true"/>
                </layout>
            </appender>
            <appender class="com.cordys.logger.appender.CordysZeroConfJDBCAppender" name="dbappender"/>
        </log4j:configuration>
    </loggerconfiguration>
</configurations>

In BOP-4 CU14, FileConnector 1.2.17, when the FileConnector configuration is 
saved and the processor is restarted, the FileConnector tab is emptied.

After saving a configuration change, Cordys will automatically prompt you 
whether the Service Container must be restarted. Then, when it is restarted, 
the first tab page of the config is automatically selected. When you click on 
the last tab again, you will see only a white area. It is necessary to close 
the page and request the properties again in order to see them.

Purpose of code changes on this branch:

 Implemented CreateDirectory WebService as a part of Enhancement request 13. Checked-in sources in the branch "enhancements_12_13" . 


When reviewing my code changes, please focus on:

Added Files:

1.com.cordys.coe.ac.fileconnector.methods.CreateDirectoryMethod
2.test\java\com\cordys\coe\ac\fileconnector\methods\CreateDirectoryMethodTest.ja
va

Modified files :

1.src\java\com\cordys\coe\ac\fileconnector\FileConnector.java
2.src\content\methodsets\Method Set FileConnector.xml


After the review, I'll merge this branch into:
/trunk

The documentation still mentions that file shares are not supported for some 
actions e.g. polling. For example remarks in sections 1.1 and 4.4.
In other locations it is mentioned that using shares *is* possible. It is also 
mentioned in the release notes for certain versions that problems in the 
handling of file shares are solved.
This is confusing. The manual should be correct and specific about these things.
Also "All relative folders are relative to the FileConnector installation 
folder" (4.4) is not correct, at least not in BOP-4: It is relative to 
<Cordys-installdir>\lib.
Furthermore the table of contents needs to be updated. When I try that in my 
downloaded copy, I get the message that there are no table of contents entries 
found in the document.

Purpose of code changes on this branch:
To read excel files with readfilerecords method.

When reviewing my code changes, please focus on:
Readfilerecords Method.

After the review, I'll merge this branch into:
/trunk

Wrong license header is used in Code. It mentions "This file is part of the 
Cordys SAP Connector."

What steps will reproduce the problem?
1. Installing a new (test) environment
2.
3.

What is the expected output? What do you see instead?
The status of the Service Container "FileConnector" is "configuration error"


What version of the product are you using? On what operating system?
BOP 4.2, Windows7, IE8

Please provide any additional information below.
See attachement for the logfile

Feature I would like to see:
  Currently ReadFile supports multibyte character sets. ReadFileRecords also needs to support multibyte character sets.

Impact if not implemented:
  We have a project that uses files written in Japanese. We want to read them using ReadFileRecords, but cannot.

Alternatives:
  None.

Feature I would like to see:
I would like an option to automatically append a new line after a record is 
added to a file with WriteFile webservice.

Impact if not implemented:
The data with each call to WriteFile is appended in a single line.

Alternatives:
Unknown

What steps will reproduce the problem?
1.
2.
3.

What is the expected output? What do you see instead?


What version of the product are you using? On what operating system?

The connector version we are using------>1.2.20
The BOP platform version we are using------>D1.000.014
The API we are using----------------------->ReadFileRecordsOperation



Please provide any additional information below.

Problems with ExcelRead.java:
* It is not in line with the rest of the code.
* Also there are quite a few compiler warnings. Raw types and unused variables.
* There is an an empty catch block. This means exceptions get ignored, which 
can't be correct. The bare minimum should AT LEAST be write the exception to 
the log file. If the processing can continue, then you log it as a warning. But 
usually you throw the exception back up and let a higher level decide what to 
do with the exception

See: 
http://code.google.com/p/cordysfilecon/source/browse/trunk/src/java/com/cordys/c
oe/ac/fileconnector/utils/ExcelRead.java

What steps will reproduce the problem?
1. Put file connector 1.2.19 or 1.2.20 in isv folder
2. Try to install from Cordys
3. When clicking "Manage Applications" Javascript error occurs in 
What is the expected output? What do you see instead?


What version of the product are you using? On what operating system?
Cordys CU19, Windows 2003 Server, IE8

Please provide any additional information below.
On line:
if(isvpNode.selectSingleNode(".//description").getAttribute("canUpgrade")=="fals
e")

Error:
'selectSingleNode(...)' is null or not an object

Feature I would like to see:
Add a tab to the file connector configuration dialog which allows editing of 
the configuration file. Just similar to the trigger configuration tab of the 
Email-IO connector.

Impact if not implemented:
Complicated error prone configuration of file reader and writer using XML store 
explorer.

Alternatives:

It should not be allowed to have non negative or zero as values for polling 
interval or track time as it might lead to thread death which is  difficult to 
diagnose on production environments.

We are using the file connector and configured the file poller. We get errors 
as in attachment. Can you tell us what this error means? The strange thing is 
that the file is from 2014-12-10, but the error only shows up at 2014-12-12. 
Both dates are also visible in the error xml. How come we only get this error 
after two days? And what does the error message mean: "SOAP request has already 
been sent and probably failed"?

What steps will reproduce the problem?
1. Create Drive mappings
2. Start the Service Container
3. Check the debug logging / try to use the drive letter

What is the expected output? What do you see instead?
Expected debug logging:
Current Network mapings : DriveLetter: 'P:'
Location: \\servername\sharename
Username: null
Password: null

Actual debug logging:
No drive mappings defined.


What version of the product are you using? On what operating system?
FileConnector 1.2.17, BOP-4 CU14

Please provide any additional information below.
ApplicationConfiguration.getNetworkDrives() says:
        int[] aiMapping = Find.match(iMappings, "fChild<" + PROP_DRIVE_MAPPING + ">");
This is old style NOM-searching which does not match the value of 
PROP_DRIVE_MAPPING: "/configuration/Configuration/drivemapping"
Changing the value to "drivemapping" fixes the problem, although of course it 
would be better to change to the new search style, and use the correct XPath 
(/configuration/Configuration/drivemappings/drivemapping)

What steps will reproduce the problem?
1. Encode a text without a new line after the last line into Base64
2. Paste the encoded text into the WriteFile method and put <encoded> to true

What is the expected output? What do you see instead?
The last line (the one without the newline) will have unreadable characters 
instead of the expected text.

What version of the product are you using? On what operating system?
FileConnector 1.2.17, BOP-4 CU14

Feature I would like to see:
Write of excel files (xls, xlsx)

Impact if not implemented:
See discussion here: 
https://wiki.cordys.com/display/dsc/Excel+Connector?focusedCommentId=191272744#c
omment-191272744

What steps will reproduce the problem?
1. I configured the file connector with correct XML Configuration
2. The file gets polled off the directory
3. I get an error

What is the expected output? What do you see instead?
I expect the webservice to be called but i get an error:
<?xml version='1.0' 
encoding='UTF-8'?><error-info><error-time>2011-09-28T14:46:24.950</error-time><c
urrent-file>/opt/sw/cordys/BOP/development/coe/fileconnector/poller/error/INCOMI
NG_AZR_FILES-00505699007011e0fd39fb43f117d71f/AW317_v30_OK.txt</current-file><or
iginal-file><path>/opt/sw/data/azr/incoming/AW317_v30_OK.txt</path><last-modifie
d>2011-09-28T14:46:06.000</last-modified></original-file><trace><![CDATA[com.cor
dys.coe.ac.fileconnector.extensions.directorypoller.states.FileStateException: 
Error triggering the webservice for 
/opt/sw/cordys/BOP/development/coe/fileconnector/poller/processing/INCOMING_AZR_
FILES-00505699007011e0fd39fb43f117d71f/AW317_v30_OK.txt
    at com.cordys.coe.ac.fileconnector.extensions.directorypoller.states.StateTrigger.sendTriggerRequest(StateTrigger.java:397)
    at com.cordys.coe.ac.fileconnector.extensions.directorypoller.states.StateTrigger.execute(StateTrigger.java:131)
    at com.cordys.coe.ac.fileconnector.extensions.directorypoller.FileProcessWorker.run(FileProcessWorker.java:95)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:662)
Caused by: com.cordys.coe.util.soap.SOAPException: [faultcode='Client', 
faultactor='', faultstring='Access is denied for the method 
'com.eibus.security.identity.CordysUserIdentityType@1d8adb0'.', 
detail='<cordys:FaultDetails 
xmlns:cordys="http://schemas.cordys.com/General/1.0/"><cordys:LocalizableMessage
 xmlns:cordys="http://schemas.cordys.com/General/1.0/"><cordys:MessageCode 
xmlns:cordys="http://schemas.cordys.com/General/1.0/">Cordys.ESBServer.Messages.
accessDeniedOnMethod</cordys:MessageCode><cordys:Insertion 
xmlns:cordys="http://schemas.cordys.com/General/1.0/">com.eibus.security.identit
y.CordysUserIdentityType@1d8adb0</cordys:Insertion></cordys:LocalizableMessage><
/cordys:FaultDetails><cordys:FaultRelatedException 
xmlns:cordys="http://schemas.cordys.com/General/1.0/"><![CDATA[LDAPException: 
java.lang.NullPointerException (80) Other
LDAPException: Server Message: java.lang.NullPointerException
java.lang.NullPointerException
    at com.eibus.directory.soap.LDAPSearchCache.getLDAPResults(LDAPSearchCache.java:386)
    at com.eibus.directory.soap.Cache.search(Cache.java:339)
    at com.eibus.directory.soap.Cache.search(Cache.java:309)
    at com.eibus.directory.soap.Cache.read(Cache.java:403)
    at com.eibus.directory.soap.Proxy.read(Proxy.java:425)
    at com.eibus.directory.soap.LDAPDirectory.read(LDAPDirectory.java:858)
    at com.eibus.soap.SOAPTransaction.isUserDisabled(SOAPTransaction.java:2351)
    at com.eibus.soap.SOAPTransaction.<init>(SOAPTransaction.java:461)
    at com.eibus.soap.SOAPTransaction.<init>(SOAPTransaction.java:175)
    at com.eibus.soap.Processor.onReceive(Processor.java:956)
    at com.eibus.soap.Processor.onReceive(Processor.java:929)
    at com.eibus.connector.nom.Connector.onReceive(Connector.java:417)
    at com.eibus.transport.Middleware$NonTransactionalWorkerThreadBody.run(Middleware.java:1722)
    at com.eibus.util.threadpool.WorkerThread.run(WorkerThread.java:64)
Caused by: java.lang.NullPointerException
    at com.eibus.directory.soap.LDAPCacheKey.hashCode(LDAPCacheKey.java:131)
    at java.util.Hashtable.get(Hashtable.java:334)
    at com.eibus.util.cache.Cache.getReadLock(Cache.java:177)
    at com.eibus.util.cache.Cache.get(Cache.java:244)
    at com.eibus.directory.soap.LDAPSearchCache.getLDAPResults(LDAPSearchCache.java:365)
    ... 13 more
]]]]><![CDATA[></cordys:FaultRelatedException>']
    at com.cordys.coe.ac.fileconnector.FileConnector$NomConnector.sendAndWait(FileConnector.java:649)
    at com.cordys.coe.ac.fileconnector.extensions.directorypoller.states.StateTrigger.sendTriggerRequest(StateTrigger.java:366)
    ... 5 more
]]></trace><file-states><state type="IN_PROCESSING" 
started="2011-09-28T14:46:24.833" 
finished="2011-09-28T14:46:24.833"><file-id>INCOMING_AZR_FILES-00505699007011e0f
d39fb43f117d71f</file-id><original-file>/opt/sw/data/azr/incoming/AW317_v30_OK.t
xt</original-file><processing-file>/opt/sw/cordys/BOP/development/coe/fileconnec
tor/poller/processing/INCOMING_AZR_FILES-00505699007011e0fd39fb43f117d71f/AW317_
v30_OK.txt</processing-file><original-size>4366</original-size><original-lastmod
ified>2011-09-28T14:46:06.000</original-lastmodified></state><state 
type="TRIGGER" started="2011-09-28T14:46:24.846" finished="" 
/></file-states></error-info>


What version of the product are you using? On what operating system?


Please provide any additional information below.
The XML Store configuration:
<configuration 
xmlns:FCDP="http://schemas.cordys.com/coe/FileConnector/Poller/1.0">
  <folder>
    <name>Incoming AZR files</name>
    <location>/opt/sw/data/azr/incoming</location>
    <filter>*.*</filter>
    <track-time>10</track-time>
    <trigger>
      <method>SendMessage</method>
      <namespace>http://www.zorgregistratie.nl/azr/schema/AZRMessagingService/v1</namespace>
      <organization>o=system,cn=cordys,cn=development,o=gen.cms.local</organization>
      <user>cn=SYSTEM,cn=authenticated users,cn=cordys,cn=development,o=gen.cms.local</user>
      <move-file>false</move-file>
      <parameters>
        <Metadata xmlns="http://www.zorgregistratie.nl/azr/CDM/Metadata/v1">
          <MessageID />
          <MessageName />
          <MessageType>AW317</MessageType>
          <MessageVersion>3.0</MessageVersion>
          <ConversationID />
          <RefMessageID />
          <MessageDescription />
          <Sender />
          <Receiver />
          <SentDate />
        </Metadata>
        <Data>
          <Message />
        </Data>
      </parameters>
    </trigger>
  </folder>
</configuration>

Feature I would like to see:
Read of Excel files (xls, xlsx)

Impact if not implemented:
See discussion here 
https://wiki.cordys.com/display/dsc/Excel+Connector?focusedCommentId=191272744#c
omment-191272744

[deleted issue]

Feature I would like to see:
The business use case demads that the ReadFileRecords method should include the 
probable error records in the response of teh method so that those chucnks can 
be written to a separate file for further analysis of the data inconsistency at 
the source.

Impact if not implemented:
Without this enhancement the solution becomes very trivial with the current 
capabilities of the file connector with the ReadFileRecords method. Usually the 
solution at high level would be validate-correct-validate-correct-process.

Alternatives:
No alternatives possible at present to the best of my knowledge except that the 
generation of the input data file ensures that there are no errors in the data 
being provided. (As per the configurations for the ReadFile)

Patch Details:
The attached patch enhances the ReadFileRecords method for single 
recordsequence. It does not impact anything w.r.t. backwards compatibility. It 
does not alter any of the other response parameters of the method. 
The solution needs future improvements to provide the exact details like line 
number, start and end offsets of the error records which is not yet implemented.

What steps will reproduce the problem?
1.Add date to excel
2.Read date from excel using ReadFileRecords method

What is the expected output? What do you see instead?
In response it should be in Cordys Date format instead it is returned as an 
integer.

What version of the product are you using? On what operating system?
BOP 4.1 CU5, File Connector 1.2.20

What steps will reproduce the problem?
1. Configure the directory poller with polling interval of 5 minutes and track 
time of 5 seconds for a folder
2. Start fileconnector and place a file in the configured input folder 
3. Wait till the run till the file is completely processed 
4. Observe the log file for the time taken for processing the file.
5. Reference: 
   https://support.cordys.com/?INCIDENT_ID=INC9948

What is the expected output? What do you see instead?
- In this case it is expected that the processing of the file should have 
started 5 seconds from placing the new file in the input folder. However, it is 
observed that the processing of the file starts after 10 minutes.


What version of the product are you using? On what operating system?
1) Cordys C3 FP4 CU3
2) Cordys CoE File Connector 1.2 Build 14

Please provide any additional information below.
The fix is incorporated in DirectoryPollerThread as shown in the attached patch.

What steps will reproduce the problem?
Read Excel file with ReadFileRecords method,
Sample Request : 
<ReadFileRecords xmlns="http://schemas.cordys.com/1.0/ac/FileConnector">
      <filename>D:/test.xlsx</filename>
      <filetype>Excel</filetype>
      <numrecords>-1</numrecords>
      <offset>0</offset>
      <validateonly>false</validateonly>
      <usetupleold>false</usetupleold>
</ReadFileRecords>


What is the expected output? What do you see instead?
Response should contain all the valid rows but now it does not return any row.

What version of the product are you using? On what operating system?
BOP 4.1 CU5, Windows 64bit, File Connector 1.2.20

SVN  DIFF file attached.

What steps will reproduce the problem?
1. Create a poller-config.xml which points to a non existent directory
2. File connector fails to start with the below error message:
com.cordys.coe.ac.fileconnector.exception.ConfigException: Unable to load 
configuration from XMLStore.
        at com.cordys.coe.ac.fileconnector.extensions.directorypoller.DirectoryPoller.getConfiguration(DirectoryPoller.java:284)
        at com.cordys.coe.ac.fileconnector.extensions.directorypoller.DirectoryPoller.initialize(DirectoryPoller.java:159)
        at com.cordys.coe.ac.fileconnector.FileConnector.loadExtensions(FileConnector.java:496)
        at com.cordys.coe.ac.fileconnector.FileConnector.open(FileConnector.java:262)
        at com.eibus.soap.Processor.open(Processor.java:755)
        at com.eibus.soap.Processor.startProcessor(Processor.java:1050)
        at com.eibus.soap.ProcessStreamsHandler.startProcessor(ProcessStreamsHandler.java:686)
        at com.eibus.soap.ProcessStreamsHandler$StreamReader.run(ProcessStreamsHandler.java:353)
Caused by: com.cordys.coe.ac.fileconnector.exception.FileException: Unable to 
create input folder: /mnt/hgfs/directory/in
        at com.cordys.coe.ac.fileconnector.extensions.directorypoller.Folder.parseFromXML(Folder.java:179)
        at com.cordys.coe.ac.fileconnector.extensions.directorypoller.Folder.<init>(Folder.java:120)
        at com.cordys.coe.ac.fileconnector.extensions.directorypoller.FolderConfiguration.parseConfiguration(FolderConfiguration.java:122)
        at com.cordys.coe.ac.fileconnector.extensions.directorypoller.FolderConfiguration.<init>(FolderConfiguration.java:62)
        at com.cordys.coe.ac.fileconnector.extensions.directorypoller.DirectoryPoller.getConfiguration(DirectoryPoller.java:280)
        ... 7 more

The message suggests that the configuration file is not found. Looking further 
down the stack trace it becomes clear that directory configured in the 
poller-config.xml is not available.

The expected error message should directly point on:
"Unable to create input folder: /mnt/hgfs/directory/in"

Purpose of code changes on this branch:
Ported the sources to bop 4.1 cu7

When reviewing my code changes, please focus on:
1. coe\fileconnector\ac\fileconnectorconfig.caf (File Connector Configuration 
UI),
2. Java Code (Changed all the Exceptions to throw with IStringResource instead 
of plain text)
3. Message Bundles (Added all the exception messages to message bundles)

After the review, I'll merge this branch into:
/trunk

Feature I would like to see:
If an exception occurs in the file poller we need it to trigger a web service / 
SOAP message. Which SOAP message to send should be configurable.

Impact if not implemented:
Have to use one of the alternatives or project specific patch.

Alternatives:

1) Use a (custom) JMX connector to listen for notifications and call our SOAP 
message.
2) Custom log consumer on a "host"

Link to changeset(s)/location:
/trunk

Purpose of code changes:
File poller doesnot process the files, if polling folder and processing folder 
is configured to different file systems

Please check http://code.google.com/p/cordysfilecon/issues/detail?id=5

Feature I would like to see:
Currently, ReadFileRecords allows only one filetype "excel" for Excel files. We 
would like to use multiple filetypes for Excel files.

Impact if not implemented:
We have a project in which there are multiple excel files whose formats are 
different. We cannot read them using one File connector service container.

Alternatives:
Create multiple File Connector service containers. Specify different 
reader-config.xml for each container. Define different configuration for each 
reader-config.

Feature I would like to see:

 FileConnector to provide APIs for creating and deleting of directories.

Impact if not implemented:

 For our current project, automation of creation and deletion of directories is a key feature. This will remove the necessity of operations team to create the directories manually. 

Alternatives:

 Write bespoke code using ws-apps component.

What steps will reproduce the problem?
1. Create a new fileconnector SG
2. enter data until you reach step 4.
3. message "This content is not in the Standards mode, and cannot be rendered 
in this browser.
To view the content, open with the Internet Explorer browser."

What is the expected output? What do you see instead?
- support non-IE browsers.
- only IE can be used

What version of the product are you using? On what operating system?
- BOP4 FP1 CA
- Fileconnector 1.2.18

Please provide any additional information below.