keeps / roda Goto Github PK

What’s the procedure to send files bigger than 2GB using the VSFTPD?

Use KEEPS horizontal logo

Is it hard to reconfigure the system not to convert the images to the TIFF format (leaving them as JPGs)?

Currently using outdated Axis 1.4, update to Axis 2.x or other newer SOAP/REST platform.

Remove logs older than X months automatically and/or find another way of storing logs that is more scalable.

If you are using your own MySQL database, it may be set to use UTF8 as the default character encoding for text/varchar columns. If so, you will get errors like:

ERROR 1071 (42000) at line 17: Specified key was too long; max key length is 1000 bytes

If you find the offending line in the database schema file, you will see that the column used in the key is declared to be id VARCHAR(500) NOT NULL. With UTF8 encoding, this results in a column needing 1500 bytes. The varchar columns used used in keys need to be declared as latin1, e.g. id VARCHAR(500) CHARACTER SET latin1 NOT NULL.

It isn't really a bug in RODA, as it depeneds on how your database is configured. But it could be something for the documentation.

E.g.: pt.gov.dgarq.roda.wui.management.user.client.images.UserManagementImageBundle
and others.

Just add a new paragraph to the readme.md

What’s the procedure to use the RODA In module? Is it finished and stable?

Atm, the properties file ldap-filter.properties is replicated in 3 different locations:

• roda-installer/src/main/resources/files/core/config/templates/ldap-filter.properties
• roda-installer/src/main/resources/files/ui/config/templates/ldap-filter.properties
• roda-installer/src/main/resources/files/migrator/config/templates/ldap-filter.properties

If a change is needed, it needs to be done in 3 different files.
Perhaps it's better to move it to a common location, like roda-installer/src/main/resources/files/common/config/templates/.

This is a minor security problem as LDAP is not accessible from outside.

Please look for the string ""

getInstance should be synchronized to avoid concurrently creating more than 1 instance.

How does the quarantine work? How can it be set or managed?

Ingest by FTP instructions are no longer valid because of changes in PAM. This support now needs to be reviewed and good instructions need to be created.

From Nicholas del Pozo:

Install instructions for LDAP + libpam-ldap both require a lot of options to be set-up. I understand that somebody with more knowledge in these areas wouldn't find it a problem, but it would good to have some 'default' values to enter recorded in the install documentation.

The log tends to grow too much too fast.
Create a plugin that removes log entries older than N days.

Make sure the log is also saved to a text file.

How does the Preservation process work? How often does it work? How can it be configured? How can it be tested?

Translate all RODA-WUI templates to english. They are located under roda-installer/src/main/resources/files/ui/config/mail/templates and roda-installer/src/main/resources/files/ui/config/templates/mail.

How can we set a development environment for the system? Do we need the Eclipse software running on Debian Linux with Maven? Which plugins are necessary? Which configurations?

Improve the content with more information:

• Who: Agent information
• When: duplicate date (which is on the bottom) with label
• What: action (in english)
• Why: e.g. preservation plan

All the information should be i18n (all) or in english.

Also, escape special characters that maybe on the outcome (e.g. the preservation action output).

Enhance documentation to clarify this.
Describe the fact that those credentials are used to connect to Web services as well.

In localization "properties" files you have items like:
"Creator.structured_text.subtype.MS_POWERPOINT = Microsoft Powerpoint"
There are a few duplicities (same text with different keys), I think that for localization will be better to use general translation of file formats like:
"fileFormat.MS_POWERPOINT = Microsoft Powerpoint"

roda-backup-mysql.sh has hardcoded mysql root password that doesn't change during the install process

The PhpMyAdmin previewer support was removed in version 1.1.0, the support now needs to be re-evaluated.

There are several classes that use String.format to create the sql queries to be executed. This may cause problems when inserting values that are differently formated depending on the locale (float, doubles).

Create a debian package to install RODA. Support last Debian version and last Ubuntu LTS.

HTTP Status 500 - Servlet execution threw an exception

type Exception report

message Servlet execution threw an exception

description The server encountered an internal error that prevented it from fulfilling this request.

exception

javax.servlet.ServletException: Servlet execution threw an exception
pt.gov.dgarq.roda.servlet.RoleFilter.doFilter(RoleFilter.java:118)
pt.gov.dgarq.roda.servlet.LdapAuthenticationFilter.doFilter(LdapAuthenticationFilter.java:356)
root cause

java.lang.NoClassDefFoundError: Could not initialize class org.apache.axis.client.AxisClient
org.apache.axis.client.Service.getAxisClient(Service.java:104)
org.apache.axis.client.Service.(Service.java:113)
pt.gov.dgarq.roda.core.stubs.LoginServiceLocator.(LoginServiceLocator.java:12)
pt.gov.dgarq.roda.core.RODAClient.getLoginService(RODAClient.java:217)
pt.gov.dgarq.roda.core.RODAClient.(RODAClient.java:202)
pt.gov.dgarq.roda.sipcreator.installer.GetRodaIn.createClassificationPlan(GetRodaIn.java:301)
pt.gov.dgarq.roda.sipcreator.installer.GetRodaIn.createGenericInstaller(GetRodaIn.java:366)
pt.gov.dgarq.roda.sipcreator.installer.GetRodaIn.sendInstaller(GetRodaIn.java:223)
pt.gov.dgarq.roda.sipcreator.installer.GetRodaIn.doPost(GetRodaIn.java:186)
pt.gov.dgarq.roda.sipcreator.installer.GetRodaIn.doGet(GetRodaIn.java:151)
javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
javax.servlet.http.HttpServlet.service(HttpServlet.java:723)
pt.gov.dgarq.roda.servlet.RoleFilter.doFilter(RoleFilter.java:118)
pt.gov.dgarq.roda.servlet.LdapAuthenticationFilter.doFilter(LdapAuthenticationFilter.java:356)
note The full stack trace of the root cause is available in the Apache Tomcat/6.0.37 logs.

Apache Tomcat/6.0.37

This files should be in caps:

install.txt
license.txt

Right now, error pages are static pages on the webapp folder and used directly by the web descriptor (web.xml).
Change the web descriptor to point out to RODA webapp menu options that open i18n error page as in Home.html

Example: Point web.xml 404 error page to /#error_404, add menu option "ERROR_404", load error page when menu option selected.

The demo system must be installed with the minimum set of tasks to be able to test the system, e.g.:

• Ingest tasks
• Checksums
• Auto approve sips
• Generate stats

What's necessary to create a valid SIP (.zip)? Any specific hierarchy inside the zip file? Any necessary text or XML file?

Dates are incorrectly stored in the database. Even task that didn't done anything have 1 hour of difference between the start of the task and the end.

This may be needed for the SCAPE Data connector API, so Plans can refer collections made of SRU queries on the characteristics of files.

Depends on #25.

When roda-wui is not the root application (ROOT.war), the PhpMyAdmin cookies don't work and a warning is issued:
"Cookies must be enabled past this point"

This happens because cookies define path as in /PhpMyAdmin, and not relative to the roda-wui application.
This can probably be worked around by changing PhpMyAdmin configuration or by putting roda-wui in ROOT.war

Google analytics code is currently hardcoded in .html pages.

At the moment, all users who can edit medadata, can also remove objects. Perhaps it would be better to separate the role in two.

The system logs (the database ones) are extensive and extremely detailed. If the system is set to perform frequent actions on the files the log will grow too much. About that information:

• Is it possible to remove them? If positive, how?
• Is it possible to reduce their level of details for their final size to be reduced?
• Can the tables be normalized to reduce their growth?

Update characterization process (use Tika or FITS)

Can the system logs (the text file ones) be removed? Can they be set to be stored in another location?

In Firefox 14 and with GWT 2.4.0 custom RODA date picker does work. Should substitute with new Calendar widget.

Possible wait for Fedora Commons 4.0, for scalability reasons.

When scheduling period was 10s, the following error occurred:

ERROR 2012-11-21 12:10:20,149 (TaskInstanceDatabaseUtility) SQLException inserting new TaskInstance - Deadlock found when trying to get lock; try restarting transaction
com.mysql.jdbc.exceptions.jdbc4.MySQLTransactionRollbackException: Deadlock found when trying to get lock; try restarting transaction
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:406)
at com.mysql.jdbc.Util.getInstance(Util.java:381)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1045)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:956)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3536)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3468)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1957)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2107)
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2642)
at com.mysql.jdbc.StatementImpl.executeUpdate(StatementImpl.java:1634)
at com.mysql.jdbc.StatementImpl.executeUpdate(StatementImpl.java:1553)
at pt.gov.dgarq.roda.core.scheduler.TaskInstanceDatabaseUtility.insertTaskInstanceParameter(TaskInstanceDatabaseUtility.java:530)
at pt.gov.dgarq.roda.core.scheduler.TaskInstanceDatabaseUtility.insertTaskInstance(TaskInstanceDatabaseUtility.java:128)
at pt.gov.dgarq.roda.core.scheduler.SchedulerManager.createTaskInstance(SchedulerManager.java:577)
at pt.gov.dgarq.roda.core.scheduler.PluginJob.execute(PluginJob.java:97)
at org.quartz.core.JobRunShell.run(JobRunShell.java:199)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:546)

The start/stop script roda-soffice-headless won't correctly shut down OpenOffice. The executable soffice is a script which calls soffice.bin. The RODA script roda-soffice-headless will kill the script soffice, if it is running, without killing the binary soffice.bin. If soffice.bin is killed first, then the script soffice will terminate anyway.

The line to change is:

killall -9 soffice 2> /dev/null || killall -9 soffice.bin

which, if modified to:

killall -9 soffice.bin 2> /dev/null || killall -9 soffice

will not leave soffice.bin running when it should have been terminated.

OpenLDAP as a sizelimit option at the configuration file (/etc/ldap/slapd.conf) which limits the numbers of results of a search.

There is a need to implement a solution, besides increasing this value, to allow searching the results without crossing the limit.

http://www.ldapadministrator.com/forum/viewtopic.php?t=14

roda-client is nothing more nothing less than roda-core-client and it should be underneath roda-core.

What’s the complete procedure to insert a new language in the system? Do all "pt_PT.properties" system files have to be translated?

The roda-core gives the following error.

ERROR 2013-06-28 18:06:12,267 (NormalizationTaskPlugin) Error executing plugin - Error executing migration - Error converting representation - Exception executing convert command - Command [soundconverter, -b, -q, -m, audio/x-wav, -s, .wav, /home/roda/roda/tomcat/apache-tomcat-6.0.37/temp/rodaSourceRep5437925939103191120temp/F0] terminated with error code 1
pt.gov.dgarq.roda.core.plugins.PluginException: Error executing plugin - Error executing migration - Error converting representation - Exception executing convert command - Command [soundconverter, -b, -q, -m, audio/x-wav, -s, .wav, /home/roda/roda/tomcat/apache-tomcat-6.0.37/temp/rodaSourceRep5437925939103191120temp/F0] terminated with error code 1
    at pt.gov.dgarq.roda.plugins.ingest.NormalizationTaskPlugin.executeNormalizationPlugin(NormalizationTaskPlugin.java:509)
    at pt.gov.dgarq.roda.plugins.ingest.NormalizationTaskPlugin.processRepresentation(NormalizationTaskPlugin.java:396)
    at pt.gov.dgarq.roda.plugins.ingest.NormalizationTaskPlugin.doTask(NormalizationTaskPlugin.java:254)
    at pt.gov.dgarq.roda.core.ingest.IngestTask.processSIP(IngestTask.java:293)
    at pt.gov.dgarq.roda.plugins.common.IngestTaskPlugin.execute(IngestTaskPlugin.java:102)
    at pt.gov.dgarq.roda.core.scheduler.PluginJob.execute(PluginJob.java:103)
    at org.quartz.core.JobRunShell.run(JobRunShell.java:199)
    at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:546)

When testing the command with the same file, the following error occurs:

roda@roda:~/roda$ soundconverter -b -q -m audio/x-wav -s .wav /home/roda/roda/tomcat/apache-tomcat-6.0.37/temp/rodaSourceRep5437925939103191120temp/F0 
SoundConverter 1.5.4
/usr/lib/python2.7/dist-packages/gtk-2.0/gtk/__init__.py:57: GtkWarning: could not open display
  warnings.warn(str(e), _gtk.Warning)
  using Gstreamer version: 0.10.36
  using gio
    'xingmux' gstreamer element not found, disabling Xing Header.
    'lame' gstreamer element not found, disabling MP3.
    'faac' gstreamer element not found, disabling AAC.

(soundconverter:6238): GConf-WARNING **: Client failed to connect to the D-BUS daemon:
Unable to autolaunch a dbus-daemon without a $DISPLAY for X11
Traceback (most recent call last):
  File "/usr/bin/soundconverter", line 449, in <module>
    profiles = _GCONF.get_list(_GCONF_PROFILE_LIST_PATH, 1)
glib.GError: No D-BUS daemon running