artefactual / atom Goto Github PK

During import we are experiencing a type error:

root@server-12851:/usr/share/nginx/server_test/src# php symfony import:bulk /home/artefactual/server/server-migration/migration/archives_migration/
PHP Fatal error:  Uncaught Error: [] operator not supported for strings in /usr/share/nginx/server_test/atom-cefd4f9b59e8d93c16f0b6b8e5625e875f0dc80c/lib/task/import/importBulkTask.class.php:224
Stack trace:
#0 /usr/share/nginx/server_test/atom-cefd4f9b59e8d93c16f0b6b8e5625e875f0dc80c/lib/task/import/importBulkTask.class.php(92): importBulkTask->dir_tree('/home/artefactu...')
#1 /usr/share/nginx/server_test/atom-cefd4f9b59e8d93c16f0b6b8e5625e875f0dc80c/vendor/symfony/lib/task/sfBaseTask.class.php(68): importBulkTask->execute(Array, Array)
#2 /usr/share/nginx/server_test/atom-cefd4f9b59e8d93c16f0b6b8e5625e875f0dc80c/vendor/symfony/lib/task/sfTask.class.php(97): sfBaseTask->doRun(Object(sfCommandManager), NULL)
#3 /usr/share/nginx/server_test/atom-cefd4f9b59e8d93c16f0b6b8e5625e875f0dc80c/vendor/symfony/lib/command/sfSymfonyCommandApplication.class.php(76): sfTask->runFromCLI(Object(sfCommandManager), NULL)
#4 /usr/share/nginx/server_test/atom-cefd4f9b59e8d93c16f0b6b8e5625e875f0dc80c/vendor/symfony/lib/command/cli.php(20): sfSymfonyCommandApplicat in /usr/share/nginx/server_test/atom-cefd4f9b59e8d93c16f0b6b8e5625e875f0dc80c/lib/task/import/importBulkTask.class.php on line 224

From @sbreker The issue is happening because of this line in importBulkTask and more specifically because of a breaking change in PHP 7.1 documented here.

Atom workers are notorious for dieing and causing 500 errors on the frontend.

Please change the default behaviour so services are restarted on failing.

Most recent relevant discussion is the first link, example configuration (copied below for ease of access) is the second.
https://groups.google.com/forum/#!msg/ica-atom-users/bblncVj_Vi0/YRLMWuosCAAJ

https://groups.google.com/forum/#!msg/ica-atom-users/X-07V3RUusw/3sHhpsrzCAAJ

[Unit]
Description=AtoM worker
After=network.target
StartLimitIntervalSec=60
StartLimitBurst=12

[Install]
WantedBy=multi-user.target

[Service]
Type=simple
User=www-data
Group=www-data
WorkingDirectory=/usr/share/nginx/atom
ExecStart=/usr/bin/php -d memory_limit=-1 -d error_reporting="E_ALL" symfony jobs:worker
ExecStop=/bin/kill -s TERM $MAINPID
Restart=always
RestartSec=5

Expected behaviour
AtoM should create an information object out of the Dublin Core metadata entered via the GUI in Archivmatica.

Current behaviour
No Information Object is created and none of the DC metadata appear in AtoM. Note that the error may be caused by the presence of a second structMap in the METS file when a transfer is placed in backlog. This is a logical structMap with LABEL="Hierarchical". For transfers run straight through Archivematica without being placed in backlog, there is only the default physical structMap. It's possible that AtoM doesn't know what to do with the fact that the dmdSec is referenced from two structMaps.

Steps to reproduce
-In Archivematica, create a transfer and place it in transfer backlog
-In the Appraisal Tab, retrieve the transfer and create a SIP from it
-Enter DC metadata via the GUI
-Create a DIP and upload it to AtoM

Your environment (version of Archivematica, OS version, etc)
AM 1.9.1, AtoM 2.4.1

We get broken image paths after having imported some PDFs using the API. Our request data payload looks like this (python):

{
        'information_object_slug': obj.metadata['atom_id'],
        'media_type': 'text',
        'mime_type': 'application/pdf',
        'uri': f"{repository_uri}/file/{obj.id}/data/pdf/{obj.id}.pdf",
        'name': f"{obj.id}.pdf"
}

Where 'uri' is an external link where the PDF document is located. Clicking on "Open original digital object" in atom's interface works - it is just the thumbnail that is broken (or non-existent?).

Hi,

when trying to start the docker stack from 2.4.x branch the build is failing with:

In file included from /pecl-memcache-NON_BLOCKING_IO_php7/php7/memcache_pool.c:38:0:
/pecl-memcache-NON_BLOCKING_IO_php7/php7/memcache_pool.c: In function 'mmc_buffer_alloc':
/usr/local/include/php/ext/standard/php_smart_string.h:76:40: error: 'newlen' undeclared (first use in this function)
  smart_string_alloc4((d), (n), (what), newlen)
                                        ^
/usr/local/include/php/ext/standard/php_smart_string.h:58:3: note: in definition of macro 'smart_string_alloc4'
   newlen = (n);            \
   ^~~~~~
/pecl-memcache-NON_BLOCKING_IO_php7/php7/memcache_pool.c:47:2: note: in expansion of macro 'smart_string_alloc'
  smart_string_alloc((&(buffer->value)), size, 0);
  ^~~~~~~~~~~~~~~~~~
/usr/local/include/php/ext/standard/php_smart_string.h:76:40: note: each undeclared identifier is reported only once for each function it appears in
  smart_string_alloc4((d), (n), (what), newlen)
                                        ^
/usr/local/include/php/ext/standard/php_smart_string.h:58:3: note: in definition of macro 'smart_string_alloc4'
   newlen = (n);            \
   ^~~~~~
/pecl-memcache-NON_BLOCKING_IO_php7/php7/memcache_pool.c:47:2: note: in expansion of macro 'smart_string_alloc'
  smart_string_alloc((&(buffer->value)), size, 0);
  ^~~~~~~~~~~~~~~~~~
make: *** [Makefile:196: php7/memcache_pool.lo] Error 1
ERROR: Service 'atom' failed to build: The command '/bin/sh -c set -xe 		&& apk add --no-cache --virtual .phpext-builddeps 

docker-compose version 1.23.2, build 1110ad01
on mac.

Cheers,

Toni

Hello, I'm trying to run Atom using Docker Compose.

Whey doing docker-compose up -d, I get

ERROR: Error: image sevein/gearmand not found

$ git branch -a
  qa/2.4.x
* stable/2.3.x
  remotes/origin/HEAD -> origin/qa/2.4.x
:

$ docker-compose --version
docker-compose version 1.8.0-rc2, build c72c966

Which gearmand can I use instead?

Hello, I'm considering deploying AtoM on a Debian 10 machine, which ships with PHP 7.3 instead of PHP 7.2 as recommended in the installation docs. Should I expect any issues due to this? Thanks.

Expected behaviour
By default, the clipboard data should be purged by the following day.

Current behaviour
Clipboard data remains cached; cron job does not appear to be working as expected.

Steps to reproduce

1. Without clearing my browser cache, I add multiple items to clipboard.

2. Additionally, I save content to multiple clipboard IDs.

3. Wait until next day to verify whether data remains on clipboard.

Content is still there. This is true for both public and authenticated users.

Your environment (version of Archivematica, operating system, other relevant details)

Tested on v 2.5.3 - 172.

Question from Support team: Is this an implementation issue or a docs issue?

For Artefactual use:

Before you close this issue, you must check off the following:

• All pull requests related to this issue are properly linked
• All pull requests related to this issue have been merged
• A testing plan for this issue has been implemented and passed (testing plan information should be included in the issue body or comments)
• Documentation regarding this issue has been written and merged (if applicable)
• Details about this issue have been added to the release notes (if applicable)

This two functions slows it down to 0.85ms. I changed it and it does 0.09ms

arElasticSearchInformationObjectPdo.class.php

private function getCollectionRootId() //slows down to 0.224ms
{
//$sql = 'SELECT id FROM information_object force index (information_object_FI_17)
// WHERE parent_id = ? AND lft < ? AND rgt > ?
//';
$sql = 'SELECT id FROM (SELECT * FROM information_object WHERE lft < ? AND rgt > ? ) a WHERE a.parent_id = ?';

return QubitPdo::fetchColumn($sql, array(QubitInformationObject::ROOT_ID, $this->lft, $this->rgt));

public function getAncestors()
{
if (!isset($this->ancestors))
{
// Find ancestors
$sql = 'SELECT
node.id,
identifier,
repository_id,
slug';
$sql .= ' FROM '.QubitInformationObject::TABLE_NAME.' node force index (information_object_FI_18)';
$sql .= ' JOIN '.QubitSlug::TABLE_NAME.' slug
ON node.id = slug.object_id';
$sql .= ' WHERE node.lft < ? AND node.rgt > ?';
$sql .= ' ORDER BY lft';

  $this->ancestors = QubitPdo::fetchAll($sql, array($this->__get('lft'), $this->__get('rgt')));

information_object_FI_18 = index on lft and rgt

When attempting to browse the Accession listings, I receive a status 500 in my browser and the following in the nginx error log:

2020/09/29 18:37:04 [error] 574#574: *60 FastCGI sent in stderr: "PHP message: PHP Parse error: syntax error, unexpected ',' in /usr/share/nginx/atom/plugins/qtAccessionPlugin/modules/accession/actions/browseAction.class.php on line 125" while reading response header from upstream, client: 10.10.10.1, server: _, request: "GET /accession/browse HTTP/1.1", upstream: "fastcgi://unix:/var/run/php-fpm.atom.sock:", host: "10.10.10.10", referrer: "http://10.10.10.10/taxonomy/list"

Hi folks,

I wanted to bring the issue of web accessibility to your attention for this project as there are a few low-hanging fruit that could be addressed in order to create a much better experience for users of assistive technology like screen readers.

Our local legislation references an international accessibility standard - The Web Content Accessibility Guidelines (WCAG) version 2.1. This document would be a great resource for the implementation of accessibility features in AtoM.

In particular, the most prevalent issues that I saw in my quick checking were:

1. Low contrast between text and its background in the Dominion theme. I haven't checked the Archives Canada theme for this issue. Examples of this issue include the link color, orange h2's, light grey h3's.
2. A lack of labels for form fields sporadically throughout the application including the search and advanced search forms

If these issues could be addressed, it would go a long way toward creating a better experience for all users of AtoM.

The index functions associated with bulk updating publication status do not get initialized with a language code to be able to read title into the index. NB. for the casual reader, this makes the search results look pretty bad - but it doesn't affect what is retrieved for the archival description.

For example, initial import (doesn't matter if it is Draft or Published) looks as follows:

After running the update publication status task we see the following:

Various fields are not correctly read and output in the ES index here.

The correct output should normally look something like:

Vs.

Docker commands used:

• docker-compose exec atom php symfony tools:purge --demo
• docker-compose exec atom php symfony csv:import --source-name=un /home/www-data/error-demo-draft.csv --index
• docker-compose exec atom php symfony tools:update-publication-status published a-simple-title

Test CSV:

error-demo-draft.zip

What are we seeing in the code

• updatePublicationStatusTask is the primary runner for this work.
• arElasticSearchModelBase is the code run by both the initial CSV import and the update task to add data to the ES index. We never progress far enough into this loop because $allowedLanguages is never set.
• QuitbitSetting is the place where the ii18n value should be read into the settings.

It looks like the bulk publication task hasn't been updated in a while so this might go back a few versions of AtoM.

The virutal set oai:virtual:top-level-records does not appear in ListSets.

When building docker images and composing them, the stated branch is qa/2.5.x, while the name of the default branch is qa/2.6.x.

Hi

The current LTS release for Ubuntu is 20.04. Because of the LTS status I followed your Ubuntu 18.04 guide.
Currenty PHP version is 7.4 and the first issue I've found is about strings with curly brackets substrings. This specific error seems easy to fix.

2021/03/18 21:05:07 [error] 1692#1692: *1 FastCGI sent in stderr: "PHP message: PHP Deprecated:  Array and string offset access syntax with curly braces is deprecated in /usr/share/nginx/atom/vendor/symfony/lib/plugins/sfPropelPlugin/lib/vendor/phing/system/io/UnixFileSystem.php on line 78PHP message: PHP Deprecated:  Array and string offset access syntax with curly braces is deprecated in /usr/share/nginx/atom/vendor/symfony/lib/plugins/sfPropelPlugin/lib/vendor/phing/system/io/UnixFileSystem.php on line 79PHP message: PHP Deprecated:  Array and string offset access syntax with curly braces is deprecated in /usr/share/nginx/atom/vendor/symfony/lib/plugins/sfPropelPlugin/lib/vendor/phing/system/io/UnixFileSystem.php on line 91PHP message: PHP Deprecated:  Array and string offset access syntax with curly braces is deprecated in /usr/share/nginx/atom/vendor/symfony/lib/plugins/sfPropelPlugin/lib/vendor/phing/system/io/UnixFileSystem.php on line 112PHP message: PHP Deprecated:  Array and string offset access syntax with curly braces is deprecated in /usr/share/nginx/atom/vendor/symfony/lib/plugins/sfPropelPlugin/lib/vendor/phing/system/io/UnixFileSystem.php on line 125PHP message: PHP Deprecated:  Array and string offset access syntax with curly braces is deprecated in /usr/share/nginx/atom/vendor/symfony/lib/plugins/sfPropelPlugin/lib/vendor/phing/system/io/UnixFileSystem.php on line 143PHP message: PHP Deprecated:  Array and string offset access syntax with curly braces is deprecated in /usr/share/nginx/atom/vendor/symfony/lib/plugins/sfPropelPlugin/lib/vendor/phing/system/io/UnixFileSystem.php on line 157PHP message: PHP Deprecated:  Array and string offset access syntax with curly braces is deprecated in /usr/share/nginx/atom/vendor/symfony/lib/plugins/sfPropelPlugin/lib/vendor/phing/system/io/UnixFileSystem.php on line 197PHP message: PHP Deprecated:  Array and string offset access syntax with curly braces is deprecated in /usr/share/nginx/atom/vendor/symfony/lib/plugins/sfPropelPlugin/lib/vendor/phing/tasks/s
2021/03/18 21:05:07 [error] 1692#1692: *1 upstream sent too big header while reading response header from upstream, client: 192.168.149.66, server: _, request: "GET /index.php/sfInstallPlugin/insertSql HTTP/1.1", upstream: "fastcgi://unix:/run/php7.4-fpm.atom.sock:", host: "192.168.141.33", referrer: "http://192.168.141.33/index.php/sfInstallPlugin/configureDatabase"

Hi,
It would be great to import more than just DC Elements in to atom, DC terms has a much larger set and covers many more of the ISAD fields that atom is already storing internally.

This is a Wishlist item which is already open somewhere but I couldn't find it.

When I try to export a report (box label or physical location) for a "file", I have these messages in the job manager:

Job 525 "arGenerateReportJob": No results found for Étiquette de boîte report.

or

Job 519 "arGenerateReportJob": No results found for Localisations physiques report.

And no report are generated.

Obviously my installation is in French. I have no problem to export the list of file of a fonds but the html/csv output does not contain the location information and this information is present.

worker log:

2018-02-06 05:10:08 > New ability: arFindingAidJob
2018-02-06 05:10:08 > New ability: arInheritRightsJob
2018-02-06 05:10:08 > New ability: arObjectMoveJob
2018-02-06 05:10:08 > New ability: arInformationObjectCsvExportJob
2018-02-06 05:10:14 > Ability not defined: qtSwordPluginWorker. Please ensure the job is in the lib/task/job directory or that the plugin is enabled.
2018-02-06 05:10:14 > New ability: arUpdatePublicationStatusJob
2018-02-06 05:10:14 > New ability: arFileImportJob
2018-02-06 05:10:14 > New ability: arInformationObjectXmlExportJob
2018-02-06 05:10:14 > New ability: arXmlExportSingleFileJob
2018-02-06 05:10:14 > New ability: arGenerateReportJob
2018-02-06 05:10:14 > New ability: arActorCsvExportJob
2018-02-06 05:10:14 > New ability: arActorXmlExportJob
2018-02-06 05:10:14 > New ability: arRepositoryCsvExportJob
2018-02-06 05:10:14 > Running worker...
2018-02-06 05:10:14 > PID 18191
2018-02-06 10:11:34 > Job started.
/sites/xxxxxxx/downloads/reports2018-02-06 10:11:34 > No results found for Étiquette de boîte report.
2018-02-06 10:11:34 > Job finished.

With AtoM now being translated via https://translations.artefactual.com/ and Weblate, it should be possible to remove references to Transifex in the AtoM source. I've spotted a handful in the build.xml e.g. here, and a cursory search across files in the 2.6.x branch shows some references in ~4 other files. Removing these will simplify the code-base for users and maintainers.

Hi, are your mail in SECURITY.md works yet? i just want to report some thing.

Hi,
I'm generating DIPs from Archivematica-1.7.2 and exporting to atom 2.4.0.

Atom won’t correctly import my entries if they have non-dc metadata; it won't just skip the stuff it doesn’t understand but skips all metadata entries.

The metadata.csv I feed in to archivematica is added to METS[...].xml. For example, if "unknown meta" is a heading entry in metadata.csv (and therefor a record in METS), no metadata is imported in to AtoM at all.

No supplied metadata imported: filename,dc.title,dc.date,”unknown meta"
All supplied metadata imported: filename,dc.title,dc.date,dc.subject

What I was expecting is the behaviour Atom exhibits for dcterms metadata: it will import dc. items but ignore dcterms if mixed in same file.

Abstract skipped, others imported: filename,dc.title,dc.date,dcterms.abstract
All supplied metadata imported: filename,dc.title,dc.date,dc.subject

The full thread+description can be found in the following google group.

https://groups.google.com/d/msg/archivematica/I_ireEfxDEA/3plLw0ILCQAJ

Hello, I'd like to know if it's possible to run multiple independent instances of AtoM on a signle server, sharing the same code-base? Have other users deployed AtoM in a similar configuration? What would be required? Thanks.

Hello,

I got the latest Docker based AtoM and build it with docker-compose. At the end I caught an error like that:

ERROR: for docker_atom_worker_1 Cannot start service atom_worker: OCI runtime create failed: container_linux.go:348: starting container process caused "exec: "/atom/src/docker/entrypoint.sh": permission denied": unknown

ERROR: for docker_atom_1 Cannot start service atom: OCI runtime create failed: container_linux.go:348: starting container process caused "exec: "/atom/src/docker/entrypoint.sh": permission denied": unknown

ERROR: for atom Cannot start service atom: OCI runtime create failed: container_linux.go:348: starting container process caused "exec: "/atom/src/docker/entrypoint.sh": permission denied": unknown

ERROR: for atom_worker Cannot start service atom_worker: OCI runtime create failed: container_linux.go:348: starting container process caused "exec: "/atom/src/docker/entrypoint.sh": permission denied": unknown

Mayby there is any idea what happend? Thanks for an answer.

Kind Regards, Maxim

In the response to an "Identify" request, ampersands may not be correctly escaped. For example:
https://catalogue.millsarchive.org/index.php/;oai?verb=Identify&
(The XML may not render correctly in your browser because of this very issue! Use "View Source".)

The response contains the text:
<repositoryName>Images & Documents</repositoryName>
This does not appear to be valid XML. The ampersand sign & should be escaped as &amp;

Got an error when building a nested set:
PDOException: SQLSTATE[HY000]: General error: trying to execute an empty query in /var/www/html/atom-2.4.1/lib/task/propel/propelBuildNestedSetTask.class.php:126
Stack trace:
#0 /var/www/html/atom-2.4.1/lib/task/propel/propelBuildNestedSetTask.class.php(120): PDO->exec('')

What's happening is the loop at line 117 run's 1 too many times if the row count is a multiple of 4000.
We had exactly 84000 records and got the error above.
So I think the fix is:

for ($i=0; $i <= (count($this->rows)-1); $i+=$incr)
{
$sql = implode("\n", array_slice($this->rows, $i, $incr));
$conn->exec($sql);
}

Hi,
In InformationObject's browseSuccess template, the value of "Hierarchy" is hard coded, c.f. this line.
Other templates, e.g. this one, make correct use of the translation key.
Cheers,
Raphaël

Helo!

Is it possible to change ldapUser.class.php to authenticate using the Active Directory attribute sAMAccountName?

Hi,

I recently came to discover that the arOaiPlugin does not convert between UTC and local time. When one uses the from and until url parameters with oai, the result will be requested from the database exactly as it was sent to Atom, where it should be converted to local time first.

Example of current behaviour (omitting dates for clarity);
I modify an object at 20:00 AEST (+10).
I make an oai request with from=09:30&until=10:30
No results are returned
I make an oai request with from=19:30&until=20:30.
OAI returns the object with the a timestamp of 10:00

Since the OAI standard dictates that all date/times are to be in UTC, the above results are incorrect and should look more like below:

I modify an object at 20:00 AEST.
I make an oai request with from=09:30&until=10:30
OAI returns the object with a timestamp of 10:00.

A solution I have put in place was to modify the setUpdateParametersFromRequest within arOaiPluginComponent.class.php. I simply run strtotime on $request->from and $request->until and this converts the date/time to my local timezone, ready to be looked up in the database.

Cheers

Hello,

If I type 1950 or 1950-12-20 or any other dates in the correct format on Identity area > Date(s) > Start (or End), the value isn't correctly stored in the database.

So, when I edit my information object, fields are filled with 0.

It works if I change to field type in database : from DATE to VARCHAR for example.

ATOM 2.6 on Ubuntu 20.04.

When viewing levels-of-description in german Objekt is listed twice

The wrong translation might be here:

where item should say "Artikel"

It appears that the finding aid $exportLanguage and $sourceLanguage variables are not used in the processing of the finding aid.

This code will never work as QubitSetting::getByNameAndScope() returns a QubitSetting object, not a string value.
The code should be:

QubitSetting::getByNameAndScope('informationobject', 'default_template')->getValue() === 'rad'

This issue goes back to 2.4 and possibly further.

AtoM version

qa/2.x (2e2ce4a)

Installation environment

Docker compose environment (Docker Desktop for Windows v20.10.7; PHP 7.4.20 (cli); Percona 8.0.22-13; Elasticsearch v5.6.16; Windows 10 Home, version 2004)

Steps to reproduce the issue

1. Run the PHPunit test suite:

docker-compose exec atom composer test

The resulting error

PHPUnit 9.5.4 by Sebastian Bergmann and contributors.

...........................F...................................  63 / 165 ( 38%)
............................................................... 126 / 165 ( 76%)
.......................................                         165 / 165 (100%)

Time: 00:00.081, Memory: 20.00 MB

There was 1 failure:

1) PhysicalObjectCsvImporterTest::testSetFilenameFileUnreadableException
Failed asserting that exception of type "sfException" is thrown.

FAILURES!
Tests: 165, Assertions: 400, Failures: 1.
Script phpunit handling the test event returned with error code 1

Expected outcome

All tests should pass

This has been talked about on the forum as a "ghost bug" that people couldn't reproduce.

When search input field on "blur" event is triggered, there is only a 150 milliseconds before it disappears. If this occurs before the mouse-up event on the suggestion, the mouse-up event is never fired due to the suggestions being no longer visible.

AtoM DIP uploads fails and nginx error log says:

2019/07/12 10:43:53 [error] 5616#5616: *124382 FastCGI sent in stderr: "PHP message: No Gearman worker available that can handle the job qtSwordPluginWorker" while reading response header from upstream, client: 51.89.136.99, server: host.accesstomemory.org, request: "POST /sword/deposit/demo HTTP/1.1", upstream: "fastcgi://unix:/var/run/php-fpm.atom.sock:", host: "host.accesstomemory.org"

Checking gearmand from atom-worker, the qtsword plugins was shown as enabled:

root@atom-host:/usr/share/nginx/udundee/src# sudo -u www-data php symfony jobs:worker
2019-07-12 12:37:59 > New ability: arFindingAidJob
2019-07-12 12:37:59 > New ability: arInheritRightsJob
2019-07-12 12:37:59 > New ability: arCalculateDescendantDatesJob
2019-07-12 12:37:59 > New ability: arObjectMoveJob
2019-07-12 12:37:59 > New ability: arInformationObjectCsvExportJob
2019-07-12 12:37:59 > New ability: qtSwordPluginWorker
2019-07-12 12:37:59 > New ability: arUpdatePublicationStatusJob
2019-07-12 12:37:59 > New ability: arFileImportJob
2019-07-12 12:37:59 > New ability: arInformationObjectXmlExportJob
2019-07-12 12:37:59 > New ability: arXmlExportSingleFileJob
2019-07-12 12:37:59 > New ability: arGenerateReportJob
2019-07-12 12:37:59 > New ability: arActorCsvExportJob
2019-07-12 12:37:59 > New ability: arActorXmlExportJob
2019-07-12 12:37:59 > New ability: arRepositoryCsvExportJob
2019-07-12 12:37:59 > New ability: arUpdateEsIoDocumentsJob
2019-07-12 12:37:59 > Running worker...
2019-07-12 12:37:59 > PID 7224

The worker and gearmand services were up:

root@atom-host:/var/log/nginx# ps aux | grep jobs:worker
root      7438  0.0  0.0  14856  1116 pts/0    S+   11:40   0:00 grep --color=auto jobs:worker
www-data 22971  0.0  0.6 361948 42592 ?        Ss   Jul10   0:48 /usr/bin/php7.2 -d memory_limit=-1 -d error_reporting=E_ALL /usr/share/nginx/host/src/symfony jobs:worker
root@atom-host:/var/log/nginx# ps aux | grep gearman
root      7440  0.0  0.0  14856  1012 pts/0    S+   11:40   0:00 grep --color=auto gearman
gearman  12852  0.0  0.0 497876  5668 ?        Ssl  Jun06  13:46 /usr/sbin/gearmand --pid-file=/run/gearman/gearmand.pid --listen=127.0.0.1 --port=4730 --daemon --log-file=/var/log/gearman-job-server/gearmand.log --queue-type=builtin

There's no errors in gearmand logs.

After restarting services in AtoM, the issue was fixed, the same DIP were upload to the same slug:

root@atom-host:/var/log/nginx# sudo service gearman-job-server restart
root@atom-host:/var/log/nginx# sudo service php7.2-fpm restart
root@atom-host:/var/log/nginx# sudo service atom-worker-host restart

No Archivematica server actions were done, so the issue seems located in AtoM server.

System Specs: AtoM 2.5.1 (4bc5202), Ubuntu Bionic, php7.2-fpm

I could reproduce the same issue in other AtoM 2.5 and Ubuntu Bionic deploy.

Hi,
I spotted a 500 error in our search results and it appears to be caused when Google triggers the "clear clipboard" functionality, specifically the url https://sparc.utas.edu.au/index.php/user/clipboardClear .

Nginx logs the following which appears to come from clipboardClearAction.class.php.

2018/11/29 01:31:43 [error] 21583#21583: *45738 FastCGI sent in stderr: "PHP message: Invalid entity type in clear clipboard:" while reading response header from upstream, 

However clipboardClearAction.class.php looks like it should include the type, which our error doesn't have.

While not a common error, I also found this result in googles index which even includes an error message.
https://archivesholdings.worldbank.org/user/clipboardClear?sf_culture=es&errorMsg=Invalid+entity+type+in+clear+clipboard%3A+

Points to how I can go about debugging this would be great, as AtoM's logs are empty.

Currently it is not possible to change the slug of an authority record once it has been created. Advice from Dan in the forum is to use SQL against the database directly to do this.
Regenerating slugs

In order to avoid this I modified the QubitActor.php file as per the following diff:

index f36bfebc2..4869dcadb 100644
--- a/lib/model/QubitActor.php
+++ b/lib/model/QubitActor.php
@@ -140,6 +140,17 @@ class QubitActor extends BaseActor
     {
       $this->parentId = QubitRepository::ROOT_ID;
     }
+
+    // Added by rferris to allow changing authority record slugs
+    $newQubitSlug = QubitSlug::getByObjectId($this->id);
+    if (isset($newQubitSlug)) {
+        $slug = QubitSlug::slugify($this->__get('authorizedFormOfName', array('sourceCulture' => true)));
+        if ($slug !== $this->slug) {
+            $this->slug = $slug;
+            $newQubitSlug->slug = $slug;
+            $newQubitSlug->save();
+        }
+    }

This allows a user to simply edit the record, change the title, hit save, and have the slug change accordingly. As far as I can tell this is effectively no different to doing this via SQL and the CLI slug utility. Whether it's the way the team at Artefactual would go about this or not, I'm not sure. I did, however, want to provide my change for your consideration.

Hi,
The Language menu have some Languages starting with capital leters and others with lowercase.
Needs to be changed at:
atom/vendor/symfony/lib/i18n/data/pt.dat
atom/vendor/symfony/lib/i18n/data/fr.dat
atom/vendor/symfony/lib/i18n/data/es.dat

Hi there does atom support a REST API ? Is there a possibility to write one as a plugin ?

Thank you

OS: debian-10-buster-v20210122
Version: 2.6.2

On a fresh VM where I only installed docker & docker-compose something is not working with elasticsearch when following this guide.

When running the sudo docker-compose -f docker/docker-compose.dev.yml exec atom php symfony tools:purge --demo there is an error where it cannot resolve some host. Full output below:

robin@atom:~/atom$ sudo docker-compose -f docker/docker-compose.dev.yml up -d --force-recreate
Building with native build. Learn about native build in Compose here: https://docs.docker.com/go/compose-native-build/
Creating network "docker_default" with the default driver
Creating docker_gearmand_1      ... done
Creating docker_memcached_1     ... done
Creating docker_nginx_1         ... done
Creating docker_atom_1          ... done
Creating docker_elasticsearch_1 ... done
Creating docker_percona_1       ... done
Creating docker_atom_worker_1   ... done
robin@atom:~/atom$ sudo docker-compose -f docker/docker-compose.dev.yml exec atom php symfony tools:purge --demo
>> schema    converting "/atom/src/config/schema.yml" to XML
>> schema    putting /atom/src/config/generated-schema.xml
>> schema    converting "/atom/src/plugins/qbAclPlugin/config/schema.yml" to XML
>> schema    putting /atom/src/plugins/qbAclPlugin/config/generated-qbAclPlugin-schema.xml
>> schema    converting "/atom/src/plugins/qtAccessionPlugin/config/schema.yml" to XML
>> schema    putting /atom/src/plugins/qtAccessionPlugin/config/generated-qtAccessionPlugin-schema.xml
>> file+     config/generated-qbAclPlugin-schema.xml
>> file-     /atom/src/plugins/qbAclPlugin/config/generated-qbAclPlugin-schema.xml
>> file+     config/generated-qtAccessionPlugin-schema.xml
>> file-     /atom/src/plugins/qtAccessionPlugin/config/generated-qtAccessionPlugin-schema.xml
>> propel    Running "insert-sql" phing task
>> file-     /atom/src/config/generated-schema.xml
>> file-     /atom/src/config/generated-qbAclPlugin-schema.xml
>> file-     /atom/src/config/generated-qtAccessionPlugin-schema.xml

                                                 
  Couldn't resolve host                          
                                                 

ERROR: 1
robin@atom:~/atom$ sudo docker-compose -f docker/docker-compose.dev.yml exec atom make -C plugins/arDominionPlugin
make: Entering directory '/atom/src/plugins/arDominionPlugin'
Running LESS compiler...
lessc --compress --relative-urls css/main.less > css/main.css
make: Leaving directory '/atom/src/plugins/arDominionPlugin'

And when checking what's running we can see that Elasticsearch has exited (it never even starts from what i can see from the logs):

robin@atom:~/atom$ sudo docker-compose -f docker/docker-compose.dev.yml ps
         Name                       Command               State                   Ports                
-------------------------------------------------------------------------------------------------------
docker_atom_1            /atom/src/docker/entrypoin ...   Up       9000/tcp                            
docker_atom_worker_1     /atom/src/docker/entrypoin ...   Up       9000/tcp                            
docker_elasticsearch_1   /bin/bash bin/es-docker          Exit 1                                       
docker_gearmand_1        docker-entrypoint.sh gearmand    Up       127.0.0.1:63005->4730/tcp           
docker_memcached_1       docker-entrypoint.sh -p 11 ...   Up       127.0.0.1:63004->11211/tcp          
docker_nginx_1           /docker-entrypoint.sh ngin ...   Up       0.0.0.0:63001->80/tcp               
docker_percona_1         /docker-entrypoint.sh mysqld     Up       127.0.0.1:63003->3306/tcp, 33060/tcp

Attaching full log output below.

Is there something I can do to get this working? I have no clue of where to start looking

atom_logs.txt

AtoM is using jQueryUI vesion 1.12.1 and when I run Tenable in the server it finds out that there are 3 severe security Issues:

https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-41184

https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-41182

https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-41183

I recommend to exchange to latest jQueryUI or at least version 1.13.0.

I'd like to suggest to increase the "visibility" of translations on AtoM's website by using the available widgets at https://translations.artefactual.com/widgets/atom/.
IMO, just because a website is translated into a specific language, it doesn't necessarily imply, that the actual application is translated as well. So, … I've done some changes on the HTML for the "Home" page (in the paragraph "Multilingual") and on the "Download" page (added a new section). Both can be reviewed in the attached file atom.zip

What is needed:

1. Weblpate's defined link for widgets at https://translations.artefactual.com/widgets/atom/ and then go to: Image multi > 4th color variant
   This could be added as a new <section> on the "Download" page … my suggestion would be to put it in front of the section about "Download and install AtoM on your webserver".
   The code would be the following:

        <section>
          <h4 id=translation-status>Translation status</h4>
          
<a href="http://translations.artefactual.com/engage/atom/?utm_source=widget">
<img src="http://translations.artefactual.com/widgets/atom/-/25/multi-auto.svg" alt="Translation status" />
</a>
          
        </section>

        <br>

Note that the h4 gets an id tag!!

2. Changes in the .po file for the website
   Source string location:
   templates/​pages/​home.html:96
   and
   Translation file:
   locale/​hr/​LC_MESSAGES/​django.po, string 95
   as follows:

All user interface elements and database content can be translated into multiple languages, using our built-in translation interface. Our translations are all generously provided by volunteer translators from the AtoM User Community. Want to see AtoM available in your language? Help us translate!

into

All user interface elements and database content can be translated into multiple languages, using our built-in translation interface. <a href="Downloads | AtoM Open Source Archival Description Software.html#translation-status">Our translations</a> are all generously provided by volunteer translators from the AtoM User Community. Want to see AtoM available in your language? <a rel="nofollow" class="external free" href="https://wiki.accesstomemory.org/Resources/Translation">Help us translate</a>!

3. Usage of a new "id" tag (using Weblate's string)
   For the newly added "id", it should be possible to use Weblate's alt-string "Translation status", specified in the widget. NOTE: It would have to be automatically converted into lower-case letter, and space characters should be replaced by hyphens (for instance in english: "Translation status" -> "translation-status")

Finally, I'm not a programmer … I have some knowledge of HTML/CSS … If something is unclear, or if can help in any way, please let me know.

Thanks,
Milo

Hello,
After upgrading to AtoM 2.4.x several functions (finding aids, exports, etc.) were failing with 500 internal server errors. After checking the logs I realized that the gearman worker simply needed to be restarted. Bingo.

I thought maybe you could mention this in the "upgrading" doc (https://www.accesstomemory.org/en/docs/2.4/admin-manual/installation/upgrading/)

We are using Vagrant to create a new entity with some new fields. The goal is add user metadata with default metadata like ESAD, DC,...

After createad the new entity We'll relate it to an existing metadata. The goal is insert it during the creation of a digital object.
These custom metadata can also be filled up during DIP Upload by Archivematica.

1. We need to know the name of the class called during the DIP Upload (which reads the DIP and inserts the metadata).

2. Following the Symfony 1.4 documentation: (http://symfony.com/legacy/doc/jobeet/1_4/en/03?orm=Propel )When We tried to create a new Class, we had some issues, as shown above:

   2.1 Change the schema.yml by adding a new table with its fields and keys. DONE
   2.2 - Run: php symfony propel:build --sql to generate .sql according to schema.yml DONE
   2.3 - Run: php symfony tools:purge --demo to recreate the database by inserting the new entity DONE
   2.4 - Run: php symfony propel:build --model to generate the created entity model DONE
   2.5 - Run: php symfony propel:build --all --not-commit to create Forms and Validators >>>> FAIL
   An error occurs:
   build-propel.xml: 474: 20: Duplicate table found: propel.
   2.6 Run: php symfony propel:generate-module qubit thousand QubitMil (project ModuleName ModelName) >>>> FAIL
   But it throws the following error:

    PHP Warning:  call_user_func() expects parameter 1 to be a valid callback, class 'QubitMilPeer' not found in /usr/share/nginx/atom/vendor/symfony/lib/plugins/sfPropelPlugin/lib/generator/sfPropelGenerator.class.php on line 134
   
    Warning: call_user_func() expects parameter 1 to be a valid callback, class 'QubitMilPeer' not found in /usr/share/nginx/atom/vendor/symfony/lib/plugins/sfPropelPlugin/lib/generator/sfPropelGenerator.class.php on line 134
    PHP Fatal error:  Call to a member function getRelations() on a non-object in /usr/share/nginx/atom/vendor/symfony/lib/plugins/sfPropelPlugin/lib/generator/sfPropelGenerator.class.php on line 137
   
    Fatal error: Call to a member function getRelations() on a non-object in /usr/share/nginx/atom/vendor/symfony/lib/plugins/sfPropelPlugin/lib/generator/sfPropelGenerator.class.php on line 137
   

I have also posted on Google groups Ica-atom-user
https://groups.google.com/forum/#!topic/ica-atom-users/DZJJj1yuB2E

Hi,

I'm trying to upgrade from my current instance 2.2.0 to 2.4.0 and install fails after continue on step "Configure Search".

Error from Apache log:

PHP Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 190867 bytes) in [path]/v2/vendor/symfony/lib/i18n/sfMessageSource.class.php on line 199, referer: http://domain.tld/v2/index.php/sfInstallPlugin/configureSearch

This error complains about memory issues but I don't think this is the problem. This server as 4GB RAM.

Questions:

1. How much memory can be necessary on install steps?
2. What happens in this step "Configure Search"?
3. Also, What happens in the next step "Load Data"?

Another problem is: after this error, I cannot continue with the instalation by reloading the install page. It seems that it thinks that the install process was complete and throws other errors...

Thanks

Atom Version: 2.4.0 (upgraded from 2.2.0)

Steps to reproduce in our instance:

1. Open http://atom.cm-tavira.pt/v2
2. Add record to clipboard
3. Click Go To Clipboard
4. Click Export
5. Click Export

Logs from Apache (only relevant information):
[Tue May 22 05:54:23.403182 2018] mod_fcgid: stderr: Unknown record property "informationObject" on "QubitEvent"
[Tue May 22 07:18:12.362738 2018] mod_fcgid: stderr: The template "indexSuccess.xml.php" does not exist or is unreadable in "".
[Tue May 22 11:32:33.673429 2018] mod_fcgid: stderr: Could not connect to 127.0.0.1:4730, referer: http://domain.tld/v2/index.php/object/export?objectType=informationObject

Thanks!

Hello,

The help text for the start and end date fields of an archival description says that acceptable date formats are YYYYMMDD, YYYY-MM-DD, YYYY-MM and YYYY, but the last two don't seem to be handled properly. If I enter a date without a valid month and day, it gets recorded as "0".

I'm using Atom 2.4 (I just tested 2.5 and the problem is still here)

Thanks