This project forked from timbrody/perl-oai-lib
HTTP::OAI
License: Other
HTTP::OAI - API for the OAI-PMH
This is a stub module, you probably want to look at HTTP::OAI::Harvester or HTTP::OAI::Repository.
You can find links to this and other OAI tools (perl, C++, java) at: http://www.openarchives.org/pmh/tools/.
Ed Summers Net::OAI::Harvester module.
Copyright 2004-2010 Tim Brody [email protected], University of Southampton.
This module is free software and is released under the BSD License (see LICENSE).
I'm trying to harvest records in oai_dc format from an OAI 2.0 repository by passing a specific setSpec to ListRecords().
The setSpec I'm using is 7375626A656374733D4F5A:4F5A3236, which corresponds to a setName of Subject = Ο: Οικονομική κρίση.
The set in question contains 233 records but I can only retrieve the last 33 when I use the latest version (4.13) of HTTP::OAI:
$ ./listrecords.pl https://anaktisis.uowm.gr/cgi/oai2 7375626A656374733D4F5A:4F5A3236
Using HTTP::OAI 4.13
2020-10-14T08:20:12Z oai:anaktisis.uowm.gr:10568
2020-10-12T07:03:10Z oai:anaktisis.uowm.gr:10572
2020-10-12T08:19:16Z oai:anaktisis.uowm.gr:10576
2020-10-16T08:19:48Z oai:anaktisis.uowm.gr:10589
2020-10-20T11:34:50Z oai:anaktisis.uowm.gr:10604
[...]
2022-07-06T08:00:52Z oai:anaktisis.uowm.gr:11753
2022-10-05T09:09:39Z oai:anaktisis.uowm.gr:12005
2022-11-25T08:00:25Z oai:anaktisis.uowm.gr:12047
2022-11-25T09:02:22Z oai:anaktisis.uowm.gr:12064
2022-11-30T09:02:28Z oai:anaktisis.uowm.gr:12066
Got 33 recordsIf I go back to version 3.27 (commit e151284), everything works fine (i.e. I can retrieve all 233 records):
$ ./listrecords.pl https://anaktisis.uowm.gr/cgi/oai2 7375626A656374733D4F5A:4F5A3236
Using HTTP::OAI 3.27
2015-04-01T07:24:06Z oai:anaktisis.uowm.gr:299
2015-04-01T12:30:38Z oai:anaktisis.uowm.gr:362
2015-11-11T09:50:10Z oai:anaktisis.uowm.gr:907
2015-05-19T10:09:23Z oai:anaktisis.uowm.gr:1793
2015-04-20T07:21:21Z oai:anaktisis.uowm.gr:2259
[...]
2022-07-06T08:00:52Z oai:anaktisis.uowm.gr:11753
2022-10-05T09:09:39Z oai:anaktisis.uowm.gr:12005
2022-11-25T08:00:25Z oai:anaktisis.uowm.gr:12047
2022-11-25T09:02:22Z oai:anaktisis.uowm.gr:12064
2022-11-30T09:02:28Z oai:anaktisis.uowm.gr:12066
Got 233 recordsgit bisect indicates that a8ae8b2 is the first bad commit (the latest good commit is 25f8de3).
Here's my code:
#!/usr/bin/perl
use strict;
use warnings;
use HTTP::OAI;
print 'Using HTTP::OAI ' . $HTTP::OAI::Harvester::VERSION . "\n\n";
my $harvester = HTTP::OAI::Harvester->new(
baseURL => $ARGV[0]
);
my $records = $harvester->ListRecords(
metadataPrefix => 'oai_dc',
set => $ARGV[1],
from => ''
);
die $records->message if $records->is_error;
my $count = 0;
while ( my $record = $records->next ) {
$count++;
print $record->datestamp . ' ' . $record->identifier . "\n";
}
print "\nGot " . $count . " records\n";On our project, we have this code:
$response->set_handler( XML::SAX::Writer->new( Output => *STDOUT ) );
$response->xslt( "/path/to/OAI.xslt" );
$response->generate;
It works correctly with 3.26. We want to support the 4.0+ API. But it seems the xslt method is no longer exported as it was previously, only in:
lib/HTTP/OAI/Response.pm:63:sub xslt { shift->headers->header('xslt',@_) })
I would also like to talk about what's changed between v3 and v4.
On some of my smokers the test suite fails like this:
# Failed test 'Request arguments'
# at t/listidentifiers.t line 22.
# Looks like you failed 1 test of 2.
t/listidentifiers.t ......
Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/2 subtests
Problem is probably that the seemingly invalid hostname domain.invalid may resolve to something if wildcard DNS records are active on a system. The long story can be seen here: tokuhirom/Furl#128
Usual fix is to add a dot (.) to the invalid hostname (not tested in your case).
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.