Namespace issue when importing Perl 5 MorphoDiTa bindings about morphodita HOT 17 CLOSED

Did you also try to use fully qualified names? I.e., if you use something like

package Qux;
use Ufal::MorphoDiTa;

Ufal::MorphoDiTa::Morpho::load('foo');

does it work?

Note that the whole issue may be a problem of MorphoDiTa only -- the SWIG itself does not export the classes automatically. I added the following to export the classes (which are in separate packages, see MorphoDiTa.pm):

@EXPORT_OK = qw(*Forms:: *TaggedForm:: *TaggedForms:: *TaggedLemma:: *TaggedLemmas::
                *TaggedLemmaForms:: *TaggedLemmasForms:: *TokenRange:: *TokenRanges::
                *Tokenizer:: *Morpho:: *Tagger:: *TagsetConverter:: *Version::);
%EXPORT_TAGS = (all => [@EXPORT_OK]);

I came with this on a trial basis, but maybe I am doing it wrong and there is a better way of doing it.

from morphodita.

Did you also try to use fully qualified names?

I did, but forgot to mention it. When using fully qualified names, the subroutine is found, which is again somewhat unexpected (other Perl modules don't behave like this).

Note that the whole issue may be a problem of MorphoDiTa only -- the SWIG itself does not export the classes automatically. I added the following to export the classes

Thanks for the pointers, I'll look into it some more!

from morphodita.

I did, but forgot to mention it. When using fully qualified names, the subroutine is found, which is again somewhat unexpected (other Perl modules don't behave like this).

I think that using fully qualified names is a standard way to use packages -- for example http://perldoc.perl.org/functions/package.html states that you can access identifiers in other packages by using package_name::. The Exporter module, if used, can also add "shortcuts" to the "current namespace", so that some names can be used without qualification.

If you find a better way of exporting the subpackages, I will be happy to fix it.

from morphodita.

I think that using fully qualified names is a standard way to use packages

You're right of course, my bad, I did a quick test to see if exported symbols remain accessible via their fully qualified names (which makes perfect sense, but you never know with Perl...) and made a stupid mistake :)

from morphodita.

I don't see the syntax mentioned in Exporter

@EXPORT_OK = qw( *Forms:: );

Are you sure it works?

from morphodita.

I don't see the syntax mentioned in Exporter

@EXPORT_OK = qw( *Forms:: );

Are you sure it works?

Are you referring to the double colon? I think this shouldn't matter, AFAIK @EXPORT_OK is just an array of strings which constitutes a whitelist of symbol names that can be passed to the import subroutine. So if the symbol Forms:: exists in the source namespace, everything should be fine.

I've found that if I add typeglob aliases of the form *Morpho:: = *Ufal::MorphoDiTa::Morpho::; etc. to the Ufal::MorphoDiTa package, then the Morpho::load subroutine is found in both cases, i.e. irrespective of whether I do use Ufal::MorphoDiTa; package Qux; or package Qux; use Ufal::MorphoDiTa;. Still not the desired behavior though.

from morphodita.

Note that without the typeglob alias mentioned above,

$ perl -e 'use Ufal::MorphoDiTa qw(:all); package Qux; print *Morpho::'
*Ufal::MorphoDiTa::Morpho::
# and the sub is found

but

$ perl -e 'package Qux; use Ufal::MorphoDiTa qw(:all); print *Morpho::'
*main::Morpho::
# and the sub is not found, obviously, since the namespace is wrong

With the typeglob alias, both of these print *Ufal::MorphoDiTa::Morpho::. This must all be caused by some arcane typeglob voodoo...

from morphodita.

What I mean, normally you list the subroutines to export, not their namespaces. I'm not sure listing the namespaces has the same effect as listing all their subroutines.

from morphodita.

After some hacking, I found out the following:

• exporting a namespace seems to be working fine. Exporter explicitly allow exporting typeglobs, so exporting a namespace using *package:: is probably fine.
• the problem is the namespace lookup. In Perl, qualified names do not seem to be using the current package as a root, but :: instead. Consider the following:

package test::inner;
sub f() { print "f\n"; }

package test;
inner::f();

The last line does not call test::inner::f, but instead fails, as it tries to use main::inner::f()

Therefore, Ufal::MorphoDiTa correctly exports all namespaces to Qux. Unfortunately, using Morpho::load in the Qux package refers to ::Morpho::Load, which is not defined.

Possible solutions I can think of:

1. import Ufal::MorphoDiTa without exporting the nested namespaces (without :all) and fully qualify everything. Probably the sanest solution.
2. import Ufal::MorphoDiTa in your package and then qualify the uses with the current package name (i.e., Qux::Morpho::load).
3. import Ufal::MorphoDiTa before defining package name, as in that case the exported namespaces are imported to the main::. Unfortunately, this imports main::Morpho everywhere (in every module using the module which imports Ufal.MorphoDiTa).

from morphodita.

In Perl, qualified names do not seem to be using the current package as a root

That's why they're called fully qualified names :-)

from morphodita.

That's why they're called fully qualified names :-)

I guess you are right :-)

I expected the lookup to work differently (as in static languages), using non-fully qualified names only: when in package A, I expected that all symbols (maybe except for symbols starting with ::) are looked up in the A package (as prefixed with A::) and if not found, looked up in the main::. But now when thinking about it, it is not possible in Perl for a symbol lookup to fail reliably -- you have to resolve every symbol to a unique fully-qualified name at first and then perform the lookup (which might for example create the symbol in the desired namespace). I am such a statically-typechecking languages guy :-)

Of course the Perl documentation explicitly states this:

Packages may themselves contain package separators, as in $OUTER::INNER::var .
This implies nothing about the order of name lookups, however. There are no
relative packages: all symbols are either local to the current package, or
must be fully qualified from the outer package name down. For instance, there
is nowhere within package OUTER that $INNER::var refers to $OUTER::INNER::var .
INNER refers to a totally separate global package.

@dlukes, can we close this? There is no way of achieving of what you expected, because of how Perl works.

from morphodita.

Of course the Perl documentation explicitly states this:

Yeah, I also finally stumbled upon that bit of documentation yesterday late at night :)

I'm not quite sure whether this is necessarily a static vs. dynamic language issue, Python does "the right thing™" in this case. But I'd rather not get up on my soapbox re: the Perl vs. Python holy war ;)

can we close this? There is no way of achieving of what you expected, because of how Perl works.

Yup. Though I guess it would be nice either not to export anything from the module (and force systematic use of fully qualified names -- no surprises there), or add a warning to the API documentation that there's this weird (though entirely consistent from Perl's point of view) behavior when importing into a namespace other than main::, that you have to prefix your calls to the imported functions with that namespace. But that's up to you :)

@foxik Would you like to edit this discussion into a short summary for the StackOverflow question and grab the points for the answer? I'll be happy to accept it. Or I can do it myself if you're not interested.

Thank you both for your help, this was really bugging me and now I can finally stop thinking about it!

from morphodita.

I have removed the :all export keyword. It makes no sense now (I added it because I expected it to work as you expected, but I tried it without package other than main). We will be releasing MorphoDiTa 2.0 soon, so I will remove it only in the new version and leave it in the 1.* not to break people code too much.

I'm not quite sure whether this is necessarily a static vs. dynamic language issue, Python does "the right thing™" in this case. But I'd rather not get up on my soapbox re: the Perl vs. Python holy war ;)

Well, I have been surprised by Python on this level too, because of the non-symmetric lookup of read-only variables and read-and-write variables. Consider for example:

imap = #create_connection

def open_mailbox():
    imap.something()

which works, versus

imap = #create_connection

def open_mailbox():
    imap.something()
    # Add reconnect in case of failure
    if something_bad:
        imap = IMAP4_stream(...)

which does not "compile".

But admittedly I should have not used a global variable, but instead pass it around, and I would not be surprised :-)

from morphodita.

which does not "compile".

That's just Python being opinionated and trying to persuade you not to mess with global state by creating a local binding on write by default ;) But this behavior is available if requested explicitly with the global keyword.

from morphodita.

I have the global keyword in the script now, but it was surprising to me that in the first example, Python is happy to access the global variable, and adding an assignment changes its opinion :-)

from morphodita.

I have the global keyword in the script now

I had a hunch I wasn't telling you anything new ;)

I guess the reasoning behind this is that global variables are often supposed to be constants, and since there's no way to declare those in Python, this is a safety measure to ensure that nobody accidentally stomps over them (because it could break unrelated code elsewhere, which is hard to debug) unless they know what they're doing. Personally, I like this design choice and it feels natural, but then again my programming background is very limited :)

from morphodita.

Although I kind of understand the motivation, it surprised me that the semantics of one line was changed by adding an innocent looking line at some other place.

Pragmatic approach is probably more useful here than symmetry and elegance. But I connect Perl more with this approach than Python ;-)

from morphodita.