Inline::Python - Testing call method with __builtin__ getattr - fails if called repeatedly on same object / attribute combination

Matplotlib_reduced.pm6

use Inline::Python;

my $py = Inline::Python.new();
$py.run('import matplotlib.pyplot');

class Matplotlib::Plot {
    method cm {
        class {
            # this getattr method, called like so : 'say $plt.cm.getattr($cmap, 'N');'
            method getattr($obj, $name) {
                $py.call('__builtin__', 'getattr', $obj, $name);
            }
            method FALLBACK($name, $idx) {
                $py.run("matplotlib.pyplot.cm.{$name}($idx)", :eval);
            }
        }.new();
    }
    method FALLBACK($name, |c) {
        $py.call('matplotlib.pyplot', $name, |c)
    }
}

class Matplotlib {
    method FALLBACK($name, |c) {
        $py.call('matplotlib', $name, |c)
    }
}

perl6_matplotlib_getattr_test.p6

use v6;
use lib '.';
use Matplotlib_reduced;

my $plt = Matplotlib::Plot.new;

my $cmap = $plt.get_cmap('seismic', 5);

# -- this works

#say 'Number of Colors : ' ~ $plt.cm.getattr($cmap, 'N'); 
#say 'Number of Colors : ' ~ $plt.cm.getattr($cmap, 'N'); 

# -- this fails on the 3rd call

#say 'Number of Colors : ' ~ $plt.cm.getattr($cmap, 'N'); 
#say 'Number of Colors : ' ~ $plt.cm.getattr($cmap, 'N'); 
#say 'Number of Colors : ' ~ $plt.cm.getattr($cmap, 'N'); 

# -- this works

#say 'Colormap Name    : ' ~ $plt.cm.getattr($cmap, 'name'); 
#say 'Colormap Name    : ' ~ $plt.cm.getattr($cmap, 'name'); 

# -- this fails on the 3rd call

#say 'Colormap Name    : ' ~ $plt.cm.getattr($cmap, 'name'); 
#say 'Colormap Name    : ' ~ $plt.cm.getattr($cmap, 'name'); 
#say 'Colormap Name    : ' ~ $plt.cm.getattr($cmap, 'name'); 

# -- this works

#say 'Number of Colors : ' ~ $plt.cm.getattr($cmap, 'N'); 
#say 'Colormap Name    : ' ~ $plt.cm.getattr($cmap, 'name'); 

# -- this fails on the 3rd call

say 'Colormap Name    : ' ~ $plt.cm.getattr($cmap, 'name'); 
say 'Number of Colors : ' ~ $plt.cm.getattr($cmap, 'N'); 
say 'Number of Colors : ' ~ $plt.cm.getattr($cmap, 'N'); 

# -- etc....

OK, .. a bit more testing
... found another matplotlib colormap 'Set1' with another attribute 'colors' attached (so long as the full colormap retrieved with get_cmap)

so I changed line 7 to this :
my $cmap = $plt.get_cmap('Set1');

and then :
# -- this works
#say 'Colormap Name : ' ~ $plt.cm.getattr($cmap, 'name');
#say 'Number of Colors : ' ~ $plt.cm.getattr($cmap, 'N');
#say 'colors : ' ~ $plt.cm.getattr($cmap, 'colors');
#say 'Colormap Name : ' ~ $plt.cm.getattr($cmap, 'name');

# -- this fails on the 5th call
say 'Colormap Name : ' ~ $plt.cm.getattr($cmap, 'name');
say 'Number of Colors : ' ~ $plt.cm.getattr($cmap, 'N');
say 'colors : ' ~ $plt.cm.getattr($cmap, 'colors');
say 'Colormap Name : ' ~ $plt.cm.getattr($cmap, 'name');
say 'Number of Colors : ' ~ $plt.cm.getattr($cmap, 'N');
say 'colors : ' ~ $plt.cm.getattr($cmap, 'colors');

Hi, you left IRC before I had a chance to respond. I wrote the Using matplotlib in Perl 6 series, so I feel compelled to help you out here 😄

It seems you want to be able to call getattr to avoid having to wrap everything manually like I had to.

I thought a better way would be is to export a top-level sub that does the call for you, then you can use if for any PythonObject you like, rather than just ColorMap objects.

# MODULE
sub getattr($obj, $name) is export {
    $py.call('__builtin__', 'getattr', $obj, $name)
}
# SCRIPT
say getattr($cmap, 'name');

But... I still ran into the same issue. I don't know why it fails after successfully calling a few times.

For now, a potential work-around is just to call the dunder method on your object directly.

This works fine for me, even after multiple calls to __getattribute__.

my $plt = Matplotlib::Plot.new;

my $cmap = $plt.get_cmap('seismic', 5);

say $cmap.__getattribute__('name');
say $cmap.__getattribute__('N');

This also requires no changes in the module.

I wonder if Inline::Python could be modified to first try attribute lookup (hasattr(obj, 'name')) when calling $pyobject.name (ie. a method call without args) and then if no attribute exists, try calling the method.

That would incur a performance hit, and possibly @niner has a better idea on how to resolve the ambiguity.

Wow. 0racle, thank you for taking the time and effort to provide your thoughts and guidance on this. Your blog series on 'Matplotlib and Perl6', together with Moritz's Perl6 Fundamentals book (Chap.12), have been my primary references, in trying to explore Matplotlib charting from the 'data processing powerhouse' that is Perl6, via Inline::Python. I've read and worked through the blog series several times, it is inspirational, and you can probably see your code in the files in this gist :). I'm just a hobbyist programmer nowadays, nevertheless it's a hobby that is fun and challenging at the same time, keeps the brain ticking over, especially with having no previous Python experience, and learning Matplotlib, in addition to trying to become reasonably proficient in Perl6. I've also just discovered 'bokeh' plotting, the charts are certainly very swish, and I'm thinking I'll try to see if I can generate them through Inline::Python too :).

I find that the best way for me to learn, is to have a practical problem to solve, and so figured I'd convert one of the Matplotlib python examples into Perl6 (#125 Small multiples for line chart[left graph]), and that's where the above 'program failing issue' was encountered.

Thanks for your guidance on generalising the 'getattr' call for use with any Python object by placing it in it's own top-level sub. That does seem to be very beneficial, I will certainly take that on board :).

Also, I'd missed the __getattribute__ dunder method, when looking through the builtin methods in Python to try to 'get a handle' on just what these colormap objects are, so thank you for pointing that method out. I've also confirmed your conclusion that repeated calls to $pyobj.__getattribute__('name') doesn't result in a program error.

I had thought of building in exception handling into the getattr method to catch cases where a specific attribute has never been set on a particular python object, but it takes me a while to make progress :)

No problem at all. I had never used matplotlib before I wrote that series... which is why I was eager at the time to document my progress, so that others such as yourselves could see the problems I ran into, and a way I solved them. As I said in the blog, often I'm not sure if the way I solved the problem was the best way, but a solution is better than none at all.

The colormap object is a matplotlib.colors.LinearSegmentedColormap, so... running with my plan of "wrap all the things"... you would define a new class (say Matplotlib::Colors::Colormap or something) that was instantiated when you called $ptl.cmap(|c). Subsequently when you call $cmap.name, you can catch that method call in your newly defined class and redirect it to __getattribute__... then as usual, punt everything else to FALLBACK.

Though your Exception idea is not bad either... but it could be extended with my idea of attribute checking. Under a FALLBACK method, if no arguments are provided, you could check Python's hasattr($name) and if True, call __getattribute__($name) on the object. Potentially within that same method you can catch exceptions and handle them accordingly.

Regarding you formatting woes... Markdown uses double underscores to denote bold. When referring to short code-specific words, wrap them in `backticks` (which will produce the result backticks, and also disable other Markdown formatting within the backticks.)

So... here's an example of how you could wrap it...

class Matplotlib::Plot {
    # ...
    method get_cmap(|c) {
        my $map = $py.call('matplotlib.pyplot', 'get_cmap', |c);
        class {
            method gist { 'ColorMap' ~ c.List.perl }
            method N { return $map.__getattribute__('N') }
            method name { return $map.__getattribute__('name') }
            method FALLBACK($name, |c) {
                $py.call('matplotlib.colors.LinearSegmentedColormap', $name, |c)
            }
        }.new()
    }

I'm using an anonymous class, but you could just as easily define a proper class outside of Matplotlib::Plot

I think - possibly - that anonymous classes have slower performance, or maybe cannot be optimised as well... at least, that's the impression I got from a bunch of commits to Rakudo that removed anonymous classes in a bunch of iterators

In any case... this works now

my $plt = Matplotlib::Plot.new;

my $cmap = $plt.get_cmap('seismic', 5);

say $cmap;      # ColorMap("seismic", 5)
say $cmap.name; # seismic
say $cmap.N;    # 5

... though it's annoying to wrap every attribute like this... I had hoped this would work...

    method FALLBACK($name, |c) {
        # If no args and attribute exists...
        if !c and $py.call('__builtin__', 'hasattr', $map, $name) {
            return $map.__getattribute__($name)
        }
        # otherwise call method
        else {
            $py.call('matplotlib.colors.LinearSegmentedColormap', $name, |c) 
        }   
    }  

and it does work... the first time, then it throws an error saying method-wrapper' object has no attribute on the second call to any attribute. I dunno, this seems like a bug in Inline::Python, but I don't want to bother the maintainer with issues I can work around with a little effort anyways. He's already put in a massive effort to make Inline::Python work as well as it does.

Here's one more option... Attributes are not methods, you don't call them... you just ask what they are... sometimes you can assign to them (if Python does not define them as a @property). So in this way, they are kind of like keys of a hash... so you can treat them as such.

    class {
        my $obj  = $py.call( ... );
        method AT-KEY($name) {
            $obj.__getattribute__($name);
        }
        method ASSIGN-KEY($name, $new-value) {
            $obj.__setattr__($name, $new-value)
        }
        method FALLBACK($name, |c) { ... }
    }

Now you could access (and assign to) attributes as if they were a Hash

$pyobject<attributename> = "new";
# calls $pyobject.ASSIGN-KEY('attributename', 'new')...
# which calls $pyobject.__setattr('attributename', 'new')

say $pyobject<attributename>; # new

Upon further experimentation, it seems like problems happen as soon as you start doing calls to methods on __builtin__. If I avoid those methods, I don't seem to hit the same problem.

For example, in my FALLBACK in the earlier comment, if I don't check __builtin__.hasattr then it works, but of course, Python will throw an exception if the attribute doesn't exists. You can put a try in front of the call to __getattribute__ and it will return an undefined value (instead of throwing) if the attribute doesn't exists.

I'm not yet sure how to put all these ideas together in a cohesive whole, but play around with them and see what you come up with.

btw python's getattr supports a third argument which is a default value in case the attribute doesn't exist.

If you write a wrapper, maybe the simplest thing to do is to create a sentinel value Any.new as the default, and if that is returned, interpret it as not found.

More thanks 0racle, and Moritz, I definitely got way more out of this than I put in :-) I hope this thread has somehow kickstarted a future new article on your Perl6/Matplotlib blog 0racle. I agree about not bugging the Inline::Python maintainer over this when there is a work around.

It all seems so straightforward when it is explained, like you have both done in your comments, I'm not sure after a month of researching I'd have come to the same understanding :-) It's given me a whole ton of concepts and reasoning to investigate. I'll carry on with my efforts on this, and try to add some results to this gist when I have them. Thanks again.