Shallow copy of AstroData instance fails with recursive loop about dragons HOT 9 CLOSED

This should be easy enough to implement -- I just copy the __deepcopy__() but create references instead of copies of the objects. I have one question though: _tables is a dict whose values are Tables, and _nddata is a list whose elements are NDData objects, so there's an extra level of complex objects. Therefore I can do a very shallow copy, or a mid-level copy that creates references to the big objects (Tables and NDData) but creates copies of the things that hold them.

So when I copy _nddata I can make the new object's _nddata (a) a reference to that list; or (b) a new list whose elements are references to the elements in the parent's list. Similarly for _tables I can (a) simply create a reference to the parent's _tables; or (b) create a new dict object whose values are references to the Table objects in the parent.

What do people think?

from dragons.

I think the mid-level option (b) is best. If you make a shallow copy and add/delete an extension or a table you don't want that to propagate to the copied object. The important aspect of the shallow copy is that you don't duplicate the data, but otherwise they should be independent. E.g. for Astropy Table I think the shallow copy will always copy the metadata, maybe a good idea to do the same here with the primary header.

from dragons.

We had some discussions about what __copy__ and __deepcopy__ should do a long time ago (e.g. https://github.com/GeminiDRSoftware/DRAGONS/compare/test/jturner), before the refactor, but I guess we never converged and that's something I did not think about during the refactor. So yes, implementing __copy__ and __deepcopy__ is the way to go.

from dragons.

I thought I had made roughly the same comment, but apparently I didn't. I think those lists should just be considered internals of the host instance that you're making a copy of.

Github has started doing this incredibly annoying thing where you press ctrl-e to get to the end of the line and it puts backticks around a word instead 😠.

from dragons.

Will the behaviour be special when copying a "single" AstroData instance? Say a primitive wants to make a copy of its input ad with the WCS or some header keywords updated on each extension (which is actually my current use case). You'd probably want to make a shallow copy of each ext, to hold the modified meta-data, which I think would also require copying each NDAstroData instance, unlike what happens when making a shallow copy of an ad with len > 1? Otherwise it seems like it could get rather fiddly to construct all the shallow copies needed to reconstruct the ad without doing a deepcopy.

from dragons.

Eg. elsewhere I was constructing extensions like this (different use case), but this doesn't even include all the ancillary bits like tables on the extension, does it? So being able to make a complete shallow copy of all the containers for a single extension would seem useful.

ad_out.append(NDAstroData(data=sky_vals, wcs=ndd.wcs,
                          meta=ndd.meta, unit=ndd.unit,
                          copy=True, window=ndd.window))

from dragons.

Right, so you want to go even further? If do ad2 = copy(ad) then you want ad2._nddata to be a completely new list, and you want its individual NDData elements to be completely new NDData objects, but you want the attributes of those elements to be references. Because if I just make references to the NDData objects then modifying the wcs will modify it in the original.

from dragons.

I would think the "natural" expectation is that if you copy an entire ad with multiple extensions, it would make a new _nddata list containing the same NDData instances as the original, but if you do [copy(ext) for ext in ad] it will produce new NDData instances too. If that seems inconsistent, I think it's just inherent in the dual nature of how AstroData is implemented. No?

from dragons.

@jehturner I've implemented something which I think does what you want in the branch bugfix/ad_shallow_copy. Please take a look and let me know if it fulfills your use case. No great urgency but I'm keen to close some of these issues.

from dragons.