Appending a list to a list causes recursion

[d00m514y3r]

reproduce:

from box import Box
a = Box({"list_of_dicts": [[{"example1": 1}]]})
a.list_of_dicts.append([{"example2": 2}])
print(a)
# {'list_of_dicts': [[{'example1': 1}], [...]]}
# expected output
# {'list_of_dicts': [[{'example1': 1}], [{'example2': 2}]]}

Using python 3.10.9 and python-box 6.0.2

[cdgriffith]

This also happens on Python 3.11 and box 7.

What's weird is that if you debug that line a.list_of_dicts.append([{"example2": 2}]) it works as expected. So no easy way to even see where that is happening...

[d00m514y3r]

Also if you do it like this it works for some reason:

a.list_of_dicts.append(BoxList([{"example2": 2}]))

[cdgriffith]

The issue comes down to reuse of id in python.

print(id([{"example2": 2}]))
print(id([{"example2": 1}]))
print(id([{"2": 1}]))
print(id([{"5": "2", "2": "4"}]))

1977990054720
1977990054720
1977990054720
1977990054720

Because as soon as the life of one object is over, that id can be reused. The id was being used as a recursion safety check. If you say, assign one to a variable so it is still alive, it will work.

from box import Box
a = Box({"list_of_dicts": [[{"example1": 1}]]})
b = [{"example2": 2}]
a.list_of_dicts.append(b)
assert a['list_of_dicts'][1] == [{"example2": 2}],  a['list_of_dicts'][1]

So what was happening is the object [[{"example1": 1}]] was given an id, say 1, and then moved into the new list as a new object, Say object with id 2.

That means the first object could be cleared from memory now that it's in a new space.

Object [{"example2": 2}] comes along and goes "oh hey, look at that, memory space 1 is free. I'll use that!"

Then BoxList goes (incorrectly), "Ah ha! This is 1, the same object we used to create the BoxList from in the beginning! It's self referencing! Just link me to myself."

Know the cause now. Just need to figure out best approach to go forward.

[cdgriffith]

Decided best course of action is to just fully remove support for self references inside of lists, as it is a rare and dangerous use-case anyways (says the guy who doesn't want to figure out how to support it.) Updated in 7.1.0