ITADN

Misunderstanding about how Python handles membership tests in `set` vs `list`

#109Closedth3w1zard1 创建于 2024-09-02
T
th3w1zard1commented
A long time ago, in a galaxy far far away, I went through the codebase and decided to improve performance by migrating all lists to sets. This was done with the understanding that I was implementing both __hash__ and __eq__ in the CaseInsensitiveWrappedStr. The implication being that this test would pass. ```python class CaseInsensitiveKey(str): __slots__ = () def __eq__(self, other): if isinstance(other, str): return self.lower() == other.lower() return NotImplemented def __hash__(self): return hash(self.lower()) class TestCaseInsensitiveKeyBehavior(unittest.TestCase): def setUp(self): self.nodepath_key = CaseInsensitiveKey("File.txt") self.caseinsens_key = "File.txt" self.casesens_key = "file.txt" def test_list_operations(self): some_list = [self.nodepath_key] assert self.nodepath_key in some_list assert self.caseinsens_key in some_list # NOTE: This is the most important difference between dict/set/list!! assert self.casesens_key in some_list def test_set_operations(self): some_hashset = {self.nodepath_key} assert self.nodepath_key in some_hashset assert self.caseinsens_key in some_hashset # THIS WILL FAIL assert self.casesens_key in some_hashset def test_setlike_dict_operations(self): some_hashset = {self.nodepath_key, "value"} assert self.nodepath_key in some_hashset assert self.casesens_key in some_hashset assert self.caseinsens_key in some_hashset # THIS WILL FAIL ``` Note the comment 'THIS WILL FAIL' which clearly illustrates the misunderstanding. In lists, python will use the __eq__ of literally everything in the list, whereas only the left side of `in`'s `__hash__` method is considered for membership tests. This affects key access with mydict[mystr], and get function with mydict.get(mystr) There is so much code using these lists, that this will probably affect the functionality of the entire library on Linux/Mac systems. Given the exhaustive tests for CaseAwarePath, the scenarios where this bug will happen is nuanced and will probably not be discovered immediately. The most straightforward solution is to ctrl+f around the codebase for set[str] | set[CaseInsensitiveWrappedStr] and replace them with lists. Probably could even use a find and replace algorithm for most of them. This would take about an hour and would slow down the whole codebase, so i'd rather not go with this strategy at this time. Suggestions would be welcome.
关闭于 2024-11-09 0 条评论