1.10 Reference Counts 引用计数
In languages like C or C++, th e programmer is responsible for dynamic allocation and deallocation of memory on the heap. In C, this is done using the functions malloc()and free(). In C++, the operators newand delete are used with essentially the same meaning and we’ll restrict the following discussion to the C case.
在像C和C++这样的语言中，程序员要负责堆栈上内存的分配和回收。在C中是使用来函数malloc()和free()来完成，而在C++中使用的是操作符new和delete，两种方式实质上含义相同，我们接下来的讨论都是针对C中的情形。
Every block of memory allocated with malloc()should eventually be returned to the pool of available memory by exactly one call to free(). It is important to call free()at the right time. If a block’s address is forgotten but free()is not called for it, the memory it occupies cannot be reused until the program terminates. This is called a memory leak. On the other hand, if a program calls free()for a block and then continues to use the block, it creates a conflict with re-use of the block through another malloc()call. This is called using freed memory. It has the same bad consequences as referencing uninitialized data — core dumps, wrong results, mysterious crashes.
每块由malloc()分配了的内存，最终都应通过调用free()归还给可用内存池。在正确的时间调用free()很重要。如果一块内存的地址被遗忘，但没有调用来free()释放内存，其占有的内存将不能被重复使用，直到程序终止。这称之为内存泄漏。另一方面，如果对一块内存调用free()来释放内存，然后又继续使用这块内存，将会引发与另一个使用malloc()来重用这块内存的情形相冲突。这称之为使用已释放内存。这和引用非初始化的数据有一样的糟糕后果——内存泄漏、错误的结果、诡异的冲突。
Common causes of memory leaks are unusual paths through the code. For instance, a function may allocate a block of memory, do some calculation, and then free the block again. Now a change in the requirements for the function may add a test to the calculation that detects an error condition and can return prematurely from the function. It’s easy to forget to free the allocated memory block when taking this premature exit, especially when it is added later to the code. Such leaks, once introduced, often go undetected for a long time: the error exit is taken only in a small fraction of all calls, and most modern machines have plenty of virtual memory, so the leak only becomes apparent in a long-running process that uses the leaking function frequently. Therefore, it’s important to prevent leaks from happening by having a coding convention or strategy that minimizes this kind of errors.
在程序代码中内存泄漏很少有共同的原因。例如，一个函数可能分配一块内存，做一些运算，然后又释放这块内存。现在可能要求为该函数的运算操作增加测试代码，用来检测错误条件，并可以从函数中提前返回。当执行这样的提前退出时，很容易忘记释放已分配的内存块，尤其是当测试代码是之后加入的。这样的内存泄漏，一旦被引入了，常常在很长的时间内都难于发现：错误的退出仅出现在总体的一小部分调用中，并且大多数现代机器都拥有充裕的虚拟内存，故内存泄漏只有在长时间运行频繁使用了内存泄漏的函数的进程中才会显著。因此，通过编程的习惯或策略最少化这类错误来防止内存泄漏很重要。
Since Python makes heavy use of malloc()and free(), it needs a strategy to avoid memory leaks as well as the use of freed memory. The chosen method is called reference counting. The principle is simple: every object contains a counter, which is incremented when a reference to the object is stored somewhere, and which is decremented when a reference to it is deleted. When the counter reaches zero, the last reference to the object has been deleted and the object is freed.
由于，Python大量使用了malloc()和free()，需要一个机制来避免内存泄漏和使用已释放内存。选中的方法被称为“引用计数”。准则很简单：每个对象都容纳了一个计数器，当对象的一个引用被存储于某处时计数器加1。当对象的一个引用被删除时计数器减1。当计数器减为0时，意味着对象的最后一个引用已经被删除并且对象已被清除释放。
An alternative strategy is called automatic garbage collection. (Sometimes, reference counting is also referred to as a garbage collection strategy, hence my use of “automatic” to distinguish the two.) The big advantage of automatic garbage collection is that the user doesn’t need to callfree()explicitly. (Another claimed advantage is an improvement in speed or memory usage — this is no hard fact however.) The disadvantage is that for C, there is no truly portable automatic garbage collector, while reference counting can be implemented portably (as long as the functionsmalloc()and free()are available — which the C Standard guarantees). Maybe some day a sufficiently portable automatic garbage collector will be available for C. Until then, we’ll have to live with reference counts.
另一个机制称之为“自动垃圾回收”（有时，引用计数也被称之为一种垃圾回收机制，因此，我用“自动”来区分二者。）自动垃圾回收机制最大的优点在于程序员不需要显式地调用函数。（另一个号称的优点是速度和内存使用的提高——然而这一点没有足够的事实支撑。）缺点在于，对于C，没有真正的可移植的自动垃圾回收器，而引用计数能可移植地实现（malloc()和free()函数可用性是C标准这一点确保了可移植性）。或许某天一个有充分可移植性的垃圾回收器在C中是可用的。在那之前，我们不得不与引用计数相伴。
While Python uses the traditional reference counting implementation, it also offers a cycle detector that works to detect reference cycles. This allows applications to not worry about creating direct or indirect circular references; these are the weakness of garbage collection implemented using only reference counting. Reference cycles consist of objects which contain (possibly indirect) references to themselves, so that each object in the cycle has a reference count which is non-zero. Typical reference counting implementations are not able to reclaim the memory belonging to any objects in a reference cycle, or referenced from the objects in the cycle, even though there are no further references to the cycle itself.
尽管Python使用传统的引用计数实现，但也提供了一个循环探测器，用于探测引用循环。这这就允许应用程序不用担心产生直接或间接的循环引用；直接或间接的循环引用是仅使用引用计数实现垃圾收集的缺陷，引用循环由包含（可能是间接的）对自身引用的对象组成，以致循环中的每个对象都有非零的引用计数。传统的引用计数的实现不能回收引用循环中任何元素的内存，也不能引用循环中的对象。即使没有更深的引用指向循环自身。
The cycle detector is able to detect garbage cycles and can reclaim them so long as there are no finalizers implemented in Python (__del__()methods). When there are such finalizers, the detector exposes the cycles through the gcmodule(specifically, the garbagevariable in that module). The gcmodule also exposes a way to run the detector (the collect()function), as well as configuration interfaces and the ability to disable the detector at runtime. The cycle detector is considered an optional component; though it is included by default, it can be disabled at build time using the –without-cycle-gcoption to the configurescript on UNIX platforms (including Mac OS X) or by removing the definition of WITH_CYCLE_GC in the ‘pyconfig.h’ header on other platforms. If the cycle detector is disabled in this way, the gcmodule will not be available.
只要没有在Python中实现收尾器（__del__()方法），循环探测器就能够检测并回收垃圾循环。当有这样的收尾器时，探测器通过gc模块（变量garbage就在这个模块中）来使循环暴露。gc模块还展示了一个运行探测器的方式（collect()函数）、配置接口以及在运行时让探测器失效的能力。循环探测器被看作一个可选的组件；尽管它被缺省地包含进来了，在UNIX平台下（包括Mac OS X）能在构建时使用configure脚本的–without-cycle-gc选项或在其他平台下可以删除头文件pyconfig.h’中的WITH_CYCLE_GC定义来使之失效。如果循环探测器以这种方式失效，gc模块将不可用。
1.10.1 Reference Counting in Python Python中的引用计数
There are two macros, Py_INCREF(x) and Py_DECREF(x), which handle the incrementing and decrementing of the reference count. Py_DECREF()also frees the object when the count reaches zero. For flexibility, it doesn’t call free()directly — rather, it makes a call through a function pointer in the object’s type object. For this purpose (and others), every object also contains a pointer to its type object.
有两个宏Py_INCREF(x)和Py_DECREF(x)用于处理引用计数的增加和减少。Py_DECREF(x)也用于将引用计数减为0时释放对象，为了灵活性，不应直接地调用free()——而应该调用对象的类型对象中的函数指针。基于这个目的（还有其他的），每个对象都维护了一个指向自已的类型对象的指针。
The big question now remains: when to use Py_INCREF(x)and Py_DECREF(x)? Let’s first introduce some terms. Nobody “owns” an object; however, you can own a reference to an object. An object’s reference count is now defined as the number of owned references to it. The owner of a reference is responsible for calling Py_-DECREF()when the reference is no longer needed. Ownership of a reference can be transferred. There are three ways to dispose of an owned reference: pass it on, store it, or call Py_DECREF(). Forgetting to dispose of an owned reference creates a memory leak.
现在剩下的最大的问题是：何时使用Py_INCREF(x)和Py_DECREF(x)？我们首先引入一些术语。没有人能“拥有”一个对象；然而，你能拥有一个指向对象的引用。对象的引用计数在这定义为指向对象的引用的数量。当不再需要引用时，引用的所有者应负责调用Py_DECREF()。引用的所有权可以转让。有三种处理已有引用的方式：转让、存储、或调用Py_DECREF()。忘记处理所拥有的引用会引发内存泄漏。
It is also possible to borrow2a reference to an object. The borrower of a reference should not callPy_DECREF(). The borrower must not hold on to the object longer than the owner from which it was borrowed. Using a borrowed reference after the owner has disposed of it risks using freed memory and should be avoided completely.3
从一个对象那借用一个引用也是可能的。引用的借用者不能调用Py_DECREF()。借用者维护对象的时间不能长于它的借出者。在所有者已经清理对象之后使用借引用会有使用已释放内存的风险，应该完全避免。
The advantage of borrowing over owning a reference is that you don’t need to take care of disposing of the reference on all possible paths through the code — in other words, with a borrowed reference you don’t run the risk of leaking when a premature exit is taken. The disadvantage of borrowing over leaking is that there are some subtle situations where in seemingly correct code a borrowed reference can be used after the owner from which it was borrowed has in fact disposed of it.
相较于“拥有引用”，“借引用”的优点在于在代码所有的分支中你都不必考虑清理引用——换句话说，使用“借引用”执行提前退出时，你不会有内存泄漏的风险。“借引用”在内存泄漏方面的缺点是：在一些微妙的情形下，表面上看似正确的代码中，“借引用”的使用可能发现在所有者实际上已经清理之后。
A borrowed reference can be changed into an owned reference by calling Py_INCREF(). This does not affect the status of the owner from which the reference was borrowed—it creates a new owned reference, and gives full owner responsibilities (the new owner must dispose of the reference properly, as well as the previous owner).
通过调用Py_INCREF()，“借引用”可以转换成“拥有引用“。这不会影响引用被借用的所有者的状态——因为创建了一个新的“拥有引用“，并有完全的所有权限（新的所有者必须先正确地处理引用，就像之间的所有者一样）。
1.10.2 Ownership Rules 所有权规则
Whenever an object reference is passed into or out of a function, it is part of the function’s interface specification whether ownership is transferred with the reference or not.
一个对象的引用无论何时传入或传出一个函数，它都是函数接口规范的一部分，而与所有权被转让与否无关。
Most functions that return a reference to an object pass on ownership with the reference. In particular, all functions whose function it is to create a new object, such asPyInt_FromLong()and Py_BuildValue(), pass ownership to the receiver. Even if the object is not actually new, you still receive ownership of a new reference to that object. For instance, PyInt_FromLong()maintains a cache of popular values and can return a reference to a cached item.
大多数返回一个对象引用的函数通过引用转让所有权。尤其是所有那些功能就是创建新对象的函数，例如PyInt_FromLong()和Py_BuildValue()，会转让所有权给接收者。即使对象实际是不是新创建的，你接收到的仍然是对象的一个新的引用。例如：PyInt_FromLong()维护了常用值的一个缓存，能返回已缓存对象。
Many functions that extract objects from other objects also transfer ownership with the reference, for instance PyObject_GetAttrString(). The picture is less clear, here, however, since a few common routines are exceptions: PyTuple_GetItem(), PyList_GetItem(),PyDict_GetItem(), and PyDict_-GetItemString()all return references that you borrow from the tuple, list or dictionary.
许多从另外的对象中析取对象的函数也会通过引用转让所有权，例如PyObject_GetAttrString()。在这，描述的不是很清楚，然而，有一些普遍的惯例是例外：PyTuple_GetItem()、PyList_GetItem()、PyDict_GetItem()、和PyDict_-GetItemString()都是返回你从元组、列表或字典借得的引用。
The function PyImport_AddModule()also returns a borrowed reference, even though it may actually creat the object it returns: this is possible because an owned reference to the object is stored in sys.modules.
PyImport_AddModule()函数也返回一个“借引用”，尽管它可能实际创建了它所返回的对象：这可能是因为对象的一个“拥有引用”存储在sys.modules中了
When you pass an object reference into another function, in general, the function borrows the reference from you—if it needs to store it, it will use Py_INCREF()to become an independent owner. There are exactly two important exceptions to this rule: PyTuple_SetItem()andPyList_SetItem(). These functions take over ownership of the item passed to them — even if they fail! (Note that PyDict_SetItem()and friends don’t take over ownership— they are “normal.”)
当人将一个对象引用传入另一个函数，一般而言，是函数从你那借用引用——如果函数需要存储引用的话，它要使用Py_INCREF()以成一个独立的所有者。对这一规则，有两个重要的例外：PyTuple_SetItem()和PyList_SetItem()。这两个函数接管传给它们的子项的所有权——即使这些子项失效了！（注间到PyDict_SetItem()那类函数不接管所有权——它们是“正常的”。）
When a C function is called from Python, it borrows references to its arguments from the caller. The caller owns a reference to the object, so the borrowed reference’s lifetime is guaranteed until the function returns. Only when such a borrowed reference must be stored or passed on, it must be turned into an owned reference by calling Py_INCREF().
当一个C函数在Python中被调用，它从调用者那里借用自已参数的引用。调用者拥有对象的一个引用，所以“借引用”的寿命在返回值退出之前都有保证。仅当这样的“借引用”要被存储或传递时，才要通过调用Py_INCREF()来转换成“拥有引用”。
The object reference returned from a C function that is called from Python must be an owned reference — owner- ship is transferred from the function to its caller.
由在Python调用的C函数返回的对象引用必须是“拥有引用”——所有权由函数转让给了它的调用者.
1.10.3 Thin Ice要如履薄冰的地方
There are a few situations where seemingly harmless use of a borrowed reference can lead to problems. These all have to do with implicit invocations of the interpreter, which can cause the owner of a reference to dispose of it. The first and most important case to know about is usingPy_DECREF()on an unrelated object while borrowing a reference to a list item. For instance:
在少数情况下，表面上看似无害的“借引用”用法也会引发问题。这些都与会导致引用所有者废弃引用的解释器隐式调用有关。首先和最重要要了解的是：在借用列表子项的引用时对未关联的对象使用Py_DECREF()。例如：
void
bug(PyObject *list)
{
PyObject *item = PyList_GetItem(list, 0);
PyList_SetItem(list, 1, PyInt_FromLong(0L));
PyObject_Print(item, stdout, 0); /* BUG! */
}
This function first borrows a reference to list[0], then replaces list[1]with the value 0, and finally prints the borrowed reference. Looks harmless, right? But it’s not!
函数先借用了list[0]的引用，接着用0值来替换list[1]，最后打印出“借引用”。看起来没问题，对吧？但不是这样！
Let’s follow the control flow into PyList_SetItem(). The list owns references to all its items, so when item 1 is replaced, it has to dispose of the original item 1. Now let’s suppose the original item 1 was an instance of a user-defined class, and let’s further suppose that the class defined a __del__()method. If this class instance has a reference count of 1, disposing of it will call its __del__()method.
让我们来跟入PyList_SetItem()的控制流。列表拥有所有子项的引用，故当子项1被替换时，它必须丢弃初始的子项1。现在让我们假设初始的子项1是一个用户定义类的实例，并且我们更进一步假设用户定义类中定义了__del__()方法。如果这个类实例的引用计数为1，丢弃它将会调用它的__del__()方法。
Since it is written in Python, the __del__()method can execute arbitrary Python code. Could it perhaps do someting to invalidate the reference to itemin bug()? You bet! Assuming that the list passed into bug() is accessible to the __del__()method, it could execute a statement to the effect of ‘del list[0]’, and assuming this was the last reference to that object, it would free the memory associated with it, thereby invalidating item.
由于是用Python写的，__del__()方法可以执行任意Python代码。它能做某些事让bug()中item的引用失效吗？的确如此！如果传入bug()函数的列表可以访问__del__()方法，将会执行一条等效于‘del list[0]’的语句，并且如果这是对象的最后一个引用，还会释放对象相关联的内存，因而使item失效。
The solution, once you know the source of the problem, is easy: temporarily increment the reference count. The correct version of the function reads:
一旦你知道了问题的根源，解决方案就是简单的了：即时地增加引用计数。函数正确的版本如下所示：
void
no_bug(PyObject *list)
{
PyObject *item = PyList_GetItem(list, 0);
Py_INCREF(item);
PyList_SetItem(list, 1, PyInt_FromLong(0L));
PyObject_Print(item, stdout, 0);
Py_DECREF(item);
}
This is a true story. An older version of Python contained variants of this bug and someone spent a considerable
amount of time in a C debugger to figure out why his __del__()methods would fail…
这是一个真实的故事。Python的一个老版本含有这个bug的变体，并且有人为了弄明白为何它的方法会不成功，而花费了相当可观的时间在C调试器中。
The second case of problems with a borrowed reference is a variant involving threads. Normally, multiple threads in the Python interpreter can’t get in each other’s way, because there is a global lock protecting Python’s entire object space. However, it is possible to temporarily release this lock using the macro Py_BEGIN_ALLOW_-THREADS, and to re-acquire it usingPy_END_ALLOW_THREADS. This is common around blocking I/O calls, to let other threads use the processor while waiting for the I/O to complete. Obviously, the following function has the same problem as the previous one:
第二个引发“借引用”相关问题的原因是涉及线程的一个变体。正常地，Python解释器中的多个线程不能进入彼此，因为有全局解释锁在保护Python的整个对象空间。不过，可以使用宏Py_BEGIN_ALLOW_THREADS来暂停解锁，用宏Py_END_ALLOW_THREADS来复原锁定。这在I/O调用的代码块中很常见，为的是在等待I/O线程完成时，其他的线程能使用处理器。明显地，下面的函数也有与前一个函数同样的问题：
void
bug(PyObject *list)
{
PyObject *item = PyList_GetItem(list, 0);
Py_BEGIN_ALLOW_THREADS
…some blocking I/O call…
Py_END_ALLOW_THREADS
PyObject_Print(item, stdout, 0); /* BUG! */
}
1.10.4 NULL Pointers 空指针
In general, functions that take object references as arguments do not expect you to pass themNULLpointers, and will dump core (or cause later core dumps) if you do so. Functions that return object references generally return NULLonly to indicate that an exception occurred. The reason for not testing for NULL arguments is that functions often pass the objects they receive on to other function — if each function were to test for NULL, there would be a lot of redundant tests and the code would run more slowly.
通常来说，以对象引用作参数的函数不会想要你传入一个空指针，而且如果你这样做了的话会泄漏内存（或在之后导致内存泄漏）。返回值为对象引用的函数通常仅在指示有异常发生时才返回NULL。不测试NULL参数的原因在于函数经常将它们接收到的对象传给其他的函数——如果每个函数都对NULL做测试，将会有大量冗余的测试而且会使代码运行的更慢。
It is better to test for NULLonly at the “source:” when a pointer that may be NULLis received, for example, from malloc()or from a function that may raise an exception.
当接收到一个可能为NULL的指针时，仅在“源头”对NULL做测试是比较好的，例如：在malloc()中或在一个可能抛出异常的函数中
The macros Py_INCREF()and Py_DECREF()do not check for NULLpointers — however, their variants Py_XINCREF()and Py_XDECREF()do.
宏Py_INCREF()和Py_DECREF()不会对NULL指针做检测——但他的变体Py_XINCREF()和Py_XDECREF()会做。
The macros for checking for a particular object type (Pytype_Check()) don’t check forNULLpointers—again, there is much code that calls several of these in a row to test an object against various different expected types, and this would generate redundant tests. There are no variants with NULL checking.
检测特定对象类型的宏（Pytype_Check()）不会检测NULL指针——再者，有很多这样的代码在一行中就调用几个这样的宏来对各种不用的类型进行检测，这样会产生冗余测试。这些宏没有带NULL指针检测的变体。
The C function calling mechanism guarantees that the argument list passed to C functions (argsin the examples) is never NULL— in fact it guarantees that it is always a tuple.4
函数调用机制确保传入函数的参数列表不会是NULL——事实上确保了参数列表总是元组。
It is a severe error to ever let a NULLpointer “escape” to the Python user.
任何时候让NULL指针“逃逸”到Python用户处，都是一个严重的错误。
4 These guarantees don’t hold when you use the “old” style calling convention – this is still found in much existing code.
当你使用旧式的调用风格（这在许多现有的代码中还存在）时这些保证不会有效
No comments:
Post a Comment