2019-07-28 | UNLOCK

从源码了解Python的函数机制

前言

python是一门动态语言,因此了解这部分的实现比其他的来的有意义的,可以帮助我们理解所谓的动态特性,以及函数热更等

结构

/img/python_fun1.png

  • PyCodeObj
  • PyFunObj

pycodeobj是虚拟机在编译代码生成pyc的时候就确定了,而这里面可能有多个函数,虚拟机把函数也定义为一个对象pyfunobj,并且是在运行的时候才生成。

看看源码pyfunobj的组成:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
typedef struct {
PyObject_HEAD
PyObject *func_code; /* A code object */
PyObject *func_globals; /* A dictionary (other mappings won't do) */
PyObject *func_defaults; /* NULL or a tuple */
PyObject *func_closure; /* NULL or a tuple of cell objects */
PyObject *func_doc; /* The __doc__ attribute, can be anything */
PyObject *func_name; /* The __name__ attribute, a string object */
PyObject *func_dict; /* The __dict__ attribute, a dict or NULL */
PyObject *func_weakreflist; /* List of weak references */
PyObject *func_module; /* The __module__ attribute, can be anything */

/* Invariant:
* func_closure contains the bindings for func_code->co_freevars, so
* PyTuple_Size(func_closure) == PyCode_GetNumFree(func_code)
* (func_closure may be NULL if PyCode_GetNumFree(func_code) == 0).
*/
} PyFunctionObject;

这两个是如何定义函数的,我们用下面的代码来查看:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
# def f():
# a = 2
# print a


# f()


def test():
def f():
a = 2
print a

f()

import dis
print dis.dis(test)

我们用compile和dis查看编译的pycodeobj和字节码

1
2
3
4
5
6
7
8
9
10           0 LOAD_CONST               1
3 MAKE_FUNCTION 0
6 STORE_FAST 0 (f)

14 9 LOAD_FAST 0 (f)
12 CALL_FUNCTION 0
15 POP_TOP
16 LOAD_CONST 0 (None)
19 RETURN_VALUE

第10行代码也就是def f():,这里并没有a = 2;print a的字节码,也就是说函数内的字节码并不是在当前的pycodeobj里,而是在另外的pycodeobj

在用compile来查看注释部分的pycodeobj关系:

1
2
3
co =  compile(open('test.py').read(), 'test.py', 'exec')
print co.co_consts[0]
print co.co_consts[0].co_name

可以看到下面结果
/img/python_fun3.png

说明f函数的实现是在嵌套的pycodeobj里面
以上就是函数的定义和实现部分,接下来只剩f()函数调用的部分

调用

来看看CALL_FUNCTIONpython是怎么执行的

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
call_function(PyObject ***pp_stack, int oparg)
{
int na = oparg & 0xff;
int nk = (oparg>>8) & 0xff;
int n = na + 2 * nk;
PyObject **pfunc = (*pp_stack) - n - 1;
PyObject *func = *pfunc;
PyObject *x, *w;

if (PyCFunction_Check(func) && nk == 0) {
......
} else {
......
if (PyFunction_Check(func))
x = fast_function(func, pp_stack, n, na, nk);
else
x = do_call(func, pp_stack, na, nk);**
......
}
......
}
return x;
}

再看看fast_function的实现

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
static PyObject *
fast_function(PyObject *func, PyObject ***pp_stack, int n, int na, int nk)
{
PyCodeObject *co = (PyCodeObject *)PyFunction_GET_CODE(func);
PyObject *globals = PyFunction_GET_GLOBALS(func);
PyObject *argdefs = PyFunction_GET_DEFAULTS(func);
PyObject **d = NULL;
int nd = 0;

PCALL(PCALL_FUNCTION);
PCALL(PCALL_FAST_FUNCTION);
if (argdefs == NULL && co->co_argcount == n && nk==0 &&
co->co_flags == (CO_OPTIMIZED | CO_NEWLOCALS | CO_NOFREE)) {
PyFrameObject *f;
PyObject *retval = NULL;
PyThreadState *tstate = PyThreadState_GET();
PyObject **fastlocals, **stack;
int i;

PCALL(PCALL_FASTER_FUNCTION);
assert(globals != NULL);
/* XXX Perhaps we should create a specialized
PyFrame_New() that doesn't take locals, but does
take builtins without sanity checking them.
*/
assert(tstate != NULL);
f = PyFrame_New(tstate, co, globals, NULL);
if (f == NULL)
return NULL;

fastlocals = f->f_localsplus;
stack = (*pp_stack) - n;

for (i = 0; i < n; i++) {
Py_INCREF(*stack);
fastlocals[i] = *stack++;
}
retval = PyEval_EvalFrameEx(f,0);
++tstate->recursion_depth;
Py_DECREF(f);
--tstate->recursion_depth;
return retval;
}
if (argdefs != NULL) {
d = &PyTuple_GET_ITEM(argdefs, 0);
nd = Py_SIZE(argdefs);
}
return PyEval_EvalCodeEx(co, globals,
(PyObject *)NULL, (*pp_stack)-n, na,
(*pp_stack)-2*nk, nk, d, nd,
PyFunction_GET_CLOSURE(func));
}

可以看到上面的过程就是创建新的PyFrameObj栈帧环境,拷贝global空间和传递PyCodeObj过去,然后又是新的执行字节码的操作,只不过这时候的字节码就是对应函数的

/img/python_fun2.png

评论加载中