Inside python interpreter
===
###### tags:`python`
:::info
Python 3.5
:::
# default builtins
sys.modules is a dictionary mapping the names of imported modules to the module object holding the code
```python
import sys
dir(sys.modules['builtins']) == dir(__builtins__)
```
True
```python
sorted(sys.modules.keys())
```
['IPython',
'IPython.core',
'IPython.core.alias',
'IPython.core.application',
'IPython.core.autocall',
'IPython.core.builtin_trap',
'IPython.core.compilerop',
'IPython.core.completer',
'IPython.core.completerlib',
'IPython.core.crashhandler',
'IPython.core.debugger',
'IPython.core.display',
'IPython.core.display_trap',
'IPython.core.displayhook',
'IPython.core.displaypub',
'IPython.core.error',
'IPython.core.events',
'IPython.core.excolors',
'IPython.core.extensions',
'IPython.core.formatters',
'IPython.core.getipython',
'IPython.core.history',
'IPython.core.hooks',
'IPython.core.inputsplitter',
'IPython.core.inputtransformer',
'IPython.core.interactiveshell',
'IPython.core.latex_symbols',
'IPython.core.logger',
'IPython.core.macro',
'IPython.core.magic',
'IPython.core.magic_arguments',
'IPython.core.magics',
'IPython.core.magics.auto',
'IPython.core.magics.basic',
'IPython.core.magics.code',
'IPython.core.magics.config',
'IPython.core.magics.display',
'IPython.core.magics.execution',
'IPython.core.magics.extension',
'IPython.core.magics.history',
'IPython.core.magics.logging',
'IPython.core.magics.namespace',
'IPython.core.magics.osm',
'IPython.core.magics.pylab',
'IPython.core.magics.script',
'IPython.core.oinspect',
'IPython.core.page',
'IPython.core.payload',
'IPython.core.payloadpage',
'IPython.core.prefilter',
'IPython.core.profiledir',
'IPython.core.pylabtools',
'IPython.core.release',
'IPython.core.shellapp',
'IPython.core.splitinput',
'IPython.core.ultratb',
'IPython.core.usage',
'IPython.display',
'IPython.extensions',
'IPython.extensions.storemagic',
'IPython.lib',
'IPython.lib.backgroundjobs',
'IPython.lib.clipboard',
'IPython.lib.display',
'IPython.lib.pretty',
'IPython.lib.security',
'IPython.paths',
'IPython.terminal',
'IPython.terminal.debugger',
'IPython.terminal.embed',
'IPython.terminal.interactiveshell',
'IPython.terminal.ipapp',
'IPython.terminal.magics',
'IPython.terminal.prompts',
'IPython.terminal.pt_inputhooks',
'IPython.terminal.ptutils',
'IPython.terminal.shortcuts',
'IPython.testing',
'IPython.testing.skipdoctest',
'IPython.utils',
'IPython.utils.PyColorize',
'IPython.utils._process_common',
'IPython.utils._process_posix',
'IPython.utils._sysinfo',
'IPython.utils.capture',
'IPython.utils.colorable',
'IPython.utils.coloransi',
'IPython.utils.contexts',
'IPython.utils.data',
'IPython.utils.decorators',
'IPython.utils.dir2',
'IPython.utils.encoding',
'IPython.utils.frame',
'IPython.utils.generics',
'IPython.utils.importstring',
'IPython.utils.io',
'IPython.utils.ipstruct',
'IPython.utils.module_paths',
'IPython.utils.openpy',
'IPython.utils.path',
'IPython.utils.process',
'IPython.utils.py3compat',
'IPython.utils.sentinel',
'IPython.utils.strdispatch',
'IPython.utils.sysinfo',
'IPython.utils.syspathcontext',
'IPython.utils.tempdir',
'IPython.utils.terminal',
'IPython.utils.text',
'IPython.utils.timing',
'IPython.utils.tokenize2',
'IPython.utils.tokenutil',
'IPython.utils.wildcard',
'__future__',
'__main__',
'__mp_main__',
'_ast',
'_bisect',
'_bootlocale',
'_bz2',
'_codecs',
'_collections',
'_collections_abc',
'_compat_pickle',
'_compression',
'_ctypes',
'_curses',
'_cython_0_25_1',
'_datetime',
'_decimal',
'_frozen_importlib',
'_frozen_importlib_external',
'_functools',
'_hashlib',
'_heapq',
'_imp',
'_io',
'_json',
'_locale',
'_lsprof',
'_lzma',
'_multiprocessing',
'_opcode',
'_operator',
'_pickle',
'_posixsubprocess',
'_random',
'_signal',
'_sitebuiltins',
'_socket',
'_sqlite3',
'_sre',
'_stat',
'_string',
'_strptime',
'_struct',
'_sysconfigdata',
'_sysconfigdata_m',
'_thread',
'_warnings',
'_weakref',
'_weakrefset',
'abc',
'apport_python_hook',
'argparse',
'array',
'ast',
'atexit',
'base64',
'bdb',
'binascii',
'bisect',
'builtins',
'bz2',
'cProfile',
'calendar',
'cmd',
'code',
'codecs',
'codeop',
'collections',
'collections.abc',
'concurrent',
'concurrent.futures',
'concurrent.futures._base',
'concurrent.futures.process',
'concurrent.futures.thread',
'contextlib',
'copy',
'copyreg',
'ctypes',
'ctypes._endian',
'ctypes.util',
'curses',
'datetime',
'dateutil',
'dateutil._common',
'dateutil.parser',
'dateutil.relativedelta',
'dateutil.tz',
'dateutil.tz._common',
'dateutil.tz.tz',
'decimal',
'decorator',
'difflib',
'dis',
'email',
'email._encoded_words',
'email._parseaddr',
'email._policybase',
'email.base64mime',
'email.charset',
'email.encoders',
'email.errors',
'email.feedparser',
'email.header',
'email.iterators',
'email.message',
'email.parser',
'email.quoprimime',
'email.utils',
'encodings',
'encodings.aliases',
'encodings.latin_1',
'encodings.utf_8',
'enum',
'errno',
'faulthandler',
'fcntl',
'fnmatch',
'functools',
'gc',
'genericpath',
'getopt',
'getpass',
'gettext',
'glob',
'grp',
'hashlib',
'heapq',
'hmac',
'html',
'html.entities',
'imp',
'importlib',
'importlib._bootstrap',
'importlib._bootstrap_external',
'importlib.abc',
'importlib.machinery',
'importlib.util',
'inspect',
'io',
'ipykernel',
'ipykernel._version',
'ipykernel.codeutil',
'ipykernel.comm',
'ipykernel.comm.comm',
'ipykernel.comm.manager',
'ipykernel.connect',
'ipykernel.datapub',
'ipykernel.displayhook',
'ipykernel.heartbeat',
'ipykernel.iostream',
'ipykernel.ipkernel',
'ipykernel.jsonutil',
'ipykernel.kernelapp',
'ipykernel.kernelbase',
'ipykernel.parentpoller',
'ipykernel.pickleutil',
'ipykernel.serialize',
'ipykernel.zmqshell',
'ipython_genutils',
'ipython_genutils._version',
'ipython_genutils.encoding',
'ipython_genutils.importstring',
'ipython_genutils.path',
'ipython_genutils.py3compat',
'ipython_genutils.text',
'ipywidgets',
'ipywidgets._version',
'ipywidgets.widgets',
'ipywidgets.widgets.domwidget',
'ipywidgets.widgets.interaction',
'ipywidgets.widgets.trait_types',
'ipywidgets.widgets.valuewidget',
'ipywidgets.widgets.widget',
'ipywidgets.widgets.widget_bool',
'ipywidgets.widgets.widget_box',
'ipywidgets.widgets.widget_button',
'ipywidgets.widgets.widget_color',
'ipywidgets.widgets.widget_controller',
'ipywidgets.widgets.widget_core',
'ipywidgets.widgets.widget_date',
'ipywidgets.widgets.widget_float',
'ipywidgets.widgets.widget_image',
'ipywidgets.widgets.widget_int',
'ipywidgets.widgets.widget_layout',
'ipywidgets.widgets.widget_link',
'ipywidgets.widgets.widget_output',
'ipywidgets.widgets.widget_selection',
'ipywidgets.widgets.widget_selectioncontainer',
'ipywidgets.widgets.widget_string',
'ipywidgets.widgets.widget_style',
'itertools',
'jedi',
'jedi._compatibility',
'jedi.api',
'jedi.api.classes',
'jedi.api.completion',
'jedi.api.helpers',
'jedi.api.interpreter',
'jedi.api.keywords',
'jedi.api.usages',
'jedi.cache',
'jedi.common',
'jedi.debug',
'jedi.evaluate',
'jedi.evaluate.analysis',
'jedi.evaluate.cache',
'jedi.evaluate.compiled',
'jedi.evaluate.compiled.fake',
'jedi.evaluate.compiled.mixed',
'jedi.evaluate.context',
'jedi.evaluate.docstrings',
'jedi.evaluate.dynamic',
'jedi.evaluate.filters',
'jedi.evaluate.finder',
'jedi.evaluate.flow_analysis',
'jedi.evaluate.helpers',
'jedi.evaluate.imports',
'jedi.evaluate.instance',
'jedi.evaluate.iterable',
'jedi.evaluate.param',
'jedi.evaluate.pep0484',
'jedi.evaluate.precedence',
'jedi.evaluate.recursion',
'jedi.evaluate.representation',
'jedi.evaluate.site',
'jedi.evaluate.stdlib',
'jedi.evaluate.sys_path',
'jedi.parser',
'jedi.parser.cache',
'jedi.parser.parser',
'jedi.parser.pgen2',
'jedi.parser.pgen2.grammar',
'jedi.parser.pgen2.parse',
'jedi.parser.pgen2.pgen',
'jedi.parser.python',
'jedi.parser.python.diff',
'jedi.parser.python.parser',
'jedi.parser.python.tree',
'jedi.parser.token',
'jedi.parser.tokenize',
'jedi.parser.tree',
'jedi.settings',
'json',
'json.decoder',
'json.encoder',
'json.scanner',
'jupyter_client',
'jupyter_client._version',
'jupyter_client.adapter',
'jupyter_client.blocking',
'jupyter_client.blocking.channels',
'jupyter_client.blocking.client',
'jupyter_client.channels',
'jupyter_client.channelsabc',
'jupyter_client.client',
'jupyter_client.clientabc',
'jupyter_client.connect',
'jupyter_client.jsonutil',
'jupyter_client.kernelspec',
'jupyter_client.launcher',
'jupyter_client.localinterfaces',
'jupyter_client.manager',
'jupyter_client.managerabc',
'jupyter_client.multikernelmanager',
'jupyter_client.session',
'jupyter_core',
'jupyter_core.paths',
'jupyter_core.version',
'keyrings',
'keyword',
'linecache',
'locale',
'logging',
'logging.handlers',
'lzma',
'marshal',
'math',
'mimetypes',
'multiprocessing',
'multiprocessing.connection',
'multiprocessing.context',
'multiprocessing.process',
'multiprocessing.reduction',
'multiprocessing.util',
'ntpath',
'numbers',
'opcode',
'operator',
'optparse',
'os',
'os.path',
'pathlib',
'pdb',
'pexpect',
'pexpect.exceptions',
'pexpect.expect',
'pexpect.pty_spawn',
'pexpect.run',
'pexpect.spawnbase',
'pexpect.utils',
'pickle',
'pickleshare',
'pkg_resources',
'pkg_resources._vendor',
'pkg_resources._vendor.packaging.__about__',
'pkg_resources._vendor.six',
'pkg_resources._vendor.six.moves',
'pkg_resources.extern',
'pkg_resources.extern.appdirs',
'pkg_resources.extern.packaging',
'pkg_resources.extern.packaging._compat',
'pkg_resources.extern.packaging._structures',
'pkg_resources.extern.packaging.markers',
'pkg_resources.extern.packaging.requirements',
'pkg_resources.extern.packaging.specifiers',
'pkg_resources.extern.packaging.version',
'pkg_resources.extern.pyparsing',
'pkg_resources.extern.six',
'pkg_resources.extern.six.moves',
'pkg_resources.extern.six.moves.urllib',
'pkgutil',
'platform',
'plistlib',
'posix',
'posixpath',
'pprint',
'profile',
'prompt_toolkit',
'prompt_toolkit.application',
'prompt_toolkit.auto_suggest',
'prompt_toolkit.buffer',
'prompt_toolkit.buffer_mapping',
'prompt_toolkit.cache',
'prompt_toolkit.clipboard',
'prompt_toolkit.clipboard.base',
'prompt_toolkit.clipboard.in_memory',
'prompt_toolkit.completion',
'prompt_toolkit.document',
'prompt_toolkit.enums',
'prompt_toolkit.eventloop',
'prompt_toolkit.eventloop.base',
'prompt_toolkit.eventloop.callbacks',
'prompt_toolkit.filters',
'prompt_toolkit.filters.base',
'prompt_toolkit.filters.cli',
'prompt_toolkit.filters.types',
'prompt_toolkit.filters.utils',
'prompt_toolkit.history',
'prompt_toolkit.input',
'prompt_toolkit.interface',
'prompt_toolkit.key_binding',
'prompt_toolkit.key_binding.bindings',
'prompt_toolkit.key_binding.bindings.basic',
'prompt_toolkit.key_binding.bindings.completion',
'prompt_toolkit.key_binding.bindings.emacs',
'prompt_toolkit.key_binding.bindings.named_commands',
'prompt_toolkit.key_binding.bindings.scroll',
'prompt_toolkit.key_binding.bindings.vi',
'prompt_toolkit.key_binding.defaults',
'prompt_toolkit.key_binding.digraphs',
'prompt_toolkit.key_binding.input_processor',
'prompt_toolkit.key_binding.manager',
'prompt_toolkit.key_binding.registry',
'prompt_toolkit.key_binding.vi_state',
'prompt_toolkit.keys',
'prompt_toolkit.layout',
'prompt_toolkit.layout.containers',
'prompt_toolkit.layout.controls',
'prompt_toolkit.layout.dimension',
'prompt_toolkit.layout.lexers',
'prompt_toolkit.layout.margins',
'prompt_toolkit.layout.menus',
'prompt_toolkit.layout.mouse_handlers',
'prompt_toolkit.layout.processors',
'prompt_toolkit.layout.prompt',
'prompt_toolkit.layout.screen',
'prompt_toolkit.layout.toolbars',
'prompt_toolkit.layout.utils',
'prompt_toolkit.mouse_events',
'prompt_toolkit.output',
'prompt_toolkit.reactive',
'prompt_toolkit.renderer',
'prompt_toolkit.search_state',
'prompt_toolkit.selection',
'prompt_toolkit.shortcuts',
'prompt_toolkit.styles',
'prompt_toolkit.styles.base',
'prompt_toolkit.styles.defaults',
'prompt_toolkit.styles.from_dict',
'prompt_toolkit.styles.from_pygments',
'prompt_toolkit.styles.utils',
'prompt_toolkit.terminal',
'prompt_toolkit.terminal.vt100_input',
'prompt_toolkit.terminal.vt100_output',
'prompt_toolkit.token',
'prompt_toolkit.utils',
'prompt_toolkit.validation',
'pstats',
'pty',
'ptyprocess',
'ptyprocess.ptyprocess',
'ptyprocess.util',
'pwd',
'pydoc',
'pydoc_data',
'pydoc_data.topics',
'pyexpat',
'pyexpat.errors',
'pyexpat.model',
'pygments',
'pygments.filter',
'pygments.filters',
'pygments.formatter',
'pygments.formatters',
'pygments.formatters._mapping',
'pygments.formatters.html',
'pygments.lexer',
'pygments.lexers',
'pygments.lexers._mapping',
'pygments.lexers.python',
'pygments.modeline',
'pygments.plugin',
'pygments.regexopt',
'pygments.style',
'pygments.styles',
'pygments.styles.default',
'pygments.token',
'pygments.unistring',
'pygments.util',
'queue',
'quopri',
'random',
're',
'reprlib',
'resource',
'runpy',
'select',
'selectors',
'shlex',
'shutil',
'signal',
'simplegeneric',
'simplejson',
'simplejson._speedups',
'simplejson.compat',
'simplejson.decoder',
'simplejson.encoder',
'simplejson.scanner',
'site',
'sitecustomize',
'six',
'six.moves',
'socket',
'sqlite3',
'sqlite3.dbapi2',
'sre_compile',
'sre_constants',
'sre_parse',
'stat',
'storemagic',
'string',
'struct',
'subprocess',
'sys',
'sysconfig',
'tempfile',
'termios',
'textwrap',
'threading',
'time',
'timeit',
'token',
'tokenize',
'tornado',
'tornado.concurrent',
'tornado.escape',
'tornado.ioloop',
'tornado.log',
'tornado.platform',
'tornado.platform.auto',
'tornado.platform.common',
'tornado.platform.interface',
'tornado.platform.posix',
'tornado.speedups',
'tornado.stack_context',
'tornado.util',
'traceback',
'traitlets',
'traitlets._version',
'traitlets.config',
'traitlets.config.application',
'traitlets.config.configurable',
'traitlets.config.loader',
'traitlets.log',
'traitlets.traitlets',
'traitlets.utils',
'traitlets.utils.bunch',
'traitlets.utils.getargspec',
'traitlets.utils.importstring',
'traitlets.utils.sentinel',
'tty',
'types',
'typing',
'typing.io',
'typing.re',
'unicodedata',
'urllib',
'urllib.parse',
'uu',
'uuid',
'warnings',
'wcwidth',
'wcwidth.table_wide',
'wcwidth.table_zero',
'wcwidth.wcwidth',
'weakref',
'xml',
'xml.parsers',
'xml.parsers.expat',
'xml.parsers.expat.errors',
'xml.parsers.expat.model',
'zipfile',
'zipimport',
'zlib',
'zmq',
'zmq.backend',
'zmq.backend.cython',
'zmq.backend.cython._device',
'zmq.backend.cython._poll',
'zmq.backend.cython._version',
'zmq.backend.cython.constants',
'zmq.backend.cython.context',
'zmq.backend.cython.error',
'zmq.backend.cython.message',
'zmq.backend.cython.socket',
'zmq.backend.cython.utils',
'zmq.backend.select',
'zmq.error',
'zmq.eventloop',
'zmq.eventloop.ioloop',
'zmq.eventloop.zmqstream',
'zmq.sugar',
'zmq.sugar.attrsettr',
'zmq.sugar.constants',
'zmq.sugar.context',
'zmq.sugar.frame',
'zmq.sugar.poll',
'zmq.sugar.socket',
'zmq.sugar.stopwatch',
'zmq.sugar.tracker',
'zmq.sugar.version',
'zmq.utils',
'zmq.utils.constant_names',
'zmq.utils.jsonapi',
'zmq.utils.strtypes']
## Built-in Modules
```python
for name in sys.builtin_module_names:
print(name)
```
_ast
_bisect
_codecs
_collections
_datetime
_elementtree
_functools
_heapq
_imp
_io
_locale
_md5
_operator
_pickle
_posixsubprocess
_random
_sha1
_sha256
_sha512
_signal
_socket
_sre
_stat
_string
_struct
_symtable
_thread
_tracemalloc
_warnings
_weakref
array
atexit
binascii
builtins
errno
faulthandler
fcntl
gc
grp
itertools
marshal
math
posix
pwd
pyexpat
select
spwd
sys
syslog
time
unicodedata
xxsubtype
zipimport
zlib
## Import Path
```python
import sys
for d in sys.path:
print(d)
```
/usr/lib/python35.zip
/usr/lib/python3.5
/usr/lib/python3.5/plat-x86_64-linux-gnu
/usr/lib/python3.5/lib-dynload
/home/splasky/.local/lib/python3.5/site-packages
/usr/local/lib/python3.5/dist-packages
/usr/lib/python3/dist-packages
/usr/local/lib/python3.5/dist-packages/IPython/extensions
/home/splasky/.ipython
```python
%%writefile example.py
DATA='a'
```
Writing example.py
```python
import example
print(example.__file__)
print(example.DATA)
```
/home/splasky/jupyter-notebook/example.py
a
> Changing the path between the initial import and the call to `reload()` means a different module may be loaded the second time.
# Import hook
```python
import sys
class NoisyImportFinder(object):
PATH_TRIGGER = 'NoisyImportFinder_PATH_TRIGGER'
def __init__(self, path_entry):
print ('Checking NoisyImportFinder support for %s' % path_entry)
if path_entry != self.PATH_TRIGGER:
print ('NoisyImportFinder does not work for %s' % path_entry)
raise ImportError()
return
def find_module(self, fullname, path=None):
print ('NoisyImportFinder looking for "%s"' % fullname)
return None
sys.path_hooks.append(NoisyImportFinder)
sys.path.insert(0, NoisyImportFinder.PATH_TRIGGER)
try:
import target_module
except Exception as e:
print ('Import failed:', e)
```
Checking NoisyImportFinder support for NoisyImportFinder_PATH_TRIGGER
NoisyImportFinder looking for "target_module"
Import failed: No module named 'target_module'
# inside the main
```python
dir(sys.modules['__main__'])
```
['In', 'NamespaceMagics', 'NoisyImportFinder', 'Out', '_', '_10', '_7', '_Jupyter', '__', '___', '__builtin__', '__builtins__', '__doc__', '__loader__', '__name__', '__package__', '__spec__', '_dh', '_getsizeof', '_i', '_i1', '_i10', '_i11', '_i12', '_i13', '_i14', '_i15', '_i16', '_i17', '_i18', '_i19', '_i2', '_i20', '_i21', '_i22', '_i23', '_i24', '_i3', '_i4', '_i5', '_i6', '_i7', '_i8', '_i9', '_ih', '_ii', '_iii', '_nms', '_oh', 'd', 'dis', 'example', 'exit', 'get_ipython', 'getsizeof', 'json', 'name', 'oo', 'pprint', 'print_function', 'quit', 'sys', 'var_dic_list']
```python
from IPython.core.interactiveshell import InteractiveShell
InteractiveShell.ast_node_interactivity = "all"
```
# How python import work?
```python
import dis
```
```python
dis.dis("import sys")
```
1 0 LOAD_CONST 0 (0)
3 LOAD_CONST 1 (None)
6 IMPORT_NAME 0 (sys)
9 STORE_NAME 0 (sys)
12 LOAD_CONST 1 (None)
15 RETURN_VALUE
```python
dis.dis("import os")
```
1 0 LOAD_CONST 0 (0)
3 LOAD_CONST 1 (None)
6 IMPORT_NAME 0 (os)
9 STORE_NAME 0 (os)
12 LOAD_CONST 1 (None)
15 RETURN_VALUE
```python
# import module dynamically
sys = __import__('sys')
print(sys)
```
<module 'sys' (built-in)>
```python
oo = __import__('os')
print(oo)
```
<module 'os' from '/usr/lib/python3.5/os.py'>
```python
%%writefile c_modules.c
#include <Python.h>
static PyObject* helloworld(PyObject* self)
{
return Py_BuildValue("s", "Hello, Python extensions!!");
}
static char helloworld_docs[] =
"helloworld( ): Any message you want to put here!!\n";
static PyMethodDef helloworld_funcs[] = {
{"helloworld", helloworld,
METH_NOARGS, helloworld_docs},
{NULL, NULL, 0, NULL}
};
static struct PyModuleDef helloworldModule = {
PyModuleDef_HEAD_INIT,
"hello", /* name of module */
helloworld_docs, /* module documentation, may be NULL */
-1, /* size of per-interpreter state of the module,
or -1 if the module keeps state in global variables. */
helloworld_funcs
};
PyMODINIT_FUNC
PyInit_helloworld(void)
{
return PyModule_Create(&helloworldModule);
}
```
Overwriting c_modules.c
```python
%%writefile setup.py
from distutils.core import setup, Extension
setup(name='helloworld', version='1.0', \
ext_modules=[Extension('helloworld', ['c_modules.c'])])
```
Overwriting setup.py
python3 ./setup.py install
## look inside how c module import into python
```python
# dis_import.py
import helloworld
helloworld.helloworld()
```
dis assembly the module
```
python -m dis dis_import.py
6 0 LOAD_CONST 0 (0)
2 LOAD_CONST 1 (None)
4 IMPORT_NAME 0 (helloworld)
6 STORE_NAME 0 (helloworld)
8 8 LOAD_NAME 0 (helloworld)
10 LOAD_METHOD 0 (helloworld)
12 CALL_METHOD 0
14 POP_TOP
16 LOAD_CONST 1 (None)
18 RETURN_VALUE
```
as same as like import python module
## take more look inside source code
IMPORT_NAME:
```c
TARGET(IMPORT_NAME) {
¦ ¦ PyObject *name = GETITEM(names, oparg);
¦ ¦ PyObject *fromlist = POP();
¦ ¦ PyObject *level = TOP();
¦ ¦ PyObject *res;
¦ ¦ res = import_name(f, name, fromlist, level);
¦ ¦ Py_DECREF(level);
¦ ¦ Py_DECREF(fromlist);
¦ ¦ SET_TOP(res);
¦ ¦ if (res == NULL)
¦ ¦ ¦ goto error;
¦ ¦ DISPATCH();
¦ }
...
static PyObject *
import_name(PyFrameObject *f, PyObject *name, PyObject *fromlist, PyObject *level)
{
_Py_IDENTIFIER(__import__);
PyObject *import_func, *res;
PyObject* stack[5];
import_func = _PyDict_GetItemId(f->f_builtins, &PyId___import__);
if (import_func == NULL) {
¦ PyErr_SetString(PyExc_ImportError, "__import__ not found");
¦ return NULL;
}
/* Fast path for not overloaded __import__. */
if (import_func == PyThreadState_GET()->interp->import_func) {
¦ int ilevel = _PyLong_AsInt(level);
¦ if (ilevel == -1 && PyErr_Occurred()) {
¦ ¦ return NULL;
¦ }
¦ res = PyImport_ImportModuleLevelObject(
¦ ¦ ¦ ¦ ¦ name,
¦ ¦ ¦ ¦ ¦ f->f_globals,
¦ ¦ ¦ ¦ ¦ f->f_locals == NULL ? Py_None : f->f_locals,
¦ ¦ ¦ ¦ ¦ fromlist,
¦ ¦ ¦ ¦ ¦ ilevel);
¦ return res;
}
Py_INCREF(import_func);
stack[0] = name;
stack[1] = f->f_globals;
stack[2] = f->f_locals == NULL ? Py_None : f->f_locals;
stack[3] = fromlist;
stack[4] = level;
res = _PyObject_FastCall(import_func, stack, 5);
Py_DECREF(import_func);
return res;
}
PyObject *
PyImport_ImportModuleLevelObject(PyObject *name, PyObject *globals,
¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦PyObject *locals, PyObject *fromlist,
¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦int level)
{
_Py_IDENTIFIER(_find_and_load);
_Py_IDENTIFIER(_handle_fromlist);
PyObject *abs_name = NULL;
PyObject *final_mod = NULL;
PyObject *mod = NULL;
PyObject *package = NULL;
PyInterpreterState *interp = PyThreadState_GET()->interp;
int has_from;
if (name == NULL) {
¦ PyErr_SetString(PyExc_ValueError, "Empty module name");
¦ goto error;
}
/* The below code is importlib.__import__() & _gcd_import(), ported to C
¦ for added performance. */
if (!PyUnicode_Check(name)) {
¦ PyErr_SetString(PyExc_TypeError, "module name must be a string");
¦ goto error;
}
if (PyUnicode_READY(name) < 0) {
¦ goto error;
}
if (level < 0) {
¦ PyErr_SetString(PyExc_ValueError, "level must be >= 0");
¦ goto error;
}
if (level > 0) {
¦ abs_name = resolve_name(name, globals, level);
¦ if (abs_name == NULL)
¦ ¦ goto error;
}
else { /* level == 0 */
¦ if (PyUnicode_GET_LENGTH(name) == 0) {
¦ ¦ PyErr_SetString(PyExc_ValueError, "Empty module name");
¦ ¦ goto error;
¦ }
¦ abs_name = name;
¦ Py_INCREF(abs_name);
}
mod = PyImport_GetModule(abs_name);
if (mod != NULL && mod != Py_None) {
¦ _Py_IDENTIFIER(__spec__);
¦ _Py_IDENTIFIER(_initializing);
¦ _Py_IDENTIFIER(_lock_unlock_module);
¦ PyObject *value = NULL;
¦ PyObject *spec;
¦ int initializing = 0;
¦ /* Optimization: only call _bootstrap._lock_unlock_module() if
¦ ¦ __spec__._initializing is true.
¦ ¦ NOTE: because of this, initializing must be set *before*
¦ ¦ stuffing the new module in sys.modules.
¦ ¦*/
¦ spec = _PyObject_GetAttrId(mod, &PyId___spec__);
¦ if (spec != NULL) {
¦ ¦ value = _PyObject_GetAttrId(spec, &PyId__initializing);
¦ ¦ Py_DECREF(spec);
¦ }
¦ if (value == NULL)
¦ ¦ PyErr_Clear();
¦ else {
¦ ¦ initializing = PyObject_IsTrue(value);
¦ ¦ Py_DECREF(value);
¦ ¦ if (initializing == -1)
¦ ¦ ¦ PyErr_Clear();
¦ ¦ if (initializing > 0) {
¦ ¦ ¦ value = _PyObject_CallMethodIdObjArgs(interp->importlib,
¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦ &PyId__lock_unlock_module, abs_name,
¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦ NULL);
¦ if (value == NULL)
¦ ¦ ¦ ¦ goto error;
¦ ¦ ¦ Py_DECREF(value);
¦ ¦ }
¦ }
}
else {
¦ static int ximporttime = 0;
¦ static int import_level;
¦ static _PyTime_t accumulated;
¦ _Py_IDENTIFIER(importtime);
¦ _PyTime_t t1 = 0, accumulated_copy = accumulated;
¦ Py_XDECREF(mod);
¦ /* XOptions is initialized after first some imports.
¦ ¦* So we can't have negative cache.
¦ ¦* Anyway, importlib.__find_and_load is much slower than
¦ ¦* _PyDict_GetItemId()
¦ ¦*/
¦ if (ximporttime == 0) {
¦ ¦ PyObject *xoptions = PySys_GetXOptions();
¦ ¦ if (xoptions) {
¦ ¦ ¦ PyObject *value = _PyDict_GetItemId(xoptions, &PyId_importtime);
¦ ¦ ¦ ximporttime = (value == Py_True);
¦ ¦ }
¦ ¦ if (ximporttime) {
¦ ¦ ¦ fputs("import time: self [us] | cumulative | imported package\n",
¦ ¦ ¦ ¦ ¦ stderr);
¦ ¦ }
¦ }
¦ if (ximporttime) {
¦ ¦ import_level++;
¦ ¦ t1 = _PyTime_GetMonotonicClock();
¦ ¦ accumulated = 0;
¦ }
¦ if (PyDTrace_IMPORT_FIND_LOAD_START_ENABLED())
¦ ¦ PyDTrace_IMPORT_FIND_LOAD_START(PyUnicode_AsUTF8(abs_name));
¦ mod = _PyObject_CallMethodIdObjArgs(interp->importlib,
¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦ &PyId__find_and_load, abs_name,
¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦ interp->import_func, NULL);
¦ if (PyDTrace_IMPORT_FIND_LOAD_DONE_ENABLED())
¦ ¦ PyDTrace_IMPORT_FIND_LOAD_DONE(PyUnicode_AsUTF8(abs_name),
¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦ mod != NULL);
¦ if (ximporttime) {
¦ ¦ _PyTime_t cum = _PyTime_GetMonotonicClock() - t1;
¦ ¦ import_level--;
¦ ¦ fprintf(stderr, "import time: %9ld | %10ld | %*s%s\n",
¦ ¦ ¦ ¦ (long)_PyTime_AsMicroseconds(cum - accumulated, _PyTime_ROUND_CEILING),
¦ ¦ ¦ ¦ (long)_PyTime_AsMicroseconds(cum, _PyTime_ROUND_CEILING),
¦ ¦ ¦ ¦ import_level*2, "", PyUnicode_AsUTF8(abs_name));
¦ ¦ accumulated = accumulated_copy + cum;
¦ }
¦ if (mod == NULL) {
¦ ¦ goto error;
¦ }
}
has_from = 0;
if (fromlist != NULL && fromlist != Py_None) {
¦ has_from = PyObject_IsTrue(fromlist);
¦ if (has_from < 0)
¦ ¦ goto error;
}
if (!has_from) {
¦ Py_ssize_t len = PyUnicode_GET_LENGTH(name);
¦ if (level == 0 || len > 0) {
¦ ¦ Py_ssize_t dot;
¦ dot = PyUnicode_FindChar(name, '.', 0, len, 1);
¦ ¦ if (dot == -2) {
¦ ¦ ¦ goto error;
¦ ¦ }
¦ ¦ if (dot == -1) {
¦ ¦ ¦ /* No dot in module name, simple exit */
¦ ¦ ¦ final_mod = mod;
¦ ¦ ¦ Py_INCREF(mod);
¦ ¦ ¦ goto error;
¦ ¦ }
¦ ¦ if (level == 0) {
¦ ¦ ¦ PyObject *front = PyUnicode_Substring(name, 0, dot);
¦ ¦ ¦ if (front == NULL) {
¦ ¦ ¦ ¦ goto error;
¦ ¦ ¦ }
¦ ¦ ¦ final_mod = PyImport_ImportModuleLevelObject(front, NULL, NULL, NULL, 0);
¦ ¦ ¦ Py_DECREF(front);
¦ ¦ }
¦ ¦ else {
¦ ¦ ¦ Py_ssize_t cut_off = len - dot;
¦ ¦ ¦ Py_ssize_t abs_name_len = PyUnicode_GET_LENGTH(abs_name);
¦ ¦ ¦ PyObject *to_return = PyUnicode_Substring(abs_name, 0,
¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦ abs_name_len - cut_off);
¦ ¦ ¦ if (to_return == NULL) {
¦ ¦ ¦ ¦ goto error;
¦ ¦ ¦ }
¦ ¦ ¦ final_mod = PyImport_GetModule(to_return);
¦ ¦ ¦ Py_DECREF(to_return);
¦ ¦ ¦ if (final_mod == NULL) {
¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦ PyErr_Format(PyExc_KeyError,
¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦"%R not in sys.modules as expected",
¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦to_return);
¦ ¦ ¦ ¦ goto error;
¦ ¦ ¦ }
¦ ¦ }
¦ }
¦ else {
¦ ¦ final_mod = mod;
¦ ¦ Py_INCREF(mod);
¦ }
}
else {
¦ final_mod = _PyObject_CallMethodIdObjArgs(interp->importlib,
¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦ &PyId__handle_fromlist, mod,
¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦ fromlist, interp->import_func,
¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦ NULL);
}
error:
Py_XDECREF(abs_name);
Py_XDECREF(mod);
Py_XDECREF(package);
if (final_mod == NULL)
¦ remove_importlib_frames();
return final_mod;
}
```
```c
/* --- Core PyObject call functions ------------------------------- */
PyObject *
_PyObject_FastCallDict(PyObject *callable, PyObject **args, Py_ssize_t nargs,
¦ ¦ ¦ ¦ ¦ PyObject *kwargs)
{
/* _PyObject_FastCallDict() must not be called with an exception set,
¦ because it can clear it (directly or indirectly) and so the
¦ caller loses its exception */
assert(!PyErr_Occurred());
assert(callable != NULL);
assert(nargs >= 0);
assert(nargs == 0 || args != NULL);
assert(kwargs == NULL || PyDict_Check(kwargs));
if (PyFunction_Check(callable)) {
¦ return _PyFunction_FastCallDict(callable, args, nargs, kwargs);
}
else if (PyCFunction_Check(callable)) {
¦ return _PyCFunction_FastCallDict(callable, args, nargs, kwargs);
}
else {
¦ PyObject *argstuple, *result;
¦ ternaryfunc call;
¦ /* Slow-path: build a temporary tuple */
¦ call = callable->ob_type->tp_call;
¦ if (call == NULL) {
¦ ¦ PyErr_Format(PyExc_TypeError, "'%.200s' object is not callable",
¦ ¦ ¦ ¦ ¦ ¦callable->ob_type->tp_name);
¦ ¦ return NULL;
¦ }
¦ argstuple = _PyStack_AsTuple(args, nargs);
¦ if (argstuple == NULL) {
¦ ¦ return NULL;
¦ }
¦ if (Py_EnterRecursiveCall(" while calling a Python object")) {
¦ ¦ Py_DECREF(argstuple);
¦ ¦ return NULL;
¦ }
¦ result = (*call)(callable, argstuple, kwargs);
¦ Py_LeaveRecursiveCall();
¦ Py_DECREF(argstuple);
¦ result = _Py_CheckFunctionResult(callable, result, NULL);
¦ return result;
}
}
```
# compile and eval
```python
compile?
```
```
Signature: compile(source, filename, mode, flags=0, dont_inherit=False, optimize=-1)
Docstring:
Compile source into a code object that can be executed by exec() or eval().
The source code may represent a Python module, statement or expression.
The filename will be used for run-time error messages.
The mode must be 'exec' to compile a module, 'single' to compile a
single (interactive) statement, or 'eval' to compile an expression.
The flags argument, if present, controls which future statements influence
the compilation of the code.
The dont_inherit argument, if true, stops the compilation inheriting
the effects of any future statements in effect in the code calling
compile; if absent or false these statements do influence the compilation,
in addition to any features explicitly specified.
Type: builtin_function_or_method
```
# bytecode
Python source code is compiled into bytecode, the internal representation of a Python program in the CPython interpreter. The bytecode is also cached in .pyc files so that executing the same file is faster the second time (recompilation from source to bytecode can be avoided). This “intermediate language” is said to run on a virtual machine that executes the machine code corresponding to each bytecode. Do note that bytecodes are not expected to work between different Python virtual machines, nor to be stable between Python releases.
> A list of bytecode instructions can be found in the documentation for the dis module.
```python
help(exec)
```
Help on built-in function exec in module builtins:
exec(source, globals=None, locals=None, /)
Execute the given source in the context of globals and locals.
The source may be a string representing one or more Python statements
or a code object as returned by compile().
The globals must be a dictionary and locals can be any mapping,
defaulting to the current globals and locals.
If only globals is given, locals defaults to it.
```python
help(eval)
```
Help on built-in function eval in module builtins:
eval(source, globals=None, locals=None, /)
Evaluate the given source in the context of globals and locals.
The source may be a string representing a Python expression
or a code object as returned by compile().
The globals must be a dictionary and locals can be any mapping,
defaulting to the current globals and locals.
If only globals is given, locals defaults to it.
所以`eval`執行的內容必須是`expression`,而`exec`是都可以執行,但是回傳值固定為None。
```python
code_str="""
def hello():
print("hello world")
hello()
"""
```
```python
exec('print(5)') # prints 5.
# exec 'print 5' if you use Python 2.x, nor the exec neither the print is a function there
exec('print(5)\nprint(6)') # prints 5{newline}6.
exec('if True: print(6)') # prints 6.
exec('5')
```
5
5
6
6
```python
eval('5') # x <- 5
eval('%d + 6' % x) # x <- 11
eval('abs(%d)' % -100) # x <- 100
eval('x = 5') # INVALID; assignment is not an expression.
eval('if 1: x = 4') # INVALID; if is a statement, not an expression.
```
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-107-40888bad4179> in <module>()
1 eval('5') # x <- 5
----> 2 eval('%d + 6' % x) # x <- 11
3 eval('abs(%d)' % -100) # x <- 100
4 eval('x = 5') # INVALID; assignment is not an expression.
5 eval('if 1: x = 4') # INVALID; if is a statement, not an expression.
TypeError: %d format: a number is required, not generator
```python
code_obj_exec=compile(code_str,'string','exec')
code_obj_exec
exec(code_obj_exec)
```
hello world
```python
eval(compile('42', '<string>', 'exec')) # code return None
```
```python
eval(compile('42', '<string>', 'eval')) # code returns 42
```
42
```python
exec(compile('42', '<string>', 'eval')) # code returns 42,but ignored by exec
```
```python
code_obj_signle=compile(code_str,'string','single')
exec(code_obj_signle)
```
Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/IPython/core/interactiveshell.py", line 2862, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "<ipython-input-112-e920440df200>", line 1, in <module>
code_obj_signle=compile(code_str,'string','single')
File "string", line 4
hello()
^
SyntaxError: invalid syntax
而single 只接受oneline statment
```python
code_obj_eval=compile(code_str,'string','eval')
```
Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/IPython/core/interactiveshell.py", line 2862, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "<ipython-input-113-686a1487d9cc>", line 1, in <module>
code_obj_eval=compile(code_str,'string','eval')
File "string", line 2
def hello():
^
SyntaxError: invalid syntax
```python
exec('print("hello")')
```
hello
```python
eval("print('hello')")
```
hello
```python
exec("3+5")
```
```python
eval("3+5")
```
8
Actually the statement "<span class="girk">eval accepts only a single expression</span>" applies only when a string (which contains Python source code) is passed to eval. Then it is internally compiled to bytecode using compile(source, '<string>', 'eval') This is where the difference really comes from.
If a code object (which contains Python bytecode) is passed to exec or eval, <span class="mark">they behave identically</span>, excepting for the fact that exec ignores the return value, still returning None always. So it is possible use eval to execute something that has statements, if you just compiled it into bytecode before instead of passing it as a string:
```python
eval(compile('if 1: print("Hello")', '<string>', 'exec'))
```
Hello
It should work.
## The longer answer, a.k.a the gory details
* The `exec` function (which was a statement in Python 2) is used for executing a dynamically created statement or program.
* The `eval` function does the same for a single expression, and returns the value of the expression.
exec and eval both accept the program/expression to be run either as a str, unicode or bytes object containing source code, or as a code object which contains Python bytecode.
If a str/unicode/bytes containing source code was passed to exec, it behaves equivalently to:
```
exec(compile(source, '<string>', 'exec'))
```
and eval similarly behaves equivalent to:
```
eval(compile(source, '<string>', 'eval'))
```
```python
def my_func(arg):
print("Called with %d" % arg)
return arg * 2
```
```python
exec('my_func(42)')
```
Called with 42
```python
eval('my_func(42)')
```
Called with 42
84
# 深入code object
## Code objects
Code objects represent byte-compiled executable Python code, or bytecode.
The difference between a code object and a function object is that the function object contains an explicit reference to the function’s globals (the module in which it was defined), while a code object contains no context; also the default argument values are stored in the function object, not in the code object (because they represent values calculated at run-time). Unlike function objects, code objects are immutable and contain no references (directly or indirectly) to mutable objects.
```
code
co_argcount number of arguments (not including keyword only arguments, * or ** args)
co_code string of raw compiled bytecode
co_cellvars tuple of names of cell variables (referenced by containing scopes)
co_consts tuple of constants used in the bytecode
co_filename name of file in which this code object was created
co_firstlineno number of first line in Python source code
co_flags bitmap of CO_* flags, read more here
co_lnotab encoded mapping of line numbers to bytecode indices
co_freevars tuple of names of free variables (referenced via a function’s closure)
co_kwonlyargcount number of keyword only arguments (not including ** arg)
co_name name with which this code object was defined
co_names tuple of names of local variables
co_nlocals number of local variables
co_stacksize virtual machine stack space required
co_varnames tuple of names of arguments and local variables
```
可以參考python的[inspect lib](https://docs.python.org/3/library/inspect.html)
## code object implement
inside `include/code.h`
```c
/* Bytecode object */
typedef struct {
PyObject_HEAD int co_argcount; /* #arguments, except *args */
int co_kwonlyargcount; /* #keyword only arguments */
int co_nlocals; /* #local variables */
int co_stacksize; /* #entries needed for evaluation stack */
int co_flags; /* CO_..., see below */
int co_firstlineno; /* first source line number */
PyObject* co_code; /* instruction opcodes */
PyObject* co_consts; /* list (constants used) */
PyObject* co_names; /* list of strings (names used) */
PyObject* co_varnames; /* tuple of strings (local variable names) */
PyObject* co_freevars; /* tuple of strings (free variable names) */
PyObject* co_cellvars; /* tuple of strings (cell variable names) */
/* The rest aren't used in either hash or comparisons, except for co_name,
used in both. This is done to preserve the name and line number
for tracebacks and debuggers; otherwise, constant de-duplication
would collapse identical functions/lambdas defined on different lines.
*/
Py_ssize_t* co_cell2arg; /* Maps cell vars which are arguments. */
PyObject* co_filename; /* unicode (where it was loaded from) */
PyObject* co_name; /* unicode (name, for reference) */
PyObject* co_lnotab; /* string (encoding addr<->lineno mapping) See
Objects/lnotab_notes.txt for details. */
void* co_zombieframe; /* for optimization only (see frameobject.c) */
PyObject* co_weakreflist; /* to support weakrefs to code objects */
/* Scratch space for extra data relating to the code object.
Type is a void* to keep the format private in codeobject.c to force
people to go through the proper APIs. */
void* co_extra;
} PyCodeObject;
```
## more about code object
### inside the code object
```python
code_obj=compile(code_str,'string','exec')
dir(code_obj)
```
['__class__',
'__delattr__',
'__dir__',
'__doc__',
'__eq__',
'__format__',
'__ge__',
'__getattribute__',
'__gt__',
'__hash__',
'__init__',
'__le__',
'__lt__',
'__ne__',
'__new__',
'__reduce__',
'__reduce_ex__',
'__repr__',
'__setattr__',
'__sizeof__',
'__str__',
'__subclasshook__',
'co_argcount',
'co_cellvars',
'co_code',
'co_consts',
'co_filename',
'co_firstlineno',
'co_flags',
'co_freevars',
'co_kwonlyargcount',
'co_lnotab',
'co_name',
'co_names',
'co_nlocals',
'co_stacksize',
'co_varnames']
### get code object code
```python
code_obj.co_code
```
b'd\x00\x00d\x01\x00\x84\x00\x00Z\x00\x00e\x00\x00\x83\x00\x00\x01d\x02\x00S'
```python
[x for x in code_obj.co_code]
```
[100,
0,
0,
100,
1,
0,
132,
0,
0,
90,
0,
0,
101,
0,
0,
131,
0,
0,
1,
100,
2,
0,
83]
```python
dis.show_code(code_obj)
```
Name: <module>
Filename: string
Argument count: 0
Kw-only arguments: 0
Number of locals: 0
Stack size: 2
Flags: NOFREE
Constants:
0: <code object hello at 0x7f01fc67f030, file "string", line 2>
1: 'hello'
2: None
Names:
0: hello
```python
dis.dis(code_obj)
```
2 0 LOAD_CONST 0 (<code object hello at 0x7f01fc67f030, file "string", line 2>)
3 LOAD_CONST 1 ('hello')
6 MAKE_FUNCTION 0
9 STORE_NAME 0 (hello)
4 12 LOAD_NAME 0 (hello)
15 CALL_FUNCTION 0 (0 positional, 0 keyword pair)
18 POP_TOP
19 LOAD_CONST 2 (None)
22 RETURN_VALUE
和上面的code object做對照
```python
iterator=dis.get_instructions(code_str)
for it in iterator:
print(it.opcode)
```
100
100
132
90
101
131
1
100
83
```python
op=code_obj.co_code[0]
op
```
100
```python
opcode.opname[op]
```
'LOAD_CONST'
可以發現opcode是python的一個module，他負責定義python interpreter會用到的instruction，所以我們才可以透過dis來disassembly python bytecode,而code object是bytecode的表現形式。
python中opcode的value和python virtual machine所定義的instruction相同(定義於opcode.h中，和dis.opmap的value互相對應)，因此可以透過實做interpreter(不管任何形式)產生的python bytecode來讓python virtual machine執行。
```python
code_obj.co_consts
```
(<code object hello at 0x7f3768227d20, file "string", line 2>, 'hello', None)
```python
code_obj.co_lnotab
```
b'\x0c\x03'
```python
code_obj.co_freevars
```
()
```python
code_obj.co_stacksize
```
3
# A Detour into Code Disassembly
```python
import dis
def hello():
print("Hello world")
```
```python
code_str="""
def print_hello(x,y):
print("hello")
return x+y
hello(10,11)
"""
```
```python
code_obj=compile(code_str,'string','exec')
dis.dis(code_obj)
```
2 0 LOAD_CONST 0 (<code object print_hello at 0x7f01fc6fde40, file "string", line 2>)
3 LOAD_CONST 1 ('print_hello')
6 MAKE_FUNCTION 0
9 STORE_NAME 0 (print_hello)
5 12 LOAD_NAME 1 (hello)
15 LOAD_CONST 2 (10)
18 LOAD_CONST 3 (11)
21 CALL_FUNCTION 2 (2 positional, 0 keyword pair)
24 POP_TOP
25 LOAD_CONST 4 (None)
28 RETURN_VALUE
|line number|bytecode offect|bytecode|bytecode argument|
|:---------:|:-------------:|:------:|:---------------:|
|5|12|LOAD_NAME|0(hello)
```python
bytecode=dis.Bytecode(hello)
bytecode
```
Bytecode(<function hello at 0x7f01fc74fd90>)
```python
help(bytecode)
```
Help on Bytecode in module dis object:
class Bytecode(builtins.object)
| The bytecode operations of a piece of code
|
| Instantiate this with a function, method, string of code, or a code object
| (as returned by compile()).
|
| Iterating over this yields the bytecode operations as Instruction instances.
|
| Methods defined here:
|
| __init__(self, x, *, first_line=None, current_offset=None)
| Initialize self. See help(type(self)) for accurate signature.
|
| __iter__(self)
|
| __repr__(self)
| Return repr(self).
|
| dis(self)
| Return a formatted view of the bytecode operations.
|
| info(self)
| Return formatted information about the code object.
|
| ----------------------------------------------------------------------
| Class methods defined here:
|
| from_traceback(tb) from builtins.type
| Construct a Bytecode from the given traceback
|
| ----------------------------------------------------------------------
| Data descriptors defined here:
|
| __dict__
| dictionary for instance variables (if defined)
|
| __weakref__
| list of weak references to the object (if defined)
```python
bytecode.dis()
```
" 3 0 LOAD_GLOBAL 0 (print)\n 3 LOAD_CONST 1 ('Hello world')\n 6 CALL_FUNCTION 1 (1 positional, 0 keyword pair)\n 9 POP_TOP\n 10 LOAD_CONST 0 (None)\n 13 RETURN_VALUE\n"
```python
bytecode.info()
```
"Name: hello\nFilename: <ipython-input-70-9b74c1c13393>\nArgument count: 0\nKw-only arguments: 0\nNumber of locals: 0\nStack size: 2\nFlags: OPTIMIZED, NEWLOCALS, NOFREE\nConstants:\n 0: None\n 1: 'Hello world'\nNames:\n 0: print"
```python
for instr in bytecode:
print(instr.opname)
```
LOAD_GLOBAL
LOAD_CONST
CALL_FUNCTION
POP_TOP
LOAD_CONST
RETURN_VALUE
```python
dis.dis(hello)
```
3 0 LOAD_GLOBAL 0 (print)
3 LOAD_CONST 1 ('Hello world')
6 CALL_FUNCTION 1 (1 positional, 0 keyword pair)
9 POP_TOP
10 LOAD_CONST 0 (None)
13 RETURN_VALUE
```python
import opcode
help(opcode)
```
Help on module opcode:
NAME
opcode
MODULE REFERENCE
https://docs.python.org/3.5/library/opcode
The following documentation is automatically generated from the Python
source files. It may be incomplete, incorrect or include features that
are considered implementation detail and may vary between Python
implementations. When in doubt, consult the module reference at the
location listed above.
DESCRIPTION
opcode module - potentially shared between dis and other modules which
operate on bytecodes (e.g. peephole optimizers).
FUNCTIONS
stack_effect(opcode, oparg=None, /)
Compute the stack effect of the opcode.
DATA
EXTENDED_ARG = 144
HAVE_ARGUMENT = 90
__all__ = ['cmp_op', 'hasconst', 'hasname', 'hasjrel', 'hasjabs', 'has...
cmp_op = ('<', '<=', '==', '!=', '>', '>=', 'in', 'not in', 'is', 'is ...
hascompare = [107]
hasconst = [100]
hasfree = [135, 136, 137, 138, 148]
hasjabs = [111, 112, 113, 114, 115, 119]
hasjrel = [93, 110, 120, 121, 122, 143, 154]
haslocal = [124, 125, 126]
hasname = [90, 91, 95, 96, 97, 98, 101, 106, 108, 109, 116]
hasnargs = [131, 140, 141, 142]
opmap = {'BEFORE_ASYNC_WITH': 52, 'BINARY_ADD': 23, 'BINARY_AND': 64, ...
opname = ['<0>', 'POP_TOP', 'ROT_TWO', 'ROT_THREE', 'DUP_TOP', 'DUP_TO...
FILE
/usr/lib/python3.5/opcode.py
```python
dis.code_info(hello)
```
"Name: hello\nFilename: <ipython-input-70-9b74c1c13393>\nArgument count: 0\nKw-only arguments: 0\nNumber of locals: 0\nStack size: 2\nFlags: OPTIMIZED, NEWLOCALS, NOFREE\nConstants:\n 0: None\n 1: 'Hello world'\nNames:\n 0: print"
```python
dis.show_code(hello)
```
Name: hello
Filename: <ipython-input-70-9b74c1c13393>
Argument count: 0
Kw-only arguments: 0
Number of locals: 0
Stack size: 2
Flags: OPTIMIZED, NEWLOCALS, NOFREE
Constants:
0: None
1: 'Hello world'
Names:
0: print
```python
help(dis.distb)
```
Help on function distb in module dis:
distb(tb=None, *, file=None)
Disassemble a traceback (default: last traceback).
```python
dis.disassemble(code_obj)
```
2 0 LOAD_CONST 0 (<code object print_hello at 0x7f01fc6fde40, file "string", line 2>)
3 LOAD_CONST 1 ('print_hello')
6 MAKE_FUNCTION 0
9 STORE_NAME 0 (print_hello)
5 12 LOAD_NAME 1 (hello)
15 LOAD_CONST 2 (10)
18 LOAD_CONST 3 (11)
21 CALL_FUNCTION 2 (2 positional, 0 keyword pair)
24 POP_TOP
25 LOAD_CONST 4 (None)
28 RETURN_VALUE
```python
dis.disco(code_obj)
```
2 0 LOAD_CONST 0 (<code object print_hello at 0x7f01fc6fde40, file "string", line 2>)
3 LOAD_CONST 1 ('print_hello')
6 MAKE_FUNCTION 0
9 STORE_NAME 0 (print_hello)
5 12 LOAD_NAME 1 (hello)
15 LOAD_CONST 2 (10)
18 LOAD_CONST 3 (11)
21 CALL_FUNCTION 2 (2 positional, 0 keyword pair)
24 POP_TOP
25 LOAD_CONST 4 (None)
28 RETURN_VALUE
這兩個function的作用是一樣的
```python
iterator=dis.get_instructions(code_str)
for it in iterator:
print(it)
```
Instruction(opname='LOAD_CONST', opcode=100, arg=0, argval=<code object print_hello at 0x7f01fc6f5030, file "<disassembly>", line 2>, argrepr='<code object print_hello at 0x7f01fc6f5030, file "<disassembly>", line 2>', offset=0, starts_line=2, is_jump_target=False)
Instruction(opname='LOAD_CONST', opcode=100, arg=1, argval='print_hello', argrepr="'print_hello'", offset=3, starts_line=None, is_jump_target=False)
Instruction(opname='MAKE_FUNCTION', opcode=132, arg=0, argval=0, argrepr='', offset=6, starts_line=None, is_jump_target=False)
Instruction(opname='STORE_NAME', opcode=90, arg=0, argval='print_hello', argrepr='print_hello', offset=9, starts_line=None, is_jump_target=False)
Instruction(opname='LOAD_NAME', opcode=101, arg=1, argval='hello', argrepr='hello', offset=12, starts_line=5, is_jump_target=False)
Instruction(opname='LOAD_CONST', opcode=100, arg=2, argval=10, argrepr='10', offset=15, starts_line=None, is_jump_target=False)
Instruction(opname='LOAD_CONST', opcode=100, arg=3, argval=11, argrepr='11', offset=18, starts_line=None, is_jump_target=False)
Instruction(opname='CALL_FUNCTION', opcode=131, arg=2, argval=2, argrepr='2 positional, 0 keyword pair', offset=21, starts_line=None, is_jump_target=False)
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=24, starts_line=None, is_jump_target=False)
Instruction(opname='LOAD_CONST', opcode=100, arg=4, argval=None, argrepr='None', offset=25, starts_line=None, is_jump_target=False)
Instruction(opname='RETURN_VALUE', opcode=83, arg=None, argval=None, argrepr='', offset=28, starts_line=None, is_jump_target=False)
```
opname - human readable name for operation
opcode - numeric code for operation
arg - numeric argument to operation (if any), otherwise None
argval - resolved arg value (if known), otherwise same as arg
argrepr - human readable description of operation argument
offset - start index of operation within bytecode sequence
starts_line - line started by this opcode (if any), otherwise None
is_jump_target - True if other code jumps to here, otherwise False
```
```python
x=dis.findlinestarts(code_obj)
next(x)
```
(0, 2)
```python
next(x)
```
(12, 5)
```python
help(dis.stack_effect)
```
Help on built-in function stack_effect in module _opcode:
stack_effect(opcode, oparg=None, /)
Compute the stack effect of the opcode.
```python
dis.stack_effect(1) # POP_TOP
```
-1
```python
help(dis.findlabels)
```
Help on function findlabels in module dis:
findlabels(code)
Detect all offsets in a byte code which are jump targets.
Return the list of offsets.
```python
def f(x):
if x == 0:
return 1
elif x==1:
return 2
```
```python
dis.disco(f.__code__)
```
2 0 LOAD_FAST 0 (x)
3 LOAD_CONST 1 (0)
6 COMPARE_OP 2 (==)
9 POP_JUMP_IF_FALSE 16
3 12 LOAD_CONST 2 (1)
15 RETURN_VALUE
4 >> 16 LOAD_FAST 0 (x)
19 LOAD_CONST 2 (1)
22 COMPARE_OP 2 (==)
25 POP_JUMP_IF_FALSE 32
5 28 LOAD_CONST 3 (2)
31 RETURN_VALUE
>> 32 LOAD_CONST 0 (None)
35 RETURN_VALUE
```python
dis.findlabels(f.__code__.co_code)
```
[16, 32]
```python
dis.opname
```
['<0>',
'POP_TOP',
'ROT_TWO',
'ROT_THREE',
'DUP_TOP',
'DUP_TOP_TWO',
'<6>',
'<7>',
'<8>',
'NOP',
'UNARY_POSITIVE',
'UNARY_NEGATIVE',
'UNARY_NOT',
'<13>',
'<14>',
'UNARY_INVERT',
'BINARY_MATRIX_MULTIPLY',
'INPLACE_MATRIX_MULTIPLY',
'<18>',
'BINARY_POWER',
'BINARY_MULTIPLY',
'<21>',
'BINARY_MODULO',
'BINARY_ADD',
'BINARY_SUBTRACT',
'BINARY_SUBSCR',
'BINARY_FLOOR_DIVIDE',
'BINARY_TRUE_DIVIDE',
'INPLACE_FLOOR_DIVIDE',
'INPLACE_TRUE_DIVIDE',
'<30>',
'<31>',
'<32>',
'<33>',
'<34>',
'<35>',
'<36>',
'<37>',
'<38>',
'<39>',
'<40>',
'<41>',
'<42>',
'<43>',
'<44>',
'<45>',
'<46>',
'<47>',
'<48>',
'<49>',
'GET_AITER',
'GET_ANEXT',
'BEFORE_ASYNC_WITH',
'<53>',
'<54>',
'INPLACE_ADD',
'INPLACE_SUBTRACT',
'INPLACE_MULTIPLY',
'<58>',
'INPLACE_MODULO',
'STORE_SUBSCR',
'DELETE_SUBSCR',
'BINARY_LSHIFT',
'BINARY_RSHIFT',
'BINARY_AND',
'BINARY_XOR',
'BINARY_OR',
'INPLACE_POWER',
'GET_ITER',
'GET_YIELD_FROM_ITER',
'PRINT_EXPR',
'LOAD_BUILD_CLASS',
'YIELD_FROM',
'GET_AWAITABLE',
'<74>',
'INPLACE_LSHIFT',
'INPLACE_RSHIFT',
'INPLACE_AND',
'INPLACE_XOR',
'INPLACE_OR',
'BREAK_LOOP',
'WITH_CLEANUP_START',
'WITH_CLEANUP_FINISH',
'RETURN_VALUE',
'IMPORT_STAR',
'<85>',
'YIELD_VALUE',
'POP_BLOCK',
'END_FINALLY',
'POP_EXCEPT',
'STORE_NAME',
'DELETE_NAME',
'UNPACK_SEQUENCE',
'FOR_ITER',
'UNPACK_EX',
'STORE_ATTR',
'DELETE_ATTR',
'STORE_GLOBAL',
'DELETE_GLOBAL',
'<99>',
'LOAD_CONST',
'LOAD_NAME',
'BUILD_TUPLE',
'BUILD_LIST',
'BUILD_SET',
'BUILD_MAP',
'LOAD_ATTR',
'COMPARE_OP',
'IMPORT_NAME',
'IMPORT_FROM',
'JUMP_FORWARD',
'JUMP_IF_FALSE_OR_POP',
'JUMP_IF_TRUE_OR_POP',
'JUMP_ABSOLUTE',
'POP_JUMP_IF_FALSE',
'POP_JUMP_IF_TRUE',
'LOAD_GLOBAL',
'<117>',
'<118>',
'CONTINUE_LOOP',
'SETUP_LOOP',
'SETUP_EXCEPT',
'SETUP_FINALLY',
'<123>',
'LOAD_FAST',
'STORE_FAST',
'DELETE_FAST',
'<127>',
'<128>',
'<129>',
'RAISE_VARARGS',
'CALL_FUNCTION',
'MAKE_FUNCTION',
'BUILD_SLICE',
'MAKE_CLOSURE',
'LOAD_CLOSURE',
'LOAD_DEREF',
'STORE_DEREF',
'DELETE_DEREF',
'<139>',
'CALL_FUNCTION_VAR',
'CALL_FUNCTION_KW',
'CALL_FUNCTION_VAR_KW',
'SETUP_WITH',
'EXTENDED_ARG',
'LIST_APPEND',
'SET_ADD',
'MAP_ADD',
'LOAD_CLASSDEREF',
'BUILD_LIST_UNPACK',
'BUILD_MAP_UNPACK',
'BUILD_MAP_UNPACK_WITH_CALL',
'BUILD_TUPLE_UNPACK',
'BUILD_SET_UNPACK',
'SETUP_ASYNC_WITH',
'<155>',
'<156>',
'<157>',
'<158>',
'<159>',
'<160>',
'<161>',
'<162>',
'<163>',
'<164>',
'<165>',
'<166>',
'<167>',
'<168>',
'<169>',
'<170>',
'<171>',
'<172>',
'<173>',
'<174>',
'<175>',
'<176>',
'<177>',
'<178>',
'<179>',
'<180>',
'<181>',
'<182>',
'<183>',
'<184>',
'<185>',
'<186>',
'<187>',
'<188>',
'<189>',
'<190>',
'<191>',
'<192>',
'<193>',
'<194>',
'<195>',
'<196>',
'<197>',
'<198>',
'<199>',
'<200>',
'<201>',
'<202>',
'<203>',
'<204>',
'<205>',
'<206>',
'<207>',
'<208>',
'<209>',
'<210>',
'<211>',
'<212>',
'<213>',
'<214>',
'<215>',
'<216>',
'<217>',
'<218>',
'<219>',
'<220>',
'<221>',
'<222>',
'<223>',
'<224>',
'<225>',
'<226>',
'<227>',
'<228>',
'<229>',
'<230>',
'<231>',
'<232>',
'<233>',
'<234>',
'<235>',
'<236>',
'<237>',
'<238>',
'<239>',
'<240>',
'<241>',
'<242>',
'<243>',
'<244>',
'<245>',
'<246>',
'<247>',
'<248>',
'<249>',
'<250>',
'<251>',
'<252>',
'<253>',
'<254>',
'<255>']
```python
sorted(dis.opmap.items(),key=lambda x:x[1])
```
[('POP_TOP', 1),
('ROT_TWO', 2),
('ROT_THREE', 3),
('DUP_TOP', 4),
('DUP_TOP_TWO', 5),
('NOP', 9),
('UNARY_POSITIVE', 10),
('UNARY_NEGATIVE', 11),
('UNARY_NOT', 12),
('UNARY_INVERT', 15),
('BINARY_MATRIX_MULTIPLY', 16),
('INPLACE_MATRIX_MULTIPLY', 17),
('BINARY_POWER', 19),
('BINARY_MULTIPLY', 20),
('BINARY_MODULO', 22),
('BINARY_ADD', 23),
('BINARY_SUBTRACT', 24),
('BINARY_SUBSCR', 25),
('BINARY_FLOOR_DIVIDE', 26),
('BINARY_TRUE_DIVIDE', 27),
('INPLACE_FLOOR_DIVIDE', 28),
('INPLACE_TRUE_DIVIDE', 29),
('GET_AITER', 50),
('GET_ANEXT', 51),
('BEFORE_ASYNC_WITH', 52),
('INPLACE_ADD', 55),
('INPLACE_SUBTRACT', 56),
('INPLACE_MULTIPLY', 57),
('INPLACE_MODULO', 59),
('STORE_SUBSCR', 60),
('DELETE_SUBSCR', 61),
('BINARY_LSHIFT', 62),
('BINARY_RSHIFT', 63),
('BINARY_AND', 64),
('BINARY_XOR', 65),
('BINARY_OR', 66),
('INPLACE_POWER', 67),
('GET_ITER', 68),
('GET_YIELD_FROM_ITER', 69),
('PRINT_EXPR', 70),
('LOAD_BUILD_CLASS', 71),
('YIELD_FROM', 72),
('GET_AWAITABLE', 73),
('INPLACE_LSHIFT', 75),
('INPLACE_RSHIFT', 76),
('INPLACE_AND', 77),
('INPLACE_XOR', 78),
('INPLACE_OR', 79),
('BREAK_LOOP', 80),
('WITH_CLEANUP_START', 81),
('WITH_CLEANUP_FINISH', 82),
('RETURN_VALUE', 83),
('IMPORT_STAR', 84),
('YIELD_VALUE', 86),
('POP_BLOCK', 87),
('END_FINALLY', 88),
('POP_EXCEPT', 89),
('STORE_NAME', 90),
('DELETE_NAME', 91),
('UNPACK_SEQUENCE', 92),
('FOR_ITER', 93),
('UNPACK_EX', 94),
('STORE_ATTR', 95),
('DELETE_ATTR', 96),
('STORE_GLOBAL', 97),
('DELETE_GLOBAL', 98),
('LOAD_CONST', 100),
('LOAD_NAME', 101),
('BUILD_TUPLE', 102),
('BUILD_LIST', 103),
('BUILD_SET', 104),
('BUILD_MAP', 105),
('LOAD_ATTR', 106),
('COMPARE_OP', 107),
('IMPORT_NAME', 108),
('IMPORT_FROM', 109),
('JUMP_FORWARD', 110),
('JUMP_IF_FALSE_OR_POP', 111),
('JUMP_IF_TRUE_OR_POP', 112),
('JUMP_ABSOLUTE', 113),
('POP_JUMP_IF_FALSE', 114),
('POP_JUMP_IF_TRUE', 115),
('LOAD_GLOBAL', 116),
('CONTINUE_LOOP', 119),
('SETUP_LOOP', 120),
('SETUP_EXCEPT', 121),
('SETUP_FINALLY', 122),
('LOAD_FAST', 124),
('STORE_FAST', 125),
('DELETE_FAST', 126),
('RAISE_VARARGS', 130),
('CALL_FUNCTION', 131),
('MAKE_FUNCTION', 132),
('BUILD_SLICE', 133),
('MAKE_CLOSURE', 134),
('LOAD_CLOSURE', 135),
('LOAD_DEREF', 136),
('STORE_DEREF', 137),
('DELETE_DEREF', 138),
('CALL_FUNCTION_VAR', 140),
('CALL_FUNCTION_KW', 141),
('CALL_FUNCTION_VAR_KW', 142),
('SETUP_WITH', 143),
('EXTENDED_ARG', 144),
('LIST_APPEND', 145),
('SET_ADD', 146),
('MAP_ADD', 147),
('LOAD_CLASSDEREF', 148),
('BUILD_LIST_UNPACK', 149),
('BUILD_MAP_UNPACK', 150),
('BUILD_MAP_UNPACK_WITH_CALL', 151),
('BUILD_TUPLE_UNPACK', 152),
('BUILD_SET_UNPACK', 153),
('SETUP_ASYNC_WITH', 154)]
```python
len(dis.opmap.items()) # numbers of the python instructions
```
114
而這些bytecode被定義在opcode module中
```python
opcode??
```
```python
dis.cmp_op
```
('<',
'<=',
'==',
'!=',
'>',
'>=',
'in',
'not in',
'is',
'is not',
'exception match',
'BAD')
```python
dis.hasconst
```
[100]
```python
dis.hasfree
```
[135, 136, 137, 138, 148]
```python
dis.hasname
```
[90, 91, 95, 96, 97, 98, 101, 106, 108, 109, 116]
```python
dis.hasjrel
```
[93, 110, 120, 121, 122, 143, 154]
```python
dis.hasjabs
```
[111, 112, 113, 114, 115, 119]
```python
dis.haslocal
```
[124, 125, 126]
```python
dis.hascompare
```
[107]
## Ref
Python dis module
<https://docs.python.org/3.7/library/dis.html?module-dis>
stack overflow 對於比較exec和eval的回答
<https://stackoverflow.com/questions/2220699/whats-the-difference-between-eval-exec-and-compile-in-python>
# Python virtual machine implementation
`Python/ceval.c`
```c
PyObject* _Py_HOT_FUNCTION
_PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag)
```
python interpreter main的進入點
```
Py_Main(int arg,wchar_t argv**)
```
python interpreter initiali
```
_Py_InitializeCore
```
PyFrameObject
`Include/frameobject.h`
```c
typedef struct _frame {
PyObject_VAR_HEAD
struct _frame *f_back; /* previous frame, or NULL */
PyCodeObject *f_code; /* code segment */
PyObject *f_builtins; /* builtin symbol table (PyDictObject) */
PyObject *f_globals; /* global symbol table (PyDictObject) */
PyObject *f_locals; /* local symbol table (any mapping) */
PyObject **f_valuestack; /* points after the last local */
/* Next free slot in f_valuestack. Frame creation sets to f_valuestack.
Frame evaluation usually NULLs it, but a frame that yields sets it
to the current stack top. */
PyObject **f_stacktop;
PyObject *f_trace; /* Trace function */
/* In a generator, we need to be able to swap between the exception
state inside the generator and the exception state of the calling
frame (which shouldn't be impacted when the generator "yields"
from an except handler).
These three fields exist exactly for that, and are unused for
non-generator frames. See the save_exc_state and swap_exc_state
functions in ceval.c for details of their use. */
PyObject *f_exc_type, *f_exc_value, *f_exc_traceback;
/* Borrowed reference to a generator, or NULL */
PyObject *f_gen;
int f_lasti; /* Last instruction if called */
/* Call PyFrame_GetLineNumber() instead of reading this field
directly. As of 2.3 f_lineno is only valid when tracing is
active (i.e. when f_trace is set). At other times we use
PyCode_Addr2Line to calculate the line from the current
bytecode index. */
int f_lineno; /* Current line number */
int f_iblock; /* index in f_blockstack */
char f_executing; /* whether the frame is still executing */
PyTryBlock f_blockstack[CO_MAXBLOCKS]; /* for try and loop blocks */
PyObject *f_localsplus[1]; /* locals+stack, dynamically sized */
} PyFrameObject;
```
## python object
```c
/* Define pointers to support a doubly-linked list of all live heap objects. */
#define _PyObject_HEAD_EXTRA \
struct _object *_ob_next; \
struct _object *_ob_prev;
#define _PyObject_EXTRA_INIT 0, 0,
#else
#define _PyObject_HEAD_EXTRA
#define _PyObject_EXTRA_INIT
#endif
/* PyObject_HEAD defines the initial segment of every PyObject. */
#define PyObject_HEAD PyObject ob_base;
#define PyObject_HEAD_INIT(type) \
{ _PyObject_EXTRA_INIT \
1, type },
#define PyVarObject_HEAD_INIT(type, size) \
{ PyObject_HEAD_INIT(type) size },
/* PyObject_VAR_HEAD defines the initial segment of all variable-size
* container objects. These end with a declaration of an array with 1
* element, but enough space is malloc'ed so that the array actually
* has room for ob_size elements. Note that ob_size is an element count,
* not necessarily a byte count.
*/
#define PyObject_VAR_HEAD PyVarObject ob_base;
#define Py_INVALID_SIZE (Py_ssize_t)-1
/* Nothing is actually declared to be a PyObject, but every pointer to
* a Python object can be cast to a PyObject*. This is inheritance built
* by hand. Similarly every pointer to a variable-size Python object can,
* in addition, be cast to PyVarObject*.
*/
typedef struct _object {
_PyObject_HEAD_EXTRA
Py_ssize_t ob_refcnt;
struct _typeobject *ob_type;
} PyObject;
typedef struct {
PyObject ob_base;
Py_ssize_t ob_size; /* Number of items in variable part */
} PyVarObject;
#define Py_REFCNT(ob) (((PyObject*)(ob))->ob_refcnt)
#define Py_TYPE(ob) (((PyObject*)(ob))->ob_type)
#define Py_SIZE(ob) (((PyVarObject*)(ob))->ob_size)
typedef void (*freefunc)(void *);
typedef void (*destructor)(PyObject *);
#ifndef Py_LIMITED_API
/* We can't provide a full compile-time check that limited-API
users won't implement tp_print. However, not defining printfunc
and making tp_print of a different function pointer type
should at least cause a warning in most cases. */
typedef int (*printfunc)(PyObject *, FILE *, int);
#endif
typedef PyObject *(*getattrfunc)(PyObject *, char *);
typedef PyObject *(*getattrofunc)(PyObject *, PyObject *);
typedef int (*setattrfunc)(PyObject *, char *, PyObject *);
typedef int (*setattrofunc)(PyObject *, PyObject *, PyObject *);
typedef PyObject *(*reprfunc)(PyObject *);
typedef Py_hash_t (*hashfunc)(PyObject *);
typedef PyObject *(*richcmpfunc) (PyObject *, PyObject *, int);
typedef PyObject *(*getiterfunc) (PyObject *);
typedef PyObject *(*iternextfunc) (PyObject *);
typedef PyObject *(*descrgetfunc) (PyObject *, PyObject *, PyObject *);
typedef int (*descrsetfunc) (PyObject *, PyObject *, PyObject *);
typedef int (*initproc)(PyObject *, PyObject *, PyObject *);
typedef PyObject *(*newfunc)(struct _typeobject *, PyObject *, PyObject *);
typedef PyObject *(*allocfunc)(struct _typeobject *, Py_ssize_t);
#ifdef Py_LIMITED_API
typedef struct _typeobject PyTypeObject; /* opaque */
#else
typedef struct _typeobject {
PyObject_VAR_HEAD
const char *tp_name; /* For printing, in format "<module>.<name>" */
Py_ssize_t tp_basicsize, tp_itemsize; /* For allocation */
/* Methods to implement standard operations */
destructor tp_dealloc;
printfunc tp_print;
getattrfunc tp_getattr;
setattrfunc tp_setattr;
PyAsyncMethods *tp_as_async; /* formerly known as tp_compare (Python 2)
or tp_reserved (Python 3) */
reprfunc tp_repr;
/* Method suites for standard classes */
PyNumberMethods *tp_as_number;
PySequenceMethods *tp_as_sequence;
PyMappingMethods *tp_as_mapping;
/* More standard operations (here for binary compatibility) */
hashfunc tp_hash;
ternaryfunc tp_call;
reprfunc tp_str;
getattrofunc tp_getattro;
setattrofunc tp_setattro;
/* Functions to access object as input/output buffer */
PyBufferProcs *tp_as_buffer;
/* Flags to define presence of optional/expanded features */
unsigned long tp_flags;
const char *tp_doc; /* Documentation string */
/* Assigned meaning in release 2.0 */
/* call function for all accessible objects */
traverseproc tp_traverse;
/* delete references to contained objects */
inquiry tp_clear;
/* Assigned meaning in release 2.1 */
/* rich comparisons */
richcmpfunc tp_richcompare;
/* weak reference enabler */
Py_ssize_t tp_weaklistoffset;
/* Iterators */
getiterfunc tp_iter;
iternextfunc tp_iternext;
/* Attribute descriptor and subclassing stuff */
struct PyMethodDef *tp_methods;
struct PyMemberDef *tp_members;
struct PyGetSetDef *tp_getset;
struct _typeobject *tp_base;
PyObject *tp_dict;
descrgetfunc tp_descr_get;
descrsetfunc tp_descr_set;
Py_ssize_t tp_dictoffset;
initproc tp_init;
allocfunc tp_alloc;
newfunc tp_new;
freefunc tp_free; /* Low-level free-memory routine */
inquiry tp_is_gc; /* For PyObject_IS_GC */
PyObject *tp_bases;
PyObject *tp_mro; /* method resolution order */
PyObject *tp_cache;
PyObject *tp_subclasses;
PyObject *tp_weaklist;
destructor tp_del;
/* Type attribute cache version tag. Added in version 2.6 */
unsigned int tp_version_tag;
destructor tp_finalize;
#ifdef COUNT_ALLOCS
/* these must be last and never explicitly initialized */
Py_ssize_t tp_allocs;
Py_ssize_t tp_frees;
Py_ssize_t tp_maxalloc;
struct _typeobject *tp_prev;
struct _typeobject *tp_next;
#endif
} PyTypeObject;
#endif
```
# Design of CPython’s Compiler
`2017-08-25 12:30:51`
In CPython, the compilation from source code to bytecode involves several steps:
* Parse source code into a parse tree (Parser/pgen.c)
* Transform parse tree into an Abstract Syntax Tree (Python/ast.c)
* Transform AST into a Control Flow Graph (Python/compile.c)
* Emit bytecode based on the Control Flow Graph (Python/compile.c)
## Important Files
* Parser/
* Python.asdl
ASDL syntax file
* asdl.py
Parser for ASDL definition files. Reads in an ASDL description and parses it into an AST that describes it.
* asdl_c.py
“Generate C code from an ASDL description.” Generates Python/Python-ast.c and Include/Python-ast.h.
* Python/
* Python-ast.c
Creates C structs corresponding to the ASDL types. Also contains code for marshalling AST nodes (core ASDL types have marshalling code in asdl.c). “File automatically generated by Parser/asdl_c.py”. This file must be committed separately after every grammar change is committed since the __version__ value is set to the latest grammar change revision number.
* asdl.c
Contains code to handle the ASDL sequence type. Also has code to handle marshalling the core ASDL types, such as number and identifier. Used by Python-ast.c for marshalling AST nodes.
* ast.c
Converts Python’s parse tree into the abstract syntax tree.
* ceval.c
Executes byte code (aka, eval loop).
* compile.c
Emits bytecode based on the AST.
* symtable.c
Generates a symbol table from AST.
* pyarena.c
Implementation of the arena memory manager.
* import.c
Home of the magic number (named MAGIC) for bytecode versioning
* Include/
* Python-ast.h
Contains the actual definitions of the C structs as generated by Python/Python-ast.c. “Automatically generated by Parser/asdl_c.py”.
* asdl.h
Header for the corresponding Python/ast.c.
* ast.h
Declares PyAST_FromNode() external (from Python/ast.c).
* code.h
Header file for Objects/codeobject.c; contains definition of PyCodeObject.
* symtable.h
Header for Python/symtable.c. struct symtable and PySTEntryObject are defined here.
* pyarena.h
Header file for the corresponding Python/pyarena.c.
* opcode.h
Master list of bytecode; if this file is modified you must modify several other files accordingly (see “Introducing New Bytecode”)
* Objects/
* codeobject.c
Contains PyCodeObject-related code (originally in Python/compile.c).
* Lib/
* opcode.py
One of the files that must be modified if Include/opcode.h is.
## Ref
https://docs.python.org/devguide/compiler.html
http://blog.csdn.net/zhsenl/article/category/2307109/1
https://leanpub.com/insidethepythonvirtualmachine/read#leanpub-auto-the-block-stack
Sign in with Wallet
Connect another wallet