CPython and Frozen modules¶
Frozen Modules¶
It is common for Firmwares to include a few (or many) python modules as ‘frozen’ modules. ‘Freezing’ modules is a way to pre-process .py modules so they’re ‘baked-in’ to MicroPython’ s firmware and use less memory. Once the code is frozen it can be quickly loaded and interpreted by MicroPython without as much memory and processing time.
Most OSS firmwares store these frozen modules as part of their repository, which allows us to:
Download the *.py from the (github) repo using
git clone
or a direct downloadExtract and store the ‘unfrozen’ modules (ie the *.py files) in a
_Frozen folder. if there are different port / boards or releases defined , there may be multiple folders such as: stubs/micropython_1_12_frozen
/esp32
/GENERIC
/RELEASE
/TINYPICO
/stm32
/GENERIC
/PYBD_SF2
generate typeshed stubs of these files. (the .pyi files will be stored alongside the .py files)
Include/use them in the configuration
ref: https://learn.adafruit.com/micropython-basics-loading-modules/frozen-modules
Collect Frozen Stubs (micropython)¶
This is run daily though the github action workflow : get-all-frozen in the micropython-stubs repo.
If you want to run this manually
Check out repos side-by-side:
micropython-stubs
micropython-stubber
micropython
micropython-lib
link repos using all_stubs symlink
checkout tag / version in the micropython folder
(for most accurate results should checkout micropython-lib for the same date)run
get-frozen
run
update_stub
create a PR for changes to the stubs repo
Postprocessing¶
You can run postprocessing for all stubs by running either of the two scripts.
There is an optional parameter to specify the location of the stub folder. The default path is ./all_stubs
update_stubs [./mystubs]
This will generate or update the .pyi
stubs for all new (and existing) stubs in the ./all_stubs
or specified folder.
From version ‘1.3.8’ the .pyi
stubs are generated using stubgen
, before that the make_stub_files.py
script was used.
Stubgen is run on each ‘collected stub folder’ (that contains a modules.json
manifest) using the options : --ignore-errors --include-private
and the resulting .pyi
files are stored in the same folder (foo.py
and foo.pyi
are stored next to each other).
In some cases stubgen
detects duplicate modules in a ‘collected stub folder’, and subsequently does not generate any stubs for any .py
module or script.
then Plan B is to run stubgen for each separate *.py
file in that folder. While this is significantly slower and according to the stubgen documentation the resulting stubs may of lesser quality, but that is better than no stubs at all.
Note: In several cases stubgen
creates folders in inappropriate locations (reason undetermined), which would cause issues when re-running stubgen
at a later time.
to compensate for this behaviour the known-incorrect .pyi files are removed before and after stubgen is run see: cleanup(modules_folder)
in utils.py