Skip to content

Attempt on optimizing the builds - #3383

Open
kuzeyron wants to merge 27 commits into
kivy:developfrom
kuzeyron:optimize-python-some-more
Open

kuzeyron wants to merge 27 commits into
kivy:developfrom
kuzeyron:optimize-python-some-more

Conversation

@kuzeyron

@kuzeyron kuzeyron commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

This PR is about optimizing the builds and strip out data that isn't needed and therefor the first launch on Androids would be faster when it comes to unpacking the data.

This PR requires testing.

The compressed filesize in comparison is about 2.4MB lighter and the unpacked data is a lot more lighter than before.
So in my books it's 2.4MB * 4 = 9.6MB of free space if you are publishing it to the Play Store with four architectures.

Next plan is to eliminate setuptools from Kivy itself because it's roughly 2.5MB in size and if I recall it correctly it's only used once with Kivy. Another thing that comes to Kivy is one of the fonts named "DeJaVue" that is nearly 1MB in size and I believe Kivy isn't even using all those characters coming with the font.

Comment thread pythonforandroid/recipes/python3/__init__.py Outdated
Comment thread pythonforandroid/recommendations.py Outdated
'idlelib',
'lib2to3',
'msilib',
'multiprocessing',

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is that? Can't Android app use Python native ThreadPool for instance? If they do, then this would break

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is what it's using with the develop branch:

    stdlib_dir_blacklist = {
        '__pycache__',
        'test',
        'tests',
        'lib2to3',
        'ensurepip',
        'idlelib',
        'tkinter',
    }
    ```
    I just sorted the lines and added multiprocessing as Android shouldn't be able to use that.
    And msilbi is safe to remove.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removing multiprocessing saves about 170 KiB compressed. Even if the process part of multiprocessing is unsupported on Android, removing the entire package also breaks its thread APIs.
It's not hypothetical: CodeRED imports multiprocessing.dummy.Process and uses it during application startup
Please keep multiprocessing for now.

This PR didn't just sort and remove multiprocessing, other were dropped too, but are more defensible:

  • curses, about 12 KiB, depends on the disabled _curses
  • turtle and turtledemo, about 100 KiB combined, depend on omitted Tkinter
  • venv, about 20 KiB, is not practically usable in the packaged Android runtime
  • msilib is absent from Python 3.14 but p4a supports older Python overrides. Removing it saves about 27 KiB with Python 3.10 and is safe because _msi is Windows-only
  • pydoc_data saves about 160 KiB. Removing it actually help("topics") and help("keywords"), but I could not find a current public p4a app relying on those APIs. So OK to drop.

These are not necessarily trivial changes and required more investigation than the original explanation suggested. As the scope increases, it becomes easier to miss compatibility regressions like the multiprocessing one.
That list is mostly fine once multiprocessing is restored.
Since this part has now been reviewed and clarified, could we move these blacklist changes into a dedicated PR? This would give us a small/quick win while the rest of the unresolved topics of PR are being discussed.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When python.org is publishing their wheels for Androids their goal is to remove all these I have stripped out. And they are even stripping out more than I currently have.


site_packages_dir_blacklist = {
'__pycache__',
'*.dist-info',

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couldn't this break importlib.metadata.version() that some app may use to show current version?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I inspected most of those folders and to me it's just spam.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think dist-info is spam because it contains metadata that can be used at runtime like I shared.
For instance if you app/package does __<package-name>__.version or version('package-name') type of thing.
One concrete example with pydantic: https://github.com/pydantic/pydantic/blob/v2.13.5/pydantic/networks.py#L969

This is why I prefer that we don't increase the scope too much of this PR because it's easy to miss regressions like this one. Let's keep the PR minimal with one area e.g. site_packages_dir_blacklist or build flags for instance so t's easier to keep the focus during review and testing

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

spamtome spamtome2 Pretty much spam to me.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes but still consumed by some packages and doesn't save a lot of space

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not about saving space with dist-infos. It's about having less data to unpack.

Comment thread pythonforandroid/recipes/python3/__init__.py
Comment thread pythonforandroid/util.py Outdated
@AndreMiras

Copy link
Copy Markdown
Member

Thanks for digging into this, the size savings are worth having. But this PR now mixes several changes that each carry a different tradeoff for users, which makes it hard to review and to spot regressions. I've left inline comments on the ones I think need their own discussion.
Could you narrow this PR to the smallest low-risk piece.
For example turtledemo, turtle.pyc and msilib (Windows installer support only) these deserve a small low risk PR that's easy to merge straight away.

@kuzeyron

Copy link
Copy Markdown
Contributor Author

Thanks for digging into this, the size savings are worth having. But this PR now mixes several changes that each carry a different tradeoff for users, which makes it hard to review and to spot regressions. I've left inline comments on the ones I think need their own discussion. Could you narrow this PR to the smallest low-risk piece. For example turtledemo, turtle.pyc and msilib (Windows installer support only) these deserve a small low risk PR that's easy to merge straight away.

I was pretty clear that This PR requires testing. Not just the CI's. Also from other users.

@kuzeyron

Copy link
Copy Markdown
Contributor Author
diff The difference by optimizing this recipe.

'*.dist-info',
'bin',
'tests',
'setuptools',

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, I'm really not sure about removing setuptools, let's restore.
Also some app use it e.g. https://github.com/mdelacorte/appElectricidadAndroid/blob/65964f69981976adbaa46aa93d702b80c3233482/main.py#L26

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Someone will have to fix the other recipes after this PR. Setuptools is included for no other reasons than to compile C++ files for Kivy and shouldn't even include the package itself in aab/apk. It serves no purpose having it there.

Comment thread pythonforandroid/recipes/python3/__init__.py Outdated
Comment thread pythonforandroid/recipes/python3/__init__.py Outdated
Comment thread pythonforandroid/recipes/python3/__init__.py Outdated
Comment thread pythonforandroid/recipes/python3/__init__.py
Comment thread tests/test_util.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants