Skip to content

Commit e891a91

Browse files
vstinnergpsheadethanfurmanEclips4
authored
gh-161705: Prevent shadowing stdlib in gzip/tarfile/zipfile CLI (#161706)
Remove sys.path[0] in gzip, tarfile and zipfile command line interface to prevent shadowing standard library modules from extracted files. * Add sys._path0 variable. * pymain_sys_path_add_path0() sets sys._path0. * Add runpy._remove_sys_path0(). * gzip, tarfile and zipfile main functions now call _remove_sys_path0() to prevent shadowing stdlib modules from extracted files. Co-authored-by: Gregory P. Smith <68491+gpshead@users.noreply.github.com> Co-authored-by: Ethan Furman <ethan@stoneleaf.us> Co-authored-by: Kirill Podoprigora <kirill.bast9@mail.ru>
1 parent 4e10d4f commit e891a91

14 files changed

Lines changed: 194 additions & 21 deletions

File tree

‎Doc/c-api/init_config.rst‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1227,8 +1227,8 @@ PyConfig
12271227
12281228
.. c:member:: int safe_path
12291229
1230-
If equals to zero, ``Py_RunMain()`` prepends a potentially unsafe path to
1231-
:data:`sys.path` at startup:
1230+
If equals to zero, ``Py_RunMain()`` prepends a potentially unsafe
1231+
:data:`sys._path0` to :data:`sys.path` at startup:
12321232
12331233
* If :c:member:`argv[0] <PyConfig.argv>` is equal to ``L"-m"``
12341234
(``python -m module``), prepend the current working directory.

‎Doc/library/sys.rst‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1651,6 +1651,24 @@ always available. Unless explicitly noted otherwise, all variables are read-only
16511651
* Module :mod:`site` This describes how to use .pth files to
16521652
extend :data:`sys.path`.
16531653

1654+
.. data:: _path0
1655+
1656+
Path prepended to :data:`sys.path` at Python startup:
1657+
1658+
* Current directory if running a module
1659+
with ``python -m module`` command line.
1660+
* Script's directory if running a script
1661+
with ``python script.py``.
1662+
* Otherwise an empty string, which means the current working directory,
1663+
for ``python -c code`` or ``python`` (REPL) command line.
1664+
1665+
If :option:`-P` or :option:`-I` command line options are used or if
1666+
:envvar:`PYTHONSAFEPATH` environment variable is used, :data:`sys._path0` is
1667+
``None`` and :data:`sys.path` is left unchanged.
1668+
1669+
.. versionadded:: next
1670+
1671+
16541672
.. data:: path_hooks
16551673

16561674
A list of callables that take a path argument to try to create a

‎Doc/using/cmdline.rst‎

Lines changed: 11 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -322,10 +322,12 @@ Miscellaneous options
322322
Run Python in isolated mode. This also implies :option:`-E`, :option:`-P`
323323
and :option:`-s` options.
324324

325-
In isolated mode :data:`sys.path` contains neither the script's directory nor
326-
the user's site-packages directory. All ``PYTHON*`` environment
327-
variables are ignored, too. Further restrictions may be imposed to prevent
328-
the user from injecting malicious code.
325+
In isolated mode, Python does not prepend the potentially unsafe path
326+
:data:`sys._path0` to :data:`sys.path`, and ignores all ``PYTHON*``
327+
environment variables, e.g. :envvar:`PYTHONPATH` and :envvar:`PYTHONHOME`,
328+
that might be set.
329+
Further restrictions may be imposed to prevent the user from injecting
330+
malicious code.
329331

330332
.. versionadded:: 3.4
331333

@@ -353,14 +355,8 @@ Miscellaneous options
353355

354356
.. option:: -P
355357

356-
Don't prepend a potentially unsafe path to :data:`sys.path`:
357-
358-
* ``python -m module`` command line: Don't prepend the current working
359-
directory.
360-
* ``python script.py`` command line: Don't prepend the script's directory.
361-
If it's a symbolic link, resolve symbolic links.
362-
* ``python -c code`` and ``python`` (REPL) command lines: Don't prepend an
363-
empty string, which means the current working directory.
358+
Don't prepend the potentially unsafe path :data:`sys._path0` to
359+
:data:`sys.path`.
364360

365361
See also the :envvar:`PYTHONSAFEPATH` environment variable, and :option:`-E`
366362
and :option:`-I` (isolated) options.
@@ -820,7 +816,9 @@ conflict.
820816
.. envvar:: PYTHONSAFEPATH
821817

822818
If this is set to a non-empty string, don't prepend a potentially unsafe
823-
path to :data:`sys.path`: see the :option:`-P` option for details.
819+
path :data:`sys._path0` to :data:`sys.path`.
820+
821+
See also the :option:`-P` option.
824822

825823
.. versionadded:: 3.11
826824

‎Lib/gzip.py‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -746,6 +746,11 @@ def opener(path, flags):
746746

747747
def main():
748748
from argparse import ArgumentParser
749+
import runpy
750+
751+
# Prevent shadowing a stdlib module from the extracted file
752+
runpy._remove_sys_path0()
753+
749754
parser = ArgumentParser(description=
750755
"A simple command line interface for the `gzip` module: act like `gzip`, "
751756
"but do not delete the input file.",

‎Lib/runpy.py‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -315,6 +315,26 @@ def run_path(path_name, init_globals=None, run_name=None):
315315
pass
316316

317317

318+
def _remove_sys_path0():
319+
"""
320+
Remove sys._path0 from sys.path. Do nothing if sys._path0 is
321+
sys._stdlib_dir or if it has already been removed. Similar to using -P
322+
(safe path) command line option.
323+
"""
324+
path0 = sys._path0
325+
if path0 is None:
326+
return
327+
328+
if path0 == sys._stdlib_dir:
329+
return
330+
331+
# Remove all path0 copies, not only the first one.
332+
# We are intentionally not trying to normalize & match alternate
333+
# representations of path0, this is only intended to remove copies
334+
# of the exact value the runtime itself may have added.
335+
sys.path[:] = [path for path in sys.path if path != path0]
336+
337+
318338
if __name__ == "__main__":
319339
# Run the module specified as the next command line argument
320340
if len(sys.argv) < 2:

‎Lib/tarfile.py‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3172,6 +3172,10 @@ def is_tarfile(name):
31723172

31733173
def main():
31743174
import argparse
3175+
import runpy
3176+
3177+
# Prevent shadowing stdlib modules from extracted files
3178+
runpy._remove_sys_path0()
31753179

31763180
description = 'A simple command-line interface for tarfile module.'
31773181
parser = argparse.ArgumentParser(description=description)

‎Lib/test/test_runpy.py‎

Lines changed: 91 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,7 @@
2121
verbose,
2222
)
2323
from test import support
24+
from test.support import os_helper
2425
from test.support.import_helper import forget, make_legacy_pyc, unload
2526
from test.support.os_helper import create_empty_file, temp_dir, FakePath
2627
from test.support.script_helper import make_script, make_zip_script
@@ -917,5 +918,95 @@ def test_pymain_run_module(self):
917918
self.assertSigInt(["-m", ham.stem], cwd=ham.parent)
918919

919920

921+
class MiscTests(unittest.TestCase):
922+
def test_remove_sys_path0(self):
923+
# Check that all path0 copies are removed
924+
PATH0 = '/path/zero'
925+
STDLIB_PATH = '/to/stdlib'
926+
OTHER_PATH = '/other/path'
927+
sys_path = [PATH0, STDLIB_PATH, PATH0, '', OTHER_PATH, PATH0]
928+
expected = [STDLIB_PATH, '', OTHER_PATH]
929+
with (support.swap_attr(sys, '_stdlib_dir', STDLIB_PATH),
930+
support.swap_attr(sys, '_path0', PATH0),
931+
support.swap_attr(sys, 'path', sys_path)):
932+
runpy._remove_sys_path0()
933+
self.assertEqual(sys.path, expected)
934+
935+
# Calling it twice does nothing
936+
runpy._remove_sys_path0()
937+
self.assertEqual(sys.path, expected)
938+
939+
# Do nothing if sys._path0 is sys._stdlib_dir
940+
sys_path = ['', STDLIB_PATH, OTHER_PATH]
941+
expected = sys_path.copy()
942+
with (support.swap_attr(sys, '_stdlib_dir', STDLIB_PATH),
943+
support.swap_attr(sys, '_path0', STDLIB_PATH),
944+
support.swap_attr(sys, 'path', sys_path)):
945+
runpy._remove_sys_path0()
946+
self.assertEqual(sys.path, expected)
947+
948+
# Do nothing if sys._path0 is None
949+
sys_path = ['', STDLIB_PATH, OTHER_PATH]
950+
expected = sys_path.copy()
951+
with (support.swap_attr(sys, '_stdlib_dir', STDLIB_PATH),
952+
support.swap_attr(sys, '_path0', None),
953+
support.swap_attr(sys, 'path', sys_path)):
954+
runpy._remove_sys_path0()
955+
self.assertEqual(sys.path, expected)
956+
957+
def run_python(self, *args, PYTHONPATH=None):
958+
cmd = [sys.executable, *args]
959+
env = support.make_clean_env()
960+
if PYTHONPATH:
961+
env['PYTHONPATH'] = PYTHONPATH
962+
proc = subprocess.run(cmd, text=True, stderr=subprocess.PIPE, env=env)
963+
self.assertEqual(proc.returncode, 0, proc.stderr)
964+
965+
@support.requires_subprocess()
966+
def test_remove_sys_path0_run_python(self):
967+
CODE = textwrap.dedent("""
968+
import runpy, sys
969+
970+
# Remove sys.path[0]
971+
orig_path = sys.path.copy()
972+
path0 = sys._path0
973+
runpy._remove_sys_path0()
974+
975+
if path0 is not None and path0 in sys.path:
976+
raise Exception(f"path0 {path0!r} found in sys.path {sys.path!r}")
977+
if not sys.flags.safe_path:
978+
if sys.path == orig_path:
979+
raise Exception("sys.path was not changed")
980+
else:
981+
if sys.path != orig_path:
982+
raise Exception("sys.path was changed")
983+
984+
# Calling it twice must do nothing
985+
clean_path = sys.path.copy()
986+
runpy._remove_sys_path0()
987+
if sys.path != clean_path:
988+
raise Exception("sys.path was changed")
989+
""")
990+
991+
# Test "python -c code"
992+
self.run_python('-c', CODE)
993+
self.run_python('-P', '-c', CODE)
994+
995+
with tempfile.TemporaryDirectory() as tmpdir:
996+
name = 'test_remove_sys_path0'
997+
filename = os.path.join(tmpdir, f'{name}.py')
998+
with open(filename, 'w') as fp:
999+
fp.write(CODE)
1000+
1001+
# Test "python script.py"
1002+
self.run_python(filename)
1003+
self.run_python('-P', filename)
1004+
1005+
with os_helper.change_cwd(tmpdir):
1006+
# Test "python -m module"
1007+
self.run_python('-m', name)
1008+
self.run_python('-P', '-m', name, PYTHONPATH=tmpdir)
1009+
1010+
9201011
if __name__ == "__main__":
9211012
unittest.main()

‎Lib/test/test_sys.py‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1493,6 +1493,15 @@ class PythonType:
14931493
self.assertFalse(is_immortal(b'abc'.decode()))
14941494
self.assertFalse(is_immortal('abc'.encode()))
14951495

1496+
def test_path(self):
1497+
self.assertIsInstance(sys.path, list)
1498+
for path in sys.path:
1499+
self.assertIsInstance(path, str)
1500+
1501+
def test_path0(self):
1502+
self.assertTrue(sys._path0 is None or isinstance(sys._path0, str),
1503+
sys._path0)
1504+
14961505

14971506
@test.support.cpython_only
14981507
@test.support.force_not_colorized_test_class

‎Lib/test/test_zipfile/test_core.py‎

Lines changed: 17 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -58,6 +58,15 @@ def get_files(test):
5858
test.assertFalse(f.closed)
5959

6060

61+
@contextlib.contextmanager
62+
def save_restore_sys_path():
63+
old_path = sys.path.copy()
64+
try:
65+
yield
66+
finally:
67+
sys.path[:] = old_path
68+
69+
6170
class LazyImportTest(unittest.TestCase):
6271
@cpython_only
6372
def test_lazy_import(self):
@@ -5987,17 +5996,18 @@ def test_add_comment(self):
59875996
def test_cli_with_metadata_encoding(self):
59885997
errmsg = "Non-conforming encodings not supported with -c."
59895998
args = ["--metadata-encoding=shift_jis", "-c", "nonesuch", "nonesuch"]
5990-
with captured_stdout() as stdout:
5991-
with captured_stderr() as stderr:
5999+
with (captured_stdout() as stdout,
6000+
captured_stderr() as stderr,
6001+
save_restore_sys_path()):
59926002
self.assertRaises(SystemExit, zipfile.main, args)
59936003
self.assertEqual(stdout.getvalue(), "")
59946004
self.assertIn(errmsg, stderr.getvalue())
59956005

5996-
with captured_stdout() as stdout:
6006+
with captured_stdout() as stdout, save_restore_sys_path():
59976007
zipfile.main(["--metadata-encoding=shift_jis", "-t", TESTFN])
59986008
listing = stdout.getvalue()
59996009

6000-
with captured_stdout() as stdout:
6010+
with captured_stdout() as stdout, save_restore_sys_path():
60016011
zipfile.main(["--metadata-encoding=shift_jis", "-l", TESTFN])
60026012
listing = stdout.getvalue()
60036013
for name in self.file_names:
@@ -6016,7 +6026,9 @@ def test_cli_with_metadata_encoding_extract(self):
60166026
except UnicodeEncodeError:
60176027
self.skipTest(f'cannot encode file name {fn!a}')
60186028

6019-
zipfile.main(["--metadata-encoding=shift_jis", "-e", TESTFN, TESTFN2])
6029+
with save_restore_sys_path():
6030+
zipfile.main(["--metadata-encoding=shift_jis", "-e",
6031+
TESTFN, TESTFN2])
60206032
listing = os.listdir(TESTFN2)
60216033
for name in self.file_names:
60226034
self.assertIn(name, listing)

‎Lib/zipfile/__init__.py‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2926,6 +2926,10 @@ def _get_code(self, pathname, basename):
29262926

29272927
def main(args=None):
29282928
import argparse
2929+
import runpy
2930+
2931+
# Prevent shadowing stdlib modules from extracted files
2932+
runpy._remove_sys_path0()
29292933

29302934
description = 'A simple command-line interface for zipfile module.'
29312935
parser = argparse.ArgumentParser(description=description)

0 commit comments

Comments
 (0)