PyPy v7.3.17: release of python 2.7 and 3.10, released 2024-08-28

The PyPy team is proud to release version 7.3.17 of PyPy.

This release includes a new RISC-V JIT backend, an improved REPL based on work by the CPython team, and better JIT optimizations of integer operations. Special shout-outs to Logan Chien for the RISC-V backend work, to Nico Rittinghaus for better integer optimization in the JIT, and the CPython team that has worked on the repl.

The release includes two different interpreters:

  • PyPy2.7, which is an interpreter supporting the syntax and the features of Python 2.7 including the stdlib for CPython 2.7.18+ (the + is for backported security updates)

  • PyPy3.10, which is an interpreter supporting the syntax and the features of Python 3.10, including the stdlib for CPython 3.10.14.

The interpreters are based on much the same codebase, thus the dual release. This is a micro release, all APIs are compatible with the other 7.3 releases. It follows after 7.3.16 release on April 23, 2024.

We recommend updating. You can find links to download the releases here:

We would like to thank our donors for the continued support of the PyPy project. If PyPy is not quite good enough for your needs, we are available for direct consulting work. If PyPy is helping you out, we would love to hear about it and encourage submissions to our blog via a pull request to https://github.com/pypy/pypy.org

We would also like to thank our contributors and encourage new people to join the project. PyPy has many layers and we need help with all of them: bug fixes, PyPy and RPython documentation improvements, or general help with making RPython’s JIT even better.

If you are a python library maintainer and use C-extensions, please consider making a HPy / CFFI / cppyy version of your library that would be performant on PyPy. In any case, both cibuildwheel and the multibuild system support building wheels for PyPy.

RISC-V backend for the JIT

PyPy’s JIT has added support for generating 64-bit RISC-V machine code at runtime (RV64-IMAD, specifically). So far we are not releasing binaries for any RISC-V platforms, but there are instructions on how to cross-compile binaries.

REPL Improvements

The biggest user-visible change of the release is new features in the repl of PyPy3.10. CPython 3.13 has adopted and extended PyPy’s pure-Python repl, adding a number of features and fixing a number or bugs in the process. We have backported and added the following features:

  • Prompts and tracebacks use terminal colors, as well as terminal hyperlinks for file names.

  • Bracketed paste enable pasting several lines of input into the terminal without auto-indentation getting in the way.

  • A special interactive help browser (F1), history browser (F2), explicit paste mode (F3).

  • Support for Ctrl-<left/right> to jump over whole words at a time.

See the CPython documentation for further details. Thanks to Łukasz Langa, Pablo Galindo Salgado and the other CPython devs involved in this work.

Better JIT optimizations of integer operations

The optimizers of PyPy’s JIT have become much better at reasoning about and optimizing integer operations. This is done with a new “knownbits” abstract domain. In many programs that do bit-manipulation of integers, some of the bits of the integer variables of the program can be statically known. Here’s a simple example:

x = a | 1
...
if x & 1:
    ...
else:
    ...

With the new abstract domain, the JIT can optimize the if-condition to True, because it already knows that the lowest bit of x must be set. This optimization applies to all Python-integers that fit into a machine word (PyPy optimistically picks between two different representations for int, depending on the size of the value). Unfortunately there is very little impact of this change on almost all Python code, because intensive bit-manipulation is rare in Python. However, the change leads to significant performance improvements in Pydrofoil (the RPython-based RISC-V/ARM emulators that are automatically generated from high-level Sail specifications of the respective ISAs, and that use the RPython JIT to improve performance).

PyPy versions and speed.pypy.org

The keen-eyed will have noticed no mention of Python version 3.9 in the releases above. Typically we will maintain only one version of Python3, but due to PyPy3.9 support on conda-forge we maintained multiple versions from the first release of PyPy3.10 in PyPy v7.3.12 (Dec 2022). Conda-forge is sunsetting its PyPy support, which means we can drop PyPy3.9. Since that was the major driver of benchmarks at https://speed.pypy.org, we revamped the site to showcase PyPy3.9, PyPy3.10, and various versions of cpython on the home page. For historical reasons, the “baseline” for comparison is still cpython 3.7.19.

We will keep the buildbots building PyPY3.9 until the end of August, these builds will still be available on the nightly builds tab of the buildbot.

What is PyPy?

PyPy is a Python interpreter, a drop-in replacement for CPython It’s fast (PyPy and CPython performance comparison) due to its integrated tracing JIT compiler.

We also welcome developers of other dynamic languages to see what RPython can do for them.

We provide binary builds for:

  • x86 machines on most common operating systems (Linux 32/64 bits, Mac OS 64 bits, Windows 64 bits)

  • 64-bit ARM machines running Linux (aarch64) and macos (macos_arm64).

PyPy supports Windows 32-bit, Linux PPC64 big- and little-endian, Linux ARM 32 bit, RISC-V RV64IMAFD Linux, and s390x Linux but does not release binaries. Please reach out to us if you wish to sponsor binary releases for those platforms. Downstream packagers provide binary builds for debian, Fedora, conda, OpenBSD, FreeBSD, Gentoo, and more.

Changelog

For all versions

  • DOCS: Fix source links in docs when moving from heptapod to github (#3216)

  • DOCS: Mention externals mercurial repo for windows build (#4921)

Bugfixes

  • Don’t crash on constfolding field reads from null pointers in the RPython translator (#4941)

  • Fix const char declaration in cffi gdbm (#4931)

  • Restore error message of TypeError generated when calling tuple.__getitem__ with invalid types (#4954)

  • id(obj) should really be an int, not a long

  • Bump MACOSX_DEPLOYMENT_TARGET to 10.15 on x86_64 and 11.0 on arm64 (#4975)

  • Ignore whitespace when parsing complex numbers (#4980)

  • Add mmap.MAP_POPULATE on linux (#4991)

Speedups and enhancements

  • Optimize for array length from new_array for non-virtual arrays

  • Add look_inside_iff for W_TupleObject._getslice

  • Allow implicit string literal concatenation as docstring (#4973)

  • Improve trace of adding a new attribute to an already big instance and to a list of unboxed fields

  • Improve the code generated by some list methods in the JIT and the RPython support code:

    • calling find or count doesn’t need the list object itself, just the underlying storage

    • unroll in more situations when creating a list

    • make list multiplication use ll_alloc_and_set if the list has length 1

    • copy lists in multiplication with log2(n) arraycopys instead of just copying n times (string multiplication does it like this too)

    • save one copy of the lhs list when adding two list

  • Add a jit driver for dict.__eq__ and an especially fast implementation for mapdict equality

  • Make the JIT generate better code for certain function calling patterns with *args

  • Optimize x * -1 and x // -1 to -x in the JIT.

  • Speed up binary search on the utf8 index storage. This slightly speeds up unicode/str.find on very large strings.

  • Optimize integers using the knownbits abstract domain (shout-out to Nico Rittinghaus).

Python 3.10

Bugfixes

  • Fix leak of global named resources when using multiprocessing spawn (python/cpython#90549)

  • Include pyexpatns.h when building pyexpat

  • Add IO_REPARSE_TAG* constants to stats.py (#4953)

  • Add some missing sqlite3 symbols and defines.

  • getindex_w works differently between py2 and py3, leading to error message confusion

  • Fix typo HAVE_MS_WINDOWS-> MS_WINDOWS which impacted the stats module (#4952)

  • Compare lower case exe names when creating a virtual environment (#4958)

  • Add missing typecodes attribute to array

  • Add cmath.tau, using math.pi * 2.0 to define it.

  • Remove dead code, make stats logic like CPython (#4976)

  • Add Py_UNREACHABLE to pymacro.h (#4982)

  • Don’t segfault trying to generate a better SyntaxError msg. Also follow CPython’s logic about RAISE_SYNTAX_ERROR_INVALID_TARGET more closely, to get the correct locations (#4897)

  • Fix long-standing bugs in pyrepl (from the CPython port of PyPy’s pyrepl)

    • Support ctrl-left and ctrl-right in the console (#906)

    • Implement bracketed paste (#2880)

  • Update bundled setuptools to 65.5.0

  • Fix value of readonly flag in PyMemoryView_FromMemory (#4993)

  • Make sure to call the atexit handlers after receiving sigint (#4995). This is done by calling space.finish().

  • Make stack depth computation in the bytecode compiler more precise. Stack depths of various instructions were off by one for historical reasons. the errors carefully balanced each other out. Now code that does exception handling has smaller maximum bytecode stack sizes.

  • Do not try __int__ in PyLong_AsLong and PyLong_AsInt (#4949)

  • Make math.isinf.__self__ == math to follow CPython’s behaviour (#4959)

  • Fix post-build in-place venv creation (#4958)

  • Fix converting python object to unsigned long (in C) object attribute (#4996)

  • Add missing sysconfig value LDCXXSHARED (#5004)

  • Add missing token.SOFT_KEYWORD (#4962)

  • Update vendored CFFI version to 1.17.0 from 1.17.0dev1. No real changes, this will ensure pip install cffi does nothing.

  • Implement more of asyncio’s _overlap module on windows (#4008)

  • Disallow HAVE_LCHMOD like CPython on linux, since glibc has a dummy implementation that always errors out.

Speedups and enhancements

  • Move the import fast path from IMPORT_NAME to __import__, which speeds up explicitly calling the latter.

  • Update OpenSSL in the “embedded” builds to 3.3.1, addressing a performance regression in urllib3 tests (#4877)

  • Speed up unicode encode/decode errormessage handling (#4972). Before this fix, the surrogateescape codec was quadratic in some corner cases.

  • Backport pyrepl and _colorize from CPython3.13

  • Make TextIOWrapper.tell() about 100x faster (#5009). Also fix some problems with seek and _multibytecodecs.