Skip to content

Add Python 3.14 support, experimental subinterpreter/freethreading support#1279

Merged
elprans merged 5 commits intomasterfrom
py314
Oct 21, 2025
Merged

Add Python 3.14 support, experimental subinterpreter/freethreading support#1279
elprans merged 5 commits intomasterfrom
py314

Conversation

@elprans
Copy link
Member

@elprans elprans commented Oct 13, 2025 •
edited
Loading

The bulk of the changes here is a rewrite of recordobj.c to use modern
CPython API to properly isolate the module (PEP 489, PEP 573, PEP 630).
This, along with Cython flags, enables support for safely importing
asyncpg in subinterpreters. The Record freelist is now
thread-specific, so asyncpg should be thread-safe at the C level.
Support for subinterpreters and freethreading is EXPERIMENTAL.

@ddanier
Copy link

ddanier commented Oct 14, 2025

As of no pre-built package being available for asyncpg on Python 3.14 you cannot install the package on "slim" docker images, cause gcc is not installed there. This is an issue as you tend to use those "slim" variants when you care about security. Of course it is possible to get this working by either compiling this package yourself on a non-"slim" image and then use a multistage build or switching to the non-"slim" version, but that either does increase complexity or reduce security significantly.

What I want to say: I really would like this to get finished and merged soon. Thanks for all the effort! ;-)

@elprans elprans changed the title Add Python 3.14 support, drop EOL 3.8 Add Python 3.14 support, experimental subinterpreter/freethreading support Oct 16, 2025
@elprans elprans requested review from 1st1 and fantix October 16, 2025 18:51
@elprans elprans force-pushed the py314 branch 2 times, most recently from e742ff7 to 7037527 Compare October 16, 2025 18:53
@elprans elprans requested a review from msullivan October 16, 2025 18:54
fantix
fantix reviewed Oct 16, 2025
View reviewed changes
@elprans elprans force-pushed the py314 branch 3 times, most recently from 483125b to fd6d675 Compare October 17, 2025 04:36
@elprans elprans requested a review from fantix October 17, 2025 04:57
@frankie567 frankie567 mentioned this pull request Oct 18, 2025
Open
1st1
1st1 reviewed Oct 20, 2025
View reviewed changes
1st1
1st1 reviewed Oct 20, 2025
View reviewed changes
asyncpg/record/recordobj.c
@@ -0,0 +1,1281 @@
/* Big parts of this file are copied (with modifications) from
Copy link
Member

@1st1 1st1 Oct 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any reason for this file to be moved? No diff now, all code is like new which makes review harder

Copy link
Member Author

@elprans elprans Oct 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There were some odd Cython issues initially, but I'll see if I can move it back.

Copy link
Member Author

@elprans elprans Oct 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, @1st1, please take another look.

1st1
1st1 reviewed Oct 20, 2025
View reviewed changes
Copy link
Member

@1st1 1st1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we move recordobj.c back where it was?

@elprans
Copy link
Member Author

elprans commented Oct 20, 2025

Can we move recordobj.c back where it was?

I couldn't find a way to do it. Cython does not expose a way to plug into its generated module state, so record must be a standalone module, not a concatenation. Why is this important?

@1st1
Copy link
Member

1st1 commented Oct 20, 2025

I couldn't find a way to do it. Cython does not expose a way to plug into its generated module state, so record must be a standalone module, not a concatenation. Why is this important?

Just for me to review the thing, it's a big file. Were there any changes in it?

@elprans elprans requested a review from 1st1 October 20, 2025 16:55
@elprans elprans force-pushed the py314 branch 2 times, most recently from f5bb6f8 to 8eed349 Compare October 20, 2025 17:16
@elprans
Add Python 3.14 support, experimental subinterpreter/freethreading su…
ecb4c19 
…pport

The bulk of the changes here is a rewrite of `recordobj.c` to use modern
CPython API to properly isolate the module (PEP 489, PEP 573, PEP 630).
This, along with Cython flags, enables support for safely importing
`asyncpg` in subinterpreters.  The `Record` freelist is now
thread-specific, so asyncpg should be thread-safe *at the C level*.
Support for subinterpreters and freethreading is EXPERIMENTAL.
1st1
1st1 reviewed Oct 20, 2025
View reviewed changes
1st1
1st1 reviewed Oct 20, 2025
View reviewed changes
asyncpg/protocol/record/recordobj.c
v_is_tuple = 1;
}
else if (ApgRecord_CheckExact(v)) {
else if (v_type == state->ApgRecord_Type) {
Copy link
Member

@1st1 1st1 Oct 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd keep the ApgRecord_CheckExact macros, just make it accept state as first arg.

1st1
1st1 reviewed Oct 20, 2025
View reviewed changes
asyncpg/protocol/record/recordobj.c Outdated
cmp = vlen >= wlen;
break;
default:
return NULL; /* cannot happen */
Copy link
Member

@1st1 1st1 Oct 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Py_UNREACHABLE();

1st1
1st1 reviewed Oct 20, 2025
View reviewed changes
1st1
1st1 approved these changes Oct 20, 2025
View reviewed changes
Copy link
Member

@1st1 1st1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks ok but please run tests in refleak hunt mode

msullivan
msullivan approved these changes Oct 20, 2025
View reviewed changes
@elprans elprans merged commit 9e42642 into master Oct 21, 2025
46 checks passed
@elprans elprans deleted the py314 branch October 21, 2025 19:20
@bfontaine bfontaine mentioned this pull request Nov 17, 2025
Closed
@bfontaine
Copy link

bfontaine commented Nov 21, 2025

When is this expected to be released?

elprans added a commit that referenced this pull request Nov 24, 2025
@elprans
asyncpg v0.31.0
71775a6 
Enable Python 3.14 with experimental subinterpreter/freethreading
support.

Improvements
============

* Add Python 3.14 support, experimental subinterpreter/freethreading support (#1279)
  (by @elprans in 9e42642)

* Avoid performing type introspection on known types (#1243)
  (by @elprans in 5c9986c)

* Make `prepare()` not use named statements by default when cache is disabled (#1245)
  (by @elprans in 5b14653)

* Implement connection service file functionality (#1223)
  (by @AndrewJackson2020 in 1d63bb1)

Fixes
=====

* Fix multi port connection string issue (#1222)
  (by @AndrewJackson2020 in 01c0db7)

* Avoid leaking connections if _can_use_connection fails (#1269)
  (by @yuliy-openai in e94302d)

Other
=====

* Drop support for EOL Python 3.8 (#1281)
  (by @elprans in 6c2c490)
@elprans elprans mentioned this pull request Nov 24, 2025
Merged
bg-admin7 pushed a commit to bg-open-source/asyncpg that referenced this pull request Dec 10, 2025
Merge tag 'v0.31.0' into non-blocking-conns-v3
acc3154 
asyncpg v0.31.0

Enable Python 3.14 with experimental subinterpreter/freethreading
support.

Improvements
============

* Add Python 3.14 support, experimental subinterpreter/freethreading support (MagicStack#1279)
  (by @elprans in 9e42642)

* Avoid performing type introspection on known types (MagicStack#1243)
  (by @elprans in 5c9986c)

* Make `prepare()` not use named statements by default when cache is disabled (MagicStack#1245)
  (by @elprans in 5b14653)

* Implement connection service file functionality (MagicStack#1223)
  (by @AndrewJackson2020 in 1d63bb1)

Fixes
=====

* Fix multi port connection string issue (MagicStack#1222)
  (by @AndrewJackson2020 in 01c0db7)

* Avoid leaking connections if _can_use_connection fails (MagicStack#1269)
  (by @yuliy-openai in e94302d)

Other
=====

* Drop support for EOL Python 3.8 (MagicStack#1281)
  (by @elprans in 6c2c490)

# -----BEGIN PGP SIGNATURE-----
#
# iQIzBAABCAAdFiEES/rQWfT9RAppNY3YXEaHeAYth78FAmkk6UMACgkQXEaHeAYt
# h7+LBw//fX/3/nHrEUctYR7A/WQqg2oj3nvk10b3OQjzHUGyHq0DhkSC7/H9P0FI
# vl0j2n+BfKcKxOw4OgbxDq5fff/a4ScsejVwsKqQEEDH9tiXAL2cIId1yvNWC9hJ
# 8WxmkJzbFHdrl1D3/pxUv15UoaMSZD1DtPXaYEupRyEKXVOfQ/ush5GU5M3vA19R
# aWrAFtbMd3MMfaBYEO1TQtZNCV3n3Dv8fmIwv8qPbqJt2lsk4HOE8F3Be/RGiaPZ
# dI9eGnZ+qmlYC/mZYKNywaBlm4v/hHhuGwSh8WdkY6h4lFkBl8HipuPpoNeWO/fd
# k5wl1Q2HA8i2qEokz7EzYPRYjOPZwXphxscRXJQbdlCNyfJ+lPTu3uAQ08RTnoGU
# +JROauwgtQknnZEGtRWNerRcbZE1D+SgkLhGkEk6+tcIJ0G2XoCsn7A/u9LMmZuN
# 7lcFnLEdDnUSaLx7SZgcbOKP1Acg1W2Vk375JxXrymGqq5FPss8cdg+KOFMNuljT
# v/RtKvAP3yXpR448uPQ6M4BNVJ5my3UwdPrHwpGNkSxaqKBGGtryBG4LLw43yPCg
# FPFO9P//z+59RsmAvilaOhT/pLhqRvOJTkaUt5bt6Fw+5SzCfNfciNkl4rBC+9EQ
# LC/NQmO9cFYlpnO7s3uF0M+JTLV1u0CJp5G5USiQiEU+tVlaH6E=
# =bEvL
# -----END PGP SIGNATURE-----
# gpg: Signature made Tue 25 Nov 2025 12:24:51 AM CET
# gpg:                using RSA key 4BFAD059F4FD440A69358DD85C468778062D87BF
# gpg: Can't check signature: No public key
bg-admin7 pushed a commit to bg-open-source/asyncpg that referenced this pull request Dec 10, 2025
Merge tag 'v0.31.0' into non-blocking-conns-v4
64ddc43 
asyncpg v0.31.0

Enable Python 3.14 with experimental subinterpreter/freethreading
support.

Improvements
============

* Add Python 3.14 support, experimental subinterpreter/freethreading support (MagicStack#1279)
  (by @elprans in 9e42642)

* Avoid performing type introspection on known types (MagicStack#1243)
  (by @elprans in 5c9986c)

* Make `prepare()` not use named statements by default when cache is disabled (MagicStack#1245)
  (by @elprans in 5b14653)

* Implement connection service file functionality (MagicStack#1223)
  (by @AndrewJackson2020 in 1d63bb1)

Fixes
=====

* Fix multi port connection string issue (MagicStack#1222)
  (by @AndrewJackson2020 in 01c0db7)

* Avoid leaking connections if _can_use_connection fails (MagicStack#1269)
  (by @yuliy-openai in e94302d)

Other
=====

* Drop support for EOL Python 3.8 (MagicStack#1281)
  (by @elprans in 6c2c490)

# -----BEGIN PGP SIGNATURE-----
#
# iQIzBAABCAAdFiEES/rQWfT9RAppNY3YXEaHeAYth78FAmkk6UMACgkQXEaHeAYt
# h7+LBw//fX/3/nHrEUctYR7A/WQqg2oj3nvk10b3OQjzHUGyHq0DhkSC7/H9P0FI
# vl0j2n+BfKcKxOw4OgbxDq5fff/a4ScsejVwsKqQEEDH9tiXAL2cIId1yvNWC9hJ
# 8WxmkJzbFHdrl1D3/pxUv15UoaMSZD1DtPXaYEupRyEKXVOfQ/ush5GU5M3vA19R
# aWrAFtbMd3MMfaBYEO1TQtZNCV3n3Dv8fmIwv8qPbqJt2lsk4HOE8F3Be/RGiaPZ
# dI9eGnZ+qmlYC/mZYKNywaBlm4v/hHhuGwSh8WdkY6h4lFkBl8HipuPpoNeWO/fd
# k5wl1Q2HA8i2qEokz7EzYPRYjOPZwXphxscRXJQbdlCNyfJ+lPTu3uAQ08RTnoGU
# +JROauwgtQknnZEGtRWNerRcbZE1D+SgkLhGkEk6+tcIJ0G2XoCsn7A/u9LMmZuN
# 7lcFnLEdDnUSaLx7SZgcbOKP1Acg1W2Vk375JxXrymGqq5FPss8cdg+KOFMNuljT
# v/RtKvAP3yXpR448uPQ6M4BNVJ5my3UwdPrHwpGNkSxaqKBGGtryBG4LLw43yPCg
# FPFO9P//z+59RsmAvilaOhT/pLhqRvOJTkaUt5bt6Fw+5SzCfNfciNkl4rBC+9EQ
# LC/NQmO9cFYlpnO7s3uF0M+JTLV1u0CJp5G5USiQiEU+tVlaH6E=
# =bEvL
# -----END PGP SIGNATURE-----
# gpg: Signature made Tue 25 Nov 2025 12:24:51 AM CET
# gpg:                using RSA key 4BFAD059F4FD440A69358DD85C468778062D87BF
# gpg: Can't check signature: No public key
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@1st1 1st1 1st1 approved these changes

@msullivan msullivan msullivan approved these changes

@fantix fantix Awaiting requested review from fantix

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@elprans @ddanier @1st1 @bfontaine @msullivan @fantix