net-misc/curl: add 8.9.1; enable HTTP/3 (QUIC) by default

The support for HTTP/3 is stable enough and we haven't seen any
reported issues since unmasking the USE where possible.

HTTP/3 is default-enabled using dev-libs/openssl[quic],
with the USE_EXPAND and REQUIRED_USE enforcing required deps
and enabling users to select an alternative implementation.

The only currently-supported option for this is net-libs/ngtcp2[gnutls],
though it is expected that as other TLS backends implement QUIC
(or are supported by other backends) that this will increase.

Of note is that HTTP/3 support and MultiSSL are mutually exclusive at
this point in time. There does not appear to be a significant benefit
to MutliSSL in 2024:

> An original driving factor for this feature was to allow Schannel with
> other TLS backends on Windows so that users could access "native" CA certs
> in the Windows CA store. Subsequently, curl has introduced support in multiple
> TLS backends to use native CAs.
> This reduces the need for many people to use and switch TLS backends.
> There are probably also other use cases, since the TLS backends are not all alike.

We're not dropping MutliSSL support, however HTTP/3 as the default seems
like the option that provides the greater utility as a default
configuration.

This is not to say that MultiSSL support is inherently incompatible with
HTTP/3, it will just require a sufficiently motivated developer
to implement the code.

Closes: https://bugs.gentoo.org/936627
Signed-off-by: Matt Jolly <kangie@gentoo.org>

3 files changed, 401 insertions, 7 deletions

diff --git a/net-misc/curl/Manifest b/net-misc/curl/Manifest
index 531b8c06008f..fb1e98c7b273 100644
--- a/net-misc/curl/Manifest
+++ b/net-misc/curl/Manifest
@@ -4,3 +4,5 @@ DIST curl-8.8.0.tar.xz 2748860 BLAKE2B c14903bad4cbd1752a5335afa6bcc78be1a484692
 DIST curl-8.8.0.tar.xz.asc 488 BLAKE2B d80c0ff357b344d7ec2b975a92f1eeb7557993b61a69e7adaaeab89c9b5a53ddade5104fe1a0ad260145db9c90fc0aae36dfc22320492db6696f290da9ff675b SHA512 37b501770225dff6b1e7bde1157f556f10ec1c597fcbbb5c8b8c370efb97a3a70f585f2f5c201b96380d68466696474a5f65a07da59b704678d6927567d25359
 DIST curl-8.9.0.tar.xz 2781828 BLAKE2B 3302ce98d937bb398fc1abcc1c403796503099e06919ea3b104c873a6fb6cd79328ea9684f5118f63ebb20bad18b94ebfbe92e87716fc24b91dcc92ff2d304b7 SHA512 922c726cfa3a73954927a32f485248d7a53a3348638a6a01add1bc0a67a7d2ee9cdb7c78b6db84bb7e2fab9d2d5487a96d9071832198b63a86d2caaef85c9310
 DIST curl-8.9.0.tar.xz.asc 488 BLAKE2B 7f35383fd98fe0947be9cb0bfb4737a185f40bb3c3e7ab001cb1bef026dec654a01059e225c0d9774c2c5a57a6ee00a4ccf4be8eb1a2db17fe7b2cdbdd06b2f1 SHA512 44cc7053ac0fddcb5131e7806fcd793d70bd49c5549b2548bbcbe60fdf913f450e45861ff6497b30eb00fd84483302ff9b6c3aea6b66728d6e54dd7ffc388408
+DIST curl-8.9.1.tar.xz 2782364 BLAKE2B 6e38e20e2b03ab5bfbb8d9797442dfdd9644fc80d7b1f7c1efb1f44e0d730524e82ccf7413b2c6f4555bd61ae42f91ec7c0201e2c0d563811c85164aa234aada SHA512 a0fe234402875db194aad4e4208b7e67e7ffc1562622eea90948d4b9b0122c95c3dde8bbe2f7445a687cb3de7cb09f20e5819d424570442d976aa4c913227fc7
+DIST curl-8.9.1.tar.xz.asc 488 BLAKE2B 437268f6e5ba5db73f205fd87f3ded1e5fc200e8bf63a83cdb7e21dfbf2f4a4620e598cd0bf5d8fa1548ade08d45b386599542cd988df46a238b85790409f42e SHA512 18acd58436d70900ab6912b84774da2c451b9dbfc83d6d00f85bbbe7894b67075918e58956fdb753fcc1486e4f10caa31139d7c68b037d7c83dc2e9c2fae9f9b
diff --git a/net-misc/curl/curl-8.9.1.ebuild b/net-misc/curl/curl-8.9.1.ebuild
new file mode 100644
index 000000000000..c8446b6178ce
--- /dev/null
+++ b/net-misc/curl/curl-8.9.1.ebuild
@@ -0,0 +1,380 @@
+# Copyright 1999-2024 Gentoo Authors
+# Distributed under the terms of the GNU General Public License v2
+
+EAPI=8
+
+# Maintainers should subscribe to the 'curl-distros' ML for backports etc
+# https://daniel.haxx.se/blog/2024/03/25/curl-distro-report/
+# https://lists.haxx.se/listinfo/curl-distros
+
+VERIFY_SIG_OPENPGP_KEY_PATH=/usr/share/openpgp-keys/danielstenberg.asc
+inherit autotools multilib-minimal multiprocessing prefix toolchain-funcs verify-sig
+
+DESCRIPTION="A Client that groks URLs"
+HOMEPAGE="https://curl.se/"
+
+if [[ ${PV} == 9999 ]]; then
+	inherit git-r3
+	EGIT_REPO_URI="https://github.com/curl/curl.git"
+else
+	SRC_URI="
+		https://curl.se/download/${P}.tar.xz
+		verify-sig? ( https://curl.se/download/${P}.tar.xz.asc )
+	"
+	KEYWORDS="~alpha ~amd64 ~arm ~arm64 ~hppa ~ia64 ~loong ~m68k ~mips ~ppc ~ppc64 ~riscv ~s390 ~sparc ~x86 ~amd64-linux ~x86-linux ~arm64-macos ~ppc-macos ~x64-macos ~x64-solaris"
+fi
+
+LICENSE="BSD curl ISC test? ( BSD-4 )"
+SLOT="0"
+IUSE="+adns +alt-svc brotli debug +ftp gnutls gopher +hsts +http2 +http3 idn +imap kerberos ldap mbedtls +openssl +pop3"
+IUSE+=" +psl +progress-meter +quic rtmp rustls samba +smtp ssh ssl sslv3 static-libs test telnet +tftp websockets zstd"
+# These select the default tls implementation / which quic impl to use
+IUSE+=" +curl_quic_openssl curl_quic_ngtcp2 curl_ssl_gnutls curl_ssl_mbedtls +curl_ssl_openssl curl_ssl_rustls"
+RESTRICT="!test? ( test )"
+
+# Only one default ssl / quic provider can be enabled
+# The default provider needs its USE satisfied
+# HTTP/3 and MultiSSL are mutually exclusive; it's not clear if MultiSSL offers any benefit at all in the modern day.
+# https://github.com/curl/curl/commit/65ece771f4602107d9cdd339dff4b420280a2c2e
+REQUIRED_USE="
+	quic? (
+		^^ (
+			curl_quic_openssl
+			curl_quic_ngtcp2
+		)
+		http3
+	)
+	ssl? (
+		^^ (
+			curl_ssl_gnutls
+			curl_ssl_mbedtls
+			curl_ssl_openssl
+			curl_ssl_rustls
+		)
+	)
+	curl_quic_openssl? (
+		curl_ssl_openssl
+		!gnutls
+		!mbedtls
+		!rustls
+	)
+	curl_quic_ngtcp2? (
+		curl_ssl_gnutls
+		!mbedtls
+		!openssl
+		!rustls
+	)
+	curl_ssl_gnutls? ( gnutls )
+	curl_ssl_mbedtls? ( mbedtls )
+	curl_ssl_openssl? ( openssl )
+	curl_ssl_rustls? ( rustls )
+	http3? ( alt-svc quic )
+"
+
+# cURL's docs and CI/CD are great resources for confirming supported versions
+# particulary for fast-moving targets like HTTP/2 and TCP/2 e.g.:
+# - https://github.com/curl/curl/blob/master/docs/INTERNALS.md (core dependencies + minimum versions)
+# - https://github.com/curl/curl/blob/master/docs/HTTP3.md (example of a feature that moves quickly)
+# - https://github.com/curl/curl/blob/master/.github/workflows/quiche-linux.yml (CI/CD for TCP/2)
+# However 'supported' vs 'works' are two entirely different things; be sane but
+# don't be afraid to require a later version.
+# ngtcp2 = https://bugs.gentoo.org/912029 - can only build with one tls backend at a time.
+RDEPEND="
+	>=sys-libs/zlib-1.1.4[${MULTILIB_USEDEP}]
+	adns? ( >=net-dns/c-ares-1.16.0:=[${MULTILIB_USEDEP}] )
+	brotli? ( app-arch/brotli:=[${MULTILIB_USEDEP}] )
+	http2? ( >=net-libs/nghttp2-1.15.0:=[${MULTILIB_USEDEP}] )
+	http3? ( >=net-libs/nghttp3-1.1.0[${MULTILIB_USEDEP}] )
+	idn? ( >=net-dns/libidn2-2.0.0:=[static-libs?,${MULTILIB_USEDEP}] )
+	kerberos? ( >=virtual/krb5-0-r1[${MULTILIB_USEDEP}] )
+	ldap? ( >=net-nds/openldap-2.0.0:=[static-libs?,${MULTILIB_USEDEP}] )
+	psl? ( net-libs/libpsl[${MULTILIB_USEDEP}] )
+	quic? (
+		curl_quic_openssl? ( >=dev-libs/openssl-3.3.0:=[quic,${MULTILIB_USEDEP}] )
+		curl_quic_ngtcp2? ( >=net-libs/ngtcp2-1.2.0[gnutls,ssl,-openssl,${MULTILIB_USEDEP}] )
+	)
+	rtmp? ( media-video/rtmpdump[${MULTILIB_USEDEP}] )
+	ssh? ( >=net-libs/libssh2-1.0.0[${MULTILIB_USEDEP}] )
+	ssl? (
+		gnutls? (
+			app-misc/ca-certificates
+			>=net-libs/gnutls-3.1.10:=[static-libs?,${MULTILIB_USEDEP}]
+			dev-libs/nettle:=[${MULTILIB_USEDEP}]
+		)
+		mbedtls? (
+			app-misc/ca-certificates
+			net-libs/mbedtls:=[${MULTILIB_USEDEP}]
+		)
+		openssl? (
+			>=dev-libs/openssl-0.9.7:=[sslv3(-)=,static-libs?,${MULTILIB_USEDEP}]
+		)
+		rustls? (
+			>=net-libs/rustls-ffi-0.13.0:=[${MULTILIB_USEDEP}]
+		)
+	)
+	zstd? ( app-arch/zstd:=[${MULTILIB_USEDEP}] )
+"
+
+DEPEND="${RDEPEND}"
+
+BDEPEND="
+	dev-lang/perl
+	virtual/pkgconfig
+	test? (
+		sys-apps/diffutils
+		http2? ( >=net-libs/nghttp2-1.15.0:=[utils,${MULTILIB_USEDEP}] )
+		http3? ( net-libs/nghttp2:=[utils,${MULTILIB_USEDEP}] )
+	)
+	verify-sig? ( sec-keys/openpgp-keys-danielstenberg )
+"
+
+DOCS=( CHANGES README docs/{FEATURES.md,INTERNALS.md,FAQ,BUGS.md,CONTRIBUTE.md} )
+
+MULTILIB_WRAPPED_HEADERS=(
+	/usr/include/curl/curlbuild.h
+)
+
+MULTILIB_CHOST_TOOLS=(
+	/usr/bin/curl-config
+)
+
+QA_CONFIG_IMPL_DECL_SKIP=(
+	__builtin_available
+	closesocket
+	CloseSocket
+	getpass_r
+	ioctlsocket
+	IoctlSocket
+	mach_absolute_time
+	setmode
+	_fseeki64
+	# custom AC_LINK_IFELSE code fails to link even without -Werror
+	OSSL_QUIC_client_method
+)
+
+PATCHES=(
+	"${FILESDIR}"/${PN}-prefix-2.patch
+	"${FILESDIR}"/${PN}-respect-cflags-3.patch
+)
+
+src_prepare() {
+	default
+
+	eprefixify curl-config.in
+	eautoreconf
+}
+
+multilib_src_configure() {
+	# We make use of the fact that later flags override earlier ones
+	# So start with all ssl providers off until proven otherwise
+	# TODO: in the future, we may want to add wolfssl (https://www.wolfssl.com/)
+	local myconf=()
+
+	myconf+=( --without-ca-fallback --with-ca-bundle="${EPREFIX}"/etc/ssl/certs/ca-certificates.crt  )
+	if use ssl; then
+		myconf+=( --without-gnutls --without-mbedtls --without-rustls )
+
+		if use gnutls; then
+			multilib_is_native_abi && einfo "SSL provided by gnutls"
+			myconf+=( --with-gnutls )
+		fi
+		if use mbedtls; then
+			multilib_is_native_abi && einfo "SSL provided by mbedtls"
+			myconf+=( --with-mbedtls )
+		fi
+		if use openssl; then
+			multilib_is_native_abi && einfo "SSL provided by openssl"
+			myconf+=( --with-ssl --with-ca-path="${EPREFIX}"/etc/ssl/certs )
+		fi
+		if use rustls; then
+			multilib_is_native_abi && einfo "SSL provided by rustls"
+			myconf+=( --with-rustls )
+		fi
+		if use curl_ssl_gnutls; then
+			multilib_is_native_abi && einfo "Default SSL provided by gnutls"
+			myconf+=( --with-default-ssl-backend=gnutls )
+		elif use curl_ssl_mbedtls; then
+			multilib_is_native_abi && einfo "Default SSL provided by mbedtls"
+			myconf+=( --with-default-ssl-backend=mbedtls )
+		elif use curl_ssl_openssl; then
+			multilib_is_native_abi && einfo "Default SSL provided by openssl"
+			myconf+=( --with-default-ssl-backend=openssl )
+		elif use curl_ssl_rustls; then
+			multilib_is_native_abi && einfo "Default SSL provided by rustls"
+			myconf+=( --with-default-ssl-backend=rustls )
+		else
+			eerror "We can't be here because of REQUIRED_USE."
+			die "Please file a bug, hit impossible condition w/ USE=ssl handling."
+		fi
+
+	else
+		myconf+=( --without-ssl )
+		einfo "SSL disabled"
+	fi
+
+	# These configuration options are organized alphabetically
+	# within each category.  This should make it easier if we
+	# ever decide to make any of them contingent on USE flags:
+	# 1) protocols first.  To see them all do
+	# 'grep SUPPORT_PROTOCOLS configure.ac'
+	# 2) --enable/disable options second.
+	# 'grep -- --enable configure | grep Check | awk '{ print $4 }' | sort
+	# 3) --with/without options third.
+	# grep -- --with configure | grep Check | awk '{ print $4 }' | sort
+
+	myconf+=(
+		$(use_enable alt-svc)
+		--enable-basic-auth
+		--enable-bearer-auth
+		--enable-digest-auth
+		--enable-kerberos-auth
+		--enable-negotiate-auth
+		--enable-aws
+		--enable-dict
+		--disable-ech
+		--enable-file
+		$(use_enable ftp)
+		$(use_enable gopher)
+		$(use_enable hsts)
+		--enable-http
+		$(use_enable imap)
+		$(use_enable ldap)
+		$(use_enable ldap ldaps)
+		--enable-ntlm
+		$(use_enable pop3)
+		--enable-rt
+		--enable-rtsp
+		$(use_enable samba smb)
+		$(use_with ssh libssh2)
+		$(use_enable smtp)
+		$(use_enable telnet)
+		$(use_enable tftp)
+		--enable-tls-srp
+		$(use_enable adns ares)
+		--enable-cookies
+		--enable-dateparse
+		--enable-dnsshuffle
+		--enable-doh
+		--enable-symbol-hiding
+		--enable-http-auth
+		--enable-ipv6
+		--enable-largefile
+		--enable-manual
+		--enable-mime
+		--enable-netrc
+		$(use_enable progress-meter)
+		--enable-proxy
+		--enable-socketpair
+		--disable-sspi
+		$(use_enable static-libs static)
+		--enable-pthreads
+		--enable-threaded-resolver
+		--disable-versioned-symbols
+		--without-amissl
+		--without-bearssl
+		$(use_with brotli)
+		--with-fish-functions-dir="${EPREFIX}"/usr/share/fish/vendor_completions.d
+		$(use_with http2 nghttp2)
+		--without-hyper
+		$(use_with idn libidn2)
+		$(use_with kerberos gssapi "${EPREFIX}"/usr)
+		--without-libgsasl
+		$(use_with psl libpsl)
+		--without-msh3
+		$(use_with http3 nghttp3)
+		$(use_with curl_quic_ngtcp2 ngtcp2)
+		$(use_with curl_quic_openssl openssl-quic)
+		--without-quiche
+		$(use_with rtmp librtmp)
+		--without-schannel
+		--without-secure-transport
+		--without-test-caddy
+		--without-test-httpd
+		--without-test-nghttpx
+		$(use_enable websockets)
+		--without-winidn
+		--without-wolfssl
+		--with-zlib
+		$(use_with zstd)
+		--with-zsh-functions-dir="${EPREFIX}"/usr/share/zsh/site-functions
+	)
+
+	if use debug; then
+		myconf+=(
+			--enable-debug
+		)
+	fi
+
+	if use test && multilib_is_native_abi && ( use http2 || use http3 ); then
+		myconf+=(
+			--with-test-nghttpx="${BROOT}/usr/bin/nghttpx"
+		)
+	fi
+
+	if [[ ${CHOST} == *mingw* ]] ; then
+		myconf+=(
+			--disable-pthreads
+		)
+	fi
+
+	ECONF_SOURCE="${S}" econf "${myconf[@]}"
+
+	if ! multilib_is_native_abi; then
+		# Avoid building the client (we just want libcurl for multilib)
+		sed -i -e '/SUBDIRS/s:src::' Makefile || die
+		sed -i -e '/SUBDIRS/s:scripts::' Makefile || die
+	fi
+
+}
+
+multilib_src_compile() {
+	default
+
+	if multilib_is_native_abi; then
+		# Shell completions
+		! tc-is-cross-compiler && emake -C scripts
+	fi
+}
+
+# There is also a pytest harness that tests for bugs in some very specific
+# situations; we can rely on upstream for this rather than adding additional test deps.
+multilib_src_test() {
+	# See https://github.com/curl/curl/blob/master/tests/runtests.pl#L5721
+	# -n: no valgrind (unreliable in sandbox and doesn't work correctly on all arches)
+	# -v: verbose
+	# -a: keep going on failure (so we see everything which breaks, not just 1st test)
+	# -k: keep test files after completion
+	# -am: automake style TAP output
+	# -p: print logs if test fails
+	# Note: if needed, we can skip specific tests. See e.g. Fedora's packaging
+	# or just read https://github.com/curl/curl/tree/master/tests#run.
+	# Note: we don't run the testsuite for cross-compilation.
+	# Upstream recommend 7*nproc as a starting point for parallel tests, but
+	# this ends up breaking when nproc is huge (like -j80).
+	# The network sandbox causes tests 241 and 1083 to fail; these are typically skipped
+	# as most gentoo users don't have an 'ip6-localhost'
+	multilib_is_native_abi && emake test TFLAGS="-n -v -a -k -am -p -j$((2*$(makeopts_jobs))) !241 !1083"
+}
+
+multilib_src_install() {
+	emake DESTDIR="${D}" install
+
+	if multilib_is_native_abi; then
+		# Shell completions
+		! tc-is-cross-compiler && emake -C scripts DESTDIR="${D}" install
+	fi
+}
+
+multilib_src_install_all() {
+	einstalldocs
+	find "${ED}" -type f -name '*.la' -delete || die
+	rm -rf "${ED}"/etc/ || die
+}
+
+pkg_postinst() {
+	if use debug; then
+		ewarn "USE=debug has been selected, enabling debug codepaths and making cURL extra verbose."
+		ewarn "Use this _only_ for testing. Debug builds should _not_ be used in anger."
+		ewarn "hic sunt dracones; you have been warned."
+	fi
+}
diff --git a/net-misc/curl/curl-9999.ebuild b/net-misc/curl/curl-9999.ebuild
index b42cca9c5152..c8446b6178ce 100644
--- a/net-misc/curl/curl-9999.ebuild
+++ b/net-misc/curl/curl-9999.ebuild
@@ -26,14 +26,16 @@ fi
 
 LICENSE="BSD curl ISC test? ( BSD-4 )"
 SLOT="0"
-IUSE="+adns +alt-svc brotli debug +ftp gnutls gopher +hsts +http2 http3 idn +imap kerberos ldap mbedtls +openssl +pop3"
-IUSE+=" +psl +progress-meter quic rtmp rustls samba +smtp ssh ssl sslv3 static-libs test telnet +tftp websockets zstd"
+IUSE="+adns +alt-svc brotli debug +ftp gnutls gopher +hsts +http2 +http3 idn +imap kerberos ldap mbedtls +openssl +pop3"
+IUSE+=" +psl +progress-meter +quic rtmp rustls samba +smtp ssh ssl sslv3 static-libs test telnet +tftp websockets zstd"
 # These select the default tls implementation / which quic impl to use
-IUSE+=" curl_quic_openssl curl_quic_ngtcp2 curl_ssl_gnutls curl_ssl_mbedtls +curl_ssl_openssl curl_ssl_rustls"
+IUSE+=" +curl_quic_openssl curl_quic_ngtcp2 curl_ssl_gnutls curl_ssl_mbedtls +curl_ssl_openssl curl_ssl_rustls"
 RESTRICT="!test? ( test )"
 
 # Only one default ssl / quic provider can be enabled
 # The default provider needs its USE satisfied
+# HTTP/3 and MultiSSL are mutually exclusive; it's not clear if MultiSSL offers any benefit at all in the modern day.
+# https://github.com/curl/curl/commit/65ece771f4602107d9cdd339dff4b420280a2c2e
 REQUIRED_USE="
 	quic? (
 		^^ (
@@ -50,8 +52,18 @@ REQUIRED_USE="
 			curl_ssl_rustls
 		)
 	)
-	curl_quic_openssl? ( openssl )
-	curl_quic_ngtcp2? ( gnutls )
+	curl_quic_openssl? (
+		curl_ssl_openssl
+		!gnutls
+		!mbedtls
+		!rustls
+	)
+	curl_quic_ngtcp2? (
+		curl_ssl_gnutls
+		!mbedtls
+		!openssl
+		!rustls
+	)
 	curl_ssl_gnutls? ( gnutls )
 	curl_ssl_mbedtls? ( mbedtls )
 	curl_ssl_openssl? ( openssl )
@@ -71,9 +83,9 @@ RDEPEND="
 	>=sys-libs/zlib-1.1.4[${MULTILIB_USEDEP}]
 	adns? ( >=net-dns/c-ares-1.16.0:=[${MULTILIB_USEDEP}] )
 	brotli? ( app-arch/brotli:=[${MULTILIB_USEDEP}] )
-	http2? ( >=net-libs/nghttp2-1.12.0:=[${MULTILIB_USEDEP}] )
+	http2? ( >=net-libs/nghttp2-1.15.0:=[${MULTILIB_USEDEP}] )
 	http3? ( >=net-libs/nghttp3-1.1.0[${MULTILIB_USEDEP}] )
-	idn? ( net-dns/libidn2:=[static-libs?,${MULTILIB_USEDEP}] )
+	idn? ( >=net-dns/libidn2-2.0.0:=[static-libs?,${MULTILIB_USEDEP}] )
 	kerberos? ( >=virtual/krb5-0-r1[${MULTILIB_USEDEP}] )
 	ldap? ( >=net-nds/openldap-2.0.0:=[static-libs?,${MULTILIB_USEDEP}] )
 	psl? ( net-libs/libpsl[${MULTILIB_USEDEP}] )