httpx 1.8.0 has been released.
HTTPX.get("https://gitlab.com/honeyryderchuck/httpx
<https://gitlab.com/honeyryderchuck/httpx>")
HTTPX is an HTTP client library for the Ruby programming language.
Among its features, it supports:
* HTTP/2 and HTTP/1.x protocol versions
* Concurrent requests by default
* Simple and chainable API
* Proxy Support (HTTP(S), CONNECT tunnel, Socks4/4a/5)
* Simple Timeout System
* Lightweight by default (require what you need)
And also:
* Compression (gzip, deflate, brotli)
* Streaming Requests
* Authentication (Basic Auth, Digest Auth, AWS Sigv4)
* Expect 100-continue
* Multipart Requests
* Cookies
* HTTP/2 Server Push
* H2C Upgrade
* Automatic follow redirects
* International Domain Names
* GRPC
* Circuit breaker
* WebDAV
* SSRF Filter
* Response caching
* HTTP/2 bidirectional streaming
* QUERY HTTP verb
* Server Sent Events
* Tracing
* Request Hedging
* Datadog integration
* Faraday integration
* Webmock integration
* Sentry integration
Here are the updates since the last release:
# 1.9.0
## Features
### `:hedging` plugin
The `:hedging` plugin implements a strategy called "request hedging",
whereby a request is sent two times, over different connections, and
picking up whichever arrives first.
http = HTTP.plugin(:hedging)
http.get("http://www.example.com")
# sent two times, only the fastest one comes back...
You can read more about it in Hedging · Wiki · OS / httpx · GitLab .
### `:proxy/http` plugin: new options `:proxy_headers` and
`:enable_proxy_tunnel`
The `:proxy/http` plugin (which gets automatically loaded by the `:proxy`
plugin) gains two new options.
#### `:proxy_headers`
Can be passed a separate set of HTTP headers, which are going to be sent to
the proxy when establishing the CONNECT proxy tunnel.
http = HTTPX.plugin(:proxy).with(proxy_headers: { "x-proxy-1" => "foo" },
headers: { "x-header-1" => "bar" }).get("https://example.com")
# -> CONNECT example.com HTTP/1.1
# -> Host: example.com
# -> X-Proxy-1: foo
# ...
# <- 200 OK
# ...
# -> GET / HTTP/1.1
# -> X-Headers-1: bar
# ...
#### `:enable_proxy_tunnel`
When set to `true`, it forces the establishment of a CONNECT proxy tunnel
before sending the request. Bear in mind that this will be the norm for
`https://` requests, so only use this in cases where the requests are to be
sent to an `http://` URL and a CONNECT proxy tunnel is required.
## Improvements
* `:request_timeout` will also observe the connect phase, in case a request
is sent to an idle connection. This means that, if you're using
`:request_timeout`, you no longer need to specify `:connect_timeout`.
* The same will happen if you're using `:total_request_timeout`.
* do not require "io/wait" anymore for ruby 3.2 or higher.
* `:proxy/http` plugin: raise a `HTTPX::ConnectTimeoutError` if there was a
timeout error while waiting for the CONNECT request to complete.
## Bugfixes
* `:auth` plugin: fixed clause inverted check, which caused a new token to
always be generated on retry.
* `:auth` plugin: take auth_header_expires_at into account when deciding to
regenerate on retry (token may have expired by the time the request will be
retried).
* `:tracing` plugin: fixed request initial time calculation in order to
take into account the time to wait for a PING correctly.
* `:tracing` plugin: fixed request initial time calculation for pending
requests which were waiting for the connect phase and errored meanwhile.
* `:proxy/http` plugin: `:headers` will not be sent anymore to the proxy
when establishing the CONNECT proxy tunnel.
* socket connect timeout bookkeeping is fixed, where in a multi request
scenario, connect timeout errors shall not be raised too early if an error
happens before the remainder of connections elapses.
* `:proxy` plugin: allow proxy params to be frozen, which also fixes an
issue where proxies identified as unhealthy by a previous request, would
still be retried on for different hostnames.
* `:callbacks` plugin: fix broken visibility of session `#close` method.
* native resolver: retry dns queries on not-imp and refused errors (treats
them as transient resolver errors).
* `webmock` adapter: fix emission of `:response` callback so that this only
happens after the body is fully buffered (which makes the `:callbacks`
plugin `.on_response_completed` callback works the same way when and when
not stubbed).
* `:addresses`: fix support for the documented variant where a hostname/ips
hash is passed.
* `http2`: fix interests calculation, so that:
* a connection expects to read from connections which still have
in-transit streams.
* an HTTP/2 with no available flow control window in its streams will
wait for incoming WINDOW_UPDATE frames.
* `http2`: pause buffering the request body if there's no available control
window in the corresponding stream or connection.
* `http2`: reset the ping timer too if a connection receives a GOAWAY frame
while waiting for a PING to be ack'ed.
* wait for read or write for any operation, considering that
`OpenSSL::SSL::SSLSocket` objects may return both `.wait_readable` and
`:wait_writable` for reads and writes.
* when using `:max_requests`, also account for inflight requests when
deciding to send it to the parser or the pending queue.
* fixed `HTTPX::Options#merge` which was not freezing the resulting options
object, something which happens when enabling plugins, which made resulting
sessions unusable across Ractors.
* fixed the `Alt-Svc` header parser to be able to parse values with
multiple authorities.
* fixed request params handling so that they are passed to all requests
invoked (instead of only the first of them).
## Chore
* Emit a warning message whenever the connection is deactivated with
pending requests; the message contains an internal state which should help
debug it.
# 1.8.4
## Improvements
* send requests which were not sent, or were not processed by the server,
when the server sends back a GOAWAY frame (which is deemed recoverable,
like when there's no error), after forcing a reconnection (instead of
bubbling up the GOAWAY error for all of them).
## Bugfixes
* `:retries` plugin: also retry requests which failed due to a RST_STREAM
frame with REFUSED_STREAM error code.
* `:retries` plugin: resend requests which were piped to a closed
connection internally.
* `:proxy` plugin: fix issue related with reconnecting inactive connections
shut down by the peer, when used with the `:persistent` plugin, where the
CONNECT request wasn't being sent to establish the tunnel.
* fixed compatibility issue starting with the `json` gem 3.0.0, when
calling `HTTPX::Response#json`.
* avoid bookkeeping issue in session, where the `on_response_arrived` on a
request may be called more than once (which it can happen in the
`:follow_redirects` plugin, where a request may have completed due to an
error, while the request is waiting to be resent due to `retry-after`).
# 1.8.3
## Bugfixes
* options print a warning instead of raising an error on unknown option
* The approach introduced in the previous version, of erroring on
something people have been accidentally relying on, is too strict in
hindsight for a patch version. This was reverted, and instead, a
deprecation warning will be printed, to give time for folks to migrate.
* The format of option messages (exceptions and warnings) also slightly
changes to wrap the option in quotes, for proper highlighting.
# 1.8.2
## Improvements
* `:proxy` plugin: allow setting `:no_proxy` option to `"*"` (or `["*"]`)
to allow bypassing proxy for all requests.
* Extending sessions with unknown options will now raise an error, instead
of silently ignoring them (consistent with the initialization case now):
session = HTTPX.with(unknown_option: 1) # will raise a HTTPX::Error:
unknown option: unknown_option
* Use `HTTP2::Connection#closed?` internally (instead of `#state`) to check
for connection state; this will, in a future release of `http-2`,
seamlessly support the new `:closing` state.
* Add missing `response` attribute reader to `HTTPX::RequestTimeoutError`.
* Improve the correctness of calling `#dup` on requests and connections.
## Bugfixes
* `:auth` plugin: generate token first, then set the "expires_at" timestamp
(bearer tokens, specially if JWT, may carry information about when it
expires).
* `:auth` plugin: do not check for token freshness on a retry, instead
force token regeneration.
* `:retries` plugin: do not apply "retry after" delay after a ping timeout
error (the ping timeout already implies a delay, so retry on a new
connection should be immediate).
* `:retries` plugin: do not retry on any HTTP/2 frame error (only on GOAWAY
or PING frame errors).
* `:persistent` plugin: do not decrement request remaining retries on ping
or GOAWAY frame errors.
* `:tracing` plugin: set connection initialization time to nil no
termination (so that it doesn't get reused on persistent connections after
reuse).
* `:follow_redirects` plugin: fixed issue when a timeout error would be
raised but only bubbles up to the root request, not properly closing the
in-flight follow up request, and leaving the request handling phase in a
broken state.
* datadog adapter: the connection initialization time is now set only on
connect or first request received, the reason being, making it more
accurate.
* webmock adapter: `.to_timeout` helper will make the request raise a
`HTTPX::RequestTimeoutError` instead of a plain `TimeoutError` (otherwise,
if using persistent sessions, it'd trigger a retry instead and mess with
webmock request count bookkeeping).
* native resolver: do not transition to `:open` if the socket was closed
during resolution (which happens when an error is raised while the query
bytes are buffered).
* fix for when setting a new session callback on an ssl context object
which has been frozen after the first connection attempt (done by binding
the ssl context callback to an internal ivar which can be changed).
* connection: protect against `Errno::ETIMEDOUT` errors on socket close by
forcing transition to closed.
* do not set a ping timer for HTTP/1.1 on keep-alive flow, as there's no
ping and the connection is reestablished.
# 1.8.1
## Improvements
* the session will now eagerly write initial data to connections instead of
waiting for I/O the first time (saves a syscall).
* removed all rescue blocks rescuing `StandardError` internally; this could
interfere with usages of `httpx` inside `Timeout.timeout` blocks, which may
emit an exception at any point, including in blocks where that exception
could be rescued, leaving the session in a potentially irrecoverable state
and looping.
* debug logs change the display of ids from 'tid:123, fid:123' to 'tid=123
fid:123'.
* and when using the `datadog` adapter, all related correlation ids
(trace id, span id, etc) will also be added to the identifiers in the same
format
* plugins built on top of the `:retries` plugin will skip the `retry_after`
interval when certain conditions are met.
* `:persistent` plugin: request will be retried immediately, when the
error happens after the keep alive interval expired.
* `:auth` plugin: request will be retried immediately if it failed due to
a "401 Unauthorized Error".
* `:ssrf_filter` plugin: removed public/private references of exception
message.
## Bugfixes
* `:auth` plugin: handle edge case where `:auth_header_expires_at` returns
no value.
* `:digest_auth` plugin: do not retry if there was an issue building the
authorization header after the probe.
* `webmock` adapter: prevent mocked requests from opening a connection to
the peer server that will not be used.
* clear and cancel request timers on completion or reset, ensuring that
they won't be retained way past the point where the request is gone.
* connection: cancel and discard ping timer before closing (removes timer
reference)
* connection: fix logic where purging-while-resending requests to a
coalesced connection was leaving references in the old connection.
* moved an `on_body_chunk` callback to the `:callbacks` plugin, where it's
used and expected to be defined.
* `http2`: avoid closing the connection on internal errors which are
supposed to be specific to a request, such as the max headers or max
response body size checks.
* system resolver will use an internal exception when using
`Timeout.timeout` on resolutions to avoid conflicts with the default
timeout exception class.
* prevent edge case where `#alpn_protocol` was being called on a
`TCPSocket`.
* fixed v1.8.0 regression causing connection to loop when an error happens
while connecting.
* resolver cache: fixed cache eviction logic, where alias answers weren't
being evicted, causing resolutions to stall after a certain number of
resolutions.