httpx 1.5.0 has been released.
HTTPX.get("https://gitlab.com/honeyryderchuck/httpx
<https://gitlab.com/honeyryderchuck/httpx>")
HTTPX is an HTTP client library for the Ruby programming language.
Among its features, it supports:
* HTTP/2 and HTTP/1.x protocol versions
* Concurrent requests by default
* Simple and chainable API
* Proxy Support (HTTP(S), CONNECT tunnel, Socks4/4a/5)
* Simple Timeout System
* Lightweight by default (require what you need)
And also:
* Compression (gzip, deflate, brotli)
* Streaming Requests
* Authentication (Basic Auth, Digest Auth, AWS Sigv4)
* Expect 100-continue
* Multipart Requests
* Cookies
* HTTP/2 Server Push
* H2C Upgrade
* Automatic follow redirects
* International Domain Names
* GRPC
* Circuit breaker
* WebDAV
* SSRF Filter
* Response caching
* HTTP/2 bidirectional streaming
* QUERY HTTP verb
* Datadog integration
* Faraday integration
* Webmock integration
* Sentry integration
Here are the updates since the last release:
# 1.5.0
## Features
### `:stream_bidi` plugin
The `:stream_bidi` plugin enables bidirectional streaming support (an
HTTP/2 only feature!). It builds on top of the `:stream` plugin, and uses
its block-based syntax to process incoming frames, while allowing the user
to pipe more data to the request (from the same, or another thread/fiber).
http = HTTPX.plugin(:stream_bidi)
request = http.build_request(
"POST",
"https://your-origin.com/stream",
headers: { "content-type" => "application/x-ndjson" },
body: ["{\"message\":\"started\"}\n"]
)
chunks = []
response = http.request(request, stream: true)
Thread.start do
response.each do |chunk|
handle_data(chunk)
end
end
# now send data...
request << "{\"message\":\"foo\"}\n"
request << "{\"message\":\"bar\"}\n"
# ...
You can read more about it in
https://honeyryderchuck.gitlab.io/httpx/wiki/Stream-Bidi
### `:query` plugin
The `:query` plugin adds public methods supporting the `QUERY` HTTP verb:
http = HTTPX.plugin(:query)
http.query("https://example.com/gquery", body: "foo=bar") # QUERY /gquery
....
You can read more about it in
https://honeyryderchuck.gitlab.io/httpx/wiki/Query
this functionality was added as a plugin for explicit opt-in, as it's
experimental (RFC for the new HTTP verb is still in draft).
### `:response_cache` plugin filesystem based store
The `:response_cache` plugin supports setting the filesystem as the
response cache store (instead of just storing them in memory, which is the
default `:store`).
# cache store in the filesystem, writes to the temporary directory from the
OS
http = HTTPX.plugin(:response_cache, response_cache_store: :file_store)
# if you want a separate location
http = HTTPX.plugin(:response_cache).with(response_cache_store:
HTTPX::Plugins::ResponseCache::FileStore.new("/path/to/dir"))
You can read more about it in
### `:close_on_fork` option
A new option `:close_on_fork` can be used to ensure that a session object
which may have open connections will not leak them in case the process is
forked (this can be the case of `:persistent` plugin enabled sessions which
have add usage before fork):
http = HTTPX.plugin(:persistent, close_on_fork: true)
# http may have open connections here
fork do
# http has no connections here
end
You can read more about it in
Connection Pools · honeyryder .
### `:debug_redact` option
The `:debug_redact` option will, when enabled, replace parts of the debug
logs (enabled via `:debug` and `:debug_level` options) which may contain
sensitive information, with the `"[REDACTED]"` placeholder.
You can read more about it in
Debugging · honeyryder .
### `:max_connections` pool option
A new `:max_connections` pool option (settable under `:pool_options`) can
be used to defined the maximum number **overall** of connections for a pool
("in-transit" or "at-rest"); this complements, and supersedes when used,
the already existing `:max_connections_per_origin`, which does the same per
connection origin.
HTTPX.with(pool_options: { max_connections: 100 })
You can read more about it in
Connection Pools · honeyryder .
### Subplugins
An enhancement to the plugins architecture, it allows plugins to define
submodules ("subplugins") which are loaded if another plugin is in use, or
is loaded afterwards.
You can read more about it in
Custom Plugins · honeyryder .
## Improvements
* `:persistent` plugin: several improvements around reconnections of
failure:
* reconnections will only happen for "connection broken" errors (and will
discard reconnection on timeouts)
* reconnections won't exhaust retries
* `:response_cache` plugin: several improements:
* return cached response if not stale, send conditional request otherwise
(it was always doing the latter).
* consider immutable (i.e. `"Cache-Control: immutable"`) responses as
never stale.
* `:datadog` adapter: decorate spans with more tags (header, kind,
component, etc...)
* timers operations have been improved to use more efficient algorithms and
reduce object creation.
## Bugfixes
* ensure that setting request timeouts happens before the request is
buffered (the latter could trigger a state transition required by the
former).
* `:response_cache` plugin: fix `"Vary"` header handling by supporting a
new plugin option, `:supported_vary_headers`, which defines which headers
are taken into account for cache key calculation.
* fixed query string encoded value when passed an empty hash to the
`:query` param and the URL already contains query string.
* `:callbacks` plugin: ensure the callbacks from a session are copied when
a new session is derived from it (via a `.plugin` call, for example).
* `:callbacks` plugin: errors raised from hostname resolution should bubble
up to user code.
* fixed connection coalescing selector monitoring in cases where the
coalescable connecton is cloned, while other branches were simplified.
* clear the connection write buffer in corner cases where the remaining
bytes may be interpreted as GOAWAY handshake frame (and may cause
unintended writes to connections already identified as broken).
* remove idle connections from the selector when an error happens before
the state changes (this may happen if the thread is interrupted during name
resolution).
## Chore
`httpx` makes extensive use of features introduced in ruby 3.4, such as
`Module#set_temporary_name` for otherwise plugin-generated anonymous
classes (improves debugging and issue reporting), or
`String#append_as_bytes` for a small but non-negligible perf boost in
buffer operations. It falls back to the previous behaviour when used with
ruby 3.3 or lower.
Also, and in preparation for the incoming ruby 3.5 release, dependency of
the `cgi` gem (which will be removed from stdlib) was removed.
# 1.4.4
## Improvements
* `:stream` plugin: response will now be partially buffered in order to
i.e. inspect response status or headers on the response body without
buffering the full response
* this fixes an issue in the `down` gem integration when used with the
`:max_size` option.
* do not unnecessarily probe for connection liveness if no more requests
are inflight, including failed ones.
* when using persistent connections, do not probe for liveness right after
reconnecting after a keep alive timeout.
## Bugfixes
* `:persistent` plugin: do not exhaust retry attempts when probing for (and
failing) connection liveness.
* since the introduction of per-session connection pools, and
consequentially due to the possibility of multiple inactive connections for
the same origin being in the pool, which may have been terminated by the
peer server, requests would fail before being able to establish a new
connection.
* prevent retrying to connect the TCP socket object when an SSLSocket
object is already in place and connecting.
# 1.4.3
## Bugfixes
* `webmock` adapter: reassign headers to signature after callbacks are
called (these may change the headers before virtual send).
* do not close request (and its body) right after sending, instead only on
response close
* prevents retries from failing under the `:retries` plugin
* fixes issue when using `faraday-multipart` request bodies
* retry request with HTTP/1 when receiving an HTTP/2 GOAWAY frame with
`HTTP_1_1_REQUIRED` error code.
* fix wrong method call on HTTP/2 PING frame with unrecognized code.
* fix EOFError issues on connection termination for long running
connections which may have already been terminated by peer and were wrongly
trying to complete the HTTP/2 termination handshake.
# 1.4.2
## Bugfixes
* faraday: use default reason when none is matched by
Net::HTTP::STATUS_CODES
* native resolver: keep sending DNS queries if the socket is available, to
avoid busy loops on select
* native resolver fixes for Happy Eyeballs v2
* do not apply resolution delay if the IPv4 IP was not resolved via DNS
* ignore ALIAS if DNS response carries IP answers
* do not try to query for names already awaiting answer from the resolver
* make sure all types of errors are propagated to connections
* make sure next candidate is picked up if receiving NX_DOMAIN_NOT_FOUND
error from resolver
* raise error happening before any request is flushed to respective
connections (avoids loop on non-actionable selector termination).
* fix "NoMethodError: undefined method `after' for nil:NilClass", happening
for requests flushed into persistent connections which errored, and were
retried in a different connection before triggering the timeout callbacks
from the previously-closed connection.
## Chore
* Refactor of timers to allow for explicit and more performant single timer
interval cancellation.
* default log message restructured to include info about process, thread
and caller.
# 1.4.1
## Bugfixes
* several `datadog` integration bugfixes
* only load the `datadog` integration when the `datadog` sdk is loaded
(and not other gems that may define the `Datadog` module, like `dogstatsd`)
* do not trace if datadog integration is loaded but disabled
* distributed headers are now sent along (when the configuration is
enabled, which it is by default)
* fix for handling multiple `GOAWAY` frames coming from the server (node.js
servers seem to send multiple frames on connection timeout)
* fix regression for when a url is used with `httpx` which is not `http://`
or `https://` (should raise `HTTPX::UnsupportedSchemaError`)
* worked around `IO.copy_stream` which was emitting incorrect bytes for
HTTP/2 requests which bodies larger than the maximum supported frame size.
* multipart requests: make sure that a body declared as `Pathname` is
opened for reading in binary mode.
* `webmock` integration: ensure that request events are emitted (such as
plugins and integrations relying in it, such as `datadog` and the OTel
integration)
* native resolver: do not propagate successful name resolutions for
connections which were already closed.
* native resolver: fixed name resolution stalling, in a multi-request to
multi-origin scenario, when a resolution timeout would happen.
## Chore
* refactor of the happy eyeballs and connection coalescing logic to not
rely on callbacks, and instead on instance variable management (makes code
more straightforward to read).