.. _whatsnew_changes_7.7:

%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Changes in Varnish-Cache 7.7
%%%%%%%%%%%%%%%%%%%%%%%%%%%%

For information about updating your current Varnish deployment to the new
version, see :ref:`whatsnew_upgrading_7.7`.

**NOTE**: In this Varnish-Cache release, we changed how timestamps are taken for
the http2 protocol, which could look like a performance regression, but is not.
See :ref:`whatsnew_changes_7.7_h2_timestamps`.

A more detailed and technical account of changes in Varnish-Cache, with links to
issues that have been fixed and pull requests that have been merged, may be
found in the `change log`_.

.. _change log: https://github.com/varnishcache/varnish-cache/blob/master/doc/changes.rst

Security
========

VSV 15:
~~~~~~~

The client connection is now always closed when a malformed request
is received.

varnishd
========

Parameters
~~~~~~~~~~

The bitfield parameters ``debug``, ``experimental``, ``feature``,
``vcc_feature`` and ``vsl_mask`` now consistently support the special values
``all`` and ``none``. The output format of ``varnishadm param.show`` has been
adjusted to always output the parameter value relative to either ``all`` or
``none`` in a format which would also be accepted by ``varnishadm param.set``
and the ``varnishd -p`` option.

The new ``http_req_overflow_status`` parameter now allows to optionally send a
response with a status between ``400`` and ``499`` (inclusive) if a request
exceeds ``http_req_size``.  The default of ``0`` keeps the existing behavior
to just close the connection in this case.

The new ``ban_any_variant`` parameter allows to configure the maximum number
of possibly non matching variants evaluated against the ban list during
lookup. The default value of 10000 avoids excessive time spent for ban checks
during lookups, which could cause noticeable delays for cases with a very high
number of bans and/or variants (in the 1000s).

Setting ``ban_any_variant`` to ``0`` changes the behavior of the lookup-time
ban check to only consider matching objects for tests against the ban list,
which can be considered a bugfix, depending on the exact interpretation of the
semantics of ban expressions with regards to variants. ``0`` will become the
new default in a future release of Varnish-Cache.

The varnishd parameters ``max_restarts`` and ``max_retries`` have been
made more consistent, which prevents a potential panic that could be
triggered when reducing the value of ``max_restarts`` under certain
conditions.

Jails
~~~~~

The ``linux`` jail gained control of transparent huge pages (THP) settings: The
``transparent_hugepage`` suboption can be set to ``ignore`` to do nothing,
``enable`` to enable THP (actually, disable the disable), ``disable`` to disable
THP or ``try-disable`` to try do disable, but not emit an error if disabling
fails. ``try-disable`` is the default.

Error handling from the jail subsystem has been streamlined to avoid some
confusing and/or contradictory error messages as well as turn assertion failures
into error messages.

Other changes in varnishd
~~~~~~~~~~~~~~~~~~~~~~~~~

An issue has been fixed which could cause a crash when ``varnishd`` receives
an invalid ``Content-Range`` header from a backend.

The management process now logs the PID of the process from which it
received a signal.

Changes to VCL
==============

VCL variables
~~~~~~~~~~~~~

Two new VCL variables ``req.filters`` and ``bereq.filters`` can now be used to
pass request bodies through a list of VFPs and backend request bodies through a
list of VDPs respectively. This can be useful for processing or transforming
request bodies as they go through varnish. See :ref:`vcl-var(7)` for more
details.

VCL now supports ``unset req.grace`` and ``unset req.ttl`` to reset the
respective variables to the "no effect" value, which is also the default.

The scope of VCL variables ``req.is_hitmiss`` and ``req.is_hitpass`` is now
restricted to ``vcl_miss, vcl_deliver, vcl_pass, vcl_synth`` and ``vcl_pass,
vcl_deliver, vcl_synth``, respectively.

The ``Content-Length`` header is now consistently removed after ``unset
bereq.body`` on the backend side.

``req.hash`` is now also readable from ``vcl_synth`` and ``vcl_pipe``.

Other changes to VCL
~~~~~~~~~~~~~~~~~~~~

Behavior of the VCL ``include`` statement with the ``+glob`` option has been
clarified to not search directories in ``vcl_path``. Using ``+glob`` includes
with a relative path that does not start with "./" will now result in a VCL
compile failure.

Validation of the ``PROXY2`` ``PP2_TYPE_AUTHORITY`` TLV sent with ``.via``
backends has been corrected: IP addresses are no longer accepted as an
authority and port numbers are automatically removed.

``return (fail(...))`` can now take strings returned from a vmod.

varnishadm
==========

* The CLI command ``backend.list -j`` now outputs IPs/port information.

Generic Logging (VSL)
=====================

affecting ``varnishlog``, ``varnishncsa`` and ``varnishtop``:

.. _whatsnew_changes_7.7_h2_timestamps:

http2 related timestamps
~~~~~~~~~~~~~~~~~~~~~~~~

Timestamps for http/2 requests have been corrected and made similar to how they
are taken for http/1.

For http/1, the start time, internally called "t_first", is taken as soon as any
part of the request (headers) is received. Previously, http/2 took it later,
possibly much later if long header lines were involved. http/2 now takes it the
same way as http/1 when the first bit of the first HEADERS frame of the request
arrives.

Timing behavior for http/1 and http/2 is different and can not be directly
compared. But with this change, the ``Timestamp`` VSL records for http/2 now at
least reflect reality better.

**NOTE** that after upgrading Varnish-Cache, processing and response times for
http/2 will now be reported as worse than before the upgrade, potentially *much*
worse. This is **NOT** a performance regression, but rather due to the corrected
timestamps, which arguably were wrong for http/2.

http2 logging
~~~~~~~~~~~~~

For http/2, normal client behavior like timeouts or closed connection was logged
with a ``SessError`` tag and ``ENHANCE_YOUR_CALM`` in additional ``Debug`` log
records. This behavior was misleading and has been corrected.

http/2 error detail reporting in ``Debug`` log records has been clarified:
Connection errors are now prefixed with ``H2CE_``, and stream errors with
``H2SE_``, respectively.

http/2 ``BogoHeader`` log records now contain the first offending byte value in
hex.

Interactive mode in varnishstat, varnishtop and varnishhist
===========================================================

Handling of curses errors in the interactive mode of ``varnishstat``,
``varnishtop`` and ``varnishhist`` has been streamlined and one wrong assertion
has been fixed, which could cause a crash with certain terminal types as set
through the ``TERM`` environment variable.

varnishncsa
===========

The ``hitmiss`` and ``hitpass`` handling indicators have been added to the
``Varnish:handling`` format of ``varnishncsa``.

``varnishncsa`` now handles headers unset and changed from VCL more
consistently: request headers are logged as they were received from the client
and as they were sent to the backend, while response headers are logged as they
were sent to the client and as they were received from the backend.

varnishstat
===========

Pressing the ``0`` key in ``varnishstat`` interactive (curses) mode now resets
averages.

The backend ``happy`` VSC bitfield is now set to all ones for backends with no
configured probe.

varnishtest
===========

``varnishtest`` can now send arbitrary http/2 settings frames and arbitrary
PROXY2 tlvs.

``varnishtest`` has been changed to always set a ``VARNISH_DEFAULT_N``
environment variable to ensure that ``varnish`` invoked from ``varnishtest``
always has a valid workdir.

Changes for developers and VMOD authors
=======================================

``miniobj.h``: Helper macros ``SIZEOF_FLEX_OBJ()`` and ``ALLOC_FLEX_OBJ()`` have
been added to facilitate use of structs with flexible array members.

The acceptor code has been refactored for basic support of pluggable acceptors.

Two fields have been added to the VMOD data registered with varnish-cache:

- ``vcs`` for Version Control System is intended as an identifier from the
  source code management system, e.g. the git revision, to identify the exact
  source code which was used to build a VMOD binary.

- ``version`` is intended as a more user friendly identifier as to which
  version of a vmod a binary represents.

The panic output and the ``debug.vmod`` CLI command output now contain these
identifiers.

Where supported by the compiler and linker, the ``vcs`` identifier is also
reachable via the ``.vmod_vcs`` section of the vmod shared object ELF file and
can be extracted, for example, using ``readelf -p.vmod_vcs <file>``

To set the version, ``vmodtool.py`` now accepts a ``$Version`` stanza in vmod
vcc files. If ``$Version`` is not present, an attempt is made to extract
``PACKAGE_STRING`` from an automake ``Makefile``, otherwise ``NOVERSION`` is
used as the version identifier.

A new facility has been added allowing transport delivery functions to disembark
the worker thread which had been handling a request's VCL code during delivery
by returning ``VTR_D_DISEMBARK`` from the ``vtr_deliver_f`` function.

This will enable future optimizations to make transport protocol code more
efficient.

To enable this facility, a new request processing step ``finish`` has been added
once delivery is complete.

*eof*
