.. _4.0_Upgrading:

Upgrading to Buildbot 4.0
=========================

Upgrading a Buildbot instance from 3.x to 4.0 may require some work to achieve.

The recommended upgrade procedure is as follows:

  - Upgrade to the last released BuildBot version in 3.x series.

  - Remove usage of the deprecated APIs.
    All usages of deprecated APIs threw a deprecation warning at the point of use.
    If the code does not emit deprecation warnings, it's in a good shape in this regard.
    You may need to run the master on a real workload in order to force all deprecated code paths to be exercised.

  - Upgrade to the latest Buildbot 4.0.x release.

  - (Optional) Upgrade to newest Buildbot 4.x.
    The newest point release will contain bug fixes and functionality improvements.

Web frontend
------------

Buildbot 4.0 replaces the AngularJS-based web frontend with a new React-based one.
In simple Buildbot installations there is nothing that needs to be done except to install compatible versions of any www plugins that are used.
The following plugins are maintained as part of Buildbot and can be upgraded seamlessly by just installing new, compatible version:

 - ``buildbot-www`` (main web frontend)
 - ``buildbot-console-view``
 - ``buildbot-grid-view``
 - ``buildbot-waterfall-view``

Custom plugins
~~~~~~~~~~~~~~

If the Buildbot installation uses plugins that are developed outside Buildbot, these will need to be rewritten to use the new Buildbot plugin APIs that expect the plugin to be written in React.
In such case the best approach is to rewrite any custom plugins into React while still using Buildbot 3.x and convert to 4.x once everything is ready.

More specifically, the recommended approach is as follows:

 - Upgrade to the last released BuildBot version in 3.x series.

 - Prepare the development environment

     - Install ``buildbot-www-react`` with the same version.

     - Install any Buildbot plugins that already have a version that is compatible with React.
       For example ``buildbot-console-view`` has a React equivalent ``buildbot-react-console-view``.

     - Add ``'base_react': {}`` key-value pair to the www plugin dictionary.
       For example, in the default installation the configuration would look like this:
       ``c['www'] = {port: 8080, plugins={'base_react': {}}}``.

       Other enabled plugins will need their keys in the dictionary changed.
       For example, the compatibility Buildbot plugins will have the following names:

         - ``console_view`` as ``react_console_view``
         - ``grid_view`` as ``react_grid_view``
         - ``waterfall_view`` as ``react_waterfall_view``

 - Rewrite any custom Buildbot plugins into React and new Buildbot plugin APIs in the development environment.

 - Replace the production setup with what was tested in the development environment section above.

 - Upgrade to Buildbot 4.x series

 - Buildbot plugins with the word ``react`` in the name are temporarily used for migration testing. After a successful migration
   to Buildbot 4.x, you should replace them with plugins without the word ``react`` in the plugin name.

     - Uninstall migration plugins

       .. code-block:: none

           pip uninstall buildbot-www-react buildbot-react-console-view buildbot-react-grid-view buildbot-react-waterfall-view  buildbot-react-wsgi-dashboards

     - Install production plugins

       .. code-block:: none

           pip install buildbot-www buildbot-console-view buildbot-grid-view buildbot-waterfall-view buildbot-react-wsgi-dashboards

     - Update the www plugin dictionary

       Only update the ones you use in your installation.

       Replace:

         .. code-block:: python

           c['www'] = dict(port=8010,
               plugins=dict('base_react': {}, react_console_view={}, react_grid_view={}, react_waterfall_view={}, react_wsgi_dashboards={}))

       With:

         .. code-block:: python

           c['www'] = dict(port=8010,
               plugins=dict(console_view={}, grid_view={}, waterfall_view={}, wsgi_dashboards={}))

GerritChangeSource and GerritEventLogPoller
-------------------------------------------

Events between ``GerritChangeSource`` and ``GerritEventLogPoller`` are no longer deduplicated.
The equivalent is setting ``GerritChangeSource`` with both SSH and HTTP APIs. The ``http_url``
should be set to ``baseURL`` of argument ``GerritEventLogPoller`` without the ``/a`` suffix included.
``http_auth`` should be set to ``auth`` argument of ``GerritEventLogPoller``.

Build status generators
-----------------------

The ``subject`` argument of ``BuildStatusGenerator`` and ``BuildSetStatusGenerator`` has been removed.
The equivalent is setting the ``subject`` argument of the message formatter.

Message formatters
------------------

The ``wantLogs`` argument to message formatters has been removed.
The equivalent is setting both ``want_logs`` and ``want_logs_content`` to the previous value of ``wantLogs``.

The ``wantSteps`` and ``wantProperties`` arguments have been renamed to ``want_steps`` and ``want_properties`` respectively.

GerritStatusPush
----------------

The ``reviewCB``, ``reviewArg``, ``startCB``, ``startArg``, ``summaryCB``, ``summaryArg``,
``builders`` , ``wantSteps``, ``wantLogs`` arguments of ``GerritStatusPush`` have been deprecated.
The upgrade strategy is as follows:

 - ``reviewCB``, ``reviewArg``, ``startCB``, ``startArg``:
   Use :bb:reportgen:`BuildStartEndStatusGenerator` report generator (``generators`` argument).
   Depending on ``reviewCB`` complexity, use :ref:`MessageFormatter` or
   :ref:`MessageFormatterFunctionRaw` message formatters. To override default handling of
   ``Verified`` and ``Reviewed`` labels, adjust extra information emitted by message formatter.
   E.g. ``{"labels": {"Verified": 1}}``.

 - ``summaryCB``, ``summaryArg``:
   Use :bb:reportgen:`BuildSetStatusGenerator` or :bb:reportgen:`BuildSetCombinedStatusGenerator`
   report generator  (``generators`` argument). Depending on ``summaryCB`` complexity,
   use :ref:`MessageFormatter` or :ref:`MessageFormatterFunctionRaw` message formatters.
   To override default handling of ``Verified`` and ``Reviewed`` labels, adjust extra
   information emitted by message formatter. E.g. ``{"labels": {"Verified": 1}}``.

 - ``builders`` - use ``builders`` argument of replacement report generator
 - ``wantSteps`` - use ``want_steps`` argument of replacement message formatter.
 - ``wantLogs`` - use ``want_logs`` argument of replacement message formatter

buildbot.util.croniter
----------------------

``buildbot.util.croniter`` module has been removed.
The replacement is ``croniter`` package from PyPI.

Migration to ``croniter`` involves ensuring that the input times are passed as time-aware ``datetime`` objects.

The original ``buildbot.util.croniter`` code always assumed the input time is in the current timezone.
The ``croniter`` package assumes the input time is in UTC timezone.


Endpoint attributes
-------------------

``buildbot.data.base.Endpoint`` no longer provides ``isRaw`` and ``isCollection`` attributes.
The equivalent in Buildbot 4.x is setting the ``kind`` attribute to ``EndpointKind.RAW`` and ``EndpointKind.COLLECTION`` respectively.

Changes to BuildStep attributes
-------------------------------

BuildBot no longer supports changing ``BuildStep`` attributes after a step is created during configuration.
Changing attributes of BuildStep instances that are not yet part of any build is most likely an error.
This is because such instances are only being used to configure a builder as a source to create real steps from.
In this scenario any attribute changes are ignored as far as build configuration is concerned.

For customizing BuildStep after an instance has already been created `set_step_arg(name, value)` function has been added.
