Skip to content

Commit

Permalink
ovsdb-server: Allow user-provided config files.
Browse files Browse the repository at this point in the history 
OVSDB server maintains a temporary file with the current database
configuration for the case it is restarted by a monitor process
after a crash.  On startup the configuration from command line
arguments is stored there in a JSON format, also whenever user
changes the configuration with different UnixCtl commands, those
changes are getting added to the file.  When restarted from the
crash it reads the configuration from the file and continues
with all the necessary remotes and databases.

This change allows it to be an external user-provided file that
OVSDB server will read the configuration from.  The file can be
specified with a --config-file command line argument and it is
mutually exclusive with most other command line arguments that
set up remotes or databases, it is also mutually exclusive with
use of appctl commands that modify same configurations, e.g.
add/remove-db or add/remove-remote.

If the user wants to change the configuration of a running server,
they may change the file and call ovsdb-server/reload appctl.
OVSDB server will open a file, read and parse it, compare the
new configuration with the current one and adjust the running
configuration as needed.  OVSDB server will try to keep existing
databases and connections intact, if the change can be applied
without disrupting the normal operation.

User-provided files are not trustworthy, so extra checks were
added to ensure a correct file format.  If the file cannot be
correctly parsed, e.g. contains invalid JSON, no changes will
be applied and the server will keep using the previous
configuration until the next reload.

If config-file is provided for active-backup databases, permanent
disconnection of one of the backup databases no longer leads to
switching all other databases to 'active'.  Only the disconnected
one will transition, since all of them have their own records in
the configuration file.

With this change, users can run all types of databases within
the same ovsdb-server process at the same time.

Simple configuration may look like this:

 {
    "remotes": {
        "punix:db.sock": {},
        "pssl:6641": {
            "inactivity-probe": 16000,
            "read-only": false,
            "role": "ovn-controller"
        }
    },
    "databases": {
        "conf.db": {},
        "sb.db": {
            "service-model": "clustered"
        },
        "OVN_Northbound": {
            "service-model": "relay",
            "source": {
                "ssl:[fe:::1]:6642,ssl:[fe:::2]:6642": {
                    "max-backoff": 8000,
                    "inactivity-probe": 10000
                }
            }
        }
    }
 }

Signed-off-by: Ilya Maximets <[email protected]>
Signed-off-by: 0-day Robot <[email protected]>
  • Loading branch information
@igsilya @ovsrobot
igsilya authored and ovsrobot committed Dec 14, 2023
1 parent d097ab9 commit ce89651
Show file tree
Hide file tree
Showing 5 changed files with 513 additions and 76 deletions.
86 changes: 84 additions & 2 deletions Documentation/ref/ovsdb.7.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -155,6 +155,22 @@ standalone database, configure the server to listen on a "connection method"
that the client can reach, then point the client to that connection method.
See `Connection Methods`_ below for information about connection methods.

Open vSwitch 3.3 introduced support for configuration files via
``--config-file`` command line option. The configuration file for a server
with a **standalone** database may look like this::

{
"remotes": { "<connection method>": {} },
"databases": {
"<database file>": {
"service-model": "standalone"
}
}
}

The ``"service-model"`` key can be omitted. In this case ``ovsdb-server``
will infer the service model from the database file itself.

Active-Backup Database Service Model
------------------------------------

Expand All @@ -177,10 +193,36 @@ database file from the active server. Then use
connects to the active server. At that point, the backup server will fetch a
copy of the active database and keep it up-to-date until it is killed.

Open vSwitch 3.3 introduced support for configuration files via
``--config-file`` command line option. The configuration file for a backup
server in this case may look like this::

{
"remotes": { "<connection method>": {} },
"databases": {
"<database file>": {
"service-model": "active-backup",
"backup": true,
"source": {
"<active>": {
"inactivity-probe": <integer>,
"max-backoff": <integer>
}
}
}
}
}

All the fields in the ``"<database file>"`` description above are required.
Options for the ``"<active>"`` connection method (``"inactivity-probe"``, etc.)
can be omitted.

When the active server in an active-backup server pair fails, an administrator
can switch the backup server to an active role with the ``ovs-appctl`` command
``ovsdb-server/disconnect-active-ovsdb-server``. Clients then have read/write
access to the now-active server. Of course, administrators are slow to respond
access to the now-active server. When the ``--config-file`` is in use, the
same can be achieved by changing the ``"backup"`` value in the file and running
``ovsdb-server/reload`` command. Of course, administrators are slow to respond
compared to software, so in practice external management software detects the
active server's failure and changes the backup server's role. For example, the
"Integration Guide for Centralized Control" in the OVN documentation describes
Expand Down Expand Up @@ -236,6 +278,22 @@ To set up a clustered database, first initialize it on a single node by running
arguments, the ``create-cluster`` command can create an empty database or copy
a standalone database's contents into the new database.

Open vSwitch 3.3 introduced support for configuration files via
``--config-file`` command line option. The configuration file for a server
with a **clustered** database may look like this::

{
"remotes": { "<connection method>": {} },
"databases": {
"<database file>": {
"service-model": "clustered"
}
}
}

The ``"service-model"`` key can be omitted. In this case ``ovsdb-server``
will infer the service model from the database file itself.

To configure a client to use a clustered database, first configure all of the
servers to listen on a connection method that the client can reach, then point
the client to all of the servers' connection methods, comma-separated. See
Expand Down Expand Up @@ -505,6 +563,29 @@ server. ``<relay source>`` could contain a comma-separated list of connection
methods, e.g. to connect to any server of the clustered database.
Multiple relay servers could be started for the same relay source.

Open vSwitch 3.3 introduced support for configuration files via
``--config-file`` command line option. The configuration file for a relay
database server in this case may look like this::

{
"remotes": { "<connection method>": {} },
"databases": {
"<DB_NAME>": {
"service-model": "relay",
"source": {
"<relay source>": {
"inactivity-probe": <integer>,
"max-backoff": <integer>
}
}
}
}
}

Both the ``"service-model"`` and the ``"source"`` are required. Options for
the ``"<relay source>"`` connection method (``"inactivity-probe"``, etc.)
can be omitted.

Since the way relays handle read and write transactions is very similar
to the clustered model where "cluster" means "set of relay servers connected
to the same relay source", "follower" means "relay server" and the "leader"
Expand Down Expand Up @@ -629,7 +710,8 @@ Creating a Database

Creating and starting up the service for a new database was covered
separately for each database service model in the `Service
Models`_ section, above.
Models`_ section, above. Single ``ovsdb-server`` process may serve
any number of databases with different service models at the same time.

Backing Up and Restoring a Database
-----------------------------------
Expand Down
19 changes: 19 additions & 0 deletions Documentation/topics/ovsdb-relay.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,25 @@ started like this::
$ ...
$ ovsdb-server --remote=ptcp:6642:172.16.0.K relay:OVN_Southbound:$REMOTES

Open vSwitch 3.3 introduced support for configuration files via
``--config-file`` command line option. The configuration file for a relay
database servers in this case may look like this::

{
"remotes": { "ptcp:6642:172.16.0.X": {} },
"databases": {
"OVN_Southbound": {
"service-model": "relay",
"source": {
"$REMOTES": {}
}
}
}
}

See ``ovsdb-server(1)`` and ``Relay Service Model`` in ``ovsdb(7)`` for more
configuration options.

Every relay server could connect to any of the cluster members of their choice,
fairness of load distribution is achieved by shuffling remotes.

Expand Down
4 changes: 4 additions & 0 deletions NEWS
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,10 @@ Post-v3.2.0
from older version is supported but it may trigger more leader elections
during the process, and error logs complaining unrecognized fields may
be observed on old nodes.
* New command line option --config-file that allows a fine control over
remotes and database configuration, including setting options for
connection methods for relays and active-backup replication.
For more details see ovsdb-server(1) and ovsdb(7).
- ovs-appctl:
* 'ofproto/trace' now reports OpenFlow rules that make up a conjunctive
flow match.
Expand Down
96 changes: 91 additions & 5 deletions ovsdb/ovsdb-server.1.in
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ ovsdb\-server \- Open vSwitch database server
[\fIdatabase\fR]\&...
[\fIrelay:schema_name:remote\fR]\&...
[\fB\-\-remote=\fIremote\fR]\&...
[\fB\-\-config\-file=\fIfile\fR]
[\fB\-\-run=\fIcommand\fR]
.so lib/daemon-syn.man
.so lib/service-syn.man
Expand Down Expand Up @@ -44,6 +45,11 @@ If none of database files or relay databases is specified, the default is
initialized using, for example, \fBovsdb\-tool\fR's \fBcreate\fR,
\fBcreate\-cluster\fR, or \fBjoin\-cluster\fR command.
.PP
All types of databases can alternatively be added using a configuration
file provided via \fB\-\-config\-file\fR option. This option is mutually
exclusive with specifying \fIdatabase\fR on the command line. For a detailed
description of a configuration file format see \fBovsdb\fR(7).
.PP
This OVSDB implementation supports standalone, active-backup, relay and
clustered database service models, as well as database replication.
See the Service Models section of \fBovsdb\fR(7) for more information.
Expand Down Expand Up @@ -105,6 +111,74 @@ It is an error for \fIcolumn\fR to have another type.
.IP
To connect or listen on multiple connection methods, use multiple
\fB\-\-remote\fR options.
.IP
Alternatively, remotes can be specified in a "remotes" section of a
configuration file, if provided using \fB\-\-config\-file\fR option.
\fB\-\-config\-file\fR and \fB\-\-remote\fR options are mutually
exclusive.
.
.IP "\fB\-\-config-file=\fIfile\fR"
Specifies a configuration file for \fBovsdb\-server\fR. This \fIfile\fR
can contain connection methods and databases used by the server.
The \fIfile\fR contains a JSON object with two main elements:
.RS
.IP "\fBremotes\fR"
JSON object that contains a set of connection methods in a following format:
"\fItarget\fR": { "\fIoption\fR": \fIvalue\fR, ... }. Where \fItarget\fR
is in the same format as \fIremote\fR in \fB\-\-remote\fR option.
\fIoption\fR can be \fBmax-backoff\fR (integer), \fBinactivity-probe\fR
(integer), \fBread-only\fR (boolean), \fBrole\fR (string) or \fBdscp\fR
(integer) with their allowed \fIvalue\fRs respectively. The meaning of these
\fIoption\fRs is the same as in configuration of \fIremote\fR via a database
row with \fB\-\-remote\fR option.
.IP "\fBdatabases\fR"
JSON object that describes databases that should be added to the
\fBovsdb\-server\fR in the following format: "\fIname\fR":{ "\fIoption\fR":
\fIvalue\fR, ... }. Where \fIname\fR is either a file name of a previously
created and initialized database or a schema name in case of relay
databases. Available \fIoption\fRs are:
.RS
.IP "\fBservice-model\fR (string)"
Describes the service model of this database. One of: \fBstandalone\fR,
\fBclustered\fR, \fBactive-backup\fR or \fBrelay\fR. This option is
required for all types, except for standalone and clustered. For these
databases the service model will be inferred from the file, if not
specified explicitly. \fBovsdb-server\fR will refuse to add a database
if the specified \fBservice-model\fR doesn't match with the provided file.
.IP "\fBsource\fR (JSON object; active-backup or relay)"
Describes the connection method to the active database or to the relay
source. It is a JSON object with exactly one element in the same format
as elements of "\fBremotes\fR", except that \fBread-only\fR and \fBrole\fR
options are not applicable. E.g. \fB"source": { "unix:db.sock": {
"inactivity-probe": 10000, "max-backoff": 8000 } }\fR
.IP "\fBbackup\fR (boolean; active-backup only)"
If set to \fBtrue\fR, \fBovsdb-server\fR will use this database as a
backup for the specified \fBsource\fR. Will be served as an active
database otherwise.
.IP "\fBexclude-tables\fR (JSON array of strings; active-backup only)"
List of table names that should be excluded from replication in backup mode,
e.g. \fB"exclude-tables": [ "Table_One", "Table_Two" ]\fR.
.RE
.RE
.IP
Content of the most basic configuration file may look like this:
\fB{ "remotes": { "pssl:6640": {} }, "databases": { "conf.db": {} } }\fR
.IP
Examples of configuration files for different service models can be
found in in \fBovsdb\fR(7).
.IP
\fB\-\-config-file\fR option is mutually exclusive with \fB\-\-remote\fR
as well as with specifying \fIdatabase\fR on a command line. It is also
mutually exclusive with all the \fBActive-Backup Options\fR and all the
\fBRUNTIME MANAGEMENT COMMANDS\fR that can change the configuration of
the server in conflict with the content of the file, i.e. all the commands
that manipulate with remotes and databases. Read-only commands can still
be used.
.IP
In case of changes in the \fIfile\fR, users should run
\fBovsdb-server/reload\fR command with \fBovs-appctl\fR(8) in order for
changes to take effect.
.RE
.
.IP "\fB\-\-run=\fIcommand\fR]"
Ordinarily \fBovsdb\-server\fR runs forever, or until it is told to
Expand Down Expand Up @@ -178,6 +252,8 @@ allow the syncing options to be specified using command line options,
yet start the server, as the default, active server. To switch the
running server to backup mode, use \fBovs-appctl(1)\fR to execute the
\fBovsdb\-server/connect\-active\-ovsdb\-server\fR command.
.PP
These options are mutually exclusive with \fB\-\-config\-file\fR.
.SS "Public Key Infrastructure Options"
The options described below for configuring the SSL public key
infrastructure accept a special syntax for obtaining their
Expand Down Expand Up @@ -230,6 +306,8 @@ clients.
Adds a remote, as if \fB\-\-remote=\fIremote\fR had been specified on
the \fBovsdb\-server\fR command line. (If \fIremote\fR is already a
remote, this command succeeds without changing the configuration.)
.IP
Mutually exclusive with \fB\-\-config\-file\fR option.
.
.IP "\fBovsdb\-server/remove\-remote \fIremote\fR"
Removes the specified \fIremote\fR from the configuration, failing
Expand All @@ -241,6 +319,8 @@ configuring a \fBdb:\fIdb\fB,\fItable\fB,\fIcolumn\fR remote.
(You can remove a database source with \fBovsdb\-server/remove\-remote
\fBdb:\fIdb\fB,\fItable\fB,\fIcolumn\fR, but not individual
remotes found indirectly through the database.)
.IP
Mutually exclusive with \fB\-\-config\-file\fR option.
.
.IP "\fBovsdb\-server/list\-remotes"
Outputs a list of the currently configured remotes named on
Expand All @@ -254,6 +334,8 @@ Adds the \fIdatabase\fR to the running \fBovsdb\-server\fR. \fIdatabase\fR
could be a database file or a relay description in the following format:
\fIrelay:schema_name:remote\fR. The database file must already have been
created and initialized using, for example, \fBovsdb\-tool create\fR.
.IP
Mutually exclusive with \fB\-\-config\-file\fR option.
.
.IP "\fBovsdb\-server/remove\-db \fIdatabase\fR"
Removes \fIdatabase\fR from the running \fBovsdb\-server\fR. \fIdatabase\fR
Expand All @@ -268,6 +350,8 @@ Any public key infrastructure options specified through this database
(e.g. \fB\-\-private\-key=db:\fIdatabase,\fR... on the command line)
will be disabled until another database with the same name is added
again (with \fBovsdb\-server/add\-db\fR).
.IP
Mutually exclusive with \fB\-\-config\-file\fR option.
.
.IP "\fBovsdb\-server/list\-dbs"
Outputs a list of the currently configured databases added either through
Expand All @@ -286,6 +370,9 @@ These commands query and update the role of \fBovsdb\-server\fR within
an active-backup pair of servers. See \fBActive-Backup Options\fR,
above, and \fBActive-Backup Database Service Model\fR in
\fBovsdb\fR(7) for more information.
.PP
All \fBActive-Backup Commands\fR that change the state of \fBovsdb\-server\fR
are mutually exclusive with \fB\-\-config\-file\fR option.
.
.IP "\fBovsdb\-server/set\-active\-ovsdb\-server \fIserver"
Sets the active \fIserver\fR from which \fBovsdb\-server\fR connects through
Expand Down Expand Up @@ -324,11 +411,10 @@ Gets the tables that are currently excluded from synchronization.
Prints a summary of replication run time information. The \fBstate\fR
information is always provided, indicating whether the server is running
in the \fIactive\fR or the \fIbackup\fR mode.
When running in backup mode, replication connection status, which
can be either \fIconnecting\fR, \fIreplicating\fR or \fIerror\fR, are shown.
When the connection is in \fIreplicating\fR state, further output shows
the list of databases currently replicating, and the tables that are
excluded.
For all databases with active-backup service model, replication connection
status, which can be either \fIconnecting\fR, \fIreplicating\fR or
\fIerror\fR, are shown. When the connection is in \fIreplicating\fR state,
further output shows the tables that are currently excluded from replication.
.
.SS "Cluster Commands"
These commands support the \fBovsdb\-server\fR clustered service model.
Expand Down
Loading

0 comments on commit ce89651

Please sign in to comment.