CLUSTERER Module

Marius Cristian Eseanu

   OpenSIPS Solutions

Edited by

Robert-Vladut Patrascu

   Copyright © 2015 - 2017 www.opensips-solutions.com
     __________________________________________________________

   Table of Contents

   1. Admin Guide

        1.1. Overview
        1.2. Dependencies

              1.2.1. OpenSIPS Modules
              1.2.2. External Libraries or Applications

        1.3. Exported Parameters

              1.3.1. db_url
              1.3.2. db_table
              1.3.3. id_col
              1.3.4. cluster_id_col
              1.3.5. node_id_col
              1.3.6. url_col
              1.3.7. state_col
              1.3.8. no_ping_retries_col
              1.3.9. priority_col
              1.3.10. sip_addr_col
              1.3.11. description_col
              1.3.12. current_id
              1.3.13. ping_interval
              1.3.14. ping_timeout
              1.3.15. node_timeout
              1.3.16. db_mode
              1.3.17. current_info
              1.3.18. neighbor_info
              1.3.19. sync_packet_max_size

        1.4. Exported Functions

              1.4.1. cluster_send_req(cluster_id, dst_id, msg[,
                      tag])

              1.4.2. cluster_send_rpl(cluster_id, dst_id, msg,
                      tag)

              1.4.3. cluster_broadcast_req(cluster_id, msg[, tag])

              1.4.4. cluster_check_addr(cluster_id, ip, addr_type)

        1.5. Exported MI Functions

              1.5.1. clusterer_reload
              1.5.2. clusterer_list
              1.5.3. clusterer_list_topology
              1.5.4. clusterer_set_status
              1.5.5. cluster_send_mi
              1.5.6. cluster_broadcast_mi
              1.5.7. clusterer_list_cap

        1.6. Exported Events

              1.6.1. E_CLUSTERER_REQ_RECEIVED
              1.6.2. E_CLUSTERER_RPL_RECEIVED

        1.7. Usage Example

   2. Developer Guide

        2.1. Available Functions

              2.1.1. get_nodes(cluster_id)
              2.1.2. free_nodes(list)
              2.1.3. set_state(cluster_id, state)
              2.1.4. check_addr(cluster_id, su)
              2.1.5. get_my_id()
              2.1.6. send_to(packet, cluster_id, node_id)
              2.1.7. send_all(packet, cluster_id)
              2.1.8. get_next_hop(cluster_id, node_id)
              2.1.9. free_next_hop(next_hop)
              2.1.10. register_module(mod_name, cb, auth_check,
                      accept_clusters_ids, no_accept_clusters)

   List of Examples

   1.1. Set db_url parameter
   1.2. Set db_table parameter
   1.3. Set id_col parameter
   1.4. Set cluster_id_col parameter
   1.5. Set node_id_col parameter
   1.6. Set url_col parameter
   1.7. Set state_col parameter
   1.8. Set no_ping_retries_col parameter
   1.9. Set priority_col parameter
   1.10. Set sip_addr_col parameter
   1.11. Set description_col parameter
   1.12. Set current_id parameter
   1.13. Set ping_interval parameter
   1.14. Set ping_timeout parameter
   1.15. Set node_timeout parameter
   1.16. Set db_mode parameter
   1.17. Set current_info parameter
   1.18. Set neighbor_info parameter
   1.19. Set sync_packet_max_size parameter
   1.20. cluster_send_req() usage
   1.21. cluster_send_rpl() usage
   1.22. cluster_broadcast_req() usage
   1.23. cluster_check_addr() usage
   1.24. clusterer_list usage
   1.25. clusterer_list_topology usage
   1.26. clusterer_list_cap usage
   1.27. Example database content - clusterer table
   1.28. Node A configuration
   1.29. Node B configuration

Chapter 1. Admin Guide

1.1. Overview

   The clusterer module is used to organize multiple OpenSIPS
   instances into groups(clusters) in which the nodes can
   communicate with each other in order to replicate, share
   information or perform distributed tasks. The distributed logic
   is performed either by different modules that use the clusterer
   interface (i.e. the dialog module can replicate
   dialogs/profiles, the ratelimit module can share pipes across
   multiple instances etc.) or at the script level. The clusterer
   module itself only provides an interface to send/receive BIN
   packets and get notifications about node availability. It
   achieves this by internally learning the cluster topology and
   state of the nodes. Provisioning the nodes within a cluster is
   done over the database or through the configuration script. The
   node-related information can be checked and triggered to be
   reloaded by sending commands over the MI interface.

   The topology established by the clusterer module is an overlay
   of nodes where the "links" represent communication availability
   at BIN interface level. For this purpose, a probing mechanism
   is used, consisting of regular pings to all nodes in a cluster
   for which replies must be received within a given interval. All
   nodes in the cluster exchange information about the state of
   their links with other nodes and compute a "routing table"
   which gives a next hop for each destination. The metric for the
   shortest path is the number of hops. When there is no direct
   link to a destination, the BIN packet sent by a module is
   transparently routed through the cluster.

   Note that an OpenSIPS instance can belong to multiple clusters,
   communicating and establishing the topology separately for each
   one. In order to provision this in the database or the script,
   each node has an unique ID at global level, which can be
   referenced in each cluster.

   An OpenSIPS instance can dynamically learn all the nodes in the
   cluster that are not provisioned in the database. For instances
   that do not load any node info from the DB, it is enough to
   define at least one neigbour in the script in order to discover
   all the cluster components.

1.2. Dependencies

1.2.1. OpenSIPS Modules

   The following modules must be loaded before this module:
     * a database module.
     * proto_bin module.

1.2.2. External Libraries or Applications

   The following libraries or applications must be installed
   before running OpenSIPS with this module loaded:
     * None.

1.3. Exported Parameters

1.3.1. db_url

   The database url.

   Default value is “NULL”.

   Example 1.1. Set db_url parameter
...
modparam("clusterer", "db_url",
        "mysql://opensips:opensipsrw@localhost/opensips")
...

1.3.2. db_table

   The name of the table storing the clustering information.

   Default value is “clusterer”.

   Example 1.2. Set db_table parameter
...
modparam("clusterer", "db_table", "clusterer")
...

1.3.3. id_col

   The name of the column storing an id for the table rows.

   Default value is “id”.

   Example 1.3. Set id_col parameter
...
modparam("clusterer", "id_col", "id")
...

1.3.4. cluster_id_col

   The name of the column to store the id of a cluster.

   Default value is “cluster_id”.

   Example 1.4. Set cluster_id_col parameter
...
modparam("clusterer", "cluster_id_col", "cluster_id")
...

1.3.5. node_id_col

   The name of the column to store the id of an instance. The
   values must be greater than 0.

   Default value is “node_id”.

   Example 1.5. Set node_id_col parameter
...
modparam("clusterer", "node_id_col", "node_id")
...

1.3.6. url_col

   The name of the column containing the instance url. The values
   must be greater than 0.

   Default value is “url”.

   Example 1.6. Set url_col parameter
...
modparam("clusterer", "url_col", "url")
...

1.3.7. state_col

   The name of the column storing the state of the
   node(enabled/disabled).

   Default value is “state”.

   Example 1.7. Set state_col parameter
...
modparam("clusterer", "state_col", "state")
...

1.3.8. no_ping_retries_col

   The name of the column containing the maximum number of ping
   retries before the link with the neighbour node is considered
   down.

   Default value is “no_ping_retries”.

   Example 1.8. Set no_ping_retries_col parameter
...
modparam("clusterer", "no_ping_retries_col", "no_ping_retries")
...

1.3.9. priority_col

   The name of the column storing the node priority to be chosen
   as next hop in case of same length(number of hops) paths when
   rerouting messages.

   Default value is “priority”.

   Example 1.9. Set priority_col parameter
...
modparam("clusterer", "priority_col", "priority")
...

1.3.10. sip_addr_col

   The name of the column containing a SIP address for the node.

   Default value is “sip_addr”.

   Example 1.10. Set sip_addr_col parameter
...
modparam("clusterer", "sip_addr_col", "sip_addr")
...

1.3.11. description_col

   The name of the column containing a node description.

   Default value is “description”.

   Example 1.11. Set description_col parameter
...
modparam("clusterer", "description_col", "description")
...

1.3.12. current_id

   The id of the current instance. This parameter must be equal
   with one of the node_id fields in the database.

   No default value. This parameter must be explicitly set to a
   value greater than zero.

   Example 1.12. Set current_id parameter
...
modparam("clusterer", "current_id", 1)
...

1.3.13. ping_interval

   The interval in seconds between regular pings sent to a
   neighbour node.

   Default value is “4”

   Example 1.13. Set ping_interval parameter
...
modparam("clusterer", "ping_interval", 1)
...

1.3.14. ping_timeout

   The time in milliseconds to wait for a reply to a previously
   sent ping before retrying or considering the link with the
   neighbour node down. This is also the interval between
   successive retries if the send fails.

   Default value is “1000”

   Example 1.14. Set ping_timeout parameter
...
modparam("clusterer", "ping_timeout", 500)
...

1.3.15. node_timeout

   The time in seconds to wait before pinging is restarted for a
   failed node.

   Default value is “60”

   Example 1.15. Set node_timeout parameter
...
modparam("clusterer", "node_timeout", 10)
...

1.3.16. db_mode

   Specifies whether the node specifciation information for the
   current instance, as well as the other instances in the
   cluster, should be loaded from the database or configured in
   the script. A value of “0” means that the DB is not used.

   Default value is “1”

   Example 1.16. Set db_mode parameter
...
modparam("clusterer", "db_mode", 0)
...

1.3.17. current_info

   Node specification similar to the information provided by a row
   in the clusterer DB table corresponding to the current
   instance. At least the cluster_id and url propreties should be
   defined. This parameter can be set multiple times in order to
   include the current node in multiple clusters.

   Parameter format: multiple "prop=value" property definitions
   separated by ',' where the name of the properties is the same
   as the DB column names.

   This parameter is mandatory if db_mode is set to “0”.

   Example 1.17. Set current_info parameter
...
modparam("clusterer", "current_info", "cluster_id=1, url=bin:192.168.0.5
:5566")
...

1.3.18. neighbor_info

   Node specification similar to the information provided by a row
   in the clusterer DB table corresponding to another instance in
   the current cluster. At least the cluster_id, node_id and url
   propreties should be defined. This parameter can be set
   multiple times.

   Parameter format: multiple "prop=value" property definitions
   separated by ',' where the name of the properties is the same
   as the DB column names.

   This parameter should be set at least once if db_mode is set to
   0 in order to properly learn and advertise the cluster
   topology.

   Example 1.18. Set neighbor_info parameter
...
modparam("clusterer", "neighbor_info", "cluster_id=1,node_id=2,url=bin:1
92.168.0.6:5566")
...

1.3.19. sync_packet_max_size

   The maximum size of the BIN packets sent while doing data
   synchronization. This is only a suggested value as the actual
   size of the packets may be slightly larger.

   Default value is “65535”.

   Example 1.19. Set sync_packet_max_size parameter
...
modparam("clusterer", "sync_packet_max_size", 32765)
...

1.4. Exported Functions

1.4.1.  cluster_send_req(cluster_id, dst_id, msg[, tag])

   This function is used to send a generic, request-like message,
   containing custom data, to a specific node in a cluster,
   directly from the script. The message is not a "request" per se
   but according to the logic on the receiving side, that node can
   send back a reply. In order to correlate a received reply with
   the request sent out, the function returns, through the tag
   parameter, a randomly generated communication tag, which is
   sent along in the the original message, that can be checked
   against the tag received in a reply.

   Meaning of the parameters is as follows:
     * cluster_id - the cluster ID of the destination node;
     * dst_id - the ID of the destiantion node;
     * msg - actual message payload;
     * tag - randomly generated communication tag. This is an
       optional output parameter and must be a variable if
       provided.

   The function can return the following values:
     * 1 - successfuly sent message to destination node or a valid
       next hop
     * -1 - current node is disabled so sending is impossbile
     * -2 - destination node is not reachable through any path
       according to the discovered topology
     * -3 - destination node or valid next hop appear to be
       reachable but send failed or other OpenSIPS internal error

   This function can be used from REQUEST_ROUTE, FAILURE_ROUTE,
   ONREPLY_ROUTE, BRANCH_ROUTE, LOCAL_ROUTE and EVENT_ROUTE.

   Example 1.20. cluster_send_req() usage
...
# send a request
cluster_send_req("1", "1", "Check USER: $fU", "$var(req_tag)");
# wait for reply
$avp(filter) = "tag=" + $var(req_tag);
async(wait_for_event("E_CLUSTERER_RPL_RECEIVED", "$avp(filter"), "5"), r
pl_resume);
# done
...
route[rpl_resume] {
  xlog("Received reply: $avp(msg)\n");
}
...

1.4.2.  cluster_send_rpl(cluster_id, dst_id, msg, tag)

   This function is used to send a generic, reply-like message,
   containing custom data, to a specific node in a cluster,
   directly from the script. The message is marked as a "reply" so
   this function should ony be used for replying to a previously
   request-like message received. In order for the other node,
   which initially sent a request, to be able to correlate it with
   this reply, a communication tag, received along with the
   request, should be passed to the function.

   Meaning of the parameters is as follows:
     * cluster_id - the cluster ID of the destination node;
     * dst_id - the ID of the destiantion node;
     * msg - actual message payload;
     * tag - communication tag. This is an input parameter and
       must be a variable.

   The function can return the following values:
     * 1 - successfuly sent message to destination node or a valid
       next hop
     * -1 - current node is disabled so sending is impossbile
     * -2 - destination node is not reachable through any path
       according to the discovered topology
     * -3 - destination node or valid next hop appear to be
       reachable but send failed or other OpenSIPS internal error

   This function can be used from REQUEST_ROUTE, FAILURE_ROUTE,
   ONREPLY_ROUTE, BRANCH_ROUTE, LOCAL_ROUTE and EVENT_ROUTE.

   Example 1.21. cluster_send_rpl() usage
...
event_route[E_CLUSTERER_REQ_RECEIVED] {
  fetch_event_params("$avp(cl_id);$avp(src_id);$avp(rcv_msg);$avp(rcv_ta
g)");
  ...
  cluster_send_rpl("$avp(cl_id)", "$avp(src_id)", "$var(my_reply)", "$av
p(rcv_tag)");
}
...

1.4.3.  cluster_broadcast_req(cluster_id, msg[, tag])

   This function has a similar behaviour to the cluster_send_req()
   function with the exception that the message is sent to all the
   nodes in the specified cluster.

   The function can return the following values:
     * 1 - successfuly sent message to at least one node;
     * -1 - current node is disabled so sending is impossbile;
     * -2 - all nodes in the cluster are unreachable according to
       the discovered topology;
     * -3 - send failed for all nodes in the cluster or other
       OpenSIPS internal error.

   The meaning of the parameters is the same as for
   cluster_send_req().

   This function can be used from REQUEST_ROUTE, FAILURE_ROUTE,
   ONREPLY_ROUTE, BRANCH_ROUTE, LOCAL_ROUTE and EVENT_ROUTE.

   Example 1.22. cluster_broadcast_req() usage
...
cluster_broadcast_req("$var(cl_id)", "$var(share_data)");
...

1.4.4.  cluster_check_addr(cluster_id, ip, addr_type)

   This function checks whether the given IP address belongs to
   one of the nodes in the cluster.

   The addr_type parameter can be set to in order to select the
   address of the node that the comparison is made against, with
   the possible values of:
     * bin - node's BIN interface listener;
     * sip - node's DB provisioned SIP address.

   The addr_type parameter is optional and the default behaviour
   is to compare against the SIP address.

   This function can be used from REQUEST_ROUTE, FAILURE_ROUTE,
   ONREPLY_ROUTE, BRANCH_ROUTE, LOCAL_ROUTE and EVENT_ROUTE.

   Example 1.23. cluster_check_addr() usage
...
if (cluster_check_addr("1", "$si")) {
        ...
}
...

1.5. Exported MI Functions

1.5.1.  clusterer_reload

   Reloads data from the clusterer database. The currently
   established topology will be lost and the node will rediscover
   the new topology.

   Name: clusterer_reload

   Parameters:none

   MI FIFO Command Format:
                :clusterer_reload
                _empty_line_

1.5.2.  clusterer_list

   Lists information(node id, URL, link state with that node etc.)
   about the other nodes in each cluster.

   Name: clusterer_list

   Parameters:none

   MI FIFO Command Format:
                :clusterer_list
                _empty_line_

   Example 1.24. clusterer_list usage
$ ./opensipsctl fifo clusterer_list
Cluster:: 1
        Node:: 4 DB_ID=4 URL=bin:127.0.0.4:7774 Enabled=1 Link_state=Up
     Next_hop=4 Description=none
        Node:: 3 DB_ID=3 URL=bin:127.0.0.3:7773 Enabled=1 Link_state=Dow
n    Next_hop=4 Description=none
        Node:: 2 DB_ID=2 URL=bin:127.0.0.2:7772 Enabled=1 Link_state=Pro
be   Next_hop=4 Description=none
Cluster:: 2
        Node:: 5 DB_ID=5 URL=bin:127.0.0.4:7775 Enabled=1 Link_state=Up
     Next_hop=5 Description=none

1.5.3.  clusterer_list_topology

   Lists each cluster's topology from the current node's
   perspective as an adjacency list. A node appears as a neighbour
   if the link with that node is up.

   Note that if a node id appears in multiple clusters, it refers
   to the same instance that belongs to different clusters, for
   which it has a different topology.

   Name: clusterer_list_topology

   Parameters:none

   MI FIFO Command Format:
                :clusterer_list_topology
                _empty_line_

   Example 1.25. clusterer_list_topology usage
$ ./opensipsctl fifo clusterer_list_topology
Cluster:: 1
        Node:: 1 Neighbours=4
        Node:: 4 Neighbours=1 2 3
        Node:: 3 Neighbours=2 4
        Node:: 2 Neighbours=3 4
Cluster:: 2
        Node:: 1 Neighbours=5
        Node:: 5 Neighbours=1

1.5.4.  clusterer_set_status

   Sets the status(Enabled/Disabled) of the current node in a
   specified cluster. A disabled node does not send any messages
   and ignores received ones thus appearing as a failed node in
   the topology.

   Name: clusterer_set_status

   Parameters:
     * cluster_id - indicates the id of the cluster.
     * status - indicates the new status(0 - Disabled, 1 -
       Enabled).

   MI FIFO Command Format:
                :clusterer_set_status:
                1
                0
                _empty_line_

1.5.5.  cluster_send_mi

   Dispatches a given MI command to be run on a specific node in
   the cluster.

   Name: cluster_send_mi

   Parameters:
     * cluster_id - id of the cluster.
     * destination - id of the destination node
     * cmd_name - name of the MI command to be run
     * the parameters of the MI command to be run - optional or
       required, depending on the command

   MI FIFO Command Format:
                :cluster_mi:
                1
                3
                lb_reload
                _empty_line_

1.5.6.  cluster_broadcast_mi

   Dispatches a given MI command to be run on all the nodes in a
   cluster. The command is also executed locally.

   Name: cluster_broadcast_mi

   Parameters:
     * cluster_id - id of the cluster.
     * cmd_name - name of the MI command to be run
     * the parameters of the MI command to be run - optional or
       required, depending on the command

   MI FIFO Command Format:
                :cluster_mi:
                1
                dr_reload_status
                partition_5
                _empty_line_

1.5.7.  clusterer_list_cap

   Lists the registered capabilities and their states.

   Name: clusterer_list

   Parameters:none

   MI FIFO Command Format:
                :clusterer_list_cap
                _empty_line_

   Example 1.26. clusterer_list_cap usage
$ ./opensipsctl fifo clusterer_list
Cluster:: 1
        Capability:: dialog-dlg-repl State=Ok
        Capability:: usrloc-contact-repl State=not synced

1.6. Exported Events

1.6.1.  E_CLUSTERER_REQ_RECEIVED

   This event is raised when a generic, request-like, clusterer
   message is received. This type of message is sent directly from
   the script and not by an OpenSIPS module.

   Parameters:
     * cluster_id - The cluster ID of the source node.
     * src_id - The ID of the source node.
     * msg - The actual message payload.
     * tag - The communication tag of this message, generated by
       the source node. This could be used to send a reply
       corresponding to the received message by providing the tag
       to the cluster_send_rpl() function.

1.6.2.  E_CLUSTERER_RPL_RECEIVED

   This event is raised when a generic, reply-like, clusterer
   message is received. This type of message is sent directly from
   the script and not by an OpenSIPS module.

   Parameters:
     * cluster_id - The cluster ID of the source node.
     * src_id - The ID of the source node.
     * msg - The actual message payload.
     * tag - The communication tag of this message. This could be
       used to match the received reply with a request sent with
       the cluster_send_req() or cluster_broadcast_req()
       functions.

1.7. Usage Example

   This section provides an usage example for replicating
   ratelimit pipes between two OpenSIPS instances. It uses the
   clusterer module to manage the replicating nodes, and the
   proto_bin modules to send the replicated information.

   The setup topology is simple: we have two OpenSIPS nodes
   running on two separate machines (although they could run on
   the same machine as well): Node A has IP 192.168.0.5 and Node B
   has IP 192.168.0.6. Both have, besides the traffic listeners
   (UDP, TCP, etc.), bin listeners bound on port 5566. These
   listeners will be used by the ratelimit module to replicate the
   pipes. Therefore, we have to provision them in the clusterer
   table.

   Example 1.27. Example database content - clusterer table
+----+------------+---------+----------------------+-------+------------
-----+----------+------------------------+
| id | cluster_id | node_id | url                  | state | no_ping_ret
ries | priority | sip_addr | description |
+----+------------+---------+----------------------+-------+-----------+
------------+-----------------+----------+
|  1 |          1 |       1 | bin:192.168.0.5:5566 |     1 |
   3 |       50 | NULL     | Node A      |
|  2 |          1 |       2 | bin:192.168.0.6:5566 |     1 |
   3 |       50 | NULL     | Node B      |
+----+------------+---------+----------------------+-------+-----------+
------------+-----------------+----------+

     * “cluster_id” - this column represents the identifier of the
       cluster. All nodes within a group/cluster should have the
       same id (in our example, both nodes have ID 1). The values
       must be greater than 0.
     * “node_id” - this represents the identifier of the
       machine/node, and each instance within a cluster should
       have a different ID. The values must be greater than 0. In
       our example, Node A will have ID 1, and Node B ID 2.
     * “url” - this indicates the URL where the instance will
       receive the replication information. In our example, each
       node will receive the date over the bin protocol
     * “state” - this is the state of the machine: 1 means
       Enabled, 0 means Disabled; if we had a third machine that
       we didn't want to use for the moment, we would have set the
       state to 0
     * “ls_seq_no” and “top_seq_no” are fields used for the
       probing and topology discovery mechanisms, and should be
       set to 0 by default; they are automatically updated by the
       clusterer module and you shouldn't change them even if a
       node fails or you disable it
     * “no_ping_retries” - is used to specify the maximum number
       of ping retries before the link with a node is considered
       down
     * “priority” - is used to specify the node priority to be
       chosen as next hop in case of same length(number of hops)
       paths when rerouting messages; it is not relevant for this
       two-node topology example
     * “sip_addr” - is a SIP address for the node with currently
       no application in replication scenarios; reserved for
       further development of other modules which might use the
       clusterer module for communication
     * “description” - is an opaque value used to identify the
       node

   After provisioning the two nodes in the database, we have to
   configure the two instances of OpenSIPS. First, we configure
   Node A:

   Example 1.28. Node A configuration
...
listen = bin:192.168.0.5:5566 # bin listener for Node A

loadmodule "proto_bin.so"

loadmodule "clusterer.so"
modparam("clusterer", "db_url", "mysql://opensips@192.168.0.7/opensips")
modparam("clusterer", "current_id", 1) # node_id for Node A

loadmodule "ratelimit.so"
# replicate pipes to cluster id 1
modparam("ratelimit", "replicate_pipes_to", 1)
# accept replicated data from nodes within cluster 1
modparam("ratelimit", "accept_pipes_from", 1)
...

   Similarly, the configuration for Node B is as follows:

   Example 1.29. Node B configuration
...
listen = bin:192.168.0.6:5566 # bin listener for Node B

loadmodule "proto_bin.so"

loadmodule "clusterer.so"
# ideally, use the same database for both nodes
modparam("clusterer", "db_url", "mysql://opensips@192.168.0.7/opensips")
modparam("clusterer", "current_id", 2) # node_id for Node B

loadmodule "ratelimit.so"
# replicate pipes to cluster id 1
modparam("ratelimit", "replicate_pipes_to", 1)
# accept replicated data from nodes within cluster 1
modparam("ratelimit", "accept_pipes_from", 1)
...

   Note that the node_id parameter for Node B is 2. Starting the
   two OpenSIPS instances with the above configurations provides
   your platform the ability to used shared ratelimit pipes in a
   very efficient and scalable way.

Chapter 2. Developer Guide

2.1. Available Functions

2.1.1.  get_nodes(cluster_id)

   This function will return a list of all the reachable nodes(if
   the direct link is down/probing, a path through intermediary
   nodes is considered) in the specified cluster.

   The returned nodes structure:
...
typedef struct clusterer_node {
    int node_id;
    union sockaddr_union addr;
    str sip_addr;
    str description;
    struct clusterer_node *next;
} clusterer_node_t;
...

   Meaning of the parameters is as follows:
     * int cluster_id - the cluster id

2.1.2.  free_nodes(list)

   This function will free the lits of nodes returned by
   get_nodes.

   Meaning of the parameters is as follows:
     * clusterer_node_t *list - list header

2.1.3.  set_state(cluster_id, state)

   This function sets the state(enabled/disabled) of the current
   node in the specified cluster.

   Meaning of the parameters is as follows:
     * int cluster_id - the cluster id
     * enum cl_node_state state - the new state; possible values:
          + STATE_DISABLED
          + STATE_ENABLED

2.1.4.  check_addr(cluster_id, su)

   This function checks if a given address belongs to one of the
   nodes in the cluster.

   Meaning of the parameters is as follows:
     * int cluster_id - the cluster id
     * union sockaddr_union* su - socket address

2.1.5.  get_my_id()

   This function will return the id of the current node.

2.1.6.  send_to(packet, cluster_id, node_id)

   This functon will send the given BIN packet to the specified
   node in the cluster. If the direct link is down/probing, it
   will send the packet to an intermediary node if the destination
   node is reachable through another path in the cluster topology.

   Meaning of the parameters is as follows:
     * bin_packet_t packet - the packet to be sent
     * int cluster_id - the cluster id
     * int node_id - the id of the destination node

   The function returns one of the following:
     * CLUSTERER_SEND_SUCCES - successfuly sent packet to
       destination node or a valid next hop
     * CLUSTERER_CURR_DISABLED - current node is disabled so
       sending is impossbile
     * CLUSTERER_DEST_DOWN - destination node is not reachable
       through any path according to the discovered topology
     * CLUSTERER_SEND_ERR - destination node or valid next hop
       appear to be reachable but send failed

2.1.7.  send_all(packet, cluster_id)

   Send the given BIN packet to all the nodes in the specified
   cluster. The function operates similarly to send_to.

   Meaning of the parameters is as follows:
     * bin_packet_t packet - the packet to be sent
     * int cluster_id - the cluster id

   The function returns one of the following:
     * CLUSTERER_SEND_SUCCES - successfuly sent packet to at least
       one node
     * CLUSTERER_CURR_DISABLED - current node is disabled so
       sending is impossbile
     * CLUSTERER_DEST_DOWN - all nodes in the cluster are
       unreachable according to the discovered topology
     * CLUSTERER_SEND_ERR - send failed for all nodes in the
       cluster

2.1.8.  get_next_hop(cluster_id, node_id)

   This function returns the next hop from the computed shortest
   path to the given destination node in the specified cluster.
   This is the node that is the actual destination for the send_to
   and send_all functions when the direct link with the intended
   destination is down. The function returns the same structure as
   get_nodes.

   Meaning of the parameters is as follows:
     * int cluster_id - the cluster id
     * int node_id - the node id of the destination for which the
       next hop is returned.

2.1.9.  free_next_hop(next_hop)

   This function will free the next hop returned by get_next_hop.

   Meaning of the parameters is as follows:
     * clusterer_node_t *next_hop - next hop to be freed

2.1.10.  register_module(mod_name, cb, auth_check,
accept_clusters_ids, no_accept_clusters)

   This function registers an OpenSIPS module in order to receive
   BIN packets and cluster notifications. A certain module can
   accept packets from multiple clusters and provides a single
   callback function that will be called for each received packet.
   This function will also be called to notify cluster events like
   nodes becoming reachable/unreachable.

   Meaning of the parameters is as follows:
     * char *mod_name - module name
     * clusterer_cb_f cb - callback function
     * int auth_check - 0 - no check, 1 - for every BIN packet
       received check if source IP belongs to one of the nodes in
       the cluster
     * int* accept_clusters_ids - array of cluster ids from which
       packets are accepted
     * int no_accept_clusters - length of accept_clusters_ids
       array

   The callback function prototype:
...
typedef void (*clusterer_cb_f)(enum clusterer_event ev,bin_packet_t *, i
nt packet_type,
                struct receive_info *ri, int cluster_id, int src_id, int
 dest_id);
...

   Possble values for the event signaled through ev parameter of
   the callback funtion:
     * CLUSTER_RECV_MSG - received BIN message
     * CLUSTER_ROUTE_FAILED - failed to route a received BIN
       packet destined for another node in the cluster
     * CLUSTER_NODE_UP - a node became reachable
     * CLUSTER_NODE_DOWN - a node became unreachable
