Load Balancing and Failover
Load Balancing and Failover
If you have multiple Internet sources, such as a Hotspot and a modem or two modems, then you will want to be able to manage those sources and direct traffic to them in a manner that suits you. This is known as Load Balancing.
Load Balancing is the black box between your network and the Internet. It takes the traffic from the network, applies the Rules to it and then sends it to the Internet through the different sources. It is strictly about directing traffic and does not deal with Internet speeds or bandwidth.
The Rules
We can define any number of rules that are to be applied to the traffic that is passing through the black box and it will determine if a particular rule applies to that data packet. These rules can include things like the following.
- Use the Wifi Hotspot unless it is not available and then switch to Modem 1. Switch back to the Hotspot if it is once again available.
- All traffic from a particular device on the network should be sent through Modem 2.
- Split all traffic from the network equally between Modem 1 and Modem 2.
- Sent all traffic to a specific IP Address through the Hotspot and all the rest goes through Modem 1.
- Send all traffic using a specified port through the WAN.
Or any combination of the above.
Before we can start to implement any Load Balancing system we have to define the Rules we will be using. This must be done in some detail in order to make implementation possible.
What traffic is this rule applied to?
The first step is to define what traffic the rule will apply to. There are a number of options that can be selected to identify the proper data packets.
- Source IP Address : the IP Address of a device on your network. This would identify data packets from a specific device.
- Source Port : a single port or multiple ports on your network that the data packets are associated with.
- Destination IP Address : the IP Address of the web site the data packets are being sent to or being received from.
- IPset : an IPSet rule that matches the IP Address of the web site the data packets are being sent to or being received from.
- Destination Port : the port or ports at the Destination Address that the data packets are being sent to.
- Protocol : the type of data being sent or received such as TCP or UDP.
The above options can be combined in a single Rule to provide a very specific set of data packets.
What do we do with this traffic?
Once we have identified the specific data packets we are interested in, we must now decide what we want to do with them. Will we be doing failover, loadbalancingor a combination of both? What Internet sources are involved in this Rule?
We must chose which action we want to do and then decide which Internet sources will be used with this action. There are 4 actions that can take place on data packets, failover, loadbalance, failover w/loadbalance and loadbalance w/failover.
- Failover is when you want all the specified data packets to go through a specific Internet source. If that source is not available then all the packets will go through another Internet source. When the first source is available again all packets will resume using it.
- Loadbalance is when the specified data packets are sent using multiple Internet sources. The Balance Ratio determines the percentage of the total data packets that are sent through each Internet source.
- Failover with loadbalance is when you want all the specified data packets to go through a specific Internet source. If that source is unavailable then the packets will be sent using multiple Internet sources that are loadbalanced. The Balance Ratio determines the percentage of the total data packets that are sent through each of these Internet sources.
- Loadbalance with failover is when the specified data packets are sent using multiple Internet sources. The Balance Ratio determines the percentage of the total data packets that are sent through each of these Internet sources. If none of these sources are available then the data packets will be sent using another Internet source.
Some Examples of Rules
By looking at a few examples of Rules we will be able to see how they can be defined so we can take that information and implement our Load Balancing System.
Example 1
All traffic goes through the Wifi Hotspot (WWAN) unless that source is disconnected. In that case all traffic goes through Modem 1. When the Wifi Hotspot is connected again return to using it rather than Modem 1.
- Action : failover
- Destination IP Address : 0.0.0.0/0 (this defines all destination addresses. ie. all traffic)
- Primary Internet : Wifi Hotspot (WWAN).
- Secondary Internet : Modem 1 (WAN1)
Example 2
Split all traffic equally between Modem 1 and Modem 2.
- Action : loadbalance
- Destination IP Address : 0.0.0.0/0 (this defines all destination addresses. ie. all traffic)
- Internet source : Modem 1 (WAN1).
- Internet source : Modem 2 (WAN2)
- Balance Ratio : 1 : 1 (Modem 1 to Modem 2)
Example 3
All traffic from the device on your network with the IP Address of 192.168.1.20 goes through Modem 2.
- Action : failover
- Source IP Address : 192.168.1.20
- Destination IP Address : 0.0.0.0/0 (this defines all destination addresses. ie. all traffic)
- Primary Internet : Modem 2 (WAN2).
Example 4
All traffic on the network goes through the Wifi Hotspot. If that source is unavailable then split the traffic between Modem 1 and Modem 2 with Modem 1 getting twice as much traffic as Modem 2.
- Action : failover with loadbalance
- Destination IP Address : 0.0.0.0/0 (this defines all destination addresses. ie. all traffic)
- Primary Internet : Hotspot (WWAN).
- Secondary Internet source : Modem 1 (WAN1).
- Secondary Internet source : Modem 2 (WAN2)
- Balance Ratio : 2 : 1 (Modem 1 to Modem 2)
The Load Balancing System
With the knowledge of how to create Rules for our Load Balancing black box we are ready to design and implement the Load Balancing System. Go to the Networkmenu and the Load Balancing sub menu. Click on the Configuration tab at the top.
Here you see the 4 sections that make up the Load Balancing System, Interfaces, Members, Policies and Rules.
- Interfaces are the physical Internet sources such as Hotspot or Modem.
- Members are a pool of Interfaces with a cost and balancing weight attached to them.
- Policies determine how traffic is distributed among its Members.
- Rules match traffic and assign it to a Policy
When implementing the System you must start with Interfaces and work your way to Rules but when designing it is easier to work in the other direction, starting with Rules.
Interfaces Section
To see the Interfaces section click on the Interfaces tab.
Interfaces are the physical Internet sources that are attached to the router. ROOter supports 4 Interfaces.
- WAN is wired Internet via the Wan port.
- WWAN is a Wifi Hotspot..
- WAN1 is Modem 1.
- WAN2 is Modem 2.
Along with the physical Internet source the Interface entries aso include information that is used by the Load Balancing System to determine if the Internet source is connected or not. This is done by pinging a server or servers and checking if there is a response to it.
Interfaces that are not connected to the Internet can be left in this section as they incur no extra overhead. Under normal circumstance you do not need to change any of the settings used to determine if the Interface is connected but, in some cases, this may help to avoid false indications that the Interface is down. To modify the settings for a specific Interface, scroll the section to the left until the Edit button appears. Click on this.
It should be noted that WAN1 and WAN2 will show as Not Enabled and should be left that way when editing this information. If you have enabled Load Balancingfor the modem in Connection Info they will be automatically enabled here when they connect to the Internet and automatically disabled when they are removed.
An Interface uses pinging a server to determine if the connection is up or down. You can customize the number of pings and the way they are interpreted to suit your system and connection.
- Tracking IP is the IP Address of a site you wish to ping. An IP Address must be entered in this box in order for Connection Monitoring to work. By clicking on the green plus at the end of the box you can have multiple IP Addresses that you wish to ping. If multiple Tracking IPs are defined then these are pinged one after the other and the number of responses are kept track of for that ping session.
- Tracking Reliability is the number of Tracking IPs that must respond each time a ping session is done in order for the connection to be deemed up or down. It must be less than or equal to the number of Tracking IPs defined.
- Ping Count is the number of packets sent during a ping to a Tracking IP. A larger number of packets can ensure that a ping response occurs in congested areas.
- Ping Interval is the amount of time between each ping session. The longer this interval the less data that is used pinging the Tracking IPs but it is also slower to determine if a connection is up or down.
- Ping Timeout is the amount of time that the Connection Monitor waits after sending a ping for its response. If no response is obtained in this time then the ping is assumed to be lost. This time may be increased if congestion is slowing down your connection.
- Interface Down is the number of ping sessions that failed to return the number of pings defined by Tracking Reliability as required before the connection is deemed to be down.
- Interface Up is the number of ping sessions that returned a response of the number of pings defined by Tracking Reliability as required before the connection is deemed to be up.
The default values will work quite well in most situations but Interfaces that suffer from congestion or other problems may have to be customized to avoid false indications of the connection being down.
Members Section
To see the Members section click on the Members tab.
Members are a pool of Interfaces that have a cost and balance ratio connected to them. The same Interface can appear in different Member entries as it may have a different cost or balance ratio in each.
The Cost of a Member is known as the Metric and lower values mean a lower cost. The Balance Ratio is known as the Weight and higher values mean more traffic goes to that Member.
To create a new Member, enter its name on the line and click the ADD button.
You chose the Interface associated with this Member, its Cost ( Metric ) and its Balance Ratio ( Weight ) and the click on Save&Apply.
You can create as many Members as you want to build up a pool of Interfaces with different costs and balance ratios. To make it easier when constructing the Load Balancing System, you should create a new Member for each Internet source used in every Rule in the above examples.
Policies Section
To see the Policies section click on the Policies tab.
A Policy is a group of one or more Members that determines how traffic sent to it is handled. This is done by using the Cost and Balance Ratio of the Members.
In any situation a Policy will always chose the Member with the lowest Cost to use first. This means that all traffic will be sent using the Member with the lowest Cost. If that Member is not connected then it will chose the Member with the next lowest Cost.
If all Members have a different Cost then the Policy will do Failover from one Member to the other. However, if two or more Members have the same Cost the Policy will split the traffic between them based on the Balance Ratio of each. In this case the Policy is doing load-balancing.
To create a new Policy, enter its name on the line and click the ADD button.
From the pool of Members add the ones that you need for this Policy. Remember that lower Cost Members will receive the traffic first and equal Cost Members will share the traffic.
The Last Resort is how traffic is to be handled if all Members are offline. In most cases you would use unreachable if no Internet source is available. Click on Save&Apply to create the Policy.
If several Members of the Policy have the same Cost then traffic is split between them using the Balance Ratio.
If the Members have the same Balance Ratio then the traffic is split equally between them. If they are different then the Member with the higher value gets the most traffic. The lower the Balance Ratio value the less traffic the Member receives. If one Member has a Balance Ratio value of 3 and the other has a value of 1 then the first Member will handle 3 data packets for every 1 handled by the second Member.
You can create as many Policies as you need here as every Rule must have a unique Policy associated with it.
Rules Section
To see the Rules section click on the Rules tab.
Rules are used to match specific traffic to a Policy which then determines how that traffic is handled.
To create a new Rule, enter its name on the line and click the ADD button.
The entries on this page are used to identify very specific data packets and assign them to the chosen Policy. The more entries that are entered here, the more specific the data packet becomes.
- Source Address : the IP Address of a device on your network. This would identify data packets from a specific device.
- Source Port : a single port or multiple ports on your network that the data packets are associated with.
- Destination Address : the IP Address of the web site the data packets are being sent to or being received from.
- IPset : an IPSet rule that matches the IP Address of the web site the data packets are being sent to or being received from.
- Destination Port : the port or ports at the Destination Address that the data packets are being sent to.
- Protocol : the type of data being sent or received such as TCP or UDP.
- Policy Assigned : the Policy associated with this Rule.
After the Rule has been created, click on Save&Apply.
In most cases, you will have more than one Rule defined to handle the traffic from your network. Some will be for specific data packets while others will be more general. When the Load Balancing System is using Rules to match to a data packet it starts at the top of the Rules list and works its way down until a match is made.
Because of this, the Rules that deal with specific data packets must be at the top of the list with the more general Rules below them. Otherwise, a match will be made with a general rule and the specific Rulle will never be used. The more specific the Rule is then the higher up the Rule list it must be.
Rules and IPSet
In the Rule definition there is an IPSet entry that needs some explaination. IPSet is used in place of the Destination Address and enables using URLs rather than an IP Address for the Destination. This is required when matching to data packets for sites such as Netflix or YouTube. These sites have hundreds of IP Addresses associated with their URL so you never know the exact IP Address you are working with.
See DNSMASQ-IPSET for details on how to create an IPSet that can be used in the Rules section.
Creating the Examples
After having a closer look at the sections of the Load Balancing System, we are ready to implement our system. We will look at what is needed to define the exmaples previously shown above.
Example 1
All traffic goes through the Wifi Hotspot (WWAN) unless that source is disconnected. In that case all traffic goes through Modem 1. When the Wifi Hotspot is connected again return to using it rather than Modem 1.
- Action : failover
- Destination IP Address : 0.0.0.0/0 (this defines all destination addresses. ie. all traffic)
- Primary Internet : Wifi Hotspot (WWAN).
- Secondary Internet : Modem 1 (WAN1)
In this example we use 2 Internet sources so we will need 2 Members.
|
Member Name |
Interface |
Metric |
Weight |
|
|
Ex1_hotspot |
WWAN |
1 |
1 |
|
|
Ex1_modem1 |
WAN1 |
2 |
1 |
We will need a Policy to use these Members to handle specific traffic according to the Member's Cost ( Metric ) and Balance Ratio ( Weight ).
|
Policy Name |
Member |
Cost |
Balance Ratio |
|
|
Ex1_failover |
Ex1_hotspot |
1 |
1 |
|
|
Ex1_modem1 |
2 |
1 |
Because the Cost of Ex1_hotspot is less than Ex1_modem1 it will handle all the traffic unless it is not connected. Balance Ratio is ignored in this situation.
Last is the Rule to assign the specific data packets to our Policy.
|
Rule Name |
Policy |
Destination IP |
||
|
Ex1_rule |
Ex1_failover |
0.0.0.0/0 |
This Rule sends all traffic from the network to the Policy named Ex1-failover for handling.
Example 2
Split all traffic equally between Modem 1 and Modem 2.
- Action : loadbalance
- Destination IP Address : 0.0.0.0/0 (this defines all destination addresses. ie. all traffic)
- Internet source : Modem 1 (WAN1).
- Internet source : Modem 2 (WAN2)
- Balance Ratio : 1 : 1 (Modem 1 to Modem 2)
In this example we use 2 Internet sources so we will need 2 Members.
|
Member Name |
Interface |
Metric |
Weight |
|
|
Ex2_modem1 |
WAN1 |
1 |
1 |
|
|
Ex2_modem2 |
WAN2 |
1 |
1 |
We will need a Policy to use these Members to handle specific traffic according to the Member's Cost ( Metric ) and Balance Ratio ( Weight ).
|
Policy Name |
Member |
Cost |
Balance Ratio |
|
|
Ex2_balance |
Ex2_modem1 |
1 |
1 |
|
|
Ex2_modem2 |
1 |
1 |
Because the Cost of the two Members is the same this Policy will do loadbalancing using the Member's Balance Ratio. Since they are the same the traffic will be split equally between the two Members.
Last is the Rule to assign the specific data packets to our Policy.
|
Rule Name |
Policy |
Destination IP |
||
|
Ex2_rule |
Ex2_balance |
0.0.0.0/0 |
This Rule sends all traffic from the network to the Policy named Ex2_balance for handling.
Example 3
All traffic from the device on your network with the IP Address of 192.168.1.20 goes through Modem 2.
- Action : failover
- Source IP Address : 192.168.1.20
- Destination IP Address : 0.0.0.0/0 (this defines all destination addresses. ie. all traffic)
- Primary Internet : Modem 2 (WAN2).
In this example we use only one Internet source so we will need a single Member.
|
Member Name |
Interface |
Metric |
Weight |
|
|
Ex3_modem2 |
WAN2 |
1 |
1 |
We will need a Policy to use this Member to handle specific traffic. Cost and Balance Ratio are ignored with only one Member. This can be considered a Failover situation.
|
Policy Name |
Member |
Cost |
Balance Ratio |
|
|
Ex3_failover |
Ex3_modem2 |
1 |
1 |
Last is the Rule to assign the specific data packets to our Policy.
|
Rule Name |
Policy |
Source IP |
Destination IP |
|
|
Ex3_rule |
Ex3_failovere |
192.168.1.20 |
0.0.0.0/0 |
This Rule sends all traffic from the network device with the IP Address of 192.168.1.20 to the Policy named Ex3_failovere for handling.
Example 4
All traffic on the network goes through the Wifi Hotspot. If that source is unavailable then split the traffic between Modem 1 and Modem 2 with Modem 1 getting twice as much traffic as Modem 2.
- Action : failover with loadbalance
- Destination IP Address : 0.0.0.0/0 (this defines all destination addresses. ie. all traffic)
- Primary Internet : Hotspot (WWAN).
- Secondary Internet source : Modem 1 (WAN1).
- Secondary Internet source : Modem 2 (WAN2)
- Balance Ratio : 2 : 1 (Modem 1 to Modem 2)
In this example we use 3 Internet sources so we will need 3 Members.
|
Member Name |
Interface |
Metric |
Weight |
|
|
Ex4_hotspot |
WWAN |
1 |
1 |
|
|
Ex4_modem1 |
WAN1 |
2 |
2 |
|
|
Ex4_modem2 |
WAN2 |
2 |
1 |
We will need a Policy to use these Members to handle specific traffic according to the Member's Cost ( Metric ) and Balance Ratio ( Weight ).
|
Policy Name |
Member |
Cost |
Balance Ratio |
|
|
Ex4_failover |
Ex4_hotspot |
1 |
1 |
|
|
Ex4_modem1 |
2 |
2 |
||
|
Ex4_modem2 |
2 |
1 |
Since the Cost of Ex4_hotspot is lower than the other Members it will handle all the traffic if it is connected. If it is not connected then traffic will go to the Member with the next lowest Cost. Since the other two Members have an equal Cost the Policy will split the traffic between them based on their Balance Ratios. In this case, Ex4_modem1 will receive twice the traffic as Ex4_modem2.
Last is the Rule to assign the specific data packets to our Policy.
|
Rule Name |
Policy |
Destination IP |
||
|
Ex4_rule |
Ex4_failover |
0.0.0.0/0 |
This Rule sends all traffic from the network to the Policy named Ex4_failover for handling.
And a Final Example
We will finish the Load Balancing examples with a more complex real world setup.
You have two Internet sources, wired Internet through the WAN that is slower but has a large bandwidth allowance and a high speed LTE modem that has limitations on its bandwidth. Your aim is to maximum the use of these Internet sources but not incur bandwidth overages.
These points are what you want to accomplish with the Load Balancing System.
- Your family are avid players of the Brand-X online game and you wish to use the LTE modem for this but will use the wired Internet if the modem is not connected.
- All traffic to Netflix and YouTube must only use the wired Internet. The modem will not be used for this.
- Your personal computer will use the LTE modem first if available.
- Your children's computer uses the wired Internet only.
- The rest of the household devices use the wired Internet first and the modem second.
Each of the above points will require a new Rule to properly handle the specific traffic.
Online Gaming
For this you will need two Members set up to do failover.
|
Member Name |
Interface |
Metric |
Weight |
|
|
Game_WAN |
WAN |
2 |
1 |
|
|
Game_modem |
WAN1 |
1 |
1 |
To handle the game traffic we need a Policy with the two Members associated with it.
|
Policy Name |
Member |
Cost |
Balance Ratio |
|
|
Game_policy |
Game_modem |
1 |
1 |
|
|
Game_WAN |
2 |
1 |
Since the modem has a lower Cost than the wired Internet it will be used for all traffic as long as it is connected.
And now the Rule. In the Rule we need to identify data packets that are going to the game server from various devices on the network. This can be done by :
- Destination IP Address : knowing the IP Address of the server that hosts the game.
- Source Port : the Port on the router that is opened for this game.
- Destination Port : the Port on the server that accepts data from you.
Or a combination of all three. In this example we will assume that the game requires the use of Ports 3078 and 3079 on the router. Traffic assiociated with those Ports will always be for the game.
|
Rule Name |
Policy |
Source Port |
||
|
Game_rule |
Game_policy |
3078,3079 |
This Rule will direct all traffic associated with Ports 3078 and 3079 to the Game_policy Policy for handling.
Netflix and YouTube
For this you will need one Member only.
|
Member Name |
Interface |
Metric |
Weight |
|
|
Video_WAN |
WAN |
1 |
1 |
To handle the video traffic we need a Policy with the single Member associated with it.
|
Policy Name |
Member |
Cost |
Balance Ratio |
|
|
Video_policy |
Video_WAN |
1 |
1 |
With only a single Internet source if that source is disconnected then no traffic will move.
And the Rule to associate data packets going to Netflix and YouTube with the Video-policy.
|
Rule Name |
Policy |
IPset |
||
|
Video_rule |
Video_policy |
video |
See DNSMASQ-IPSET for details on how to create an IPSet that can be used to match traffic to Netflix and YouTube..
Your Computer
For this you will need two Members set up to do failover.
|
Member Name |
Interface |
Metric |
Weight |
|
|
Your_WAN |
WAN |
2 |
1 |
|
|
Your_modem |
WAN1 |
1 |
1 |
To handle the traffic from your computer we need a Policy with the two Members associated with it.
|
Policy Name |
Member |
Cost |
Balance Ratio |
|
|
Your-policy |
Your_modem |
1 |
1 |
|
|
Your_WAN |
2 |
1 |
Since the modem has a lower Cost than the wired Internet it will be used for all traffic as long as it is connected.
And now the Rule. In the Rule we need to identify data packets that come from your computer and that can be done by the Source IP Address of your computer. Let's assume it is 192.168.1.30.
|
Rule Name |
Policy |
Source Address |
||
|
Your_rule |
Your_policy |
192.168.1.30 |
This Rule will direct all traffic associated with 192.168.1.30 to the Your-policy Policy for handling.
Your Children
For this you will need one Member only.
|
Member Name |
Interface |
Metric |
Weight |
|
|
Kids_WAN |
WAN |
1 |
1 |
To handle the traffic from your children's computer we need a Policy with the single Member associated with it.
|
Policy Name |
Member |
Cost |
Balance Ratio |
|
|
Kids_policy |
Kids_WAN |
1 |
1 |
With only a single Internet source if that source is disconnected then no traffic will move.
And the Rule to associate data packets from your children's computer with the Kids_policy. Again, this is done by using the Source IP Address of the computer. Let's assume this is 192.168.1.25.
|
Rule Name |
Policy |
Source Address |
||
|
Video_rule |
Kids_policy |
192.168.1.25 |
Default
For this you will need two Members set up to do failover.
|
Member Name |
Interface |
Metric |
Weight |
|
|
Default_WAN |
WAN |
1 |
1 |
|
|
Default_modem |
WAN1 |
2 |
1 |
To handle the traffic from the other devices we need a Policy with the two Members associated with it.
|
Policy Name |
Member |
Cost |
Balance Ratio |
|
|
Default_policy |
Default_modem |
2 |
1 |
|
|
Default_WAN |
1 |
1 |
Since the WAN has a lower Cost than the modem it will be used for all traffic as long as it is connected.
And now the Rule. In the Rule we need to identify data packets that go to all destinations
|
Rule Name |
Policy |
Destination Address |
||
|
Default_rule |
Default_policy |
0.0.0.0/0 |
This Rule will direct all traffic to the Default-policy Policy for handling.
For this Load Balancing System we have created 5 Rules to handle the traffic from the network. The final step is to order them so that traffic destined for one Rule is not grabbed by another Rule higher up the list. The most specific Rule must be at the top of the list and the least specific at the bottom.
A good ordering for these Rules would be :
- Online Gaming
- Netflix and YouTube
- Your Computer
- Children's Computer
- Default
This orders the Rules from very specific Game traffic down to the Default traffic. To change the position of a Rule in the Rules list, use the Sort arrows to move them up and down.
Click on Save&Apply after all your changes are finished.