Load Balancing: A First Look

Once a project has been running for a while, its user base grows along with the traffic. At that point, too many user requests start to overwhelm the server (a single machine) and it can no longer keep up. That’s where load balancing comes in.

• Say I have 600k users — the load then looks like this:

graph TD;
    A["nginx入口服务器"]-->B1["nginx负载1(20w)"];
    A-->B2["nginx负载2(20w)"];
    A-->B3["nginx负载3(20w)"];

Distributing the load this way is fine. But at some point traffic suddenly spikes past 600k requests. There’s no time to add more servers, which brings us back to the original problem: the server goes down. So you have to rate-limit the load servers — once the request volume is exceeded, further requests are blocked.

• Say I have 1.6 million users — the load then looks like this:

graph TD;
    A["nginx主服务器"]-->A1;
    A-->A2;
    A-->A3;
    A1["nginx入口服务器（55w）"]-->B11["nginx负载1(20w)"];
    A1-->B12["nginx负载2(20w)"];
    A1-->B13["nginx负载3(20w)"];
    A2["nginx入口服务器（55w）"]-->B21["nginx负载1(20w)"];
    A2-->B22["nginx负载2(20w)"];
    A2-->B23["nginx负载3(20w)"];
    A3["nginx入口服务器（55w）"]-->B31["nginx负载1(20w)"];
    A3-->B32["nginx负载2(20w)"];
    A3-->B33["nginx负载3(20w)"];

• Say I have 5 million users and want to run a lucky draw.

You need to think this through. If the event really draws 5 million users or more, the servers will be overloaded. That means some load servers will start rejecting requests, and the rejected users get routed to another load server instead. Clearly, if this continues, the whole service will collapse. At this point you should consider deploying the lucky draw separately, with the event servers used only as short-term, auto-scaling servers. The load then looks like this:

graph TD;
    A["nginx主服务器"]-->A1;
    A-->A2;
    A-->A3;
    A["nginx主服务器"]-->H1["活动服务器(30w)"];
    A-->H2["活动服务器(30w)"];
    A-->H3["活动服务器(30w)"];
    A-->H4["活动服务器(30w)"];
    A-->H5["活动服务器(30w)"];
    A-->H6["活动服务器(30w)"];
    A-->H7["活动服务器(30w)"];
    A1["nginx入口服务器（55w）"]-->B11["nginx负载1(20w)"];
    A1-->B12["nginx负载2(20w)"];
    A1-->B13["nginx负载3(20w)"];
    A2["nginx入口服务器（55w）"]-->B21["nginx负载1(20w)"];
    A2-->B22["nginx负载2(20w)"];
    A2-->B23["nginx负载3(20w)"];
    A3["nginx入口服务器（55w）"]-->B31["nginx负载1(20w)"];
    A3-->B32["nginx负载2(20w)"];
    A3-->B33["nginx负载3(20w)"];

• Say you don’t want a single failure to bring down the other services.

  Then consider splitting the services apart — that is, microservices.

graph TD;
    A(入口)-->B1(用户层);
    A-->B2(订单);
    A-->B3(活动);
    A-->B4(商品);
    A-->B5(物流);
    A-->B6(日志);
    A-->B7(客服);
    A-->B8(支付);
    B1-->C1(用户信息);
    B1-->C2(用户任务);
    B2-->D1(订单生成);
    B2-->D2(订单支付);
    B2-->D3(购物车);
    B3-->E1(秒杀);
    B3-->E2(团购);
    B3-->E3(竞拍);
    B4-->F1(商品搜索);
    B4-->F2(商品详情);
    B4-->F3(商品推荐);
    B5-->G1(信息更新);
    B5-->G2(服务商对接);
    B6-->H1(请求日志);
    B6-->H2(交易日志);
    B6-->H3(浏览日志);
    B7-->I1(纠纷处理);
    B7-->I2(订单处理);
    B7-->I3(投诉处理);
    B8-->J1(支付宝);
    B8-->J2(微信);
    B8-->J3(银联);
    B8-->J4(paypal);
    B8-->J5(第三方支付);