Spring Cloud's service fault tolerance component Hystrix
1. Introduction to Spring Cloud Hystrix
Spring Cloud Hystrix is implemented based on Netflix's open source framework Hystrix and is used as a service fault-tolerant component. Hystrix components are like fuses in daily life, which can fuse and protect high-concurrency Web application systems. Hystrix has powerful functions such as service degradation, service fuse, thread isolation, request caching, request merging, and service monitoring.
2. Hystrix-based service downgrade
In order to achieve the effect of service degradation, we need to provide an available service and an unavailable service in the service provider (by setting the sleep time). Then, in the consumer application, add Hystrix-related dependencies to implement service degradation. Specific steps are as follows:
2.1, service provider (eureka-provider) modification
The service provider only needs to modify the test Controller class and add an unavailable interface, as follows:
@Controller
public class ProviderController {
Logger logger = Logger.getLogger(ProviderController.class);
@RequestMapping("/provider")
@ResponseBody
public String provider(){
logger.info("call service provider ProviderController of provider()method!");
return "Hello World!";
}
@RequestMapping("/unavailable")
@ResponseBody
public String unavailable() throws InterruptedException {
logger.info("call service provider ProviderController of unavailable()method!");
Thread.sleep(1000 * 10);
return "Hello World!";
}
}
In the above code, we added an unavailable() method, which makes the service unavailable by sleeping for 10 seconds (Hystrix will consider the service unavailable by default if it exceeds 3 seconds, and the timeout period can be configured through the configuration file).
2.2. Introduce Hystrix dependencies on the consumer side (eureka-consumer)
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-hystrix</artifactId>
<version>1.4.7.RELEASE</version>
</dependency>
2.3. Enable Hystrix function
Add the annotation @EnableCircuitBreaker to the startup class to start Hystrix related functions.
@SpringBootApplication
@EnableDiscoveryClient
@EnableFeignClients(defaultConfiguration = FeignLoggingConfig.class)
@EnableCircuitBreaker
public class EurekaConsumerApplication {
public static void main(String[] args) {
SpringApplication.run(EurekaConsumerApplication.class, args);
}
}
2.4. Define a HystrixService class to realize the integration of Hystrix functions
implements a HystrixService class, then annotates a method with @HystrixCommand, and then defines the callback method when the method fails or times out through the fallbackMethod attribute, as shown below:
@Service
public class HystrixService {
@Autowired
private FeignApiClient feignApiClient;
@HystrixCommand(fallbackMethod = "fallback")
public String provider(){
return feignApiClient.provider();
}
@HystrixCommand(fallbackMethod = "fallback")
public String unavailable(){
return feignApiClient.unavailable();
}
public String fallback(){
return "error";
}
}
2.5. Define the test class HystrixController
defines a test class, which injects a HystrixService instance, and defines two test interfaces, one for calling available services and one for calling unavailable services.
@RestController
public class HystrixController {
Logger logger = Logger.getLogger(HystrixController.class);
@Autowired
private HystrixService hystrixService;
@RequestMapping("/provider")
public String provider(){
logger.info("execute consumer FeignController of provider()method!");
return hystrixService.provider();
}
@RequestMapping("/unavailable")
public String unavailable(){
logger.info("execute consumer FeignController of unavailable()method!");
return hystrixService.unavailable();
}
}
2.6, start and verify
After the startup is successful, visit http://localhost:8010/provider and http://localhost:8010/unavailable respectively. The available interface will return the correct result, while the unavailable interface directly accesses "error", that is The return value in the callback method. At this point, the simple usage of Hystrix has been realized, and we will learn about the detailed configuration and other usage of Hystrix in detail later.
3. Dependency isolation
In Hystrix, the use of bulkhead mode (Bulkhead) to achieve thread pool isolation, it will create an independent thread pool for each Hystrix command, so that when a dependent service wrapped in a Hystrix command is delayed too high, It also only affects the call to the dependent service, and will not spread to other services.
In addition to using thread pools, semaphores can also be used in Hystrix to control the concurrency of a single dependent service. The overhead of semaphores is much smaller than that of thread pools, but it cannot set timeouts and achieve asynchronous access. So, use semaphores only if the dependent service is reliable enough.
How to use Hystrix to achieve dependency isolation? In fact, in the previous example, when we use
When @HystrixCommand is annotated, the Hystrix component will automatically implement isolation for this service call. Therefore, dependency isolation and service degradation basically exist and occur at the same time.
Bulkhead isolates critical resources such as connection pools, memory, and CPU for each workload or service. Using bulkheads avoids scenarios where a single workload (or service) consumes all resources, causing other services to fail. This pattern primarily increases the resiliency of the system by preventing cascading failures caused by one service.
4. Request a merge
Hystrix supports automatic merging of multiple requests into one request. By merging, the number of threads and network connections required for concurrent execution of HystrixCommand can be reduced, which greatly saves overhead and improves system efficiency.
4.1. Implemented by inheriting the HystrixCollapser and HystrixCommand classes respectively
First, implement a basic Service method for processing business logic, in which there must be a business method findAll() that can batch data. code show as below:
@Service
public class HystrixService {
Logger logger = Logger.getLogger(HystrixService.class);
//@HystrixCommand
public List<String> findAll(List<Integer> ids) {
logger.info("got inside findAll method, start executing the batch!");
for(int i=0;i<ids.size();i++){
logger.info("ID:" + ids.get(i));
}
//Equivalent to calling the data returned by Provider
List<String> list = Arrays.asList("Zhang San","Li Si","Wang Wu");
return list;
}
}
Then you need to create a HystrixBatchCommon class, which inherits the HystrixCommand class and is mainly used to implement batch method calls. At the same time, you can add some Hystrix-related features, such as service degradation, dependency isolation, etc.
public class HystrixBatchCommon extends HystrixCommand<List<String>> {
Logger logger = Logger.getLogger(HystrixBatchCommon.class);
private HystrixService hystrixService;
private List<Integer> ids;
public HystrixBatchCommon(HystrixService hystrixService, List<Integer> ids){
super(Setter.withGroupKey(HystrixCommandGroupKey.Factory.asKey("ExampleGroup")).
andCommandKey(HystrixCommandKey.Factory.asKey("GetValueForKey")));
this.hystrixService =hystrixService ;
this.ids = ids;
}
@Override
protected List<String> run() throws Exception {
List<String> list = hystrixService.findAll(ids);
return list;
}
@Override
protected List<String> getFallback() {
logger.info("HystrixBatchCommon An error message appears!");
return null;
}
}
Implement a HystrixCollapseCommand class, which inherits the HystrixCollapser class, mainly implements the merging of requests, and then implements the batch method invocation through HystrixBatchCommon. Among them, in the definition of the HystrixCollapser abstract class, three different types are specified:
- The first is the return type of the merged batch request
- The second is the type returned by a single request
- The third is the request parameter type
Then it overrides three methods:
-
The first: the getRequestArgument() method is used to define the get request parameters.
-
The second: the specific implementation method of the createCommand() method used to merge requests to generate batch commands
-
The third: The mapResponseToRequests() method is used to process the results returned by the batch command. It is necessary to split the batch results and pass them to each request before merging.
public class HystrixCollapseCommand extends HystrixCollapser<List,String,Integer> {
Logger logger = Logger.getLogger(HystrixCollapseCommand.class); private HystrixService hystrixService; private Integer id; public HystrixCollapseCommand(HystrixService hystrixService, Integer id){ //Set the merge request window to 1000 milliseconds to facilitate testing super(Setter.withCollapserKey(HystrixCollapserKey.Factory.asKey("testCollapseCommand")). andCollapserPropertiesDefaults(HystrixCollapserProperties.Setter().withTimerDelayInMilliseconds(1000))); this.hystrixService = hystrixService; this.id = id; } @Override public Integer getRequestArgument() { return this.id; } @Override protected HystrixCommand<List<String>> createCommand(Collection<CollapsedRequest<String, Integer>> collapsedRequests) { logger.info("HystrixCollapseCommand========>"); //List of UserId s by number of requests List<Integer> ids = new ArrayList<>(collapsedRequests.size()); //By requesting, the request parameters in 100 milliseconds are taken out and loaded into the collection ids.addAll(collapsedRequests.stream().map(CollapsedRequest::getArgument).collect(Collectors.toList())); //Return the UserBatchCommand object and automatically execute the run method of UserBatchCommand HystrixCommand<List<String>> re =new HystrixBatchCommon(hystrixService, ids); return re; } @Override protected void mapResponseToRequests(List<String> batchResponse, Collection<CollapsedRequest<String, Integer>> collapsedRequests) { int count = 0 ; for(CollapsedRequest<String,Integer> collapsedRequest : collapsedRequests){ //Fetch the results in order from the batch response collection String user = batchResponse.get(count++); //Put the result back into the response body of the original Request collapsedRequest.setResponse(user); } }}
Finally, write a test method to verify the merge effect of the request, as follows:
@RequestMapping("/mergeRequest2")
public String mergeRequest2(Integer id) throws ExecutionException, InterruptedException {
logger.info("execute consumer FeignController of mergeRequest2()method!");
//HystrixRequestContext context = HystrixRequestContext.initializeContext();
// Future<String> f1 = new HystrixCollapseCommand(hystrixService, id).queue();
HystrixRequestContext context = HystrixRequestContext.initializeContext();
try {
Future<String> f1 = new HystrixCollapseCommand(hystrixService, 1).queue();
Future<String> f2 = new HystrixCollapseCommand(hystrixService, 2).queue();
Future<String> f3 = new HystrixCollapseCommand(hystrixService, 3).queue();
Thread.sleep(3000);
Future<String> f4 = new HystrixCollapseCommand(hystrixService, 5).queue();
Future<String> f5 = new HystrixCollapseCommand(hystrixService, 6).queue();
String u1 = f1.get();
String u2 = f2.get();
String u3 = f3.get();
String u4 = f4.get();
String u5 = f5.get();
System.out.println(u1);
System.out.println(u2);
System.out.println(u3);
System.out.println(u4);
System.out.println(u5);
} catch (Exception e) {
e.printStackTrace();
}finally {
context.close();
}
return null;//f1.get().toString();
}
After starting the project, visit the address http://localhost:8020/mergeRequest2, and the content appears in the console, in which the findAll method is only executed twice, indicating that the merger has taken effect.
Question: In the actual work process, how is the request suitable used? The request result cannot be returned directly here, otherwise the merge will fail. Conjecture: It should be implemented by means of asynchronous messages and the like, waiting for verification.
4.2. Request merging through annotations
The way to implement annotations is relatively simple. You only need to modify the Service method of business processing. You need a single request processing method to add the @HystrixCollapser annotation. The logic in this method will not actually be executed. It is mainly used to request mergers, and then In the actual batch method, the @HystrixCommand annotation is used for merged processing.
@Service
public class HystrixService {
Logger logger = Logger.getLogger(HystrixService.class);
@HystrixCollapser(batchMethod = "findAll", scope = com.netflix.hystrix.HystrixCollapser.Scope.GLOBAL,
collapserProperties = {
//Requests with a request interval within 5000ms will be merged into one request, the default is 10ms
@HystrixProperty(name = "timerDelayInMilliseconds", value = "500"),
//Sets the maximum number of requests allowed in a batch before triggering batch execution.
@HystrixProperty(name = "maxRequestsInBatch", value = "200")
})
public Future<String> getById(Integer id) {
logger.info("do not execute the code");
return null;
}
@HystrixCommand
public List<String> findAll(List<Integer> ids) {
logger.info("got inside findAll method, start executing the batch!");
for(int i=0;i<ids.size();i++){
logger.info("ID:" + ids.get(i));
}
//Equivalent to calling the data returned by Provider
List<String> list = Arrays.asList("Zhang San","Li Si","Wang Wu");
return list;
}
}
The verification of the annotation method cannot be performed by the method provided above. We provide a separate method in the HystrixController class as follows:
@RequestMapping("/mergeRequest")
public String mergeRequest(Integer id) throws ExecutionException, InterruptedException {
logger.info("execute consumer FeignController of mergeRequest()method!");
Future<String> future = hystrixService.getById(id);
return null;
}
Then verify through the batch request method of postman, first save several requests, and then execute, as follows:
After the request, you can see the following content in the console, indicating that the two requests are merged into one and executed.
5. Monitoring panel
We mentioned earlier that the method annotated by @HystrixCommand will be recorded by Hystrix. How to view the recorded data?
5.1. Monitoring Node Instances
In order to monitor the calling of the interface in the service, it is necessary to add related dependencies, that is, introduce the actuator dependency on the consumer side (eureka-consumer), as follows:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
Then, add relevant configuration in application.properties, that is, expose endpoints
#expose all endpoints management.endpoints.web.exposure.include=* #It is also possible to set endpoints that are not exposed management.endpoints.web.exposure.exclude=beans
The endpoint information is as follows:
After the configuration is complete, restart the service. Visit the address http://192.168.1.87:8020/actuator/health, the page will display the following content, indicating that the service is running normally:
{"status":"UP"}
Visit http://192.168.1.87:8020/actuator/hystrix.stream address. At this time, the following content will appear, which is the data of service usage.
If the following 404 page appears, it is generally a management.endpoints.web.exposure.include configuration problem, just check its configuration.
5.2. Monitoring panel
In the front, we can view the data monitored by the application by visiting the address http://192.168.1.87:8020/actuator/hystrix.stream, but these data are in json format, which is very inconvenient to intuitively understand the operation of the service. In fact, Hystrix has already provided a set of visual monitoring clients, namely Hystrix Dashboard. Let's build the Hystrix Dashboard service, and the process is very simple.
In order to ensure the independence of Hystrix Dashboard, we created a separate sub-module to build Hystrix Dashboard, the project name is: hystrix-dashboard.
First, you need to add relevant dependency information, as follows:
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-hystrix</artifactId>
<version>1.4.7.RELEASE</version>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-hystrix-dashboard</artifactId>
<version>1.4.7.RELEASE</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
Then, modify the application.properties configuration file
spring.application.name=hystrix-dashboard server.port=8050 #Set the address to interact with Eureka Server. eureka.client.serviceUrl.defaultZone=http://127.0.0.1:8000/eureka/
Finally, create a startup class and add relevant annotations, as shown below:
@SpringBootApplication
@EnableHystrixDashboard
public class HystrixDashboardApplication {
public static void main(String[] args) {
SpringApplication.run(HystrixDashboardApplication.class, args);
}
}
After completing the above configuration, restart the project. Visit http://localhost:8050/hystrix address, you can see the monitoring home page of Hystrix Dashboard, as shown below:
However, there is no relevant monitoring information and content on the monitoring page of Hystrix Dashboard. We can know from the page prompts that if we want to monitor those data, fill in the corresponding url and click the "Monitor Stream" button to view it. Here we choose to use hystrix.stream to monitor the situation of a single application, that is, the input url is http://192.168.1.87:8020/actuator/hystrix.stream, and then click the button to view.
At this time, if an "Unable to connect to Command Metric Stream." error occurs, the page is as follows:
Then, check the hystrix-dashboard console and find that the following log is printed, indicating that the hystrix.dashboard.proxyStreamAllowList configuration is missing.
2020-11-11 14:20:34.909 WARN 70100 --- [nio-8050-exec-9] ashboardConfiguration$ProxyStreamServlet : Origin parameter: http://192.168.1.87:8020/actuator/hystrix.stream is not in the allowed list of proxy host names. If it should be allowed add it to hystrix.dashboard.proxyStreamAllowList.
At this time, you can add the following configuration to the application.properties configuration file. It should be noted that this configuration does not support "*". If you need to configure multiple IP s, use "," to split.
hystrix.dashboard.proxyStreamAllowList=192.168.1.87
Then restart, and then refresh the page, you will find the following interface, indicating success:
Description of visual content: a solid circle, a curve, and an indicator description.
- Solid circle: There are two meanings. It represents the health of the instance through the change of color, and its health decreases from green, yellow, orange, red. In addition to the color change, the size of the solid circle also changes according to the request traffic of the instance. The larger the traffic, the larger the solid circle. Therefore, through the display of the solid circle, we can quickly find fault instances and high-stress instances in a large number of instances.
- Curve: Used to record relative changes in traffic (in the near term), we can use it to observe the rising and falling trend of traffic.
- The indicators are described as follows:
