Integrate FastDFS middleware to realize distributed file management

1. Introduction to FastDFS

FastDFS is an open source lightweight distributed file system. It manages files. Its functions include file storage, file synchronization, file upload, file download, etc. It solves the problems of mass storage and load balancing.

1.1 core roles

FastDFS is composed of trackerserver, storageserver and client.

  1. Tracking server
    The coordinator of FastDFS is responsible for managing all storage servers and groups. After each storage is started, it will connect to the tracker to inform it of its own group and other information, and maintain a periodic heartbeat. The tracker establishes a mapping table from group to [storage server list] according to the heartbeat information of storage.

  2. Storage server
    In a group, there are multiple storage machines in a group. The data is backed up to each other. The storage space is subject to the storage with the smallest capacity in the group. Therefore, it is recommended that multiple storage in the group be configured as the same as possible to avoid waste of storage space.

  3. client
    The initiator of the service request interacts with the tracker server or storage node through a proprietary interface using TCP/IP protocol.

1.2 operation process

  1. The storage service regularly uploads status information to the tracking service;
  2. The client initiates the request;
  3. The tracker synchronizes the memory status and returns the storage service port and IP;
  4. The client performs file operations (upload, download), etc.

2. SpringBoot integrates FastDFS

2.1 core steps

1),allocation FastDFS execution environment 
2),File upload configuration
3),integration Swagger2 Test interface

2.2 core dependencies

<!-- FastDFS rely on -->
<dependency>
    <groupId>com.github.tobato</groupId>
    <artifactId>fastdfs-client</artifactId>
    <version>1.26.5</version>
</dependency>
<!-- Swagger2 Core dependency -->
<dependency>
    <groupId>io.springfox</groupId>
    <artifactId>springfox-swagger2</artifactId>
    <version>2.6.1</version>
</dependency>
<dependency>
    <groupId>io.springfox</groupId>
    <artifactId>springfox-swagger-ui</artifactId>
    <version>2.6.1</version>
</dependency>

2.3 configuring FastDFS

  1. Core profile
fdfs:
  # Link timeout
  connect-timeout: 60
  # Read time
  so-timeout: 60
  # Generate thumbnail parameters
  thumb-image:
    width: 150
    height: 150
  tracker-list: 192.168.72.130:22122
  1. Core configuration class
@Configuration
@Import(FdfsClientConfig.class)
// The problem of Jmx repeatedly registering bean s
@EnableMBeanExport(registration = RegistrationPolicy.IGNORE_EXISTING)
public class DfsConfig {
}

2) File tool class

@Component
public class FileDfsUtil {
    private static final Logger LOGGER = LoggerFactory.getLogger(FileDfsUtil.class);
    @Resource
    private FastFileStorageClient storageClient ;
    /**
     * Upload file
     */
    public String upload(MultipartFile multipartFile) throws Exception{
        String originalFilename = multipartFile.getOriginalFilename().
                                  substring(multipartFile.getOriginalFilename().
                                  lastIndexOf(".") + 1);
        StorePath storePath = this.storageClient.uploadImageAndCrtThumbImage(
                              multipartFile.getInputStream(),
                              multipartFile.getSize(),originalFilename , null);
        return storePath.getFullPath() ;
    }
    /**
     * Delete file
     */
    public void deleteFile(String fileUrl) {
        if (StringUtils.isEmpty(fileUrl)) {
            LOGGER.info("fileUrl == >>File path is empty...");
            return;
        }
        try {
            StorePath storePath = StorePath.parseFromUrl(fileUrl);
            storageClient.deleteFile(storePath.getGroup(), storePath.getPath());
        } catch (Exception e) {
            LOGGER.info(e.getMessage());
        }
    }
}

2.4 file upload configuration

spring:
  application:
    name: ware-fast-dfs
  servlet:
    multipart:
      enabled: true
      max-file-size: 10MB
      max-request-size: 20MB

2.5 configuring Swagger2

It is mainly used to generate the test interface for file uploading.

1) Configuring code classes

@Configuration
public class SwaggerConfig {
    @Bean
    public Docket createRestApi() {
        return new Docket(DocumentationType.SWAGGER_2)
                .apiInfo(apiInfo())
                .select()
                .apis(RequestHandlerSelectors.basePackage("com.fast.dfs"))
                .paths(PathSelectors.any())
                .build();
    }
    private ApiInfo apiInfo() {
        return new ApiInfoBuilder()
                .title("SpringBoot utilize Swagger structure API document")
                .description("apply RestFul style, Founder: cicada Yixiao")
                .termsOfServiceUrl("https://github.com/cicadasmile")
                .version("version 1.0")
                .build();
    }
}

2) Startup class annotation
@EnableSwagger2

3. Demo case

3.1 interface code

@RestController
public class FileController {
    @Resource
    private FileDfsUtil fileDfsUtil ;
    /**
     * File upload
     */
    @ApiOperation(value="Upload file", notes="test FastDFS File upload")
    @RequestMapping(value = "/uploadFile",headers="content-type=multipart/form-data", method = RequestMethod.POST)
    public ResponseEntity<String> uploadFile (@RequestParam("file") MultipartFile file){
        String result ;
        try{
            String path = fileDfsUtil.upload(file) ;
            if (!StringUtils.isEmpty(path)){
                result = path ;
            } else {
                result = "Upload failed" ;
            }
        } catch (Exception e){
            e.printStackTrace() ;
            result = "Service exception" ;
        }
        return ResponseEntity.ok(result);
    }
    /**
     * File deletion
     */
    @RequestMapping(value = "/deleteByPath", method = RequestMethod.GET)
    public ResponseEntity<String> deleteByPath (){
        String filePathName = "group1/M00/00/00/wKhIgl0n4AKABxQEABhlMYw_3Lo825.png" ;
        fileDfsUtil.deleteFile(filePathName);
        return ResponseEntity.ok("SUCCESS") ;
    }
}

3.2 execution process

1,access http://Localhost:7010/swagger-ui HTML test interface
2,Call the file upload interface to get the file in the FastDFS Path to service
3,Browser access: http://192.168.72.130/group1/M00/00/00/wKhIgl0n4AKABxQEABhlMYw_3Lo825.png
4,Call the delete interface to delete the pictures on the server
5,Clear the browser cache and access the picture again Url,Return 404

Tags: Java Spring Spring Boot FastDFS

Posted by classifieds100 on Tue, 31 May 2022 12:46:11 +0530