批处理服务是在单个任务中执行多个命令的过程。 在本章中,将学习如何在Spring Boot应用程序中创建批处理服务。
在开始学习之前,请考虑一个示例,如何将CSV文件内容保存到HSQLDB中。
要创建批处理服务程序,需要在构建配置文件中添加Spring Boot Starter Batch依赖项和HSQLDB依赖项。
Maven用户可以在pom.xml 文件中添加以下依赖项。
<dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-batch</artifactId> </dependency> <dependency> <groupId>org.hsqldb</groupId> <artifactId>hsqldb</artifactId> </dependency>
Gradle用户可以在build.gradle 文件中添加以下依赖项。
compile("org.springframework.boot:spring-boot-starter-batch") compile("org.hsqldb:hsqldb")
现在,在资源目录 - src/main/resources
下添加简单的CSV数据文件,并将文件命名为file.csv,如图所示 -
Max,Su Da, Lee Sanfeng, Zhang
接下来,为HSQLDB编写一个SQL脚本 - 在资源目(src/main/resources
)录下创建一个文件: request_fail_hystrix_timeout
DROP TABLE USERS IF EXISTS; CREATE TABLE USERS ( user_id BIGINT IDENTITY NOT NULL PRIMARY KEY, first_name VARCHAR(20), last_name VARCHAR(20) );
创建一个POJO类,如图所示 -
package com.zyiz.batchservicedemo; public class User { private String lastName; private String firstName; public User() { } public User(String firstName, String lastName) { this.firstName = firstName; this.lastName = lastName; } public void setFirstName(String firstName) { this.firstName = firstName; } public String getFirstName() { return firstName; } public String getLastName() { return lastName; } public void setLastName(String lastName) { this.lastName = lastName; } @Override public String toString() { return "firstName: " + firstName + ", lastName: " + lastName; } }
现在,创建一个中间处理器,在从CSV文件读取数据之后和将数据写入SQL之前执行操作。
package com.zyiz.batchservicedemo; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.batch.item.ItemProcessor; public class UserItemProcessor implements ItemProcessor<User, User> { private static final Logger log = LoggerFactory.getLogger(UserItemProcessor.class); @Override public User process(final User user) throws Exception { final String firstName = user.getFirstName().toUpperCase(); final String lastName = user.getLastName().toUpperCase(); final User transformedPerson = new User(firstName, lastName); log.info("Converting (" + user + ") into (" + transformedPerson + ")"); return transformedPerson; } }
创建一个Batch配置文件,从CSV读取数据并写入SQL文件,如下所示。需要在配置类文件中添加@EnableBatchProcessing
注释。 @EnableBatchProcessing
注释用于启用Spring Boot应用程序的批处理操作。
package com.zyiz.batchservicedemo; import javax.sql.DataSource; import org.springframework.batch.core.Job; import org.springframework.batch.core.Step; import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing; import org.springframework.batch.core.configuration.annotation.JobBuilderFactory; import org.springframework.batch.core.configuration.annotation.StepBuilderFactory; import org.springframework.batch.core.launch.support.RunIdIncrementer; import org.springframework.batch.item.database.BeanPropertyItemSqlParameterSourceProvider; import org.springframework.batch.item.database.JdbcBatchItemWriter; import org.springframework.batch.item.file.FlatFileItemReader; import org.springframework.batch.item.file.mapping.BeanWrapperFieldSetMapper; import org.springframework.batch.item.file.mapping.DefaultLineMapper; import org.springframework.batch.item.file.transform.DelimitedLineTokenizer; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; import org.springframework.core.io.ClassPathResource; @Configuration @EnableBatchProcessing public class BatchConfiguration { @Autowired public JobBuilderFactory jobBuilderFactory; @Autowired public StepBuilderFactory stepBuilderFactory; @Autowired public DataSource dataSource; @Bean public FlatFileItemReader<User> reader() { FlatFileItemReader<User> reader = new FlatFileItemReader<User>(); reader.setResource(new ClassPathResource("file.csv")); reader.setLineMapper(new DefaultLineMapper<User>() { { setLineTokenizer(new DelimitedLineTokenizer() { { setNames(new String[] { "firstName", "lastName" }); } }); setFieldSetMapper(new BeanWrapperFieldSetMapper<User>() { { setTargetType(User.class); } }); } }); return reader; } @Bean public UserItemProcessor processor() { return new UserItemProcessor(); } @Bean public JdbcBatchItemWriter<User> writer() { JdbcBatchItemWriter<User> writer = new JdbcBatchItemWriter<User>(); writer.setItemSqlParameterSourceProvider(new BeanPropertyItemSqlParameterSourceProvider<User>()); writer.setSql("INSERT INTO USERS (first_name, last_name) VALUES (:firstName, :lastName)"); writer.setDataSource(dataSource); return writer; } @Bean public Job importUserJob(JobCompletionNotificationListener listener) { return jobBuilderFactory.get("importUserJob").incrementer( new RunIdIncrementer()).listener(listener).flow(step1()).end().build(); } @Bean public Step step1() { return stepBuilderFactory.get("step1").<User, User>chunk(10).reader(reader()).processor(processor()).writer(writer()).build(); } }
reader()
方法用于从CSV文件中读取数据,而writer()
方法用于将数据写入SQL。
接下来,将编写一个作业完成通知监听器类 - 用于在作业完成后通知。
package com.zyiz.batchservicedemo; import java.sql.ResultSet; import java.sql.SQLException; import java.util.List; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.batch.core.BatchStatus; import org.springframework.batch.core.JobExecution; import org.springframework.batch.core.listener.JobExecutionListenerSupport; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.jdbc.core.JdbcTemplate; import org.springframework.jdbc.core.RowMapper; import org.springframework.stereotype.Component; @Component public class JobCompletionNotificationListener extends JobExecutionListenerSupport { private static final Logger log = LoggerFactory.getLogger(JobCompletionNotificationListener.class); private final JdbcTemplate jdbcTemplate; @Autowired public JobCompletionNotificationListener(JdbcTemplate jdbcTemplate) { this.jdbcTemplate = jdbcTemplate; } @Override public void afterJob(JobExecution jobExecution) { if (jobExecution.getStatus() == BatchStatus.COMPLETED) { log.info("!!! JOB FINISHED !! It's time to verify the results!!"); List<User> results = jdbcTemplate.query( "SELECT first_name, last_name FROM USERS", new RowMapper<User>() { @Override public User mapRow(ResultSet rs, int row) throws SQLException { return new User(rs.getString(1), rs.getString(2)); } }); for (User person : results) { log.info("Found <" + person + "> in the database."); } } } }
现在,创建一个可执行的JAR文件,并使用以下Maven或Gradle命令运行Spring Boot应用程序。
对于Maven,使用如下所示的命令 -
mvn clean install
在“BUILD SUCCESS”之后,在target
目录下找到JAR文件。
对于Gradle,可以使用如下所示的命令 -
gradle clean build
在“BUILD SUCCESSFUL”之后,可以在build/libs
目录下找到JAR文件。
使用此处给出的命令运行JAR文件 -
java –jar <JARFILE>
在控制台窗口中看到作业执行过程输出。
应该少了数据库在application.resources中的配置 提交时间:2019-08-29