我需要能够使用spring数据CrudRepository的saveAll()方法,并忽略对唯一约束的重复插入。
在我当前的项目中,我目前使用的是spring boot-
问题是我有一个唯一的约束,文件可能与旧行一起有新行。通过POI处理Excel文件非常繁重,我通过java.util.Set将计算结果保存在内存中,直到我到达必须使用CrudRepository的saveAll()方法保留Set的时间点。
我目前遇到:错误:重复的键值违反了唯一约束“csspd_avoid_duplicates”
org.springframework.transaction.UnexpectedRollbackException:事务已自动回滚,因为它已标记为仅回滚
并且导入在第一次遇到重复时停止。
对于同一项目,我在循环中单次插入(单 save())时遇到了类似的问题,我已经设法使用以下代码片段解决了这个问题:
try {
repository.save(entity);
} catch (DataIntegrityViolationException e) {
this.log.debug("Duplicate found, skipping");
}
我也尝试过同样的方法,但没有运气。
这是一个有趣的方法:
public boolean triggerExcelImport(Path path) {
try {
ExcelDataDTO excelExtractedData;
if (path == null) {
excelExtractedData = excelImporterService.importExcelFiles();
} else {
excelExtractedData = excelImporterService.importStandardXLSXFiles(path);
}
staticProductDataRepository.saveAll(excelExtractedData.getProductData()); // <-- This saveAll might have duplicates
for (Set<AllocationChartData> allocationList : excelExtractedData.getAllocationMap().values()) {
allocationChartDataRepository.saveAll(allocationList); // <-- This saveAll might have duplicates
}
performanceRepository.saveAll(excelExtractedData.getPerformanceData()); // <-- This saveAll might have duplicates
return true;
} catch (DataIntegrityViolationException e) {
this.log.debug("Duplicate found, skipping");
}
return false;
}
我希望导入过程能够实际完成导入,避免在第一个副本上终止。该解决方案必须是独立于数据库的(我正在使用 Postgres 和 Oracle 通过 liquibase 上下文进行管理)
这是我遇到的实际例外:
org.springframework.transaction.UnexpectedRollbackException: Transaction silently rolled back because it has been marked as rollback-only
at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:755) ~[spring-tx-5.1.6.RELEASE.jar:5.1.6.RELEASE]
at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:714) ~[spring-tx-5.1.6.RELEASE.jar:5.1.6.RELEASE]
at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:533) ~[spring-tx-5.1.6.RELEASE.jar:5.1.6.RELEASE]
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:304) ~[spring-tx-5.1.6.RELEASE.jar:5.1.6.RELEASE]
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98) ~[spring-tx-5.1.6.RELEASE.jar:5.1.6.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:688) ~[spring-aop-5.1.6.RELEASE.jar:5.1.6.RELEASE]
at com.juliusbaer.ims.toolbox.services.CenshareService$$EnhancerBySpringCGLIB$$8e7529c1.triggerCenshareExcelImport(<generated>) ~[classes/:?]
at com.juliusbaer.ims.toolbox.services.FolderWatchService.readAndCleanUpFile(FolderWatchService.java:168) ~[classes/:?]
at com.juliusbaer.ims.toolbox.services.FolderWatchService.initializeFileImportAtStartup(FolderWatchService.java:99) ~[classes/:?]
at com.juliusbaer.ims.toolbox.services.FolderWatchService.setUp(FolderWatchService.java:93) ~[classes/:?]
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:363) ~[spring-beans-5.1.6.RELEASE.jar:5.1.6.RELEASE]
at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeInitMethods(InitDestroyAnnotationBeanPostProcessor.java:307) ~[spring-beans-5.1.6.RELEASE.jar:5.1.6.RELEASE]
at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:136) ~[spring-beans-5.1.6.RELEASE.jar:5.1.6.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:414) ~[spring-beans-5.1.6.RELEASE.jar:5.1.6.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1770) ~[spring-beans-5.1.6.RELEASE.jar:5.1.6.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:593) ~[spring-beans-5.1.6.RELEASE.jar:5.1.6.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:515) ~[spring-beans-5.1.6.RELEASE.jar:5.1.6.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:320) ~[spring-beans-5.1.6.RELEASE.jar:5.1.6.RELEASE]
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222) [spring-beans-5.1.6.RELEASE.jar:5.1.6.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:318) [spring-beans-5.1.6.RELEASE.jar:5.1.6.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199) [spring-beans-5.1.6.RELEASE.jar:5.1.6.RELEASE]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:849) [spring-beans-5.1.6.RELEASE.jar:5.1.6.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:877) [spring-context-5.1.6.RELEASE.jar:5.1.6.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:549) [spring-context-5.1.6.RELEASE.jar:5.1.6.RELEASE]
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:142) [spring-boot-2.1.4.RELEASE.jar:2.1.4.RELEASE]
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:775) [spring-boot-2.1.4.RELEASE.jar:2.1.4.RELEASE]
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:397) [spring-boot-2.1.4.RELEASE.jar:2.1.4.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:316) [spring-boot-2.1.4.RELEASE.jar:2.1.4.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1260) [spring-boot-2.1.4.RELEASE.jar:2.1.4.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1248) [spring-boot-2.1.4.RELEASE.jar:2.1.4.RELEASE]
at com.juliusbaer.ims.toolbox.Application.main(Application.java:12) [classes/:?]
谢谢你的帮助。
编辑:为了给你一个更简单的解释,惟一约束不是基于单个标识符id,而是由多个列(总共8列)组成
您可以使用@Query
注释来定义您自己的自定义repo方法来完成此操作。
@Query("INSERT IGNORE INTO table (COLUMNS) values (...)")
public List<S> saveAllInsertIgnore(List<> list);
参考:
https://stackoverflow.com/a/70574062/6784846
为什么不在想要唯一性的列上为对象实现equals()
和hashCode()
函数?将您的对象存储在集合中,它将自动删除重复项,然后您可以毫无例外地使用saveAll()
。