博客
关于我
强烈建议你试试无所不能的chatGPT,快点击我
JDBC多线程插入大量数据.md
阅读量:4181 次
发布时间:2019-05-26

本文共 3360 字,大约阅读时间需要 11 分钟。

package com.xiaobu.base.util;import com.google.common.util.concurrent.ThreadFactoryBuilder;import lombok.extern.slf4j.Slf4j;import java.sql.Connection;import java.sql.PreparedStatement;import java.util.concurrent.*;import java.util.concurrent.atomic.AtomicLong;/** * @author xiaobu * @version JDK1.8.0_171 * @date on  2020/12/8 9:07 * @description */@Slf4jpublic class MultiTableInport {
private static final String TABLE_PRE = "random_code_"; private static final Long COUNT=50000000L; private static ThreadFactory threadFactory = new ThreadFactoryBuilder().setNameFormat("thread-pool-%s").build(); public static void multiTableInport(){
for (int index = 1; index < 201; index++) {
long atomic=(index-1)*COUNT; AtomicLong atomicIndex = new AtomicLong(atomic); String tableName = TABLE_PRE + index; multiThreadImport(10,tableName,atomicIndex); } } public static void multiThreadImport(final int threadNum,String tableName,AtomicLong atomicIndex) {
final CountDownLatch cdl = new CountDownLatch(threadNum); long starttime = System.currentTimeMillis(); ThreadPoolExecutor executor = new ThreadPoolExecutor(threadNum, 15, 60, TimeUnit.SECONDS, new ArrayBlockingQueue<>(50000), threadFactory); for (int k = 1; k <= threadNum; k++) {
executor.execute(() -> {
Connection conn = JDBCUtils.getConnection(); PreparedStatement preparedStatement = null; try {
assert conn != null; conn.setAutoCommit(false); String sql = "INSERT INTO " + tableName + "(id) values(?)"; preparedStatement = conn.prepareStatement(sql); for (long i = 1; i <= COUNT / threadNum; i++) {
long id = atomicIndex.incrementAndGet(); preparedStatement.setLong(1, id); preparedStatement.addBatch(); if (i % 10000 == 0) {
preparedStatement.executeBatch(); conn.commit(); preparedStatement.clearBatch(); } } preparedStatement.executeBatch(); preparedStatement.clearBatch(); conn.commit(); preparedStatement.close(); } catch (Exception e) {
e.printStackTrace(); } finally {
cdl.countDown(); JDBCUtils.closeStatement(preparedStatement); JDBCUtils.closeConnection(conn); } }); } try {
cdl.await(); long spendtime = System.currentTimeMillis() - starttime; System.out.println(threadNum + "个线程花费时间:" + spendtime/1000+"S"); log.info("{}个线程花费时间:{}S" ,threadNum , spendtime/1000); } catch (InterruptedException e) {
e.printStackTrace(); } executor.shutdown(); } public static void main(String[] args) {
multiTableInport(); }}
CREATE TABLE `random_code_1` (  `id` bigint(20) NOT NULL,  PRIMARY KEY (`id`)) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;

转载地址:http://xzgai.baihongyu.com/

你可能感兴趣的文章
《Android系统学习》之JAVA与C混合编程——JNI
查看>>
《C预处理》之#ifndef
查看>>
《Linux内核编程》第十三章:Linux对进程内存的二级页式管理
查看>>
ARM协处理器
查看>>
《miniOS分析》前言
查看>>
《Linux内核编程》第十四章:Linux驱动基础
查看>>
Linux平台下ARM-Linux交叉编译工具链
查看>>
Window平台下ADS自带ARMCC编译工具链
查看>>
micro2440/tiny6410使用JLINK直接烧录nand flash
查看>>
C编译器、连接器与可执行机器码文件
查看>>
android linker 浅析
查看>>
802.11 traffic id
查看>>
Android系统wifi分析-手动连接过程
查看>>
设置IP别名Shell脚本
查看>>
Source Insight 宏-单行注释
查看>>
levelDB源码分析-Arena
查看>>
levelDB源码分析-SSTable
查看>>
平滑升级Nginx的Shell脚本
查看>>
SSH远程会话管理工具
查看>>
canvas标签设长宽是在css中还是在标签中
查看>>