Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
首先获取表的最小最大id,根据配置文件的bulk Size参数切分区间,创建$Maxprocs个worker
比如 mysql表最小id为1,最大为100万,bulk Size=5000,Maxprocs=20
会先创建20个worker等待执行任务,根据最小最大id生成下面的子任务
where id>0 and id<=5000
where id>5000 and id<=10000
where id>10000 and id<=15000
每个worker从taskChan取任务,执行完将任务结果写入resultChan通道 最后主进程汇总任务的结果
➜ go-mysql-transfer git:(master) ✗ go run main.go -stock true
2022-12-24 19:24:20.868626 I | process id: 32925
2022-12-24 19:24:20.868650 I | GOMAXPROCS :24
2022-12-24 19:24:20.868653 I | source mysql(127.0.0.1:3306)
2022-12-24 19:24:20.868657 I | destination rabbitmq(amqp://guest@guest@*****:5672/)
2022-12-24 19:24:20.918500 I | bulk size: 10
2022-12-24 19:24:20.918725 I | 开始导出 blog.blog
2022-12-24 19:24:20.918744 I | minSql select min(id) as min_id from blog.blog
2022-12-24 19:24:20.918754 I | maxSql select max(id) as max_id from blog.blog
2022-12-24 19:24:20.919563 I | minId:1,maxId:67
2022-12-24 19:24:20.923962 I | blog.blog 导入数据 10 条
2022-12-24 19:24:20.949198 I | blog.blog 导入数据 9 条
2022-12-24 19:24:20.955680 I | blog.blog 导入数据 9 条
2022-12-24 19:24:20.958201 I | blog.blog 导入数据 8 条
2022-12-24 19:24:20.958359 I | blog.blog 导入数据 7 条
2022-12-24 19:24:20.959305 I | blog.blog 导入数据 7 条
2022-12-24 19:24:20.959734 I | blog.blog 导入数据 10 条
2022-12-24 19:24:20.959745 I | 共耗时 :41(毫秒)
2022-12-24 19:24:20.959749 I | 表:blog.blog 共导入 60条