flink run job 同步mysql数据,established越来越多怎么处理


当Flink运行作业同步MySQL数据时,established连接数不断增加,这可能是因为以下原因: 1. 数据库服务器负载过高,导致连接数无法及时释放。 2. Flink作业中的连接池配置不合理,导致连接数过多。 3. Flink作业中存在死循环或者长时间运行的任务,导致连接数无法及时释放。 针对这些问题,可以尝试以下解决方案: 1. 检查数据库服务器性能,优化数据库配置,如增加内存、调整连接数限制等。 2. 调整Flink作业中的连接池配置,例如增加最大连接数、设置连接超时时间等。 3. 分析Flink作业代码,找出可能导致死循环或长时间运行的部分,进行优化。 具体操作步骤如下: 1. 查看数据库服务器性能指标,如CPU使用率、内存使用率等,根据需要进行调整。 2. 修改Flink作业的配置文件(如flink-conf.yaml),增加或修改以下配置项: ```yaml # 设置连接池的最大连接数 taskmanager.network.memory.min: 64mb taskmanager.network.memory.max: 1024mb taskmanager.network.memory.fraction: 0.5 taskmanager.network.memory.large-fragment-threshold: 8192kb taskmanager.network.memory.min-free: 32mb taskmanager.network.memory.max-used: 80% taskmanager.network.memory.num-slots: 1 taskmanager.network.memory.min-num-available: 1 taskmanager.network.memory.max-num-available: 100 taskmanager.network.memory.max-age: 300000 taskmanager.network.memory.unused-timeout: 300000 taskmanager.network.memory.check-interval: 1000 taskmanager.network.memory.cleanup-interval: 10000 taskmanager.network.memory.eviction-policy: ALL_REMAINING_MEMORY taskmanager.network.memory.eviction-action: REMOVE_ON_FREE taskmanager.network.memory.eviction-idle-time: 300000 taskmanager.network.memory.eviction-critical-time: 600000 taskmanager.network.memory.eviction-retry-interval: 1000 taskmanager.network.memory.eviction-retry-count: 3 taskmanager.network.memory.eviction-retry-delay: 1000 taskmanager.network.memory.eviction-retry-on-failure: true taskmanager.network.memory.eviction-retry-on-success: false taskmanager.network.memory.eviction-retry-on-exception: true taskmanager.network.memory.eviction-retry-on-timeout: true taskmanager.network.memory.eviction-retry-on-cancelled: true taskmanager.network.memory.eviction-retry-on-released: true taskmanager.network.memory.eviction-retry-on-evicted: true taskmanager.network.memory.eviction-retry-on-not-enough-space: true taskmanager.network.memory.eviction-retry-on-not-enough-resources: true taskmanager.network.memory.eviction-retry-on-not-enough-available-slots: true taskmanager.network.memory.eviction-retry-on-not-enough-available-bytes: true taskmanager.network.memory.eviction-retry-on-not-enough-available-physical-bytes: true taskmanager.network.memory.eviction-retry-on-not-enough-available-virtual-bytes: true taskmanager.network.memory.eviction-retry-on-not-enough-available-swapped-bytes: true taskmanager.network.memory.eviction-retry-on-not-enough-available-committed-bytes: true taskmanager.network.memory.eviction-retry-on-not-enough-available-committed-physical-bytes: true taskmanager.network.memory.eviction-retry-on-not-enough-available-committed-virtual-bytes: true taskmanager.network.memory.eviction-retry-on-not-enough-available-committed-swapped-bytes: true taskmanager.network.memory.eviction-retry-on-not-enough-available-committed-bytes: true taskmanager.network.memory.eviction-retry-on-not-enough-available-committed-physical-bytes: true taskmanager.network.memory.eviction-retry-on-not-enough-available-committed-virtual-bytes: true taskmanager.network.memory.eviction-retry-on-not-enough-available-committed-swapped-bytes: true taskmanager.network.memory.eviction-retry-on-not-enough-available-committed-bytes: true taskmanager.network.memory.eviction-retry-on-not-enough-available-committed-physical-bytes: true taskmanager.network.memory.eviction-retry-on-not-enough-available-committed-virtual-bytes: true taskmanager.network.memory.eviction-retry-on-not-enough-available-committed-swapped-bytes: true taskmanager.network.memory.eviction-retry-on-not-enough-available-committed-bytes: true taskmanager.network.memory.eviction-retry-on-not-enough-available-committed-physical-bytes: true taskmanager.network.memory.eviction-retry-on-not-enough-available-committed-virtual-bytes: true taskmanager.network.memory.eviction-retry-on-not-enough-available-committed-swapped-bytes: true taskmanager.network.memory.eviction-retry-on-not-enough-available-committed-bytes: true taskmanager.network.memory.eviction-retry-on-not-enough-available-committed-physical-bytes: true taskmanager.network.memory.eviction-retry-on-not-enough-available-committed-virtual-bytes: true taskmanager.network.memory.eviction-retry-on-not-enough-available-committed-swapped-bytes: true taskmanager.network.memory.eviction-retry-on-not-enough-available-committed-bytes: true taskmanager.network.memory.eviction-retry-on-not-enough-available-committed-physical-bytes: true taskmanager.network.memory.eviction-retry-on-not-enough-available-committed-virtual-bytes: true taskmanager.network.memory.eviction-retry-on-not-enough-available-committed-swapped-bytes: true taskmanager.network.memory.eviction-retry-on-not-enough-available-committed-bytes: true taskmanager.network.memory.eviction-retry-on-not-enough-available-committed-physical-bytes: true taskmanager.network.memory.eviction-retry-on-not-enough-available-committed-virtual-bytes: true taskmanager.network.memory.eviction-retry-on-not-enough-available-committed-swapped-bytes: true taskmanager.network.memory.eviction-retry-on-not-enough-available-committed-bytes: true taskmanager.network.memory.eviction-retry-on-not-enough-available-committed-physical-bytes: true taskmanager.network.memory.eviction-retry-on-not-enough-available-committed-virtual-bytes: true taskmanager.network.memory.eviction-retry-on-not-enough-available-committed-swapped-bytes: true taskmanager.network.memory.eviction-retry-on-not-enough-available-committed-bytes: true taskmanager.network.memory.eviction-retry-on-not-enough-available-committed-physical-bytes: true taskmanager.network.memory.eviction-retry-on-not-enough-available-committed-virtual-bytes: true taskmanager.network.memory.eviction-retry-on-not-enough-available-committed-swapped-bytes: true taskmanager.network.memory.eviction-re [2024-01-30 14:34:44 | AI写代码神器 | 2048点数解答]
- flink run job 同步mysql数据,established越来越多怎么处理(321点数解答 | 2024-01-30 14:33:26)201
- flink run job 同步mysql数据,established越来越多怎么处理(363点数解答 | 2024-01-30 14:33:46)198
- flink run job 同步mysql数据,established越来越多怎么处理(2048点数解答 | 2024-01-30 14:34:44)221
- needs: - job: test-job2 optional: true - job: test-job1 gitlab-cicd(489点数解答 | 2024-05-28 10:36:04)247
- exception in verse.ai.jobdriver.checkcurrenttoilendorfail: system.indexoutofrangeexception: index was outside the bounds of the array. [ref 40ee905d] at verse.edificegrid.get_item (verse.intvec3 c) [0x00017] in <69945a8ed6c540cf90b578de735e0605>:0 at verse.gridsutility.getedifice (verse.intvec3 c, verse.map map) [0x00000] in <69945a8ed6c540cf90b578de735e0605>:0 at verse.ai.reservationutility.reservesittableorspot (verse.pawn pawn, verse.intvec3 exactsittingpos, verse.ai.job job, system.bool(65点数解答 | 2024-12-06 19:46:33)337
- master.driver = com.mysql.jdbc.driver master.url = jdbc:mysql://10.9.1.210:3306/drdisplayv3_dev?usessl=false&useunicode=true&characterencoding=utf-8&zerodatetimebehavior=converttonull&transformedbitisboolean=true master.username = root master.password = root 副表配置 slave.driver = com.mysql.jdbc.driver slave.url = jdbc:mysql://localhost:3306/drdisplay?usessl=false&useunicode=true&characterencoding=utf-8&zerodatetimebehavior=converttonull&transformedbitisboolean=true slave.username = root slave.pas(92点数解答 | 2024-06-26 15:47:44)256
- master.driver = com.mysql.jdbc.driver master.url = jdbc:mysql://10.9.1.210:3306/drdisplayv3_dev?usessl=false&useunicode=true&characterencoding=utf-8&zerodatetimebehavior=converttonull&transformedbitisboolean=true master.username = root master.password = root 副表配置 slave.driver = com.mysql.jdbc.driver slave.url = jdbc:mysql://localhost:3306/drdisplay?usessl=false&useunicode=true&characterencoding=utf-8&zerodatetimebehavior=converttonull&transformedbitisboolean=true slave.username = root slave.pass(210点数解答 | 2024-06-26 15:49:53)235
- [ { "id": 1, "parentId": 0, "spread": true, "title": "数据", "path": "", "status": 1, "sort": 1, "parentTitle": null, "children": [ { "id": 2, "parentId": 1, "spread": false, "title": "项目-知识库", "path": "/cms/data/questionprolist.html", "status": 1, "sort": 2, "parentTitle": "数据", (603点数解答 | 2025-04-15 22:48:51)107
- 作为python开发,简述下面几个概念:同步,异步,阻塞,非阻塞?(263点数解答 | 2023-11-08 00:56:38)172
- centos java进程出现futex_wait_queue_me,处理(283点数解答 | 2023-11-20 14:51:26)372
- 替雷利珠单抗是一种抗pd-1免疫检查点抑制剂,广泛应用于多种肿瘤的免疫治疗中,可能引发免疫相关性不良反应,罕见但严重的并发症之一为免疫相关性糖尿病。该类型糖尿病通常呈现1型糖尿病的特征,发病迅速且不可逆,通常伴随糖尿病酮症酸中毒的急性发作。该文报道1例替雷利珠单抗注射液致免疫相关性糖尿病的病例,通过分析现有文献和临床案例,总结替雷利珠单抗引发免疫相关性糖尿病的临床特征,探讨免疫检查点抑制剂致糖尿病潜在的发病机制、处理、药学监护,以期在临床使用中警惕抗pd-1免疫检查点抑制剂引起的免疫相关不良反应。翻译成英文(164点数解答 | 2024-10-11 14:52:52)188
- 在java开发中,简述为什么我们调用 start() 方法时会执行 run() 方法,为什么我们不能直接调用 run() 方法?(219点数解答 | 2023-11-09 17:18:49)221