# 9.1 Kvstore ## 需求 1. ntyco需要作为kvstore的submodule,通过git clone一次下载。 **完成**。 2. README需要包含编译步骤，测试方案与可行性，性能数据。 **完成**。 3. 全量持久化保存数据集。 **BUG FIX**。 4. 持久化的性能数据。 5. 特殊字符，可以解决redis的resp协议。 **完成**。 6. 实现配置文件，把日志级别，端口ip，主从模式，持久化方案。 **完成**。 7. 持久化落盘用io_uring，加载配置文件用mmap。 **完成**。 8. 主从同步的性能，开启与关闭性能做到5%?。 9. 主从同步600w条,出现的coredump。 **完成**。 10. 主从同步用ebpf实现。 **BUG FIX**。 11. 内存池测试qps与虚拟内存，物理内存。 12. 实现一个内存泄露检测组件。 **完成**。 ## 环境安装与编译 ```shell # xml sudo apt install libxml2 libxml2-dev # hiredis client sudo apt install -y libhiredis-dev # bpftrace sudo apt install -y bpftrace git clone git@gitlab.0voice.com:lianyiheng/9.1-kvstore.git cd 9.1-kvstore/ git submodule update --init --recursive ./init.sh make ``` ## 测试 ### 测试1：性能测试 测试条件： 1. 不启用持久化。 2. 不启用主从同步。 2. pipline： 1. RSET 100w 条， p:i v:i -> +OK 2. RGET 100w 条， p:i -> +v:i 3. RDEL 100w 条。 p:i -> +OK 3. 重复 15 次. 4. 本机发送请求。 内存分配: malloc ```bash lian@ubuntu:~/share/9.1-kvstore$ ./test-redis/testcase 192.168.10.129 8888 3 Connected to 192.168.10.129:8888 BATCH (N=3000000) --> time_used=3294 ms, qps=910746 BATCH (N=3000000) --> time_used=3501 ms, qps=856898 BATCH (N=3000000) --> time_used=3457 ms, qps=867804 BATCH (N=3000000) --> time_used=3351 ms, qps=895255 BATCH (N=3000000) --> time_used=3320 ms, qps=903614 BATCH (N=3000000) --> time_used=3551 ms, qps=844832 BATCH (N=3000000) --> time_used=3354 ms, qps=894454 BATCH (N=3000000) --> time_used=3475 ms, qps=863309 BATCH (N=3000000) --> time_used=3404 ms, qps=881316 BATCH (N=3000000) --> time_used=3460 ms, qps=867052 BATCH (N=3000000) --> time_used=3392 ms, qps=884433 BATCH (N=3000000) --> time_used=3427 ms, qps=875401 BATCH (N=3000000) --> time_used=3441 ms, qps=871839 BATCH (N=3000000) --> time_used=3471 ms, qps=864304 BATCH (N=3000000) --> time_used=3354 ms, qps=894454 BATCH (N=3000000) --> time_used=3447 ms, qps=870322 BATCH (N=3000000) --> time_used=3364 ms, qps=891795 BATCH (N=3000000) --> time_used=3200 ms, qps=937500 BATCH (N=3000000) --> time_used=3159 ms, qps=949667 BATCH (N=3000000) --> time_used=3482 ms, qps=861573 BATCH (N=3000000) --> time_used=3474 ms, qps=863557 BATCH (N=3000000) --> time_used=3591 ms, qps=835421 BATCH (N=3000000) --> time_used=3466 ms, qps=865551 BATCH (N=3000000) --> time_used=3425 ms, qps=875912 BATCH (N=3000000) --> time_used=3346 ms, qps=896592 BATCH (N=3000000) --> time_used=3532 ms, qps=849377 BATCH (N=3000000) --> time_used=3471 ms, qps=864304 BATCH (N=3000000) --> time_used=3616 ms, qps=829646 BATCH (N=3000000) --> time_used=3403 ms, qps=881575 BATCH (N=3000000) --> time_used=3426 ms, qps=875656 BATCH (N=3000000) --> time_used=3493 ms, qps=858860 BATCH (N=3000000) --> time_used=3411 ms, qps=879507 BATCH (N=3000000) --> time_used=3422 ms, qps=876680 BATCH (N=3000000) --> time_used=3556 ms, qps=843644 BATCH (N=3000000) --> time_used=3285 ms, qps=913242 BATCH (N=3000000) --> time_used=3486 ms, qps=860585 BATCH (N=3000000) --> time_used=3427 ms, qps=875401 BATCH (N=3000000) --> time_used=3563 ms, qps=841987 BATCH (N=3000000) --> time_used=3304 ms, qps=907990 BATCH (N=3000000) --> time_used=3582 ms, qps=837520 BATCH (N=3000000) --> time_used=3468 ms, qps=865051 BATCH (N=3000000) --> time_used=3360 ms, qps=892857 BATCH (N=3000000) --> time_used=3426 ms, qps=875656 BATCH (N=3000000) --> time_used=3186 ms, qps=941619 BATCH (N=3000000) --> time_used=3251 ms, qps=922792 BATCH (N=3000000) --> time_used=3400 ms, qps=882352 BATCH (N=3000000) --> time_used=3446 ms, qps=870574 BATCH (N=3000000) --> time_used=3302 ms, qps=908540 BATCH (N=3000000) --> time_used=3072 ms, qps=976562 BATCH (N=3000000) --> time_used=3458 ms, qps=867553 average qps:880462 ALL TESTS PASSED. ``` 内存分配: 自实现内存池 ```bash lian@ubuntu:~/share/9.1-kvstore$ ./test-redis/testcase 192.168.10.129 8888 3 Connected to 192.168.10.129:8888 BATCH (N=3000000) --> time_used=3552 ms, qps=844594 BATCH (N=3000000) --> time_used=3404 ms, qps=881316 BATCH (N=3000000) --> time_used=3083 ms, qps=973078 BATCH (N=3000000) --> time_used=3315 ms, qps=904977 BATCH (N=3000000) --> time_used=3462 ms, qps=866551 BATCH (N=3000000) --> time_used=3334 ms, qps=899820 BATCH (N=3000000) --> time_used=3358 ms, qps=893388 BATCH (N=3000000) --> time_used=3423 ms, qps=876424 BATCH (N=3000000) --> time_used=3319 ms, qps=903886 BATCH (N=3000000) --> time_used=3327 ms, qps=901713 BATCH (N=3000000) --> time_used=3157 ms, qps=950269 BATCH (N=3000000) --> time_used=3241 ms, qps=925640 BATCH (N=3000000) --> time_used=3301 ms, qps=908815 BATCH (N=3000000) --> time_used=3345 ms, qps=896860 BATCH (N=3000000) --> time_used=3319 ms, qps=903886 BATCH (N=3000000) --> time_used=3312 ms, qps=905797 BATCH (N=3000000) --> time_used=3337 ms, qps=899011 BATCH (N=3000000) --> time_used=3309 ms, qps=906618 BATCH (N=3000000) --> time_used=3385 ms, qps=886262 BATCH (N=3000000) --> time_used=3328 ms, qps=901442 BATCH (N=3000000) --> time_used=3194 ms, qps=939261 BATCH (N=3000000) --> time_used=3309 ms, qps=906618 BATCH (N=3000000) --> time_used=3262 ms, qps=919681 BATCH (N=3000000) --> time_used=3314 ms, qps=905250 BATCH (N=3000000) --> time_used=3382 ms, qps=887049 BATCH (N=3000000) --> time_used=3296 ms, qps=910194 BATCH (N=3000000) --> time_used=3331 ms, qps=900630 BATCH (N=3000000) --> time_used=3279 ms, qps=914913 BATCH (N=3000000) --> time_used=2996 ms, qps=1001335 BATCH (N=3000000) --> time_used=3387 ms, qps=885739 BATCH (N=3000000) --> time_used=3346 ms, qps=896592 BATCH (N=3000000) --> time_used=3241 ms, qps=925640 BATCH (N=3000000) --> time_used=3353 ms, qps=894721 BATCH (N=3000000) --> time_used=3366 ms, qps=891265 BATCH (N=3000000) --> time_used=3387 ms, qps=885739 BATCH (N=3000000) --> time_used=3382 ms, qps=887049 BATCH (N=3000000) --> time_used=3358 ms, qps=893388 BATCH (N=3000000) --> time_used=3372 ms, qps=889679 BATCH (N=3000000) --> time_used=3467 ms, qps=865301 BATCH (N=3000000) --> time_used=3243 ms, qps=925069 BATCH (N=3000000) --> time_used=3191 ms, qps=940144 BATCH (N=3000000) --> time_used=3365 ms, qps=891530 BATCH (N=3000000) --> time_used=3218 ms, qps=932256 BATCH (N=3000000) --> time_used=3332 ms, qps=900360 BATCH (N=3000000) --> time_used=3172 ms, qps=945775 BATCH (N=3000000) --> time_used=3211 ms, qps=934288 BATCH (N=3000000) --> time_used=3124 ms, qps=960307 BATCH (N=3000000) --> time_used=3043 ms, qps=985869 BATCH (N=3000000) --> time_used=3086 ms, qps=972132 BATCH (N=3000000) --> time_used=3201 ms, qps=937207 average qps:911106 ALL TESTS PASSED. ``` 内存分配：jemalloc ```shell lian@ubuntu:~/share/9.1-kvstore$ ./test-redis/testcase 192.168.10.129 8888 3 Connected to 192.168.10.129:8888 BATCH (N=3000000) --> time_used=3197 ms, qps=938379 BATCH (N=3000000) --> time_used=3221 ms, qps=931387 BATCH (N=3000000) --> time_used=3360 ms, qps=892857 BATCH (N=3000000) --> time_used=3292 ms, qps=911300 BATCH (N=3000000) --> time_used=3407 ms, qps=880540 BATCH (N=3000000) --> time_used=3317 ms, qps=904431 BATCH (N=3000000) --> time_used=3337 ms, qps=899011 BATCH (N=3000000) --> time_used=3384 ms, qps=886524 BATCH (N=3000000) --> time_used=3355 ms, qps=894187 BATCH (N=3000000) --> time_used=3379 ms, qps=887836 BATCH (N=3000000) --> time_used=3243 ms, qps=925069 BATCH (N=3000000) --> time_used=3377 ms, qps=888362 BATCH (N=3000000) --> time_used=3212 ms, qps=933997 BATCH (N=3000000) --> time_used=3248 ms, qps=923645 BATCH (N=3000000) --> time_used=3234 ms, qps=927643 BATCH (N=3000000) --> time_used=3152 ms, qps=951776 BATCH (N=3000000) --> time_used=3089 ms, qps=971188 BATCH (N=3000000) --> time_used=3287 ms, qps=912686 BATCH (N=3000000) --> time_used=3079 ms, qps=974342 BATCH (N=3000000) --> time_used=3261 ms, qps=919963 BATCH (N=3000000) --> time_used=3123 ms, qps=960614 BATCH (N=3000000) --> time_used=3234 ms, qps=927643 BATCH (N=3000000) --> time_used=3056 ms, qps=981675 BATCH (N=3000000) --> time_used=3040 ms, qps=986842 BATCH (N=3000000) --> time_used=3187 ms, qps=941324 BATCH (N=3000000) --> time_used=3311 ms, qps=906070 BATCH (N=3000000) --> time_used=3155 ms, qps=950871 BATCH (N=3000000) --> time_used=3318 ms, qps=904159 BATCH (N=3000000) --> time_used=3372 ms, qps=889679 BATCH (N=3000000) --> time_used=3254 ms, qps=921942 BATCH (N=3000000) --> time_used=3386 ms, qps=886001 BATCH (N=3000000) --> time_used=3413 ms, qps=878992 BATCH (N=3000000) --> time_used=3474 ms, qps=863557 BATCH (N=3000000) --> time_used=3412 ms, qps=879249 BATCH (N=3000000) --> time_used=3414 ms, qps=878734 BATCH (N=3000000) --> time_used=3325 ms, qps=902255 BATCH (N=3000000) --> time_used=3346 ms, qps=896592 BATCH (N=3000000) --> time_used=3345 ms, qps=896860 BATCH (N=3000000) --> time_used=3582 ms, qps=837520 BATCH (N=3000000) --> time_used=3412 ms, qps=879249 BATCH (N=3000000) --> time_used=3370 ms, qps=890207 BATCH (N=3000000) --> time_used=3375 ms, qps=888888 BATCH (N=3000000) --> time_used=3190 ms, qps=940438 BATCH (N=3000000) --> time_used=3324 ms, qps=902527 BATCH (N=3000000) --> time_used=3253 ms, qps=922225 BATCH (N=3000000) --> time_used=3230 ms, qps=928792 BATCH (N=3000000) --> time_used=3294 ms, qps=910746 BATCH (N=3000000) --> time_used=3295 ms, qps=910470 BATCH (N=3000000) --> time_used=3148 ms, qps=952986 BATCH (N=3000000) --> time_used=3228 ms, qps=929368 average qps:914031 ALL TESTS PASSED. ``` ### 面试题 1. 为什么会实现kvstore，使用场景在哪里？ 2. reactor, ntyco, io_uring的三种网络模型的性能差异？ 3. 多线程的kvstore该如何改进？ 4. 私有协议如何设计会更加安全可靠？ 5. 协议改进以后，对已有的代码有哪些改变？ 6. kv引擎实现了哪些？ 7. 每个kv引擎的使用场景，以及性能差异？ 8. 测试用例如何实现？并且保证代码覆盖率超过90% 9. 网络并发量如何？qps如何？ 10. 能够跟哪些系统交互使用？ ### 架构设计 ![image](https://disk.0voice.com/p/py)