lian/zvfs
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

zvfs: 重构核心读写逻辑并新增 POSIX hook；引入目录元数据与伪 fd；分离创建、打开、关闭、删除的逻辑；

Browse Source
This commit is contained in:
1iaan
2026-02-22 14:46:17 +00:00
parent f216f73dda
commit 31dc307d0b
17 changed files with 1931 additions and 1727 deletions
Download Patch File Download Diff File Expand all files Collapse all files

295
README.md
View File

@@ -1,92 +1,257 @@
# 5.3.1 Zv2404fs
## Getting started
To make it easy for you to get started with GitLab, here's a list of recommended next steps.
Already a pro? Just edit this README.md and make it your own. Want to make it easy? [Use the template at the bottom](#editing-this-readme)!
## Add your files
- [ ] [Create](https://docs.gitlab.com/ee/user/project/repository/web_editor.html#create-a-file) or [upload](https://docs.gitlab.com/ee/user/project/repository/web_editor.html#upload-a-file) files
- [ ] [Add files using the command line](https://docs.gitlab.com/ee/gitlab-basics/add-file.html#add-a-file-using-the-command-line) or push an existing Git repository with the following command:
```
cd existing_repo
git remote add origin http://gitlab.0voice.com/2404_vip/5.3.1-zv2404fs.git
git branch -M main
git push -uf origin main
## usage
```shell
```
## Integrate with your tools
- [ ] [Set up project integrations](http://gitlab.0voice.com/2404_vip/5.3.1-zv2404fs/-/settings/integrations)
## SPDK
1. blob_store: blob仓库，管理多个blob对象。
2. blob: 存储对象，逻辑上连续，物理上不一定连续。相当于文件。
3. cluster: 分配单元，一个 blob 可以由多个 cluster 构成，扩容即分配新的 cluster。相当于文件系统的block group。
4. page: IO单元，一个 cluster 等于多个 page。
## Collaborate with your team
文件系统
- [ ] [Invite team members and collaborators](https://docs.gitlab.com/ee/user/project/members/)
- [ ] [Create a new merge request](https://docs.gitlab.com/ee/user/project/merge_requests/creating_merge_requests.html)
- [ ] [Automatically close issues from merge requests](https://docs.gitlab.com/ee/user/project/issues/managing_issues.html#closing-issues-automatically)
- [ ] [Enable merge request approvals](https://docs.gitlab.com/ee/user/project/merge_requests/approvals/)
- [ ] [Automatically merge when pipeline succeeds](https://docs.gitlab.com/ee/user/project/merge_requests/merge_when_pipeline_succeeds.html)
## 架构设计
```scss
| 应用程序
| (POSIX API: open/read/write/close)
| LD_PRELOAD 拦截层
| (简单路径判断和转发到zvfs)
| zvfs 文件系统层
| (blob 操作)
| SPDK Blobstore
| 块设备 (Malloc0)
```
## Test and Deploy
### 磁盘布局
```scss
BlobStore:
|—— Super Blob（元数据，使用SPDK的Super Blob锚定）
|——超级块
|——目录项/目录日志
|—— Blob 1 (文件A...)
|—— Blob 2 (文件B...)
|—— Blob N (文件C...)
```
Use the built-in continuous integration in GitLab.
- [ ] [Get started with GitLab CI/CD](https://docs.gitlab.com/ee/ci/quick_start/index.html)
- [ ] [Analyze your code for known vulnerabilities with Static Application Security Testing(SAST)](https://docs.gitlab.com/ee/user/application_security/sast/)
- [ ] [Deploy to Kubernetes, Amazon EC2, or Amazon ECS using Auto Deploy](https://docs.gitlab.com/ee/topics/autodevops/requirements.html)
- [ ] [Use pull-based deployments for improved Kubernetes management](https://docs.gitlab.com/ee/user/clusters/agent/)
- [ ] [Set up protected environments](https://docs.gitlab.com/ee/ci/environments/protected_environments.html)
### 数据结构
#### Super Blob（元数据）
```scss
[超级块]
- magic_number: 0x5A563146 (ZV1F)
- version: 1
***
[目录项]
- filename[256]: 文件名
- blob_id: 对应的数据blob ID
- file_size: 文件实际大小（字节）
- allocated_clusters: 已分配的cluster数量
- is_valid: 标记是否有效（用于删除）
```
# Editing this README
```c
/* 目录项（内存中的目录） */
typedef struct {
char filename[256];
spdk_blob_id blob_id;
uint64_t file_size; // 文件逻辑大小（字节）
uint32_t allocated_clusters; // 已分配的cluster数量
bool is_valid; // false 表示已删除
int32_t open_count; // 打开的文件句柄数量
} zvfs_dirent_t;
When you're ready to make this README your own, just edit this file and use the handy template below (or feel free to structure it however you want - this is just a starting point!). Thank you to [makeareadme.com](https://www.makeareadme.com/) for this template.
/* 文件系统全局结构 */
typedef struct zvfs {
struct spdk_blob_store *bs;
struct spdk_io_channel *channel;
struct spdk_blob *super_blob; // 承载目录日志的blob
uint64_t io_unit_size; // page大小，单位字节
## Suggestions for a good README
Every project is different, so consider which of these sections apply to yours. The sections used in the template are suggestions for most open source projects. Also keep in mind that while a README can be too long and detailed, too long is better than too short. If you think your README is too long, consider utilizing another form of documentation rather than cutting out information.
/* 目录 */
zvfs_dirent_t *dirents; // 目录项数组 #define ZVFS_MAX_FILES 1024
uint32_t dirent_count; // 当前有效项数
## Name
Choose a self-explaining name for your project.
/* 伪FD表 */
struct zvfs_file *fd_table[ZVFS_MAX_FD]; // // e.g., #define ZVFS_MAX_FD 64
int fd_base; // 伪FD起始值，如10000
int openfd_count;
## Description
Let people know what your project can do specifically. Provide context and add a link to any reference visitors might be unfamiliar with. A list of Features or a Background subsection can also be added here. If there are alternatives to your project, this is a good place to list differentiating factors.
/* 元数据 */
uint32_t magic; // 0x5A563146 (ZV1F)
uint32_t version; // 1
} zvfs_t;
## Badges
On some READMEs, you may see small images that convey metadata, such as whether or not all the tests are passing for the project. You can use Shields to add some to your README. Many services also have instructions for adding a badge.
/* 打开的文件句柄 */
typedef struct zvfs_file {
zvfs_t *fs;
struct spdk_blob *blob;
zvfs_dirent_t *dirent; // 指回目录项 file_size/allocated_clusters
uint64_t current_offset; // 当前读写位置
int flags; // O_RDONLY / O_RDWR / O_CREAT 等
int pseudo_fd;
## Visuals
Depending on what you are making, it can be a good idea to include screenshots or even a video (you'll frequently see GIFs rather than actual videos). Tools like ttygif can help, but check out Asciinema for a more sophisticated method.
/* 临时DMA缓冲区（可选：每个file一个，避免每次malloc） */
void *dma_buf;
uint64_t dma_buf_size;
} zvfs_file_t;
```
## Installation
Within a particular ecosystem, there may be a common way of installing things, such as using Yarn, NuGet, or Homebrew. However, consider the possibility that whoever is reading your README is a novice and would like more guidance. Listing specific steps helps remove ambiguity and gets people to using your project as quickly as possible. If it only runs in a specific context like a particular programming language version or operating system or has dependencies that have to be installed manually, also add a Requirements subsection.
### 工作流程
#### mount
hook POSIX API没有很好的调用时机，单线程目前采用懒加载。
```scss
1. [创建块设备]
- spdk_bdev_create_bs_dev_ext
2. [初始化文件系统]
- spdk_bs_init 或者 spdk_bs_load（已有数据时）
- spdk_bs_get_io_unit_size 获取io单元大小(page)
- spdk_bs_alloc_io_channel 分配blobstore的读写入口
3. [读取元数据]
- spdk_bs_get_super_blob 获取 Super Blob ID
- spdk_bs_open_blob 打开 Super Blob
- 读取超级块，校验 magic
- 读取目录项数组，加载到内存 dirents
4. [创建zvfs_t结构体]
- 创建 zvfs_t 结构体
- 填充 bs/channel/super_blob/dirents 等字段
```
#### open
##### O_RDONLY / O_RDWR
```scss
1. [文件名查找]
- 遍历 dirents，匹配 filename 且 is_valid=true
- 找不到返回 -ENOENT
2. [打开blob]
- spdk_bs_open_blob(dirent->blob_id)
- dirent->open_count++
- fs->openfd_count++
3. [分配文件句柄]
- 创建 zvfs_file_t，dirent 指针指向目录项
- 分配伪FD，写入 fd_table
5. [返回伪FD]
```
## Usage
Use examples liberally, and show the expected output if you can. It's helpful to have inline the smallest example of usage that you can demonstrate, while providing links to more sophisticated examples if they are too long to reasonably include in the README.
##### O_CREAT
```scss
1. [文件名查找]
- 遍历 dirents，若 filename 已存在且 is_valid=true，返回 -EEXIST
- 找一个 is_valid=false 的空槽位；没有空槽则追加（dirent_count < max_files）
2. [创建blob]
- spdk_bs_create_blob → 得到 blob_id
- spdk_bs_open_blob → 得到 blob 句柄
- spdk_blob_resize 初始分配空间
- spdk_blob_sync_md 持久化 cluster 分配
3. [写目录]
- 填充 filename/blob_id/file_size=0/is_valid=true
- dirent->open_count = 1
4. [创建文件句柄]
- 创建 zvfs_file_t
- 分配伪FD，写入 fd_table
5. [返回伪FD]
## Support
Tell people where they can go to for help. It can be any combination of an issue tracker, a chat room, an email address, etc.
```
> 说明：目录变更只写内存，unmount 时统一持久化。
## Roadmap
If you have ideas for releases in the future, it is a good idea to list them in the README.
### read
读写都以字节为单位，offset / count 单位为字节；根据 io_unit_size 做对齐计算。
## Contributing
State if you are open to contributions and what your requirements are for accepting them.
```scss
1. [参数]
- fd
- buffer
- count
- offset(隐含)
2. [边界检查]
- 实际可读 = min(count, dirent->file_size - current_offset)
- 实际可读为0则返回0
3. [计算Blob位置]
- start_page = current_offset / io_unit_size
- page_offset = current_offset % io_unit_size
- num_pages = (page_offset + 实际可读 + io_unit_size - 1) / io_unit_size
4. [DMA读取]
- 非对齐读(offset != 0 || count 不是整页)
- 需要DMA临时缓冲区（spdk_dma_zmalloc）
- spdk_blob_io_read(blob, channel, dma_buffer, start_page, num_pages, ...)
- 从 dma_buffer + page_offset 拷贝到用户 buffer
- 对齐
- 仍使用DMA缓冲区执行读取，再拷贝到用户buffer
5. [更新offset]
- current_offset += 实际可读
6. [返回实际读取字节数]
```
> 说明：SPDK需要DMA可用的内存，应用提供的用户缓冲区通常不满足要求。即便对齐也不能直接提交给spdk_blob_io_*，应使用DMA缓冲作为跳板；未来通过注册内存池可优化直传。
For people who want to make changes to your project, it's helpful to have some documentation on how to get started. Perhaps there is a script that they should run or some environment variables that they need to set. Make these steps explicit. These instructions could also be useful to your future self.
### write
```scss
1. [参数]
- fd
- buffer
- count
- offset(隐含)
2. [检查空间是否足够]
- 需要大小 = current_offset + count
- 若超过 allocated_clusters 对应容量：
- spdk_blob_resize 扩容
- spdk_blob_sync_md
- 更新 dirent->allocated_clusters
3. [计算写入位置]
- start_page / page_offset / num_pages（同read）
4. [DMA写入]
- 非对齐写(offset != 0 || count 不是整页)
- 读取涉及的首尾page到DMA临时缓冲区
- 修改对应位置的数据
- 写回：spdk_blob_io_write(blob, channel, dma_buffer, start_page, num_pages, ...)
- 对齐
- 仍通过DMA缓冲区提交写入
5. [更新状态]
- current_offset += count
- dirent->file_size = max(dirent->file_size, current_offset)
6. [返回写入字节数]
```
You can also document commands to lint the code or run tests. These steps help to ensure high code quality and reduce the likelihood that the changes inadvertently break something. Having instructions for running tests is especially helpful if it requires external setup, such as starting a Selenium server for testing in a browser.
## Authors and acknowledgment
Show your appreciation to those who have contributed to the project.
### close
```scss
1. [关闭Blob]
- spdk_blob_close(file->blob)
- dirent->open_count--
- fs->openfd_count++
- 若 open_count == 0 且 is_valid == false（已unlink）：spdk_bs_delete_blob, 清空dirent
- 若 openfd_count == 0 则 unmount
2. [释放缓冲区]
- 释放 dma_buf
- 清除 fd_table[pseudo_fd]
- free(zvfs_file_t)
3. [返回0]
```
### unlink
```scss
1. [查找目录项]
- 遍历 dirents，匹配 filename 且 is_valid=true
- 找不到返回 -ENOENT
2. [标记删除]
- dirent->is_valid = false
3. [判断是否立即删除]
- open_count == 0：spdk_bs_delete_blob，清空该槽位
- open_count > 0：延迟，最后一个 close 负责删除
4. [返回0]
```
## License
For open source projects, say how it is licensed.
### unmount
```scss
1. [关闭channel]
- spdk_bs_free_io_channel
2. [关闭BlobStore]
- spdk_bs_unload
3. [释放FS]
- free(fs)
```
## Project status
If you have run out of energy or time for your project, put a note at the top of the README saying that development has slowed down or stopped completely. Someone may choose to fork your project or volunteer to step in as a maintainer or owner, allowing your project to keep going. You can also make an explicit request for maintainers.
### 其他方案
如果不使用`LD_PRELOAD`hook，可以使用FUSE。\
FUSE是一种内核文件系统程序，挂载在文件目录上，对这个目录的访问，会使用这个文件系统程序。\
文件系统程序会将请求转发给应用层程序，这里的应用层程序可以是SPDK。这样就不用管其他的操作。