Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LocalFile的实现中有两个bug #129

Open
acmol opened this issue Nov 2, 2017 · 1 comment
Open

LocalFile的实现中有两个bug #129

acmol opened this issue Nov 2, 2017 · 1 comment

Comments

@acmol
Copy link
Contributor

acmol commented Nov 2, 2017

  1. ReadLine函数中,如果文件中有\0(例如hive导出的文件会用\0作分割符),则\0后的数据会被截断丢弃。
  2. max_size比较大时,ReadLine函数性能极差,大部分时间都花在resize操作中,将后边的内存memset为0的过程中了。

问题1可以直接修复。
问题2我认为可能是接口设计的问题,目前的接口下要么得多一次内存拷贝,要么得像现在这样在读取前进行resize,所以,我认为可能需要改File基类,添加一个bool LocalFile::ReadLine(StringPiece* line, size_t max_size)的接口。

不知道各位commiter认为是否可以按我说的方案修复。

@acmol
Copy link
Contributor Author

acmol commented Nov 2, 2017

如果可以,我这边后续会把我们这边相关的修改提交一个pull request

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant