Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[New Feature] Development List TO DO #20

Open
peeyee opened this issue Aug 16, 2023 · 2 comments
Open

[New Feature] Development List TO DO #20

peeyee opened this issue Aug 16, 2023 · 2 comments
Labels
enhancement New feature or request good first issue Good for newcomers help wanted Extra attention is needed

Comments

@peeyee
Copy link
Collaborator

peeyee commented Aug 16, 2023

To do list for new Contributors

Contributions are welcome, here is a list for new contributors:

No Plugin Content Priority Difficulty Contributer Status Issue/PR
1 clickhouse   low medium      
2 hdf5 add docs about the perfect way of loading data and use cases. medium low      
3 httpClient support for windows high medium      
4 kafka support for kerberos low medium      
5 kafka support for windows high medium      
6 kdb support for using loadTextEx medium medium      
7 kdb support for reading  compressed data of type q IPC (1), snappy (3), L4Z (4) high high      
8 mqtt support for arm low medium      
9 mqtt publish stream engine calculation results directly to the mqtt server medium medium      
10 mysql add docs about return type medium low      
11 mysql support for type blob high high      
12 mysql support for type decimal high high      
13 oracle   high high      
14 parquet support for reading column of  type repeated high high      
15 parquet support for windows high medium      
16 signal support for rfft, irfft high high      
17 xgboost upgrade the library of xgboost to version 1.7 medium medium      
18 zip support for arm low medium      
19 zip support for compressing folders high medium      
@peeyee peeyee added the enhancement New feature or request label Aug 16, 2023
@peeyee peeyee pinned this issue Aug 16, 2023
@peeyee peeyee added the help wanted Extra attention is needed label Aug 16, 2023
@peeyee peeyee changed the title [New Feature] Develop List TO DO [New Feature] Development List TO DO Aug 16, 2023
@peeyee peeyee added the good first issue Good for newcomers label Aug 16, 2023
@FlyingOE
Copy link

Item No.6 kdb+'s tasks: Is it referring to direct file access (similar to loadFile), bypassing q process (similar to loadTable)?

@peeyee
Copy link
Collaborator Author

peeyee commented Aug 17, 2023

Yes, the way of loadTable/File is totally loading the data into DolphinDB's in-memory table. Thus comes a problem if we do not have enough memory,especially for community users.
So we want to have a method loadTextEx that directly loading data into DolphinDB's DFS table, meanwhile reduce the memory usage, for example, each batch costs 512MB.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request good first issue Good for newcomers help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

2 participants