Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Current status of leaf? #108

Open
anp opened this issue Jun 15, 2016 · 45 comments
Open

Current status of leaf? #108

anp opened this issue Jun 15, 2016 · 45 comments

Comments

@anp
Copy link

anp commented Jun 15, 2016

From this post:

Which is why Max and I will suspend the development of Leaf and focus on new ventures.

Is this accurate? If so, should the README mention so?

Autumn is also still listed on http://rust-lang.org/friends.html, is Autumn still a thing? If so, will the company continue to use Rust?
@MichaelHirn
Copy link
Member

Hi @dikaiosune,

We are procrastinating really hard on updating the README, although we should because Autumn is in the process of shutting down.

Also, no one stepped up to take over Leaf which means Leaf is rather dead at the moment, too.

@neuralvis
Copy link

@MichaelHirn I can make an effort to take over leaf, and it's associated libraries. But I will be needing a lot of help and assistance in assimilating the code. I believe there is a lot of value in collenchyma framework - especially in building robust code for large scale data visualization and analysis pipelines.

@kination
Copy link

I also agree with @vyslexic too. Still lot of people are interested on leaf, and it seems bad idea to close it. Instead of shutting down, how about take over to someone else who wants too?

@KodrAus
Copy link
Contributor

KodrAus commented Oct 11, 2016 •
edited
Loading

@vyslexic I'm a hobbyist with an expensive graphics card, so don't have a lot of GPU experience but would be happy to step up and help if someone with more experience took it on too.

I was working through the documentation book when Autumn wound down.

@neuralvis
Copy link

@MichaelHirn , @hobofan - I am happy to carry forward your work with autumnai. How would you guys like to proceed ? You can either add me to the autumnai org or I can start maintaining a mirror.

@KodrAus, @djKooks - we can use all the help we can to develop the community and the software.

@kination
Copy link

@vyslexic good to hear that! Please share the progress of carrying it on.

@hobofan
Copy link
Member

hobofan commented Oct 12, 2016 •
edited
Loading

Hey everyone, @MichaelHirn and me wanted to write down a well formulated statement, but I never seem to find the time for it, so I'll just write out right now what we decided on:

We both don't feel comfortable handing over the maintainership over Leaf's and the other Autumn repositories to a maintainer we don't know well, and also don't have the time to vet one. Another motivation for that is, that both Leaf and Collenchyma can develop into very different directions from here on, and we don't want to endorse one over the other. If the projects would have had more time to mature and grow a healthy community around them, that would probably be a different story, but that's not the situation we are in right now.

So we feel that the best way moving forward to ensure that the code we built continues to provide value, and that you, the community can start to build a healthy project is for anyone wanting to continue with the project to do a hard fork.

Feel free to discuss forming a hard fork and linking them in the issues of the relevant repositories. We will try our best to make them easy to find by linking them in the READMEs, so that anyone wanting to contribute can find the new projects.

@MichaelHirn
Copy link
Member

Thanks, @hobofan.

I agree, but am open to being convinced otherwise. If taking over Autumn instead of forking it, is more relevant to your plans, @vyslexic, I'm happy to do a 15 min call in the next few days. (Max, happy to have you on the call if you want to.)

@kination
Copy link

Thanks to @MichaelHirn, @hobofan . @vyslexic please share us the result about it. Lots of people are interested on it.

@neuralvis
Copy link

@djKooks - my apologies for intermittent responses. It has been a busy last few days.
@hobofan, @MichaelHirn - thanks for clarifying the recommended approach moving forward. I understand your concerns, and fully respect them. My intention wasn't to "take over" as maintainer, but to merely be added as a contributor to the group, so I can take an active role in handling issues, and arbitrating pull requests etc. I am content with working on a fork as well.

Explaining the motivation and plans behind my interest in leaf/collenchyma will require a long blog post, and I hope to get there some day (soonish!). But briefly - my background and contributions in the past few years (~8yrs) has been in the area of high performance visualization, in-situ analytics, and data mining for large scale (~petabytes) scientific (HPC) applications. A recurring frustration that I often encounter is the lack of robust, scalable, fault-tolerant software (for analytics and visualization) that "just works" on large scale HPC systems.

The "state-of-the-art" software libraries such as Paraview, VTK are around ~20 years old, and do not support the computing paradigms (both hardware and software) that are evolving (hybrid architectures, fine-grained concurrency, deep learning etc). Solutions like Hadoop and Spark may be engineered but do not map very well to the problem domains we are looking into (more on this on my yet-to-come blog).

This is where I believe leaf/collenchyma can really add value. First, they are based on Rust, and therefore inherit all the guarantees that come with the language. Second, a library such as leaf/collenchyma can be combined with a fault-tolerant, soft real-time system (erlang, elixir) to provide a data analysis/machine intelligence/visualization framework with the following benefits:

  • A highly-available run-time system that is
  • Massively scalable on-demand,
  • Is fault-tolerant and "self-reparing" to some extent,
  • and can scale-up and scale-out, effectively at minimal development cost.

This is a long-term goal, of course, and I would hope to achieve this with the help of many other contributors that have expressed interest here, and elsewhere in my community.
I hope this answers a few questions you may have had. I come from a C/C++ family of languages, so I would say I am an intermediate Rust programmer. It will still take me a while to get up-to-speed with the Rust code in these repositories.

Happy to chat more on Slack/Skype/Hangout if needed ...

@MichaelHirn
Copy link
Member

Thank you, @vyslexic. I am looking forward to the blog post, to learn more about it - sounds very cool.

I think (@hobofan, let me know if you disagree), that it would be best, given your objections, that to me, seem more complementary to Rust/Leaf/Collenchyma's general environment than to Leaf/Collenchyma-the library and the initial goals that we had, to start a fork and see from there.

@neverfox
Copy link

Also interested and would like to join in on any discussions.

1 similar comment
@sunface
Copy link

sunface commented Oct 25, 2016

Also interested and would like to join in on any discussions.

@kination
Copy link

@vyslexic do you have some next plan or schedule? It would be helpful to people who wants to support the project.

@dlight
Copy link

dlight commented Oct 25, 2016

Just a thought: perhaps a software foundation like the Apache foundation could accept this project under their umbrella?

@ehiggs
Copy link

ehiggs commented Oct 28, 2016

Apache is not a dumping ground for OSS. Apache also doesn't magically provide resources to a project.

@castarco
Copy link

Hi, I'm very interested on this project. I'd love to contribute to it. I'm not an expert, but I love Rust and ML, and I'm working on a PHP extension programmed in Rust which could use Leaf as a dependency.

@kination
Copy link

kination commented Jan 3, 2017

Hello~
Is there some progress on plan in this project?

@ghost
Copy link

ghost commented Jan 4, 2017

Suggestion: Perhaps @alexandermorozov's fork could become a new venue for development? At a glance, his contributions appear to be the most active outside the core AutumnAI folks.

@hobofan @MichaelHirn - If Alexander is willing to be considered a "hard fork" maintainer, even temporarily, would you be willing to link to his repo from the README here?

I'd really love to see leaf live. It's an OpenCL-as-first-class learning framework written in an amazing high-performance language that makes mistakes hard. Right now, for OpenCL, the choices are pretty bleak (improving super rapidly but still..); Torch, if you like Lua and Magic, or Tensorflow, if you like proprietary compiler frameworks (required for OpenCL). Leaf could really carve out a following by approaching the hot machine-learning-learning segment. Using OpenCL, people can learn ML on the integrated graphics of their CPUs, they don't need to invest in expensive NVidia hardware!

So please, let's elect or nominate a new interim BDFL and see where Leaf can go? :)

@alexandermorozov
Copy link
Contributor

@cathalgarvey Initially I didn't offer to take on maintainership because I wanted to explore graph-based approach mostly for anticipated performace reasons.

Currently I don't have enough time, energy and interest to move Leaf forward or to maintain the project. If situation changes I'll do the work in my own fork and @hobofan and @MichaelHirn could later decide if they like the direction and results and if they want to make it official.

With regards to OpenCL IIRC there is only a memory allocator in collenchyma. Generic tensor operations and NN specific stuff are yet to be implemented.

@ghost
Copy link

ghost commented Jan 8, 2017 via email

@kination
Copy link

@MichaelHirn @hobofan Hello~

It looks like still lot of people are interest in this project, and some of are keep updating PR though it cannot be pulled. Could you let people know just how this project will be(like just closing or handing over maintainership to somebody) or it is still thinking about it? It will help people here to go on to next step.

It is sad to let this great project unsettled.

@hobofan
Copy link
Member

hobofan commented Jan 11, 2017

@djKooks We already did that. As of now there are no plans from us to maintain this repository.

Our recommended method of moving forward for the community is still to do a hard fork, but as I the comments here read, nobody seems to be interested enough in working on the project and actively driving it forward in a maintainership role.

@kination
Copy link

@hobofan Thanks for quick answer. One more, do you have some criteria of a maintainer?

It will a hard job to be a maintainer of quite a massive project. Most of engineer would have their job, and don't have time to participate in full-time. How about making a community for managing this project? It would be more better if each people share there time together.
Is there someone following this issue who is interest in?

@alexandermorozov
Copy link
Contributor

@cathalgarvey One problem is that there are no code PRs since shutdown, only small documentation fixes. People will not likely contribute much effort to a dead project, so what is dead tends to stay dead. Also Leaf with other crates is quite complex in itself, so there is a rather high barrier to entry. To move forward someone should put effort in and continue development, after that others may want to join in. Another path is to fork it, then conduct a massive PR campaign to rekindle interest in Leaf but it's not very likely to succeed.

I'm currently restarting a private project that may benefit from a NN lib in rust in the future. I'm quite familiar with Leaf, so if there would be perceived advantages of developing own tools instead of using Tensorflow/... I may continue work on Leaf in my fork. That's not to dissuade anyone from forking, of course do it if you have inclination! I have quite a lot going on and may not get around to work on it again, life is quite unpredictable )

@kination
Copy link

@alexandermorozov thanks for good message. I'm also looking on leaf but because it's not quite long since I started studying Rust/ML, it is hard going in.
Hope someone make this work again.

@castarco
Copy link

castarco commented Jan 12, 2017 via email

@alexandermorozov
Copy link
Contributor

I've had a little time to merge most of PRs into my "fork" manually except #68 (seems to be already fixed), #75 (author has deleted his branch), autumnai/collenchyma-nn#46 (it's a great work but after my refactoring it's unmergeble, I'll adapt it later). I've also changed cargo files to point to my forks on github. As of now leaf-examples fails to compile, everything else should.

Short-term plans:

  • fix and merge Refactor plugin impls to be generic; implement ndim convolution (no grad for now); fix conv tests. collenchyma-nn#46,
  • consider applying collenchyma/decoupling branch in my repo. While it adds some decoupling, I want it for properly typed memory in user API of SharedTensor. There are tradeoffs though,
  • see what can be done about Arc<RWLock<_>> everywhere in leaf code,
  • IDevice, IHardware, IMemory, IFramework, IBinary traits, DeviceType, MemoryType -- read code and figure out what does what, and if there is a chance to reduce complexity,
  • see what can be done about feature flags and conditional compilation. Currently some combinations of flags produce warnings or compilation failures. If decoupling makes it in, things could be possibly simplified,
  • there are lots of unwraps in leaf code. Maybe reduce error boilerplate in collenchyma with error-chain,
  • ensure that NNs really work. There were some issues, they have been fixed, but I'm not sure that learning really works as it should,
  • try to keep docs up-to-date; fix statements about the scope of the project,

Long term plans (if there is a long term):

  • proper hard fork (select name, do fork, create org, releases on crates.io),
  • consider using a tensor lib for native memory (instead of flat slices), ndarray maybe.

@ghost
Copy link

ghost commented Jan 15, 2017 via email

@kination
Copy link

@alexandermorozov Seems good~Is it okay to fork on your project?

@alexandermorozov
Copy link
Contributor

@cathalgarvey I like this suggestion ) Though it'll be a long time until hard fork makes sense. I think it'll take me about 3 months to work through the list of short-term issues at the current pace. That's if everything in life stays more or less the same.

@djKooks Yes, feel free to fork it!

@botev
Copy link

botev commented Jan 18, 2017

@cathalgarvey @alexandermorozov @djKooks @castarco @hobofan I recently posted this suggestion for a full Graph based framework/compiler with autodiff in rust on reddit https://www.reddit.com/message/inbox/. The main goal of that would be to reuse the backend/lower level compute of what Leaf already has for a start, but use a graph based framework to make it very flexible and comparable to other frameworks in the wild. I'm currently finishing writing some of the major parts of the graph with the autodiff, however I have no expertise in how to make this then translate to the low level part. We had and some discussion on reddit that might be of use to you guys.

@hobofan
Copy link
Member

hobofan commented Jan 18, 2017

Correct link to the reddit discussion: https://www.reddit.com/r/rust/comments/5of7t8/is_anyone_interested_in_trying_to_build_a_graph/

@botev
Copy link

botev commented Jan 18, 2017

I've made a gitter room where potentially if we get to be in at the same time can have a slightly more legthy discussion: https://gitter.im/rust-ml/Lobby?utm_source=share-link&utm_medium=link&utm_campaign=share-link

Sorry for the bad link last time -.-

@botev
Copy link

botev commented Jan 27, 2017

@alexandermorozov I've saw you are interested in graph interface and I just want to make an update on it. I currently implemented it for a very limited number of ops. One difference with all other current ML frameworks is that it validate shapes(including dynamic) at graph construction, thus this gives you "graph construction" time guarantee that if there is no error, the resulting operations are valid (This is the spirit of rust and in contrast to Theano and Tf where you can get this cryptic error messages at runtime). Currently the interface looks like this:

    let g = gir::Graph::default();
    let x = &f_var!(g, (784, "n"), "input");
    let y = &f_var!(g, ("n"), "target");
    let y_t = &transpose(y)?;
    let w1 = &f_param!(g, (1, 784), "w1")?;
    let b1 = &f_param!(g, (1), "b1")?;
    let h1 = &tanh((mat_mul(w1, x)? + b1))?;
    let error = sum_all((h1 - y_t) * (h1 - y_t))? / dim0(y)?;
    let params = &vec![w1, b1];
    let grads = gir::derivative::gradient(error, params)?;

However, currently the project lacks any backend at all for executing the graph. Any help on that front would be most helpful.

@kazimuth
Copy link

kazimuth commented Feb 8, 2017

@MichaelHirn @hobofan

After nearly a year, the README has still not been updated stating that this project has been completely abandoned (barring a hard fork sometime in the future). It's confusing and misleading, since the project has thousands of stars and a lot of apparent polish.

A single commit adding "this project is dead, don't expect updates or support" to the top of the README would do wonders to keep people from wasting time trying to learn it.

Similarly, it's entirely non-obvious that autumnai.com is defunct, and a header there would be helpful as well.

@Enet4
Copy link

Enet4 commented Feb 8, 2017

@kazimuth On that end, I agree that the project should not leave a false sense of security. Adding the "no maintenance intended" badge (No Maintenance Intended) should be sufficiently alarming, but we also don't want people to be steered away from a potential fork, so we should also leave a link to this issue or the project where all future progress will take place.

@jonysy
Copy link

jonysy commented Mar 7, 2017

Hello all!

I and a few others are considering hard-forking Leaf since we've already become familiar with it.

From an HN discussion regarding the flexibility of Leaf's current design:

[HN commenter]: I think in particular the layer-based approach makes it awkward to represent more complicated models since not every model can be decomposed into backward/forward passes.

M. Goisser: I am very optimistic concerning our plans to make Leaf work in a distributed fashion. It's not quite finalized yet, but the rough outline consists of abstracting other parts of the Network away as a special type of layer and using capnproto + zeromq for transportation. Sadly I haven't come across a good implementation of model parallelism to learn from, so that might become a bit tricky.

I think there is still a lot of room for improvement for layer-based networks, but even if the graph-based approach will become more prelevant I think the Rust trait system gives us great tools to migrate and reuse a lot from our layer based approach.

@MichaelHirn @hobofan Would you two mind elaborating on this? The question came up a lot during Leaf's lifetime, but I couldn't find a clear, detailed answer..

@Boscop
Copy link

Boscop commented Mar 15, 2017

Is there a new maintained hard fork now? It's sad to see all this hard work go to waste..

@Binero
Copy link

Binero commented Mar 16, 2017

@Boscop At least one guy seems to be maintaining it, haven't tried it though: https://github.com/ratpoison-io/leaf

@jonysy
Copy link

jonysy commented Mar 18, 2017 •
edited
Loading

@Boscop @Binero here's a quick overview:

It seems that @drahnr is currently working off of @alexandermorozov's fork (last commit was 2 days ago at the time of typing this post), which hopefully means that I can abandon my leaf fork. In any case, it would make for an awesome community project.

@jonysy
Copy link

jonysy commented Mar 18, 2017

I'd be willing to adapt Leaf for Parenchyma if there's interest.

How can we go about having a community-maintained fork of Leaf? Any suggestions would be appreciated.

@drahnr
Copy link

drahnr commented Mar 18, 2017 •
edited
Loading

@jonysy I would definitely be interested regarding the changes and goals of parenchyma vs today's collenchyma :) drop a mail or find me on gitter, I am open for discussion and merging efforts

@jonysy
Copy link

jonysy commented Mar 18, 2017

@drahnr 👍 great! I sent you a message via Gitter.

@drahnr
Copy link

drahnr commented Apr 10, 2017 •
edited
Loading

just a quick update: I am working on cuda 5.1 support which is almost done and adaptations on the cudnn backend for collenchyma-nn (partially based on @DiamondLovesYou 's refactorings adopted to the new rw tensor API introduced by @alexandermorozov ).

Afterwards I am planning to rust-ify the leaf interface and polish the examples.

@hobofan hobofan mentioned this issue Jun 4, 2017
Closed
@DeadZen DeadZen mentioned this issue Jun 4, 2017
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests