Struct Sugar

[bvssvni]

Rendered

This idea came out of the discussion http://discuss.rust-lang.org/t/struct-sugar/551/29

Reddit discussions:

http://www.reddit.com/r/rust/comments/2hgspp/struct_sugar_and_using_it_for_keyword_and_default/
http://www.reddit.com/r/rust/comments/2hpu5k/extended_struct_sugar_named_parameters_defaults/

I felt it necessary to iterate on the idea alone, due to the many concerns and constraints, where the discussion did not give a full picture. However, the feedback from the community played a great role in the design.

Certain views were rejected because I could not see how they would work in practice. Under "alternatives", I try to give a justification of why I think these do not cover the same use cases. I did not cover the other RFCs for optional arguments, because they are different in syntax and do not address refactoring.

I don't have much knowledge of how the compiler works, so I hoped somebody else would draft up this RFC. The iteration on the design has taken quite a bit of time, and there are likely alternatives or refinement that I have not thought of. Therefore I want this RFC to be treated as an initial draft, while the final spec could be made in another RFC (hopefully not by me, haha).

It is probably useful to have more examples to avoid misunderstanding. I reserved a "corner cases" section for this.

[pnkfelix]

The desugaring needs to distinguish "optional" from "non-optional" arguments to decide whether to emit a call to the Optional::to_option() method or not.

Based on my cursory skim of the RFC, this decision (i.e. how "optional" and "non-optional" is distinguished) is made based solely on ... where the type of the corresponding field of the struct looks like Option<T>? Is that correct? How does that handle e.g. pub struct Wrapper<T>(Option<T>); ?

Or is it done for any type that implements the Optional trait?

[pnkfelix]

(also, I'm not sure that the RFC repository is the appropriate place to iterate on a draft RFC in the manner proposed here. I do appreciate that you explicitly stated that to be your intent, but it might be better for the early drafting process to be contained to the discuss site.)

[mahkoh]

Syntax for marking members of a struct public

Syntax for making all fields public is not useful. This is already easy unless you have an editor problem. On the other hand, this seems to be useful:

#[field_visibility = "pub"]
struct X {
    x: int,
    priv y: int,
}

And then maybe add a crate attribute #![field_visibility = "pub"] so that those who care about this can have all fields public by default.

[arthurprs]

Personally I don't like it. It encourages methods with tons of parameters and adds complexity to the language itself and API designs.

[bvssvni]

@pnkfelix The distinguishing between calls to .to_option() is the precense of the equal sign after the type parameter. When this is not present, it will not desugar for that parameter even if it implements the Optional trait.

[bvssvni]

@pnkfelix I did not get the feedback I wanted in the discussion thread that addressed corner cases, because there were too many and people could not see the overall picture, so without doing anything, I feared the idea would perish. I think this version could work, but will of course not proceed without discussing it in more detail.

There is a lot of potential here that might make Rust more interesting for the game industry, and instead of sacrificing safety, I think it is worth looking at other parts of the language. The focus on safety is precisely why some professional game developer perceive Rust as a "big idea" language, but the response has not only been one sided critical, but also full of constructive ideas around refactoring. I've read some blogs, and there are interesting ideas around the creative process of modelling real world connections through the language, starting from simple and then refactor to a more robust design. I wanted to put this in the motivation part, but I think the idea should stand on its own, and not be justified by how people are "supposed" to program.

The reason I've worked on this RFC is to see if I can start a discussion around refactoring, because it seems like an area where game programmers complain about language design. By showing that we care about this area, maybe we can get this group more interested in Rust?

[bvssvni]

@mahkoh When you refactor a struct, you are not adding knowledge to the source, so the number of keystrokes required to get it done should be as few as possible. I chose pub: because it requires 4 characters, it is recognizable from similar conventions in C++, and it does not add line noise that requires thinking to understand. If you can come up with something that beats the syntax within these constraints, then it would be great! As a reminder, this is only a proposal, so the final design can still change.

https://twitter.com/Jonathan_Blow/status/480579846708293632

So I am worried that my code is going to be cluttered by putting "pub" in front of every damn thing. But we'll see.

[mahkoh]

If you're trying to appeal to that guy then add a crate attribute that makes all fields public by default. He'll be all over that because that's how he wants to do it in his language IIRC.

[bvssvni]

@arthurprs This is meant to add a feature to Rust without putting complexity into the language. It addresses a concern around the practicality of modeling real world connections, which semantics must be compressed in a pragmatic way, because it is not well defined. Can you come up with something that makes it better, or if you can define your terms of how methods should look like?

[bvssvni]

@mahkoh :) I don't think we should go that far. However, if you have 10 members in a struct, you save 30 keystrokes. Try convince somebody in the design industry that 30 mouse clicks is OK to do something they do every day in Photoshop!

[mahkoh]

I counted and it's at most 11 keystrokes.

[bvssvni]

@mahkoh I did not know this existed. Perhaps this could be done simpler?

[mahkoh]

I'm not sure what you're referring to.

[bvssvni]

@mahkoh The #[field_visibility = "pub"] attribute. Do you think it could be possible to desugar pub: into this attribute? Is it possible to override it to make a member private?

Tested with priv to override, but it is only a reserved keyword. http://is.gd/wmjiIC

Could be something like this:

pub: struct Foo {
    priv x: uint, // Override
    y: uint,
}

[mahkoh]

I don't think such an attribute exists.

[bvssvni]

@mahkoh Oh, I thought you where referring to an attribute that existed. I like pub: better because it's shorter.

[ftxqxd]

The syntax in this proposal will be impossible to resolve unambiguously if type ascription is introduced (being able to specify the types of expressions like x: int). I personally would much rather have a way of being able to specify the types of arbitrary expressions than the ability to save a few characters in some circumstances and have a way of adding default and named parameters to functions that seems to me like a bit of a hack (creating a struct for the purposes of arguments alone seems strange).

[bvssvni]

@P1start Can you give some examples or links to type ascription? I've never considered this. It looks like the variable is at the left side of the colon? Are you sure this is ambiguous? I need to know this to see if this concern is solvable.

This is about refactoring existing code into reusable parts, where named syntax for calling functions is equivalent to the construction of structs. Structs are not created for arguments, but they are equivalent, to make refactoring easier. I'm sure you misunderstood that part.

How much is a "few" characters? 10%, 20%, 50%? I didn't bother type out the generated code for all the examples, because then the RFC would be too long. Have you read the alternatives? Do you think Rust should live without optional arguments? How should this be solved?

[reem]

It's worth considering that Rust currently has very little syntax sugar. Adopting a proposal like this one, which suggests adding a lot of non-trivial sugar seems like it would be a fairly large departure from the philosophy of the language until now. As a result I don't think this is a good fit.

Also, it should not take 30 keystrokes to add pub to the front of 10 lines, if you use a reasonable editor. In vim it's C-V(j*10)Ipub:

[bvssvni]

@reem The sugar is non-trivial because the problem is non-trivial. If you have a better suggestion for optional arguments, then it would be great!

Sorry, but I don't take arguments about editors seriously. Let's not go there.

[bvssvni]

The problem:

• Named syntax for calling functions
• Optional arguments
• Refactoring

Nobody so far has addressed anything related to it.

[reem]

Sorry, wasn't trying to start an editor war or anything like that. I was just suggesting that you usually don't have to type all the pubs manually.

I'm not certain I see why this is an enormous problem - many libraries have solved this using the "builder" pattern today.

As far as solving this problem, really the main things that are needed are:

• Named parameters
• Optional parameters

Those could be accomplished without any of the additional struct sugar by simply allowing light sugar around named arguments that desugars to either Optional::some(x) or Optional::none(). There is a lot of other sugar in here that I think could be cut from this proposal.

That said, I still don't know if this is really worth the extra complexity.

Rust is a relatively WYSIWYG language and I the additional struct sugar proposed in this RFC is not in that same vein IMO.

[blaenk]

I agree with @p1start, @SiegeLord, and @reem. I've heard this argument of creating a struct to wrap many parameters (and also to name them). It can make sense in certain situations, but in ad-hoc situations, when all of the arguments are not necessarily completely related, it seems too heavy and unwieldy to me. It doesn't feel right to have to shoe-horn the disparate arguments together into the same struct for what amounts to an ephemeral use case (e.g. a single function) in order to get named/optional arguments in a function call.

I would much prefer support for more general primitives, like actual named and default parameters similar to what was proposed in PR #257, paired with something similar to C++11's initializer lists (and maybe uniform initialization syntax). It seems to me like those two things---especially the initializer lists---would be more widely applicable (e.g. for building hashmaps, vectors, general structures, etc.).

Even without those things, I would prefer not to have this sugar because I feel that it runs pretty deep and has many specific use cases (i.e. too specialized) and corner cases.

[bvssvni]

@reem @blaenk Thanks. This is the kind of feedback I appreciate. I need some time to get into the issues you address.

[ftxqxd]

@bvssvni To take an example from the RFC:

send_form(first_name: "Mike", last_name: "Brown", age: 56);

This happens to be unambiguous, as string and integer literals are not valid types. However, it only takes a small refactoring to make it ambiguous:

let foo = "Mike";
let bar = "Brown";
let int = 56;
send_form(first_name: foo, last_name: bar, age: int);

The parser certainly can’t know that foo and bar are variables, not types, and in fact int could refer to either a variable or a type. So this could be parsed as either a function call with three parameters, or a function call with one parameter which is a struct.

Structs are not created for arguments, but they are equivalent, to make refactoring easier.

Ah, I didn’t notice that this RFC also adds named/optional parameters. I don’t really see how they are related to the struct sugar, though.

[bvssvni]

@reem The builder pattern is good for library design, but it is not refactor friendly. This is something you want to add in the later phases, but not during development when you want to test ideas. I'm not sure what you are thinking about keeping Optional but cut most of the sugar. Do you think of a macro or something? Keep in mind I wrote this to give a unified picture, it does not mean we should go all the way if this means too much complexity. The RFC might give the wrong impression because of the examples, but the ideas behind it are simple. I just had to make sure every corner case I could find was covered.

@blaenk This proposal does not put restrictions on how you want to call functions compared to the current syntax. For example, you can call a function with two structs by putting braces around the separate arguments. Optional arguments should be supported without using structs. You want more built-in features, which is opposite of @reem thinks, it seems. Do you have any thoughts considering reorder safety etc? This is something other people gave early feedback on, so it does not look like a single solution will satisfy everyone. Or, perhaps some of it is compatible?

Rust already lets you write constructors and builder patterns, so I don't think adding new kinds of constructors will boost productivity. What I am looking for is a way to work on the data representation without fighting with the compiler more than necessary. Perhaps solutions to some of these concerns:

• When moving a local declared struct to global scope as an intermediate step before fleshing out the API
• Describe intention of using optional arguments or not
• A sugar that keeps amount of typing low when refactoring
• Writing code that solves problems around data modelling, not intended to end up as a library

I have been working on libraries in Rust for a while and felt like I was putting the head in the sand, so I tested if Rust could be used the way typical to ad-hoc programming with lots of refactoring. There were moments I did not feel doing anything productive, just typing and typing to make the compiler happy. It is already hard to make the ideas working, and I am not interested in some "clever" design like generalized constructors, I just want the simple type of programming to be simpler.

One lesson I find important is that some immediate steps along the road would be good, and not think about the final design upfront. This is easily measured by counting the number of keystrokes and mouse clicks to get the code compiling. I believe it is possible to design a sugar around this, where you optimize for refactoring.

Actually I wish this sugar was there already. If I could turn it on, I would not have to think that much about how to organize data, but let the design come naturally from working with the code. I am considering learning to hack the compiler, so I could test it out for real in a branch. It might be a fun challenge, but I don't know how much work it will be.

[bvssvni]

@P1Start The function expects the type, which it uses to infer the type of the arguments. If the function takes multiple arguments, type inference works as usual, but with named syntax to reorder the parameters.

About the part with named syntax and optional arguments: Perhaps you could point out in the RFC where this is unclear?

[ftxqxd]

@bvssvni I’m not sure I entirely understand you. What I’m trying to say is that if Rust gained type ascription, which is quite simply permitting expressions of the form expr: ty, then there is a parser ambiguity in expressions (not declarations) like foo(bar: baz), because bar: baz in that context could be either giving a type, baz, to bar, or calling the function foo with a named argument, bar, with value baz.

About the part with named syntax and optional arguments: Perhaps you could point out in the RFC where this is unclear?

The RFC introduces named parameters like so:

When there is a single argument to a function, or, a single argument besides self in a method, the struct sugar looks like a named parameter syntax:

But the examples that follow don’t use any structs at all, so I don’t see how this relates to the struct sugar other than looking similar. Is this supposed to be implicitly creating an anonymous struct or something?

[kemurphy]

Massive -1 from me; while I agree the ergonomics need work, all of these solutions look really ugly and unintuitive to my eyes. The only thing I see here that makes of sense is having a local with a matching field name, but I'd imagine that would work more like Character { first_name: "Joe", last_name, .. }. For all fields of a struct to default to pub, I'd imagine something more like #[pub_fields] pub struct Foo { ... }. Structs expanding to named arguments is mixing two things that shouldn't be mixed, IMO. Overall I just don't like this. Feels very un-rustlike to me.

[bvssvni]

@P1Start Thanks! It will be hard to allow any kind of named syntax with type ascription, if it uses ':' to assign the argument. I hope this gets addressed. Is there an RFC for this?

There is no anonymous struct, the syntax is only designed to be similar to allow refactoring without breaking code. The benefit is to be able to switch between them, making structs a tool for refactoring code. For example, you can start out with a simple function and put in the arguments you need, then split it into structs, add optional arguments, and play around with it before you start thinking of methods, traits and API design. This is desireable where the data represents the idea you want to test, not the API. This often includes many members, many optional parameters, ad-hoc constructors etc. In the current syntax you have to type and design upfront to do this style of programming, which is verbose and makes it harder to navigate the code. I want a sugar which lets you make design around data, with unit tests for data, interactions based on data etc. The whole RFC is focusing on this kind of refactoring.

[bvssvni]

@kemurphy Thanks for the feedback! Curious:

Structs expanding to named arguments is mixing two things that shouldn't be mixed

Can you elaborate on what you mean by that?

[bvssvni]

@kemurphy From the Character { first_name: "Joe", last_name, .. } example, how do you think .. should work? Or, do you want to type out all the fields for every struct? I don't sure if you're against the RFC as a whole, or want to break it up, or whether you want to keep the current syntax.

You also want the ergonomics to be better, which this RFC addresses, including discussing alternatives, so which part did you find unintuitive? I tried to design something minimal that met the constraints from early feedback. There were several views, while seemingly intuitive, that simply did not work out. I think the problem here is non-trivial, but the solution in this RFC addresses all the concerns I am aware about, which gives us a place to start. Maybe we could discuss ways to get closer to a better solution? I am very interested in solving this problem, so if you have ideas of what I can do to improve the design, it would be great!

[bvssvni]

To summary the feedback so far:

• Everybody seem to agree that the ergonomics should be improved (+1)
• A way to make every member public should be possible, while the particular syntax is not agreed upon (+1?)
• There is a disagreement about whether optional parameters should be sugar or put into the language (pros/cons?)
• The non-trivial parts has been criticized to be unintuitive (conceptual model?)
• Incompatibility with type ascription, from @P1start (-1)

What is missing:

• Discussing the benefits of this sugar vs alternatives
• Additional corner cases which the RFC does not address

[bvssvni]

Someone on reddit suggested putting 'pub:' inside the struct on a new line like C++. Could this be ambiguous with declaration of a new member?

[oli-obk]

why not something like

pub struct Character<'a> {
   pub {
     first_name: &'a str,
     last_name: &'a str,
   }
}

refactoring requires an indent for all lines changing visibility and two lines require actual editing.
would also be a little more diff-friendly

[bvssvni]

@oli-obk That's nicer than my suggestion.

[thestinger]

I don't really understand the motivation for this added complexity. I don't see an improvement from indenting a block of public fields rather than prefixing them with pub. I think the current ergonomics are great, as the defaults are sane and the syntax for making data public is clean. I don't think types with a ridiculous number of fields should be encouraged. It's better to compose an API out of small pieces.

[bvssvni]

@thestinger Libraries with theoretical underpinnings usually have few fields and refactors well, while code that models informal relationships are much harder to model. The ergonomics of a programming language can easily be tested against such cases. I sat down for a couple days trying to write this kind of code, according to the practices recommended by professional game developers, and I still believe there is a pain point where refactoring plays a great role in productivity and you start noticing the difference. When you move a function or struct declared within another function to the module level, you have to put pub in front over every member. The time you loose when forgetting to do this is: compiling the project + reason about what the compiler says + amount of typing + getting distracted from what you were thinking on. It is not a matter of only the number of characters, it affects how you work, and this happens repeatedly. The reason I proposed pub: in front of the struct is because it is easy to type, and when it is easy to type it becomes easy to remember and might save you a round trip with the compiler. Another way improving the ability to refactor the code, is when you start out with a function and this evolves into a struct. Now you have to put the struct name everywhere where the function is called,. This costs you many round trips with the compiler. And then, you want to rename the structs to something else... Perhaps you are lucky and can do it with an IDE. Then language is designed to be used with an IDE and not designed for ergonomics from ground up. Then you have optional arguments which changes from being required to optional and the other way around... So this is the kind of thinking that motivates the RFC. It got nothing to do with API design.

[aturon]

@bvssvni I'm catching up on some RFCs that have fallen through the cracks; I'm sorry to have taken so long to get back to you on this one!

While I think there are some good ideas in this RFC, in general we have been steadfastly postponing this kind of feature. Shipping 1.0 requires keeping a focused scope, and this kind of feature would be a great addition in a later 1.X release.

I'm going to close this RFC for the time being, and link to it from the postponement issue, where the general design space in this area can be discussed. Let's plan to revisit these questions after 1.0.

Thanks for the RFC!

[bvssvni]

@aturon Thanks! I was writing this RFC with the idea of a backward compatible alternative, and the idea seems so unconventional that it deserved some initial discussion. That was my only goal and I don't expect it to end up in Rust in this form, but perhaps some of the ideas can user later.