type-safety with wing

[revitalbarletz]

@OlafConijn
Hi all, when I was reading up on wing-lang a couple of days ago, it got me thinking about type-safety….
I rely heavily on type safety for both static analysis of my program as well as the convenience of being able to use auto-complete. I program primarily in TypeScript and must say I am deeply impressed with TypeScripts type system. Static analysis as described at the end of this section is peanuts.
If i imagine a cloud-native programming language I could imagine it look like the following:

bring cloud;

interface Dog {
 name: str;
 age: num;
}

let bucket = new cloud.Bucket<Dog | Cat>();

bucket.on_file_added(inflight (key: str, object: Dog | Cat): void => {
 //object is of type Dog | Cat
 if (!object is Dog) {
  //object must be Cat
  console.log("meow");
  return;
 }
 //object must be Dog
 ... do a dog thing

now... i believe the pre-flight code to be assembled differently from the in-flight code. I did notice the compiler for the in-flight code to output javascript code (makes sense), but what I wondered is the following…..

If, instead of assembling in-flights to javascript, the compiler would emit an typescript AST... this might allow you to use the typescript type checker against the AST (including all of its features)
It might also allow re-use of other tools like the VS.language service for typescript or just for rapid prototyping of language features. The AST can (obviously) be used to also emit the javascript code that needs to run in the actual function.

I have not had a lot of change to read about Wing and also am not familiar with the vision you have for the language. Maybe becoming a front-end compiler to typescripts (front-end compiler) is not what you guys are aiming at.
Still thought it was an idea worth surfacing

@lsegal
I think this thread closely matches my takeaway that Wing should much more closely mimic TypeScript's syntax and in should in general just be a preprocessor layer on top of (or worst case a fork of) TypeScript's grammar/compiler so that there does not need to be a reinvention of type system, semantics, optimizations, etc. It would mean spending less time spent having to fix basic things like empty arrays not compiling and more time on the important stuff.

@OlafConijn
that sounds about right!
i don't think it means you'll have to do much concessions on syntax (as long as you keep your own parser). having worked with the typescript compiler before i might be able to come up with a small example of how this works (if/when time permits). would be happy to share

[OlafConijn]

hi!

I wrote a small proof of concept program that shows how the Typescript type-checker can be used to analyze an AST.
The intent is to show how (parts) of the Typescript compiler can be re-used while using a different programming language (as long as the programming language is able to generate a typescript AST).

I haven't found a well-defined extension point to register a Parser (which would have been nice), but with a bit of poking into the typescript library at runtime, I got it working.

source, here: https://github.com/OlafConijn/fake-parser-poc

[OlafConijn]

I took some time to take the proof of concept a little further and committed the result in a new repository.
This new repository uses the winglang parser and translates the winglang ast to a typescript ast. e.g. variable_assignment_statement

The test files (there is one for variable assignments and another where I got started with control flow) take wing code and use the typescript AST to convert this to typescript, javascript and print types of variables.

so far I ran into a number of things that I am not entirely sure about.
edit: I figured these things out! still getting to know how the parser works 😅

[pcholakov]

This line caught my eye:

let bucket = new cloud.Bucket<Dog | Cat>();

Interesting idea to make the bucket itself typed! How do you think about checking these types? For example, if this bucket maps to an AWS S3 bucket, a different program could use a bare S3 client to store an object in it in a way that bypasses checks. Or an older/newer version of the same program could insert an instance of Fox which our program doesn't understand.

This has (potentially) deeper runtime implications than compile-time type checking. I say "potentially" because you could always do the TypeScript/JavaScript thing, and pretend the problem doesn't exist once it leaves the compiler. But it feels to me like that's an interesting opportunity to define such semantics for the "inflight" phase.

[Chriscbr]

We are definitely considering features like this for Wing (parameterized resources) though it may be on our longer term roadmap. You're right, that doing something like this would probably require developing stable ABI for serializing any kinds of types onto the cloud. For example, the same logic applies to

let fn = new cloud.Function<str, num>(/* inflight code */);

Supposing this means the function accepts a string and returns a number, it would be handy for these conventions to be encoded in a simple format (like JSON encoded in UTF8) so that any resources that are created in Wing can "play well" with other libraries and microservices etc. In the shorter term I think we can start by letting users create HTTP endpoints and generate OpenAPI specifications based on them etc. (a cloud.Api resource in other words)

[lsegal]

For example, if this bucket maps to an AWS S3 bucket, a different program could use a bare S3 client to store an object in it in a way that bypasses checks. Or an older/newer version of the same program could insert an instance of Fox which our program doesn't understand.

It is pretty common for typed systems to define best-effort declarations at external boundaries. You see this often with database mappers (which might map a users table to a User type, but I can of course mutate the database directly or even migrate the schema to a point where the type has the wrong declarations) but the best possible example here is simply found in JSON. Go does this by defining (typed) objects to deserialize JSON strings into, but the strings themselves cannot be guaranteed to follow the serialization format; there are many valid situations that could cause the payload to begin returning types differently than described by the program (bad data was injected, the schema has changed, etc).

This is the classic type checking problem with remote resources; it's not sufficient to have compile time safety when dealing with external objects, and you should always expect to be handling runtime errors in these cases. Do you have any specific examples of other languages dealing with this case in a novel way?

[OlafConijn]

+1 to what @lsegal wrote.

would be really usefull though if, when deploying, you get feedback about the schemachanges and whether this breaks compatibility for prodocurs or subscribers.

[yoav-steinberg]

We're playing around with the idea of stateful vs stateless cloud resources. For example a cloud.Queue in many cases can be sateless. So having something like a cloud.Queue<UserUpdate | UserPurge | NewUser>() will introduce runtime safety without the worries of a cloud.Bucket<User>(). I think just that is a lot.
And I agree with @OlafConijn that some mechanism of notifying you before deployment of a schema break for all stateful resources would also be very useful.