Photo by ciboulette from Pexels

Flow’s Improved Handling of Generic Types

tl;dr: Flow has improved its handling of generic types by banning unsafe behaviors previously allowed and clarifying error messages.

We’ve improved Flow’s ability to detect errors in generic function and class definitions, and also introduced a new restriction on how generic types can be used. This post describes the new restriction and how to upgrade.

The escaped-generics restriction

In Flow 0.136, we introduced a new check that detects, and raises an error, when Flow infers that an unannotated variable or parameter has a type that includes a generic, but is not inside the scope where that generic is defined. We call this detecting when a generic “escapes” from it’s scope, and it can happen in situations like this [example]:

let external_var = 42;
function f<T>(x: T, should_escape: bool): T | number {
  if (should_escape) {
    external_var = x;
  }
  return external_var;
}

Since it’s unannotated, Flow will infer the type of external_var, and find it to be T | number, but this isn’t really a type that makes sense for it to have: T doesn’t exist in the scope in which it’s defined! We call this T escaping its scope. Previously this was allowed by Flow, but it had the potential of producing confusing error messages, it wasn’t clear what it means for a variable to have a generic type when the generic is not in scope, and it blocked ongoing work on improving Flow’s generic system as a whole. Other languages likewise object to generic escapes: for example, TypeScript warns that external_var’s type cannot be determined and uses any instead.

The new error we’ve introduced only applies when a generic escapes into an unannotated variable (or property or function return), because if an annotation exists, Flow won’t need to infer anything, and we already ban out-of-scope generics from being explicitly annotated.

The easiest way to fix escaped-generics errors is to simply provide a type annotation, and the error message used by the escape check can usually point to exactly where the annotation is needed. In some cases, as with this example, there may not be a reasonable annotation to use, which likely indicates a more fundamental problem in the program.

this types are also generics

It’s important to note that everything described here also applies to the type this when used inside a class definition. Flow models the this-type as a generic whose upper bound is its enclosing class, and for good reason: a generic can be thought of as a range of types from empty up to its upper bound (which is mixed if not otherwise specified), and since classes can be extended, the this type likewise represents a range of types that include all possible subclasses of the enclosing class.

Detailed example

Previously, Flow would allow escaping to occur without errors, but when the escaped generic is used (as in return external_var above), it would often produce a confusing error:

6:   return external_var;
            ^ Cannot return `external_var` because `T` [1] is 
              incompatible with `T` [2]. [incompatible-return]
References:
2: function f<T>(x: T, should_escape: bool): T | number {
                    ^ [1]
2: function f<T>(x: T, should_escape: bool): T | number {
                                             ^ [2]

This is confusing, because how can T be incompatible with itself? On the other hand, Flow needs to produce some error here, because the escaped generic could otherwise be used to break type soundness without the programmer realizing it [example]:

...
f<string>("hello world", true); 
// now external_var is set to "hello world"
var not_a_boolean: boolean | number = f<boolean>(false, false); 
// we now have a string masquerading as a boolean | number!

Here, because T escapes its scope and then re-enters it, we’re able to return a value that is actually at runtime a string into a position that expects it to be a boolean—or at least we would be able to if Flow didn’t report an error telling us that something about f was wrong.

The change we’ve made for v0.136 is that now, any generic escape is treated as an error, so in addition to the “T is incompatible with T” error, Flow also produces a more useful error at the site where the generic actually escapes:

4:     external_var = x;
                      ^ Cannot assign `x` to `external_var` 
                        because type variable `T` [1] cannot 
                        escape from the scope in which it was 
                        defined [2] (try adding a type 
                        annotation to external_var` [3]). 
                        [escaped-generic]
References:
2: function f<T>(x: T, should_escape: bool): T | number {
                    ^ [1]
2: function f<T>(x: T, should_escape: bool): T | number {
              ^ [2]
1: let external_var = 42;
       ^ [3]

We made this change because escaped generic types rarely make any sense — even in cases where they don’t actually cause unsoundness or produce errors, they always have the potential of creating confusing error messages. In addition, we’re making general improvements to Flow’s type-checking of generics, which will remove altogether the confusing error messages above, but which require this additional check to ensure soundness in cases like the above example.

How to upgrade by automatically adding annotations

To make it easier to fix the new escaped-generic errors, in Flow 0.137 we’ve added a new recipe to Flow’s codemod command, which adds annotations to variables that contain escaped generics. This follows the same approach as adding annotations for Types First but should be a smaller change for most codebases.

To add annotations to fix escaped-generics, you can run the following command:

flow codemod annotate-escaped-generics --write /path/to/folder

This command update the project at /path/to/folder in-place. This will likely fix many escaped-generics errors, but it does require manual follow up: there may still be some annotations that need to be added manually, in some cases the new annotations will themselves cause errors, and some programs (the program shown above, for example) may need more significant changes because no annotation would allow the program to typecheck as written.

See the documentation for the annotate-exports codemod for more guidance on using the codemod command.