Much ado about null

null has a controversial history. It's been called "the billion-dollar mistake" by its creator. Most languages implement null, but those few that do not (such as Rust) are generally lauded for their smart design by eliminating a class of errors entirely. Many developers (myself included) have argued that code that uses null or nullable parameters/returns is intrinsically a code smell to be avoided, while others (also myself included, naturally) have argued that is too draconian of a stance to take.

Anna Filina has a very good three-part series (properties, parameters, and returns) on how to avoid null in PHP generally. Alexis King has a related post (excuse the Haskell) on how to avoid needing edge cases like null in the first place.

However, I want to go a bit deeper and try to understand null from a different angle, and tease out the nuance of why one would really want to use it, and thus what we should be doing instead. To get there we'll have to go through a tiny little bit of type theory, but it will be quick and painless, I swear.

A language-specific problem

One of the difficulties of null is that most languages have a feature called null (or sometimes nil, just to be confusing), but that doesn't necessarily mean the same thing in all languages. That would be too easy. For instance, what Tony Hoare called his "billion-dollar mistake" was not null itself! It was null-references. Specifically, in the design of Algol 60 it was convenient to allow a pointer to point to address 0 to indicate "actually this pointer is invalid." That easy shortcut made implementation convenient, but also meant that every single time you wanted to use a pointer you had to first ensure it pointed at something other than address 0, or Weird And Dangerous Things(tm) could happen.

At the time, many developers favored "just let my code run" rather than "prevent me from shooting myself in the foot", so that idea was preferable. After a billion perforated feet language designers generally know that's a bad idea, but many developers, even seasoned developers, still would rather blow their own foot off so null pointer references and similar foot-guns still exist. Java, for instance, makes all objects nullable, which is effectively the same thing.

I am not here to defend null pointer references. They are already buried.

A wee bit of types

Instead, I want to talk about null in languages where nullability is optional and opt-in, such as PHP. And for that, we need to first talk about what it means for null to be a type, and what it means for anything to be a type.

To do that, we need to start with some terminology that will drive the rest of this discussion.

A Type is a set of possible values. It's a logical list. Those values could be finite or infinite. For instance, int is the type (set) of all integers, which is nominally infinite. bool is the type (set) of just the values true and false, which is finite.
A Product Type is a compound type that is composed of two or more other types, next to each other. For instance, "one int and one bool" is a product type. In most languages today this is represented by a class.
A Union Type is a type whose list of possible values is simply the combination of two other types. int|bool contains all possible values of int, plus all possible values of bool. However, there may not be any intrinsic tracking of which subset the value came from (whether it's an int or a bool).
A unit type is a finite type that has only one legal value. While technically one can have more than one unit type (e.g., multiple enums with only a single case), a unit type by definition carries no useful information so all unit types are logically equivalent.
A bottom type is a finite type that has zero legal values. It is, technically, a subtype of every other possible type (since all of its possible values are also valid values of all other types, on the technicality that it has no possible values).
A top type is a type that accepts literally anything. It is, technically, a super-type of every other possible type.

It may not be common to think of PHP in these terms, but PHP has every single one of these.

It has a list of predefined types, and user code can define additional product types called classes.
Although PHP cannot, yet, define a free-standing union type, we can define a union type for a property, parameter, or return value.
PHP supports enums, which are a way to define a custom type with a finite list of possible values.
never is the PHP bottom type. There is no value that is of type never, but you can use it as a return type to indicate that a function never has a return value.
The mixed type is PHP's top-type. Every value passes a type-check for mixed. Omitting a type in any location is equivalent (from a type theory perspective at least) to declaring it mixed.
And PHP has a unit type, called null. Its only legal value is... null. Yes, null refers to both the type and the value. It's only slightly confusing (and common).

PHP has no concept of a null pointer reference, so the "billion-dollar mistake" security hole that is null pointer references does not apply. Why, then, is null still problematic in PHP, especially when recent versions have given us such great new tools for working with null like ?? or ?->?

To answer that, we need to talk a little bit about functions. (For this discussion, "function" always includes "method"; everything we're going to say about one applies to the other.)

A function is an operation from an Input to an Output. Both the Input and the Output have some known Type. Even if you don't specify a type, PHP interprets that the same as if you'd specified mixed, the top type.
The fancy name for the input of a function is its domain. The fancy name for its output is range or codomain.
A total function is a function that is defined for every possible input. That is, any input you feed it that matches the specified input type has a corresponding output value. add(int, int): int is a total function over integers, because for any possible integers you feed into it there is an integer that it can return.
A partial function is a function that is not defined for every possible input. That is, there are values that pass the type check of the inputs for which no reasonable output is possible. divide(float, float): float is a partial function, because there are float values you can pass to it, specifically 0, for which there is no meaningful return value.

With all that laid out, here's the key observation that is going to drive the rest of our discussion:

Total functions require less error handling than partial functions

Total functions and errors

By definition, a total function has a meaningful return value for every possible input. That means there is, by definition, no possible input you can provide for which a meaningful return value isn't possible.

We don't need to check the return value of add(int, int): int to ensure that it's an int, or that there was such an integer, or that there was a database connection failure, or anything else along those lines. If the function truly is total, then it can only fail in one of two ways: Either the parser/compiler complains beforehand (because I tried to pass a string to it, which is invalid input), or I will get a runtime error (because the whole system is broken and therefore all bets are off). "If it compiles then it's right" is the goal, and while we can never truly achieve that, total functions get us a long way toward that goal.

A key value proposition for robust type systems in code is to make it possible to represent more operations as total functions. For example, if we had the ability to define a type for "a floating point number other than 0", then divide(float, nonZeroFloat): float would be a total function. We've now moved the need to error check that potential 0 value out of the code and into the definition of the problem itself. (That kind of highly complex type rule is known as "Dependent type theory" and is off-topic for today, so don't worry about it.)

Languages vary widely in how precise and nuanced you can get in your type definitions. PHP 8.1 is, I would argue, somewhat middle of the road overall, though one of the more advanced among interpreted languages.

Partial functions and errors

By definition, a partial function has values that are legal inputs, for which there is no meaningful output. That means if a legal but unmatched input is provided, the function needs to... do something else. That "something else" is error handling, and it takes many forms. I won't go into all the possible forms of error handling here (that's an article unto itself, at least), but generally speaking there's two popular approaches: Return a sentinel value or throw an exception.

A sentinel value is a value that is a legal output in itself, but the developer knows out-of-band has special meaning. For example, "returns 0 on error" from a function that returns integers is using 0 as a sentinel value, because it's a legal integer but has an extra special "guarded" meaning. (Hence the name "sentinel," as in a guardian.) The caller needs to always remember, however, that 0 has that special meaning. Or, perhaps, that 0 may have that special meaning. Depending on the function, 0 may also be a legal non-sentinel value. And that's where trouble happens.

One could argue that a partial function that returns a sentinel value is, strictly speaking, now a total function, since it has a return value. However, while the return value is within the type universe allowed, it is semantically still only partial because not every input has a semantically meaningful return, even if it has a compiler-safe return.

The other common option is to throw an exception. An exception is a typed value that may carry additional context information, but more importantly breaks the flow of control. An exception is a statement of "something is totally broken and I don't know WTF to do with it, halp!". It's appropriate when a partial function hits a case that the developer didn't anticipate or has no way of handling in a reasonable way, so gives up.

Exceptions are very expensive in some languages, including PHP. They're also destructive. You don't necessarily know that the state of the program is now reasonable. It may or may not be safe to continue doing anything. While you, the code writer, may know from context that a particular exception is non-world-destroying, there is no systematic way for the program to know that, the way it can know that a total function is always safe.

`null` as a sentinel

One way of avoiding the problem of sentinel values sharing the same type as real, meaningful return values is to expand the function's codomain (return types). For instance, if we could instead have a codomain that spanned multiple types, as a union, we could say that return values in one set are "real" return values and those in the other set are error return values.

A nullable type in PHP, like ?int, is in practice a shorthand for the union type int|null. That is, we're making a union of int and the unit type, which gives us as potentially legal values "all integers that exist... plus null". At least from a type theory perspective, that converts a partial function into a total function. divide(float, float): float|null is a total function, in that for any two input floats there is always a possible output that is either float or null.

That allows us to use null as a sentinel value, reserving all possible floating point values for actual, meaningful results.

Very old parts of the PHP standard library, which dated from before PHP had explicit types at all, often used false as a sentinel value. For example, strpos() technically has a return type of int|false, because its output range is any integer, or a boolean, specifically false. (The only reason PHP has false as a declarable type is to support these old and horrible functions. Do not use it yourself. Ever. If you do, you are wrong. There is no exception to that statement.) Of course, PHP being weakly typed 0 (which is an entirely reasonable return value from strpos()) and false are equivalent (==), leading to all sorts of stupid bugs.

This is why sentinel values that are also equal or equivalent to a meaningful output value are a very bad idea.

The problem with `null`

Using the unit type as a sentinel has its advantages, simplicity being key among them.

Except... a null may not be a valid input for the next function you want to use that value in. So this kind of return type widening technically doesn't render error handling unnecessary, it externalizes it from the function. Which... can still be a win, but not quite as big a win as we made it sound originally. Aw.

What we have, then, is null, via a union type, as the universal sentinel value. It's a built-in readily available type/value that indicates "there was no otherwise meaningful output for the type-safe input you provided." But... it doesn't tell you more than that.

The problem with null is that it's a universal sentinel value. In context, it could mean

your input was invalid in some way the type system cannot capture.
your input was invalid due to some external constraint (eg, not found in a database).
there is no value here that exists, and that's OK according to our business rules.
no action needs to be taken
this value hasn't been provided yet, but it will be later, and we've accounted for that (or possibly not)
various other meanings

Just null on its own doesn't differentiate between these possible meanings. Moreover, it could mean different things for the same value when passed or returned to two different functions. To divide(float, float): float|null it means "input invalid", but to setRate(float|null): void it means "use default". But is that really correct? Should invalid input silently translate to using a default value? Maybe, but probably not.

The root issue here is that null is insufficiently expressive of all the ways in which a function could be partial, so using it as a universal sentinel to convert a partial function to a total function is still a foot-gun.

Because it is the de facto universal sentinel, though, PHP (like many languages) has added several native language features to make working with it easier. Of note, ?? and ??= make providing a fallback value for null easier, while ?-> makes propagating a null easier.

However, the latter is extremely problematic precisely because that propagated null may change semantic meaning from one function to another.

Checking sentinels

Sentinel values, or error returns more generally, have an annoying problem that if you don't always check for them then they're useless or worse; but always checking for them, when they are rarely returned, is costly both for developer time and execution time. But you can guarantee the one time you forget to check it is the one time it's going to be critically necessary.

Consider the square-root function.

sqrt(int): float - This is a partial function, and if passed a negative value the program must die painfully (or ideally not even compile).
sqrt(int): float|null - This is a technically total function, but you must always check that the return value is not null, and a simple truthiness check won't work because null == 0 (in PHP; it does different weird things in other languages) and 0 is an entirely legal return value.
sqrt(int): float throws \InvalidArgumentException - This is a partial function, and you know that if the function returns then the return value is valid. However, if a negative number is passed the program enters a state of ¯\_(ツ)_/¯. Moreover, someone still needs to catch that exception somewhere or the whole program will die painfully.

Checking exceptions

Exceptions, as noted, are not an error condition. They are a failure condition. They indicate a case that the author could not foresee and thus could not handle, and thus nothing is trustworthy anymore.

At the risk of sounding judgemental, if you're writing a sqrt() function and didn't foresee someone passing in a negative value, the problem exists between the chair and keyboard. That is a very foreseeable circumstance, and as a responsible programmer you should write some error path for it of some kind. So no, exceptions are not appropriate here.

A concrete example

To use an example most of us will be familiar with, consider these two functions:

function getProduct(int $id): Product {}

function getProductsByColor(string $color): array {}

As written, it's fairly self-evident what they do in the happy path: The first returns a single Product object, the second returns a list of Product objects. But there are far more unhappy paths than happy paths (true of nearly any meaningful code), so let's consider those.

If getProduct() is called with a non-integer, that's a type error. Either the compiler or the runtime will fail out on us, which is correct because the input is not in the function's domain. All is well.
If getProductsByColor() is called with a non-string, that's a type error. Either the compiler or the runtime will fail out on us, which is correct because the input is not in the function's domain. All is well.
If getProduct() is called with an integer that does not map to an existing Product... we got nothing. This is bad.
If getProductsByColor() is called with a string that does not correspond to a known color... we got nothing. This is bad.
If getProductsByColor() is called with a string that is a valid color but corresponds to no valid products, what should happen?
If the database used to store product information catches fire during the function call... this is also bad, but there's decidedly little we can do about it.

Case 6 is out of our control, and not something we should be asked to anticipate. Throwing an exception in that case is entirely reasonable.

Cases 1 and 2 are already handled for us by the type system. They are "correct by construction," which is the fancy way of saying "if it compiles, it's right (at least as far as this particular problem is concerned)." We want this. The more we can make correct-by-construction, the better.

Case 5 has a self-evident answer. An empty list is a valid and semantically meaningful return from this function, and readily representable by the type system. In fact, it already is, so no further action is needed. (But always consider this case when making sure you have your error bases covered!)

Case 4 is not immediately obvious, but we can convert it to a correct-by-construction case by changing the parameter type from string to Color. Color is an object we define that encapsulates whatever our definition of "valid color" is for this problem space. It could be a validated RGB tuple, but in this case probably an Enum is probably more correct (as our products likely come in a fixed set of colors for which we have names). This is up to your domain design.

The interesting case is case 3. The ID matches whatever format we have for Product identifiers, but there simply is no such product in the system. Now what?

The traditional approach is to do one of two things:

Change the return type to ?Product (aka `Product|n