You have a Common Lisp program running, multiple threads listening for network connections, others processing data in the background. You were really careful making sure that there are no data access conflicts. Concurrency, check. You use Emacs and SLIME to connect to your server, inspect its state. Interactivity, check. You realize a global hash table contains an incorrect value; no problem, just evaluate a simple REMHASH expression. Mutability, check. Unfortunately another thread happens to read the hash table while the form is being evaluated. Game over.

You want concurrency, loosely defined as “having multiple execution threads making progress in the same unit of time”, because it allows for faster programs, processing more data faster. But now your programs are much more complex because you need to protect data accesses to avoid deadlocks, starvation, memory corruption and other unpleasant consequences.

You want interactivity, because it is so convenient to be able to inspect and update the state of your program at any time. But now you can access any data whenever you want, data that may be read or written by concurrent threads.

You want mutability, because it is practical.

But you cannot have it all.

Dropping interactivity

The most popular choice is to drop interactivity: since there is no one to randomly access runtime data, there are no concurrency issues. C, Rust, Go, the list of languages that go this way is long.

You are efficient and safe, but you lose the ability to work with your systems during execution. Frustrating during development, or to debug and fix complex systems where restarting is not always an option.

Dropping concurrency

You can still have multiple threads of course, but nothing in your program can be read or written at the same time. Python and Ruby are good examples of languages that drop concurrency by using a GIL (Global Interpreter Lock).

You can execute multiple concurrent threads, but any access to Python or Ruby data are protected by a global lock. You can run your program in a Python shell or with Pry in Ruby, reading and writing data at will in a safe way.

Unfortunately it is slow. Most concurrent programs only require synchronization for a tiny subset of all data accesses; tiny on purpose since synchronization is slow. Locking all accesses has a very real cost that must be paid.

Dropping mutability

What if you could not really change data? Instead of reading the value of a variable, you could have the runtime systematically (and safely) copy the value and return it to you. Threads would communicate by, again, sending copies of values. Of course you cannot go and directly modify any value anywhere; but still the message-based approach means you can still safely update state at any time by sending messages to threads.

Erlang made this choice, and on paper it solves a lot of problems. You can run threads (“processes” in Erlang) that operate concurrently safely since they communicate with messages containing copied values instead of accessing shared memory. And you can interact with the system at will.

But again, there is no free lunch: copying data is slow. Very slow. Of course you can optimize data representations, avoid copying binary blobs (they are ref-counted in Erlang) because it would be untenable. But it will still be horribly slow.

You cannot have it all

As usual choosing a language is choosing a set of trade-offs. Most performance-sensitive programs drop interactivity so that they can limit shared data accesses as much as possible. Developers wanting the convenience of interactivity have to tolerate the extra cost of a global lock, or of an immutable data model.

Common Lisp developers will of course brag about how they can have it all, but they will always have to keep in mind that anything they do in their REPL can potentially break their program.