That said, the wisdom of the quip has been widely lost in many fields. In many fields data is "modeled" with huge regression models with dozens of parameters or even neural networks with billions of parameters.

> In 1953, Enrico Fermi criticized Dyson’s model by quoting Johnny von Neumann: “With four parameters I can fit an elephant, and with five I can make him wiggle his trunk.”[1]. This quote is intended to tell Dyson that while his model may appear complex and precise, merely increasing the number of parameters to fit the data does not necessarily imply that the model has real physical significance.

For those who are interested, you can watch Freeman Dyson recount this conversation in his own words in an interview: https://youtu.be/hV41QEKiMlM

What can't be fit by declaring the amount of dark matter that must be present fits the data? It's unfalsifiable, just because we haven't found it, doesn't mean it doesn't exist. Even worse than string/M-theory which at least has math.

It is actually a satisfying theory with regard to the Occam razor. We don't have to change our laws of physics to explain the abnormal rotations of galaxy, we just need "stuff" that we can't see and yet interact gravitationally. When we have stuff like neutrinos, it is not that far fetched. In fact, though unlikely given our current understanding of physics, dark matter could be neutrinos.

If, as it turn out, the invisible stuff we call dark matter doesn't follow the laws of physics as we know them, then the dark matter theory is falsified and we need a new one (or at least some tweaks). And it may actually be the case as a recent paper claims that gravitational lensing doesn't match the predictions of the dark matter theory.

The main competitor to dark matter is modified gravity, which calls for no new stuff, but changes the equations for gravity. For the Occam razor, adding some random term to an equation is not really better than adding some invisible but well characterized stuff, especially when we consider that the equation in question is extremely well tested. It is, of course, also falsifiable.

The problem right now is not that these theories are unfalsifiable, it is that they are already pretty much falsified in their current form (dark matter less than modified gravity), and some rework is needed.

There are many theories to explain dark matter observations. MOND is not a competitor with 'dark matter', because MOND is a theory and it tries to explain some aspects (spiral galaxy rotation) of what is observed as the dark matter problem, which consists of many more observations. There is no competition here. There are other theories to explain dark matter, like dark matter particle theories involving neutrinos or whatever, and these may be called competitors, but dark matter itself is not a theory, but a problem statement.

Whereas you can have many proposals for what dark matter is, provided it is capable of being almost entirely only gravitationally interacting, and there's enough of it.

MOND has had the problem that depending which MOND you're talking about, it still doesn't explain all the dark matter (so now you're pulling free parameters on top of free parameters).

This obviously can match almost anything and it has extremely low predictive power (many future observations may differ from predictions, which can be accounted by some dark matter whose distribution was previously unknown), so it is a much worse explanation than a modified theory of gravity that would have only a finite number of additional parameters.

The point is that even with current observational data there's no reasonable distribution of dark matter that correctly explains all evidence that we have.

Your intuition that "if I have an infinite number of degrees of freedom anything at all can be fit" is leading you astray here.

At minimum this is a ~200 billion parameter model, and more if you’re looking at smaller structures.

Even this is granting too much: "seeing it" and "seeing its effects" are the same thing. No one has ever "directly seen", in the sense that internet DM skepticism demands, anything other than a photon.

The problem with dark matter is that there does not exist any second relationship from which to verify its existence, like in the case of normal matter, which takes part in a variety of interactions that lead to measurable effects, which can be compared.

The amount and the location of dark matter is computed from the gravitational forces that explain the observed movements of the bodies, but there are no additional relationships with any other data, which could corroborate the computed distribution of dark matter. That is what some people mean by "seeing".

This is exactly it! Dark matter is strictly defined by its effects. The only 'theory' part is a belief that it's caused by yet to be found particle that's distributed to fit observations. Take all the gravitational anomalies that we can't explain with ordinary matter, then arbitrarily distribute an imaginary 'particle' that solves them: that's DM.

The problem is that the language used to talk about DM is wrong. It's not that DM doesn't interact with EM, or the presence of DM is causing the galaxies to rotate faster than by observed mass. These are all putting the cart before the horse. What we have is unexplained gravitational effects being attributed to a hypothetical particle. If we discovered a new unexplained gravitational property, we would merely add that to the list of DM's attributes rather than say "oh then it can't be DM".

Radical departure may well be needed, for other reasons too.

You're being too kind. It's worse. Especially when (in my understanding anyway) that added term doesn't even explain all the things dark matter does.

This is partially because there are two ways to detect dark-matter. The first is gravitational lensing. The second is the rotatinal speed of galaxies. There are some galaxies that need less Dark Matter to explain their rotational speed. We can then cross check whether those galaxies cause less gravitational lensing.

Besides that, the gravitational lensing of galaxies being stronger than the bright matter in the galaxies can justify is hard to explain without dark matter.

Not even anything that extreme. What's ruled out is interaction via electromagnetism (or if you want to get really nit-picky, electromagnetic interaction with a strength above some extremely low threshold).

Besides the idea 'not all mass can be seen optically' is not that surprising. The many theories on what that mass might be are all speculation, but they are treated as such.

Both of these are pretty much ruled out though: you can't plausibly add enough brown dwarfs, and if it's black holes then you should see more lensing events towards nearby stars given how many you'd need.

But they're both concrete predictions which are falsifiable (or boundable such that they can't be the dominant contributors).

Tons of things - just like there are tons of things that can't be fit by declaring the amount of electromagnetically-interacting matter that must be present fits the data.

You can fit anything you like by positing new and more complicated laws of physics, but that's not what's going on here. Dark matter is ordinary mass gravitating in an ordinary way: the observed gravitational lensing needs to match up with the rotation curves needs to match up with the velocity distributions of galaxies in clusters; you don't strictly need large scale homogeneity and isotropy but you really really want it, etc. Lambda-CDM doesn't handle everything perfectly (which in itself demonstrates that it's not mindless overfitting) but neither does anything else.

Which MOND does: it creates huge problems fitting into GR.

Whereas dark matter as just regular mass that interacts poorly by other means does not.

Nobody probably believes MOND as such is some fundamental theory, rather as a "theory" it's sort of a stepping stone. Also MOND is used often interchangeably (and confusingly) with modified gravity theories in general.

Hodgin and Huxley did ground-breaking work on squid's giant axon and modelled neural activity. They had multiple parameters extracted from 'curve fitting' of recorded potential and injected currents which were much later mapped to sodium channels. Similarly, another process to potassium channels.

I woudnt worry too much having multiple parameters -- even four when 3 can't just explain the model.

The physics equivalent is something like eternal inflation as an explanation for apparent fine-tuning - except that even if it's correct it's still absolutely nowhere near as complex or as contingent as biology.

For example, how many parameters does the Standard Model have? It's not clear what you count as a parameter. Do you count the group structure, the other mathematical structure that has been "fitted" through decades of comparisons with experiments?

I'm always making the joke (observation) that ML (AI) is just curve-fitting. Whether "just curve-fitting" is enough to produce something "intelligent" is, IMO, currently unanswered, largely due to differing viewpoints on the meaning of "intelligent".

In this case they're demonstrating some very clean, easy-to-understand curve-fitting, but it's really the same process -- come up with a target, optimize over a loss function, and hope that it generalizes, (this one, obviously, does not. But the elephant is cute.)

This raises the question Neumann was asking -- why have so many parameters? Ironically (or maybe just interestingly), we've done a lot with a ton of parameters recently, answering it with "well, with a lot of parameters you can do cool things".

Continual "curve fitting" to the real world can create intelligence. What is missing is not something inside the model. It's missing a mechanism to explore, search and expand its experience.

Our current crop of LLMs ride on human experience, they have not largely participated in creating their own experiences. That's why people call it imitation learning or parroting. But once models become more agentic they can start creating useful experiences on their own. AlphaZero did it.

1. The world around you 2. The experiences within your (really, the past view of the world around you) 3. Innateness of you (sure, this could be 2 but I think it's also something else) 4. The experience you find + the way you change yourself to impact (1), (2), and (3)

If you think of intelligence as all of these, then you're making the assumption that all that's required for (2), (3), and (4) is "agentic systems", which I think skips a few steps (as the author of an agent framework myself...). All this is to say that "what makes intelligence" is largely unsolved, and nobody really knows, because we actually don't understand this ourselves.

I'm going to need a citation on this bold claim. And by that I mean in the same vein as what Carl Sagan would say

  Extraordinary claims require extraordinary evidence

Can't we create an agent system which can search the internet and choose what data to train itself with?

Otherwise this can quickly devolve into the common useless semantic discussion.

By spelling out what brains are doing it becomes very obvious that it's all simply a sequence of chemical reactions - and yet here we are, having experiences. Software will never have a human experience - but neither will a chimp, or an octopus, or a Zeta-Reticulan.

Mammalian neurons are not the only possible substrate for intelligence; if they're the only possible substrate for consciousness, then the fact that we're conscious is an inexplicable miracle.

Consider the (algorithmic) mechanical process of screwing in a screw into a board. This screw has an "experience" and therefore intelligence. So... The screw is intelligent? Very low intelligence, but intelligent according to this definition.

But we have an even bigger problem. There's the metaset of experiences, that's the collection of several screws (or the screw, board, and screwdriver together). So we now have a meta intelligence! And we have several because there's the different operations on these sets to perform.

You might be okay with this or maybe you're saying it needs memory. If the later you hopefully quickly realize this means a classic computer is intelligent but due to the many ways information can be stored it does not solve our above conundrum.

So we must then come to the conclusion that all things AND any set of things have intelligence. Which kinda makes the whole discussion meaningless. Or, we must need a more refined definition of intelligence which more closely reflects what people actually are trying to convey when they use this word.

Neither, what I'm saying is that the observable correlates of experience are the observable correlates of intelligence - saying that "humans are X therefore humans are Y, software is X but software is not Y" is special pleading. The most defensible positions here are illusionism about consciousness altogether (humans aren't Y) or a sort of soft panpsychism (X really does imply Y). Personally I favor the latter. Some sort of threshold model where the lights turn on at a certain point seems pretty sketchy to me, but I guess isn't ruled out. But GP, as I understand them, is claiming that biology doesn't even supervene on physics, which is a wild claim.

> Or, we must need a more refined definition of intelligence which more closely reflects what people actually are trying to convey when they use this word.

Well that's the thing, I don't think people are trying to convey any particular thing. I think they're trying to find some line - any line - which allows them to write off non-animal complex systems as philsophically uninteresting. Same deal as people a hundred years ago trying to find a way to strictly separate humans from nonhuman animals.

And where you decide to assume that non-computable physics happens in the brain based on no evidence?

What a waste of time. You "addressed" it in a completely meaningless way.

Also LLMs just straight up do overfit, which makes them function as a database, but a really bad one. So while more parameters might just be better, that feels like a cop-out to the real problem. TBD what scaling issues we hit in the future.

In fact, that entire notion of early stopping is due to this. We use a validation set as a pseudo test set to inject information into our optimization products without leaking information from the test set (why you shouldn't choose parameters based on test results. That is spoilage. Doesn't matter if it's status quo, it's spoilage)

But we also need to consider that a lack of divergence between train/val does not mean there isn't overfittng. Divergence implies overfittng but the inverse statement is not true. I state this because it's both relevant here and an extremely common mistake.

I think you're being too optimistic, and I'm a pretty optimistic person. Maybe it is because I work in ML, but I've had to explain to a large number of people this concept. This doesn't matter if it is academia or industry. It is true for both management and coworkers. As far as I can tell, people seem very happy to operate under the assumption that benchmark results are strong indicators of real world performance __without__ the need to consider assumptions of your metrics or data. I've even proven this to a team at a trillion dollar company where I showed a model with lower test set performance had more than double the performance on actual customer data. Response was "cool, but we're training a much larger model on more data, so we're going to use that because it is a bit better than yours." My point was that the problem still exists in that bigger model with more data, but that increased params and data do a better job at hiding the underlying (and solvable!) issues.

In other words, in my experience people are happy to be Freeman Dyson in the conversation Calavar linked[0] and very upset to hear Fermi's critique: being able to fit data doesn't mean shit without either a clear model or a rigorous mathematical basis. Much of data science is happy to just curve fit. But why shouldn't they? You advance your career in the same way, by bureaucrats who understand the context of metrics even less.

I've just experienced too many people who cannot distinguish empirical results from causal models. And a lot of people who passionately insist there is no difference.

[0] https://news.ycombinator.com/item?id=40964328

This was a lovely passage from Dyson’s Web of Stories interview, and it struck a chord with me, like it clearly did with the authors too.

It happened when Dyson took the preliminary results of his work on the Pseudoscalar theory of Pions to Fermi and Fermi very quickly dismissed the whole thing. It was a shock to Dyson but freed him from wasting more time on it.

Fermi: When one does a theoretical calculation, either you have a clear physical module in mind or a rigorous mathematical basis. You have neither. How many free parameters did you use for your fitting?

Dyson: 4

Fermi: You know, Johnny Von Neumann always used to say ‘with four parameters I can fit an elephant; and with five I can make him wiggle his trunk’.

Maybe this sort of thing would be a really good tradition. Everyone must write a very silly article with some mathematical arguments in it. Then, we can all go forward with the comfort of knowing that we aren’t really at risk of breaking new grounds in appearing unserious.

It is well written and very understandable!

Why would that be a harder problem? In the case that you get a zero parameter, you could inflate it by some epsilon and the solution would basically be the same.

Not everything is continuous. Add an epsilon worth of torsion to GR and you don't get almost-GR, you get a qualitatively different theory in which potentially arbitrarily large violations of the equivalence principle are possible.

In a sense, they have done their fitting using nine parameters, of which five are zero.

If I could make a discovery in my own time without using company resources I would absolutely publish it in the most humorous way possible.

  letmeB this (F you) | you == me = B this
                      | otherwise = F you
  letmeB this (B that)            = B that
  letmeB this (App fun arg)       = letmeB this fun `App` letmeB this arg

[1] https://arxiv.org/abs/hep-ph/9306225

[2] https://en.wikipedia.org/wiki/Alpher%E2%80%93Bethe%E2%80%93G...

[3] http://strictlypositive.org/

(Seriously, though, this was a lot of fun!)

Though in that case, I would have liked for them to make it explicit. Maybe normalize it to "1", and scale the other parameters appropriately. (Because as it stands, I don't think you can reproduce their figure from their paper.)

A real-parameter (r(theta) = sum(r_k cos(k theta))) Fourier series can only draw a "wiggly circle" figure with one point on each radial ray from the origin.

A compex parameter (z(theta) = sum(e^(z_ theta))) can draw more squiggly figures (epicycles) -- the pen can backtrack as the drawing arm rotates, as each parameter can move a point somewhere on a small circle around the point computed from the previous parameter (and recursively).

Obligatory 3B1B https://m.youtube.com/watch?v=r6sGWTCMz2k

Since a complex parameter is 2 real parameters, we should compare the best 4-cosine curve to the best 2-complex-exponential curve.