Think in Math, Write in Code (2019)

I think you misunderstand this approach.

The point of writing the tests is to think about the desired behaviour of the system/module you are implementing, before your mind gets lost in all the complexities which necessarily happens during the implementation.

When you write code, and hit a wall, it’s super easy to get hyper-focused on solving that one problem, and while doing so: lose the big picture.

Writing tests first can be a way to avoid this, by thinking of the tests as a specification you think you should adhere to later, without having to worry about how you get there.

For some problems, this works really well. For others, it might not. Just don’t dismiss the idea completely :)

Not that Implement/Test can't work. As frustrating as it is, "just do something" works far better than many alternatives. In particular, with enough places doing it, somebody may succeed.

There are literally industry publications full of these.

(But I think it does apply more generally. We refer to it as Computer Science, it is often a branch of Mathematics both historically and today with some Universities still considering it a part of their Math department. Some of the industry's biggest role models/luminaries often considered themselves mathematicians first or second, such as Turing, Church, Dijkstra, Knuth, and more.)

My takes are:

1) There are a lot of IT workers in the world, and they're all online natives. So naturally they will discuss ideas, problems, etc. online. It is simply a huge community, compared to other professions.

2) Programming specifically is for many both a hobby and a profession. So being passionate about it compels many people to discuss it, just like others will do about their own hobbies.

3) Software is a very fast-moving area, and very diverse, so you will get many different takes on the same problems.

4) Posting is cheap. The second you've learned about something, like static vs dynamic typing, you can voice your opinion. And the opinions can range from beginners to CS experts, both with legit takes on the topic.

5) It is incredibly easy to reach out to other developers, with the various platforms and aggregators. In some fields it is almost impossible to connect / interact with other professionals in your field, unless you can get past the gatekeepers.

And the list goes on.

The Internet is absolutely full of this. This is purely your own bias here, for any of the trades you mentioned try looking. You will find Videos, podcast and blogs within minutes.

People love talking about their work, no matter their trade. They love giving their opinions.

[0] https://www.google.com/search?q=blaking+magazine [1] https://www.google.com/search?q=mechanics+magazines [2] https://dentistry.co.uk/dentistry-magazine-january-2023-digi...

The top-down (mathematical) approach can also fail, in cases where there's not an existing math solution, or when a perfectly spherical cow isn't an adequate representation of reality. See Minix vs Linux, or OSI vs TCP/IP.

Just get a proof of the open problem no matter how sketchy. Then iterate and refine.

But people love to reinvent the wheel without caring about abstractions, resulting in languages like Python being the defacto standard for machine learning

https://www.youtube.com/watch?v=ltLUadnCyi0

Personally, I find a mix of all three approaches (programming, pen and paper, and "pure" mathematical structural thought) to be best.

That said, I mostly agree with you, and I thought I'd share an anecdote where a math result came from a premature implementation.

I was working on maximizing the minimum value of a set of functions f_i that depend on variables X. I.e., solve max_X min_i f_i(X).

The f_i were each cubic, so F(X) = min_i f_i(X) was piecewise cubic. X was dimension 3xN, N arbitrarily large. This is intractable to solve as, F being non-smooth (derivatives are discontinuous), you can't well throw it at Newton's method or a gradient descent. Non-differentiable optimization was out of the question due to cost.

To solve this, I'd implemented an optimizer that moved one variable at a time x, such that F(x) was now a 1d piecewise cubic function that I could globally maximize with analytical methods.

This was a simple algorithm where I intersected graphs of the f_i to figure out where they're minimal, then maximize the whole thing analytically section by section.

In debugging this, something jumped out: coefficients corresponding to second and third derivative were always zero. What the hell was wrong with my implementation?? Did I compute the coefficients wrong?

After a lot of head scratching and code back and forth, I went back to the scratchpad, looked at these functions more closely, and realized they're cubic of all variables, but linear of any given variable. This should have been obvious, as it was a determinant of a matrix whose columns or rows depended linearly on the variables. Noticing this would have been 1st year math curriculum.

This changed things radically as I could now recast my maxmin problem as a Linear Program, which has very efficient numerical solvers (e.g. Dantzig's simplex algorithm). These give you the global optimum to machine precision, and are very fast on small problems. As a bonus, I could actually move three variables at once --- not just one ---, as those were separate rows of the matrix. Or I could even move N at once, as those were separate columns. This could beat all the differentiable optimization based approaches that people had been doing on all counts (quality of the extrema and speed), using regularizations of F.

The end result is what I'd consider one of the few things not busy work in my PhD thesis, an actual novel result that brings something useful to the table. To say this has been adopted at all is a different matter, but I'm satisfied with my result which, in the end, is mathematical in nature. It still baffles me that no-one had stumbled on this simple property despite the compute cycles wasted on solving this problem, which coincidentally is often stated as one of the main reasons the overarching field is still not as popular as it could be.

From this episode, I deduced two things. Firstly, the right a priori mathematical insight can save a lot of time in designing misfit algorithms, and then implementing and debugging them. I don't recall exactly, but this took me about two months or so, as I tried different approaches. Secondly, the right mathematical insight can be easy to miss. I had been blinded by the fact no-one had solved this problem before, so I assumed it must have had a hard solution. Something as trivial as this was not even imaginable to me.

Now I try to be a little more careful and not jump into code right away when meeting a novel problem, and at least consider if there isn't a way it can be recast to a simpler problem. Recasting things to simpler or known problems is basically the essence of mathematics, isn't it?

  Mathematics is not the study of numbers, but the relationships between them
  - Henry Poincaré
 

Elegance is not first. First is rough. Solving by math sounds much like what you describe. I find my structures, put them together, and find the interactions. Elegance comes after cleaning things up. It's towards the end of the process, not the beginning. We don't divine math just as you don't divine code. I'm just not sure how you get elegance from the get go.

So I find it weird that you criticize a math first approach because your description of a math approach doesn't feel all that accurate to me.

Edit: I do also want to mention that there's a correspondence between math and code. They aren't completely isomorphic because math can do a lot more and can be much more arbitrarily constructed, but the correspondence is key to understanding how these techniques are not so different.

Can you give an example of how you "linguistically" approach a problem?

I mean, even in math, description of the problems are written in natural language, but they have to be precise.

It is that Mathematics is far more general and uses a myriad of notations developed over hundreds of years and adapted to various sub-fields/domains/models as necessary. This makes it far more flexible and powerful than any programming language. That is why Multi-Paradigm languages became a thing i.e. there is a need for programming languages to provide a larger set of computation models which can then be exploited by the programmer to map his domain models (mathematical or not).

For example; why do many(most?) programmers have difficulty in transcribing algorithms given in pseudocode to their favourite language? Simply because they have not understood the algorithm at the fundamental mathematical level but have only picked up the patterns through which it is expressed in their language. Note that this is the default way our brain works and how we manage real-world complexity without really understanding everything (satirically phrased as "monkey see, monkey do"). But we can use mathematical methods and reasoning to minimize going off the rails because it forces us to make explicit our assumptions, definitions and proofs which is at the heart of problem-solving. So we use all the mathematical tools we have at hand to structure and solve a problem and only later map it to our programming language. But note that as we gain more experience this mapping becomes intuitive and we can directly think and express it in our favourite programming language.

See also my comment here - https://news.ycombinator.com/item?id=45934301

Some References:

Notation as a tool of thought by Kenneth Iverson - https://dl.acm.org/doi/10.1145/358896.358899

Predicate Logic as Programming Language by Robert Kowalski - https://www.researchgate.net/publication/221330242_Predicate...

  > Math notations, conventions and concepts usually operate under the principles of minimum description lenght.

If we look at older code we'll actually see a similar thing. People are limited in character lengths so the exact same thing happened. That's why there's still conventions like 80 char text width, though now those things serve as a style guide rather than a hard rule. It also helps that we can auto complete variables.

Your criticism is misplaced.

I think what helps is to take the time to sit down and practice going back and forth. Remember, math and code are interchangeable. All the computer can do is math. Take some code and translate it to math, take some math and translate it to code. There's easy things to see like how variables are variables, but do you always see what the loops represent? Sums and products are easy, but there's also permutations and sometimes they're there due to lack of an operator. Like how loops can be matrix multiplication, dot products, or even integration.

I highly suggest working with a language like C or Fortran to begin with and code that's more obviously math. But then move into things that aren't so obvious. Databases are a great example. When you get somewhat comfortable try code that isn't obviously math.

The reason I wouldn't suggest a language like Python is because it abstracts too much. While it's my primary language now it's harder to make that translation because you have to understand what's happening underneath or be working with a diffident mathematical system and in my experience not many engineers (or many outside math majors) are familiar with abstract algebra and beyond so these formulations are more challenging at first.

For motivation, the benefits are that you can switch modes for when a problem is easier to solve in a different context. It happens much more than you'd think. So you end up speaking like Spanglish, or some other mixture of languages. I also find it beneficial that I can formulate ideas when out and about without a computer to type code. I also find that my code can often be cleaner and more flexible as it's clearer to me what I'm doing. So it helps a lot with debugging too

Side note: with computers don't forget about statistics and things like Monte Carlo integration. We have GPUs these days and that massive parallelism can often make slower algorithms faster :). When looking at lots of computational code it's not written for the modern massively parallel environment we have today. Just some food for thought. You might find some fun solutions but also be careful of rabbit holes lol