PurelyFunctional.tv Newsletter 419: Why not write your own language?
Issue 419 - March 22, 2021 · Archives · Subscribe
Follow-up 🙃
Clojure's buzz
In last week's issue, I asked how we could drive Clojure's growth. I got some really great responses to my question last week. Mostly, they were explaining how lots of characteristics trump language quality. While Clojure is a great language, it's lacking a lot of these characteristics, like familiarity, marketing, etc.
Unfortunately, I asked the wrong question. What I meant to ask was more about buzz. I do care about growing Clojure, but I confused it with participating in the pop culture conversation. Forget growth! I just want something exciting to say about Clojure so people don't think it's dead!
As for growth, I really think this talk by Gabriel Gonzalez explains a proven growth strategy really well. You should watch this talk. The trouble is choosing something Clojure can be best-in-class at.
Design Tip 💡
Why not write your own language?
I was watching AlphaGo - The Movie over the weekend. It documents the emotions and suspense around the game where Deepmind's AlphaGo beat Lee Sedol (best Go player in the world) 4 to 1. Let me tell you, in 2008, when I got my MS in AI, nobody cared about AI (no money in it), especially neural nets. It's a vindication for the entire field.
I started looking into the technology used at Deepmind. I was not surprised, but very disappointed, to learn that they use a mix of prevalent languages. There's a lot of Python, a lot of Lua, and C++ for low-level math. A company with the resources of Alphabet behind it could have built its own language. It's certainly easier than solving Go.
Now, I can't fault them too much: they won! They built the best Go AI in the world. But I think there's a huge missed opportunity to gain several orders of magnitude of productivity by developing a custom language just for their problem domain.
First of all, a better language will let you program using the right abstractions. I really doubt C++ is the right level of abstraction. The C++ build toolchain alone could scare Godzilla back into the Pacific. Some folks at Google already recognized that and built Go(lang).
Second, control over the compiler gives you the ability to bake in domain-level optimizations. You can actually generate faster code since you understand the problem you're solving.
There are many side benefits. There are lots of precedents.
The field of AI is littered with experimental languages, especially variants of Lisp. It's a tradition to write a new Lisp for a new machine to get started with programming.
The Microsoft Excel team wrote their own C compiler to control the entire development stack, among other benefits. See this story by Joel Spolsky who worked on Excel for a time. And Joel's company, FogBugz, had its own language.
The Crash Bandicoot used its own compiler (yes, a Lisp) to get more out of the Sony Playstation hardware. Check out this excellent documentary version of the story.
There's more, but I'll leave that there.
The other side of the argument is pretty straightforward: Isn't it a lot of work to develop a language? Isn't that outside of their core competency?
Is it a lot of work to develop a language? Yes. But the benefits pay off in orders of magnitude of productivity. I hope we realize that any savings you make in getting started quickly with existing languages would be dwarfed in a few years by the gain in productivity. A multi-year project like AlphaGo, backed by practically infinite funding, can afford its own compiler. Do you really still want to be writing for loops 10 years into your company?
Isn't language outside of their core competency? Shouldn't they focus on the AI algorithms? Yes, I suppose it is. But it's definitely something they can afford. Anyone who can figure out how to wri te fast, complicated code in C++ can write a compiler. There's a lot of overlapping skills. Plus, it's something you can hire for. Finally, they're making their own hardware! You can't tell me they know how to design hardware but not software.
AlphaGo is probably doing it because they want the code to be accessible to more people. People outside the company know C++ and Python. They understand the toolchains. They can get a lot of open-source users. But maybe they don't understand the power of language. And that is disappointing.
Podcast episode🎙
I've been thinking a lot about software design and modeling. This week on the podcast, I talk about the wrong turn software design has taken. Check it out: Has software design taken a wrong turn?
Book update ✍️
Grokking Simplicity is working its way through production at Manning. I'm still waiting for the print-ready proofs!
Grokking Simplicity is available for pre-order on Amazon. One day I will ask you for a favor: a review on Amazon.
However, you can buy Grokking Simplicity on Manning's site today and use the coupon code TSSIMPLICITY for 50% off. You can buy it now and it will be shipped to you when it's printed. Meanwhile, you can read the PDF version today!
Thanks to those of you who have already purchased 😘
Quarantine update 😷
I know a lot of people are going through tougher times than I am. If you, for any reason, can't afford my courses, and you think the courses will help you, please hit reply and I will set you up. It's a small gesture I can make, but it might help.
I don't want to shame you or anybody that we should be using this time to work on our skills. The number one priority is your health and safety. I know I haven't been able to work very much, let alone learn some new skill. But if learning Clojure is important to you, and you can't afford it, just hit reply and I'll set you up. Keeping busy can keep us sane.
Stay healthy. Wash your hands. Wear a mask. Take care of loved ones.
Clojure Challenge 🤔
Last issue's challenge
- Index map - submissions
Please do participate in the discussion at the submission links above. It's active and it's a great way to get comments on your code.
This week's challenge
String difference
You are given two strings, a and b. We consider each letter to be
unique, meaning duplicates are significant. Write a function that
returns the count of letters in b which do not occur in a.
Examples
(strdiff "abc" "") ;=> {} ;; no characters in b don't occur in a
(strdiff "abc" "abc") ;=> {} ;; ditto
(strdiff "" "abc") ;=> {\a 1 \b 1 \c 1}
(strdiff "axx" "abcc") ;=> {\b 1 \c 2}
(strdiff "xxxx" "xxxxxx") ;=> {\a 2} ;; two x's in b that don't occur in
a
Thanks to this site for the challenge idea where it is considered Hard in Python. The problem has been modified from the original.
Please submit your design process as comments to this gist. Discussion is welcome.
Rock on!
Eric Normand
