PurelyFunctional.tv Newsletter 442: Human interfaces should be rich and composed

Issue 442 - September 13, 2021 · Archives · Subscribe

Hurricane update 🌬️

Well, I've been back home for a week. Our house is okay and we have finished cleaning up the debris. School for my kids starts tomorrow. Grocery stores are restocking lost food. Life is slowly returning to normal.

Design Idea 💡

Human interfaces should be rich and composed

A lot of people replied to my discussion of tangible programming (as if with Lego). There were a lot of questions about what form it might take. It's definitely a research topic needing lots of exploration. Here's one area I think is useful to explore---if not for a tangible system, but one meant to push the envelope of personal power in computing.

We need to build in more functionality to our components. While it's good to break things apart, humans expect things to work across a variety of different contexts. The interface should facilitate that.

My friend was practicing some interview problems. They were all about choosing between stacks, heaps, queues, and linked lists. We'll call those the base data structures. He chose to do the exercises in Python and realized that the Python array handled all of the challenges easily. How was it that a single data structure could do that?

It's the same in Clojure. While vectors might not be the most efficient possible stack implementation, adding and removing from the end are constant-time operations, so vectors make decent stacks (and you can reason about their performance). Clojure vectors, like Python arrays, can act like different basic data structures at different times. That lends power to the language. Instead of deciding upfront what kind of data structure you need, you choose a composed one that can work in many situations. It's usually good enough, and no solution scales forever.

While Python's array is more human-oriented, I envision even more human-oriented primitives for my democratizing programming system. One such primitive is file storage. I think there is plenty of room to improve file storage so that it works in human contexts. Nothing I propose is technically impossible today. It's mostly just about putting together existing functionality into a unified interface, like Python's array.

Here's the scenario I can describe in two sentences:

As I watch conference talks on YouTube, I would like to archive them to be safely stored later in case they are taken off of YouTube. I would also like to watch them or listen to them on my phone without using the cell data connection (downloaded while I'm on wifi).

I can currently do this manually in several tedious steps by running commands and dragging and dropping. But automating this is currently difficult. It might take several weeks of development to program it from scratch. Because the English description is only two sentences (and everything can be done manually), it should ideally only take an hour or two to automate. Here are the steps the automation needs to perform:

1. Poll the YouTube API every 10 minutes to see if I've added a new video to a list.
2. If I have, download the video from YouTube to MP4 format.
3. Upload the video to S3.
4. Convert the video to MP3 audio.
5. Upload the MP3 audio to S3.
6. Generate an RSS feed of the videos and audios in that bucket.
7. Upload the RSS to S3.

I can then subscribe to the RSS in a podcast app on my phone. When I'm on YouTube, I add videos to a special list and they should show up in my podcast app within a reasonable time.

Let's go over one piece: Amazon S3. It is nicely designed to be very general-purpose. At its heart, the interface is simple: S3 stores blobs of bytes under a key. This is a very generic interface that makes it very useful. In one sense, a component is general because it does less. It has been nicely decomposed and the interface is simple.

However, without loss of generality, there are some ergonomics added to the interface: keys are URLs, you can fetch the blob with an HTTP GET, you can list keys using a prefix (a generalization of directory listing), etc. But let's add some more.

• File indexing and search
• File format conversion (fetch an MP4 video as MP3 audio, for instance)
• RSS feeds for key prefixes, search terms, etc.
• Other protocols besides HTTP (scp, ftp, smb, etc.)

These are only a few of the things we could add. But the point is, when someone wants to build a human-level workflow, they often have to move data across different contexts. Adding these features to S3 would make steps 4-7 unnecessary.

Usually, the work is a trivial program (like generating an RSS feed from a listing of an S3 bucket), but there are lots of such connecting programs, and the deployment and management of all the pieces (operations) dominate. The more the component can build in, the easier the construction will be. That work helps the component work across different contexts. In a sense, at the human level, a component is more general because it does more.

Pandemic update 😷

I know a lot of people are going through tougher times than I am. If you, for any reason, can't afford my courses, and you think the courses will help you, please hit reply and I will set you up. It's a small gesture I can make, but it might help.

I don't want to shame you or anybody that we should be using this time to work on our skills. The number one priority is your health and safety. I know I haven't been able to work very much, let alone learn some new skill. But if learning Clojure is important to you, and you can't afford it, just hit reply and I'll set you up. Keeping busy can keep us sane.

Stay healthy. Wash your hands. Wear a mask. Get vaccinated if you can. Take care of loved ones.

Stripe Climate 🌍

I just signed up for Stripe Climate. 1% of credit card transactions for my products go to fund carbon sequestration. I hope this is the first of many changes I will make to improve my impact on the climate.

If you sell using Stripe, you can sig n up very easily. As climate change gets worse, we need to find ways to actively remove carbon from the atmosphere. This is an easy, hassle-free way to do it.

Clojure Challenge 🤔

Last issue's challenge

Issue 440

• Dealing elements - Submissions

This week's challenge

Dijkstra's Algorithm

Write a function to calculate the shortest path between two nodes in a graph using Dijkstra's Algorithm. The graph is directional and is represented as a map. The keys are pairs of nodes and the value is the cost to traverse the edge between them, from the first node to the second node. If a pair of nodes does not exist in the map, it means that there is no edge between them.

Example graph

(def graph {[:a :b] 1
            [:a :c] 2
            [:c :a] 4})

Your function should return a path, consisting of a sequence of the names of the nodes to follow, including the starting end ending nodes.

Example

(shortest-path graph :c :b) ;=> [:c :a :b]
(shortest-path graph :a :a) ;=> [:a]
(shortest-path graph :a :b) ;=> [:a :b]
(shortest-path graph :b :c) ;=> nil (because no path exists)

For a description of the algorithm, see the wikipedia page or any textbook on algorithms.

Please submit your solutions as comments on this gist.

Rock on!
Eric Normand