Delusions From Middle-Earth

Today I submitted a public comment to the Federal Communications Commission on the proposal from SpaceX to launch up to a million satellites for orbital data centers, which I blogged about last Friday. I am now working on the public comment for the Reflect Orbital proposal to put giant mirrors into space to light up the night particularly for use by solar farms. I retrieved the Reflect Orbital proposal documents from the FCC portal and was disenchanted to find that the name of their initial test satellite with an 18-meter mirror is EARENDIL-1.

This is a name that comes from Tolkien: Eärendil was a half-elf in The Silmarillion who bore on his brow a jewel — a Silmaril — that shone like a bright star. This leads to the question: why are so many tech bros obsessed with Tolkien?

A lot of people have commented on this trait lately. A writer named Samuel Arbesman compiled a list of all the tech companies he could find that have names from Tolkien (there are 22). In an essay entitled Mythic Capital, Lee Konstantinou discusses how Tolkien teaches a lot about capital and politics and the technoutopian vision of breaking free of all limits. In the New York Times Michiko Kakutani writes about how the traditionalism running through Tolkien appeals to the tech bros and that they are drawn to the themes of kingly and magical power rather than the gentle settled life of the hobbits.

It is interesting that when Tolkien first got popular in the late 1960s and early 1970s it was pulling in people mostly from the hippie counterculture. Times have changed.

But Pratchett doesn’t seem to appeal to the tech bros — I don’t see too many companies celebrating the cabbages of the Sto Plains for some reason.

The Destruction of the Night Sky

There are two proposals before the U.S. Federal Communications Commission right now that would do horrifying things to the night sky. Both are currently open to public comment through March 6 and 9, and I’m gearing up to submit a couple of comments. The FCC is the federal agency that regulates satellite launches in the United States, and they are now in the practice of rubber stamping an awful lot of these.

The first proposal is from a company called Reflect Solar that wants to put giant mirrors in space for the purpose of turning night into day for selected localities, in particular solar farms. They plan to start with an test satellite in 2026 with an 18 meter mirror, and then by 2030 have 4000 satellites in orbit at an altitude of 625 km. Eventually they imagine orbiting 250,000 satellites. The math for the amount of solar energy one can obtain this way absolutely does not work out, but even the 4000-satellite plan would be catastrophic for both professional and amateur astronomy. Visual astronomy would become an extremely risky activity, since accidentally glimpsing the reflected light in a telescope or binoculars could cause permanent eye damage.

Not to be outdone, everybody’s favorite archvillain Elon Musk is wanting to orbit up to 1,000,000 satellites for spaceborne AI data centers. There are presently 14,000 active satellites in space and low earth orbit is already getting crowded. One risk is Kessler syndrome — that is, collisions from space debris causing the generation of more debris in a chain reaction, rendering the entire orbital zone unusable. Another is impacts on atmospheric chemistry as tens of thousands of satellites burning up when they reenter may contribute to ozone depletion and climate change. Advocates of space data centers also tend to neglect the laws of physics. It is a lot harder to cool down a data center in space than on Earth, since due to the vacuum of space the only mechanism for heat transfer is radiation, not conduction or convection. (This is why vacuum thermoses keep their contents hot or cold.)

Both these proposals are now getting mainstream media coverage, such as in the New York Times and the Washington Post. The organization DarkSky International has a web page on how to comment on the proposals.

Back To The Moon

This upcoming journey doesn’t seem to be getting much attention now given the train wreck of current world events, but the United States is on the verge of launching four astronauts on a trip around the moon. This is the Artemis II space mission, which will start no earlier than February 6th. The launch vehicle arrived at its launch pad last week. There are monthly launch windows, so if the initial attempt has to be scrubbed, they will postpone to the next or subsequent months. The mission profile is similar to that of Apollo 8 in December 1968, although unlike in Apollo 8, the spacecraft will be sent on a “free return” trajectory to the moon (the spacecraft will not have to fire its engines to get back to Earth).

I am excited and nervous and will be following closely. 1968 wasn’t exactly a year of peace and harmony either, so there’s that.

The Memory Keeper

I’m continuing down my genealogical rabbithole and while reading up on WikiTree I came across a reference to an obscure but quite intriguing piece of software called The Memory Keeper. This is genealogical and historical research that is built on something called TiddlyWiki.

TiddlyWiki is personal wiki software that extremely cleverly functions entirely inside of a single HTML page. The individual wiki pages are units called “tiddlers” and the code in the HTML page sets up forms to edit and save the tiddlers. I have been using TiddlyWiki since 2017 to keep a research log for work. There is a substantial community around TiddlyWiki who have built many extensions and plugins for the system.

Memory Keeper consists of a set of these plugins and templates that have been organized around genealogical and historical research. It is not meant as a replacement for traditional genealogy software but rather to help in the research process. The trouble with most genealogy software is that the software typically is good at organizing the results of the research (individuals, their relationships in families, events, places, and sources and citations) but the software isn’t really a place to record one’s working notes. Nowadays there are many software systems for taking non-linear notes (in addition to TiddlyWiki, systems like Zettlr, Obsidian, and Scrivener come to mind). What Memory Keeper does is marry the two types of software, providing fields for genealogical data while allowing for non-linear wiki entry linking.

I’ve been testing Memory Keeper out these past couple of days and I think it will be very useful. I’m using Hosea Curtice as my test case. Here is an illustration. There is a note in the published genealogy for the Curtice family that he served in the French and Indian Wars and that lists the captain commanding his company. I easily look up what company this was, but this leads into researching the campaigns of this company and its regiment. Traditional genealogy software will not have fields to store that information, but this is ideal for a wiki-based system.

My previous work with TiddlyWiki was not very sophisticated, but I see lots of potential for Memory Keeper, particularly around keeping track of geographies in personal historical research.

Epistemological Debt

There is a concept in software engineering called technical debt. Basically, this is something that accrues when taking shortcuts in building a software system. You solve the problem that is immediately at hand, but in so doing you neglect to think through edge cases and these come back to haunt you as use of the system expands and additional pieces get built out.

I think something analogous though more sinister occurs when working with large language models (e.g. ChatGPT and its rivals) which are the core of the AI boom. I’m calling this “epistemological debt”. LLMs are by design extremely good at returning plausibly sounding text, outputs that look correct on first glance but often contain some inaccuracies. For example, using AI-based transcripts and summaries of meetings are now commonplace: this is a standard function in Zoom these days. But what happens when the summaries get saved and become the official record of the meeting without anybody checking the generated text for the inaccuracies? Given that people are only getting more and more busy one suspects these failures to review happen all the time. The inaccuracies start to accumulate, and nobody can figure out what is truth and what is not.

On Painting and Thought

I’m continuing to explore sketching with my new Neocolor II crayons, and here is a painting I did today of one of the Sugar Bee apples from today’s grocery shop run. I’m starting to learn how the Neocolors work as their own distinct medium. They go on the paper very smoothly — it’s a wax crayon — and it’s easy to spread the pigments around with a wet paintbrush. Once the paper is dry again, you can draw on it with more crayon in another layer. I also picked up a trick from a video about drawing birds with Neocolor IIs. The artist in this video uses a plastic palette with a rough surface. After drawing on the rough surface with a crayon, one can pick up the pigment directly with a wet paintbrush, thus turning the crayon into what is effectively watercolor paint. This can solve some problems posed by only using the crayons directly on the paper, such as being able to create a smooth wash, or being able to paint details with a fine brush. I made up an instant rough palette surface by using steel wool on a yogurt container top, and tested this approach out.

It is interesting that learning how materials behave — in this case a new art medium — is as far as I can intuit is the domain of non-linguistic thought. When I wanted to add yellow spotting on top of the red of the apple, I just knew that my little rounded flat travel brush would be a good tool for this. I don’t believe language had anything to do with this thought.

This is a consequential realization because the trillions that are being invested right now in AI are being built for the most part on the manipulation of language. To the best of my knowledge the heart of today’s AI boom is large language models (LLMs). There was a piece published in The Verge today about how this is likely a philosophical error. The article is entitled “Large language mistake” and has the subheading “Cutting-edge research shows language is not the same as intelligence. The entire AI bubble is built on ignoring it.” The article draws upon a perspective piece published in Nature last year entitled “Language is primarily a tool for communication rather than thought”, arguing its case from contemporary neuroscience and linguistics. I’m not expecting AIs to know how to paint watercolor anytime soon.

Transcribing Catalan With My New Workstation

Thanks to the Easy Languages folks, I learned the power of target language subtitling of video content in language learning, and this has been a big part of my Catalan studies. The Easy Languages approach is to do double subtitling e.g. for Catalan this is subtitles in both Catalan and English. But it is also very helpful to watch videos that are singly subtitled in the target language, e.g. Catalan subtitles for Catalan video, and I have watching these where I can find them. The YouTube channel Català al Natural does this specifically for language learning, and as I’ve described earlier I have watched many episodes of the TV series El Foraster this way.

But most of the Catalan content on YouTube has no subtitling available, which limits its utility to a beginner in the language. What to do? I came up with a plan for adding automated subtitling to the video content, and tried this out yesterday with much success. The workflow is as follows: a) download the YouTube video to my workstation b) run speech-to-text software over the audio channel of the downloaded video and c) add the transcribed text as subtitles as one watches the video stored locally.

This approach came together very easily using my new workstation. The details are as follows. First, I used the program yt-dlp to download the video from YouTube. The next step is the speech-to-text conversion. I used Whisper here, which I believe is the best open source speech-to-text converter, at least that is what I gathered from working with the AI institute a year-and-a-half ago. This is software from the belly of the AI beast, coming from the company OpenAI. It is multilingual, and Catalan is one of the better performing languages in the software. The output from this program consists of transcribed text with timestamps. Finally, I watched the video in the program Celluloid, which turns out to be smart enough to take the text-with-timestamps and overlay the text on the video as subtitles at the right times.

It greatly helps the accuracy of the transcription not to have to do it in real time, as the software can take advantage of looking at the language context around the current timepoint to produce a better transcription. My new workstation is very helpful here, having a graphics card with 12 GB of VRAM memory. It still takes a while: it was transcribing at a rate of about 4x real speed (that is, a 12 minute video was taking about 3 minutes to transcribe). The output seems very good, though as a beginner in the language I am not the best one to judge.

I tested this system today with a couple of recent videos from VilaWeb, and was pleased with how it helped. I might try experimenting with double subtitling a la Easy Languages, since I think that is supported by the video playback software after some fiddling.

Patterns of Liberation

About four years I was doing some literature research on information and communications technology for sustainable development and came across the writings of Douglas Schuler, a computer scientist now retired from Evergreen State College in Washington, who works on democratic technology. He is most noted for the 2008 book Liberating Voices: A Pattern Language for Communication Revolution, published by MIT Press. I revisited this book today and it seems a good work to share in our present moment. As the title suggests, it is inspired by the highly influential 1977 book A Pattern Language: Towns, Buildings, Construction by architect Christopher Alexander. Liberating Voices takes a similar approach to the latter book and provides a catalogs of patterns helpful for positive social change.

The physical book for Liberating Voices seems hard to find but much of the content is replicated in the website the Public Sphere Project. In particular, there is a section specifically on the Liberating Voices pattern language. Several examples of these patterns include Linguistic Diversity, Participatory Design, Intermediate Technology, and Voices of the Unheard. There are 136 patterns listed in the original Liberating Voices publication and these are summarized in a set of cards here. Patterns which others have submitted are also listed here.

It looks like the Public Sphere Project has gone dormant for now but many of the patterns described there for social change are timeless, and it is well worth reviewing the set for ideas on how to act.

The Jujube AI

When I was a child, my mother and I crafted an AI out of a set of matchboxes and some jujubes (the small colored gummy candies). I enjoyed reading Scientific American when I was young, and at one point found an intriguing article by Martin Gardner, the columnist who wrote Mathematical Games for 25 years. This article was entitled A Matchbox Game-Learning Machine and was published in 1962 — I probably ran across it in a reprinted book collection.

Gardner’s article shows how to build an analog learning machine to play a simple game called hexapawn which involves moving pawns on a 3×3 chessboard. The rules are given in the article linked above. At right is a photo from a much more recent Instructables article showing the setup. In the game the human player moves first as white. The machine plays the black side. The system works as follows. On top of the boxes are diagrams illustrating all the possible states of the game after moves 2, 4, and 6 (the game can last no more than 7 moves) and the possible moves for black illustrated in different colored arrows. Inside each box are colored beads (or jujubes in my case) corresponding to the colored arrows on top. After the human moves, they find the box corresponding to the state of the game, randomly draw a colored bead on the top, and have black carry out the move indicated on the top by the corresponding arrow. The bead is set aside, and if black loses the game, the bead representing the final move is discarded from the machine. That way the machine learns that the final move is an incorrect one to take.

It turns out that given optimal play, black is guaranteed to win, and it doesn’t take very many rounds for the machine to become invincible — somewhere around 30 or 40 games played.

Does this qualify as AI? Absolutely. This demonstrates that machine learning doesn’t require digital computers. Admittedly, this approach doesn’t scale very well: Gardner’s hexapawn example with 24 matchboxes was based on an earlier system for tic-tac-toe that needs over 300 matchboxes.

I am interested in other examples of analog AI. In particular, contemporary board games often have subsystems for solitaire play that can be quite hard to beat. These generally do not learn from experience, but they do respond to current states of the game by setting goals and carrying out actions.

Technically Sweet

When you see something that is technically sweet, you go ahead and do it and you argue about what to do about it only after you have had your technical success. That is the way it was with the atomic bomb.
— J. Robert Oppenheimer

I am not the first to link this Oppenheimer quote to recent developments in AI but it seems quite apt. It is striking how quickly this era of generative AI has come about. The landmark paper presenting the theoretical architecture (Attention Is All You Need) behind large language models (i.e. ChatGPT and its relatives) was published in 2017. ChatGPT itself was released in November 2022, scaling up in complexity from the prototype model presented in the Attention paper by a factor of about 800.

The arrival of generative AI for images and video is another case of rapid evolution, well presented in a Stephen Welch YouTube video on the theory behind these technologies. Today there are numerous systems for generating video from text descriptions, but it took several mathematical breakthroughs in the past five years to get to these. For instance in February 2021 research was published describing a training method for placing images and their text descriptions in the same high-dimensional numerical space, but that was just the initial step in image generation, let alone video creation.

But the model built for the 2021 research was trained on 400 million pairs of images with corresponding text, scraped from we don’t know where. This week one of the big AI companies, Anthropic, settled out of court a major copyright class action lawsuit concerning the company’s use of millions of pirated books. Also this week, a wrongful death lawsuit was filed against the company OpenAI detailing how ChatGPT coached a teenager in committing suicide. Meanwhile, it has become clear that large language model-driven systems have security flaws that one can drive proverbial trucks through. And it has become incredibly easy to use text-to-image AI systems to create fake photographs for propaganda purposes. Pursuing the technically sweet has gotten well ahead of ethics. Again.