This starts speculative and ends up in an interesting place: an algorithm for improving language. (My personal algorithm for new ideas appears to include: think about nonsense for longer than most others are prepared to tolerate.)
These words all start with horse-:
Horsefly – a fly found near horses, fair enough
Horseplay – totally understandable, but a metaphor that only makes sense if you have been friends with horses
Horsepower – sadly not like “sea power”
Horseradish – a root and a condiment
Horsetrade – now a useful way to talk about an equitable negotiation that involved a lot of back-and-forth, originally I’m sure to do with horses
Note that these are unlike: horsehair, horserider, horseshoe, etc, which are direct attributes of a horse. That is you could equally say “horse’s hair” and so on.
So it’s fun to invent other horse- words and speculate what they might mean. Some at random:
Horsefunctor
Horsehistory
Horseorbital
Horseroman
Horsesane
For example, #1: Rob Miller suggested on Twitter: Black Beauty is my favourite horseroman.
And that’s neat, right? “Roman” as in roman à clef, a novel not-so-secretly based on real life events, and Black Beauty is a novel about a horse. BUT ALSO maybe a horseroman could be like the famously equine senator appointed by Nero in Ancient Rome.
Let’s blend those two and throw in some semantic drift, then define horseroman as one of those thought leaders who is appointed by the people in charge, but is actually way out of their comfort zone, and answers all questions with lengthy anecdotes about their own life. Like a bad TED talk.
For example, #2: Peter Bleakley pointed out that: The “horse” in “horseradish” is nothing to do with horses. It’s cognate with “coarse” and indicates “inedible”. Same as “horse chestnut”.
So let’s take that and apply it!
Maybe horsehistory could be the socially uncomfortable parts of our history that we brush under the carpet? A useful new term!
(Thank you Tom Carden for continuing this conversation on Twitter yesterday.)
The first analysis from a horsehistorical perspective
This section is a tangent but I want to show you that the concept is useful.
I was never really taught about the British Empire or colonisation at school. Sure I knew about it, but it was always in the background, taken for granted. Sure there were some ugly aspects but aren’t there always.
Then in my 30s I visited a museum in Kolkata and unexpectedly began to learn about the atrocities committed by my country, and the feeling of sickness and shame that started on that day has never left me. I have continued to educate myself. What’s worse is that I did know some of the events, but I hadn’t stopped to consider them.
Empire is not, in the UK, ignored history. We all know it. But when you grow up with something from before you can speak, and leave it unanalysed, you accept facts that you would never accept as an adult. I imagine it’s a little like abuse: if you grow up in an abusive household, it takes work as an adult to realise: that wasn’t normal! That was not ok!
Now some of this is unconscious, but some of it is very much deliberate. We have leaders who do know history, who are able to talk about Empire (what it was, what we did, what we are still doing), but keep silent and make use of the idea of Empire (“Global Britain”, now). And because there’s utility in maintaining this myth in its unanalysed state, a kind of systemic resistance arises: anybody who does attempt to talk about Empire in an adult, clear-eyed way is aggressively shouted down. See the current culture war about “decolonising the curriculum” in universities which, to my mind, is simply about saying: let’s not take this history for granted.
My pet theory:
You can’t really talk about society as an individual, but give me this rope for a second. “British society” is aware of Empire and its atrocities, but conscious acceptance of that knowledge is repressed – for whatever reason: because it runs counter to our identity, because it is inconvenient, because then we would have to do something about it, because it was awful, take your pick.
In short, the history of the British Empire is impossible for British society to digest.
In an individual, repressed feelings find other ways to come out. So the repressed idea of colonisation came out as Brexit, which was this almost fanatical belief that we were being colonised by a larger state, EU. Or, to be blunt: that the pains that the British Empire had inflicted on others were now being inflicted on us.
I read this as a way to psychologically square the circle of the repression: to say, it’s ok to avoid looking at the history of Empire squarely in the face, because look it’s fair now, we’ve been punished.
Anyway, that’s my amateur read.
It’s also the first application of horsehistory: the study of undigestable histories and what they do to us. Agree or disagree with my analysis in this instance, the area opened up is interesting.
Maybe this study could also look at the ways that horsehistories become regular histories, and how we could take British society through that journey (several other countries with atrocities in their past have been able to look at their history with clear eyes, act accordingly, and are healthier for it).
Or we might also examine how horsehistories ossify over time (or not), or catalogue them globally, or re-analyse existing histories.
A new word becomes a new lens for understanding the world.
Words as coordinates in the space of all possible concepts
Back to the horse- prefix. What does it do to words?
It’s not a straight modifier. Horseplay is not the play of a horse (not any longer), nor horseradish a horse’s radish. It’s an unexpectedly transformative operator, in a way that I don’t yet understand.
Maybe: it’s a matrix rotation in embedding space?
To unpack that:
An “embedding” is how machine learning encodes concepts.
A good example is word2vec, an old technique (old meaning 2013) that takes words and translates them into coordinates in a multi-dimensional space of all possible concepts. The set of coordinates is called the embedding.
What’s neat is that the embeddings can be mathematically combined. That is to say:
king - man + woman = queen
If you take the coordinates for king, subtract man and add woman, you get the coordinates for queen. Approximately…
The resulting vector from “king-man+woman” doesn’t exactly equal “queen”, but “queen” is the closest word to it from the 400,000 word embeddings we have in this collection.
That example is from the EXCELLENT guide from Jay Allamar, The Illustrated Word2vec.
So thinking about the concept horsehistory using this model and attempting to decompose it, what we see is that it’s not the mathematical addition of the horse embedding and the history embedding. The horse- prefix has mutated the history embedding somehow and turned it into something else. That mutation is what I’m referring to as a matrix rotation.
Let me try another way:
Have you every tried miracle fruit? It’s a berry with unusual property – it doesn’t have a taste itself, but it changes other tastes. In particular it rotates sour to sweet.
So I had some of these berries and was drinking beer, and the beer tasted like Fanta. Amazing! Then some time later, suddenly as I was crossing a road, the berries wore off and I tasted the inside of my mouth as if for the first time, and good grief it was disgusting.
The horse- prefix is the miracle berry of words.
And also:
New words are addresses to previously unused embeddings in concept space.
The invention of new words provides new scaffolding for thought
I think what I’m convinced by, with horsehistory, is that it’s worth developing new words.
In the sci-fi novel Native Tongue (Bookshop.org) about aliens, linguistics, and a fierce patriarchy, Suzette Haden Elgin supposes a new language for women. The creation of this artificial language is an act of resistance and also way to carve out a space for unique feminist thought and being. (It’s a stunning book.)
In Native Tongue, discovering a new word in this new language is a big deal: a new word, a new concept, a new “Encoding” as Elgin calls it, a new valid embedding in concept phase space, we might say. Finding a new Encoding rarely happens! Each Encoding is hard won. If somebody discovers/invents one or two, that is huge news!
And so it is for us, I think. Discovering a new concept that isn’t simply a metaphorical framing, today, is rare and propels thought. Back to “Brexit” for a second: it derived from “Grexit,” itself a neologism (for Greece leaving the EU), but having been coined it was possible to poke at it the concept, to discuss it, to ask about how it could happen and if so when and what it would mean, and so on, and without the word I believe the process would have unfolded in another way entirely.
Yes we can initially refer to these same concepts in other more cumbersome ways, but as single words it is possible to combine and manipulate more ideas that are more complex, and then they take on their own reality. Cheap referents have value.
(Metaphorically, I’ve long believed that this process is what the Old Testament story of Moses receiving the Commandments is about, at least partially. There is an arduous journey - up a mountain - at which point some simple rules that are received, literally inscribed into rock. The rules are straightforward and lead to a long-term healthy and stable society, at least in this belief framework, but are hard to arrive at from first principles. The foundational ideas are too unwieldy and require a rare perspective. Similarly, great minds dedicate their lives to climbing their own mountains to claim complex insights similarly inscribed in simple terms and, having received the new concept in a graspable formulation and bringing it down the mountain for us, our whole society benefits.)
The question is: is it possible to move beyond Native Tongue? It’s slow. Can we automate the process of concept discovery?
A reward function for the invention of new words
Yes there are ways to invent new words with AI. Take the website ThisWordDoesNotExist.com which generates a random word and a plausible definition (making use of GPT-2, the ancestor of GPT-3 which, as previously discussed is an idea machine). For example here’s one generated word:
tokou
a black wine made from fermented soybeans cooked in molasses and yeast; “various African wines and tokou grape”
The problem is that tokou isn’t as useful as, say, horsehistory (at least on first glance). We could sit there, refreshing the website, trying out each word to see if it’s handy, but that’s a bottleneck in the process and besides I would lack the domain expertise in most cases. So any algorithm will have automate that process too.
Here’s the algorithm I propose.
1. Invent candidate words and their definitions.
Either invent at random, or use the method I ran by hand above: collide multiple embeddings and jiggle the result with semantic drift. That was how the horseroman concept was generated. An AI can do this.
Now we effectively have two languages: English, which is the language we speak, and English-Prime, which is identical save for the addition of this single new word.
2. Translate all of human knowledge into English-Prime.
This isn’t as hard as is sounds.
Google Translate has used, since November 2016, a system called Google Neural Machine Translation. It’s a machine learning method of translation between language pairs.
But look closer at how it works…
“Visual interpretation of the results shows that these models learn a form of interlingua representation for the multilingual model between all involved language pairs,” the researchers wrote in the paper.
An interlingua is a type of artificial language, which is used to fulfil a purpose. In this case, the interlingua was used within the AI to explain how unseen material could be translated.
… The data within the network allowed the team to interpret that the neural network was “encoding something” about the semantics of a sentence rather than comparing phrase-to-phrase translations.
In short, Google Translate has an intermediary language that exists only in the form of embeddings.
So, even without training data already written our new English-Prime, we know how to represent it in language space: it’s identical to English but with a single extra embedding available. And if we know that, Google Translate can translate into it. (Exactly how is left as an exercise for the reader.)
Which means that step 2 is to automatically translate all of Wikipedia into English-Prime.
3. The reward function: test whether the new word is useful
What does useful mean? Machine learning has the idea of a “reward function”: how do we state what good looks like? If we can do that, the process can run automatically.
For horsehistory it meant that the word acted as an intuition pump (philosopher Daniel Dennett’s term): by examining what it could mean, it took me to a place where complex ideas were reached and could then be articulated more efficiently.
So what is our reward function? Simply:
Compare the word count of Wikipedia in English and the word count of Wikipedia in English-Prime. If the latter is shorter, i.e. more efficient, then the proposed new word is useful.
RINSE AND REPEAT.
Each new term is a new place to start thinking. Give each its own journal and academic conference and see what happens.
Let me summarise.
By way of speculating about a single new field of study, horsehistory, I have proposed a general method for the automatic generation of many new fields of study.
In the same way that Magnus Carlsen is a “centaur” chess player, a player of chess greater than any other human because he has trained with a custom AI and benefited from the wisdom of this machine which is effectively 200 data-years old, the method I propose leads to a new discipline of centaur philosophers, thinkers who are able to systemically reveal new scaffoldings for thought, far beyond what would ordinarily be reached in a single human lifetime, to more rapidly develop and examine new ideas for the betterment of society at large.
Or the whole thing is horsefeathers. Take your pick.
‘Yes, we’ll see them together some Saturday afternoon then,’ she said. ‘I won’t have any hand in your not going to Cathedral on Sunday morning. I suppose we must be getting back. What time was it when you looked at your watch just now?’ "In China and some other countries it is not considered necessary to give the girls any education; but in Japan it is not so. The girls are educated here, though not so much as the boys; and of late years they have established schools where they receive what we call the higher branches of instruction. Every year new schools for girls are opened; and a great many of the Japanese who formerly would not be seen in public with their wives have adopted the Western idea, and bring their wives into society. The marriage laws have been arranged so as to allow the different classes to marry among[Pg 258] each other, and the government is doing all it can to improve the condition of the women. They were better off before than the women of any other Eastern country; and if things go on as they are now going, they will be still better in a few years. The world moves. "Frank and Fred." She whispered something to herself in horrified dismay; but then she looked at me with her eyes very blue and said "You'll see him about it, won't you? You must help unravel this tangle, Richard; and if you do I'll--I'll dance at your wedding; yours and--somebody's we know!" Her eyes began forewith. Lawrence laughed silently. He seemed to be intensely amused about something. He took a flat brown paper parcel from his pocket. making a notable addition to American literature. I did truly. "Surely," said the minister, "surely." There might have been men who would have remembered that Mrs. Lawton was a tough woman, even for a mining town, and who would in the names of their own wives have refused to let her cross the threshold of their homes. But he saw that she was ill, and he did not so much as hesitate. "I feel awful sorry for you sir," said the Lieutenant, much moved. "And if I had it in my power you should go. But I have got my orders, and I must obey them. I musn't allow anybody not actually be longing to the army to pass on across the river on the train." "Throw a piece o' that fat pine on the fire. Shorty," said the Deacon, "and let's see what I've got." "Further admonitions," continued the Lieutenant, "had the same result, and I was about to call a guard to put him under arrest, when I happened to notice a pair of field-glasses that the prisoner had picked up, and was evidently intending to appropriate to his own use, and not account for them. This was confirmed by his approaching me in a menacing manner, insolently demanding their return, and threatening me in a loud voice if I did not give them up, which I properly refused to do, and ordered a Sergeant who had come up to seize and buck-and-gag him. The Sergeant, against whom I shall appear later, did not obey my orders, but seemed to abet his companion's gross insubordination. The scene finally culminated, in the presence of a number of enlisted men, in the prisoner's wrenching the field-glasses away from me by main force, and would have struck me had not the Sergeant prevented this. It was such an act as in any other army in the world would have subjected the offender to instant execution. It was only possible in—" "Don't soft-soap me," the old woman snapped. "I'm too old for it and I'm too tough for it. I want to look at some facts, and I want you to look at them, too." She paused, and nobody said a word. "I want to start with a simple statement. We're in trouble." RE: Fruyling's World "MACDONALD'S GATE" "Read me some of it." "Well, I want something better than that." HoME大香蕉第一时间
ENTER NUMBET 0016humsocc.org.cn ggzddd.com.cn www.wbit.org.cn ofykjg.com.cn www.mlhy.net.cn pchao.com.cn sylmgjdl.org.cn tuolian.net.cn www.weiyigo.com.cn www.rjchain.com.cn
This starts speculative and ends up in an interesting place: an algorithm for improving language. (My personal algorithm for new ideas appears to include: think about nonsense for longer than most others are prepared to tolerate.)
These words all start with horse-:
Note that these are unlike: horsehair, horserider, horseshoe, etc, which are direct attributes of a horse. That is you could equally say “horse’s hair” and so on.
So it’s fun to invent other horse- words and speculate what they might mean. Some at random:
For example, #1: Rob Miller suggested on Twitter:
And that’s neat, right? “Roman” as in roman à clef, a novel not-so-secretly based on real life events, and Black Beauty is a novel about a horse. BUT ALSO maybe a horseroman could be like the famously equine senator appointed by Nero in Ancient Rome.
Let’s blend those two and throw in some semantic drift, then define horseroman as one of those thought leaders who is appointed by the people in charge, but is actually way out of their comfort zone, and answers all questions with lengthy anecdotes about their own life. Like a bad TED talk.
For example, #2: Peter Bleakley pointed out that:
So let’s take that and apply it!
Maybe horsehistory could be the socially uncomfortable parts of our history that we brush under the carpet? A useful new term!
(Thank you Tom Carden for continuing this conversation on Twitter yesterday.)
The first analysis from a horsehistorical perspective
This section is a tangent but I want to show you that the concept is useful.
I was never really taught about the British Empire or colonisation at school. Sure I knew about it, but it was always in the background, taken for granted. Sure there were some ugly aspects but aren’t there always.
Then in my 30s I visited a museum in Kolkata and unexpectedly began to learn about the atrocities committed by my country, and the feeling of sickness and shame that started on that day has never left me. I have continued to educate myself. What’s worse is that I did know some of the events, but I hadn’t stopped to consider them.
Empire is not, in the UK, ignored history. We all know it. But when you grow up with something from before you can speak, and leave it unanalysed, you accept facts that you would never accept as an adult. I imagine it’s a little like abuse: if you grow up in an abusive household, it takes work as an adult to realise: that wasn’t normal! That was not ok!
Now some of this is unconscious, but some of it is very much deliberate. We have leaders who do know history, who are able to talk about Empire (what it was, what we did, what we are still doing), but keep silent and make use of the idea of Empire (“Global Britain”, now). And because there’s utility in maintaining this myth in its unanalysed state, a kind of systemic resistance arises: anybody who does attempt to talk about Empire in an adult, clear-eyed way is aggressively shouted down. See the current culture war about “decolonising the curriculum” in universities which, to my mind, is simply about saying: let’s not take this history for granted.
My pet theory:
You can’t really talk about society as an individual, but give me this rope for a second. “British society” is aware of Empire and its atrocities, but conscious acceptance of that knowledge is repressed – for whatever reason: because it runs counter to our identity, because it is inconvenient, because then we would have to do something about it, because it was awful, take your pick.
In short, the history of the British Empire is impossible for British society to digest.
In an individual, repressed feelings find other ways to come out. So the repressed idea of colonisation came out as Brexit, which was this almost fanatical belief that we were being colonised by a larger state, EU. Or, to be blunt: that the pains that the British Empire had inflicted on others were now being inflicted on us.
I read this as a way to psychologically square the circle of the repression: to say, it’s ok to avoid looking at the history of Empire squarely in the face, because look it’s fair now, we’ve been punished.
Anyway, that’s my amateur read.
It’s also the first application of horsehistory: the study of undigestable histories and what they do to us. Agree or disagree with my analysis in this instance, the area opened up is interesting.
Maybe this study could also look at the ways that horsehistories become regular histories, and how we could take British society through that journey (several other countries with atrocities in their past have been able to look at their history with clear eyes, act accordingly, and are healthier for it).
Or we might also examine how horsehistories ossify over time (or not), or catalogue them globally, or re-analyse existing histories.
A new word becomes a new lens for understanding the world.
Words as coordinates in the space of all possible concepts
Back to the horse- prefix. What does it do to words?
It’s not a straight modifier. Horseplay is not the play of a horse (not any longer), nor horseradish a horse’s radish. It’s an unexpectedly transformative operator, in a way that I don’t yet understand.
Maybe: it’s a matrix rotation in embedding space?
To unpack that:
An “embedding” is how machine learning encodes concepts.
A good example is word2vec, an old technique (old meaning 2013) that takes words and translates them into coordinates in a multi-dimensional space of all possible concepts. The set of coordinates is called the embedding.
What’s neat is that the embeddings can be mathematically combined. That is to say:
king - man + woman = queen
If you take the coordinates for king, subtract man and add woman, you get the coordinates for queen. Approximately…
That example is from the EXCELLENT guide from Jay Allamar, The Illustrated Word2vec.
So thinking about the concept horsehistory using this model and attempting to decompose it, what we see is that it’s not the mathematical addition of the horse embedding and the history embedding. The horse- prefix has mutated the history embedding somehow and turned it into something else. That mutation is what I’m referring to as a matrix rotation.
Let me try another way:
Have you every tried miracle fruit? It’s a berry with unusual property – it doesn’t have a taste itself, but it changes other tastes. In particular it rotates sour to sweet.
So I had some of these berries and was drinking beer, and the beer tasted like Fanta. Amazing! Then some time later, suddenly as I was crossing a road, the berries wore off and I tasted the inside of my mouth as if for the first time, and good grief it was disgusting.
The horse- prefix is the miracle berry of words.
And also:
New words are addresses to previously unused embeddings in concept space.
The invention of new words provides new scaffolding for thought
I think what I’m convinced by, with horsehistory, is that it’s worth developing new words.
In the sci-fi novel Native Tongue (Bookshop.org) about aliens, linguistics, and a fierce patriarchy, Suzette Haden Elgin supposes a new language for women. The creation of this artificial language is an act of resistance and also way to carve out a space for unique feminist thought and being. (It’s a stunning book.)
In Native Tongue, discovering a new word in this new language is a big deal: a new word, a new concept, a new “Encoding” as Elgin calls it, a new valid embedding in concept phase space, we might say. Finding a new Encoding rarely happens! Each Encoding is hard won. If somebody discovers/invents one or two, that is huge news!
And so it is for us, I think. Discovering a new concept that isn’t simply a metaphorical framing, today, is rare and propels thought. Back to “Brexit” for a second: it derived from “Grexit,” itself a neologism (for Greece leaving the EU), but having been coined it was possible to poke at it the concept, to discuss it, to ask about how it could happen and if so when and what it would mean, and so on, and without the word I believe the process would have unfolded in another way entirely.
Yes we can initially refer to these same concepts in other more cumbersome ways, but as single words it is possible to combine and manipulate more ideas that are more complex, and then they take on their own reality. Cheap referents have value.
(Metaphorically, I’ve long believed that this process is what the Old Testament story of Moses receiving the Commandments is about, at least partially. There is an arduous journey - up a mountain - at which point some simple rules that are received, literally inscribed into rock. The rules are straightforward and lead to a long-term healthy and stable society, at least in this belief framework, but are hard to arrive at from first principles. The foundational ideas are too unwieldy and require a rare perspective. Similarly, great minds dedicate their lives to climbing their own mountains to claim complex insights similarly inscribed in simple terms and, having received the new concept in a graspable formulation and bringing it down the mountain for us, our whole society benefits.)
The question is: is it possible to move beyond Native Tongue? It’s slow. Can we automate the process of concept discovery?
A reward function for the invention of new words
Yes there are ways to invent new words with AI. Take the website ThisWordDoesNotExist.com which generates a random word and a plausible definition (making use of GPT-2, the ancestor of GPT-3 which, as previously discussed is an idea machine). For example here’s one generated word:
The problem is that tokou isn’t as useful as, say, horsehistory (at least on first glance). We could sit there, refreshing the website, trying out each word to see if it’s handy, but that’s a bottleneck in the process and besides I would lack the domain expertise in most cases. So any algorithm will have automate that process too.
Here’s the algorithm I propose.
1. Invent candidate words and their definitions.
Either invent at random, or use the method I ran by hand above: collide multiple embeddings and jiggle the result with semantic drift. That was how the horseroman concept was generated. An AI can do this.
Now we effectively have two languages: English, which is the language we speak, and English-Prime, which is identical save for the addition of this single new word.
2. Translate all of human knowledge into English-Prime.
This isn’t as hard as is sounds.
Google Translate has used, since November 2016, a system called Google Neural Machine Translation. It’s a machine learning method of translation between language pairs.
But look closer at how it works…
In short, Google Translate has an intermediary language that exists only in the form of embeddings.
So, even without training data already written our new English-Prime, we know how to represent it in language space: it’s identical to English but with a single extra embedding available. And if we know that, Google Translate can translate into it. (Exactly how is left as an exercise for the reader.)
Which means that step 2 is to automatically translate all of Wikipedia into English-Prime.
3. The reward function: test whether the new word is useful
What does useful mean? Machine learning has the idea of a “reward function”: how do we state what good looks like? If we can do that, the process can run automatically.
For horsehistory it meant that the word acted as an intuition pump (philosopher Daniel Dennett’s term): by examining what it could mean, it took me to a place where complex ideas were reached and could then be articulated more efficiently.
So what is our reward function? Simply:
Compare the word count of Wikipedia in English and the word count of Wikipedia in English-Prime. If the latter is shorter, i.e. more efficient, then the proposed new word is useful.
RINSE AND REPEAT.
Each new term is a new place to start thinking. Give each its own journal and academic conference and see what happens.
Let me summarise.
By way of speculating about a single new field of study, horsehistory, I have proposed a general method for the automatic generation of many new fields of study.
In the same way that Magnus Carlsen is a “centaur” chess player, a player of chess greater than any other human because he has trained with a custom AI and benefited from the wisdom of this machine which is effectively 200 data-years old, the method I propose leads to a new discipline of centaur philosophers, thinkers who are able to systemically reveal new scaffoldings for thought, far beyond what would ordinarily be reached in a single human lifetime, to more rapidly develop and examine new ideas for the betterment of society at large.
Or the whole thing is horsefeathers. Take your pick.