AI Layers, "Filling Potholes", and Non-Chat Interfaces
Frameworks for investing and building in AI
Welcome to the 26th edition of Black Box. This is the second of a series on some of my recent thoughts on generative AI that I’m calling Thesis StAItments; read the first part here.
Several generative AI themes have come up repeatedly between my day job as chief of staff at Lex and my side hustle as a scout for Punch Capital. They have finally coalesced into working theories, which I present here in hopes of sparking discussion:
AI layers
Every tech wave has its own insults; for example, crypto had “paper hands”, “NGMI”, “bagholder”, and others. Generative AI has been less combative so far, but “GPT wrapper” still gets a lot of founders heated. Well, I think there is a new belittlement on the block: “AI layer”. An AI layer automates certain workflows in existing “host” software which would otherwise be manual or require several steps. For example, Boondoggle enables CRMs to automate contact entry, data analysis, and status updates by passively reading emails and LinkedIn and Twitter messages.
The criticism is that AI layers are temporary because their hosts will build these capabilities in-house at some point. After all, their incentive to do so as an end-to-end platform is much greater than that of the AI layers, which are obligate symbiotes precisely because they are not. This means AI layers exist in pockets of “catch up” time that inevitably close.
While I think this intuition is generally right, I believe some AI layers could be worthwhile investments. There are the usual considerations of how long it takes the host to innovate, pricing, UI/UX, customizability, and so on; but I think it comes down to the niche that the AI layer targets and whether the host tends to vertically integrate. For example, Boondoggle serves the same users as Salesforce, which is designed to be a complete solution. But an AI layer in e-commerce infrastructure would not necessarily attract the wrath of Shopify, which has a history of investing in “category winners” to inform an analogous solution for SMBs (e.g., Klaviyo / Shopify Email or Recharge / Shopify Subscriptions). AI layers building in markets that are underserved and non-competitive to their hosts could be wedges into their own end-to-end platforms.
“Filling potholes”
There has been a lot of hand-wringing over generative AI favoring Big Tech and incumbents like OpenAI given the insane resources and talent needed. For a while, I was not only part of this camp but questioned if any “normal” AI startup, e.g., with less than a billion dollars or 10 IOI gold medals, had a chance at becoming a generational company. I wanted to believe otherwise though, which is why I was delighted to come across this take on Sora from journalist Brian Merchant:
If Altman wants to keep the money flowing, he has to keep moving the goalposts, producing shiny, exciting new tools and talking the biggest game this side of Elon Musk — to that end, Sora fits the bill… It’s not that Sora is generating new and amazing scenes based on the words you’re typing… [Rather,] it’s automating the act of stitching together renderings of extant video and images.
While there is some truth in his accusation of “proliferating mythologies”, I think this does not give OpenAI enough credit. On the other hand, perhaps I had given the incumbents too much. If generative AI is like a wild jungle, these companies are the ones clearing new roads. They have the machines to go and discover new areas. But in their haste to press forward, they do a rough job and leave behind many potholes — impossibly high compute and energy costs, hallucinations, privacy and security risks, etc.
These potholes are each multi-billion problems. Filling them is the work of this generation of startups. Already, companies like Nomic, Truth Systems, and Credal are shoveling gravel, and many more will be built. There are AI opportunities yet that will be won by the small and scrappy.
Non-chat interfaces
One of my favorite Lex features is the ability to at-mention the eponymous AI in a comment. This allows users to focus Lex on just the text highlighted in the comment, which is really helpful for specific requests:
What I like about this is not the fine control it affords but the idea of asking Lex for feedback as the user would a human collaborator. Lex has a chatbot too, but something about that form factor feels artificial and impersonal to me. I think it is because so many other AI tools have chat interfaces, lots of which are frankly not very good. That association creates poor expectations that limits how people think about using the AI. Conversely, no such beliefs exist for human-native interactions like tagging others in a comment.
These mental blocks are surprisingly powerful! For example, although Lex has a built-in chatbot, some users have told me that they also use ChatGPT for “informal” questions like synonyms and fact-checking. One of our most highly requested features is the ability to talk to Lex, i.e., through speech to text. One user told me that simply looking at the words that she had written restricted her thinking and she could be more creative if she were speaking into her phone on the couch.
Yet the only non-chat interface that I have seen outside of Lex is this clever email-based GPT wrapper called HaiHai. It analogously treats the LLM as a human recipient who responds according to the address and content of the email, e.g., emailing recipes@haihai.ai will return a recipe that follows any dietary restrictions in the original message. Though whimsical, HaiHai is a great example of how non-chat interfaces can change the way that we think about and use AI.
If you are building an AI layer that fills a pothole with a non-chat interface, please let me know! Regardless, I would love to hear your thoughts. ∎
Thanks to Hunter at Vanik as well as Nam and Anh at Truth Systems for helping me think through these topics. Let me know your thoughts @jwang_18 or over LinkedIn!