OpenAI DevDay Roundup

Monday was OpenAI’s first DevDay. If you have 45 minutes, the replay is worth a watch:

https://www.youtube.com/watch?v=U9mJuUkhUzk

They blitzed through a lot of content. I asked ChatGPT to summarize what was said (by feeding it the transcript of the YouTube video); you can read its summary here. I find it interesting that its summary, in a sense, demonstrates both the power and limitations of AI: it notes what was said, but not what it might mean. AI has a view, but not an opinion.

I, on the other hand, have lots of opinions. So let me share what I found interesting in the presentation. Because this is opinion, I could be wrong. I won’t be offended if you think so.

Rise of The Citizen AI User Creator?

OpenAI has a business model, which is to sell API access, and a public relations model, which is to present itself to the public and regulators as a non-threatening and munificent provider of tools that can enhance our lives.

I’m sure there are many facets to this PR model, but a significant one has to be getting as much of the public using and benefiting from GPT as possible. They’ve been successful at it, that’s clear: there’s already about 100 million weekly active users of ChatGPT that Sam Altman called out in his keynote.

It’s interesting who they chose to represent those 100 million in the video he played as part of his keynote: it featured mostly non-business users. The titles for the 9 subjects were: Artist, Founder, Artist, Physician1, Futurist, Centenarian2, Engineer, Student, andArtist. At most 3 were “professional” users and the rest ordinary people. But they all testified as to how ChatGPT made a real, meaningful difference in their lives. This is the community of users that OpenAI puts forward.

The new concept of people publishing their own “GPTs” is clearly aimed at nurturing that community by enabling them to extend ChatGPT and share it with others — and eventually be able to charge for it (although we don’t yet know how or what the vigorish is).

For example, imagine you’re a high school, and you put your class descriptions and syllabi into your “GPT”. Then, parents and students can ask questions about classes at registration time without calling the office. Do that well, and you might make it into next year’s video!

By investing thought, planning, & effort, people who have real needs (or have identified needs in others) have the chance to produce a valuable tool. That will be great.

But the same thing that democratizes the building of AI tools also makes it effortless to add in a superficial bit of cosmetic gloss to GPT and proclaim it your own. That probably suits OpenAI just fine — the more people who talk about how they’ve created their own AI Bot the better, even if many of them are just AI’s version of a flashlight app.

For the rest of us, that’s not so great: we’re going to be deluged with people who are flacking their useless custom GPT or are offering to tell us the 3 secret tricks3 to make our own. I suggest you give it a try if you can: OpenAI’s editor walks you through the process. And view things with a gimlet eye.

Developer Announcements

Improved Models … and Speed?

New versions of both GPT-4 and GPT-3.5 were announced. The pricing went down while the capabilities went up. One thing I found funny is that of all the announcements, the one I really want, consistent speed, wan’t on the list.4

When it comes to speed, however, there’s a lot of between-the-lines going on. Coincident with DevDay was this update to the rate limits documentation:

What makes this especially noteworthy is the text that it replaced on the website that was there the week before:

The highlighting is mine.5 We’re in the dark about what it takes to get good performance out of GPT, probably because OpenAI doesn’t want to make commitments6. And now they’re no longer even saying “we may” give you better performance if you spend more money! But I can verify that, when I moved past $50 in spending, my response times got significantly faster. (If you want to jump tiers instantly, just buy enough credits). I appreciate that response times are hard to manage for an out of control juggernaut, but our users who find a 5 second delay acceptable probably won’t put up with a 30 second one that randomly happens. So, yes, response time is going to be a pressing issue for me.

Assistants … vs GPTs

Here’s a chart I created to make sense of the difference between GPTs and Assistants:

A GPT7 and an Assistant can basically do the same things, in different ways. With the assistant the developer is responsible for doing a lot more of the work (with a commensurate increase in control). The most interesting things about GPTs is the ability to call a private API to do things via an “action”.

At first, I thought the big news with Assistants was that they were stateful, keeping conversation history on their own, but I was wrong. The big newsthe ability to upload documents to them for a form of Retrieval Augmented Generation and their ability to do calculations (“code interpreter”). This will make it far easier to do complicated things.

Final Thoughts

It would be an overstatement to say that OpenAI is playing chess while the other LLM vendors are playing checkers. But not that much of one. OpenAI is winning over the public (how many people would participate a video talking about how LLAMA-7B has changed their lives in a relatable way?), and they are building some strong product differentiators (uploading files, code execution, actions) that puts them ahead of the competition.8

A postscript: Elon Musk tried to one-up OpenAI and announced Grok, his new LLM from xAI. If you want to sign up for the waiting list, you have to be a verified user on X (Twitter). That’s a crock, Grok.