Categories
Uncategorized

Building a Command Line Application using GPT3

GPT3 allows a simple and accessible API for accessing and using AI in an application. The application I wanted to create was a command line utility to quickly lookup commands and associated command line switches.

Cbot, available on Github.

The language processing of GPT3 is well suited to process natural language questions and provide results across a wide multitude of obscure commands that would otherwise be difficult to memorize.

Getting Started

The development process starts in the playground of the OpenAI website. The examples section provides a number of examples, my command line bot was a modified version of a translation example.

You provide a number of examples and can then modify the options within GPT3 to increase or reduce randomness and provide alternatives for start/stop sequences. The beauty of GPT3 is that you don’t need to provide a lot of examples for the software to get really good at knowing the types of responses that you want.

With 5-6 samples the application was reliably producing useful results. The OpenAI playground makes it easy to export the API call as either a CURL command or as a short block of Python code.

The resulting Python code uses a library called openai and is otherwise six lines of code, two of those are actually unused. (yes I reported this bug.)

Google Colab

I had never previously written much Python code and Google Colab tool was a great way to get started. It’s essentially an interactive Python editing and debugging tool, and since it’s interactive and in the cloud there isn’t much setup needed.

I copied the exported code and started playing around. If you have a GPT3 API key you can try my initial version here.

In playing interactively, I was more easily able to figure out how to parse the JSON and I realized that including the platform name (Mac, Windows, Linux) helped the bot determine the appropriate platform specific commands and platform specific options and folders. This really improved the quality of results.

OpenAI API and limitations

The current version of the software doesn’t allow fine-tuning of the GPT3 completion API using your own database. For use-cases such as customer service or email completion, the ability to further train GPT3 on specific large data sets would be particularly useful. I do think this is coming as fine tuning was part of GPT2, but it hasn’t yet made its way into GPT3.

The other thing to note about the API is that it does take some practice and exploration. When providing a set of examples, it was difficult to unit-test the examples against known and desirable results. Very subtle changes in the prompts would yield very different results and the software would occasionally get stuck in a loop, feeding off of its own content.

Building the bot

Once I had the basic code working in Google Colab, I was able to get a version running on my computer. I had never programmed in Python and I ended up actually using GPT3 to help “auto-complete” some of my functions. I would do this by going back and forth with the playground with blocks of code. It wasn’t perfect but it felt much more natural and collaborative than the alternative of jumping back and forth between StackOverflow pages.

The core bot used the basic prompt from the playground example and a SQLite database to keep track of requests/responses and act as a local cache. This is likely overkill at this point but I thought it could be interesting if a general database of questions and answers could be compiled, filtered, enhanced and sorted over time. The database also acts as a speed and cost buffer since the GPT3 API is not free and not always the fastest.

Some examples:

$>cbot "How do I count the number of lines in a file?"
  wc -l filename.txt

$>cbot "How do I get the mime type of a file?"        
  file filename.txt

$>cbot "How do I create a file with the text 'hello world'"   
  echo hello world > hello.txt

$>cbot "How do I open php in interactive mode?" 
  php -a

$>cbot "How do I set my email using git config?"
  git config --global user.email "[email protected]"

$>cbot What is the current date
  date

Advanced Tricks

After using the bot for a few weeks I started to add some more advanced functionality.

The -x option allows you execute the command directly. The -c option allows you to copy the answer into the clipboard (a little safer than just executing it.)

$>cbot -x "how do I put my computer to sleep"
  Sleeping now...  

The -g option allows you to ask general questions.

$>cbot -g "Who was the 23rd president?"
  Herbert Hoover  

Lastly, you can use cbot to save command shortcuts. So if you’ve remembered an obscure command you can save it for later.

$>cbot -s listall "ls -la"
  Saving Shortcut
$>cbot -x listall  

Open Source

While this code is open source, the problem is that currently OpenAI isn’t available openly and that API keys are only available to select developers. This makes it currently impossible to publish open-source software that is meant to be used by typical end-users.

I am hopeful that end-user API keys will be made available so that open-source AI software and tools can be made more broadly available.

Watch the YouTube Video

Try the Code

Follow me on Twitter

Categories
innovation technology

GTP3 – Will AI replace programmers?

Huge thank you to those who checked out my last video, I was taken aback by the reaction and it was awesome to interact with so many people from around the world.

Reactions tended to break down into two categories.

  • This is so cool
  • OMG – my job! Will AI replace programmers?

I want to show some additional GPT3 demos and I also explain some of the different types of AI, machine learning, and how it’s amazing and cool but you don’t have to worry about your job, at least not YET! We do however need to worry about AI having bias and being racist, sexist, and more.

Let’s start with some recent highlights…

A context aware dictionary that knows the definition of a word based on the context.
An example of image recognition paired with GPT3 to show good and bad ingredients in a product.

An example of using GPT3 as a function within a spreadsheet.
An example of CSS and layout generation using GPT3
https://thoughts.sushant-kumar.com/. An example of a quote generator based on GPT3
An example of UI generation within Figma using GPT3
An example of using GPT3 to write SQL queries

These demos are amazing progress but in order to unsertand why programmer jobs aren’t in gepordy you need to understand the basics of AI. There are four categories to consider:

  • 1. Reactive machines – Big Blue / Chess-playing – they look at the environment and react.
  • 2. Machines with memory – These are AI’s that can remember things over time and learn from their environment. Self-driving cars are an example of this.
  • 3. Theory of mind – This is when a machine can think and understand that others can think. It can put itself in the shoes of someone else and serve basic needs and functions in a general way. This is called Artificial General Intelligence
  • 4. Self-aware – This is when a machine can has the abilities of the previous categories and can also understand it’s own existance. This is in the realm of science fiction and both categories #3 & 4 are theoretical areas of research and we’re not close to these yet.

GPT3 is mostly #1. While it has a lot of data it’s not designed to remember things from session to session. The model is pre-trained, that’s the PT in GPT and you can think of it as the world’s most sophisticated auto-complete. Similar to how when you start typing and Google completes the sentence. GPT is able to complete questions, code, html files, and more. Because it’s trained on so much data the auto-complete has context but not memory. It’s incredibly good but it’s not perfect and it isn’t tested as valid.

Most of the time the output of GPT3 will be a starting point, not the final product. In the examples above the HTML, SQL, CSS, and text that is produced is most likely to be a starting point but its quality and fidelity, while impressive is unlikely to be a final result.

As I said GPT3 is an amazing piece of technology and I can understand why people may worry about their job. Technology can cause this concern and it’s been going on since Aristotle and Ancient Greeks. Farmers have worried about tractors. Scribes worried about the printing press and mathematicians and typists worried about computers. There’s a term for this is Technological unemployment.

While technology can eliminate or shift jobs it also tends to create new jobs and new opportunities. Even if GPT3 is really good, the world will still need engineers, designers, poets, and creators, perhaps more than ever.

The problem with AI

I tend to be an optimist but there are areas that still need a lot of work when it comes to AI and in particular, bias tends to be a real problem.

Here Chukwuemeka shows an image recognition that isn’t trained with diversity in mind…

This is why diversity in technology is so important and it’s also why we need to be careful about the data that’s driving and powering the worlds most powerful autocomplete. AI tends to work off of large collections of data. This can be imaging data, text data. If we’re not careful about the input data and testing it can produce problems.

In another study out of MIT Joy Buolamwini explores the notion of algorithmic bias and how it can be a huge problem.

Joy has a great TedX talk on this topic if you want to learn more.

As developers start incorporating GPT3 into their products and technologies, it’s important that they consider all sorts of biases that may be in the data.

Bad jokes, offensive ideas, historical inaccuracies, propaganda, sexism, racism, and more. In the billions of tokens that GPT3 has processed, it’s gotten good at auto-completing many things including some that we may offensive, inaccurate, or even dangerous.

Sam Altman – one of the founders of OpenAI touched on this recently in response to Jerome Pesenti, the head of AI at Facebook:

It’s great that OpenAI is taking bias seriously and it’s important that engineers building and incorporating AI into their products consider how their training data may have biases.

Thank you to everyone who watched my last video and checked out this post. I’m incredibly grateful to you for your feedback and comments. If you’re new to the blog, I tend to talk about entrepreneurship, technology, design, so if you like that sort of thing you can sign-up to get updates when I post. You can also subscribe on YouTube if you prefer.

Categories
innovation technology

GPT 3 Demo and Explanation

Last week GPT3 was released by OpenAI and it’s nothing short of groundbreaking. It’s the largest leap in artificial intelligence we’ve seen in a long time and the implications of these advances will be felt for a really long time.

GPT 3 can write poetry, translate text, chat convincingly, and answer abstract questions. It’s being used to code, design, and much more.

I’m going to give you the basics and background of GPT3 and show you some of the amazing creations that have started to circle the Internet in just the first week of the technology being available to a limited set of developers.

Let’s start with a few examples of what’s possible.

Demonstration of GPT-3 designing user interface components:

The designer is able to describe the interface that they want and the GPT3 plug-in to Figma is able to generate the UI.

GPT3 creating a simple react application:

Here the developer describes the React application that they want and the AI writes a function with the hooks and events needed to function correctly.
A couple of examples of GPT3 creating paragraphs of text based on initial cues of what is needed.
In this example GPT3 is able to complete an Excel table of data.
This example demonstrates a web plug-in to find answers within a Wikipedia article.
Example of using GPT3 as an answer engine for arbitrary questions.

Background

GTP3 comes from a company called OpenAI. OpenAI was founded by Elon Musk and Sam Altman (former president of Y-combinator the startup accelerator). OpenAI was founded with over a Billion invested to collaborate and create human-level AI for the benefit of the human race.

OpenAI has been developing it’s technology for a number of years. One of the early papers published was on Generative Pre-Training. The idea behind generative pre-training is that while most AI’s are trained on labeled data, there’s a ton of data that isn’t labeled. If you can evaluate the words and use them to train and tune the AI it can start to create predictions of future text on the unlabeled data. You repeat the process until predictions start to converge.

The original GPT stands for Generative Pre Training and the original GPT used 7000 books as the basis of training. The new GPT3 is trained on a lot more… In fact it’s trained on 410 billion tokens from crawling the Internet. 67 Billion from books. 3 Billion from Wikipedia and much more. In total it’s 175 Billion parameters and 570GB of filtered text (over 45 Terrabytes of unfiltered text)

Over an ExaFLOP day of compute needed to train the full data set.

The amount of computing power that was used to pre-train the model is astounding. It’s over an exaflop day of computer power. One second of exaflop computer power would allow you to run a calculation per second for over 37 Trillion years.

The GPT3 technology is currently in limited Beta and early access developers are just starting to produce demonstrations of the technology. As the limited Beta expands you can expect to see a lot more interesting and deep applications of the technology. I believe it’ll shape the future of the Internet and how we use software and technology.

Links: