Types of Generative AI: Exploring Diverse Applications

Generative AI is a type of artificial intelligence that can create novel content. This content may look like the material researchers used to train the algorithm, but it represents a unique, original creation. You can use generative AI to create text documents, images, video, audio and sound, code, data, and more. Depending on the type of content you want the AI to generate, you can employ different types of generative AI models, like generative adversarial networks (GANs), variational autoencoders (VAEs), or transformer models, to accomplish your task. 

Explore types of generative artificial intelligence, what kind of work you can do with them, and some examples of generative AI apps and the companies that provide those models. 

What is generative AI?

Generative AI is a type of deep-learning model capable of creating novel text, videos, images, and more. These sophisticated models can understand patterns in the materials developers use to train them. They can learn how different parts of data relate to one another and use this information to deliver a fake item that looks like it could fit into that original set of training data. 

Developers train the AI model on a vast amount of data, including samples similar to the material the model will ultimately create, such as publicly available data sets, documents, books, code, images, audio, and video data. During training, the AI model begins to understand patterns in the training materials and how different items relate to one another. For example, an AI model can understand how the position of words in a sentence can change that sentence’s meaning. 

Then, during tuning, the model is refined for specific purposes—a text generator will concentrate on text, while an image generator will concentrate on images, and so on. After the model is ready to begin generating new content, developers continue to evaluate and fine-tune it as it creates new outputs so that it can constantly improve. 

Types of generative AI

You can define types of generative AI in various ways, such as different types of generative AI models, use cases, or companies providing specific models. Researchers presenting at the 57th Hawaii International Conference on System Sciences in 2024 proposed that professionals in the field define the taxonomy of generative AI using five categories [1]:

1. Generators: Create novel content based on input

2. Reimaginators: Interpret or innovate existing data in new ways

3. Synthesizers: Create synthetic data such as training data

4. Assistants: Offer support with specialized knowledge 

5. Enablers: Offer infrastructure and computational power

To get a closer look at different types of generative AI models, what they do, and what you can do with them, you can explore how each model works. You could also explore model types by the type of task you want help with, whether generating text, images, music and audio, videos, or code. 

Types of generative AI models 

Four common types of generative AI models include: 

• Generative adversarial networks: A generative adversarial network, or GAN, represents two dueling neural networks, the generator and the discriminator. The generator aims to create fake content that is indistinguishable from training data, and the discriminator aims to detect the fake. The two neural networks go back and forth until the generator wins, or the discriminator can’t tell the difference between the fake item and the training data. 

• Variational autoencoders: A variational autoencoder, or VAE, encodes or compresses data down into a simplified version that contains all of the most important elements while omitting details. After encoding, the decoder then rebuilds the original data set, generating new details around the most important elements. This introduces a bit of randomness that helps create unique items or variations on the original input. 

• Transformer-based models: A transformer model is a deep learning model that understands text by breaking it down into tokens, which are small components of text that can contain a character, a part of a word, or a short phrase. The model then turns the tokens into numerical vectors and analyzes how they relate to one another. Transformers also have a self-attention mechanism that gives them the ability to understand how important some words in a sentence are compared to others. Large language models, like Chat-GPT, use transformer-based models. (GPT stands for “generative pre-trained transformer.”) 

• Diffusion models: Diffusion models are a type of generative AI that adds noise (random sets of data points) to the input to distort the data, studies how that process alters the data, and then rebuilds a reverse-diffused original version of the input data. This process helps the AI model understand how the elements of the data relate to one another. After training, the model can use what it understands about the patterns of its training materials to generate content that meets your prompt's request. 

Types of generative AI use cases 

Another way to think about types of generative AI is to consider the main task the model can accomplish. Some generative AI models can generate multiple types of content, but some have limits in what they can provide or have specific uses in which they excel. Some of the more common types of generative AI use cases you may find include: 

• Text generation: Generative AI can help you produce text that explains concepts or answers questions. Generative AI can also create drafts of documents or assemble pieces of documents, like outlines or citations. Examples of models you might use to create text include ChatGPT, Google Gemini, or Perplexity AI. 

• Image generation: You may use generative AI to create new images, make changes to existing images, and produce images in a variety of styles. You can even prompt AI to employ a specific medium, such as sketching, photorealistic images, or an oil painting. To create images, you may use an AI model like DALL-E 2 from OpenAI, Midjourney, or Stable Diffusion from Stability AI.

• Music and audio generation: You can use generative AI to create audio data and music. Just like it does with text or images, generative AI can detect and understand the patterns in music and create similar, original works. You can also use generative AI to create an audio file of speech based on a text prompt. Two examples of models you may use for this task include Aiva AI and Soundful.

• Video generation: Generative AI can create videos, animations, and special effects. The functionality of the AI you use depends on what material was input during training. So, for example, an AI trained in video editing could help you add special effects, while other algorithms can create original videos. A few examples of video-generation AI apps include Canva, DeepAI, and Invideo. 

• Code generation: You can use generative AI to create code from scratch or to autocomplete code as you write it. The documentation for many programming languages is available online, which makes it possible for developers to use this data to train AI models that are fluent in these languages (which is similar to the way AI models can learn multiple human languages). A few examples of AI apps you may use to generate code include Google’s Gemini, Vertex AI, and Sonar. 

Generative AI examples

A final way to think about different types of generative AI is to consider the companies developing the models. Some companies, like OpenAI, are developing models that you can use and adapt to a broader range of purposes, while others, like Hugging Face, develop models for specific uses. (Hugging Face uses AI to create other AI and machine-learning solutions.) Generative AI companies include: 

• Aiva AI

• PwC

• Perplexity AI

• Synthesia

• Canva

• Anthropic

• Cohere

• Copysmith

• ElevenLabs

• Mistral AI

• Qualtrics

• Stability AI

• Afiniti AI

Explore types of generative AI on Coursera

You can use generative AI to help you create new content or make your creative process faster and easier. If you want to learn more about generative AI, consider taking an online course to learn new skills. For example, you could enroll in Generative AI for Everyone, offered by DeepLearning.AI, to explore generative AI tools, using AI for productivity, and creating an AI strategy.