How to work with ChatGPT and GPT-4 models - Azure OpenAI Service (2023)

Works with ChatGPT and GPT 4 models

The following code snippet shows the simplest way to use the ChatGPT and GPT-4 models with the Chat Completion API. If this is your first time using these models programmatically, we recommend starting with ours.

GPT-4 models are currently only available on request.Existing Azure OpenAI customers can do thisRequest access by filling out this form.

import osimport openaiopenai.api_type = "azure"openai.api_version = "2023-05-15" openai.api_base = os.getenv("OPENAI_API_BASE") # The Azure OpenAI resource of value endpoint.openai.api_KEI_KEY(os" ") ; Response = openai.ChatCompletion.create( engine="gpt-35-turbo", # The implementation name you chose when implementing the ChatGPT or GPT-4 model. messages=[ {"role": "system" , " content " : "Assistant is a large language model trained by OpenAI."}, {"role": "user", "content": "Who was the founder of Microsoft?"} ])print(response)print( response [ 'option'][0]['message']['content'])

Production

{ "choices": [ { "finish_reason": "stop", "index": 0, "message": { "content": "The founders of Microsoft are Bill Gates and Paul Allen. They founded the company in 1975 with . " , "role": "assistant" } } ], "created": 1679014551, "id": "chatcmpl-6usfn2yyjkbmESe3G4jaQR6bsScO1", "model": "gpt-3.5-turbo-0301", "chat": ", " usage ": { "completion_tokens": 86, "prompt_tokens": 37, "total_tokens": 123 }}

Each answer contains oneend_reason. The possible values ​​forend_reasonIs:

• I'm waiting: The API returned the full model output.
• Long: Incomplete model output due to max_tokens parameter or token limit.
• content filter: Excluded content due to being flagged by our content filters.
• null:The API response is still in progress or incomplete.

Examine the settingmax_tokensto a slightly higher value than usual, e.g. 300 or 500. This ensures that the model does not stop generating text before it reaches the end of the message.

model releases

In contrast to the previous models GPT-3 and GPT-3.5gpt-35-turboCombined modelgpt-4Andgpt-4-32kThe models continue to be updated. If you create oneintroductionFor these models, you must also specify a model version.

Currently only version0301is available for ChatGPT and0314for GPT-4 models. We will continue to provide updates in the future. Model payback times can be found in ourmodel makerPage.

Working with the Chat Completion API

OpenAI has trained the ChatGPT and GPT-4 models to accept input formatted as chat. The message parameter uses an array of dictionaries with a conversation organized by role.

The format for a simple chat output is:

{"role": "system", "content": "Give the model context and/or instructions"},{"role": "user", "content": "User messages go here"}

A conversation with a sample answer followed by a question looks like this:

{"role": "system", "content": "Provide context and/or instructions for the model."},{"role": "user", "content": "Here is a sample question." }, {"role": "assistant", "content": "See sample answers here."},{"role": "user", "content": "First question/message that the model will actually answer. "}

role of the system

The system role, also called system message, is listed at the beginning of the table. This message contains the first instructions for the model. You can provide various information to the system role, including:

• A brief description of the wizard
• Helper Personality Traits
• Policies or rules for the assistant to follow
• Data or information required for the model, e.g. B. Related questions from a FAQ

You can customize the system role to suit your use case or just add basic instructions. The system role/message is optional, but it is recommended to specify at least one key role for best results.

Messages

After the system role, you can insert a series of messages in betweenuserAndAssistant.

{"role": "user", "content": "What is thermodynamics?"}

To trigger a response from the model, end with a user message indicating it is the helper's turn to respond. You can also insert a series of example messages between the user and the assistant to learn how to shoot.

Examples of message prompts

The following section shows examples of different message styles you can use with the ChatGPT and GPT-4 models. These examples are just a starting point, and you can experiment with different prompts to customize the behavior for your own use cases.

Simple example

If you want the ChatGPT model to behave as followschat.openai.com, you can use a simple system message like "The wizard is a large language model trained by OpenAI".

{"role": "system", "content": "Assistant is a large language model trained by OpenAI."},{"role": "user", "content": "Who were the founders of Microsoft? "}

Example with instructions

In some scenarios, you might want to give the model additional instructions to set guard rails for the model's capabilities.

{"role": "system", "content": "Assistant is an intelligent chatbot designed to help users answer their tax questions. Instructions: - Only answer tax-related questions. - If you are unsure of an answer, you should be able to say "I don't know" or "I'm not sure" and suggest users go to the IRS website for more information."},{"role": "user", " content ": "When is my honey due?"}

Use grounding data

You can also include relevant data or information in the system message to give the model additional context in the conversation. If you only need to provide a small amount of information, you can encrypt it in the system message. If you have a large amount of data that the model needs to know, you can use thatThe integrationor a product thatAzure Cognitive Searchto retrieve the most relevant information at the time of the request.

{"role": "system", "content": "The Assistant is an intelligent chatbot designed to help users answer technical questions about the Azure OpenAI service. Please answer questions using the box below only. If you join If you are unsure of an answer, please contact us You can say “I don't know.” Framework: - The Azure OpenAI service provides REST API access to OpenAI's powerful language models, including the GPT-3 -, Codex, and Embeddings model series.—The Azure OpenAI service provides customers with advanced artificial language intelligence using OpenAI GPT-3, Codex, and DALL-E models with the security and business promise of Azure. Azure OpenAI develops APIs together with OpenAI, ensuring compatibility and a smooth transition from one to another - At Microsoft, we are committed to advancing AI with human-centric principles Microsoft has made significant investments to protect against misuse and unintended harm to contribute. This also includes that applicants must demonstrate clearly defined use cases that include the Microsoft principles for the responsible use of artificial intelligence.},{" role" : "user", "content": "What is the Azure OpenAI service?" }

Improve learning with Conversation Completion

You can also provide some sample shots for the model. The process for learning some shots has changed slightly due to the new prompt format. You can now include a series of messages between the user and the assistant as sample downloads in the command prompt. These examples can be used to derive answers to common questions, to initialize the model, or to teach the model specific behaviors.

This is just an example of how you can use Shot Learning with ChatGPT and GPT-4. You can experiment with different approaches to see what works best for your use case.

{"role": "system", "content": "O Assistant is an intelligent chatbot designed to help users answer their tax questions. "},{"role": "user", "content": "When must Do I do this? Do you have to file my taxes by April 18? falls on a Saturday in 2023. See https://www.irs.gov/filing/individuals/when-to-file for details."} ,{"role": " user", "content": "How can I check the status of my tax refund?"},{"role": "assistant", "content": "You can check the status of your tax refund at https: //check www.irs.gov/refund " }

Use chat integration for non-chat scenarios

The Chat Closing API is designed for use in multi-round conversations, but works well for non-chat scenarios as well.

For example, for a device extraction script, you can use the following command prompt:

{"role": "system", "content": "You are a helper designed to extract entities from text. Users input a text string and you respond with entities extracted from the text as a JSON object. Here is an example format output: { "name": "", "company": "", "phone_number": ""}"}, {"role": "user", "content": "Hello. My name is Robert Smith. I'm call from Contoso Insurance, Delaware. My colleague mentioned that you're interested in learning more about our comprehensive benefits policy. When you get a chance, could you please call me back at (555) 346-9322 so we can discuss benefits?" }

Create a simple chat loop

The examples so far have shown you the basic mechanics of interacting with the Chat Completion API. This example shows you how to create a chat loop that performs the following actions:

• It takes continuous input to the console and properly formats it as part of the message table as user role content.
• Prints responses written to the console and formatted and added to the message array as helper roll content.

This means that each time a new question is asked, a running transcript of the previous conversation is sent along with the last question. Because the model has no memory, you must send an updated transcript with each new question, otherwise the model will lose the context of previous questions and answers.

import osimport openaiopenai.api_type = "azure"openai.api_version = "2023-05-15" openai.api_base = os.getenv("OPENAI_API_BASE") # The endpoint value of the Azure OpenAI resource .openai.api_KEI) =_( os " ) entertainment=[{"role": "system", "content": "You are a useful helper."}]while(True): user_input = input() talk.append({"role": "user" , " content": user_input}) answer = openai.ChatCompletion.create( engine="gpt-3.5-turbo", # The implementation name you chose when implementing the ChatGPT or GPT-4 model. messages = chat ) chat .append ( {" role": "assistant", "content": reply['choice'][0]['message']['content']}) print("\n" + reply['choice' ] [0 ] [' message']['content'] + "\n")

When you run the code above, a blank console window appears. Type your first question in the window, then press Enter. Once the answer comes back, you can repeat the process and ask more questions.

handling conversations

The previous example runs until you reach the model's token limit. For each question asked and answered:MessagesThe array gets bigger. The token limit forgpt-35-turbois 4096 tokens, while the token limit forgpt-4Andgpt-4-32kare 8192 and 32768, respectively. These limits include the number of tokens from both the sent message table and the model response. The number of tokens in the message table combined with their valuemax_tokensThe parameter must remain below these limits, otherwise you will receive an error message.

It is your responsibility to ensure that the prompt and completion are within the token limit. That means you'll have to keep track of token counts on longer conversations and just send the model a prompt that's within the limit.

The code sample below shows a simple chat loop example with a technique to process a count of 4096 tokens using OpenAI's Tiktoken library.

Code requires Tiktok0,3,0. If you have an older version, run itPip Installer Tiktoken – Upgrade.

import tiktokenimport openaiimport osopenai.api_type = "azure"openai.api_version = "2023-05-15" openai.api_base = os.getenv("OPENAI_API_BASE") # The endpoint value of the Azure OpenAI resource "apiosgekey" os. OPENAI_API_KEY")system_message = {"role": "system", "content": "You are a useful helper."}max_response_tokens = 250token_limit= 4096conversation=[]conversation.append(system_message)nustem_message)nustem_message"decles_message" 3.5-turbo -0301 "): binding = tiktoken.encoding_for_model(model) num_tokens = 0 for message in messages: num_tokens += 4 # follow each message{rolle/name}\n{content}\ n for key, value in message.items(): num_tokens += len(encoding.encode(value)) if key == "name": # If name is present, the role is skipped. num_tokens += -1 # role is always required and always 1 token num_tokens += 2 # is initialized with each responseHelper returns num_tokenswhile(True): user_input = input("") conversation.append({"role": "user", "content": user_input}) conv_history_tokens = num_tokens_from_messages(conversation) while (conv_historyxresponse) share tokens conversation[ 1 ] conv_history_tokens = num_tokens_from_messages(conversation) response = openg.bo-5, pleg-engine-com, #openi. Deployment name you chose when deploying ChatGPT or GPT-4 model. messages = chat, temp=.7, max_tokens=max_response_tokens, ) talk.append({"role": "assistant", "content": answer['selection'][0]['message']['content'] }) print("\n" + reply['option'][0]['message']['content'] + "\n")

In this example, the oldest messages in the chat transcript are removed when the number of tokens is reached.vonis used instead ofclick()For efficiency, we'll start at index 1 to always keep the system message and only remove user/helper messages. Over time, this method of driving the conversation can cause the quality of the conversation to degrade as the model gradually loses context in the earlier parts of the conversation.

An alternative approach is to limit the talk duration to the maximum token duration or to a specific number of rounds. If the maximum token limit is reached and the model loses context when you allow the conversation to continue, you can ask the user to start a new conversation and delete the message board to start a whole new conversation with the full available token limit.

The token counting part of the code presented earlier is a simplified version of one of theseExamples of OpenAI cookbooks.

Next Step

• Learn more about Azure OpenAI.
• Start with the ChatGPT modelChatGPT quick start.
• For more examples seeAzure OpenAI samples GitHub repository

Working with ChatGPT models

The following code snippet shows the simplest way to use ChatGPT models with ChatML. If this is your first time using these models programmatically, we recommend starting with ours.

import osimport openaiopenai.api_type = "azure"openai.api_base = "https://{your-resource-name}.openai.azure.com/"openai.api_version = "2023-05-15"openai.api_key = os. getenv("OPENAI_API_KEY")response = openai.Completion.create( engine="gpt-35-turbo", # The implementation name you chose when implementing the ChatGPT model prompt="<|im_start|>system\nDer Assistant is a large language model trained by OpenAI.\n<|im_end|>\n<|im_start|>user\nWho were the founders of Microsoft?\n<|im_end|>\n<|im_start|>assistant \n", temperature = 0 , max_tokens=500, top_p=0.5, stop=["<|im_end|>"])print(response['choice'][0]['text'])

The<|im_end|>The token indicates the end of a message. We recommend inclusion<|im_end|>Token as a break sequence to ensure that the model stops generating text when the end of the message is reached. For more information on special tokens, seeChat Markup Language (ChatML)Unit.

Examine the settingmax_tokensto a slightly higher value than usual, e.g. 300 or 500. This ensures that the model does not stop generating text before it reaches the end of the message.

model releases

In contrast to the previous models GPT-3 and GPT-3.5gpt-35-turboCombined modelgpt-4Andgpt-4-32kThe models continue to be updated. If you create oneintroductionFor these models, you must also specify a model version.

Currently only version0301is available for ChatGPT. We will continue to provide updates in the future. Model payback times can be found in ourmodel makerPage.

Working with Chat Markup Language (ChatML)

OpenAI trained ChatGPT with special tokens that delineate the different parts of the prompt. The prompt begins with a system message used to start the model, followed by a series of messages between the user and the helper.

A simple ChatML prompt looks like this:

<|im_start|>system Provide context and/or instructions to the model.<|im_end|> <|im_start|>user User message goes here<|im_end|> <|im_start|>helper

system message

The system message is inserted at the beginning of the intermediate prompt<|im_start|>systemAnd<|im_end|>Brands. This message contains the first instructions for the model. You can enter various information in the system message, such as:

• A brief description of the wizard
• Helper Personality Traits
• Policies or rules for the assistant to follow
• Data or information required for the model, e.g. B. Related questions from a FAQ

You can customize the system message for your use or just add a simple system message. The system message is optional, but it is recommended that you provide at least a basic message for best results.

Messages

After the system message, you can insert a series of messages in betweenuserAndAssistant. Every message must begin with<|in_start|>symbol followed by the role (userDieAssistant) and we end with<|im_end|>flight

<|im_start|>user What is thermodynamics?<|im_end|>

To trigger a response from the model, the prompt must end with<|in_start|>AssistentIcon indicating it is the helper's turn to respond. You can also add messages between the user and the helper to the prompt to learn something.

Short examples

The following section shows examples of different message styles you can use with the ChatGPT and GPT-4 models. These examples are just a starting point, and you can experiment with different prompts to customize the behavior for your own use cases.

Simple example

If you want the ChatGPT and GPT-4 models to behave as followschat.openai.com, you can use a simple system message like "The wizard is a large language model trained by OpenAI".

<|im_start|>systemAssistant is a large language model trained by OpenAI.<|im_end|><|im_start|>userWho were the founders of Microsoft?<|im_end|><|im_start|>assistant

Example with instructions

In some scenarios, you might want to give the model additional instructions to set guard rails for the model's capabilities.

<|im_start|>systemAssistant is an intelligent chatbot designed to help users answer their tax questions. Instructions: - Only answer tax-related questions. - If you are unsure of an answer, you can say "I don't know" or "I'm not sure" and suggest users visit the IRS website for more information.<|im_end|><|im_start |> userWhen is my tax due?<|im_end|><|im_start|>helper

Use grounding data

You can also include relevant data or information in the system message to give the model additional context in the conversation. If you only need to provide a small amount of information, you can encrypt it in the system message. If you have a large amount of data that the model needs to know, you can use thatThe integrationor a product thatAzure Cognitive Searchto retrieve the most relevant information at the time of the request.

<|im_start|>systemAssistant is an intelligent chatbot designed to help users answer technical questions about the Azure OpenAI Service. Only answer questions using the box below. If you are unsure of an answer, you can say "I don't know." Context: - The Azure OpenAI service provides REST API access to OpenAI's powerful language models, including the GPT-3, Codex, and Embedding model series. - Azure OpenAI service offers customers an advanced AI language with OpenAI GPT-3, Codex and Embedding models DALL-E with the Azure security and promise of the company. Azure OpenAI develops the APIs alongside OpenAI, which ensures compatibility and a smooth transition from one to the other. - At Microsoft, we are committed to advancing artificial intelligence, guided by human-first principles. Microsoft has made significant investments to help protect against misuse and accidental harm. This also includes that applicants must demonstrate clearly defined use cases that take into account the Microsoft principles for the responsible use of artificial intelligence<|im_end|><|im_start| >userWhat is Azure OpenAI? Services?< | im_end|><|im_start|>helper

Enhance learning with ChatML

You can also provide some sample shots for the model. The process for learning some shots has changed slightly due to the new prompt format. You can now include a series of messages between the user and the assistant as sample downloads in the command prompt. These examples can be used to derive answers to common questions, to initialize the model, or to teach the model specific behaviors.

This is just an example of how you can use the Shooting Tutorial with ChatGPT. You can experiment with different approaches to see what works best for your use case.

<|im_start|>systemAssistant is an intelligent chatbot designed to help users answer their tax questions. <|im_end|><|im_start|>user By when do I have to file my tax return?<|im_end|><|im_start|>helper In 2023 you have to file your tax return by April 18 at the latest. The date is after the usual April 15 deadline, as April 15 falls on a Saturday in 2023. See https://www.irs.gov/filing/individuals/when-to-file<|im_end | for more details ><| im_start|>userHow do I check the status of my tax refund?<|im_end|><|im_start|>helper You can check the status of your tax refund by going to https://www.irs.gov/refunds<| go im_end | >

Using conversational markup for non-conversational scenarios

ChatML is intended to make it easier to manage multi-round conversations, but it also works well for non-chat scenarios.

For example, for a device extraction script, you can use the following command prompt:

<|im_start|>systemDu is a helper for extracting entities from text. Users enter a text string and you respond with entities extracted from the text as a JSON object. Here is an example of your output format: { "name": "", "company": "", "phone_number": ""}<|im_end|><|im_start|>userHello. My name is Robert Smith. I'm calling from Contoso Insurance, Delaware. My colleague mentioned that you are interested in learning more about our comprehensive benefits policy. Could you call me back at (555) 346-9322 when you have a chance to discuss the benefits?<|im_end|><|im_start|>assistant

Prevent insecure user logins

It's important to add restrictions to your application to ensure safe use of conversational markup.

We recommend that you prevent end users from including special tokens in their input data, e.g<|in_start|>And<|im_end|>. We also recommend including additional validation to ensure that the messages you send to the model are properly formatted and conform to the Chat Markup Language format described in this document.

You can also provide instructions in the system message to tell the model how to respond to specific types of user input. For example, you can instruct the model to only reply to messages on a specific topic. You can also reinforce this behavior with some examples.

handling conversations

The token limit forgpt-35-turboit is 4096 marks. This limit includes the number of tokens for both prompt and completion. The number of tokens in the prompt combined with their valuemax_tokensThe parameter must remain below 4096, otherwise you will receive an error message.

It is your responsibility to ensure that the prompt and completion are within the token limit. This means that on longer conversations you can keep track of the token count and just send the model a prompt that is within the token limit.

The following code sample shows a simple example of how to track individual messages in chat.

import osimport openaiopenai.api_type = "azure"openai.api_base = "https://{your-resource-name}.openai.azure.com/" #This corresponds to the Azure OpenAI resource endpoint valueopenai.api_version = "2023-05 - 15" openai.api_key = os.getenv("OPENAI_API_KEY")# Define a function to create prompt from system messages and chat messages def create_prompt(system_message, messages): prompt = system_message for message in messages: prompt + = f return "\n< |im_start|>{message['sender']}\n{message['text']}\n<|im_end|>" += "\n<|im_start|>assistant\n " prompt# Define user inputs and systemmessages_input = "" system_message = f"<|im_start|>system\n{''}\n<|im_end|>"# Create message list to track chat messages = [{"sender": "user", "text": user_input}]response = openai.Completion.create( engine=" gpt - 35 -turbo", # The implementation name you chose when implementing the ChatGPT model. prompt=create_prompt(system_message, messages), Temperature=0.5, max_tokens=250, top_p=0.9, Frequency_penalty=0, present_penalty=0, stop = [ '<|im_end|>'])messages.append ({"sender": "assistant", "text": Reply['choice'][0]['text']})print(reply['choice ' ] [0]['text'])

It stays under the token limit

The easiest approach to staying under the token limit is to remove the oldest messages in the conversation when you reach the token limit.

You can choose to always include as many tokens as possible while staying under the limit, or you can always include a set number of previous messages provided those messages stay within the limit. It's important to remember that longer messages take longer to generate a response and have a higher cost than shorter messages.

You can calculate the number of tokens in a string usingTick ​​Thank youPython library as shown below.

tiktoken cl100k_base = tiktoken.get_encoding("cl100k_base") enc = tiktoken.Encoding( name="gpt-35-turbo", pat_str=cl100k_base._pat_str, mergeable_ranks=cl100k_">mergeable_ranks=cl100k_">Künstler. 100264, „< |im_end|>": 100265 } ) tokens = enc.encode( "<|im_start|>bruger\nHej<|im_end|><|im_start|>βοηθός",allow_special= {"<|im_start|> ", "< |im_end|>"} ) Assert len(tokens) == 7 Assert Tokens == [100264, 882, 198, 9906, 100265, 100264, 78191]

Next Step

• Learn more about Azure OpenAI.
• Start with the ChatGPT modelChatGPT quick start.
• For more examples seeAzure OpenAI samples GitHub repository