20 Dec 2023
12215字
41分
CC BY 4.0
（除特别声明或转载文章外）
如果这篇博客帮助到你，可以请我喝一杯咖啡~
Prompt 手册(openai官方系列)
这里的 prompt是针对 GPT4 才能达到良好效果，示例采用英文是因为模型问题，换成中文不保证能达到同样的效果。
一，六步获得更好的结果
给予清楚的命令
如果觉得输出太长，让他给出简短的答案，如果给出的答案太简单，让他输出专家级别的回答，如果你不喜欢输出格式，给你喜欢的格式，别让模型天天猜你你想要啥，多给约束
策略：
- 给予细节
- 采用角色
- 清楚划分输入的不同部分
- 指定完成任务需要步骤
- 提供列子
- 指定输出长度
提供参考文本
LLM有幻觉问题，尤其针对复杂问题或者需要引用或 Urls
策略：
分解复杂任务
把复杂任务分解成多个简单任务，让模型输出结果，然后把结果拼接起来(工作流)
策略：
- 根据意图分类找到最相关的指令
- 长对话需要总结或过滤之前的内容
- 总结长文档并递归重构
给模型时间去思考
COT
策略：
- 在结论前指导
- 在内部隐藏推理过程
- 询问是否错过或者忽视了之前的某些细节
使用额外的工具
通过把模型的输出和外部工具连接获得更好的效果。比如 RAG
策略：
- 用基于嵌入的搜索来高效知识检索
- 调用额外的API或代码计算加强
- 给予模型权限去控制特殊功能函数
系统的测试
为了防止prompt只针对特定例子有良好效果需要制作通用测试
策略：
详细示例
给予清楚的命令
1. 给定细节
| 负面例子 |
正面例子 |
| How do I add numbers in Excel? |
How do I add up a row of dollar amounts in Excel? I want to do this automatically for a whole sheet of rows with all the totals ending up on the right in a column called “Total”. |
| Who’s president? |
Who was the president of Mexico in 2021, and how frequently are elections held? |
| Write code to calculate the Fibonacci sequence. |
Write a TypeScript function to efficiently calculate the Fibonacci sequence. Comment the code liberally to explain what each piece does and why it’s written that way. |
| Summarize the meeting notes. |
Summarize the meeting notes in a single paragraph. Then write a markdown list of the speakers and each of their key points. Finally, list the next steps or action items suggested by the speakers, if any. |
2. 采用角色
| SYSTEM （系统） |
When I ask for help to write something, you will reply with a document that contains at least one joke or playful comment in every paragraph. |
| USER （用户） |
Write a thank you note to my steel bolt vendor for getting the delivery in on time and in short notice. This made it possible for us to deliver an important order. |
3. 清楚划分输入的不同部分
| USER （用户） |
Summarize the text delimited by triple quotes with a haiku.
"""insert text here""" |
在 playground 中尝试
| SYSTEM （系统） |
You will be provided with a pair of articles (delimited with XML tags) about the same topic. First summarize the arguments of each article. Then indicate which of them makes a better argument and explain why. |
| USER （用户） |
< article> insert first article here < /article>
< article> insert second article here < /article> |
在 playground 中尝试
| SYSTEM （系统） |
You will be provided with a thesis abstract and a suggested title for it. The thesis title should give the reader a good idea of the topic of the thesis but should also be eye-catching. If the title does not meet these criteria, suggest 5 alternatives. |
| USER （用户） |
Abstract: insert abstract here
Title: insert title here |
4. 指定完成任务需要步骤
| SYSTEM （系统） |
Use the following step-by-step instructions to respond to user inputs.
Step 1 - The user will provide you with text in triple quotes. Summarize this text in one sentence with a prefix that says "Summary: ".
Step 2 - Translate the summary from Step 1 into Spanish, with a prefix that says "Translation: ".
|
| USER （用户） |
"""insert text here""" |
5. 提供例子
| SYSTEM （系统） |
Answer in a consistent style.
|
| USER （用户） |
Teach me about patience. |
| Assistant（辅助） |
The river that carves the deepest valley flows from a modest spring; the grandest symphony originates from a single note; the most intricate tapestry begins with a solitary thread. |
| USER （用户） |
Teach me about the ocean. |
6. 限制长度
| USER （用户） |
Summarize the text delimited by triple quotes in about 50 words.
"""insert text here""" |
| USER （用户） |
Summarize the text delimited by triple quotes in 2 paragraphs.
"""insert text here""" |
| USER （用户） |
Summarize the text delimited by triple quotes in 3 bullet points.
"""insert text here""" |
提供参考文本
1. 用参考文本回答
| SYSTEM （系统） |
Use the provided articles delimited by triple quotes to answer questions. If the answer cannot be found in the articles, write "I could not find an answer."
|
| USER （用户） |
<insert articles, each delimited by triple quotes>
Question: < insert question here>
|
2. 同时引用参考文本内容
| SYSTEM （系统） |
You will be provided with a document delimited by triple quotes and a question. Your task is to answer the question using only the provided document and to cite the passage(s) of the document used to answer the question. If the document does not contain the information needed to answer this question then simply write: "Insufficient information." If an answer to the question is provided, it must be annotated with a citation. Use the following format for to cite relevant passages ({"citation": …}).
|
| USER （用户） |
"""< insert document here>"""
Question: < insert question here>
|
划分子任务
1. 根据意图分类找到最相关的指令
| SYSTEM （系统） |
You will be provided with customer service queries. Classify each query into a primary category and a secondary category. Provide your output in json format with the keys: primary and secondary.
Primary categories: Billing, Technical Support, Account Management, or General Inquiry.
Billing secondary categories:
- Unsubscribe or upgrade
- Add a payment method
- Explanation for charge
- Dispute a charge
Technical Support secondary categories:
- Troubleshooting
- Device compatibility
- Software updates
Account Management secondary categories:
- Password reset
- Update personal information
- Close account
- Account security
General Inquiry secondary categories:
- Product information
- Pricing
- Feedback
- Speak to a human
|
| USER （用户） |
I need to get my internet working again.
|
故障排除
| SYSTEM （系统） |
You will be provided with customer service inquiries that require troubleshooting in a technical support context. Help the user by:
- Ask them to check that all cables to/from the router are connected. Note that it is common for cables to come loose over time.
- If all cables are connected and the issue persists, ask them which router model they are using
- Now you will advise them how to restart their device:
-- If the model number is MTD-327J, advise them to push the red button and hold it for 5 seconds, then wait 5 minutes before testing the connection.
-- If the model number is MTD-327S, advise them to unplug and replug it, then wait 5 minutes before testing the connection.
- If the customer's issue persists after restarting the device and waiting 5 minutes, connect them to IT support by outputting {"IT support requested"}.
- If the user starts asking questions that are unrelated to this topic then confirm if they would like to end the current chat about troubleshooting and classify their request according to the following scheme:
<insert primary/secondary classification scheme from above here>
|
| USER （用户） |
I need to get my internet working again.
|
2. 总结或过滤对话
- 就一但对话长度达到某个阈值，触发总结
- 后台异步总结
- 动态筛选之前相关的部分
3.总结重构
思考
1. 指定方案
以评估学生数学解法对错为例
| SYSTEM （系统） |
Determine if the student's solution is correct or not.
|
| USER （用户） |
Problem Statement: I'm building a solar power installation and I need help working out the financials.
- Land costs $100 / square foot
- I can buy solar panels for $250 / square foot
- I negotiated a contract for maintenance that will cost me a flat $100k per year, and an additional $10 / square foot
What is the total cost for the first year of operations as a function of the number of square feet.
Student's Solution: Let x be the size of the installation in square feet.
1. Land cost: 100x
2. Solar panel cost: 250x
3. Maintenance cost: 100,000 + 100x
Total cost: 100x + 250x + 100,000 + 100x = 450x + 100,000 |
| Assistant（辅助） |
The student's solution is correct. |
可以看到解法错误，通过下面方式先生成自己的解法
| SYSTEM （系统） |
First work out your own solution to the problem. Then compare your solution to the student's solution and evaluate if the student's solution is correct or not. Don't decide if the student's solution is correct until you have done the problem yourself.
|
| USER （用户） |
Problem Statement: I'm building a solar power installation and I need help working out the financials.
- Land costs $100 / square foot
- I can buy solar panels for $250 / square foot
- I negotiated a contract for maintenance that will cost me a flat $100k per year, and an additional $10 / square foot
What is the total cost for the first year of operations as a function of the number of square feet.
Student's Solution: Let x be the size of the installation in square feet.
1. Land cost: 100x
2. Solar panel cost: 250x
3. Maintenance cost: 100,000 + 100x
Total cost: 100x + 250x + 100,000 + 100x = 450x + 100,000 |
| Assistant（辅助） |
Let x be the size of the installation in square feet.
1. Land cost: 100x
2. Solar panel cost: 250x
3. Maintenance cost: 100,000 + 10x
Total cost: 100x + 250x + 100,000 + 10x = 360x + 100,000
The student's solution is incorrect. They made an error in the maintenance cost calculation, using 100x instead of 10x. The correct total cost for the first year of operations is 360x + 100,000. |
2. 隐藏中间结果
| SYSTEM （系统） |
Follow these steps to answer the user queries.
Step 1 - First work out your own solution to the problem. Don't rely on the student's solution since it may be incorrect. Enclose all your work for this step within triple quotes (""").
Step 2 - Compare your solution to the student's solution and evaluate if the student's solution is correct or not. Enclose all your work for this step within triple quotes (""").
Step 3 - If the student made a mistake, determine what hint you could give the student without giving away the answer. Enclose all your work for this step within triple quotes (""").
Step 4 - If the student made a mistake, provide the hint from the previous step to the student (outside of triple quotes). Instead of writing "Step 4 - ..." write "Hint:".
|
| USER （用户） |
Problem Statement: < insert problem statement>
Student Solution: < insert student solution>
|
或者通过一系列询问达到隐藏效果
3.询问遗忘或丢失
| SYSTEM （系统） |
You will be provided with a document delimited by triple quotes. Your task is to select excerpts which pertain to the following question: "What significant paradigm shifts have occurred in the history of artificial intelligence."
Ensure that excerpts contain all relevant context needed to interpret them - in other words don't extract small snippets that are missing important context. Provide output in JSON format as follows:
[{"excerpt": "..."},
...
{"excerpt": "..."}]
|
| USER （用户） |
"""< insert document here>""" |
| Assistant（辅助） |
[{"excerpt": "the model writes an excerpt here"},
...
{"excerpt": "the model writes another excerpt here"}] |
| USER （用户） |
Are there more relevant excerpts? Take care not to repeat excerpts. Also ensure that excerpts contain all relevant context needed to interpret them - in other words don't extract small snippets that are missing important context. |
使用工具
1.文本嵌入向量来高效检索
2.执行代码或者调用API
| SYSTEM （系统） |
You can write and execute Python code by enclosing it in triple backticks, e.g. ```code goes here```. Use this to perform calculations.
|
| USER （用户） |
Find all real-valued roots of the following polynomial: 3*x**5 - 5*x**4 - 3*x**3 - 7*x - 10.
|
或者
| SYSTEM （系统） |
You can write and execute Python code by enclosing it in triple backticks. Also note that you have access to the following module to help users send messages to their friends:
```python
import message
message.write(to="John", message="Hey, want to meetup after work?")
```
|
注意安全性问题
3. 给予模型权限
系统测试
好的评估
| 检测不同 |
95%的置信区间的采样大小 |
| 30% |
-10 |
| 10% |
-100 |
| 3% |
-1000 |
| 1% |
-10000 |
1.带参考答案评估
| SYSTEM （系统） |
You will be provided with text delimited by triple quotes that is supposed to be the answer to a question. Check if the following pieces of information are directly contained in the answer:
- Neil Armstrong was the first person to walk on the moon.
- The date Neil Armstrong first walked on the moon was July 21, 1969.
For each of these points perform the following steps:
1 - Restate the point.
2 - Provide a citation from the answer which is closest to this point.
3 - Consider if someone reading the citation who doesn't know the topic could directly infer the point. Explain why or why not before making up your mind.
4 - Write "yes" if the answer to 3 was yes, otherwise write "no".
Finally, provide a count of how many "yes" answers there are. Provide this count as {"count": }.</td>
</tr>
</table>
- 两个都满足的例子
| SYSTEM （系统） |
< insert system message above>
|
| USER （用户） |
"""Neil Armstrong is famous for being the first human to set foot on the Moon. This historic event took place on July 21, 1969, during the Apollo 11 mission."""
|
- 只满足一点的
| SYSTEM （系统） |
< insert system message above>
|
| USER （用户） |
"""Neil Armstrong made history when he stepped off the lunar module, becoming the first person to walk on the moon."""
|
- 都不满足的
| SYSTEM （系统） |
< insert system message above>
|
| USER （用户） |
"""In the summer of '69, a voyage grand,
Apollo 11, bold as legend's hand.
Armstrong took a step, history unfurled,
"One small step," he said, for a new world."""
|
- **变体**
| SYSTEM （系统） |
Use the following steps to respond to user inputs. Fully restate each step before proceeding. i.e. "Step 1: Reason...".
Step 1: Reason step-by-step about whether the information in the submitted answer compared to the expert answer is either: disjoint, equal, a subset, a superset, or overlapping (i.e. some intersection but not subset/superset).
Step 2: Reason step-by-step about whether the submitted answer contradicts any aspect of the expert answer.
Step 3: Output a JSON object structured like: {"type_of_overlap": "disjoint" or "equal" or "subset" or "superset" or "overlapping", "contradiction": true or false}
|
- 不合格的答案
| SYSTEM （系统） |
< insert system message above>
|
| USER （用户） |
Question: """What event is Neil Armstrong most famous for and on what date did it occur? Assume UTC time."""
Submitted Answer: """Didn't he walk on the moon or something?"""
Expert Answer: """Neil Armstrong is most famous for being the first person to walk on the moon. This historic event occurred on July 21, 1969."""
|
- 与专家矛盾的答案
| SYSTEM （系统） |
< insert system message above>
|
| USER （用户） |
Question: """What event is Neil Armstrong most famous for and on what date did it occur? Assume UTC time."""
Submitted Answer: """On the 21st of July 1969, Neil Armstrong became the second person to walk on the moon, following after Buzz Aldrin."""
Expert Answer: """Neil Armstrong is most famous for being the first person to walk on the moon. This historic event occurred on July 21, 1969."""
|
- 带细节的正确答案
| SYSTEM （系统） |
< insert system message above>
|
| USER （用户） |
Question: """What event is Neil Armstrong most famous for and on what date did it occur? Assume UTC time."""
Submitted Answer: """At approximately 02:56 UTC on July 21st 1969, Neil Armstrong became the first human to set foot on the lunar surface, marking a monumental achievement in human history."""
Expert Answer: """Neil Armstrong is most famous for being the first person to walk on the moon. This historic event occurred on July 21, 1969."""
|
|
