merge frontier branch (#1620)
* Zhipu sdk update 适配最新的智谱SDK，支持GLM4v (#1502) * 适配 google gemini 优化为从用户input中提取文件 * 适配最新的智谱SDK、支持glm-4v * requirements.txt fix * pending history check --------- Co-authored-by: binary-husky <qingxu.fu@outlook.com> * Update "生成多种Mermaid图表" plugin: Separate out the file reading function (#1520) * Update crazy_functional.py with new functionality deal with PDF * Update crazy_functional.py and Mermaid.py for plugin_kwargs * Update crazy_functional.py with new chart type: mind map * Update SELECT_PROMPT and i_say_show_user messages * Update ArgsReminder message in get_crazy_functions() function * Update with read md file and update PROMPTS * Return the PROMPTS as the test found that the initial version worked best * Update Mermaid chart generation function * version 3.71 * 解决issues #1510 * Remove unnecessary text from sys_prompt in 解析历史输入 function * Remove sys_prompt message in 解析历史输入 function * Update bridge_all.py: supports gpt-4-turbo-preview (#1517) * Update bridge_all.py: supports gpt-4-turbo-preview supports gpt-4-turbo-preview * Update bridge_all.py --------- Co-authored-by: binary-husky <96192199+binary-husky@users.noreply.github.com> * Update config.py: supports gpt-4-turbo-preview (#1516) * Update config.py: supports gpt-4-turbo-preview supports gpt-4-turbo-preview * Update config.py --------- Co-authored-by: binary-husky <96192199+binary-husky@users.noreply.github.com> * Refactor 解析历史输入 function to handle file input * Update Mermaid chart generation functionality * rename files and functions --------- Co-authored-by: binary-husky <qingxu.fu@outlook.com> Co-authored-by: hongyi-zhao <hongyi.zhao@gmail.com> Co-authored-by: binary-husky <96192199+binary-husky@users.noreply.github.com> * 接入mathpix ocr功能 (#1468) * Update Latex输出PDF结果.py 借助mathpix实现了PDF翻译中文并重新编译PDF * Update config.py add mathpix appid & appkey * Add 'PDF翻译中文并重新编译PDF' feature to plugins. --------- Co-authored-by: binary-husky <96192199+binary-husky@users.noreply.github.com> * fix zhipuai * check picture * remove glm-4 due to bug * 修改config * 检查MATHPIX_APPID * Remove unnecessary code and update function_plugins dictionary * capture non-standard token overflow * bug fix #1524 * change mermaid style * 支持mermaid 滚动放大缩小重置,鼠标滚动和拖拽 (#1530) * 支持mermaid 滚动放大缩小重置,鼠标滚动和拖拽 * 微调未果 先stage一下 * update --------- Co-authored-by: binary-husky <qingxu.fu@outlook.com> Co-authored-by: binary-husky <96192199+binary-husky@users.noreply.github.com> * ver 3.72 * change live2d * save the status of ``clear btn` in cookie * 前端选择保持 * js ui bug fix * reset btn bug fix * update live2d tips * fix missing get_token_num method * fix live2d toggle switch * fix persistent custom btn with cookie * fix zhipuai feedback with core functionality * Refactor button update and clean up functions * tailing space removal * Fix missing MATHPIX_APPID and MATHPIX_APPKEY configuration * Prompt fix、脑图提示词优化 (#1537) * 适配 google gemini 优化为从用户input中提取文件 * 脑图提示词优化 * Fix missing MATHPIX_APPID and MATHPIX_APPKEY configuration --------- Co-authored-by: binary-husky <qingxu.fu@outlook.com> * 优化“PDF翻译中文并重新编译PDF”插件 (#1602) * Add gemini_endpoint to API_URL_REDIRECT (#1560) * Add gemini_endpoint to API_URL_REDIRECT * Update gemini-pro and gemini-pro-vision model_info endpoints * Update to support new claude models (#1606) * Add anthropic library and update claude models * 更新bridge_claude.py文件，添加了对图片输入的支持。修复了一些bug。 * 添加Claude_3_Models变量以限制图片数量 * Refactor code to improve readability and maintainability * minor claude bug fix * more flexible one-api support * reformat config * fix one-api new access bug * dummy * compat non-standard api * version 3.73 --------- Co-authored-by: XIao <46100050+Kilig947@users.noreply.github.com> Co-authored-by: Menghuan1918 <menghuan2003@outlook.com> Co-authored-by: hongyi-zhao <hongyi.zhao@gmail.com> Co-authored-by: Hao Ma <893017927@qq.com> Co-authored-by: zeyuan huang <599012428@qq.com>
This commit is contained in:
binary-husky
@@ -11,13 +11,12 @@
|
||||
"""
|
||||
|
||||
import os
|
||||
import json
|
||||
import time
|
||||
import gradio as gr
|
||||
import logging
|
||||
import traceback
|
||||
import requests
|
||||
import importlib
|
||||
from toolbox import get_conf, update_ui, trimmed_format_exc, encode_image, every_image_file_in_path
|
||||
|
||||
picture_system_prompt = "\n当回复图像时,必须说明正在回复哪张图像。所有图像仅在最后一个问题中提供,即使它们在历史记录中被提及。请使用'这是第X张图像:'的格式来指明您正在描述的是哪张图像。"
|
||||
Claude_3_Models = ["claude-3-sonnet-20240229", "claude-3-opus-20240229"]
|
||||
|
||||
# config_private.py放自己的秘密如API和代理网址
|
||||
# 读取时首先看是否存在私密的config_private配置文件（不受git管控），如果有，则覆盖原config文件
|
||||
@@ -56,7 +55,8 @@ def predict_no_ui_long_connection(inputs, llm_kwargs, history=[], sys_prompt="",
|
||||
"""
|
||||
from anthropic import Anthropic
|
||||
watch_dog_patience = 5 # 看门狗的耐心, 设置5秒即可
|
||||
prompt = generate_payload(inputs, llm_kwargs, history, system_prompt=sys_prompt, stream=True)
|
||||
if inputs == "": inputs = "空空如也的输入栏"
|
||||
message = generate_payload(inputs, llm_kwargs, history, stream=True, image_paths=None)
|
||||
retry = 0
|
||||
if len(ANTHROPIC_API_KEY) == 0:
|
||||
raise RuntimeError("没有设置ANTHROPIC_API_KEY选项")
|
||||
@@ -65,15 +65,16 @@ def predict_no_ui_long_connection(inputs, llm_kwargs, history=[], sys_prompt="",
|
||||
try:
|
||||
# make a POST request to the API endpoint, stream=False
|
||||
from .bridge_all import model_info
|
||||
anthropic = Anthropic(api_key=ANTHROPIC_API_KEY)
|
||||
anthropic = Anthropic(api_key=ANTHROPIC_API_KEY, base_url=model_info[llm_kwargs['llm_model']]['endpoint'])
|
||||
# endpoint = model_info[llm_kwargs['llm_model']]['endpoint']
|
||||
# with ProxyNetworkActivate()
|
||||
stream = anthropic.completions.create(
|
||||
prompt=prompt,
|
||||
max_tokens_to_sample=4096, # The maximum number of tokens to generate before stopping.
|
||||
stream = anthropic.messages.create(
|
||||
messages=message,
|
||||
max_tokens=4096, # The maximum number of tokens to generate before stopping.
|
||||
model=llm_kwargs['llm_model'],
|
||||
stream=True,
|
||||
temperature = llm_kwargs['temperature']
|
||||
temperature = llm_kwargs['temperature'],
|
||||
system=sys_prompt
|
||||
)
|
||||
break
|
||||
except Exception as e:
|
||||
@@ -82,15 +83,19 @@ def predict_no_ui_long_connection(inputs, llm_kwargs, history=[], sys_prompt="",
|
||||
if retry > MAX_RETRY: raise TimeoutError
|
||||
if MAX_RETRY!=0: print(f'请求超时，正在重试 ({retry}/{MAX_RETRY}) ……')
|
||||
result = ''
|
||||
try:
|
||||
try:
|
||||
for completion in stream:
|
||||
result += completion.completion
|
||||
if not console_slience: print(completion.completion, end='')
|
||||
if observe_window is not None:
|
||||
if completion.type == "message_start" or completion.type == "content_block_start":
|
||||
continue
|
||||
elif completion.type == "message_stop" or completion.type == "content_block_stop" or completion.type == "message_delta":
|
||||
break
|
||||
result += completion.delta.text
|
||||
if not console_slience: print(completion.delta.text, end='')
|
||||
if observe_window is not None:
|
||||
# 观测窗，把已经获取的数据显示出去
|
||||
if len(observe_window) >= 1: observe_window[0] += completion.completion
|
||||
if len(observe_window) >= 1: observe_window[0] += completion.delta.text
|
||||
# 看门狗，如果超过期限没有喂狗，则终止
|
||||
if len(observe_window) >= 2:
|
||||
if len(observe_window) >= 2:
|
||||
if (time.time()-observe_window[1]) > watch_dog_patience:
|
||||
raise RuntimeError("用户取消了程序。")
|
||||
except Exception as e:
|
||||
@@ -98,6 +103,10 @@ def predict_no_ui_long_connection(inputs, llm_kwargs, history=[], sys_prompt="",
|
||||
|
||||
return result
|
||||
|
||||
def make_media_input(history,inputs,image_paths):
|
||||
for image_path in image_paths:
|
||||
inputs = inputs + f'<br/><br/><div align="center"><img src="file={os.path.abspath(image_path)}"></div>'
|
||||
return inputs
|
||||
|
||||
def predict(inputs, llm_kwargs, plugin_kwargs, chatbot, history=[], system_prompt='', stream = True, additional_fn=None):
|
||||
"""
|
||||
@@ -109,23 +118,34 @@ def predict(inputs, llm_kwargs, plugin_kwargs, chatbot, history=[], system_promp
|
||||
chatbot 为WebUI中显示的对话列表，修改它，然后yeild出去，可以直接修改对话界面内容
|
||||
additional_fn代表点击的哪个按钮，按钮见functional.py
|
||||
"""
|
||||
if inputs == "": inputs = "空空如也的输入栏"
|
||||
from anthropic import Anthropic
|
||||
if len(ANTHROPIC_API_KEY) == 0:
|
||||
chatbot.append((inputs, "没有设置ANTHROPIC_API_KEY"))
|
||||
yield from update_ui(chatbot=chatbot, history=history, msg="等待响应") # 刷新界面
|
||||
return
|
||||
|
||||
|
||||
if additional_fn is not None:
|
||||
from core_functional import handle_core_functionality
|
||||
inputs, history = handle_core_functionality(additional_fn, inputs, history, chatbot)
|
||||
|
||||
raw_input = inputs
|
||||
logging.info(f'[raw_input] {raw_input}')
|
||||
chatbot.append((inputs, ""))
|
||||
yield from update_ui(chatbot=chatbot, history=history, msg="等待响应") # 刷新界面
|
||||
have_recent_file, image_paths = every_image_file_in_path(chatbot)
|
||||
if len(image_paths) > 20:
|
||||
chatbot.append((inputs, "图片数量超过api上限(20张)"))
|
||||
yield from update_ui(chatbot=chatbot, history=history, msg="等待响应")
|
||||
return
|
||||
|
||||
if any([llm_kwargs['llm_model'] == model for model in Claude_3_Models]) and have_recent_file:
|
||||
if inputs == "" or inputs == "空空如也的输入栏": inputs = "请描述给出的图片"
|
||||
system_prompt += picture_system_prompt # 由于没有单独的参数保存包含图片的历史，所以只能通过提示词对第几张图片进行定位
|
||||
chatbot.append((make_media_input(history,inputs, image_paths), ""))
|
||||
yield from update_ui(chatbot=chatbot, history=history, msg="等待响应") # 刷新界面
|
||||
else:
|
||||
chatbot.append((inputs, ""))
|
||||
yield from update_ui(chatbot=chatbot, history=history, msg="等待响应") # 刷新界面
|
||||
|
||||
try:
|
||||
prompt = generate_payload(inputs, llm_kwargs, history, system_prompt, stream)
|
||||
message = generate_payload(inputs, llm_kwargs, history, stream, image_paths)
|
||||
except RuntimeError as e:
|
||||
chatbot[-1] = (inputs, f"您提供的api-key不满足要求，不包含任何可用于{llm_kwargs['llm_model']}的api-key。您可能选择了错误的模型或请求源。")
|
||||
yield from update_ui(chatbot=chatbot, history=history, msg="api-key不满足要求") # 刷新界面
|
||||
@@ -138,17 +158,17 @@ def predict(inputs, llm_kwargs, plugin_kwargs, chatbot, history=[], system_promp
|
||||
try:
|
||||
# make a POST request to the API endpoint, stream=True
|
||||
from .bridge_all import model_info
|
||||
anthropic = Anthropic(api_key=ANTHROPIC_API_KEY)
|
||||
anthropic = Anthropic(api_key=ANTHROPIC_API_KEY, base_url=model_info[llm_kwargs['llm_model']]['endpoint'])
|
||||
# endpoint = model_info[llm_kwargs['llm_model']]['endpoint']
|
||||
# with ProxyNetworkActivate()
|
||||
stream = anthropic.completions.create(
|
||||
prompt=prompt,
|
||||
max_tokens_to_sample=4096, # The maximum number of tokens to generate before stopping.
|
||||
stream = anthropic.messages.create(
|
||||
messages=message,
|
||||
max_tokens=4096, # The maximum number of tokens to generate before stopping.
|
||||
model=llm_kwargs['llm_model'],
|
||||
stream=True,
|
||||
temperature = llm_kwargs['temperature']
|
||||
temperature = llm_kwargs['temperature'],
|
||||
system=system_prompt
|
||||
)
|
||||
|
||||
break
|
||||
except:
|
||||
retry += 1
|
||||
@@ -158,10 +178,14 @@ def predict(inputs, llm_kwargs, plugin_kwargs, chatbot, history=[], system_promp
|
||||
if retry > MAX_RETRY: raise TimeoutError
|
||||
|
||||
gpt_replying_buffer = ""
|
||||
|
||||
|
||||
for completion in stream:
|
||||
if completion.type == "message_start" or completion.type == "content_block_start":
|
||||
continue
|
||||
elif completion.type == "message_stop" or completion.type == "content_block_stop" or completion.type == "message_delta":
|
||||
break
|
||||
try:
|
||||
gpt_replying_buffer = gpt_replying_buffer + completion.completion
|
||||
gpt_replying_buffer = gpt_replying_buffer + completion.delta.text
|
||||
history[-1] = gpt_replying_buffer
|
||||
chatbot[-1] = (history[-2], history[-1])
|
||||
yield from update_ui(chatbot=chatbot, history=history, msg='正常') # 刷新界面
|
||||
@@ -172,57 +196,52 @@ def predict(inputs, llm_kwargs, plugin_kwargs, chatbot, history=[], system_promp
|
||||
chatbot[-1] = (chatbot[-1][0], f"[Local Message] 异常 \n\n{tb_str}")
|
||||
yield from update_ui(chatbot=chatbot, history=history, msg="Json异常" + tb_str) # 刷新界面
|
||||
return
|
||||
|
||||
|
||||
|
||||
|
||||
# https://github.com/jtsang4/claude-to-chatgpt/blob/main/claude_to_chatgpt/adapter.py
|
||||
def convert_messages_to_prompt(messages):
|
||||
prompt = ""
|
||||
role_map = {
|
||||
"system": "Human",
|
||||
"user": "Human",
|
||||
"assistant": "Assistant",
|
||||
}
|
||||
for message in messages:
|
||||
role = message["role"]
|
||||
content = message["content"]
|
||||
transformed_role = role_map[role]
|
||||
prompt += f"\n\n{transformed_role.capitalize()}: {content}"
|
||||
prompt += "\n\nAssistant: "
|
||||
return prompt
|
||||
|
||||
def generate_payload(inputs, llm_kwargs, history, system_prompt, stream):
|
||||
def generate_payload(inputs, llm_kwargs, history, stream, image_paths):
|
||||
"""
|
||||
整合所有信息，选择LLM模型，生成http请求，为发送请求做准备
|
||||
"""
|
||||
from anthropic import Anthropic, HUMAN_PROMPT, AI_PROMPT
|
||||
|
||||
conversation_cnt = len(history) // 2
|
||||
|
||||
messages = [{"role": "system", "content": system_prompt}]
|
||||
messages = []
|
||||
|
||||
if conversation_cnt:
|
||||
for index in range(0, 2*conversation_cnt, 2):
|
||||
what_i_have_asked = {}
|
||||
what_i_have_asked["role"] = "user"
|
||||
what_i_have_asked["content"] = history[index]
|
||||
what_i_have_asked["content"] = [{"type": "text", "text": history[index]}]
|
||||
what_gpt_answer = {}
|
||||
what_gpt_answer["role"] = "assistant"
|
||||
what_gpt_answer["content"] = history[index+1]
|
||||
if what_i_have_asked["content"] != "":
|
||||
if what_gpt_answer["content"] == "": continue
|
||||
if what_gpt_answer["content"] == timeout_bot_msg: continue
|
||||
what_gpt_answer["content"] = [{"type": "text", "text": history[index+1]}]
|
||||
if what_i_have_asked["content"][0]["text"] != "":
|
||||
if what_i_have_asked["content"][0]["text"] == "": continue
|
||||
if what_i_have_asked["content"][0]["text"] == timeout_bot_msg: continue
|
||||
messages.append(what_i_have_asked)
|
||||
messages.append(what_gpt_answer)
|
||||
else:
|
||||
messages[-1]['content'] = what_gpt_answer['content']
|
||||
messages[-1]['content'][0]['text'] = what_gpt_answer['content'][0]['text']
|
||||
|
||||
what_i_ask_now = {}
|
||||
what_i_ask_now["role"] = "user"
|
||||
what_i_ask_now["content"] = inputs
|
||||
if any([llm_kwargs['llm_model'] == model for model in Claude_3_Models]) and image_paths:
|
||||
base64_images = []
|
||||
for image_path in image_paths:
|
||||
base64_images.append(encode_image(image_path))
|
||||
what_i_ask_now = {}
|
||||
what_i_ask_now["role"] = "user"
|
||||
what_i_ask_now["content"] = []
|
||||
for base64_image in base64_images:
|
||||
what_i_ask_now["content"].append({
|
||||
"type": "image",
|
||||
"source": {
|
||||
"type": "base64",
|
||||
"media_type": "image/jpeg",
|
||||
"data": base64_image,
|
||||
}
|
||||
})
|
||||
what_i_ask_now["content"].append({"type": "text", "text": inputs})
|
||||
else:
|
||||
what_i_ask_now = {}
|
||||
what_i_ask_now["role"] = "user"
|
||||
what_i_ask_now["content"] = [{"type": "text", "text": inputs}]
|
||||
messages.append(what_i_ask_now)
|
||||
prompt = convert_messages_to_prompt(messages)
|
||||
|
||||
return prompt
|
||||
|
||||
|
||||
return messages
|
||||
Reference in New Issue
Block a user