diff --git a/README.md b/README.md index ceeea7c0..3167cf9a 100644 --- a/README.md +++ b/README.md @@ -40,7 +40,7 @@ To translate this project to arbitrary language with GPT, read and run [`multi_l 功能（⭐= 近期新增功能） | 描述 --- | --- -⭐[接入新模型](https://github.com/binary-husky/gpt_academic/wiki/%E5%A6%82%E4%BD%95%E5%88%87%E6%8D%A2%E6%A8%A1%E5%9E%8B)！ | 百度[千帆](https://cloud.baidu.com/doc/WENXINWORKSHOP/s/Nlks5zkzu)与文心一言, [通义千问](https://modelscope.cn/models/qwen/Qwen-7B-Chat/summary)，上海AI-Lab[书生](https://github.com/InternLM/InternLM)，讯飞[星火](https://xinghuo.xfyun.cn/)，[LLaMa2](https://huggingface.co/meta-llama/Llama-2-7b-chat-hf)，智谱API，DALLE3 +⭐[接入新模型](https://github.com/binary-husky/gpt_academic/wiki/%E5%A6%82%E4%BD%95%E5%88%87%E6%8D%A2%E6%A8%A1%E5%9E%8B)！ | 百度[千帆](https://cloud.baidu.com/doc/WENXINWORKSHOP/s/Nlks5zkzu)与文心一言, 通义千问[Qwen](https://modelscope.cn/models/qwen/Qwen-7B-Chat/summary)，上海AI-Lab[书生](https://github.com/InternLM/InternLM)，讯飞[星火](https://xinghuo.xfyun.cn/)，[LLaMa2](https://huggingface.co/meta-llama/Llama-2-7b-chat-hf)，[智谱API](https://open.bigmodel.cn/)，DALLE3, [DeepseekCoder](https://coder.deepseek.com/) 润色、翻译、代码解释 | 一键润色、翻译、查找论文语法错误、解释代码 [自定义快捷键](https://www.bilibili.com/video/BV14s4y1E7jN) | 支持自定义快捷键 模块化设计 | 支持自定义强大的[插件](https://github.com/binary-husky/gpt_academic/tree/master/crazy_functions)，插件支持[热更新](https://github.com/binary-husky/gpt_academic/wiki/%E5%87%BD%E6%95%B0%E6%8F%92%E4%BB%B6%E6%8C%87%E5%8D%97) @@ -104,36 +104,38 @@ Latex论文一键校对 | [插件] 仿Grammarly对Latex文章进行语法、拼 ### 安装方法I：直接运行 (Windows, Linux or MacOS) 1. 下载项目 -```sh -git clone --depth=1 https://github.com/binary-husky/gpt_academic.git -cd gpt_academic -``` + + ```sh + git clone --depth=1 https://github.com/binary-husky/gpt_academic.git + cd gpt_academic + ``` 2. 配置API_KEY -在`config.py`中，配置API KEY等设置，[点击查看特殊网络环境设置方法](https://github.com/binary-husky/gpt_academic/issues/1) 。[Wiki页面](https://github.com/binary-husky/gpt_academic/wiki/项目配置说明)。 + 在`config.py`中，配置API KEY等设置，[点击查看特殊网络环境设置方法](https://github.com/binary-husky/gpt_academic/issues/1) 。[Wiki页面](https://github.com/binary-husky/gpt_academic/wiki/项目配置说明)。 -「 程序会优先检查是否存在名为`config_private.py`的私密配置文件，并用其中的配置覆盖`config.py`的同名配置。如您能理解该读取逻辑，我们强烈建议您在`config.py`旁边创建一个名为`config_private.py`的新配置文件，并把`config.py`中的配置转移（复制）到`config_private.py`中（仅复制您修改过的配置条目即可）。 」 + 「 程序会优先检查是否存在名为`config_private.py`的私密配置文件，并用其中的配置覆盖`config.py`的同名配置。如您能理解该读取逻辑，我们强烈建议您在`config.py`旁边创建一个名为`config_private.py`的新配置文件，并把`config.py`中的配置转移（复制）到`config_private.py`中（仅复制您修改过的配置条目即可）。 」 -「 支持通过`环境变量`配置项目，环境变量的书写格式参考`docker-compose.yml`文件或者我们的[Wiki页面](https://github.com/binary-husky/gpt_academic/wiki/项目配置说明)。配置读取优先级: `环境变量` > `config_private.py` > `config.py`。 」 + 「 支持通过`环境变量`配置项目，环境变量的书写格式参考`docker-compose.yml`文件或者我们的[Wiki页面](https://github.com/binary-husky/gpt_academic/wiki/项目配置说明)。配置读取优先级: `环境变量` > `config_private.py` > `config.py`。 」 3. 安装依赖 -```sh -# （选择I: 如熟悉python, python推荐版本 3.9 ~ 3.11）备注：使用官方pip源或者阿里pip源, 临时换源方法：python -m pip install -r requirements.txt -i https://mirrors.aliyun.com/pypi/simple/ -python -m pip install -r requirements.txt + ```sh + # （选择I: 如熟悉python, python推荐版本 3.9 ~ 3.11）备注：使用官方pip源或者阿里pip源, 临时换源方法：python -m pip install -r requirements.txt -i https://mirrors.aliyun.com/pypi/simple/ + python -m pip install -r requirements.txt -# （选择II: 使用Anaconda）步骤也是类似的 (https://www.bilibili.com/video/BV1rc411W7Dr)： -conda create -n gptac_venv python=3.11 # 创建anaconda环境 -conda activate gptac_venv # 激活anaconda环境 -python -m pip install -r requirements.txt # 这个步骤和pip安装一样的步骤 -``` + # （选择II: 使用Anaconda）步骤也是类似的 (https://www.bilibili.com/video/BV1rc411W7Dr)： + conda create -n gptac_venv python=3.11 # 创建anaconda环境 + conda activate gptac_venv # 激活anaconda环境 + python -m pip install -r requirements.txt # 这个步骤和pip安装一样的步骤 + ```
如果需要支持清华ChatGLM2/复旦MOSS/RWKV作为后端，请点击展开此处

【可选步骤】如果需要支持清华ChatGLM2/复旦MOSS作为后端，需要额外安装更多依赖（前提条件：熟悉Python + 用过Pytorch + 电脑配置够强）： + ```sh # 【可选步骤I】支持清华ChatGLM2。清华ChatGLM备注：如果遇到"Call ChatGLM fail 不能正常加载ChatGLM的参数" 错误，参考如下： 1：以上默认安装的为torch+cpu版，使用cuda需要卸载torch重新安装torch+cuda； 2：如因本机配置不够无法加载模型，可以修改request_llm/bridge_chatglm.py中的模型精度, 将 AutoTokenizer.from_pretrained("THUDM/chatglm-6b", trust_remote_code=True) 都修改为 AutoTokenizer.from_pretrained("THUDM/chatglm-6b-int4", trust_remote_code=True) python -m pip install -r request_llms/requirements_chatglm.txt @@ -155,39 +157,39 @@ AVAIL_LLM_MODELS = ["gpt-3.5-turbo", "api2d-gpt-3.5-turbo", "gpt-4", "api2d-gpt- 4. 运行 -```sh -python main.py -``` + ```sh + python main.py + ``` ### 安装方法II：使用Docker 0. 部署项目的全部能力（这个是包含cuda和latex的大型镜像。但如果您网速慢、硬盘小，则不推荐使用这个） [![fullcapacity](https://github.com/binary-husky/gpt_academic/actions/workflows/build-with-all-capacity.yml/badge.svg?branch=master)](https://github.com/binary-husky/gpt_academic/actions/workflows/build-with-all-capacity.yml) -``` sh -# 修改docker-compose.yml，保留方案0并删除其他方案。然后运行： -docker-compose up -``` + ``` sh + # 修改docker-compose.yml，保留方案0并删除其他方案。然后运行： + docker-compose up + ``` 1. 仅ChatGPT+文心一言+spark等在线模型（推荐大多数人选择） [![basic](https://github.com/binary-husky/gpt_academic/actions/workflows/build-without-local-llms.yml/badge.svg?branch=master)](https://github.com/binary-husky/gpt_academic/actions/workflows/build-without-local-llms.yml) [![basiclatex](https://github.com/binary-husky/gpt_academic/actions/workflows/build-with-latex.yml/badge.svg?branch=master)](https://github.com/binary-husky/gpt_academic/actions/workflows/build-with-latex.yml) [![basicaudio](https://github.com/binary-husky/gpt_academic/actions/workflows/build-with-audio-assistant.yml/badge.svg?branch=master)](https://github.com/binary-husky/gpt_academic/actions/workflows/build-with-audio-assistant.yml) -``` sh -# 修改docker-compose.yml，保留方案1并删除其他方案。然后运行： -docker-compose up -``` + ``` sh + # 修改docker-compose.yml，保留方案1并删除其他方案。然后运行： + docker-compose up + ``` P.S. 如果需要依赖Latex的插件功能，请见Wiki。另外，您也可以直接使用方案4或者方案0获取Latex功能。 2. ChatGPT + ChatGLM2 + MOSS + LLAMA2 + 通义千问（需要熟悉[Nvidia Docker](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html#installing-on-ubuntu-and-debian)运行时） [![chatglm](https://github.com/binary-husky/gpt_academic/actions/workflows/build-with-chatglm.yml/badge.svg?branch=master)](https://github.com/binary-husky/gpt_academic/actions/workflows/build-with-chatglm.yml) -``` sh -# 修改docker-compose.yml，保留方案2并删除其他方案。然后运行： -docker-compose up -``` + ``` sh + # 修改docker-compose.yml，保留方案2并删除其他方案。然后运行： + docker-compose up + ``` ### 安装方法III：其他部署姿势 @@ -208,9 +210,11 @@ docker-compose up # Advanced Usage ### I：自定义新的便捷按钮（学术快捷键） + 任意文本编辑器打开`core_functional.py`，添加条目如下，然后重启程序。（如按钮已存在，那么前缀、后缀都支持热修改，无需重启程序即可生效。） 例如 -``` + +```python "超级英译中": { # 前缀，会被加在你的输入之前。例如，用来描述你的要求，例如翻译、解释代码、润色等等 "Prefix": "请翻译把下面一段内容成中文，然后用一个markdown表格逐一解释文中出现的专有名词：\n\n", @@ -219,6 +223,7 @@ docker-compose up "Suffix": "", }, ``` +

@@ -295,6 +300,7 @@ Tip：不指定文件直接点击 `载入对话历史存档` 可以查看历史h ### II：版本: + - version 3.70（todo）: 优化AutoGen插件主题并设计一系列衍生插件 - version 3.60: 引入AutoGen作为新一代插件的基石 - version 3.57: 支持GLM3，星火v3，文心一言v4，修复本地模型的并发BUG @@ -315,7 +321,7 @@ Tip：不指定文件直接点击 `载入对话历史存档` 可以查看历史h - version 3.0: 对chatglm和其他小型llm的支持 - version 2.6: 重构了插件结构，提高了交互性，加入更多插件 - version 2.5: 自更新，解决总结大工程源代码时文本过长、token溢出的问题 -- version 2.4: (1)新增PDF全文翻译功能; (2)新增输入区切换位置的功能; (3)新增垂直布局选项; (4)多线程函数插件优化。 +- version 2.4: 新增PDF全文翻译功能; 新增输入区切换位置的功能 - version 2.3: 增强多线程交互性 - version 2.2: 函数插件支持热重载 - version 2.1: 可折叠式布局 @@ -337,6 +343,7 @@ GPT Academic开发者QQ群：`610599535` 1. `master` 分支: 主分支，稳定版 2. `frontier` 分支: 开发分支，测试版 +3. 如何接入其他大模型：[接入其他大模型](request_llms/README.md) ### V：参考与学习 diff --git a/check_proxy.py b/check_proxy.py index 977ac276..e2ba3f1a 100644 --- a/check_proxy.py +++ b/check_proxy.py @@ -5,7 +5,6 @@ def check_proxy(proxies): try: response = requests.get("https://ipapi.co/json/", proxies=proxies, timeout=4) data = response.json() - # print(f'查询代理的地理位置，返回的结果是{data}') if 'country_name' in data: country = data['country_name'] result = f"代理配置 {proxies_https}, 代理所在地：{country}" @@ -47,8 +46,8 @@ def backup_and_download(current_version, remote_version): os.makedirs(new_version_dir) shutil.copytree('./', backup_dir, ignore=lambda x, y: ['history']) proxies = get_conf('proxies') - r = requests.get( - 'https://github.com/binary-husky/chatgpt_academic/archive/refs/heads/master.zip', proxies=proxies, stream=True) + try: r = requests.get('https://github.com/binary-husky/chatgpt_academic/archive/refs/heads/master.zip', proxies=proxies, stream=True) + except: r = requests.get('https://public.gpt-academic.top/publish/master.zip', proxies=proxies, stream=True) zip_file_path = backup_dir+'/master.zip' with open(zip_file_path, 'wb+') as f: f.write(r.content) @@ -111,11 +110,10 @@ def auto_update(raise_error=False): try: from toolbox import get_conf import requests - import time import json proxies = get_conf('proxies') - response = requests.get( - "https://raw.githubusercontent.com/binary-husky/chatgpt_academic/master/version", proxies=proxies, timeout=5) + try: response = requests.get("https://raw.githubusercontent.com/binary-husky/chatgpt_academic/master/version", proxies=proxies, timeout=5) + except: response = requests.get("https://public.gpt-academic.top/publish/version", proxies=proxies, timeout=5) remote_json_data = json.loads(response.text) remote_version = remote_json_data['version'] if remote_json_data["show_feature"]: @@ -127,8 +125,7 @@ def auto_update(raise_error=False): current_version = json.loads(current_version)['version'] if (remote_version - current_version) >= 0.01-1e-5: from colorful import print亮黄 - print亮黄( - f'\n新版本可用。新版本:{remote_version}，当前版本:{current_version}。{new_feature}') + print亮黄(f'\n新版本可用。新版本:{remote_version}，当前版本:{current_version}。{new_feature}') print('（1）Github更新地址:\nhttps://github.com/binary-husky/chatgpt_academic\n') user_instruction = input('（2）是否一键更新代码（Y+回车=确认，输入其他/无输入+回车=不更新）？') if user_instruction in ['Y', 'y']: @@ -154,7 +151,7 @@ def auto_update(raise_error=False): print(msg) def warm_up_modules(): - print('正在执行一些模块的预热...') + print('正在执行一些模块的预热 ...') from toolbox import ProxyNetworkActivate from request_llms.bridge_all import model_info with ProxyNetworkActivate("Warmup_Modules"): diff --git a/config.py b/config.py index 59d7be15..46d1d90d 100644 --- a/config.py +++ b/config.py @@ -94,12 +94,12 @@ DEFAULT_FN_GROUPS = ['对话', '编程', '学术', '智能体'] # 模型选择是 (注意: LLM_MODEL是默认选中的模型, 它*必须*被包含在AVAIL_LLM_MODELS列表中 ) LLM_MODEL = "gpt-3.5-turbo" # 可选 ↓↓↓ -AVAIL_LLM_MODELS = ["gpt-3.5-turbo-1106","gpt-4-1106-preview", +AVAIL_LLM_MODELS = ["gpt-3.5-turbo-1106","gpt-4-1106-preview","gpt-4-vision-preview", "gpt-3.5-turbo-16k", "gpt-3.5-turbo", "azure-gpt-3.5", "api2d-gpt-3.5-turbo", 'api2d-gpt-3.5-turbo-16k', "gpt-4", "gpt-4-32k", "azure-gpt-4", "api2d-gpt-4", - "chatglm3", "moss", "newbing", "claude-2"] -# P.S. 其他可用的模型还包括 ["zhipuai", "qianfan", "llama2", "qwen", "gpt-3.5-turbo-0613", "gpt-3.5-turbo-16k-0613", "gpt-3.5-random" + "chatglm3", "moss", "claude-2"] +# P.S. 其他可用的模型还包括 ["zhipuai", "qianfan", "deepseekcoder", "llama2", "qwen", "gpt-3.5-turbo-0613", "gpt-3.5-turbo-16k-0613", "gpt-3.5-random" # "spark", "sparkv2", "sparkv3", "chatglm_onnx", "claude-1-100k", "claude-2", "internlm", "jittorllms_pangualpha", "jittorllms_llama"] @@ -278,11 +278,31 @@ NUM_CUSTOM_BASIC_BTN = 4 │ ├── BAIDU_CLOUD_API_KEY │ └── BAIDU_CLOUD_SECRET_KEY │ -├── "newbing" Newbing接口不再稳定，不推荐使用 +├── "zhipuai" 智谱AI大模型chatglm_turbo +│ ├── ZHIPUAI_API_KEY +│ └── ZHIPUAI_MODEL +│ +└── "newbing" Newbing接口不再稳定，不推荐使用 ├── NEWBING_STYLE └── NEWBING_COOKIES +本地大模型示意图 +│ +├── "chatglm3" +├── "chatglm" +├── "chatglm_onnx" +├── "chatglmft" +├── "internlm" +├── "moss" +├── "jittorllms_pangualpha" +├── "jittorllms_llama" +├── "deepseekcoder" +├── "qwen" +├── RWKV的支持见Wiki +└── "llama2" + + 用户图形界面布局依赖关系示意图 │ ├── CHATBOT_HEIGHT 对话窗的高度 @@ -293,7 +313,7 @@ NUM_CUSTOM_BASIC_BTN = 4 ├── THEME 色彩主题 ├── AUTO_CLEAR_TXT 是否在提交时自动清空输入框 ├── ADD_WAIFU 加一个live2d装饰 -├── ALLOW_RESET_CONFIG 是否允许通过自然语言描述修改本页的配置，该功能具有一定的危险性 +└── ALLOW_RESET_CONFIG 是否允许通过自然语言描述修改本页的配置，该功能具有一定的危险性 插件在线服务配置依赖关系示意图 @@ -305,7 +325,7 @@ NUM_CUSTOM_BASIC_BTN = 4 │ ├── ALIYUN_ACCESSKEY │ └── ALIYUN_SECRET │ -├── PDF文档精准解析 -│ └── GROBID_URLS +└── PDF文档精准解析 + └── GROBID_URLS """ diff --git a/crazy_functions/Latex输出PDF结果.py b/crazy_functions/Latex输出PDF结果.py index a2545ddd..f3919edc 100644 --- a/crazy_functions/Latex输出PDF结果.py +++ b/crazy_functions/Latex输出PDF结果.py @@ -73,6 +73,7 @@ def move_project(project_folder, arxiv_id=None): # align subfolder if there is a folder wrapper items = glob.glob(pj(project_folder,'*')) + items = [item for item in items if os.path.basename(item)!='__MACOSX'] if len(glob.glob(pj(project_folder,'*.tex'))) == 0 and len(items) == 1: if os.path.isdir(items[0]): project_folder = items[0] @@ -214,7 +215,6 @@ def Latex英文纠错加PDF对比(txt, llm_kwargs, plugin_kwargs, chatbot, histo # <-------------- we are done -------------> return success - # ========================================= 插件主程序2 ===================================================== @CatchException diff --git a/crazy_functions/crazy_utils.py b/crazy_functions/crazy_utils.py index 5788194c..afe079f4 100644 --- a/crazy_functions/crazy_utils.py +++ b/crazy_functions/crazy_utils.py @@ -1,4 +1,4 @@ -from toolbox import update_ui, get_conf, trimmed_format_exc, get_log_folder +from toolbox import update_ui, get_conf, trimmed_format_exc, get_max_token import threading import os import logging @@ -92,7 +92,7 @@ def request_gpt_model_in_new_thread_with_ui_alive( # 【选择处理】 尝试计算比例，尽可能多地保留文本 from toolbox import get_reduce_token_percent p_ratio, n_exceed = get_reduce_token_percent(str(token_exceeded_error)) - MAX_TOKEN = 4096 + MAX_TOKEN = get_max_token(llm_kwargs) EXCEED_ALLO = 512 + 512 * exceeded_cnt inputs, history = input_clipping(inputs, history, max_token_limit=MAX_TOKEN-EXCEED_ALLO) mutable[0] += f'[Local Message] 警告，文本过长将进行截断，Token溢出数：{n_exceed}。\n\n' @@ -224,7 +224,7 @@ def request_gpt_model_multi_threads_with_very_awesome_ui_and_high_efficiency( # 【选择处理】 尝试计算比例，尽可能多地保留文本 from toolbox import get_reduce_token_percent p_ratio, n_exceed = get_reduce_token_percent(str(token_exceeded_error)) - MAX_TOKEN = 4096 + MAX_TOKEN = get_max_token(llm_kwargs) EXCEED_ALLO = 512 + 512 * exceeded_cnt inputs, history = input_clipping(inputs, history, max_token_limit=MAX_TOKEN-EXCEED_ALLO) gpt_say += f'[Local Message] 警告，文本过长将进行截断，Token溢出数：{n_exceed}。\n\n' diff --git a/crazy_functions/latex_fns/latex_toolbox.py b/crazy_functions/latex_fns/latex_toolbox.py index afa65bf9..4555ff18 100644 --- a/crazy_functions/latex_fns/latex_toolbox.py +++ b/crazy_functions/latex_fns/latex_toolbox.py @@ -283,10 +283,10 @@ def find_tex_file_ignore_case(fp): dir_name = os.path.dirname(fp) base_name = os.path.basename(fp) # 如果输入的文件路径是正确的 - if os.path.exists(pj(dir_name, base_name)): return pj(dir_name, base_name) + if os.path.isfile(pj(dir_name, base_name)): return pj(dir_name, base_name) # 如果不正确，试着加上.tex后缀试试 if not base_name.endswith('.tex'): base_name+='.tex' - if os.path.exists(pj(dir_name, base_name)): return pj(dir_name, base_name) + if os.path.isfile(pj(dir_name, base_name)): return pj(dir_name, base_name) # 如果还找不到，解除大小写限制，再试一次 import glob for f in glob.glob(dir_name+'/*.tex'): diff --git a/docs/waifu_plugin/waifu-tips.js b/docs/waifu_plugin/waifu-tips.js index 8f9533a1..d1b214d7 100644 --- a/docs/waifu_plugin/waifu-tips.js +++ b/docs/waifu_plugin/waifu-tips.js @@ -258,39 +258,7 @@ function loadTipsMessage(result) { }); window.showWelcomeMessage = function(result) { - var text; - if (window.location.href == live2d_settings.homePageUrl) { - var now = (new Date()).getHours(); - if (now > 23 || now <= 5) text = getRandText(result.waifu.hour_tips['t23-5']); - else if (now > 5 && now <= 7) text = getRandText(result.waifu.hour_tips['t5-7']); - else if (now > 7 && now <= 11) text = getRandText(result.waifu.hour_tips['t7-11']); - else if (now > 11 && now <= 14) text = getRandText(result.waifu.hour_tips['t11-14']); - else if (now > 14 && now <= 17) text = getRandText(result.waifu.hour_tips['t14-17']); - else if (now > 17 && now <= 19) text = getRandText(result.waifu.hour_tips['t17-19']); - else if (now > 19 && now <= 21) text = getRandText(result.waifu.hour_tips['t19-21']); - else if (now > 21 && now <= 23) text = getRandText(result.waifu.hour_tips['t21-23']); - else text = getRandText(result.waifu.hour_tips.default); - } else { - var referrer_message = result.waifu.referrer_message; - if (document.referrer !== '') { - var referrer = document.createElement('a'); - referrer.href = document.referrer; - var domain = referrer.hostname.split('.')[1]; - if (window.location.hostname == referrer.hostname) - text = referrer_message.localhost[0] + document.title.split(referrer_message.localhost[2])[0] + referrer_message.localhost[1]; - else if (domain == 'baidu') - text = referrer_message.baidu[0] + referrer.search.split('&wd=')[1].split('&')[0] + referrer_message.baidu[1]; - else if (domain == 'so') - text = referrer_message.so[0] + referrer.search.split('&q=')[1].split('&')[0] + referrer_message.so[1]; - else if (domain == 'google') - text = referrer_message.google[0] + document.title.split(referrer_message.google[2])[0] + referrer_message.google[1]; - else { - $.each(result.waifu.referrer_hostname, function(i,val) {if (i==referrer.hostname) referrer.hostname = getRandText(val)}); - text = referrer_message.default[0] + referrer.hostname + referrer_message.default[1]; - } - } else text = referrer_message.none[0] + document.title.split(referrer_message.none[2])[0] + referrer_message.none[1]; - } - showMessage(text, 6000); + showMessage('欢迎使用GPT-Academic', 6000); }; if (live2d_settings.showWelcomeMessage) showWelcomeMessage(result); var waifu_tips = result.waifu; diff --git a/docs/waifu_plugin/waifu-tips.json b/docs/waifu_plugin/waifu-tips.json index 229d5a14..30f26315 100644 --- a/docs/waifu_plugin/waifu-tips.json +++ b/docs/waifu_plugin/waifu-tips.json @@ -83,8 +83,8 @@ "很多强大的函数插件隐藏在下拉菜单中呢。", "红色的插件，使用之前需要把文件上传进去哦。", "想添加功能按钮吗？读读readme很容易就学会啦。", - "敏感或机密的信息，不可以问chatGPT的哦！", - "chatGPT究竟是划时代的创新，还是扼杀创造力的毒药呢？" + "敏感或机密的信息，不可以问AI的哦！", + "LLM究竟是划时代的创新，还是扼杀创造力的毒药呢？" ] } ], "click": [ @@ -92,8 +92,6 @@ "selector": ".waifu #live2d", "text": [ "是…是不小心碰到了吧", - "萝莉控是什么呀", - "你看到我的小熊了吗", "再摸的话我可要报警了！⌇●﹏●⌇", "110吗，这里有个变态一直在摸我(ó﹏ò｡)" ] diff --git a/request_llms/README.md b/request_llms/README.md index 92b856e3..288bc135 100644 --- a/request_llms/README.md +++ b/request_llms/README.md @@ -1,79 +1,35 @@ -# 如何使用其他大语言模型 - -## ChatGLM - -- 安装依赖 `pip install -r request_llms/requirements_chatglm.txt` -- 修改配置，在config.py中将LLM_MODEL的值改为"chatglm" - -``` sh -LLM_MODEL = "chatglm" -``` -- 运行！ -``` sh -`python main.py` -``` - -## Claude-Stack - -- 请参考此教程获取 https://zhuanlan.zhihu.com/p/627485689 - - 1、SLACK_CLAUDE_BOT_ID - - 2、SLACK_CLAUDE_USER_TOKEN - -- 把token加入config.py - -## Newbing - -- 使用cookie editor获取cookie（json） -- 把cookie（json）加入config.py （NEWBING_COOKIES） - -## Moss -- 使用docker-compose - -## RWKV -- 使用docker-compose - -## LLAMA -- 使用docker-compose - -## 盘古 -- 使用docker-compose +P.S. 如果您按照以下步骤成功接入了新的大模型，欢迎发Pull Requests（如果您在自己接入新模型的过程中遇到困难，欢迎加README底部QQ群联系群主） ---- -## Text-Generation-UI (TGUI，调试中，暂不可用) +# 如何接入其他本地大语言模型 -### 1. 部署TGUI -``` sh -# 1 下载模型 -git clone https://github.com/oobabooga/text-generation-webui.git -# 2 这个仓库的最新代码有问题，回滚到几周之前 -git reset --hard fcda3f87767e642d1c0411776e549e1d3894843d -# 3 切换路径 -cd text-generation-webui -# 4 安装text-generation的额外依赖 -pip install accelerate bitsandbytes flexgen gradio llamacpp markdown numpy peft requests rwkv safetensors sentencepiece tqdm datasets git+https://github.com/huggingface/transformers -# 5 下载模型 -python download-model.py facebook/galactica-1.3b -# 其他可选如 facebook/opt-1.3b -# facebook/galactica-1.3b -# facebook/galactica-6.7b -# facebook/galactica-120b -# facebook/pygmalion-1.3b 等 -# 详情见 https://github.com/oobabooga/text-generation-webui +1. 复制`request_llms/bridge_llama2.py`，重命名为你喜欢的名字 -# 6 启动text-generation -python server.py --cpu --listen --listen-port 7865 --model facebook_galactica-1.3b -``` +2. 修改`load_model_and_tokenizer`方法，加载你的模型和分词器（去该模型官网找demo，复制粘贴即可） -### 2. 修改config.py +3. 修改`llm_stream_generator`方法，定义推理模型（去该模型官网找demo，复制粘贴即可） -``` sh -# LLM_MODEL格式: tgui:[模型]@[ws地址]:[ws端口] , 端口要和上面给定的端口一致 -LLM_MODEL = "tgui:galactica-1.3b@localhost:7860" -``` +4. 命令行测试 + - 修改`tests/test_llms.py`（聪慧如您，只需要看一眼该文件就明白怎么修改了） + - 运行`python tests/test_llms.py` -### 3. 运行！ -``` sh -cd chatgpt-academic -python main.py -``` +5. 测试通过后，在`request_llms/bridge_all.py`中做最后的修改，把你的模型完全接入到框架中（聪慧如您，只需要看一眼该文件就明白怎么修改了） + +6. 修改`LLM_MODEL`配置，然后运行`python main.py`，测试最后的效果 + + +# 如何接入其他在线大语言模型 + +1. 复制`request_llms/bridge_zhipu.py`，重命名为你喜欢的名字 + +2. 修改`predict_no_ui_long_connection` + +3. 修改`predict` + +4. 命令行测试 + - 修改`tests/test_llms.py`（聪慧如您，只需要看一眼该文件就明白怎么修改了） + - 运行`python tests/test_llms.py` + +5. 测试通过后，在`request_llms/bridge_all.py`中做最后的修改，把你的模型完全接入到框架中（聪慧如您，只需要看一眼该文件就明白怎么修改了） + +6. 修改`LLM_MODEL`配置，然后运行`python main.py`，测试最后的效果 \ No newline at end of file diff --git a/request_llms/bridge_all.py b/request_llms/bridge_all.py index 88848a98..8dece548 100644 --- a/request_llms/bridge_all.py +++ b/request_llms/bridge_all.py @@ -543,6 +543,22 @@ if "zhipuai" in AVAIL_LLM_MODELS: # zhipuai }) except: print(trimmed_format_exc()) +if "deepseekcoder" in AVAIL_LLM_MODELS: # deepseekcoder + try: + from .bridge_deepseekcoder import predict_no_ui_long_connection as deepseekcoder_noui + from .bridge_deepseekcoder import predict as deepseekcoder_ui + model_info.update({ + "deepseekcoder": { + "fn_with_ui": deepseekcoder_ui, + "fn_without_ui": deepseekcoder_noui, + "endpoint": None, + "max_token": 4096, + "tokenizer": tokenizer_gpt35, + "token_cnt": get_token_num_gpt35, + } + }) + except: + print(trimmed_format_exc()) # <-- 用于定义和切换多个azure模型 --> AZURE_CFG_ARRAY = get_conf("AZURE_CFG_ARRAY") diff --git a/request_llms/bridge_deepseekcoder.py b/request_llms/bridge_deepseekcoder.py new file mode 100644 index 00000000..2242eec7 --- /dev/null +++ b/request_llms/bridge_deepseekcoder.py @@ -0,0 +1,88 @@ +model_name = "deepseek-coder-6.7b-instruct" +cmd_to_install = "未知" # "`pip install -r request_llms/requirements_qwen.txt`" + +import os +from toolbox import ProxyNetworkActivate +from toolbox import get_conf +from .local_llm_class import LocalLLMHandle, get_local_llm_predict_fns +from threading import Thread + +def download_huggingface_model(model_name, max_retry, local_dir): + from huggingface_hub import snapshot_download + for i in range(1, max_retry): + try: + snapshot_download(repo_id=model_name, local_dir=local_dir, resume_download=True) + break + except Exception as e: + print(f'\n\n下载失败，重试第{i}次中...\n\n') + return local_dir +# ------------------------------------------------------------------------------------------------------------------------ +# 🔌💻 Local Model +# ------------------------------------------------------------------------------------------------------------------------ +class GetCoderLMHandle(LocalLLMHandle): + + def load_model_info(self): + # 🏃‍♂️🏃‍♂️🏃‍♂️ 子进程执行 + self.model_name = model_name + self.cmd_to_install = cmd_to_install + + def load_model_and_tokenizer(self): + # 🏃‍♂️🏃‍♂️🏃‍♂️ 子进程执行 + with ProxyNetworkActivate('Download_LLM'): + from transformers import AutoTokenizer, AutoModelForCausalLM, TextIteratorStreamer + model_name = "deepseek-ai/deepseek-coder-6.7b-instruct" + # local_dir = f"~/.cache/{model_name}" + # if not os.path.exists(local_dir): + # tokenizer = download_huggingface_model(model_name, max_retry=128, local_dir=local_dir) + tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True) + self._streamer = TextIteratorStreamer(tokenizer) + model = AutoModelForCausalLM.from_pretrained(model_name, trust_remote_code=True) + if get_conf('LOCAL_MODEL_DEVICE') != 'cpu': + model = model.cuda() + return model, tokenizer + + def llm_stream_generator(self, **kwargs): + # 🏃‍♂️🏃‍♂️🏃‍♂️ 子进程执行 + def adaptor(kwargs): + query = kwargs['query'] + max_length = kwargs['max_length'] + top_p = kwargs['top_p'] + temperature = kwargs['temperature'] + history = kwargs['history'] + return query, max_length, top_p, temperature, history + + query, max_length, top_p, temperature, history = adaptor(kwargs) + history.append({ 'role': 'user', 'content': query}) + messages = history + inputs = self._tokenizer.apply_chat_template(messages, return_tensors="pt").to(self._model.device) + generation_kwargs = dict( + inputs=inputs, + max_new_tokens=max_length, + do_sample=False, + top_p=top_p, + streamer = self._streamer, + top_k=50, + temperature=temperature, + num_return_sequences=1, + eos_token_id=32021, + ) + thread = Thread(target=self._model.generate, kwargs=generation_kwargs, daemon=True) + thread.start() + generated_text = "" + for new_text in self._streamer: + generated_text += new_text + # print(generated_text) + yield generated_text + + + def try_to_import_special_deps(self, **kwargs): pass + # import something that will raise error if the user does not install requirement_*.txt + # 🏃‍♂️🏃‍♂️🏃‍♂️ 主进程执行 + # import importlib + # importlib.import_module('modelscope') + + +# ------------------------------------------------------------------------------------------------------------------------ +# 🔌💻 GPT-Academic Interface +# ------------------------------------------------------------------------------------------------------------------------ +predict_no_ui_long_connection, predict = get_local_llm_predict_fns(GetCoderLMHandle, model_name, history_format='chatglm3') \ No newline at end of file diff --git a/request_llms/bridge_llama2.py b/request_llms/bridge_llama2.py index e6da4b75..bfa3c14a 100644 --- a/request_llms/bridge_llama2.py +++ b/request_llms/bridge_llama2.py @@ -12,7 +12,7 @@ from threading import Thread # ------------------------------------------------------------------------------------------------------------------------ # 🔌💻 Local Model # ------------------------------------------------------------------------------------------------------------------------ -class GetONNXGLMHandle(LocalLLMHandle): +class GetLlamaHandle(LocalLLMHandle): def load_model_info(self): # 🏃‍♂️🏃‍♂️🏃‍♂️ 子进程执行 @@ -87,4 +87,4 @@ class GetONNXGLMHandle(LocalLLMHandle): # ------------------------------------------------------------------------------------------------------------------------ # 🔌💻 GPT-Academic Interface # ------------------------------------------------------------------------------------------------------------------------ -predict_no_ui_long_connection, predict = get_local_llm_predict_fns(GetONNXGLMHandle, model_name) \ No newline at end of file +predict_no_ui_long_connection, predict = get_local_llm_predict_fns(GetLlamaHandle, model_name) \ No newline at end of file diff --git a/request_llms/bridge_qwen.py b/request_llms/bridge_qwen.py index afd886bf..85a4d80c 100644 --- a/request_llms/bridge_qwen.py +++ b/request_llms/bridge_qwen.py @@ -15,7 +15,7 @@ from .local_llm_class import LocalLLMHandle, get_local_llm_predict_fns # ------------------------------------------------------------------------------------------------------------------------ # 🔌💻 Local Model # ------------------------------------------------------------------------------------------------------------------------ -class GetONNXGLMHandle(LocalLLMHandle): +class GetQwenLMHandle(LocalLLMHandle): def load_model_info(self): # 🏃‍♂️🏃‍♂️🏃‍♂️ 子进程执行 @@ -64,4 +64,4 @@ class GetONNXGLMHandle(LocalLLMHandle): # ------------------------------------------------------------------------------------------------------------------------ # 🔌💻 GPT-Academic Interface # ------------------------------------------------------------------------------------------------------------------------ -predict_no_ui_long_connection, predict = get_local_llm_predict_fns(GetONNXGLMHandle, model_name) \ No newline at end of file +predict_no_ui_long_connection, predict = get_local_llm_predict_fns(GetQwenLMHandle, model_name) \ No newline at end of file diff --git a/request_llms/bridge_zhipu.py b/request_llms/bridge_zhipu.py index a1e0de59..915a13ef 100644 --- a/request_llms/bridge_zhipu.py +++ b/request_llms/bridge_zhipu.py @@ -1,6 +1,7 @@ import time from toolbox import update_ui, get_conf, update_ui_lastest_msg +from toolbox import check_packages, report_exception model_name = '智谱AI大模型' @@ -37,6 +38,14 @@ def predict(inputs, llm_kwargs, plugin_kwargs, chatbot, history=[], system_promp chatbot.append((inputs, "")) yield from update_ui(chatbot=chatbot, history=history) + # 尝试导入依赖，如果缺少依赖，则给出安装建议 + try: + check_packages(["zhipuai"]) + except: + yield from update_ui_lastest_msg(f"导入软件依赖失败。使用该模型需要额外依赖，安装方法```pip install --upgrade zhipuai```。", + chatbot=chatbot, history=history, delay=0) + return + if validate_key() is False: yield from update_ui_lastest_msg(lastmsg="[Local Message] 请配置ZHIPUAI_API_KEY", chatbot=chatbot, history=history, delay=0) return diff --git a/request_llms/local_llm_class.py b/request_llms/local_llm_class.py index 091707a7..413df03f 100644 --- a/request_llms/local_llm_class.py +++ b/request_llms/local_llm_class.py @@ -198,7 +198,7 @@ class LocalLLMHandle(Process): if res.startswith(self.std_tag): new_output = res[len(self.std_tag):] std_out = std_out[:std_out_clip_len] - # print(new_output, end='') + print(new_output, end='') std_out = new_output + std_out yield self.std_tag + '\n```\n' + std_out + '\n```\n' elif res == '[Finish]': diff --git a/tests/test_llms.py b/tests/test_llms.py index 6285f030..8b685972 100644 --- a/tests/test_llms.py +++ b/tests/test_llms.py @@ -15,7 +15,8 @@ if __name__ == "__main__": # from request_llms.bridge_jittorllms_pangualpha import predict_no_ui_long_connection # from request_llms.bridge_jittorllms_llama import predict_no_ui_long_connection # from request_llms.bridge_claude import predict_no_ui_long_connection - from request_llms.bridge_internlm import predict_no_ui_long_connection + # from request_llms.bridge_internlm import predict_no_ui_long_connection + from request_llms.bridge_deepseekcoder import predict_no_ui_long_connection # from request_llms.bridge_qwen import predict_no_ui_long_connection # from request_llms.bridge_spark import predict_no_ui_long_connection # from request_llms.bridge_zhipu import predict_no_ui_long_connection diff --git a/themes/contrast.py b/themes/contrast.py index d407d92a..bf48808d 100644 --- a/themes/contrast.py +++ b/themes/contrast.py @@ -1,6 +1,8 @@ +import os import gradio as gr from toolbox import get_conf CODE_HIGHLIGHT, ADD_WAIFU, LAYOUT = get_conf('CODE_HIGHLIGHT', 'ADD_WAIFU', 'LAYOUT') +theme_dir = os.path.dirname(__file__) def adjust_theme(): @@ -57,7 +59,7 @@ def adjust_theme(): button_cancel_text_color_dark="white", ) - with open('themes/common.js', 'r', encoding='utf8') as f: + with open(os.path.join(theme_dir, 'common.js'), 'r', encoding='utf8') as f: js = f"" # 添加一个萌萌的看板娘 @@ -67,7 +69,9 @@ def adjust_theme(): """ - gradio_original_template_fn = gr.routes.templates.TemplateResponse + if not hasattr(gr, 'RawTemplateResponse'): + gr.RawTemplateResponse = gr.routes.templates.TemplateResponse + gradio_original_template_fn = gr.RawTemplateResponse def gradio_new_template_fn(*args, **kwargs): res = gradio_original_template_fn(*args, **kwargs) res.body = res.body.replace(b'', f'{js}'.encode("utf8")) @@ -79,7 +83,7 @@ def adjust_theme(): print('gradio版本较旧, 不能自定义字体和颜色') return set_theme -with open("themes/contrast.css", "r", encoding="utf-8") as f: +with open(os.path.join(theme_dir, 'contrast.css'), "r", encoding="utf-8") as f: advanced_css = f.read() -with open("themes/common.css", "r", encoding="utf-8") as f: +with open(os.path.join(theme_dir, 'common.css'), "r", encoding="utf-8") as f: advanced_css += f.read() diff --git a/themes/default.py b/themes/default.py index 6fa2ba5c..64b67209 100644 --- a/themes/default.py +++ b/themes/default.py @@ -60,7 +60,7 @@ def adjust_theme(): with open(os.path.join(theme_dir, 'common.js'), 'r', encoding='utf8') as f: js = f"" - + # 添加一个萌萌的看板娘 if ADD_WAIFU: js += """ @@ -68,7 +68,9 @@ def adjust_theme(): """ - gradio_original_template_fn = gr.routes.templates.TemplateResponse + if not hasattr(gr, 'RawTemplateResponse'): + gr.RawTemplateResponse = gr.routes.templates.TemplateResponse + gradio_original_template_fn = gr.RawTemplateResponse def gradio_new_template_fn(*args, **kwargs): res = gradio_original_template_fn(*args, **kwargs) res.body = res.body.replace(b'', f'{js}'.encode("utf8")) diff --git a/themes/gradios.py b/themes/gradios.py index 96a9c54e..9db13415 100644 --- a/themes/gradios.py +++ b/themes/gradios.py @@ -1,7 +1,9 @@ -import gradio as gr import logging +import os +import gradio as gr from toolbox import get_conf, ProxyNetworkActivate CODE_HIGHLIGHT, ADD_WAIFU, LAYOUT = get_conf('CODE_HIGHLIGHT', 'ADD_WAIFU', 'LAYOUT') +theme_dir = os.path.dirname(__file__) def dynamic_set_theme(THEME): set_theme = gr.themes.ThemeClass() @@ -13,7 +15,6 @@ def dynamic_set_theme(THEME): return set_theme def adjust_theme(): - try: set_theme = gr.themes.ThemeClass() with ProxyNetworkActivate('Download_Gradio_Theme'): @@ -23,7 +24,7 @@ def adjust_theme(): if THEME.startswith('huggingface-'): THEME = THEME.lstrip('huggingface-') set_theme = set_theme.from_hub(THEME.lower()) - with open('themes/common.js', 'r', encoding='utf8') as f: + with open(os.path.join(theme_dir, 'common.js'), 'r', encoding='utf8') as f: js = f"" # 添加一个萌萌的看板娘 @@ -33,7 +34,9 @@ def adjust_theme(): """ - gradio_original_template_fn = gr.routes.templates.TemplateResponse + if not hasattr(gr, 'RawTemplateResponse'): + gr.RawTemplateResponse = gr.routes.templates.TemplateResponse + gradio_original_template_fn = gr.RawTemplateResponse def gradio_new_template_fn(*args, **kwargs): res = gradio_original_template_fn(*args, **kwargs) res.body = res.body.replace(b'', f'{js}'.encode("utf8")) @@ -46,7 +49,5 @@ def adjust_theme(): logging.error('gradio版本较旧, 不能自定义字体和颜色:', trimmed_format_exc()) return set_theme -# with open("themes/default.css", "r", encoding="utf-8") as f: -# advanced_css = f.read() -with open("themes/common.css", "r", encoding="utf-8") as f: +with open(os.path.join(theme_dir, 'common.css'), "r", encoding="utf-8") as f: advanced_css = f.read() diff --git a/themes/green.py b/themes/green.py index a29a0fa1..326981e3 100644 --- a/themes/green.py +++ b/themes/green.py @@ -1,6 +1,8 @@ +import os import gradio as gr from toolbox import get_conf CODE_HIGHLIGHT, ADD_WAIFU, LAYOUT = get_conf('CODE_HIGHLIGHT', 'ADD_WAIFU', 'LAYOUT') +theme_dir = os.path.dirname(__file__) def adjust_theme(): try: @@ -73,7 +75,7 @@ def adjust_theme(): chatbot_code_background_color_dark="*neutral_950", ) - with open('themes/common.js', 'r', encoding='utf8') as f: + with open(os.path.join(theme_dir, 'common.js'), 'r', encoding='utf8') as f: js = f"" # 添加一个萌萌的看板娘 @@ -83,11 +85,13 @@ def adjust_theme(): """ - - with open('themes/green.js', 'r', encoding='utf8') as f: + + with open(os.path.join(theme_dir, 'green.js'), 'r', encoding='utf8') as f: js += f"" - - gradio_original_template_fn = gr.routes.templates.TemplateResponse + + if not hasattr(gr, 'RawTemplateResponse'): + gr.RawTemplateResponse = gr.routes.templates.TemplateResponse + gradio_original_template_fn = gr.RawTemplateResponse def gradio_new_template_fn(*args, **kwargs): res = gradio_original_template_fn(*args, **kwargs) res.body = res.body.replace(b'', f'{js}'.encode("utf8")) @@ -99,8 +103,7 @@ def adjust_theme(): print('gradio版本较旧, 不能自定义字体和颜色') return set_theme - -with open("themes/green.css", "r", encoding="utf-8") as f: +with open(os.path.join(theme_dir, 'green.css'), "r", encoding="utf-8") as f: advanced_css = f.read() -with open("themes/common.css", "r", encoding="utf-8") as f: +with open(os.path.join(theme_dir, 'common.css'), "r", encoding="utf-8") as f: advanced_css += f.read() diff --git a/version b/version index cf14e82b..5f6de09c 100644 --- a/version +++ b/version @@ -1,5 +1,5 @@ { - "version": 3.60, + "version": 3.61, "show_feature": true, - "new_feature": "修复多个BUG <-> AutoGen多智能体插件测试版 <-> 修复本地模型在Windows下的加载BUG <-> 支持文心一言v4和星火v3 <-> 支持GLM3和智谱的API <-> 解决本地模型并发BUG <-> 支持动态追加基础功能按钮 <-> 新汇报PDF汇总页面 <-> 重新编译Gradio优化使用体验" + "new_feature": "修复潜在的多用户冲突问题 <-> 接入Deepseek Coder <-> AutoGen多智能体插件测试版 <-> 修复本地模型在Windows下的加载BUG <-> 支持文心一言v4和星火v3 <-> 支持GLM3和智谱的API <-> 解决本地模型并发BUG <-> 支持动态追加基础功能按钮" }