Compare commits
133 Commits
version3.1
...
version3.3
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
b6d2766e59 | ||
|
|
73ce471a0e | ||
|
|
4e113139c8 | ||
|
|
081acc6404 | ||
|
|
1a999497d7 | ||
|
|
6137963355 | ||
|
|
22bffdb737 | ||
|
|
75adcbffeb | ||
|
|
4451770061 | ||
|
|
09c413a272 | ||
|
|
ddb6c90a8f | ||
|
|
71590426f9 | ||
|
|
b3e5cdb3a5 | ||
|
|
6595ab813e | ||
|
|
d1efbd26da | ||
|
|
f04683732e | ||
|
|
cb0241db78 | ||
|
|
a097b6cd03 | ||
|
|
487ffe7888 | ||
|
|
51424a7d08 | ||
|
|
06e8e8f9a6 | ||
|
|
0512b311f8 | ||
|
|
81d53d0726 | ||
|
|
a141c5ccdc | ||
|
|
e361d741c3 | ||
|
|
f5bc58dbde | ||
|
|
e7b73f3041 | ||
|
|
ed8db8c8ae | ||
|
|
df97213d3b | ||
|
|
97443d1f83 | ||
|
|
59bed52faf | ||
|
|
3814c3a915 | ||
|
|
d98d0a291e | ||
|
|
ee94fa6dc4 | ||
|
|
d2e46f6684 | ||
|
|
5948dcacd5 | ||
|
|
3041858e7f | ||
|
|
9c2a6bc413 | ||
|
|
1cf8b6c6c8 | ||
|
|
781ef4487c | ||
|
|
4a494354b1 | ||
|
|
385c775aa5 | ||
|
|
518385dea2 | ||
|
|
4d1eea7bd5 | ||
|
|
9cb51ccc70 | ||
|
|
94dc398163 | ||
|
|
65317e33af | ||
|
|
06fbdf43af | ||
|
|
ab61418410 | ||
|
|
0785ff2aed | ||
|
|
676fe40d39 | ||
|
|
0b89673ee9 | ||
|
|
2f4e050612 | ||
|
|
87d963bda5 | ||
|
|
07807e4653 | ||
|
|
2b96217f2b | ||
|
|
13342c2988 | ||
|
|
95f8b2824a | ||
|
|
4065d6e234 | ||
|
|
d3dcd432e8 | ||
|
|
7d14de79bf | ||
|
|
15c6b52b5f | ||
|
|
c0f1b5bc8e | ||
|
|
bd62c6be68 | ||
|
|
70bd21f09a | ||
|
|
a0f15f1512 | ||
|
|
4575046ce1 | ||
|
|
33ea7391b5 | ||
|
|
e90eee2d8e | ||
|
|
7d44210a48 | ||
|
|
206f4138b6 | ||
|
|
6d2807f499 | ||
|
|
f1234937c6 | ||
|
|
7beea951c6 | ||
|
|
6f7e8076c7 | ||
|
|
ae24fab441 | ||
|
|
880be21bf7 | ||
|
|
559b3cd6bb | ||
|
|
9d9df8aa57 | ||
|
|
64548d33a9 | ||
|
|
c3cafd8d6f | ||
|
|
e9a6efef7f | ||
|
|
89a75e26c3 | ||
|
|
1139d395f2 | ||
|
|
e20070939c | ||
|
|
3236fcca21 | ||
|
|
5353eba376 | ||
|
|
7339b06acb | ||
|
|
ce1fc3a999 | ||
|
|
a9a489231a | ||
|
|
e889590a91 | ||
|
|
9481405f6f | ||
|
|
7317d79a3c | ||
|
|
a46e0111cd | ||
|
|
01a377d747 | ||
|
|
50258b781e | ||
|
|
dd1ba222ae | ||
|
|
b7d4adeccc | ||
|
|
3f82208062 | ||
|
|
5f319061d7 | ||
|
|
2c2a8ea549 | ||
|
|
90e1eef61f | ||
|
|
325406a650 | ||
|
|
bff4a87914 | ||
|
|
de0ed4a6f5 | ||
|
|
0ff838443e | ||
|
|
cfbfb68618 | ||
|
|
b42f2f745f | ||
|
|
9945d5048a | ||
|
|
f0ff1f2c64 | ||
|
|
7dd73e1330 | ||
|
|
4cfbacdb26 | ||
|
|
3bb4b4c92a | ||
|
|
bda025bc50 | ||
|
|
c24ff30f8c | ||
|
|
53189aea4e | ||
|
|
4b7a954fc8 | ||
|
|
2b9261bc39 | ||
|
|
4812513cbc | ||
|
|
4bd8475d95 | ||
|
|
ac219f40c5 | ||
|
|
26af2b1bb4 | ||
|
|
97385c98fc | ||
|
|
74e3cd4c6f | ||
|
|
205a6952a2 | ||
|
|
dd593d9f25 | ||
|
|
5d23420553 | ||
|
|
20bec70160 | ||
|
|
9b5f088793 | ||
|
|
3a561a70db | ||
|
|
11e33ec657 | ||
|
|
d1926725d3 | ||
|
|
2f9a4e1618 |
1
.gitignore
vendored
1
.gitignore
vendored
@@ -145,3 +145,4 @@ cradle*
|
||||
debug*
|
||||
private*
|
||||
crazy_functions/test_project/pdf_and_word
|
||||
crazy_functions/test_samples
|
||||
|
||||
137
README.md
137
README.md
@@ -1,10 +1,15 @@
|
||||
|
||||
> **Note**
|
||||
>
|
||||
> 本项目依赖的Gradio组件的新版pip包(Gradio 3.26~3.27)有严重bug。所以,请在安装时严格选择requirements.txt中**指定的版本**。
|
||||
>
|
||||
> `pip install -r requirements.txt -i https://mirrors.aliyun.com/pypi/simple/`
|
||||
>
|
||||
|
||||
# <img src="docs/logo.png" width="40" > ChatGPT 学术优化
|
||||
|
||||
**如果喜欢这个项目,请给它一个Star;如果你发明了更好用的快捷键或函数插件,欢迎发issue或者pull requests**
|
||||
|
||||
If you like this project, please give it a Star. If you've come up with more useful academic shortcuts or functional plugins, feel free to open an issue or pull request. We also have a [README in English](img/README_EN.md) translated by this project itself.
|
||||
If you like this project, please give it a Star. If you've come up with more useful academic shortcuts or functional plugins, feel free to open an issue or pull request. We also have a README in [English|](docs/README_EN.md)[日本語|](docs/README_JP.md)[Русский|](docs/README_RS.md)[Français](docs/README_FR.md) translated by this project itself.
|
||||
|
||||
> **Note**
|
||||
>
|
||||
@@ -12,7 +17,7 @@ If you like this project, please give it a Star. If you've come up with more use
|
||||
>
|
||||
> 2.本项目中每个文件的功能都在自译解[`self_analysis.md`](https://github.com/binary-husky/chatgpt_academic/wiki/chatgpt-academic%E9%A1%B9%E7%9B%AE%E8%87%AA%E8%AF%91%E8%A7%A3%E6%8A%A5%E5%91%8A)详细说明。随着版本的迭代,您也可以随时自行点击相关函数插件,调用GPT重新生成项目的自我解析报告。常见问题汇总在[`wiki`](https://github.com/binary-husky/chatgpt_academic/wiki/%E5%B8%B8%E8%A7%81%E9%97%AE%E9%A2%98)当中。
|
||||
>
|
||||
|
||||
> 3.已支持OpenAI和API2D的api-key共存,可在配置文件中填写如`API_KEY="openai-key1,openai-key2,api2d-key3"`。需要临时更换`API_KEY`时,在输入区输入临时的`API_KEY`然后回车键提交后即可生效。
|
||||
|
||||
<div align="center">
|
||||
|
||||
@@ -20,30 +25,33 @@ If you like this project, please give it a Star. If you've come up with more use
|
||||
--- | ---
|
||||
一键润色 | 支持一键润色、一键查找论文语法错误
|
||||
一键中英互译 | 一键中英互译
|
||||
一键代码解释 | 可以正确显示代码、解释代码
|
||||
一键代码解释 | 显示代码、解释代码、生成代码、给代码加注释
|
||||
[自定义快捷键](https://www.bilibili.com/video/BV14s4y1E7jN) | 支持自定义快捷键
|
||||
[配置代理服务器](https://www.bilibili.com/video/BV1rc411W7Dr) | 支持配置代理服务器
|
||||
模块化设计 | 支持自定义高阶的函数插件与[函数插件],插件支持[热更新](https://github.com/binary-husky/chatgpt_academic/wiki/%E5%87%BD%E6%95%B0%E6%8F%92%E4%BB%B6%E6%8C%87%E5%8D%97)
|
||||
[配置代理服务器](https://www.bilibili.com/video/BV1rc411W7Dr) | 支持代理连接OpenAI/Google等,秒解锁ChatGPT互联网[实时信息聚合](https://www.bilibili.com/video/BV1om4y127ck/)能力
|
||||
模块化设计 | 支持自定义强大的[函数插件](https://github.com/binary-husky/chatgpt_academic/tree/master/crazy_functions),插件支持[热更新](https://github.com/binary-husky/chatgpt_academic/wiki/%E5%87%BD%E6%95%B0%E6%8F%92%E4%BB%B6%E6%8C%87%E5%8D%97)
|
||||
[自我程序剖析](https://www.bilibili.com/video/BV1cj411A7VW) | [函数插件] [一键读懂](https://github.com/binary-husky/chatgpt_academic/wiki/chatgpt-academic%E9%A1%B9%E7%9B%AE%E8%87%AA%E8%AF%91%E8%A7%A3%E6%8A%A5%E5%91%8A)本项目的源代码
|
||||
[程序剖析](https://www.bilibili.com/video/BV1cj411A7VW) | [函数插件] 一键可以剖析其他Python/C/C++/Java/Lua/...项目树
|
||||
读论文 | [函数插件] 一键解读latex论文全文并生成摘要
|
||||
Latex全文翻译、润色 | [函数插件] 一键翻译或润色latex论文
|
||||
读论文、翻译论文 | [函数插件] 一键解读latex/pdf论文全文并生成摘要
|
||||
Latex全文[翻译](https://www.bilibili.com/video/BV1nk4y1Y7Js/)、[润色](https://www.bilibili.com/video/BV1FT411H7c5/) | [函数插件] 一键翻译或润色latex论文
|
||||
批量注释生成 | [函数插件] 一键批量生成函数注释
|
||||
Markdown[中英互译](https://www.bilibili.com/video/BV1yo4y157jV/) | [函数插件] 看到上面5种语言的[README](https://github.com/binary-husky/chatgpt_academic/blob/master/docs/README_EN.md)了吗?
|
||||
chat分析报告生成 | [函数插件] 运行后自动生成总结汇报
|
||||
[arxiv小助手](https://www.bilibili.com/video/BV1LM4y1279X) | [函数插件] 输入arxiv文章url即可一键翻译摘要+下载PDF
|
||||
[PDF论文全文翻译功能](https://www.bilibili.com/video/BV1KT411x7Wn) | [函数插件] PDF论文提取题目&摘要+翻译全文(多线程)
|
||||
[谷歌学术统合小助手](https://www.bilibili.com/video/BV19L411U7ia) | [函数插件] 给定任意谷歌学术搜索页面URL,让gpt帮你选择有趣的文章
|
||||
公式/图片/表格显示 | 可以同时显示公式的tex形式和渲染形式,支持公式、代码高亮
|
||||
多线程函数插件支持 | 支持多线调用chatgpt,一键处理海量文本或程序
|
||||
[Arxiv小助手](https://www.bilibili.com/video/BV1LM4y1279X) | [函数插件] 输入arxiv文章url即可一键翻译摘要+下载PDF
|
||||
[谷歌学术统合小助手](https://www.bilibili.com/video/BV19L411U7ia) | [函数插件] 给定任意谷歌学术搜索页面URL,让gpt帮你[写relatedworks](https://www.bilibili.com/video/BV1GP411U7Az/)
|
||||
互联网信息聚合+GPT | [函数插件] 一键让ChatGPT先Google搜索,再回答问题,信息流永不过时
|
||||
公式/图片/表格显示 | 可以同时显示公式的[tex形式和渲染形式](https://user-images.githubusercontent.com/96192199/230598842-1d7fcddd-815d-40ee-af60-baf488a199df.png),支持公式、代码高亮
|
||||
多线程函数插件支持 | 支持多线调用chatgpt,一键处理[海量文本](https://www.bilibili.com/video/BV1FT411H7c5/)或程序
|
||||
启动暗色gradio[主题](https://github.com/binary-husky/chatgpt_academic/issues/173) | 在浏览器url后面添加```/?__dark-theme=true```可以切换dark主题
|
||||
[多LLM模型](https://www.bilibili.com/video/BV1wT411p7yf)支持,[API2D](https://api2d.com/)接口支持 | 同时被GPT3.5、GPT4和[清华ChatGLM](https://github.com/THUDM/ChatGLM-6B)伺候的感觉一定会很不错吧?
|
||||
更多LLM模型接入 | 新加入Newbing测试接口(新必应AI)
|
||||
huggingface免科学上网[在线体验](https://huggingface.co/spaces/qingxu98/gpt-academic) | 登陆huggingface后复制[此空间](https://huggingface.co/spaces/qingxu98/gpt-academic)
|
||||
…… | ……
|
||||
|
||||
</div>
|
||||
|
||||
|
||||
- 新界面(修改config.py中的LAYOUT选项即可实现“左右布局”和“上下布局”的切换)
|
||||
- 新界面(修改`config.py`中的LAYOUT选项即可实现“左右布局”和“上下布局”的切换)
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/230361456-61078362-a966-4eb5-b49e-3c62ef18b860.gif" width="700" >
|
||||
</div>
|
||||
@@ -91,8 +99,8 @@ cd chatgpt_academic
|
||||
|
||||
在`config.py`中,配置 海外Proxy 和 OpenAI API KEY,说明如下
|
||||
```
|
||||
1. 如果你在国内,需要设置海外代理才能够顺利使用 OpenAI API,设置方法请仔细阅读config.py(1.修改其中的USE_PROXY为True; 2.按照说明修改其中的proxies)。
|
||||
2. 配置 OpenAI API KEY。你需要在 OpenAI 官网上注册并获取 API KEY。一旦你拿到了 API KEY,在 config.py 文件里配置好即可。
|
||||
1. 如果你在国内,需要设置海外代理才能够顺利使用OpenAI API,设置方法请仔细阅读config.py(1.修改其中的USE_PROXY为True; 2.按照说明修改其中的proxies)。
|
||||
2. 配置 OpenAI API KEY。支持任意数量的OpenAI的密钥和API2D的密钥共存/负载均衡,多个KEY用英文逗号分隔即可,例如输入 API_KEY="OpenAI密钥1,API2D密钥2,OpenAI密钥3,OpenAI密钥4"
|
||||
3. 与代理网络有关的issue(网络超时、代理不起作用)汇总到 https://github.com/binary-husky/chatgpt_academic/issues/1
|
||||
```
|
||||
(P.S. 程序运行时会优先检查是否存在名为`config_private.py`的私密配置文件,并用其中的配置覆盖`config.py`的同名配置。因此,如果您能理解我们的配置读取逻辑,我们强烈建议您在`config.py`旁边创建一个名为`config_private.py`的新配置文件,并把`config.py`中的配置转移(复制)到`config_private.py`中。`config_private.py`不受git管控,可以让您的隐私信息更加安全。)
|
||||
@@ -100,19 +108,17 @@ cd chatgpt_academic
|
||||
|
||||
3. 安装依赖
|
||||
```sh
|
||||
# (选择一)推荐
|
||||
python -m pip install -r requirements.txt
|
||||
# (选择I: 如熟悉python)推荐
|
||||
python -m pip install -r requirements.txt
|
||||
# 备注:使用官方pip源或者阿里pip源,其他pip源(如一些大学的pip)有可能出问题,临时换源方法:python -m pip install -r requirements.txt -i https://mirrors.aliyun.com/pypi/simple/
|
||||
|
||||
# (选择二)如果您使用anaconda,步骤也是类似的:
|
||||
# (选择二.1)conda create -n gptac_venv python=3.11
|
||||
# (选择二.2)conda activate gptac_venv
|
||||
# (选择二.3)python -m pip install -r requirements.txt
|
||||
|
||||
# 备注:使用官方pip源或者阿里pip源,其他pip源(如一些大学的pip)有可能出问题,临时换源方法:
|
||||
# python -m pip install -r requirements.txt -i https://mirrors.aliyun.com/pypi/simple/
|
||||
# (选择II: 如不熟悉python)使用anaconda,步骤也是类似的:
|
||||
# (II-1)conda create -n gptac_venv python=3.11
|
||||
# (II-2)conda activate gptac_venv
|
||||
# (II-3)python -m pip install -r requirements.txt
|
||||
```
|
||||
|
||||
如果需要支持清华ChatGLM,需要额外安装更多依赖(不熟悉python者、电脑配置不佳者,建议不要尝试):
|
||||
如果需要支持清华ChatGLM后端,需要额外安装更多依赖(前提条件:熟悉python + 电脑配置够强):
|
||||
```sh
|
||||
python -m pip install -r request_llm/requirements_chatglm.txt
|
||||
```
|
||||
@@ -125,54 +131,48 @@ python main.py
|
||||
5. 测试函数插件
|
||||
```
|
||||
- 测试Python项目分析
|
||||
input区域 输入 `./crazy_functions/test_project/python/dqn` , 然后点击 "解析整个Python项目"
|
||||
- 测试自我代码解读
|
||||
(选择1)input区域 输入 `./crazy_functions/test_project/python/dqn` , 然后点击 "解析整个Python项目"
|
||||
(选择2)展开文件上传区,将python文件/包含python文件的压缩包拖拽进去,在出现反馈提示后, 然后点击 "解析整个Python项目"
|
||||
- 测试自我代码解读(本项目自译解)
|
||||
点击 "[多线程Demo] 解析此项目本身(源码自译解)"
|
||||
- 测试实验功能模板函数(要求gpt回答历史上的今天发生了什么),您可以根据此函数为模板,实现更复杂的功能
|
||||
- 测试函数插件模板函数(要求gpt回答历史上的今天发生了什么),您可以根据此函数为模板,实现更复杂的功能
|
||||
点击 "[函数插件模板Demo] 历史上的今天"
|
||||
- 函数插件区下拉菜单中有更多功能可供选择
|
||||
```
|
||||
|
||||
## 安装-方法2:使用docker (Linux)
|
||||
## 安装-方法2:使用Docker
|
||||
|
||||
1. 仅ChatGPT(推荐大多数人选择)
|
||||
|
||||
``` sh
|
||||
# 下载项目
|
||||
git clone https://github.com/binary-husky/chatgpt_academic.git
|
||||
cd chatgpt_academic
|
||||
# 配置 海外Proxy 和 OpenAI API KEY
|
||||
# 配置 “海外Proxy”, “API_KEY” 以及 “WEB_PORT” (例如50923) 等
|
||||
用任意文本编辑器编辑 config.py
|
||||
# 安装
|
||||
docker build -t gpt-academic .
|
||||
# 运行
|
||||
#(最后一步-选择1)在Linux环境下,用`--net=host`更方便快捷
|
||||
docker run --rm -it --net=host gpt-academic
|
||||
|
||||
# 测试函数插件
|
||||
## 测试函数插件模板函数(要求gpt回答历史上的今天发生了什么),您可以根据此函数为模板,实现更复杂的功能
|
||||
点击 "[函数插件模板Demo] 历史上的今天"
|
||||
## 测试给Latex项目写摘要
|
||||
input区域 输入 ./crazy_functions/test_project/latex/attention , 然后点击 "读Tex论文写摘要"
|
||||
## 测试Python项目分析
|
||||
input区域 输入 ./crazy_functions/test_project/python/dqn , 然后点击 "解析整个Python项目"
|
||||
|
||||
函数插件区下拉菜单中有更多功能可供选择
|
||||
#(最后一步-选择2)在macOS/windows环境下,只能用-p选项将容器上的端口(例如50923)暴露给主机上的端口
|
||||
docker run --rm -it -p 50923:50923 gpt-academic
|
||||
```
|
||||
|
||||
2. ChatGPT+ChatGLM(需要对docker非常熟悉 + 电脑配置足够强)
|
||||
2. ChatGPT+ChatGLM(需要对Docker熟悉 + 读懂Dockerfile + 电脑配置够强)
|
||||
|
||||
``` sh
|
||||
# 修改dockerfile
|
||||
# 修改Dockerfile
|
||||
cd docs && nano Dockerfile+ChatGLM
|
||||
# How to build | 如何构建 (Dockerfile+ChatGLM在docs路径下,请先cd docs)
|
||||
# 构建 (Dockerfile+ChatGLM在docs路径下,请先cd docs)
|
||||
docker build -t gpt-academic --network=host -f Dockerfile+ChatGLM .
|
||||
# How to run | 如何运行 (1) 直接运行:
|
||||
# 运行 (1) 直接运行:
|
||||
docker run --rm -it --net=host --gpus=all gpt-academic
|
||||
# How to run | 如何运行 (2) 我想运行之前进容器做一些调整:
|
||||
# 运行 (2) 我想运行之前进容器做一些调整:
|
||||
docker run --rm -it --net=host --gpus=all gpt-academic bash
|
||||
```
|
||||
|
||||
|
||||
## 安装-方法3:其他部署方式
|
||||
## 安装-方法3:其他部署方式(需要云服务器知识与经验)
|
||||
|
||||
1. 远程云服务器部署
|
||||
请访问[部署wiki-1](https://github.com/binary-husky/chatgpt_academic/wiki/%E4%BA%91%E6%9C%8D%E5%8A%A1%E5%99%A8%E8%BF%9C%E7%A8%8B%E9%83%A8%E7%BD%B2%E6%8C%87%E5%8D%97)
|
||||
@@ -180,6 +180,8 @@ docker run --rm -it --net=host --gpus=all gpt-academic bash
|
||||
2. 使用WSL2(Windows Subsystem for Linux 子系统)
|
||||
请访问[部署wiki-2](https://github.com/binary-husky/chatgpt_academic/wiki/%E4%BD%BF%E7%94%A8WSL2%EF%BC%88Windows-Subsystem-for-Linux-%E5%AD%90%E7%B3%BB%E7%BB%9F%EF%BC%89%E9%83%A8%E7%BD%B2)
|
||||
|
||||
3. 如何在二级网址(如`http://localhost/subpath`)下运行
|
||||
请访问[FastAPI运行说明](docs/WithFastapi.md)
|
||||
|
||||
## 安装-代理配置
|
||||
1. 常规方法
|
||||
@@ -191,7 +193,9 @@ docker run --rm -it --net=host --gpus=all gpt-academic bash
|
||||
|
||||
---
|
||||
|
||||
## 自定义新的便捷按钮(学术快捷键自定义)
|
||||
## 自定义新的便捷按钮 / 自定义函数插件
|
||||
|
||||
1. 自定义新的便捷按钮(学术快捷键)
|
||||
任意文本编辑器打开`core_functional.py`,添加条目如下,然后重启程序即可。(如果按钮已经添加成功并可见,那么前缀、后缀都支持热修改,无需重启程序即可生效。)
|
||||
例如
|
||||
```
|
||||
@@ -207,19 +211,25 @@ docker run --rm -it --net=host --gpus=all gpt-academic bash
|
||||
<img src="https://user-images.githubusercontent.com/96192199/226899272-477c2134-ed71-4326-810c-29891fe4a508.png" width="500" >
|
||||
</div>
|
||||
|
||||
2. 自定义函数插件
|
||||
|
||||
编写强大的函数插件来执行任何你想得到的和想不到的任务。
|
||||
本项目的插件编写、调试难度很低,只要您具备一定的python基础知识,就可以仿照我们提供的模板实现自己的插件功能。
|
||||
详情请参考[函数插件指南](https://github.com/binary-husky/chatgpt_academic/wiki/%E5%87%BD%E6%95%B0%E6%8F%92%E4%BB%B6%E6%8C%87%E5%8D%97)。
|
||||
|
||||
|
||||
---
|
||||
|
||||
|
||||
## 部分功能展示
|
||||
|
||||
### 图片显示:
|
||||
1. 图片显示:
|
||||
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/228737599-bf0a9d9c-1808-4f43-ae15-dfcc7af0f295.png" width="800" >
|
||||
</div>
|
||||
|
||||
|
||||
### 如果一个程序能够读懂并剖析自己:
|
||||
2. 本项目的代码自译解(如果一个程序能够读懂并剖析自己):
|
||||
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/226936850-c77d7183-0749-4c1c-9875-fd4891842d0c.png" width="800" >
|
||||
@@ -229,7 +239,7 @@ docker run --rm -it --net=host --gpus=all gpt-academic bash
|
||||
<img src="https://user-images.githubusercontent.com/96192199/226936618-9b487e4b-ab5b-4b6e-84c6-16942102e917.png" width="800" >
|
||||
</div>
|
||||
|
||||
### 其他任意Python/Cpp项目剖析:
|
||||
3. 其他任意Python/Cpp/Java/Go/Rect/...项目剖析:
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/226935232-6b6a73ce-8900-4aee-93f9-733c7e6fef53.png" width="800" >
|
||||
</div>
|
||||
@@ -238,33 +248,44 @@ docker run --rm -it --net=host --gpus=all gpt-academic bash
|
||||
<img src="https://user-images.githubusercontent.com/96192199/226969067-968a27c1-1b9c-486b-8b81-ab2de8d3f88a.png" width="800" >
|
||||
</div>
|
||||
|
||||
### Latex论文一键阅读理解与摘要生成
|
||||
4. Latex论文一键阅读理解与摘要生成
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/227504406-86ab97cd-f208-41c3-8e4a-7000e51cf980.png" width="800" >
|
||||
</div>
|
||||
|
||||
### 自动报告生成
|
||||
5. 自动报告生成
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/227503770-fe29ce2c-53fd-47b0-b0ff-93805f0c2ff4.png" height="300" >
|
||||
<img src="https://user-images.githubusercontent.com/96192199/227504617-7a497bb3-0a2a-4b50-9a8a-95ae60ea7afd.png" height="300" >
|
||||
<img src="https://user-images.githubusercontent.com/96192199/227504005-efeaefe0-b687-49d0-bf95-2d7b7e66c348.png" height="300" >
|
||||
</div>
|
||||
|
||||
### 模块化功能设计
|
||||
6. 模块化功能设计
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/229288270-093643c1-0018-487a-81e6-1d7809b6e90f.png" height="400" >
|
||||
<img src="https://user-images.githubusercontent.com/96192199/227504931-19955f78-45cd-4d1c-adac-e71e50957915.png" height="400" >
|
||||
</div>
|
||||
|
||||
|
||||
### 源代码转译英文
|
||||
7. 源代码转译英文
|
||||
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/229720562-fe6c3508-6142-4635-a83d-21eb3669baee.png" height="400" >
|
||||
</div>
|
||||
|
||||
8. 互联网在线信息综合
|
||||
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/233575247-fb00819e-6d1b-4bb7-bd54-1d7528f03dd9.png" width="800" >
|
||||
<img src="https://user-images.githubusercontent.com/96192199/233779501-5ce826f0-6cca-4d59-9e5f-b4eacb8cc15f.png" width="800" >
|
||||
|
||||
</div>
|
||||
|
||||
|
||||
|
||||
## Todo 与 版本规划:
|
||||
- version 3.2+ (todo): 函数插件支持更多参数接口
|
||||
- version 3.3+ (todo): NewBing支持
|
||||
- version 3.2: 函数插件支持更多参数接口 (保存对话功能, 解读任意语言代码+同时询问任意的LLM组合)
|
||||
- version 3.1: 支持同时问询多个gpt模型!支持api2d,支持多个apikey负载均衡
|
||||
- version 3.0: 对chatglm和其他小型llm的支持
|
||||
- version 2.6: 重构了插件结构,提高了交互性,加入更多插件
|
||||
@@ -276,6 +297,8 @@ docker run --rm -it --net=host --gpus=all gpt-academic bash
|
||||
- version 2.0: 引入模块化函数插件
|
||||
- version 1.0: 基础功能
|
||||
|
||||
chatgpt_academic开发者QQ群:734063350
|
||||
|
||||
## 参考与学习
|
||||
|
||||
```
|
||||
|
||||
15
config.py
15
config.py
@@ -45,7 +45,7 @@ MAX_RETRY = 2
|
||||
|
||||
# OpenAI模型选择是(gpt4现在只对申请成功的人开放,体验gpt-4可以试试api2d)
|
||||
LLM_MODEL = "gpt-3.5-turbo" # 可选 ↓↓↓
|
||||
AVAIL_LLM_MODELS = ["gpt-3.5-turbo", "api2d-gpt-3.5-turbo", "gpt-4", "api2d-gpt-4", "chatglm"]
|
||||
AVAIL_LLM_MODELS = ["gpt-3.5-turbo", "api2d-gpt-3.5-turbo", "gpt-4", "api2d-gpt-4", "chatglm", "newbing"]
|
||||
|
||||
# 本地LLM模型如ChatGLM的执行方式 CPU/GPU
|
||||
LOCAL_MODEL_DEVICE = "cpu" # 可选 "cuda"
|
||||
@@ -56,3 +56,16 @@ CONCURRENT_COUNT = 100
|
||||
# 设置用户名和密码(不需要修改)(相关功能不稳定,与gradio版本和网络都相关,如果本地使用不建议加这个)
|
||||
# [("username", "password"), ("username2", "password2"), ...]
|
||||
AUTHENTICATION = []
|
||||
|
||||
# 重新URL重新定向,实现更换API_URL的作用(常规情况下,不要修改!!)
|
||||
# 格式 {"https://api.openai.com/v1/chat/completions": "在这里填写重定向的api.openai.com的URL"}
|
||||
API_URL_REDIRECT = {}
|
||||
|
||||
# 如果需要在二级路径下运行(常规情况下,不要修改!!)(需要配合修改main.py才能生效!)
|
||||
CUSTOM_PATH = "/"
|
||||
|
||||
# 如果需要使用newbing,把newbing的长长的cookie放到这里
|
||||
NEWBING_STYLE = "creative" # ["creative", "balanced", "precise"]
|
||||
NEWBING_COOKIES = """
|
||||
your bing cookies here
|
||||
"""
|
||||
@@ -19,12 +19,25 @@ def get_crazy_functions():
|
||||
from crazy_functions.解析项目源代码 import 解析一个Lua项目
|
||||
from crazy_functions.解析项目源代码 import 解析一个CSharp项目
|
||||
from crazy_functions.总结word文档 import 总结word文档
|
||||
from crazy_functions.解析JupyterNotebook import 解析ipynb文件
|
||||
from crazy_functions.对话历史存档 import 对话历史存档
|
||||
function_plugins = {
|
||||
|
||||
"解析整个Python项目": {
|
||||
"Color": "stop", # 按钮颜色
|
||||
"Function": HotReload(解析一个Python项目)
|
||||
},
|
||||
"保存当前的对话": {
|
||||
"AsButton":False,
|
||||
"Function": HotReload(对话历史存档)
|
||||
},
|
||||
"[测试功能] 解析Jupyter Notebook文件": {
|
||||
"Color": "stop",
|
||||
"AsButton":False,
|
||||
"Function": HotReload(解析ipynb文件),
|
||||
"AdvancedArgs": True, # 调用时,唤起高级参数输入区(默认False)
|
||||
"ArgsReminder": "若输入0,则不解析notebook中的Markdown块", # 高级参数输入区的显示提示
|
||||
},
|
||||
"批量总结Word文档": {
|
||||
"Color": "stop",
|
||||
"Function": HotReload(总结word文档)
|
||||
@@ -168,25 +181,48 @@ def get_crazy_functions():
|
||||
"AsButton": False, # 加入下拉菜单中
|
||||
"Function": HotReload(Markdown英译中)
|
||||
},
|
||||
|
||||
|
||||
})
|
||||
|
||||
###################### 第三组插件 ###########################
|
||||
# [第三组插件]: 尚未充分测试的函数插件,放在这里
|
||||
try:
|
||||
from crazy_functions.下载arxiv论文翻译摘要 import 下载arxiv论文并翻译摘要
|
||||
function_plugins.update({
|
||||
"一键下载arxiv论文并翻译摘要(先在input输入编号,如1812.10695)": {
|
||||
"Color": "stop",
|
||||
"AsButton": False, # 加入下拉菜单中
|
||||
"Function": HotReload(下载arxiv论文并翻译摘要)
|
||||
}
|
||||
})
|
||||
|
||||
except Exception as err:
|
||||
print(f'[下载arxiv论文并翻译摘要] 插件导入失败 {str(err)}')
|
||||
|
||||
from crazy_functions.下载arxiv论文翻译摘要 import 下载arxiv论文并翻译摘要
|
||||
function_plugins.update({
|
||||
"一键下载arxiv论文并翻译摘要(先在input输入编号,如1812.10695)": {
|
||||
"Color": "stop",
|
||||
"AsButton": False, # 加入下拉菜单中
|
||||
"Function": HotReload(下载arxiv论文并翻译摘要)
|
||||
}
|
||||
})
|
||||
|
||||
from crazy_functions.联网的ChatGPT import 连接网络回答问题
|
||||
function_plugins.update({
|
||||
"连接网络回答问题(先输入问题,再点击按钮,需要访问谷歌)": {
|
||||
"Color": "stop",
|
||||
"AsButton": False, # 加入下拉菜单中
|
||||
"Function": HotReload(连接网络回答问题)
|
||||
}
|
||||
})
|
||||
|
||||
from crazy_functions.解析项目源代码 import 解析任意code项目
|
||||
function_plugins.update({
|
||||
"解析项目源代码(手动指定和筛选源代码文件类型)": {
|
||||
"Color": "stop",
|
||||
"AsButton": False,
|
||||
"AdvancedArgs": True, # 调用时,唤起高级参数输入区(默认False)
|
||||
"ArgsReminder": "输入时用逗号隔开, *代表通配符, 加了^代表不匹配; 不输入代表全部匹配。例如: \"*.c, ^*.cpp, config.toml, ^*.toml\"", # 高级参数输入区的显示提示
|
||||
"Function": HotReload(解析任意code项目)
|
||||
},
|
||||
})
|
||||
from crazy_functions.询问多个大语言模型 import 同时问询_指定模型
|
||||
function_plugins.update({
|
||||
"询问多个GPT模型(手动指定询问哪些模型)": {
|
||||
"Color": "stop",
|
||||
"AsButton": False,
|
||||
"AdvancedArgs": True, # 调用时,唤起高级参数输入区(默认False)
|
||||
"ArgsReminder": "支持任意数量的llm接口,用&符号分隔。例如chatglm&gpt-3.5-turbo&api2d-gpt-4", # 高级参数输入区的显示提示
|
||||
"Function": HotReload(同时问询_指定模型)
|
||||
},
|
||||
})
|
||||
###################### 第n组插件 ###########################
|
||||
return function_plugins
|
||||
|
||||
@@ -12,7 +12,7 @@ def validate_path():
|
||||
sys.path.append(root_dir_assume)
|
||||
|
||||
validate_path() # validate path so you can run from base directory
|
||||
|
||||
from colorful import *
|
||||
from toolbox import get_conf, ChatBotWithCookies
|
||||
proxies, WEB_PORT, LLM_MODEL, CONCURRENT_COUNT, AUTHENTICATION, CHATBOT_HEIGHT, LAYOUT, API_KEY = \
|
||||
get_conf('proxies', 'WEB_PORT', 'LLM_MODEL', 'CONCURRENT_COUNT', 'AUTHENTICATION', 'CHATBOT_HEIGHT', 'LAYOUT', 'API_KEY')
|
||||
@@ -79,14 +79,52 @@ def test_下载arxiv论文并翻译摘要():
|
||||
for cookies, cb, hist, msg in 下载arxiv论文并翻译摘要(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, web_port):
|
||||
print(cb)
|
||||
|
||||
test_解析一个Python项目()
|
||||
test_Latex英文润色()
|
||||
test_Markdown中译英()
|
||||
test_批量翻译PDF文档()
|
||||
test_谷歌检索小助手()
|
||||
test_总结word文档()
|
||||
test_下载arxiv论文并翻译摘要()
|
||||
test_解析一个Cpp项目()
|
||||
def test_联网回答问题():
|
||||
from crazy_functions.联网的ChatGPT import 连接网络回答问题
|
||||
# txt = "“我们称之为高效”是什么梗?"
|
||||
# >> 从第0份、第1份、第2份搜索结果可以看出,“我们称之为高效”是指在游戏社区中,用户们用来形容一些游戏策略或行为非常高效且能够带来好的效果的用语。这个用语最初可能是在群星(Stellaris)这个游戏里面流行起来的,后来也传播到了其他游戏中,比如巨像(Titan)等游戏。其中第1份搜索结果中的一篇文章也指出,“我们称之为高效”这 一用语来源于群星(Stellaris)游戏中的一个情节。
|
||||
# txt = "为什么说枪毙P社玩家没有一个冤枉的?"
|
||||
# >> 它们都是关于一个知乎用户所发的帖子,引用了一群游戏玩家对于需要对P社玩家进行枪毙的讨论,这个话题的本质是玩家们对于P 社游戏中的政治与历史元素的不同看法,以及其中不少玩家以极端立场宣扬的想法和言论,因此有人就以枪毙这些玩家来回应此类言论。但是这个话题本身并没有实质内容,只是一个玩笑或者恶搞,并不应该被当做真实的态度或者观点,因此这种说法没有实际意义。
|
||||
# txt = "谁是应急食品?"
|
||||
# >> '根据以上搜索结果可以得知,应急食品是“原神”游戏中的角色派蒙的外号。'
|
||||
# txt = "道路千万条,安全第一条。后面两句是?"
|
||||
# >> '行车不规范,亲人两行泪。'
|
||||
# txt = "What is in the canister?"
|
||||
# >> Rainbow Six Siege 游戏中 Smoke 的 Canister 中装有何种物质相关的官方信息。
|
||||
# txt = "失败的man是什么?"
|
||||
# >> 根据第1份搜索结果,可以得知失败的man是指一位在B站购买了蜘蛛侠COS服后穿上后被网友嘲笑的UP主,而“失败的man”是蜘蛛侠英文名“spiderman”的谐音梗,并且网友们还 给这位UP主起了“苍蝇侠”的外号。因此,失败的man是指这位UP主在穿上蜘蛛侠COS服后被网友嘲笑的情况。
|
||||
# txt = "老六是什么,起源于哪里?"
|
||||
# >> 老六是网络流行语,最初起源于游戏《CSGO》,指游戏中玩家中独来独往、游离于队伍之外的“自由人”或玩得比较菜或者玩得比较阴险的人 ,后来逐渐演变成指玩得比较阴险的玩家。
|
||||
# txt = "罗小黑战记因为什么经常被吐槽?"
|
||||
# >> 3. 更新速度。罗小黑战记的更新时间不定,时而快时而慢,给观众留下了等待的时间过长的印象。
|
||||
# txt = "沙特、伊朗最近的关系如何?"
|
||||
# >> 最近在中国的斡旋下,沙特和伊朗于3月10日达成了恢复两国外交关系的协议,这表明两国关系已经重新回到正常化状态。
|
||||
# txt = "You should have gone for the head. What does that mean?"
|
||||
# >> The phrase "You should have gone for the head" is a quote from the Marvel movies, Avengers: Infinity War and Avengers: Endgame. It was spoken by the character Thanos in Infinity War and by Thor in Endgame.
|
||||
txt = "AutoGPT是什么?"
|
||||
# >> AutoGPT是一个基于GPT-4语言模型的开源应用程序。它可以根据用户需求自主执行任务,包括事件分析、营销方案撰写、代码编程、数学运算等等,并完全不需要用户插手。它可以自己思考,给出实现的步骤和实现细节,甚至可以自问自答执 行任务。最近它在GitHub上爆火,成为了业内最热门的项目之一。
|
||||
# txt = "钟离带什么圣遗物?"
|
||||
for cookies, cb, hist, msg in 连接网络回答问题(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, web_port):
|
||||
print("当前问答:", cb[-1][-1].replace("\n"," "))
|
||||
for i, it in enumerate(cb): print亮蓝(it[0]); print亮黄(it[1])
|
||||
|
||||
def test_解析ipynb文件():
|
||||
from crazy_functions.解析JupyterNotebook import 解析ipynb文件
|
||||
txt = "crazy_functions/test_samples"
|
||||
for cookies, cb, hist, msg in 解析ipynb文件(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, web_port):
|
||||
print(cb)
|
||||
|
||||
|
||||
# test_解析一个Python项目()
|
||||
# test_Latex英文润色()
|
||||
# test_Markdown中译英()
|
||||
# test_批量翻译PDF文档()
|
||||
# test_谷歌检索小助手()
|
||||
# test_总结word文档()
|
||||
# test_下载arxiv论文并翻译摘要()
|
||||
# test_解析一个Cpp项目()
|
||||
# test_联网回答问题()
|
||||
test_解析ipynb文件()
|
||||
|
||||
input("程序完成,回车退出。")
|
||||
print("退出。")
|
||||
@@ -1,5 +1,4 @@
|
||||
import traceback
|
||||
from toolbox import update_ui, get_conf
|
||||
from toolbox import update_ui, get_conf, trimmed_format_exc
|
||||
|
||||
def input_clipping(inputs, history, max_token_limit):
|
||||
import numpy as np
|
||||
@@ -94,12 +93,12 @@ def request_gpt_model_in_new_thread_with_ui_alive(
|
||||
continue # 返回重试
|
||||
else:
|
||||
# 【选择放弃】
|
||||
tb_str = '```\n' + traceback.format_exc() + '```'
|
||||
tb_str = '```\n' + trimmed_format_exc() + '```'
|
||||
mutable[0] += f"[Local Message] 警告,在执行过程中遭遇问题, Traceback:\n\n{tb_str}\n\n"
|
||||
return mutable[0] # 放弃
|
||||
except:
|
||||
# 【第三种情况】:其他错误:重试几次
|
||||
tb_str = '```\n' + traceback.format_exc() + '```'
|
||||
tb_str = '```\n' + trimmed_format_exc() + '```'
|
||||
print(tb_str)
|
||||
mutable[0] += f"[Local Message] 警告,在执行过程中遭遇问题, Traceback:\n\n{tb_str}\n\n"
|
||||
if retry_op > 0:
|
||||
@@ -173,7 +172,7 @@ def request_gpt_model_multi_threads_with_very_awesome_ui_and_high_efficiency(
|
||||
if max_workers == -1: # 读取配置文件
|
||||
try: max_workers, = get_conf('DEFAULT_WORKER_NUM')
|
||||
except: max_workers = 8
|
||||
if max_workers <= 0 or max_workers >= 20: max_workers = 8
|
||||
if max_workers <= 0: max_workers = 3
|
||||
# 屏蔽掉 chatglm的多线程,可能会导致严重卡顿
|
||||
if not (llm_kwargs['llm_model'].startswith('gpt-') or llm_kwargs['llm_model'].startswith('api2d-')):
|
||||
max_workers = 1
|
||||
@@ -220,14 +219,14 @@ def request_gpt_model_multi_threads_with_very_awesome_ui_and_high_efficiency(
|
||||
continue # 返回重试
|
||||
else:
|
||||
# 【选择放弃】
|
||||
tb_str = '```\n' + traceback.format_exc() + '```'
|
||||
tb_str = '```\n' + trimmed_format_exc() + '```'
|
||||
gpt_say += f"[Local Message] 警告,线程{index}在执行过程中遭遇问题, Traceback:\n\n{tb_str}\n\n"
|
||||
if len(mutable[index][0]) > 0: gpt_say += "此线程失败前收到的回答:\n\n" + mutable[index][0]
|
||||
mutable[index][2] = "输入过长已放弃"
|
||||
return gpt_say # 放弃
|
||||
except:
|
||||
# 【第三种情况】:其他错误
|
||||
tb_str = '```\n' + traceback.format_exc() + '```'
|
||||
tb_str = '```\n' + trimmed_format_exc() + '```'
|
||||
print(tb_str)
|
||||
gpt_say += f"[Local Message] 警告,线程{index}在执行过程中遭遇问题, Traceback:\n\n{tb_str}\n\n"
|
||||
if len(mutable[index][0]) > 0: gpt_say += "此线程失败前收到的回答:\n\n" + mutable[index][0]
|
||||
|
||||
42
crazy_functions/对话历史存档.py
Normal file
42
crazy_functions/对话历史存档.py
Normal file
@@ -0,0 +1,42 @@
|
||||
from toolbox import CatchException, update_ui
|
||||
from .crazy_utils import request_gpt_model_in_new_thread_with_ui_alive
|
||||
|
||||
def write_chat_to_file(chatbot, file_name=None):
|
||||
"""
|
||||
将对话记录history以Markdown格式写入文件中。如果没有指定文件名,则使用当前时间生成文件名。
|
||||
"""
|
||||
import os
|
||||
import time
|
||||
if file_name is None:
|
||||
file_name = 'chatGPT对话历史' + time.strftime("%Y-%m-%d-%H-%M-%S", time.localtime()) + '.html'
|
||||
os.makedirs('./gpt_log/', exist_ok=True)
|
||||
with open(f'./gpt_log/{file_name}', 'w', encoding='utf8') as f:
|
||||
for i, contents in enumerate(chatbot):
|
||||
for content in contents:
|
||||
try: # 这个bug没找到触发条件,暂时先这样顶一下
|
||||
if type(content) != str: content = str(content)
|
||||
except:
|
||||
continue
|
||||
f.write(content)
|
||||
f.write('\n\n')
|
||||
f.write('<hr color="red"> \n\n')
|
||||
|
||||
res = '对话历史写入:' + os.path.abspath(f'./gpt_log/{file_name}')
|
||||
print(res)
|
||||
return res
|
||||
|
||||
@CatchException
|
||||
def 对话历史存档(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, web_port):
|
||||
"""
|
||||
txt 输入栏用户输入的文本,例如需要翻译的一段话,再例如一个包含了待处理文件的路径
|
||||
llm_kwargs gpt模型参数,如温度和top_p等,一般原样传递下去就行
|
||||
plugin_kwargs 插件模型的参数,暂时没有用武之地
|
||||
chatbot 聊天显示框的句柄,用于显示给用户
|
||||
history 聊天历史,前情提要
|
||||
system_prompt 给gpt的静默提醒
|
||||
web_port 当前软件运行的端口号
|
||||
"""
|
||||
|
||||
chatbot.append(("保存当前对话", f"[Local Message] {write_chat_to_file(chatbot)}"))
|
||||
yield from update_ui(chatbot=chatbot, history=history) # 刷新界面 # 由于请求gpt需要一段时间,我们先及时地做一次界面更新
|
||||
|
||||
@@ -50,7 +50,7 @@ def 多文件翻译(file_manifest, project_folder, llm_kwargs, plugin_kwargs, ch
|
||||
pfg.file_contents.append(file_content)
|
||||
|
||||
# <-------- 拆分过长的Markdown文件 ---------->
|
||||
pfg.run_file_split(max_token_limit=2048)
|
||||
pfg.run_file_split(max_token_limit=1500)
|
||||
n_split = len(pfg.sp_file_contents)
|
||||
|
||||
# <-------- 多线程润色开始 ---------->
|
||||
|
||||
102
crazy_functions/联网的ChatGPT.py
Normal file
102
crazy_functions/联网的ChatGPT.py
Normal file
@@ -0,0 +1,102 @@
|
||||
from toolbox import CatchException, update_ui
|
||||
from .crazy_utils import request_gpt_model_in_new_thread_with_ui_alive, input_clipping
|
||||
import requests
|
||||
from bs4 import BeautifulSoup
|
||||
from request_llm.bridge_all import model_info
|
||||
|
||||
def google(query, proxies):
|
||||
query = query # 在此处替换您要搜索的关键词
|
||||
url = f"https://www.google.com/search?q={query}"
|
||||
headers = {'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.61 Safari/537.36'}
|
||||
response = requests.get(url, headers=headers, proxies=proxies)
|
||||
soup = BeautifulSoup(response.content, 'html.parser')
|
||||
results = []
|
||||
for g in soup.find_all('div', class_='g'):
|
||||
anchors = g.find_all('a')
|
||||
if anchors:
|
||||
link = anchors[0]['href']
|
||||
if link.startswith('/url?q='):
|
||||
link = link[7:]
|
||||
if not link.startswith('http'):
|
||||
continue
|
||||
title = g.find('h3').text
|
||||
item = {'title': title, 'link': link}
|
||||
results.append(item)
|
||||
|
||||
for r in results:
|
||||
print(r['link'])
|
||||
return results
|
||||
|
||||
def scrape_text(url, proxies) -> str:
|
||||
"""Scrape text from a webpage
|
||||
|
||||
Args:
|
||||
url (str): The URL to scrape text from
|
||||
|
||||
Returns:
|
||||
str: The scraped text
|
||||
"""
|
||||
headers = {
|
||||
'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.61 Safari/537.36',
|
||||
'Content-Type': 'text/plain',
|
||||
}
|
||||
try:
|
||||
response = requests.get(url, headers=headers, proxies=proxies, timeout=8)
|
||||
if response.encoding == "ISO-8859-1": response.encoding = response.apparent_encoding
|
||||
except:
|
||||
return "无法连接到该网页"
|
||||
soup = BeautifulSoup(response.text, "html.parser")
|
||||
for script in soup(["script", "style"]):
|
||||
script.extract()
|
||||
text = soup.get_text()
|
||||
lines = (line.strip() for line in text.splitlines())
|
||||
chunks = (phrase.strip() for line in lines for phrase in line.split(" "))
|
||||
text = "\n".join(chunk for chunk in chunks if chunk)
|
||||
return text
|
||||
|
||||
@CatchException
|
||||
def 连接网络回答问题(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, web_port):
|
||||
"""
|
||||
txt 输入栏用户输入的文本,例如需要翻译的一段话,再例如一个包含了待处理文件的路径
|
||||
llm_kwargs gpt模型参数,如温度和top_p等,一般原样传递下去就行
|
||||
plugin_kwargs 插件模型的参数,暂时没有用武之地
|
||||
chatbot 聊天显示框的句柄,用于显示给用户
|
||||
history 聊天历史,前情提要
|
||||
system_prompt 给gpt的静默提醒
|
||||
web_port 当前软件运行的端口号
|
||||
"""
|
||||
history = [] # 清空历史,以免输入溢出
|
||||
chatbot.append((f"请结合互联网信息回答以下问题:{txt}",
|
||||
"[Local Message] 请注意,您正在调用一个[函数插件]的模板,该模板可以实现ChatGPT联网信息综合。该函数面向希望实现更多有趣功能的开发者,它可以作为创建新功能函数的模板。您若希望分享新的功能模组,请不吝PR!"))
|
||||
yield from update_ui(chatbot=chatbot, history=history) # 刷新界面 # 由于请求gpt需要一段时间,我们先及时地做一次界面更新
|
||||
|
||||
# ------------- < 第1步:爬取搜索引擎的结果 > -------------
|
||||
from toolbox import get_conf
|
||||
proxies, = get_conf('proxies')
|
||||
urls = google(txt, proxies)
|
||||
history = []
|
||||
|
||||
# ------------- < 第2步:依次访问网页 > -------------
|
||||
max_search_result = 5 # 最多收纳多少个网页的结果
|
||||
for index, url in enumerate(urls[:max_search_result]):
|
||||
res = scrape_text(url['link'], proxies)
|
||||
history.extend([f"第{index}份搜索结果:", res])
|
||||
chatbot.append([f"第{index}份搜索结果:", res[:500]+"......"])
|
||||
yield from update_ui(chatbot=chatbot, history=history) # 刷新界面 # 由于请求gpt需要一段时间,我们先及时地做一次界面更新
|
||||
|
||||
# ------------- < 第3步:ChatGPT综合 > -------------
|
||||
i_say = f"从以上搜索结果中抽取信息,然后回答问题:{txt}"
|
||||
i_say, history = input_clipping( # 裁剪输入,从最长的条目开始裁剪,防止爆token
|
||||
inputs=i_say,
|
||||
history=history,
|
||||
max_token_limit=model_info[llm_kwargs['llm_model']]['max_token']*3//4
|
||||
)
|
||||
gpt_say = yield from request_gpt_model_in_new_thread_with_ui_alive(
|
||||
inputs=i_say, inputs_show_user=i_say,
|
||||
llm_kwargs=llm_kwargs, chatbot=chatbot, history=history,
|
||||
sys_prompt="请从给定的若干条搜索结果中抽取信息,对最相关的两个搜索结果进行总结,然后回答问题。"
|
||||
)
|
||||
chatbot[-1] = (i_say, gpt_say)
|
||||
history.append(i_say);history.append(gpt_say)
|
||||
yield from update_ui(chatbot=chatbot, history=history) # 刷新界面 # 界面更新
|
||||
|
||||
145
crazy_functions/解析JupyterNotebook.py
Normal file
145
crazy_functions/解析JupyterNotebook.py
Normal file
@@ -0,0 +1,145 @@
|
||||
from toolbox import update_ui
|
||||
from toolbox import CatchException, report_execption, write_results_to_file
|
||||
fast_debug = True
|
||||
|
||||
|
||||
class PaperFileGroup():
|
||||
def __init__(self):
|
||||
self.file_paths = []
|
||||
self.file_contents = []
|
||||
self.sp_file_contents = []
|
||||
self.sp_file_index = []
|
||||
self.sp_file_tag = []
|
||||
|
||||
# count_token
|
||||
from request_llm.bridge_all import model_info
|
||||
enc = model_info["gpt-3.5-turbo"]['tokenizer']
|
||||
def get_token_num(txt): return len(
|
||||
enc.encode(txt, disallowed_special=()))
|
||||
self.get_token_num = get_token_num
|
||||
|
||||
def run_file_split(self, max_token_limit=1900):
|
||||
"""
|
||||
将长文本分离开来
|
||||
"""
|
||||
for index, file_content in enumerate(self.file_contents):
|
||||
if self.get_token_num(file_content) < max_token_limit:
|
||||
self.sp_file_contents.append(file_content)
|
||||
self.sp_file_index.append(index)
|
||||
self.sp_file_tag.append(self.file_paths[index])
|
||||
else:
|
||||
from .crazy_utils import breakdown_txt_to_satisfy_token_limit_for_pdf
|
||||
segments = breakdown_txt_to_satisfy_token_limit_for_pdf(
|
||||
file_content, self.get_token_num, max_token_limit)
|
||||
for j, segment in enumerate(segments):
|
||||
self.sp_file_contents.append(segment)
|
||||
self.sp_file_index.append(index)
|
||||
self.sp_file_tag.append(
|
||||
self.file_paths[index] + f".part-{j}.txt")
|
||||
|
||||
|
||||
|
||||
def parseNotebook(filename, enable_markdown=1):
|
||||
import json
|
||||
|
||||
CodeBlocks = []
|
||||
with open(filename, 'r', encoding='utf-8', errors='replace') as f:
|
||||
notebook = json.load(f)
|
||||
for cell in notebook['cells']:
|
||||
if cell['cell_type'] == 'code' and cell['source']:
|
||||
# remove blank lines
|
||||
cell['source'] = [line for line in cell['source'] if line.strip()
|
||||
!= '']
|
||||
CodeBlocks.append("".join(cell['source']))
|
||||
elif enable_markdown and cell['cell_type'] == 'markdown' and cell['source']:
|
||||
cell['source'] = [line for line in cell['source'] if line.strip()
|
||||
!= '']
|
||||
CodeBlocks.append("Markdown:"+"".join(cell['source']))
|
||||
|
||||
Code = ""
|
||||
for idx, code in enumerate(CodeBlocks):
|
||||
Code += f"This is {idx+1}th code block: \n"
|
||||
Code += code+"\n"
|
||||
|
||||
return Code
|
||||
|
||||
|
||||
def ipynb解释(file_manifest, project_folder, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt):
|
||||
from .crazy_utils import request_gpt_model_multi_threads_with_very_awesome_ui_and_high_efficiency
|
||||
|
||||
enable_markdown = plugin_kwargs.get("advanced_arg", "1")
|
||||
try:
|
||||
enable_markdown = int(enable_markdown)
|
||||
except ValueError:
|
||||
enable_markdown = 1
|
||||
|
||||
pfg = PaperFileGroup()
|
||||
|
||||
for fp in file_manifest:
|
||||
file_content = parseNotebook(fp, enable_markdown=enable_markdown)
|
||||
pfg.file_paths.append(fp)
|
||||
pfg.file_contents.append(file_content)
|
||||
|
||||
# <-------- 拆分过长的IPynb文件 ---------->
|
||||
pfg.run_file_split(max_token_limit=1024)
|
||||
n_split = len(pfg.sp_file_contents)
|
||||
|
||||
inputs_array = [r"This is a Jupyter Notebook file, tell me about Each Block in Chinese. Focus Just On Code." +
|
||||
r"If a block starts with `Markdown` which means it's a markdown block in ipynbipynb. " +
|
||||
r"Start a new line for a block and block num use Chinese." +
|
||||
f"\n\n{frag}" for frag in pfg.sp_file_contents]
|
||||
inputs_show_user_array = [f"{f}的分析如下" for f in pfg.sp_file_tag]
|
||||
sys_prompt_array = ["You are a professional programmer."] * n_split
|
||||
|
||||
gpt_response_collection = yield from request_gpt_model_multi_threads_with_very_awesome_ui_and_high_efficiency(
|
||||
inputs_array=inputs_array,
|
||||
inputs_show_user_array=inputs_show_user_array,
|
||||
llm_kwargs=llm_kwargs,
|
||||
chatbot=chatbot,
|
||||
history_array=[[""] for _ in range(n_split)],
|
||||
sys_prompt_array=sys_prompt_array,
|
||||
# max_workers=5, # OpenAI所允许的最大并行过载
|
||||
scroller_max_len=80
|
||||
)
|
||||
|
||||
# <-------- 整理结果,退出 ---------->
|
||||
block_result = " \n".join(gpt_response_collection)
|
||||
chatbot.append(("解析的结果如下", block_result))
|
||||
history.extend(["解析的结果如下", block_result])
|
||||
yield from update_ui(chatbot=chatbot, history=history) # 刷新界面
|
||||
|
||||
# <-------- 写入文件,退出 ---------->
|
||||
res = write_results_to_file(history)
|
||||
chatbot.append(("完成了吗?", res))
|
||||
yield from update_ui(chatbot=chatbot, history=history) # 刷新界面
|
||||
|
||||
@CatchException
|
||||
def 解析ipynb文件(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, web_port):
|
||||
chatbot.append([
|
||||
"函数插件功能?",
|
||||
"对IPynb文件进行解析。Contributor: codycjy."])
|
||||
yield from update_ui(chatbot=chatbot, history=history) # 刷新界面
|
||||
|
||||
history = [] # 清空历史
|
||||
import glob
|
||||
import os
|
||||
if os.path.exists(txt):
|
||||
project_folder = txt
|
||||
else:
|
||||
if txt == "":
|
||||
txt = '空空如也的输入栏'
|
||||
report_execption(chatbot, history,
|
||||
a=f"解析项目: {txt}", b=f"找不到本地项目或无权访问: {txt}")
|
||||
yield from update_ui(chatbot=chatbot, history=history) # 刷新界面
|
||||
return
|
||||
if txt.endswith('.ipynb'):
|
||||
file_manifest = [txt]
|
||||
else:
|
||||
file_manifest = [f for f in glob.glob(
|
||||
f'{project_folder}/**/*.ipynb', recursive=True)]
|
||||
if len(file_manifest) == 0:
|
||||
report_execption(chatbot, history,
|
||||
a=f"解析项目: {txt}", b=f"找不到任何.ipynb文件: {txt}")
|
||||
yield from update_ui(chatbot=chatbot, history=history) # 刷新界面
|
||||
return
|
||||
yield from ipynb解释(file_manifest, project_folder, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, )
|
||||
@@ -1,5 +1,6 @@
|
||||
from toolbox import update_ui
|
||||
from toolbox import CatchException, report_execption, write_results_to_file
|
||||
from .crazy_utils import input_clipping
|
||||
|
||||
def 解析源代码新(file_manifest, project_folder, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt):
|
||||
import os, copy
|
||||
@@ -11,7 +12,7 @@ def 解析源代码新(file_manifest, project_folder, llm_kwargs, plugin_kwargs,
|
||||
history_array = []
|
||||
sys_prompt_array = []
|
||||
report_part_1 = []
|
||||
|
||||
|
||||
assert len(file_manifest) <= 512, "源文件太多(超过512个), 请缩减输入文件的数量。或者,您也可以选择删除此行警告,并修改代码拆分file_manifest列表,从而实现分批次处理。"
|
||||
############################## <第一步,逐个文件分析,多线程> ##################################
|
||||
for index, fp in enumerate(file_manifest):
|
||||
@@ -61,13 +62,15 @@ def 解析源代码新(file_manifest, project_folder, llm_kwargs, plugin_kwargs,
|
||||
previous_iteration_files.extend([os.path.relpath(fp, project_folder) for index, fp in enumerate(this_iteration_file_manifest)])
|
||||
previous_iteration_files_string = ', '.join(previous_iteration_files)
|
||||
current_iteration_focus = ', '.join([os.path.relpath(fp, project_folder) for index, fp in enumerate(this_iteration_file_manifest)])
|
||||
i_say = f'根据以上分析,对程序的整体功能和构架重新做出概括。然后用一张markdown表格整理每个文件的功能(包括{previous_iteration_files_string})。'
|
||||
i_say = f'用一张Markdown表格简要描述以下文件的功能:{previous_iteration_files_string}。根据以上分析,用一句话概括程序的整体功能。'
|
||||
inputs_show_user = f'根据以上分析,对程序的整体功能和构架重新做出概括,由于输入长度限制,可能需要分组处理,本组文件为 {current_iteration_focus} + 已经汇总的文件组。'
|
||||
this_iteration_history = copy.deepcopy(this_iteration_gpt_response_collection)
|
||||
this_iteration_history = copy.deepcopy(this_iteration_gpt_response_collection)
|
||||
this_iteration_history.append(last_iteration_result)
|
||||
# 裁剪input
|
||||
inputs, this_iteration_history_feed = input_clipping(inputs=i_say, history=this_iteration_history, max_token_limit=2560)
|
||||
result = yield from request_gpt_model_in_new_thread_with_ui_alive(
|
||||
inputs=i_say, inputs_show_user=inputs_show_user, llm_kwargs=llm_kwargs, chatbot=chatbot,
|
||||
history=this_iteration_history, # 迭代之前的分析
|
||||
inputs=inputs, inputs_show_user=inputs_show_user, llm_kwargs=llm_kwargs, chatbot=chatbot,
|
||||
history=this_iteration_history_feed, # 迭代之前的分析
|
||||
sys_prompt="你是一个程序架构分析师,正在分析一个项目的源代码。")
|
||||
report_part_2.extend([i_say, result])
|
||||
last_iteration_result = result
|
||||
@@ -222,8 +225,8 @@ def 解析一个Golang项目(txt, llm_kwargs, plugin_kwargs, chatbot, history, s
|
||||
yield from update_ui(chatbot=chatbot, history=history) # 刷新界面
|
||||
return
|
||||
yield from 解析源代码新(file_manifest, project_folder, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt)
|
||||
|
||||
|
||||
|
||||
|
||||
@CatchException
|
||||
def 解析一个Lua项目(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, web_port):
|
||||
history = [] # 清空历史,以免输入溢出
|
||||
@@ -243,9 +246,9 @@ def 解析一个Lua项目(txt, llm_kwargs, plugin_kwargs, chatbot, history, syst
|
||||
report_execption(chatbot, history, a = f"解析项目: {txt}", b = f"找不到任何lua文件: {txt}")
|
||||
yield from update_ui(chatbot=chatbot, history=history) # 刷新界面
|
||||
return
|
||||
yield from 解析源代码新(file_manifest, project_folder, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt)
|
||||
|
||||
|
||||
yield from 解析源代码新(file_manifest, project_folder, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt)
|
||||
|
||||
|
||||
@CatchException
|
||||
def 解析一个CSharp项目(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, web_port):
|
||||
history = [] # 清空历史,以免输入溢出
|
||||
@@ -263,4 +266,45 @@ def 解析一个CSharp项目(txt, llm_kwargs, plugin_kwargs, chatbot, history, s
|
||||
report_execption(chatbot, history, a = f"解析项目: {txt}", b = f"找不到任何CSharp文件: {txt}")
|
||||
yield from update_ui(chatbot=chatbot, history=history) # 刷新界面
|
||||
return
|
||||
yield from 解析源代码新(file_manifest, project_folder, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt)
|
||||
yield from 解析源代码新(file_manifest, project_folder, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt)
|
||||
|
||||
|
||||
@CatchException
|
||||
def 解析任意code项目(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, web_port):
|
||||
txt_pattern = plugin_kwargs.get("advanced_arg")
|
||||
txt_pattern = txt_pattern.replace(",", ",")
|
||||
# 将要匹配的模式(例如: *.c, *.cpp, *.py, config.toml)
|
||||
pattern_include = [_.lstrip(" ,").rstrip(" ,") for _ in txt_pattern.split(",") if _ != "" and not _.strip().startswith("^")]
|
||||
if not pattern_include: pattern_include = ["*"] # 不输入即全部匹配
|
||||
# 将要忽略匹配的文件后缀(例如: ^*.c, ^*.cpp, ^*.py)
|
||||
pattern_except_suffix = [_.lstrip(" ^*.,").rstrip(" ,") for _ in txt_pattern.split(" ") if _ != "" and _.strip().startswith("^*.")]
|
||||
pattern_except_suffix += ['zip', 'rar', '7z', 'tar', 'gz'] # 避免解析压缩文件
|
||||
# 将要忽略匹配的文件名(例如: ^README.md)
|
||||
pattern_except_name = [_.lstrip(" ^*,").rstrip(" ,").replace(".", "\.") for _ in txt_pattern.split(" ") if _ != "" and _.strip().startswith("^") and not _.strip().startswith("^*.")]
|
||||
# 生成正则表达式
|
||||
pattern_except = '/[^/]+\.(' + "|".join(pattern_except_suffix) + ')$'
|
||||
pattern_except += '|/(' + "|".join(pattern_except_name) + ')$' if pattern_except_name != [] else ''
|
||||
|
||||
history.clear()
|
||||
import glob, os, re
|
||||
if os.path.exists(txt):
|
||||
project_folder = txt
|
||||
else:
|
||||
if txt == "": txt = '空空如也的输入栏'
|
||||
report_execption(chatbot, history, a = f"解析项目: {txt}", b = f"找不到本地项目或无权访问: {txt}")
|
||||
yield from update_ui(chatbot=chatbot, history=history) # 刷新界面
|
||||
return
|
||||
# 若上传压缩文件, 先寻找到解压的文件夹路径, 从而避免解析压缩文件
|
||||
maybe_dir = [f for f in glob.glob(f'{project_folder}/*') if os.path.isdir(f)]
|
||||
if len(maybe_dir)>0 and maybe_dir[0].endswith('.extract'):
|
||||
extract_folder_path = maybe_dir[0]
|
||||
else:
|
||||
extract_folder_path = project_folder
|
||||
# 按输入的匹配模式寻找上传的非压缩文件和已解压的文件
|
||||
file_manifest = [f for pattern in pattern_include for f in glob.glob(f'{extract_folder_path}/**/{pattern}', recursive=True) if "" != extract_folder_path and \
|
||||
os.path.isfile(f) and (not re.search(pattern_except, f) or pattern.endswith('.' + re.search(pattern_except, f).group().split('.')[-1]))]
|
||||
if len(file_manifest) == 0:
|
||||
report_execption(chatbot, history, a = f"解析项目: {txt}", b = f"找不到任何文件: {txt}")
|
||||
yield from update_ui(chatbot=chatbot, history=history) # 刷新界面
|
||||
return
|
||||
yield from 解析源代码新(file_manifest, project_folder, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt)
|
||||
@@ -25,6 +25,35 @@ def 同时问询(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt
|
||||
retry_times_at_unknown_error=0
|
||||
)
|
||||
|
||||
history.append(txt)
|
||||
history.append(gpt_say)
|
||||
yield from update_ui(chatbot=chatbot, history=history) # 刷新界面 # 界面更新
|
||||
|
||||
|
||||
@CatchException
|
||||
def 同时问询_指定模型(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, web_port):
|
||||
"""
|
||||
txt 输入栏用户输入的文本,例如需要翻译的一段话,再例如一个包含了待处理文件的路径
|
||||
llm_kwargs gpt模型参数,如温度和top_p等,一般原样传递下去就行
|
||||
plugin_kwargs 插件模型的参数,如温度和top_p等,一般原样传递下去就行
|
||||
chatbot 聊天显示框的句柄,用于显示给用户
|
||||
history 聊天历史,前情提要
|
||||
system_prompt 给gpt的静默提醒
|
||||
web_port 当前软件运行的端口号
|
||||
"""
|
||||
history = [] # 清空历史,以免输入溢出
|
||||
chatbot.append((txt, "正在同时咨询ChatGPT和ChatGLM……"))
|
||||
yield from update_ui(chatbot=chatbot, history=history) # 刷新界面 # 由于请求gpt需要一段时间,我们先及时地做一次界面更新
|
||||
|
||||
# llm_kwargs['llm_model'] = 'chatglm&gpt-3.5-turbo&api2d-gpt-3.5-turbo' # 支持任意数量的llm接口,用&符号分隔
|
||||
llm_kwargs['llm_model'] = plugin_kwargs.get("advanced_arg", 'chatglm&gpt-3.5-turbo') # 'chatglm&gpt-3.5-turbo' # 支持任意数量的llm接口,用&符号分隔
|
||||
gpt_say = yield from request_gpt_model_in_new_thread_with_ui_alive(
|
||||
inputs=txt, inputs_show_user=txt,
|
||||
llm_kwargs=llm_kwargs, chatbot=chatbot, history=history,
|
||||
sys_prompt=system_prompt,
|
||||
retry_times_at_unknown_error=0
|
||||
)
|
||||
|
||||
history.append(txt)
|
||||
history.append(gpt_say)
|
||||
yield from update_ui(chatbot=chatbot, history=history) # 刷新界面 # 界面更新
|
||||
@@ -70,6 +70,7 @@ def 谷歌检索小助手(txt, llm_kwargs, plugin_kwargs, chatbot, history, syst
|
||||
# 尝试导入依赖,如果缺少依赖,则给出安装建议
|
||||
try:
|
||||
import arxiv
|
||||
import math
|
||||
from bs4 import BeautifulSoup
|
||||
except:
|
||||
report_execption(chatbot, history,
|
||||
@@ -80,25 +81,26 @@ def 谷歌检索小助手(txt, llm_kwargs, plugin_kwargs, chatbot, history, syst
|
||||
|
||||
# 清空历史,以免输入溢出
|
||||
history = []
|
||||
|
||||
meta_paper_info_list = yield from get_meta_information(txt, chatbot, history)
|
||||
batchsize = 5
|
||||
for batch in range(math.ceil(len(meta_paper_info_list)/batchsize)):
|
||||
if len(meta_paper_info_list[:batchsize]) > 0:
|
||||
i_say = "下面是一些学术文献的数据,提取出以下内容:" + \
|
||||
"1、英文题目;2、中文题目翻译;3、作者;4、arxiv公开(is_paper_in_arxiv);4、引用数量(cite);5、中文摘要翻译。" + \
|
||||
f"以下是信息源:{str(meta_paper_info_list[:batchsize])}"
|
||||
|
||||
if len(meta_paper_info_list[:10]) > 0:
|
||||
i_say = "下面是一些学术文献的数据,请从中提取出以下内容。" + \
|
||||
"1、英文题目;2、中文题目翻译;3、作者;4、arxiv公开(is_paper_in_arxiv);4、引用数量(cite);5、中文摘要翻译。" + \
|
||||
f"以下是信息源:{str(meta_paper_info_list[:10])}"
|
||||
inputs_show_user = f"请分析此页面中出现的所有文章:{txt},这是第{batch+1}批"
|
||||
gpt_say = yield from request_gpt_model_in_new_thread_with_ui_alive(
|
||||
inputs=i_say, inputs_show_user=inputs_show_user,
|
||||
llm_kwargs=llm_kwargs, chatbot=chatbot, history=[],
|
||||
sys_prompt="你是一个学术翻译,请从数据中提取信息。你必须使用Markdown表格。你必须逐个文献进行处理。"
|
||||
)
|
||||
|
||||
inputs_show_user = f"请分析此页面中出现的所有文章:{txt}"
|
||||
gpt_say = yield from request_gpt_model_in_new_thread_with_ui_alive(
|
||||
inputs=i_say, inputs_show_user=inputs_show_user,
|
||||
llm_kwargs=llm_kwargs, chatbot=chatbot, history=[],
|
||||
sys_prompt="你是一个学术翻译,请从数据中提取信息。你必须使用Markdown格式。你必须逐个文献进行处理。"
|
||||
)
|
||||
history.extend([ f"第{batch+1}批", gpt_say ])
|
||||
meta_paper_info_list = meta_paper_info_list[batchsize:]
|
||||
|
||||
history.extend([ "第一批", gpt_say ])
|
||||
meta_paper_info_list = meta_paper_info_list[10:]
|
||||
|
||||
chatbot.append(["状态?", "已经全部完成"])
|
||||
chatbot.append(["状态?",
|
||||
"已经全部完成,您可以试试让AI写一个Related Works,例如您可以继续输入Write a \"Related Works\" section about \"你搜索的研究领域\" for me."])
|
||||
msg = '正常'
|
||||
yield from update_ui(chatbot=chatbot, history=history, msg=msg) # 刷新界面
|
||||
res = write_results_to_file(history)
|
||||
|
||||
@@ -1,223 +1,227 @@
|
||||
# ChatGPT Academic Optimization
|
||||
> **Note**
|
||||
>
|
||||
> This English readme is automatically generated by the markdown translation plugin in this project, and may not be 100% correct.
|
||||
> This English README is automatically generated by the markdown translation plugin in this project, and may not be 100% correct.
|
||||
>
|
||||
|
||||
# <img src="logo.png" width="40" > ChatGPT Academic Optimization
|
||||
|
||||
**If you like this project, please give it a star. If you have come up with more useful academic shortcuts or functional plugins, feel free to open an issue or pull request (to the `dev` branch).**
|
||||
**If you like this project, please give it a Star. If you've come up with more useful academic shortcuts or functional plugins, feel free to open an issue or pull request. We also have a [README in English](docs/README_EN.md) translated by this project itself.**
|
||||
|
||||
> **Note**
|
||||
>
|
||||
> 1. Please note that only function plugins (buttons) marked in **red** support reading files, and some plugins are located in the **dropdown menu** in the plugin area. Additionally, we welcome and process PRs for any new plugins with the **highest priority**!
|
||||
> 1. Please note that only **functions with red color** supports reading files, some functions are located in the **dropdown menu** of plugins. Additionally, we welcome and prioritize any new plugin PRs with **highest priority**!
|
||||
>
|
||||
> 2. The functions of each file in this project are detailed in the self-translation report [self_analysis.md](https://github.com/binary-husky/chatgpt_academic/wiki/chatgpt-academic%E9%A1%B9%E7%9B%AE%E8%87%AA%E8%AF%91%E8%A7%A3%E6%8A%A5%E5%91%8A). With the version iteration, you can click on a relevant function plugin at any time to call GPT to regenerate the self-analysis report for the project. Commonly asked questions are summarized in the [`wiki`](https://github.com/binary-husky/chatgpt_academic/wiki/%E5%B8%B8%E8%A7%81%E9%97%AE%E9%A2%98).
|
||||
>
|
||||
> 3. If you are not used to the function, comments or interface with some Chinese names, you can click on the relevant function plugin at any time to call ChatGPT to generate the source code of the project in English.
|
||||
> 2. The functionality of each file in this project is detailed in the self-translation report [`self_analysis.md`](https://github.com/binary-husky/chatgpt_academic/wiki/chatgpt-academic%E9%A1%B9%E7%9B%AE%E8%87%AA%E8%AF%91%E8%A7%A3%E6%8A%A5%E5%91%8A) of the project. With the iteration of the version, you can also click on the relevant function plugins at any time to call GPT to regenerate the self-analysis report of the project. The FAQ summary is in the [`wiki`](https://github.com/binary-husky/chatgpt_academic/wiki/%E5%B8%B8%E8%A7%81%E9%97%AE%E9%A2%98) section.
|
||||
>
|
||||
|
||||
|
||||
<div align="center">
|
||||
|
||||
Function | Description
|
||||
--- | ---
|
||||
One-click refinement | Supports one-click refinement, one-click searching for grammatical errors in papers.
|
||||
One-click translation between Chinese and English | One-click translation between Chinese and English.
|
||||
One-click code interpretation | Can correctly display and interpret the code.
|
||||
[Custom shortcuts](https://www.bilibili.com/video/BV14s4y1E7jN) | Supports custom shortcuts.
|
||||
[Configure proxy server](https://www.bilibili.com/video/BV1rc411W7Dr) | Supports configuring proxy server.
|
||||
Modular design | Supports custom high-order experimental features and [function plug-ins], and plug-ins support [hot update](https://github.com/binary-husky/chatgpt_academic/wiki/%E5%87%BD%E6%95%B0%E6%8F%92%E4%BB%B6%E6%8C%87%E5%8D%97).
|
||||
[Self-program analysis](https://www.bilibili.com/video/BV1cj411A7VW) | [Function Plug-in] [One-Key Understanding](https://github.com/binary-husky/chatgpt_academic/wiki/chatgpt-academic%E9%A1%B9%E7%9B%AE%E8%87%AA%E8%AF%91%E8%A7%A3%E6%8A%A5%E5%91%8A) the source code of this project.
|
||||
[Program analysis](https://www.bilibili.com/video/BV1cj411A7VW) | [Function Plug-in] One-click can analyze other Python/C/C++/Java/Golang/Lua/Rect project trees.
|
||||
Read papers | [Function Plug-in] One-click reads the full text of a latex paper and generates an abstract.
|
||||
Latex full-text translation/refinement | [Function Plug-in] One-click translates or refines a latex paper.
|
||||
Batch annotation generation | [Function Plug-in] One-click generates function annotations in batches.
|
||||
Chat analysis report generation | [Function Plug-in] Automatically generate summary reports after running.
|
||||
[Arxiv assistant](https://www.bilibili.com/video/BV1LM4y1279X) | [Function Plug-in] Enter the arxiv paper url and you can translate the abstract and download the PDF with one click.
|
||||
[PDF paper full-text translation function](https://www.bilibili.com/video/BV1KT411x7Wn) | [Function Plug-in] Extract title and abstract of PDF papers + translate full text (multi-threaded).
|
||||
[Google Scholar integration assistant](https://www.bilibili.com/video/BV19L411U7ia) (Version>=2.45) | [Function Plug-in] Given any Google Scholar search page URL, let GPT help you choose interesting articles.
|
||||
Formula display | Can simultaneously display the tex form and rendering form of formulas.
|
||||
Image display | Can display images in Markdown.
|
||||
Multithreaded function plug-in support | Supports multi-threaded calling of chatgpt, one-click processing of massive texts or programs.
|
||||
Support for markdown tables output by GPT | Can output markdown tables that support GPT.
|
||||
Start dark gradio theme [theme](https://github.com/binary-husky/chatgpt_academic/issues/173) | Add ```/?__dark-theme=true``` to the browser URL to switch to the dark theme.
|
||||
Huggingface free scientific online experience](https://huggingface.co/spaces/qingxu98/gpt-academic) | After logging in to Huggingface, copy [this space](https://huggingface.co/spaces/qingxu98/gpt-academic).
|
||||
[Mixed support for multiple LLM models](https://www.bilibili.com/video/BV1EM411K7VH/) ([v3.0 branch](https://github.com/binary-husky/chatgpt_academic/tree/v3.0) in testing) | It must feel great to be served by both ChatGPT and [Tsinghua ChatGLM](https://github.com/THUDM/ChatGLM-6B)!
|
||||
Compatible with [TGUI](https://github.com/oobabooga/text-generation-webui) to access more language models | Access to opt-1.3b, galactica-1.3b and other models ([v3.0 branch](https://github.com/binary-husky/chatgpt_academic/tree/v3.0) under testing).
|
||||
… | ...
|
||||
One-Click Polish | Supports one-click polishing and finding grammar errors in academic papers.
|
||||
One-Key Translation Between Chinese and English | One-click translation between Chinese and English.
|
||||
One-Key Code Interpretation | Can correctly display and interpret code.
|
||||
[Custom Shortcut Keys](https://www.bilibili.com/video/BV14s4y1E7jN) | Supports custom shortcut keys.
|
||||
[Configure Proxy Server](https://www.bilibili.com/video/BV1rc411W7Dr) | Supports configuring proxy servers.
|
||||
Modular Design | Supports custom high-order function plugins and [function plugins], and plugins support [hot updates](https://github.com/binary-husky/chatgpt_academic/wiki/%E5%87%BD%E6%95%B0%E6%8F%92%E4%BB%B6%E6%8C%87%E5%8D%97).
|
||||
[Self-programming Analysis](https://www.bilibili.com/video/BV1cj411A7VW) | [Function Plugin] [One-Key Read] (https://github.com/binary-husky/chatgpt_academic/wiki/chatgpt-academic%E9%A1%B9%E7%9B%AE%E8%87%AA%E8%AF%91%E8%A7%A3%E6%8A%A5%E5%91%8A) The source code of this project is analyzed.
|
||||
[Program Analysis](https://www.bilibili.com/video/BV1cj411A7VW) | [Function Plugin] One-click can analyze the project tree of other Python/C/C++/Java/Lua/... projects
|
||||
Read the Paper | [Function Plugin] One-click interpretation of the full text of latex paper and generation of abstracts
|
||||
Latex Full Text Translation, Proofreading | [Function Plugin] One-click translation or proofreading of latex papers.
|
||||
Batch Comment Generation | [Function Plugin] One-click batch generation of function comments
|
||||
Chat Analysis Report Generation | [Function Plugin] After running, an automatic summary report will be generated
|
||||
[Arxiv Assistant](https://www.bilibili.com/video/BV1LM4y1279X) | [Function Plugin] Enter the arxiv article url to translate the abstract and download the PDF with one click
|
||||
[Full-text Translation Function of PDF Paper](https://www.bilibili.com/video/BV1KT411x7Wn) | [Function Plugin] Extract the title & abstract of the PDF paper + translate the full text (multithreading)
|
||||
[Google Scholar Integration Assistant](https://www.bilibili.com/video/BV19L411U7ia) | [Function Plugin] Given any Google Scholar search page URL, let gpt help you choose interesting articles.
|
||||
Formula / Picture / Table Display | Can display both the tex form and the rendering form of formulas at the same time, support formula and code highlighting
|
||||
Multithreaded Function Plugin Support | Supports multi-threaded calling chatgpt, one-click processing of massive text or programs
|
||||
Start Dark Gradio [Theme](https://github.com/binary-husky/chatgpt_academic/issues/173) | Add ```/?__dark-theme=true``` at the end of the browser url to switch to dark theme
|
||||
[Multiple LLM Models](https://www.bilibili.com/video/BV1wT411p7yf) support, [API2D](https://api2d.com/) interface support | It must feel nice to be served by both GPT3.5, GPT4, and [Tsinghua ChatGLM](https://github.com/THUDM/ChatGLM-6B)!
|
||||
Huggingface non-Science Net [Online Experience](https://huggingface.co/spaces/qingxu98/gpt-academic) | After logging in to huggingface, copy [this space](https://huggingface.co/spaces/qingxu98/gpt-academic)
|
||||
... | ...
|
||||
|
||||
</div>
|
||||
|
||||
<!-- - New interface (left: master branch, right: dev development frontier) -->
|
||||
- New interface (modify the `LAYOUT` option in `config.py` to switch between "left and right layout" and "up and down layout").
|
||||
|
||||
- New interface (switch between "left-right layout" and "up-down layout" by modifying the LAYOUT option in config.py)
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/230361456-61078362-a966-4eb5-b49e-3c62ef18b860.gif" width="700" >
|
||||
</div>
|
||||
|
||||
- All buttons are dynamically generated by reading `functional.py`, and custom functions can be added freely, freeing up the clipboard.
|
||||
|
||||
- All buttons are dynamically generated by reading functional.py and can add custom functionality at will, freeing up clipboard
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/231975334-b4788e91-4887-412f-8b43-2b9c5f41d248.gif" width="700" >
|
||||
</div>
|
||||
|
||||
- Refinement/Correction
|
||||
- Proofreading / correcting
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/231980294-f374bdcb-3309-4560-b424-38ef39f04ebd.gif" width="700" >
|
||||
</div>
|
||||
|
||||
- Supports markdown tables output by GPT.
|
||||
- If the output contains formulas, it will be displayed in both the tex form and the rendering form at the same time, which is convenient for copying and reading
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/230598842-1d7fcddd-815d-40ee-af60-baf488a199df.png" width="700" >
|
||||
</div>
|
||||
|
||||
- If the output contains formulas, both the tex form and the rendering form are displayed simultaneously for easy copying and reading.
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/230598842-1d7fcddd-815d-40ee-af60-baf488a199df.png" width="700" >
|
||||
</div>
|
||||
|
||||
- Don't want to read project code? Let chatgpt boast about the whole project.
|
||||
- Don't want to read the project code? Just take the whole project to chatgpt
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/226935232-6b6a73ce-8900-4aee-93f9-733c7e6fef53.png" width="700" >
|
||||
</div>
|
||||
|
||||
- Multiple large language models mixed calling. ([v3.0 branch](https://github.com/binary-husky/chatgpt_academic/tree/v3.0) in testing)
|
||||
- Multiple major language model mixing calls (ChatGLM + OpenAI-GPT3.5 + [API2D](https://api2d.com/)-GPT4)
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/232537274-deca0563-7aa6-4b5d-94a2-b7c453c47794.png" width="700" >
|
||||
</div>
|
||||
|
||||
Multiple major language model mixing call [huggingface beta version](https://huggingface.co/spaces/qingxu98/academic-chatgpt-beta) (the huggingface version does not support chatglm)
|
||||
|
||||
|
||||
## Running Directly (Windows, Linux or MacOS)
|
||||
---
|
||||
|
||||
### 1. Download the Project
|
||||
## Installation-Method 1: Run directly (Windows, Linux or MacOS)
|
||||
|
||||
1. Download project
|
||||
```sh
|
||||
git clone https://github.com/binary-husky/chatgpt_academic.git
|
||||
cd chatgpt_academic
|
||||
```
|
||||
|
||||
### 2. Configure API_KEY and Proxy Settings
|
||||
2. Configure API_KEY and proxy settings
|
||||
|
||||
In `config.py`, configure the overseas Proxy and OpenAI API KEY, as follows:
|
||||
```
|
||||
1. If you are in China, you need to set an overseas proxy to use the OpenAI API smoothly. Please read the instructions in config.py carefully (1. Modify the USE_PROXY to True; 2. Modify the proxies according to the instructions).
|
||||
2. Configure OpenAI API KEY. You need to register on the OpenAI official website and obtain an API KEY. Once you get the API KEY, configure it in the config.py file.
|
||||
3. Issues related to proxy network (network timeout, proxy not working) are summarized to https://github.com/binary-husky/chatgpt_academic/issues/1
|
||||
```
|
||||
(Note: When the program is running, it will first check whether there is a private configuration file named `config_private.py`, and use the configuration in it to overwrite the same name configuration in `config.py`. Therefore, if you can understand our configuration reading logic, we strongly recommend that you create a new configuration file next to `config.py` named `config_private.py` and transfer (copy) the configuration in `config.py` to `config_private.py`. `config_private.py` is not managed by Git, which can make your privacy information more secure.)
|
||||
|
||||
### 3. Install Dependencies
|
||||
In `config.py`, configure the overseas Proxy and OpenAI API KEY as follows:
|
||||
```
|
||||
1. If you are in China, you need to set up an overseas proxy to use the OpenAI API smoothly. Please read config.py carefully for setup details (1. Modify USE_PROXY to True; 2. Modify proxies according to the instructions).
|
||||
2. Configure the OpenAI API KEY. You need to register and obtain an API KEY on the OpenAI website. Once you get the API KEY, you can configure it in the config.py file.
|
||||
3. Issues related to proxy networks (network timeouts, proxy failures) are summarized at https://github.com/binary-husky/chatgpt_academic/issues/1
|
||||
```
|
||||
(P.S. When the program runs, it will first check whether there is a private configuration file named `config_private.py` and use the same-name configuration in `config.py` to overwrite it. Therefore, if you can understand our configuration reading logic, we strongly recommend that you create a new configuration file named `config_private.py` next to `config.py` and transfer (copy) the configuration in `config.py` to` config_private.py`. `config_private.py` is not controlled by git and can make your privacy information more secure.))
|
||||
|
||||
|
||||
3. Install dependencies
|
||||
```sh
|
||||
# (Option 1) Recommended
|
||||
# (Option One) Recommended
|
||||
python -m pip install -r requirements.txt
|
||||
|
||||
# (Option 2) If you use anaconda, the steps are also similar:
|
||||
# (Option 2.1) conda create -n gptac_venv python=3.11
|
||||
# (Option 2.2) conda activate gptac_venv
|
||||
# (Option 2.3) python -m pip install -r requirements.txt
|
||||
# (Option Two) If you use anaconda, the steps are similar:
|
||||
# (Option Two.1) conda create -n gptac_venv python=3.11
|
||||
# (Option Two.2) conda activate gptac_venv
|
||||
# (Option Two.3) python -m pip install -r requirements.txt
|
||||
|
||||
# Note: Use the official pip source or the Ali pip source. Other pip sources (such as some university pips) may have problems. Temporary substitution method:
|
||||
# Note: Use official pip source or Ali pip source. Other pip sources (such as some university pips) may have problems, and temporary replacement methods are as follows:
|
||||
# python -m pip install -r requirements.txt -i https://mirrors.aliyun.com/pypi/simple/
|
||||
```
|
||||
|
||||
### 4. Run
|
||||
If you need to support Tsinghua ChatGLM, you need to install more dependencies (if you are not familiar with python or your computer configuration is not good, we recommend not to try):
|
||||
```sh
|
||||
python -m pip install -r request_llm/requirements_chatglm.txt
|
||||
```
|
||||
|
||||
4. Run
|
||||
```sh
|
||||
python main.py
|
||||
```
|
||||
|
||||
### 5. Test Experimental Features
|
||||
5. Test function plugins
|
||||
```
|
||||
- Test C++ Project Header Analysis
|
||||
In the input area, enter `./crazy_functions/test_project/cpp/libJPG` , and then click "[Experiment] Parse the entire C++ project (input inputs the root path of the project)"
|
||||
- Test Writing Abstracts for Latex Projects
|
||||
In the input area, enter `./crazy_functions/test_project/latex/attention` , and then click "[Experiment] Read the tex paper and write an abstract (input inputs the root path of the project)"
|
||||
- Test Python Project Analysis
|
||||
In the input area, enter `./crazy_functions/test_project/python/dqn` , and then click "[Experiment] Parse the entire py project (input inputs the root path of the project)"
|
||||
- Test Self-code Interpretation
|
||||
Click "[Experiment] Please analyze and deconstruct this project itself"
|
||||
- Test Experimental Function Template (asking GPT what happened in history today), you can implement more complex functions based on this template function
|
||||
Click "[Experiment] Experimental function template"
|
||||
- Test Python project analysis
|
||||
In the input area, enter `./crazy_functions/test_project/python/dqn`, and then click "Analyze the entire Python project"
|
||||
- Test self-code interpretation
|
||||
Click "[Multithreading Demo] Interpretation of This Project Itself (Source Code Interpretation)"
|
||||
- Test experimental function template function (requires gpt to answer what happened today in history). You can use this function as a template to implement more complex functions.
|
||||
Click "[Function Plugin Template Demo] Today in History"
|
||||
- There are more functions to choose from in the function plugin area drop-down menu.
|
||||
```
|
||||
|
||||
## Use Docker (Linux)
|
||||
## Installation-Method 2: Use Docker (Linux)
|
||||
|
||||
1. ChatGPT only (recommended for most people)
|
||||
``` sh
|
||||
# Download Project
|
||||
# download project
|
||||
git clone https://github.com/binary-husky/chatgpt_academic.git
|
||||
cd chatgpt_academic
|
||||
# Configure Overseas Proxy and OpenAI API KEY
|
||||
Configure config.py with any text editor
|
||||
# Installation
|
||||
# configure overseas Proxy and OpenAI API KEY
|
||||
Edit config.py with any text editor
|
||||
# Install
|
||||
docker build -t gpt-academic .
|
||||
# Run
|
||||
docker run --rm -it --net=host gpt-academic
|
||||
|
||||
# Test Experimental Features
|
||||
## Test Self-code Interpretation
|
||||
Click "[Experiment] Please analyze and deconstruct this project itself"
|
||||
## Test Experimental Function Template (asking GPT what happened in history today), you can implement more complex functions based on this template function
|
||||
Click "[Experiment] Experimental function template"
|
||||
## (Please note that when running in docker, you need to pay extra attention to file access rights issues of the program.)
|
||||
## Test C++ Project Header Analysis
|
||||
In the input area, enter ./crazy_functions/test_project/cpp/libJPG , and then click "[Experiment] Parse the entire C++ project (input inputs the root path of the project)"
|
||||
## Test Writing Abstracts for Latex Projects
|
||||
In the input area, enter ./crazy_functions/test_project/latex/attention , and then click "[Experiment] Read the tex paper and write an abstract (input inputs the root path of the project)"
|
||||
# Test function plug-in
|
||||
## Test function plugin template function (requires gpt to answer what happened today in history). You can use this function as a template to implement more complex functions.
|
||||
Click "[Function Plugin Template Demo] Today in History"
|
||||
## Test Abstract Writing for Latex Projects
|
||||
Enter ./crazy_functions/test_project/latex/attention in the input area, and then click "Read Tex Paper and Write Abstract"
|
||||
## Test Python Project Analysis
|
||||
In the input area, enter ./crazy_functions/test_project/python/dqn , and then click "[Experiment] Parse the entire py project (input inputs the root path of the project)"
|
||||
Enter ./crazy_functions/test_project/python/dqn in the input area and click "Analyze the entire Python project."
|
||||
|
||||
More functions are available in the function plugin area drop-down menu.
|
||||
```
|
||||
|
||||
## Other Deployment Methods
|
||||
- Use WSL2 (Windows Subsystem for Linux subsystem)
|
||||
Please visit [Deploy Wiki-1] (https://github.com/binary-husky/chatgpt_academic/wiki/%E4%BD%BF%E7%94%A8WSL2%EF%BC%88Windows-Subsystem-for-Linux-%E5%AD%90%E7%B3%BB%E7%BB%9F%EF%BC%89%E9%83%A8%E7%BD%B2)
|
||||
2. ChatGPT+ChatGLM (requires strong familiarity with docker + strong computer configuration)
|
||||
|
||||
- nginx remote deployment
|
||||
Please visit [Deploy Wiki-2] (https://github.com/binary-husky/chatgpt_academic/wiki/%E8%BF%9C%E7%A8%8B%E9%83%A8%E7%BD%B2%E7%9A%84%E6%8C%87%E5%AF%BC)
|
||||
|
||||
|
||||
## Customizing New Convenient Buttons (Academic Shortcut Key Customization)
|
||||
Open functional.py and add the entry as follows, and then restart the program. (If the button has been successfully added and is visible, both the prefix and suffix support hot modification and take effect without restarting the program.)
|
||||
|
||||
For example,
|
||||
``` sh
|
||||
# Modify dockerfile
|
||||
cd docs && nano Dockerfile+ChatGLM
|
||||
# How to build | 如何构建 (Dockerfile+ChatGLM在docs路径下,请先cd docs)
|
||||
docker build -t gpt-academic --network=host -f Dockerfile+ChatGLM .
|
||||
# How to run | 如何运行 (1) 直接运行:
|
||||
docker run --rm -it --net=host --gpus=all gpt-academic
|
||||
# How to run | 如何运行 (2) 我想运行之前进容器做一些调整:
|
||||
docker run --rm -it --net=host --gpus=all gpt-academic bash
|
||||
```
|
||||
"Super English to Chinese Translation": {
|
||||
|
||||
# Prefix, which will be added before your input. For example, it is used to describe your requirements, such as translation, code interpretation, polishing, etc.
|
||||
"Prefix": "Please translate the following content into Chinese, and then use a markdown table to explain each proprietary term in the text:\n\n",
|
||||
|
||||
# Suffix, which will be added after your input. For example, in conjunction with the prefix, you can bracket your input in quotes.
|
||||
|
||||
## Installation-Method 3: Other Deployment Methods
|
||||
|
||||
1. Remote Cloud Server Deployment
|
||||
Please visit [Deployment Wiki-1] (https://github.com/binary-husky/chatgpt_academic/wiki/%E4%BA%91%E6%9C%8D%E5%8A%A1%E5%99%A8%E8%BF%9C%E7%A8%8B%E9%83%A8%E7%BD%B2%E6%8C%87%E5%8D%97)
|
||||
|
||||
2. Use WSL2 (Windows Subsystem for Linux)
|
||||
Please visit [Deployment Wiki-2](https://github.com/binary-husky/chatgpt_academic/wiki/%E4%BD%BF%E7%94%A8WSL2%EF%BC%88Windows-Subsystem-for-Linux-%E5%AD%90%E7%B3%BB%E7%BB%9F%EF%BC%89%E9%83%A8%E7%BD%B2)
|
||||
|
||||
|
||||
## Installation-Proxy Configuration
|
||||
### Method 1: Conventional method
|
||||
[Configure Proxy](https://github.com/binary-husky/chatgpt_academic/issues/1)
|
||||
|
||||
### Method Two: Step-by-step tutorial for newcomers
|
||||
[Step-by-step tutorial for newcomers](https://github.com/binary-husky/chatgpt_academic/wiki/%E4%BB%A3%E7%90%86%E8%BD%AF%E4%BB%B6%E9%97%AE%E9%A2%98%E7%9A%84%E6%96%B0%E6%89%8B%E8%A7%A3%E5%86%B3%E6%96%B9%E6%B3%95%EF%BC%88%E6%96%B9%E6%B3%95%E5%8F%AA%E9%80%82%E7%94%A8%E4%BA%8E%E6%96%B0%E6%89%8B%EF%BC%89)
|
||||
|
||||
---
|
||||
|
||||
## Customizing Convenient Buttons (Customizing Academic Shortcuts)
|
||||
Open `core_functional.py` with any text editor and add an item as follows, then restart the program (if the button has been successfully added and visible, both the prefix and suffix support hot modification without the need to restart the program to take effect). For example:
|
||||
```
|
||||
"Super English to Chinese translation": {
|
||||
# Prefix, which will be added before your input. For example, to describe your requirements, such as translation, code interpretation, polishing, etc.
|
||||
"Prefix": "Please translate the following content into Chinese and use a markdown table to interpret the proprietary terms in the text one by one:\n\n",
|
||||
|
||||
# Suffix, which will be added after your input. For example, combined with the prefix, you can put your input content in quotes.
|
||||
"Suffix": "",
|
||||
|
||||
},
|
||||
```
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/226899272-477c2134-ed71-4326-810c-29891fe4a508.png" width="500" >
|
||||
</div>
|
||||
|
||||
|
||||
If you invent a more user-friendly academic shortcut key, welcome to post an issue or pull request!
|
||||
|
||||
## Configure Proxy
|
||||
### Method 1: General Method
|
||||
Modify the port and proxy software corresponding in ```config.py```
|
||||
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/226571294-37a47cd9-4d40-4c16-97a2-d360845406f7.png" width="500" >
|
||||
<img src="https://user-images.githubusercontent.com/96192199/226838985-e5c95956-69c2-4c23-a4dd-cd7944eeb451.png" width="500" >
|
||||
</div>
|
||||
---
|
||||
|
||||
|
||||
After configuring, you can use the following command to test whether the proxy works. If everything is normal, the code below will output the location of your proxy server:
|
||||
|
||||
```
|
||||
python check_proxy.py
|
||||
```
|
||||
|
||||
### Method Two: Pure Beginner Tutorial
|
||||
[Pure Beginner Tutorial](https://github.com/binary-husky/chatgpt_academic/wiki/%E4%BB%A3%E7%90%86%E8%BD%AF%E4%BB%B6%E9%97%AE%E9%A2%98%E7%9A%84%E6%96%B0%E6%89%8B%E8%A7%A3%E5%86%B3%E6%96%B9%E6%B3%95%EF%BC%88%E6%96%B9%E6%B3%95%E5%8F%AA%E9%80%82%E7%94%A8%E4%BA%8E%E6%96%B0%E6%89%8B%EF%BC%89)
|
||||
|
||||
## Compatibility Testing
|
||||
## Some Function Displays
|
||||
|
||||
### Image Display:
|
||||
|
||||
|
||||
You are a professional academic paper translator.
|
||||
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/228737599-bf0a9d9c-1808-4f43-ae15-dfcc7af0f295.png" width="800" >
|
||||
</div>
|
||||
|
||||
|
||||
### If the program can read and analyze itself:
|
||||
### If a program can understand and analyze itself:
|
||||
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/226936850-c77d7183-0749-4c1c-9875-fd4891842d0c.png" width="800" >
|
||||
@@ -227,7 +231,7 @@ python check_proxy.py
|
||||
<img src="https://user-images.githubusercontent.com/96192199/226936618-9b487e4b-ab5b-4b6e-84c6-16942102e917.png" width="800" >
|
||||
</div>
|
||||
|
||||
### Any other Python/Cpp project analysis:
|
||||
### Analysis of any Python/Cpp project:
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/226935232-6b6a73ce-8900-4aee-93f9-733c7e6fef53.png" width="800" >
|
||||
</div>
|
||||
@@ -236,59 +240,52 @@ python check_proxy.py
|
||||
<img src="https://user-images.githubusercontent.com/96192199/226969067-968a27c1-1b9c-486b-8b81-ab2de8d3f88a.png" width="800" >
|
||||
</div>
|
||||
|
||||
### Latex paper reading comprehension and abstract generation with one click
|
||||
### One-click reading comprehension and summary generation of Latex papers
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/227504406-86ab97cd-f208-41c3-8e4a-7000e51cf980.png" width="800" >
|
||||
</div>
|
||||
|
||||
### Automatic Report Generation
|
||||
### Automatic report generation
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/227503770-fe29ce2c-53fd-47b0-b0ff-93805f0c2ff4.png" height="300" >
|
||||
<img src="https://user-images.githubusercontent.com/96192199/227504617-7a497bb3-0a2a-4b50-9a8a-95ae60ea7afd.png" height="300" >
|
||||
<img src="https://user-images.githubusercontent.com/96192199/227504005-efeaefe0-b687-49d0-bf95-2d7b7e66c348.png" height="300" >
|
||||
</div>
|
||||
|
||||
### Modular Function Design
|
||||
### Modular functional design
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/229288270-093643c1-0018-487a-81e6-1d7809b6e90f.png" height="400" >
|
||||
<img src="https://user-images.githubusercontent.com/96192199/227504931-19955f78-45cd-4d1c-adac-e71e50957915.png" height="400" >
|
||||
</div>
|
||||
|
||||
|
||||
### Translating source code to English
|
||||
### Source code translation to English
|
||||
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/229720562-fe6c3508-6142-4635-a83d-21eb3669baee.png" height="400" >
|
||||
</div>
|
||||
|
||||
## Todo and Version Planning:
|
||||
|
||||
- version 3 (Todo):
|
||||
- - Support for gpt4 and other llm
|
||||
- version 2.4+ (Todo):
|
||||
- - Summary of long text and token overflow problems in large project source code
|
||||
- - Implementation of project packaging and deployment
|
||||
- - Function plugin parameter interface optimization
|
||||
- - Self-updating
|
||||
- version 2.4: (1) Added PDF full-text translation function; (2) Added input area switching function; (3) Added vertical layout option; (4) Optimized multi-threaded function plugin.
|
||||
## Todo and version planning:
|
||||
- version 3.2+ (todo): Function plugin supports more parameter interfaces
|
||||
- version 3.1: Support for inquiring multiple GPT models at the same time! Support for api2d, support for multiple apikeys load balancing
|
||||
- version 3.0: Support for chatglm and other small llms
|
||||
- version 2.6: Refactored the plugin structure, improved interactivity, added more plugins
|
||||
- version 2.5: Self-updating, solves the problem of text being too long and token overflowing when summarizing large project source code
|
||||
- version 2.4: (1) Added PDF full text translation function; (2) Added function to switch input area position; (3) Added vertical layout option; (4) Multi-threaded function plugin optimization.
|
||||
- version 2.3: Enhanced multi-threaded interactivity
|
||||
- version 2.2: Function plug-in supports hot reloading
|
||||
- version 2.1: Collapsible layout
|
||||
- version 2.2: Function plugin supports hot reloading
|
||||
- version 2.1: Foldable layout
|
||||
- version 2.0: Introduction of modular function plugins
|
||||
- version 1.0: Basic functions
|
||||
|
||||
## References and Learning
|
||||
|
||||
## Reference and learning
|
||||
|
||||
```
|
||||
The code refers to the design of many other excellent projects, mainly including:
|
||||
The code design of this project has referenced many other excellent projects, including:
|
||||
|
||||
# Reference Project 1: Referenced the method of reading OpenAI json, recording historical inquiry records, and using gradio queue in ChuanhuChatGPT
|
||||
# Reference project 1: Borrowed many tips from ChuanhuChatGPT
|
||||
https://github.com/GaiZhenbiao/ChuanhuChatGPT
|
||||
|
||||
# Reference Project 2:
|
||||
# Reference project 2: Tsinghua ChatGLM-6B:
|
||||
https://github.com/THUDM/ChatGLM-6B
|
||||
|
||||
```
|
||||
|
||||
|
||||
|
||||
296
docs/README_FR.md
Normal file
296
docs/README_FR.md
Normal file
@@ -0,0 +1,296 @@
|
||||
> **Note**
|
||||
>
|
||||
> Ce fichier README est généré automatiquement par le plugin de traduction markdown de ce projet et n'est peut - être pas correct à 100%.
|
||||
>
|
||||
|
||||
# <img src="logo.png" width="40" > ChatGPT Optimisation Académique
|
||||
|
||||
**Si vous aimez ce projet, donnez-lui une étoile; si vous avez inventé des raccourcis académiques plus utiles ou des plugins fonctionnels, n'hésitez pas à ouvrir une demande ou une demande de traction. Nous avons également un fichier README en [anglais|](docs/README_EN.md)[japonais|](docs/README_JP.md)[russe|](docs/README_RS.md)[français](docs/README_FR.md) traduit par ce projet lui-même.**
|
||||
|
||||
> **Note**
|
||||
>
|
||||
> 1. Veuillez noter que seuls les plugins de fonction signalés en **rouge** sont capables de lire les fichiers, certains plugins se trouvent dans le **menu déroulant** de la section plugin. Nous sommes également les bienvenus avec la plus haute priorité pour traiter et accepter tout nouveau PR de plugin!
|
||||
>
|
||||
> 2. Chaque fichier dans ce projet est expliqué en détail dans l'auto-analyse [self_analysis.md](https://github.com/binary-husky/chatgpt_academic/wiki/chatgpt-academic%E9%A1%B9%E7%9B%AE%E8%87%AA%E8%AF%91%E8%A7%A3%E6%8A%A5%E5%91%8A). Avec l'itération des versions, vous pouvez également cliquer sur les plugins fonctionnels pertinents pour appeler GPT et générer un rapport d'auto-analyse projet mis à jour. Les questions fréquemment posées sont résumées dans le [wiki](https://github.com/binary-husky/chatgpt_academic/wiki/%E5%B8%B8%E8%A7%81%E9%97%AE%E9%A2%98).
|
||||
>
|
||||
|
||||
<div align="center">
|
||||
|
||||
Fonctionnalité | Description
|
||||
--- | ---
|
||||
Polissage en un clic | Prend en charge la correction en un clic et la recherche d'erreurs de syntaxe dans les documents de recherche.
|
||||
Traduction Chinois-Anglais en un clic | Une touche pour traduire la partie chinoise en anglais ou celle anglaise en chinois.
|
||||
Explication de code en un clic | Affiche et explique correctement le code.
|
||||
[Raccourcis clavier personnalisables](https://www.bilibili.com/video/BV14s4y1E7jN) | Prend en charge les raccourcis clavier personnalisables.
|
||||
[Configuration du serveur proxy](https://www.bilibili.com/video/BV1rc411W7Dr) | Prend en charge la configuration du serveur proxy.
|
||||
Conception modulaire | Prend en charge la personnalisation des plugins de fonctions et des [plugins] de fonctions hiérarchiques personnalisés, et les plugins prennent en charge [la mise à jour à chaud](https://github.com/binary-husky/chatgpt_academic/wiki/%E5%87%BD%E6%95%B0%E6%8F%92%E4%BB%B6%E6%8C%87%E5%8D%97).
|
||||
[Auto-analyse du programme](https://www.bilibili.com/video/BV1cj411A7VW) | [Plugins] [Lire en un clic](https://github.com/binary-husky/chatgpt_academic/wiki/chatgpt-academic%E9%A1%B9%E7%9B%AE%E8%87%AA%E8%AF%91%E8%A7%A3%E6%8A%A5%E5%91%8A) le code source de ce projet.
|
||||
[Analyse de programme](https://www.bilibili.com/video/BV1cj411A7VW) | [Plugins] En un clic, les projets Python/C/C++/Java/Lua/... peuvent être analysés.
|
||||
Lire le document de recherche | [Plugins] Lisez le résumé de l'article en latex et générer un résumé.
|
||||
Traduction et polissage de l'article complet en LaTeX | [Plugins] Une touche pour traduire ou corriger en LaTeX
|
||||
Génération Commentaire de fonction en vrac | [Plugins] Lisez en un clic les fonctions et générez des commentaires de fonction.
|
||||
Rapport d'analyse automatique des chats générés | [Plugins] Génère un rapport de synthèse après l'exécution.
|
||||
[Assistant arxiv](https://www.bilibili.com/video/BV1LM4y1279X) | [Plugins] Entrez l'url de l'article arxiv pour traduire le résumé + télécharger le PDF en un clic
|
||||
[Traduction complète des articles PDF](https://www.bilibili.com/video/BV1KT411x7Wn) | [Plugins] Extraire le titre et le résumé de l'article PDF + Traduire le texte entier (multithread)
|
||||
[Aide à la recherche Google Academ](https://www.bilibili.com/video/BV19L411U7ia) | [Plugins] Donnez à GPT l'URL de n'importe quelle page de recherche Google Academ pour vous aider à sélectionner des articles intéressants
|
||||
Affichage de formules/images/tableaux | Afficher la forme traduite et rendue d'une formule en même temps, plusieurs formules et surlignage du code prend en charge
|
||||
Prise en charge des plugins multithread | Prise en charge de l'appel multithread de chatgpt, traitement en masse de texte ou de programmes en un clic
|
||||
Activer le thème Gradio sombre [theme](https://github.com/binary-husky/chatgpt_academic/issues/173) au démarrage | Ajoutez ```/?__dark-theme=true``` à l'URL du navigateur pour basculer vers le thème sombre
|
||||
[Prise en charge de plusieurs modèles LLM](https://www.bilibili.com/video/BV1wT411p7yf), [prise en charge de l'interface API2D](https://api2d.com/) | Comment cela serait-il de se faire servir par GPT3.5, GPT4 et la [ChatGLM de Tsinghua](https://github.com/THUDM/ChatGLM-6B) en même temps?
|
||||
Expérience en ligne d'huggingface sans science | Après vous être connecté à huggingface, copiez [cet espace](https://huggingface.co/spaces/qingxu98/gpt-academic)
|
||||
... | ...
|
||||
|
||||
</div>
|
||||
|
||||
|
||||
Vous êtes un traducteur professionnel d'articles universitaires en français.
|
||||
|
||||
Ceci est un fichier Markdown, veuillez le traduire en français sans modifier les commandes Markdown existantes :
|
||||
|
||||
- Nouvelle interface (modifiable en modifiant l'option de mise en page dans config.py pour basculer entre les mises en page gauche-droite et haut-bas)
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/230361456-61078362-a966-4eb5-b49e-3c62ef18b860.gif" width="700" >
|
||||
</div>
|
||||
|
||||
|
||||
- Tous les boutons sont générés dynamiquement en lisant functional.py, les utilisateurs peuvent ajouter librement des fonctions personnalisées pour libérer le presse-papiers.
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/231975334-b4788e91-4887-412f-8b43-2b9c5f41d248.gif" width="700" >
|
||||
</div>
|
||||
|
||||
- Correction/amélioration
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/231980294-f374bdcb-3309-4560-b424-38ef39f04ebd.gif" width="700" >
|
||||
</div>
|
||||
|
||||
- Si la sortie contient des formules, elles seront affichées simultanément sous forme de de texte brut et de forme rendue pour faciliter la copie et la lecture.
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/230598842-1d7fcddd-815d-40ee-af60-baf488a199df.png" width="700" >
|
||||
</div>
|
||||
|
||||
- Pas envie de lire le code du projet ? Faites votre propre démo avec ChatGPT.
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/226935232-6b6a73ce-8900-4aee-93f9-733c7e6fef53.png" width="700" >
|
||||
</div>
|
||||
|
||||
- Utilisation combinée de plusieurs modèles de langage sophistiqués (ChatGLM + OpenAI-GPT3.5 + [API2D](https://api2d.com/)-GPT4)
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/232537274-deca0563-7aa6-4b5d-94a2-b7c453c47794.png" width="700" >
|
||||
</div>
|
||||
|
||||
Utilisation combinée de plusieurs modèles de langage sophistiqués en version de test [huggingface](https://huggingface.co/spaces/qingxu98/academic-chatgpt-beta) (la version huggingface ne prend pas en charge Chatglm).
|
||||
|
||||
|
||||
---
|
||||
|
||||
## Installation - Méthode 1 : Exécution directe (Windows, Linux or MacOS)
|
||||
|
||||
1. Téléchargez le projet
|
||||
```sh
|
||||
git clone https://github.com/binary-husky/chatgpt_academic.git
|
||||
cd chatgpt_academic
|
||||
```
|
||||
|
||||
2. Configuration de l'API_KEY et des paramètres de proxy
|
||||
|
||||
Dans `config.py`, configurez les paramètres de proxy et de clé d'API OpenAI, comme indiqué ci-dessous
|
||||
```
|
||||
1. Si vous êtes en Chine, vous devez configurer un proxy étranger pour utiliser l'API OpenAI en toute transparence. Pour ce faire, veuillez lire attentivement le fichier config.py (1. Modifiez l'option USE_PROXY ; 2. Modifiez les paramètres de proxies comme indiqué dans les instructions).
|
||||
2. Configurez votre clé API OpenAI. Vous devez vous inscrire sur le site web d'OpenAI pour obtenir une clé API. Une fois que vous avez votre clé API, vous pouvez la configurer dans le fichier config.py.
|
||||
3. Tous les problèmes liés aux réseaux de proxy (temps d'attente, non-fonctionnement des proxies) sont résumés dans https://github.com/binary-husky/chatgpt_academic/issues/1.
|
||||
```
|
||||
(Remarque : le programme vérifie d'abord s'il existe un fichier de configuration privé nommé `config_private.py`, et utilise les configurations de celui-ci à la place de celles du fichier `config.py`. Par conséquent, si vous comprenez notre logique de lecture de configuration, nous vous recommandons fortement de créer un nouveau fichier de configuration nommé `config_private.py` à côté de `config.py` et de transférer (copier) les configurations de celui-ci dans `config_private.py`. `config_private.py` n'est pas contrôlé par git et rend vos informations personnelles plus sûres.)
|
||||
|
||||
3. Installation des dépendances
|
||||
```sh
|
||||
# (Option 1) Recommandé
|
||||
python -m pip install -r requirements.txt
|
||||
|
||||
# (Option 2) Si vous utilisez anaconda, les étapes sont similaires :
|
||||
# (Option 2.1) conda create -n gptac_venv python=3.11
|
||||
# (Option 2.2) conda activate gptac_venv
|
||||
# (Option 2.3) python -m pip install -r requirements.txt
|
||||
|
||||
# note : Utilisez la source pip officielle ou la source pip Alibaba. D'autres sources (comme celles des universités) pourraient poser problème. Pour utiliser temporairement une autre source, utilisez :
|
||||
# python -m pip install -r requirements.txt -i https://mirrors.aliyun.com/pypi/simple/
|
||||
```
|
||||
|
||||
Si vous avez besoin de soutenir ChatGLM de Tsinghua, vous devez installer plus de dépendances (si vous n'êtes pas familier avec Python ou que votre ordinateur n'est pas assez performant, nous vous recommandons de ne pas essayer) :
|
||||
```sh
|
||||
python -m pip install -r request_llm/requirements_chatglm.txt
|
||||
```
|
||||
|
||||
4. Exécution
|
||||
```sh
|
||||
python main.py
|
||||
```
|
||||
|
||||
5. Tester les plugins de fonctions
|
||||
```
|
||||
- Test Python Project Analysis
|
||||
Dans la zone de saisie, entrez `./crazy_functions/test_project/python/dqn`, puis cliquez sur "Parse Entire Python Project"
|
||||
- Test d'auto-lecture du code
|
||||
Cliquez sur "[Démo multi-thread] Parser ce projet lui-même (auto-traduction de la source)"
|
||||
- Test du modèle de fonctionnalité expérimentale (exige une réponse de l'IA à ce qui est arrivé aujourd'hui dans l'histoire). Vous pouvez utiliser cette fonctionnalité comme modèle pour des fonctions plus complexes.
|
||||
Cliquez sur "[Démo modèle de plugin de fonction] Histoire du Jour"
|
||||
- Le menu déroulant de la zone de plugin de fonctionnalité contient plus de fonctionnalités à sélectionner.
|
||||
```
|
||||
|
||||
## Installation - Méthode 2 : Utilisation de docker (Linux)
|
||||
|
||||
|
||||
Vous êtes un traducteur professionnel d'articles académiques en français.
|
||||
|
||||
1. ChatGPT seul (recommandé pour la plupart des gens)
|
||||
``` sh
|
||||
# Télécharger le projet
|
||||
git clone https://github.com/binary-husky/chatgpt_academic.git
|
||||
cd chatgpt_academic
|
||||
# Configurer le proxy outre-mer et la clé API OpenAI
|
||||
Modifier le fichier config.py avec n'importe quel éditeur de texte
|
||||
# Installer
|
||||
docker build -t gpt-academic .
|
||||
# Exécuter
|
||||
docker run --rm -it --net=host gpt-academic
|
||||
|
||||
# Tester les modules de fonction
|
||||
## Tester la fonction modèle des modules (requiert la réponse de GPT à "qu'est-ce qui s'est passé dans l'histoire aujourd'hui ?"), vous pouvez utiliser cette fonction en tant que modèle pour implémenter des fonctions plus complexes.
|
||||
Cliquez sur "[Exemple de modèle de module] Histoire d'aujourd'hui"
|
||||
## Tester le résumé écrit pour le projet LaTeX
|
||||
Dans la zone de saisie, tapez ./crazy_functions/test_project/latex/attention, puis cliquez sur "Lire le résumé de l'article de recherche LaTeX"
|
||||
## Tester l'analyse du projet Python
|
||||
Dans la zone de saisie, tapez ./crazy_functions/test_project/python/dqn, puis cliquez sur "Analyser l'ensemble du projet Python"
|
||||
|
||||
D'autres fonctions sont disponibles dans la liste déroulante des modules de fonction.
|
||||
```
|
||||
|
||||
2. ChatGPT+ChatGLM (nécessite une grande connaissance de docker et une configuration informatique suffisamment puissante)
|
||||
``` sh
|
||||
# Modifier le dockerfile
|
||||
cd docs && nano Dockerfile+ChatGLM
|
||||
# Comment construire | 如何构建 (Dockerfile+ChatGLM在docs路径下,请先cd docs)
|
||||
docker build -t gpt-academic --network=host -f Dockerfile+ChatGLM .
|
||||
# Comment exécuter | 如何运行 (1) Directement exécuter :
|
||||
docker run --rm -it --net=host --gpus=all gpt-academic
|
||||
# Comment exécuter | 如何运行 (2) Je veux effectuer quelques ajustements dans le conteneur avant de lancer :
|
||||
docker run --rm -it --net=host --gpus=all gpt-academic bash
|
||||
```
|
||||
|
||||
## Installation - Méthode 3 : Autres méthodes de déploiement
|
||||
|
||||
1. Déploiement sur un cloud serveur distant
|
||||
Veuillez consulter le [wiki de déploiement-1](https://github.com/binary-husky/chatgpt_academic/wiki/%E4%BA%91%E6%9C%8D%E5%8A%A1%E5%99%A8%E8%BF%9C%E7%A8%8B%E9%83%A8%E7%BD%B2%E6%8C%87%E5%8D%97)
|
||||
|
||||
2. Utilisation de WSL2 (Windows Subsystem for Linux)
|
||||
Veuillez consulter le [wiki de déploiement-2](https://github.com/binary-husky/chatgpt_academic/wiki/%E4%BD%BF%E7%94%A8WSL2%EF%BC%88Windows-Subsystem-for-Linux-%E5%AD%90%E7%B3%BB%E7%BB%9F%EF%BC%89%E9%83%A8%E7%BD%B2)
|
||||
|
||||
|
||||
## Configuration de la procuration de l'installation
|
||||
### Méthode 1 : Méthode conventionnelle
|
||||
[Configuration de la procuration](https://github.com/binary-husky/chatgpt_academic/issues/1)
|
||||
|
||||
### Méthode 2 : Tutoriel pour débutant pur
|
||||
[Tutoriel pour débutant pur](https://github.com/binary-husky/chatgpt_academic/wiki/%E4%BB%A3%E7%90%86%E8%BD%AF%E4%BB%B6%E9%97%AE%E9%A2%98%E7%9A%84%E6%96%B0%E6%89%8B%E8%A7%A3%E5%86%B3%E6%96%B9%E6%B3%95%EF%BC%88%E6%96%B9%E6%B3%95%E5%8F%AA%E9%80%82%E7%94%A8%E4%BA%8E%E6%96%B0%E6%89%8B%EF%BC%89)
|
||||
|
||||
|
||||
---
|
||||
|
||||
## Personnalisation des nouveaux boutons pratiques (personnalisation des raccourcis académiques)
|
||||
Ouvrez le fichier `core_functional.py` avec n'importe quel éditeur de texte, ajoutez les éléments suivants, puis redémarrez le programme. (Si le bouton a déjà été ajouté avec succès et est visible, le préfixe et le suffixe pris en charge peuvent être modifiés à chaud sans avoir besoin de redémarrer le programme.)
|
||||
Par exemple:
|
||||
```
|
||||
"Traduction Français-Chinois": {
|
||||
# Préfixe, qui sera ajouté avant votre saisie. Par exemple, pour décrire votre demande, telle que la traduction, le débogage de code, l'amélioration, etc.
|
||||
"Prefix": "Veuillez traduire le contenu ci-dessous en chinois, puis expliquer chaque terme propre mentionné dans un tableau Markdown :\n\n",
|
||||
|
||||
# Suffixe, qui sera ajouté après votre saisie. Par exemple, en combinaison avec un préfixe, vous pouvez mettre le contenu de votre saisie entre guillemets.
|
||||
"Suffix": "",
|
||||
},
|
||||
```
|
||||
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/226899272-477c2134-ed71-4326-810c-29891fe4a508.png" width="500" >
|
||||
</div>
|
||||
|
||||
---
|
||||
|
||||
|
||||
## Présentation de certaines fonctionnalités
|
||||
|
||||
### Affichage des images:
|
||||
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/228737599-bf0a9d9c-1808-4f43-ae15-dfcc7af0f295.png" width="800" >
|
||||
</div>
|
||||
|
||||
|
||||
### Si un programme peut comprendre et décomposer lui-même :
|
||||
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/226936850-c77d7183-0749-4c1c-9875-fd4891842d0c.png" width="800" >
|
||||
</div>
|
||||
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/226936618-9b487e4b-ab5b-4b6e-84c6-16942102e917.png" width="800" >
|
||||
</div>
|
||||
|
||||
|
||||
### Analyse de tout projet Python/Cpp quelconque :
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/226935232-6b6a73ce-8900-4aee-93f9-733c7e6fef53.png" width="800" >
|
||||
</div>
|
||||
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/226969067-968a27c1-1b9c-486b-8b81-ab2de8d3f88a.png" width="800" >
|
||||
</div>
|
||||
|
||||
### Lecture et résumé générés automatiquement pour les articles en Latex
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/227504406-86ab97cd-f208-41c3-8e4a-7000e51cf980.png" width="800" >
|
||||
</div>
|
||||
|
||||
### Génération de rapports automatique
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/227503770-fe29ce2c-53fd-47b0-b0ff-93805f0c2ff4.png" height="300" >
|
||||
<img src="https://user-images.githubusercontent.com/96192199/227504617-7a497bb3-0a2a-4b50-9a8a-95ae60ea7afd.png" height="300" >
|
||||
<img src="https://user-images.githubusercontent.com/96192199/227504005-efeaefe0-b687-49d0-bf95-2d7b7e66c348.png" height="300" >
|
||||
</div>
|
||||
|
||||
### Conception de fonctionnalités modulaires
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/229288270-093643c1-0018-487a-81e6-1d7809b6e90f.png" height="400" >
|
||||
<img src="https://user-images.githubusercontent.com/96192199/227504931-19955f78-45cd-4d1c-adac-e71e50957915.png" height="400" >
|
||||
</div>
|
||||
|
||||
|
||||
### Traduction de code source en anglais
|
||||
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/229720562-fe6c3508-6142-4635-a83d-21eb3669baee.png" height="400" >
|
||||
</div>
|
||||
|
||||
## À faire et planification de version :
|
||||
- version 3.2+ (à faire) : Prise en charge de plus de paramètres d'interface de plugin de fonction
|
||||
- version 3.1 : Prise en charge de l'interrogation simultanée de plusieurs modèles GPT ! Prise en charge de l'API2d, prise en charge de la répartition de charge de plusieurs clés API
|
||||
- version 3.0 : Prise en charge de chatglm et d'autres petits llm
|
||||
- version 2.6 : Réorganisation de la structure du plugin, amélioration de l'interactivité, ajout de plus de plugins
|
||||
- version 2.5 : Mise à jour automatique, résolution du problème de dépassement de jeton et de texte trop long lors de la compilation du code source complet
|
||||
- version 2.4 : (1) Ajout de la fonctionnalité de traduction intégrale de PDF ; (2) Ajout d'une fonctionnalité de changement de position de zone de saisie ; (3) Ajout d'une option de disposition verticale ; (4) Optimisation du plugin de fonction multi-thread.
|
||||
- version 2.3 : Amélioration de l'interactivité multi-thread
|
||||
- version 2.2 : Prise en charge du rechargement à chaud du plugin de fonction
|
||||
- version 2.1 : Mise en page pliable
|
||||
- version 2.0 : Introduction du plugin de fonction modulaire
|
||||
- version 1.0 : Fonctionnalité de base
|
||||
|
||||
## Références et apprentissage
|
||||
|
||||
```
|
||||
De nombreux designs d'autres projets exceptionnels ont été utilisés pour référence dans le code, notamment :
|
||||
|
||||
# Projet 1 : De nombreuses astuces ont été empruntées à ChuanhuChatGPT
|
||||
https://github.com/GaiZhenbiao/ChuanhuChatGPT
|
||||
|
||||
# Projet 2 : ChatGLM-6B de Tsinghua :
|
||||
https://github.com/THUDM/ChatGLM-6B
|
||||
```
|
||||
|
||||
302
docs/README_JP.md
Normal file
302
docs/README_JP.md
Normal file
@@ -0,0 +1,302 @@
|
||||
> **Note**
|
||||
>
|
||||
> このReadmeファイルは、このプロジェクトのmarkdown翻訳プラグインによって自動的に生成されたもので、100%正確ではない可能性があります。
|
||||
>
|
||||
|
||||
# <img src="logo.png" width="40" > ChatGPT 学術最適化
|
||||
|
||||
**このプロジェクトが好きだったら、スターをつけてください。もし、より使いやすい学術用のショートカットキーまたはファンクションプラグインを発明した場合は、issueを発行するかpull requestを作成してください。また、このプロジェクト自体によって翻訳されたREADMEは[英語説明書|](docs/README_EN.md)[日本語説明書|](docs/README_JP.md)[ロシア語説明書|](docs/README_RS.md)[フランス語説明書](docs/README_FR.md)もあります。**
|
||||
|
||||
> **注意事項**
|
||||
>
|
||||
> 1. **赤色**のラベルが付いているファンクションプラグイン(ボタン)のみファイルを読み込めます。一部のプラグインはプラグインエリアのドロップダウンメニューにあります。新しいプラグインのPRを歓迎いたします!
|
||||
>
|
||||
> 2. このプロジェクトの各ファイルの機能は`self_analysis.md`(自己解析レポート)で詳しく説明されています。バージョンが追加されると、関連するファンクションプラグインをクリックして、GPTを呼び出して自己解析レポートを再生成することができます。一般的な質問は`wiki`にまとめられています。(`https://github.com/binary-husky/chatgpt_academic/wiki/%E5%B8%B8%E8%A7%81%E9%97%AE%E9%A2%98`)
|
||||
|
||||
|
||||
<div align="center">
|
||||
|
||||
機能 | 説明
|
||||
--- | ---
|
||||
ワンクリック整形 | 論文の文法エラーを一括で正確に修正できます。
|
||||
ワンクリック日英翻訳 | 日英翻訳には、ワンクリックで対応できます。
|
||||
ワンクリックコード説明 | コードの正しい表示と説明が可能です。
|
||||
[カスタムショートカットキー](https://www.bilibili.com/video/BV14s4y1E7jN) | カスタムショートカットキーをサポートします。
|
||||
[プロキシサーバーの設定](https://www.bilibili.com/video/BV1rc411W7Dr) | プロキシサーバーの設定をサポートします。
|
||||
モジュラーデザイン | カスタム高階関数プラグインと[関数プラグイン]、プラグイン[ホット更新]のサポートが可能です。詳細は[こちら](https://github.com/binary-husky/chatgpt_academic/wiki/%E5%87%BD%E6%95%B0%E6%8F%92%E4%BB%B6%E6%8C%87%E5%8D%97)
|
||||
[自己プログラム解析](https://www.bilibili.com/video/BV1cj411A7VW) | [関数プラグイン][ワンクリック理解](https://github.com/binary-husky/chatgpt_academic/wiki/chatgpt-academic%E9%A1%B9%E7%9B%AE%E8%87%AA%E8%AF%91%E8%A7%A3%E6%8A%A5%E5%91%8A)このプロジェクトのソースコード
|
||||
[プログラム解析機能](https://www.bilibili.com/video/BV1cj411A7VW) | [関数プラグイン] ワンクリックで別のPython/C/C++/Java/Lua/...プロジェクトツリーを解析できます。
|
||||
論文読解 | [関数プラグイン] LaTeX論文の全文をワンクリックで解読し、要約を生成します。
|
||||
LaTeX全文翻訳、整形 | [関数プラグイン] ワンクリックでLaTeX論文を翻訳または整形できます。
|
||||
注釈生成 | [関数プラグイン] ワンクリックで関数の注釈を大量に生成できます。
|
||||
チャット分析レポート生成 | [関数プラグイン] 実行後、まとめレポートを自動生成します。
|
||||
[arxivヘルパー](https://www.bilibili.com/video/BV1LM4y1279X) | [関数プラグイン] 入力したarxivの記事URLで要約をワンクリック翻訳+PDFダウンロードができます。
|
||||
[PDF論文全文翻訳機能](https://www.bilibili.com/video/BV1KT411x7Wn) | [関数プラグイン] PDF論文タイトルと要約を抽出し、全文を翻訳します(マルチスレッド)。
|
||||
[Google Scholar Integratorヘルパー](https://www.bilibili.com/video/BV19L411U7ia) | [関数プラグイン] 任意のGoogle Scholar検索ページURLを指定すると、gptが興味深い記事を選択します。
|
||||
数式/画像/テーブル表示 | 数式のTex形式とレンダリング形式を同時に表示できます。数式、コードのハイライトをサポートしています。
|
||||
マルチスレッド関数プラグインサポート | ChatGPTをマルチスレッドで呼び出すことができ、大量のテキストやプログラムを簡単に処理できます。
|
||||
ダークグラジオ[テーマ](https://github.com/binary-husky/chatgpt_academic/issues/173)の起動 | 「/?__dark-theme=true」というURLをブラウザに追加することで、ダークテーマに切り替えることができます。
|
||||
[多数のLLMモデル](https://www.bilibili.com/video/BV1wT411p7yf)をサポート、[API2D](https://api2d.com/)インターフェースをサポート | GPT3.5、GPT4、[清華ChatGLM](https://github.com/THUDM/ChatGLM-6B)による同時サポートは、とても素晴らしいですね!
|
||||
huggingface免科学上网[オンライン版](https://huggingface.co/spaces/qingxu98/gpt-academic) | huggingfaceにログイン後、[このスペース](https://huggingface.co/spaces/qingxu98/gpt-academic)をコピーしてください。
|
||||
...... | ......
|
||||
|
||||
|
||||
</div>
|
||||
|
||||
|
||||
- 新しいインターフェース(config.pyのLAYOUTオプションを変更するだけで、「左右レイアウト」と「上下レイアウト」を切り替えることができます)
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/230361456-61078362-a966-4eb5-b49e-3c62ef18b860.gif" width="700" >
|
||||
</div>
|
||||
|
||||
|
||||
- すべてのボタンは、functional.pyを読み込んで動的に生成されます。カスタム機能を自由に追加して、クリップボードを解放します
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/231975334-b4788e91-4887-412f-8b43-2b9c5f41d248.gif" width="700" >
|
||||
</div>
|
||||
|
||||
- 色を修正/修正
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/231980294-f374bdcb-3309-4560-b424-38ef39f04ebd.gif" width="700" >
|
||||
</div>
|
||||
|
||||
- 出力に数式が含まれている場合、TeX形式とレンダリング形式の両方が表示され、コピーと読み取りが容易になります
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/230598842-1d7fcddd-815d-40ee-af60-baf488a199df.png" width="700" >
|
||||
</div>
|
||||
|
||||
- プロジェクトのコードを見るのが面倒?chatgptに整備されたプロジェクトを直接与えましょう
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/226935232-6b6a73ce-8900-4aee-93f9-733c7e6fef53.png" width="700" >
|
||||
</div>
|
||||
|
||||
- 多数の大規模言語モデルの混合呼び出し(ChatGLM + OpenAI-GPT3.5 + [API2D](https://api2d.com/)-GPT4)
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/232537274-deca0563-7aa6-4b5d-94a2-b7c453c47794.png" width="700" >
|
||||
</div>
|
||||
|
||||
多数の大規模言語モデルの混合呼び出し[huggingfaceテスト版](https://huggingface.co/spaces/qingxu98/academic-chatgpt-beta)(huggigface版はchatglmをサポートしていません)
|
||||
|
||||
|
||||
---
|
||||
|
||||
## インストール-方法1:直接運転 (Windows、LinuxまたはMacOS)
|
||||
|
||||
1. プロジェクトをダウンロードします。
|
||||
```sh
|
||||
git clone https://github.com/binary-husky/chatgpt_academic.git
|
||||
cd chatgpt_academic
|
||||
```
|
||||
|
||||
2. API_KEYとプロキシ設定を構成する
|
||||
|
||||
`config.py`で、海外のProxyとOpenAI API KEYを構成して説明します。
|
||||
```
|
||||
1.あなたが中国にいる場合、OpenAI APIをスムーズに使用するには海外プロキシを設定する必要があります。構成の詳細については、config.py(1.その中のUSE_PROXYをTrueに変更し、2.手順に従ってプロキシを変更する)を詳細に読んでください。
|
||||
2. OpenAI API KEYを構成する。OpenAIのウェブサイトでAPI KEYを取得してください。一旦API KEYを手に入れると、config.pyファイルで設定するだけです。
|
||||
3.プロキシネットワークに関連する問題(ネットワークタイムアウト、プロキシが動作しない)をhttps://github.com/binary-husky/chatgpt_academic/issues/1にまとめました。
|
||||
```
|
||||
(P.S. プログラム実行時にconfig.pyの隣にconfig_private.pyという名前のプライバシー設定ファイルを作成し、同じ名前の設定を上書きするconfig_private.pyが存在するかどうかを優先的に確認します。そのため、私たちの構成読み取りロジックを理解できる場合は、config.pyの隣にconfig_private.pyという名前の新しい設定ファイルを作成し、その中のconfig.pyから設定を移動してください。config_private.pyはgitで保守されていないため、プライバシー情報をより安全にすることができます。)
|
||||
|
||||
3. 依存関係をインストールします。
|
||||
```sh
|
||||
# 選択肢があります。
|
||||
python -m pip install -r requirements.txt
|
||||
|
||||
|
||||
# (選択肢2) もしAnacondaを使用する場合、手順は同様です:
|
||||
# (選択肢2.1) conda create -n gptac_venv python=3.11
|
||||
# (選択肢2.2) conda activate gptac_venv
|
||||
# (選択肢2.3) python -m pip install -r requirements.txt
|
||||
|
||||
# 注: 公式のpipソースまたはAlibabaのpipソースを使用してください。 別のpipソース(例:一部の大学のpip)は問題が発生する可能性があります。 一時的なソースの切り替え方法:
|
||||
# python -m pip install -r requirements.txt -i https://mirrors.aliyun.com/pypi/simple/
|
||||
```
|
||||
|
||||
もしあなたが清華ChatGLMをサポートする必要がある場合、さらに多くの依存関係をインストールする必要があります(Pythonに慣れない方やコンピューターの設定が十分でない方は、試みないことをお勧めします):
|
||||
```sh
|
||||
python -m pip install -r request_llm/requirements_chatglm.txt
|
||||
```
|
||||
|
||||
4. 実行
|
||||
```sh
|
||||
python main.py
|
||||
```
|
||||
|
||||
5. 関数プラグインのテスト
|
||||
```
|
||||
- Pythonプロジェクト分析のテスト
|
||||
入力欄に `./crazy_functions/test_project/python/dqn` と入力し、「Pythonプロジェクト全体の解析」をクリックします。
|
||||
- 自己コード解読のテスト
|
||||
「[マルチスレッドデモ] このプロジェクト自体を解析します(ソースを翻訳して解読します)」をクリックします。
|
||||
- 実験的な機能テンプレート関数のテスト(GPTが「今日の歴史」に何が起こったかを回答することが求められます)。この関数をテンプレートとして使用して、より複雑な機能を実装できます。
|
||||
「[関数プラグインテンプレートデモ] 今日の歴史」をクリックします。
|
||||
- 関数プラグインエリアのドロップダウンメニューには他にも選択肢があります。
|
||||
```
|
||||
|
||||
## インストール方法2:Dockerを使用する(Linux)
|
||||
|
||||
1. ChatGPTのみ(大多数の人にお勧めです)
|
||||
``` sh
|
||||
# プロジェクトのダウンロード
|
||||
git clone https://github.com/binary-husky/chatgpt_academic.git
|
||||
cd chatgpt_academic
|
||||
# 海外プロキシとOpenAI API KEYの設定
|
||||
config.pyを任意のテキストエディタで編集する
|
||||
# インストール
|
||||
docker build -t gpt-academic .
|
||||
# 実行
|
||||
docker run --rm -it --net=host gpt-academic
|
||||
|
||||
# 関数プラグインのテスト
|
||||
## 関数プラグインテンプレート関数のテスト(GPTが「今日の歴史」に何が起こったかを回答することが求められます)。この関数をテンプレートとして使用して、より複雑な機能を実装できます。
|
||||
「[関数プラグインテンプレートデモ] 今日の歴史」をクリックします。
|
||||
## Latexプロジェクトの要約を書くテスト
|
||||
入力欄に./crazy_functions/test_project/latex/attentionと入力し、「テックス論文を読んで要約を書く」をクリックします。
|
||||
## Pythonプロジェクト分析のテスト
|
||||
入力欄に./crazy_functions/test_project/python/dqnと入力し、[Pythonプロジェクトの全解析]をクリックします。
|
||||
|
||||
関数プラグインエリアのドロップダウンメニューには他にも選択肢があります。
|
||||
```
|
||||
|
||||
2. ChatGPT + ChatGLM(Dockerに非常に詳しい人+十分なコンピューター設定が必要)
|
||||
|
||||
|
||||
|
||||
```sh
|
||||
# Dockerfileの編集
|
||||
cd docs && nano Dockerfile+ChatGLM
|
||||
# ビルド方法
|
||||
docker build -t gpt-academic --network=host -f Dockerfile+ChatGLM .
|
||||
# 実行方法 (1) 直接実行:
|
||||
docker run --rm -it --net=host --gpus=all gpt-academic
|
||||
# 実行方法 (2) コンテナに入って調整する:
|
||||
docker run --rm -it --net=host --gpus=all gpt-academic bash
|
||||
```
|
||||
|
||||
## インストール方法3:その他のデプロイ方法
|
||||
|
||||
1. クラウドサーバーデプロイ
|
||||
[デプロイwiki-1](https://github.com/binary-husky/chatgpt_academic/wiki/%E4%BA%91%E6%9C%8D%E5%8A%A1%E5%99%A8%E8%BF%9C%E7%A8%8B%E9%83%A8%E7%BD%B2%E6%8C%87%E5%8D%97)
|
||||
|
||||
2. WSL2を使用 (Windows Subsystem for Linux)
|
||||
[デプロイwiki-2](https://github.com/binary-husky/chatgpt_academic/wiki/%E4%BD%BF%E7%94%A8WSL2%EF%BC%88Windows-Subsystem-for-Linux-%E5%AD%90%E7%B3%BB%E7%BB%9F%EF%BC%89%E9%83%A8%E7%BD%B2)
|
||||
|
||||
|
||||
## インストール-プロキシ設定
|
||||
1. 通常の方法
|
||||
[プロキシを設定する](https://github.com/binary-husky/chatgpt_academic/issues/1)
|
||||
|
||||
2. 初心者向けチュートリアル
|
||||
[初心者向けチュートリアル](https://github.com/binary-husky/chatgpt_academic/wiki/%E4%BB%A3%E7%90%86%E8%BD%AF%E4%BB%B6%E9%97%AE%E9%A2%98%E7%9A%84%E6%96%B0%E6%89%8B%E8%A7%A3%E5%86%B3%E6%96%B9%E6%B3%95%EF%BC%88%E6%96%B9%E6%B3%95%E5%8F%AA%E9%80%82%E7%94%A8%E4%BA%8E%E6%96%B0%E6%89%8B%EF%BC%89)
|
||||
|
||||
|
||||
---
|
||||
|
||||
## カスタムボタンの追加(学術ショートカットキー)
|
||||
|
||||
`core_functional.py`を任意のテキストエディタで開き、以下のエントリーを追加し、プログラムを再起動してください。(ボタンが追加されて表示される場合、前置詞と後置詞はホット編集がサポートされているため、プログラムを再起動せずに即座に有効になります。)
|
||||
|
||||
例:
|
||||
```
|
||||
"超级英译中": {
|
||||
# 前置詞 - あなたの要求を説明するために使用されます。翻訳、コードの説明、編集など。
|
||||
"Prefix": "以下のコンテンツを中国語に翻訳して、マークダウンテーブルを使用して専門用語を説明してください。\n\n",
|
||||
|
||||
# 後置詞 - プレフィックスと共に使用すると、入力内容を引用符で囲むことができます。
|
||||
"Suffix": "",
|
||||
},
|
||||
```
|
||||
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/226899272-477c2134-ed71-4326-810c-29891fe4a508.png" width="500" >
|
||||
</div>
|
||||
|
||||
|
||||
---
|
||||
|
||||
## いくつかの機能の例
|
||||
|
||||
### 画像表示:
|
||||
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/228737599-bf0a9d9c-1808-4f43-ae15-dfcc7af0f295.png" width="800" >
|
||||
</div>
|
||||
|
||||
|
||||
### プログラムが自己解析できる場合:
|
||||
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/226936850-c77d7183-0749-4c1c-9875-fd4891842d0c.png" width="800" >
|
||||
</div>
|
||||
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/226936618-9b487e4b-ab5b-4b6e-84c6-16942102e917.png" width="800" >
|
||||
</div>
|
||||
|
||||
### 他のPython/Cppプロジェクトの解析:
|
||||
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/226935232-6b6a73ce-8900-4aee-93f9-733c7e6fef53.png" width="800" >
|
||||
</div>
|
||||
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/226969067-968a27c1-1b9c-486b-8b81-ab2de8d3f88a.png" width="800" >
|
||||
</div>
|
||||
|
||||
### Latex論文の一括読解と要約生成
|
||||
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/227504406-86ab97cd-f208-41c3-8e4a-7000e51cf980.png" width="800" >
|
||||
</div>
|
||||
|
||||
### 自動報告生成
|
||||
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/227503770-fe29ce2c-53fd-47b0-b0ff-93805f0c2ff4.png" height="300" >
|
||||
<img src="https://user-images.githubusercontent.com/96192199/227504617-7a497bb3-0a2a-4b50-9a8a-95ae60ea7afd.png" height="300" >
|
||||
<img src="https://user-images.githubusercontent.com/96192199/227504005-efeaefe0-b687-49d0-bf95-2d7b7e66c348.png" height="300" >
|
||||
</div>
|
||||
|
||||
### モジュール化された機能デザイン
|
||||
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/229288270-093643c1-0018-487a-81e6-1d7809b6e90f.png" height="400" >
|
||||
<img src="https://user-images.githubusercontent.com/96192199/227504931-19955f78-45cd-4d1c-adac-e71e50957915.png" height="400" >
|
||||
</div>
|
||||
|
||||
|
||||
### ソースコードの英語翻訳
|
||||
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/229720562-fe6c3508-6142-4635-a83d-21eb3669baee.png" height="400" >
|
||||
</div>
|
||||
|
||||
## Todo およびバージョン計画:
|
||||
- version 3.2+ (todo): 関数プラグインがより多くのパラメーターインターフェースをサポートするようになります。
|
||||
- version 3.1: 複数のgptモデルを同時にクエリし、api2dをサポートし、複数のapikeyの負荷分散をサポートします。
|
||||
- version 3.0: chatglmおよび他の小型llmのサポート
|
||||
- version 2.6: プラグイン構造を再構成し、相互作用性を高め、より多くのプラグインを追加しました。
|
||||
- version 2.5: 自己更新。総括的な大規模プロジェクトのソースコードをまとめた場合、テキストが長すぎる、トークンがオーバーフローする問題を解決します。
|
||||
- version 2.4: (1)PDF全文翻訳機能を追加。(2)入力エリアの位置を切り替える機能を追加。(3)垂直レイアウトオプションを追加。(4)マルチスレッド関数プラグインの最適化。
|
||||
- version 2.3: 多スレッドの相互作用性を向上させました。
|
||||
- version 2.2: 関数プラグインでホットリロードをサポート
|
||||
- version 2.1: 折りたたみ式レイアウト
|
||||
- version 2.0: モジュール化された関数プラグインを導入
|
||||
- version 1.0: 基本機能
|
||||
|
||||
## 参考および学習
|
||||
|
||||
|
||||
以下は中国語のマークダウンファイルです。日本語に翻訳してください。既存のマークダウンコマンドを変更しないでください:
|
||||
|
||||
```
|
||||
多くの優秀なプロジェクトの設計を参考にしています。主なものは以下の通りです:
|
||||
|
||||
# 参考プロジェクト1:ChuanhuChatGPTから多くのテクニックを借用
|
||||
https://github.com/GaiZhenbiao/ChuanhuChatGPT
|
||||
|
||||
# 参考プロジェクト2:清華ChatGLM-6B:
|
||||
https://github.com/THUDM/ChatGLM-6B
|
||||
```
|
||||
|
||||
291
docs/README_RS.md
Normal file
291
docs/README_RS.md
Normal file
@@ -0,0 +1,291 @@
|
||||
> **Note**
|
||||
>
|
||||
> Этот файл самовыражения автоматически генерируется модулем перевода markdown в этом проекте и может быть не на 100% правильным.
|
||||
>
|
||||
|
||||
# <img src="logo.png" width="40" > ChatGPT Academic Optimization
|
||||
|
||||
**Если вам понравился этот проект, пожалуйста, поставьте ему звезду. Если вы придумали более полезные академические ярлыки или функциональные плагины, не стесняйтесь создавать запросы на изменение или пул-запросы. Мы также имеем [README на английском языке](docs/README_EN.md), переведенный этим же проектом.
|
||||
|
||||
> **Примечание**
|
||||
>
|
||||
> 1. Пожалуйста, обратите внимание, что только функциonal plugins (buttons) с **красным цветом** могут читать файлы, некоторые из которых находятся в **выпадающем меню** плагинов. Кроме того, мы приветствуем и обрабатываем любые новые плагины с **наивысшим приоритетом**!
|
||||
>
|
||||
> 2. Функции каждого файла в этом проекте подробно описаны в собственном анализе [`self_analysis.md`](https://github.com/binary-husky/chatgpt_academic/wiki/chatgpt-academic%E9%A1%B9%E7%9B%AE%E8%87%AA%E8%AF%91%E8%A7%A3%E6%8A%A5%E5%91%8A) . При повторных итерациях вы также можете вызывать обновленный отчет функций проекта, щелкнув соответствующий функциональный плагин GPT. Часто задаваемые вопросы собраны в [`wiki`](https://github.com/binary-husky/chatgpt_academic/wiki/%E5%B8%B8%E8%A7%81%E9%97%AE%E9%A2%98) .
|
||||
|
||||
<div align="center">
|
||||
|
||||
Функция | Описание
|
||||
--- | ---
|
||||
Редактирование одним кликом | Поддержка редактирования одним кликом, поиск грамматических ошибок в академических статьях
|
||||
Переключение языков "Английский-Китайский" одним кликом | Одним кликом переключайте языки "Английский-Китайский"
|
||||
Разъяснение программного кода одним кликом | Вы можете правильно отобразить и объяснить программный код.
|
||||
[Настраиваемые сочетания клавиш](https://www.bilibili.com/video/BV14s4y1E7jN) | Поддержка настраиваемых сочетаний клавиш
|
||||
[Настройка сервера-прокси](https://www.bilibili.com/video/BV1rc411W7Dr) | Поддержка настройки сервера-прокси
|
||||
Модульный дизайн | Поддержка настраиваемых функциональных плагинов высших порядков и функциональных плагинов, поддерживающих [горячее обновление](https://github.com/binary-husky/chatgpt_academic/wiki/%E5%87%BD%E6%95%B0%E6%8F%92%E4%BB%B6%E6%8C%87%E5%8D%97)
|
||||
[Автоанализ программы](https://www.bilibili.com/video/BV1cj411A7VW) | [Функциональный плагин] [Прочтение в один клик](https://github.com/binary-husky/chatgpt_academic/wiki/chatgpt-academic%E9%A1%B9%E7%9B%AE%E8%87%AA%E8%AF%91%E8%A7%A3%E6%8A%A5%E5%91%8A) кода программы проекта
|
||||
[Анализ программы](https://www.bilibili.com/video/BV1cj411A7VW) | [Функциональный плагин] Один клик для проанализирования дерева других проектов Python/C/C++/Java/Lua/...
|
||||
Чтение статей| [Функциональный плагин] Одним кликом прочитайте весь латех (LaTex) текст статьи и сгенерируйте краткое описание
|
||||
Перевод и редактирование всех статей из LaTex | [Функциональный плагин] Перевод или редактирование LaTex-статьи всего одним нажатием кнопки
|
||||
Генерация комментариев в пакетном режиме | [Функциональный плагин] Одним кликом сгенерируйте комментарии к функциям в пакетном режиме
|
||||
Генерация отчетов пакета CHAT | [Функциональный плагин] Автоматически создавайте сводные отчеты после выполнения
|
||||
[Помощник по arxiv](https://www.bilibili.com/video/BV1LM4y1279X) | [Функциональный плагин] Введите URL статьи arxiv, чтобы легко перевести резюме и загрузить PDF-файл
|
||||
[Перевод полного текста статьи в формате PDF](https://www.bilibili.com/video/BV1KT411x7Wn) | [Функциональный плагин] Извлеките заголовок статьи, резюме и переведите весь текст статьи (многопоточно)
|
||||
[Помощник интеграции Google Scholar](https://www.bilibili.com/video/BV19L411U7ia) | [Функциональный плагин] Дайте GPT выбрать для вас интересные статьи на любой странице поиска Google Scholar.
|
||||
Отображение формул/изображений/таблиц | Одновременно отображается tex-форма и рендер-форма формул, поддержка формул, высокоскоростных кодов
|
||||
Поддержка функциональных плагинов многопоточности | Поддержка многопоточной работы с плагинами, обрабатывайте огромные объемы текста или программы одним кликом
|
||||
Запуск темной темы gradio[подробнее](https://github.com/binary-husky/chatgpt_academic/issues/173) | Добавьте / ?__dark-theme=true в конец URL браузера, чтобы переключиться на темную тему.
|
||||
[Поддержка нескольких моделей LLM](https://www.bilibili.com/video/BV1wT411p7yf), поддержка API2D | Находиться между GPT3.5, GPT4 и [清华ChatGLM](https://github.com/THUDM/ChatGLM-6B) должно быть очень приятно, не так ли?
|
||||
Альтернатива huggingface без использования научной сети [Онлайн-эксперимент](https://huggingface.co/spaces/qingxu98/gpt-academic) | Войдите в систему, скопируйте пространство [этот пространственный URL](https://huggingface.co/spaces/qingxu98/gpt-academic)
|
||||
…… | ……
|
||||
|
||||
|
||||
</div>
|
||||
|
||||
- Новый интерфейс (вы можете изменить настройку LAYOUT в config.py, чтобы переключаться между "горизонтальным расположением" и "вертикальным расположением")
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/230361456-61078362-a966-4eb5-b49e-3c62ef18b860.gif" width="700" >
|
||||
</div>
|
||||
|
||||
|
||||
Вы профессиональный переводчик научных статей.
|
||||
|
||||
- Все кнопки генерируются динамически путем чтения functional.py и могут быть легко настроены под пользовательские потребности, освобождая буфер обмена.
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/231975334-b4788e91-4887-412f-8b43-2b9c5f41d248.gif" width="700" >
|
||||
</div>
|
||||
|
||||
- Редактирование/корректирование
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/231980294-f374bdcb-3309-4560-b424-38ef39f04ebd.gif" width="700" >
|
||||
</div>
|
||||
|
||||
- Если вывод содержит формулы, они отображаются одновременно как в формате tex, так и в рендеринговом формате для удобства копирования и чтения.
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/230598842-1d7fcddd-815d-40ee-af60-baf488a199df.png" width="700" >
|
||||
</div>
|
||||
|
||||
- Лень смотреть код проекта? Просто покажите chatgpt.
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/226935232-6b6a73ce-8900-4aee-93f9-733c7e6fef53.png" width="700" >
|
||||
</div>
|
||||
|
||||
- Несколько моделей больших языковых моделей смешиваются (ChatGLM + OpenAI-GPT3.5 + [API2D] (https://api2d.com/) -GPT4)
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/232537274-deca0563-7aa6-4b5d-94a2-b7c453c47794.png" width="700" >
|
||||
</div>
|
||||
|
||||
Несколько моделей больших языковых моделей смешиваются в [бета-версии huggingface] (https://huggingface.co/spaces/qingxu98/academic-chatgpt-beta) (huggingface-версия не поддерживает chatglm).
|
||||
|
||||
|
||||
---
|
||||
|
||||
## Установка - Метод 1: Запуск (Windows, Linux или MacOS)
|
||||
|
||||
1. Скачайте проект
|
||||
```sh
|
||||
git clone https://github.com/binary-husky/chatgpt_academic.git
|
||||
cd chatgpt_academic
|
||||
```
|
||||
|
||||
2. Настройка API_KEY и настройки прокси
|
||||
|
||||
В файле `config.py` настройте зарубежный прокси и OpenAI API KEY, пояснения ниже
|
||||
```
|
||||
1. Если вы находитесь в Китае, вам нужно настроить зарубежный прокси, чтобы использовать OpenAI API. Пожалуйста, внимательно прочитайте config.py для получения инструкций (1. Измените USE_PROXY на True; 2. Измените прокси в соответствии с инструкциями).
|
||||
2. Настройка API KEY OpenAI. Вам необходимо зарегистрироваться на сайте OpenAI и получить API KEY. После получения API KEY настройте его в файле config.py.
|
||||
3. Вопросы, связанные с сетевыми проблемами (тайм-аут сети, прокси не работает), можно найти здесь: https://github.com/binary-husky/chatgpt_academic/issues/1
|
||||
```
|
||||
(Примечание: при запуске программы будет проверяться наличие конфиденциального файла конфигурации с именем `config_private.py` и использоваться в нем конфигурация параметров, которая перезаписывает параметры с такими же именами в `config.py`. Поэтому, если вы понимаете логику чтения нашей конфигурации, мы настоятельно рекомендуем вам создать новый файл конфигурации с именем `config_private.py` рядом с `config.py` и переместить (скопировать) настройки из `config.py` в `config_private.py`. `config_private.py` не подвергается контролю git, что делает конфиденциальную информацию более безопасной.)
|
||||
|
||||
|
||||
3. Установить зависимости
|
||||
```sh
|
||||
# (Выбор 1) Рекомендуется
|
||||
python -m pip install -r requirements.txt
|
||||
|
||||
# (Выбор 2) Если вы используете anaconda, то шаги будут аналогичны:
|
||||
# (Шаг 2.1) conda create -n gptac_venv python=3.11
|
||||
# (Шаг 2.2) conda activate gptac_venv
|
||||
# (Шаг 2.3) python -m pip install -r requirements.txt
|
||||
|
||||
# Примечание: используйте официальный источник pip или источник pip.aliyun.com. Другие источники pip могут вызывать проблемы. временный метод замены источника:
|
||||
# python -m pip install -r requirements.txt -i https://mirrors.aliyun.com/pypi/simple/
|
||||
```
|
||||
|
||||
Если требуется поддержка TUNA ChatGLM, необходимо установить дополнительные зависимости (если вы неудобны с python, необходимо иметь хорошую конфигурацию компьютера):
|
||||
```sh
|
||||
python -m pip install -r request_llm/requirements_chatglm.txt
|
||||
```
|
||||
|
||||
4. Запустите
|
||||
```sh
|
||||
python main.py
|
||||
```
|
||||
|
||||
5. Тестовые функции плагина
|
||||
```
|
||||
- Тестирвоание анализа проекта Python
|
||||
В основной области введите `./crazy_functions/test_project/python/dqn` , а затем нажмите "Анализировать весь проект Python"
|
||||
- Тестирование самостоятельного чтения кода
|
||||
Щелкните " [Демонстрационный режим многопоточности] Проанализируйте сам проект (расшифровка источника кода)"
|
||||
- Тестирование функций шаблонного плагина (вы можете использовать эту функцию как шаблон для более сложных функций, требующих ответа от gpt в связи с тем, что произошло сегодня в истории)
|
||||
Щелкните " [Функции шаблонного плагина] День в истории"
|
||||
- На нижней панели дополнительные функции для выбора
|
||||
```
|
||||
|
||||
## Установка - Метод 2: Использование docker (Linux)
|
||||
|
||||
|
||||
1. Только ChatGPT (рекомендуется для большинства пользователей):
|
||||
``` sh
|
||||
# Скачать проект
|
||||
git clone https://github.com/binary-husky/chatgpt_academic.git
|
||||
cd chatgpt_academic
|
||||
# Настроить прокси за границей и OpenAI API KEY
|
||||
Отредактируйте файл config.py в любом текстовом редакторе.
|
||||
# Установка
|
||||
docker build -t gpt-academic .
|
||||
# Запустить
|
||||
docker run --rm -it --net=host gpt-academic
|
||||
|
||||
# Проверка функциональности плагина
|
||||
## Проверка шаблонной функции плагина (требуется, чтобы gpt ответил, что произошло "в истории на этот день"), вы можете использовать эту функцию в качестве шаблона для реализации более сложных функций.
|
||||
Нажмите "[Шаблонный демонстрационный плагин] История на этот день".
|
||||
## Тест абстрактного резюме для проекта на Latex
|
||||
В области ввода введите ./crazy_functions/test_project/latex/attention, а затем нажмите "Чтение реферата о тезисах статьи на LaTeX".
|
||||
## Тестовый анализ проекта на Python
|
||||
Введите в область ввода ./crazy_functions/test_project/python/dqn, затем нажмите "Проанализировать весь проект на Python".
|
||||
|
||||
Выбирайте больше функциональных плагинов в нижнем выпадающем меню.
|
||||
```
|
||||
|
||||
2. ChatGPT + ChatGLM (требуется глубокое знание Docker и достаточно мощное компьютерное оборудование):
|
||||
|
||||
``` sh
|
||||
# Изменение Dockerfile
|
||||
cd docs && nano Dockerfile+ChatGLM
|
||||
# Как построить | Как запустить (Dockerfile+ChatGLM в пути docs, сначала перейдите в папку с помощью cd docs)
|
||||
docker build -t gpt-academic --network=host -f Dockerfile+ChatGLM .
|
||||
# Как запустить | Как запустить (2) я хочу войти в контейнер и сделать какие-то настройки до запуска:
|
||||
docker run --rm -it --net=host --gpus=all gpt-academic bash
|
||||
```
|
||||
|
||||
|
||||
## Установка-Метод 3: Другие способы развертывания
|
||||
|
||||
1. Развертывание на удаленном облачном сервере
|
||||
Пожалуйста, посетите [Deploy Wiki-1] (https://github.com/binary-husky/chatgpt_academic/wiki/%E4%BA%91%E6%9C%8D%E5%8A%A1%E5%99%A8%E8%BF%9C%E7%A8%8B%E9%83%A8%E7%BD%B2%E6%8C%87%E5%8D%97)
|
||||
|
||||
2. Использование WSL2 (Windows Subsystem for Linux)
|
||||
Пожалуйста, посетите [Deploy Wiki-2] (https://github.com/binary-husky/chatgpt_academic/wiki/%E4%BD%BF%E7%94%A8WSL2%EF%BC%88Windows-Subsystem-for-Linux-%E5%AD%90%E7%B3%BB%E7%BB%9F%EF%BC%89%E9%83%A8%E7%BD%B2)
|
||||
|
||||
|
||||
## Установка-Настройки прокси
|
||||
### Метод 1: Обычный способ
|
||||
[Конфигурация прокси] (https://github.com/binary-husky/chatgpt_academic/issues/1)
|
||||
|
||||
### Метод 2: Руководство новичка
|
||||
[Руководство новичка] (https://github.com/binary-husky/chatgpt_academic/wiki/%E4%BB%A3%E7%90%86%E8%BD%AF%E4%BB%B6%E9%97%AE%E9%A2%98%E7%9A%84%E6%96%B0%E6%89%8B%E8%A7%A3%E5%86%B3%E6%96%B9%E6%B3%95%EF%BC%88%E6%96%B9%E6%B3%95%E5%8F%AA%E9%80%82%E7%94%A8%E4%BA%8E%E6%96%B0%E6%89%8B%EF%BC%89)
|
||||
|
||||
|
||||
---
|
||||
|
||||
## Настройка новой удобной кнопки (настройка быстрой клавиши для научной работы)
|
||||
Откройте `core_functional.py` любым текстовым редактором, добавьте элементы, как показано ниже, затем перезапустите программу. (Если кнопка уже успешно добавлена и видна, то префикс и суффикс поддерживают горячее изменение, чтобы они оказались в действии, не нужно перезапускать программу.)
|
||||
например
|
||||
```
|
||||
"Супер анг-рус": {
|
||||
# Префикс, будет добавлен перед вашим вводом. Например, используется для описания ваших потребностей, таких как перевод, кодинг, редактирование и т. д.
|
||||
"Prefix": "Пожалуйста, переведите этот фрагмент на русский язык, а затем создайте пошаговую таблицу в markdown, чтобы объяснить все специализированные термины, которые встречаются в тексте:\n\n",
|
||||
|
||||
# Суффикс, будет добавлен после вашего ввода. Например, совместно с префиксом можно обрамить ваш ввод в кавычки.
|
||||
"Suffix": "",
|
||||
},
|
||||
```
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/226899272-477c2134-ed71-4326-810c-29891fe4a508.png" width="500" >
|
||||
</div>
|
||||
|
||||
---
|
||||
|
||||
|
||||
## Демонстрация некоторых возможностей
|
||||
|
||||
### Отображение изображений:
|
||||
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/228737599-bf0a9d9c-1808-4f43-ae15-dfcc7af0f295.png" width="800" >
|
||||
</div>
|
||||
|
||||
|
||||
### Если программа может понимать и разбирать сама себя:
|
||||
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/226936850-c77d7183-0749-4c1c-9875-fd4891842d0c.png" width="800" >
|
||||
</div>
|
||||
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/226936618-9b487e4b-ab5b-4b6e-84c6-16942102e917.png" width="800" >
|
||||
</div>
|
||||
|
||||
|
||||
### Анализ других проектов на Python/Cpp:
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/226935232-6b6a73ce-8900-4aee-93f9-733c7e6fef53.png" width="800" >
|
||||
</div>
|
||||
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/226969067-968a27c1-1b9c-486b-8b81-ab2de8d3f88a.png" width="800" >
|
||||
</div>
|
||||
|
||||
### Генерация понимания и абстрактов с помощью Латех статей в один клик
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/227504406-86ab97cd-f208-41c3-8e4a-7000e51cf980.png" width="800" >
|
||||
</div>
|
||||
|
||||
### Автоматическое создание отчетов
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/227503770-fe29ce2c-53fd-47b0-b0ff-93805f0c2ff4.png" height="300" >
|
||||
<img src="https://user-images.githubusercontent.com/96192199/227504617-7a497bb3-0a2a-4b50-9a8a-95ae60ea7afd.png" height="300" >
|
||||
<img src="https://user-images.githubusercontent.com/96192199/227504005-efeaefe0-b687-49d0-bf95-2d7b7e66c348.png" height="300" >
|
||||
</div>
|
||||
|
||||
### Модульный дизайн функций
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/229288270-093643c1-0018-487a-81e6-1d7809b6e90f.png" height="400" >
|
||||
<img src="https://user-images.githubusercontent.com/96192199/227504931-19955f78-45cd-4d1c-adac-e71e50957915.png" height="400" >
|
||||
</div>
|
||||
|
||||
|
||||
### Трансляция исходного кода на английский язык
|
||||
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/96192199/229720562-fe6c3508-6142-4635-a83d-21eb3669baee.png" height="400" >
|
||||
</div>
|
||||
|
||||
## Todo и планирование версий:
|
||||
- version 3.2+ (todo): функция плагины поддерживают более многочисленные интерфейсы параметров
|
||||
- version 3.1: поддержка одновременного опроса нескольких моделей gpt! Поддержка api2d, поддержка балансировки нагрузки множества apikey.
|
||||
- version 3.0: поддержка chatglm и других маленьких llm
|
||||
- version 2.6: реструктурировал структуру плагинов, повысил интерактивность, добавил больше плагинов
|
||||
- version 2.5: само обновление, решение проблемы слишком длинного текста и переполнения токена при переводе всего проекта исходного кода
|
||||
- version 2.4: (1) добавлена функция перевода всего PDF-документа; (2) добавлена функция изменения положения входной области; (3) добавлена опция вертикального макета; (4) оптимизация функций многопоточности плагина.
|
||||
- version 2.3: улучшение многопоточной интерактивности
|
||||
- version 2.2: функция плагинов поддерживает горячую перезагрузку
|
||||
- version 2.1: блочная раскладка
|
||||
- version 2.0: модульный дизайн функций плагина
|
||||
- version 1.0: основные функции
|
||||
|
||||
## Ссылки на изучение и обучение
|
||||
|
||||
```
|
||||
В коде использовано много хороших дизайнерских решений из других отличных проектов, в том числе:
|
||||
|
||||
# Project1: использование многих приемов из ChuanhuChatGPT
|
||||
https://github.com/GaiZhenbiao/ChuanhuChatGPT
|
||||
|
||||
# Project2: ChatGLM-6B в Тхуде:
|
||||
https://github.com/THUDM/ChatGLM-6B
|
||||
```
|
||||
|
||||
43
docs/WithFastapi.md
Normal file
43
docs/WithFastapi.md
Normal file
@@ -0,0 +1,43 @@
|
||||
# Running with fastapi
|
||||
|
||||
We currently support fastapi in order to solve sub-path deploy issue.
|
||||
|
||||
1. change CUSTOM_PATH setting in `config.py`
|
||||
|
||||
``` sh
|
||||
nano config.py
|
||||
```
|
||||
|
||||
2. Edit main.py
|
||||
|
||||
```diff
|
||||
auto_opentab_delay()
|
||||
- demo.queue(concurrency_count=CONCURRENT_COUNT).launch(server_name="0.0.0.0", server_port=PORT, auth=AUTHENTICATION, favicon_path="docs/logo.png")
|
||||
+ demo.queue(concurrency_count=CONCURRENT_COUNT)
|
||||
|
||||
- # 如果需要在二级路径下运行
|
||||
- # CUSTOM_PATH, = get_conf('CUSTOM_PATH')
|
||||
- # if CUSTOM_PATH != "/":
|
||||
- # from toolbox import run_gradio_in_subpath
|
||||
- # run_gradio_in_subpath(demo, auth=AUTHENTICATION, port=PORT, custom_path=CUSTOM_PATH)
|
||||
- # else:
|
||||
- # demo.launch(server_name="0.0.0.0", server_port=PORT, auth=AUTHENTICATION, favicon_path="docs/logo.png")
|
||||
|
||||
+ 如果需要在二级路径下运行
|
||||
+ CUSTOM_PATH, = get_conf('CUSTOM_PATH')
|
||||
+ if CUSTOM_PATH != "/":
|
||||
+ from toolbox import run_gradio_in_subpath
|
||||
+ run_gradio_in_subpath(demo, auth=AUTHENTICATION, port=PORT, custom_path=CUSTOM_PATH)
|
||||
+ else:
|
||||
+ demo.launch(server_name="0.0.0.0", server_port=PORT, auth=AUTHENTICATION, favicon_path="docs/logo.png")
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
```
|
||||
|
||||
|
||||
3. Go!
|
||||
|
||||
``` sh
|
||||
python main.py
|
||||
```
|
||||
49
main.py
49
main.py
@@ -45,7 +45,7 @@ def main():
|
||||
|
||||
gr_L1 = lambda: gr.Row().style()
|
||||
gr_L2 = lambda scale: gr.Column(scale=scale)
|
||||
if LAYOUT == "TOP-DOWN":
|
||||
if LAYOUT == "TOP-DOWN":
|
||||
gr_L1 = lambda: DummyWith()
|
||||
gr_L2 = lambda scale: gr.Row()
|
||||
CHATBOT_HEIGHT /= 2
|
||||
@@ -56,7 +56,7 @@ def main():
|
||||
cookies = gr.State({'api_key': API_KEY, 'llm_model': LLM_MODEL})
|
||||
with gr_L1():
|
||||
with gr_L2(scale=2):
|
||||
chatbot = gr.Chatbot()
|
||||
chatbot = gr.Chatbot(label=f"当前模型:{LLM_MODEL}")
|
||||
chatbot.style(height=CHATBOT_HEIGHT)
|
||||
history = gr.State([])
|
||||
with gr_L2(scale=1):
|
||||
@@ -88,9 +88,12 @@ def main():
|
||||
with gr.Row():
|
||||
with gr.Accordion("更多函数插件", open=True):
|
||||
dropdown_fn_list = [k for k in crazy_fns.keys() if not crazy_fns[k].get("AsButton", True)]
|
||||
with gr.Column(scale=1):
|
||||
with gr.Row():
|
||||
dropdown = gr.Dropdown(dropdown_fn_list, value=r"打开插件列表", label="").style(container=False)
|
||||
with gr.Column(scale=1):
|
||||
with gr.Row():
|
||||
plugin_advanced_arg = gr.Textbox(show_label=True, label="高级参数输入区", visible=False,
|
||||
placeholder="这里是特殊函数插件的高级参数输入区").style(container=False)
|
||||
with gr.Row():
|
||||
switchy_bt = gr.Button(r"请先从插件列表中选择", variant="secondary")
|
||||
with gr.Row():
|
||||
with gr.Accordion("点击展开“文件上传区”。上传本地文件可供红色函数插件调用。", open=False) as area_file_up:
|
||||
@@ -100,7 +103,7 @@ def main():
|
||||
top_p = gr.Slider(minimum=-0, maximum=1.0, value=1.0, step=0.01,interactive=True, label="Top-p (nucleus sampling)",)
|
||||
temperature = gr.Slider(minimum=-0, maximum=2.0, value=1.0, step=0.01, interactive=True, label="Temperature",)
|
||||
max_length_sl = gr.Slider(minimum=256, maximum=4096, value=512, step=1, interactive=True, label="Local LLM MaxLength",)
|
||||
checkboxes = gr.CheckboxGroup(["基础功能区", "函数插件区", "底部输入区", "输入清除键"], value=["基础功能区", "函数插件区"], label="显示/隐藏功能区")
|
||||
checkboxes = gr.CheckboxGroup(["基础功能区", "函数插件区", "底部输入区", "输入清除键", "插件参数区"], value=["基础功能区", "函数插件区"], label="显示/隐藏功能区")
|
||||
md_dropdown = gr.Dropdown(AVAIL_LLM_MODELS, value=LLM_MODEL, label="更换LLM模型/请求源").style(container=False)
|
||||
|
||||
gr.Markdown(description)
|
||||
@@ -112,7 +115,7 @@ def main():
|
||||
with gr.Row():
|
||||
resetBtn2 = gr.Button("重置", variant="secondary"); resetBtn2.style(size="sm")
|
||||
stopBtn2 = gr.Button("停止", variant="secondary"); stopBtn2.style(size="sm")
|
||||
clearBtn2 = gr.Button("清除", variant="secondary", visible=False); clearBtn.style(size="sm")
|
||||
clearBtn2 = gr.Button("清除", variant="secondary", visible=False); clearBtn2.style(size="sm")
|
||||
# 功能区显示开关与功能区的互动
|
||||
def fn_area_visibility(a):
|
||||
ret = {}
|
||||
@@ -122,11 +125,12 @@ def main():
|
||||
ret.update({area_input_secondary: gr.update(visible=("底部输入区" in a))})
|
||||
ret.update({clearBtn: gr.update(visible=("输入清除键" in a))})
|
||||
ret.update({clearBtn2: gr.update(visible=("输入清除键" in a))})
|
||||
ret.update({plugin_advanced_arg: gr.update(visible=("插件参数区" in a))})
|
||||
if "底部输入区" in a: ret.update({txt: gr.update(value="")})
|
||||
return ret
|
||||
checkboxes.select(fn_area_visibility, [checkboxes], [area_basic_fn, area_crazy_fn, area_input_primary, area_input_secondary, txt, txt2, clearBtn, clearBtn2] )
|
||||
checkboxes.select(fn_area_visibility, [checkboxes], [area_basic_fn, area_crazy_fn, area_input_primary, area_input_secondary, txt, txt2, clearBtn, clearBtn2, plugin_advanced_arg] )
|
||||
# 整理反复出现的控件句柄组合
|
||||
input_combo = [cookies, max_length_sl, md_dropdown, txt, txt2, top_p, temperature, chatbot, history, system_prompt]
|
||||
input_combo = [cookies, max_length_sl, md_dropdown, txt, txt2, top_p, temperature, chatbot, history, system_prompt, plugin_advanced_arg]
|
||||
output_combo = [cookies, chatbot, history, status]
|
||||
predict_args = dict(fn=ArgsGeneralWrapper(predict), inputs=input_combo, outputs=output_combo)
|
||||
# 提交按钮、重置按钮
|
||||
@@ -153,17 +157,22 @@ def main():
|
||||
# 函数插件-下拉菜单与随变按钮的互动
|
||||
def on_dropdown_changed(k):
|
||||
variant = crazy_fns[k]["Color"] if "Color" in crazy_fns[k] else "secondary"
|
||||
return {switchy_bt: gr.update(value=k, variant=variant)}
|
||||
dropdown.select(on_dropdown_changed, [dropdown], [switchy_bt] )
|
||||
ret = {switchy_bt: gr.update(value=k, variant=variant)}
|
||||
if crazy_fns[k].get("AdvancedArgs", False): # 是否唤起高级插件参数区
|
||||
ret.update({plugin_advanced_arg: gr.update(visible=True, label=f"插件[{k}]的高级参数说明:" + crazy_fns[k].get("ArgsReminder", [f"没有提供高级参数功能说明"]))})
|
||||
else:
|
||||
ret.update({plugin_advanced_arg: gr.update(visible=False, label=f"插件[{k}]不需要高级参数。")})
|
||||
return ret
|
||||
dropdown.select(on_dropdown_changed, [dropdown], [switchy_bt, plugin_advanced_arg] )
|
||||
def on_md_dropdown_changed(k):
|
||||
return {chatbot: gr.update(label="当前模型:"+k)}
|
||||
md_dropdown.select(on_md_dropdown_changed, [md_dropdown], [chatbot] )
|
||||
# 随变按钮的回调函数注册
|
||||
def route(k, *args, **kwargs):
|
||||
if k in [r"打开插件列表", r"请先从插件列表中选择"]: return
|
||||
if k in [r"打开插件列表", r"请先从插件列表中选择"]: return
|
||||
yield from ArgsGeneralWrapper(crazy_fns[k]["Function"])(*args, **kwargs)
|
||||
click_handle = switchy_bt.click(route,[switchy_bt, *input_combo, gr.State(PORT)], output_combo)
|
||||
click_handle.then(on_report_generated, [file_upload, chatbot], [file_upload, chatbot])
|
||||
# def expand_file_area(file_upload, area_file_up):
|
||||
# if len(file_upload)>0: return {area_file_up: gr.update(open=True)}
|
||||
# click_handle.then(expand_file_area, [file_upload, area_file_up], [area_file_up])
|
||||
cancel_handles.append(click_handle)
|
||||
# 终止按钮的回调函数注册
|
||||
stopBtn.click(fn=None, inputs=None, outputs=None, cancels=cancel_handles)
|
||||
@@ -175,7 +184,7 @@ def main():
|
||||
print(f"如果浏览器没有自动打开,请复制并转到以下URL:")
|
||||
print(f"\t(亮色主题): http://localhost:{PORT}")
|
||||
print(f"\t(暗色主题): http://localhost:{PORT}/?__dark-theme=true")
|
||||
def open():
|
||||
def open():
|
||||
time.sleep(2) # 打开浏览器
|
||||
webbrowser.open_new_tab(f"http://localhost:{PORT}/?__dark-theme=true")
|
||||
threading.Thread(target=open, name="open-browser", daemon=True).start()
|
||||
@@ -185,5 +194,13 @@ def main():
|
||||
auto_opentab_delay()
|
||||
demo.queue(concurrency_count=CONCURRENT_COUNT).launch(server_name="0.0.0.0", server_port=PORT, auth=AUTHENTICATION, favicon_path="docs/logo.png")
|
||||
|
||||
# 如果需要在二级路径下运行
|
||||
# CUSTOM_PATH, = get_conf('CUSTOM_PATH')
|
||||
# if CUSTOM_PATH != "/":
|
||||
# from toolbox import run_gradio_in_subpath
|
||||
# run_gradio_in_subpath(demo, auth=AUTHENTICATION, port=PORT, custom_path=CUSTOM_PATH)
|
||||
# else:
|
||||
# demo.launch(server_name="0.0.0.0", server_port=PORT, auth=AUTHENTICATION, favicon_path="docs/logo.png")
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
main()
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# 如何使用其他大语言模型(v3.0分支测试中)
|
||||
# 如何使用其他大语言模型
|
||||
|
||||
## ChatGLM
|
||||
|
||||
@@ -15,7 +15,7 @@ LLM_MODEL = "chatglm"
|
||||
|
||||
|
||||
---
|
||||
## Text-Generation-UI (TGUI)
|
||||
## Text-Generation-UI (TGUI,调试中,暂不可用)
|
||||
|
||||
### 1. 部署TGUI
|
||||
``` sh
|
||||
|
||||
@@ -1,16 +1,17 @@
|
||||
|
||||
"""
|
||||
该文件中主要包含2个函数
|
||||
该文件中主要包含2个函数,是所有LLM的通用接口,它们会继续向下调用更底层的LLM模型,处理多模型并行等细节
|
||||
|
||||
不具备多线程能力的函数:
|
||||
1. predict: 正常对话时使用,具备完备的交互功能,不可多线程
|
||||
不具备多线程能力的函数:正常对话时使用,具备完备的交互功能,不可多线程
|
||||
1. predict(...)
|
||||
|
||||
具备多线程调用能力的函数
|
||||
2. predict_no_ui_long_connection:在实验过程中发现调用predict_no_ui处理长文档时,和openai的连接容易断掉,这个函数用stream的方式解决这个问题,同样支持多线程
|
||||
具备多线程调用能力的函数:在函数插件中被调用,灵活而简洁
|
||||
2. predict_no_ui_long_connection(...)
|
||||
"""
|
||||
import tiktoken
|
||||
from functools import wraps, lru_cache
|
||||
from functools import lru_cache
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
from toolbox import get_conf, trimmed_format_exc
|
||||
|
||||
from .bridge_chatgpt import predict_no_ui_long_connection as chatgpt_noui
|
||||
from .bridge_chatgpt import predict as chatgpt_ui
|
||||
@@ -18,6 +19,9 @@ from .bridge_chatgpt import predict as chatgpt_ui
|
||||
from .bridge_chatglm import predict_no_ui_long_connection as chatglm_noui
|
||||
from .bridge_chatglm import predict as chatglm_ui
|
||||
|
||||
from .bridge_newbing import predict_no_ui_long_connection as newbing_noui
|
||||
from .bridge_newbing import predict as newbing_ui
|
||||
|
||||
# from .bridge_tgui import predict_no_ui_long_connection as tgui_noui
|
||||
# from .bridge_tgui import predict as tgui_ui
|
||||
|
||||
@@ -42,18 +46,39 @@ class LazyloadTiktoken(object):
|
||||
def decode(self, *args, **kwargs):
|
||||
encoder = self.get_encoder(self.model)
|
||||
return encoder.decode(*args, **kwargs)
|
||||
|
||||
|
||||
# Endpoint 重定向
|
||||
API_URL_REDIRECT, = get_conf("API_URL_REDIRECT")
|
||||
openai_endpoint = "https://api.openai.com/v1/chat/completions"
|
||||
api2d_endpoint = "https://openai.api2d.net/v1/chat/completions"
|
||||
newbing_endpoint = "wss://sydney.bing.com/sydney/ChatHub"
|
||||
# 兼容旧版的配置
|
||||
try:
|
||||
API_URL, = get_conf("API_URL")
|
||||
if API_URL != "https://api.openai.com/v1/chat/completions":
|
||||
openai_endpoint = API_URL
|
||||
print("警告!API_URL配置选项将被弃用,请更换为API_URL_REDIRECT配置")
|
||||
except:
|
||||
pass
|
||||
# 新版配置
|
||||
if openai_endpoint in API_URL_REDIRECT: openai_endpoint = API_URL_REDIRECT[openai_endpoint]
|
||||
if api2d_endpoint in API_URL_REDIRECT: api2d_endpoint = API_URL_REDIRECT[api2d_endpoint]
|
||||
if newbing_endpoint in API_URL_REDIRECT: newbing_endpoint = API_URL_REDIRECT[newbing_endpoint]
|
||||
|
||||
|
||||
# 获取tokenizer
|
||||
tokenizer_gpt35 = LazyloadTiktoken("gpt-3.5-turbo")
|
||||
tokenizer_gpt4 = LazyloadTiktoken("gpt-4")
|
||||
get_token_num_gpt35 = lambda txt: len(tokenizer_gpt35.encode(txt, disallowed_special=()))
|
||||
get_token_num_gpt4 = lambda txt: len(tokenizer_gpt4.encode(txt, disallowed_special=()))
|
||||
|
||||
|
||||
model_info = {
|
||||
# openai
|
||||
"gpt-3.5-turbo": {
|
||||
"fn_with_ui": chatgpt_ui,
|
||||
"fn_without_ui": chatgpt_noui,
|
||||
"endpoint": "https://api.openai.com/v1/chat/completions",
|
||||
"endpoint": openai_endpoint,
|
||||
"max_token": 4096,
|
||||
"tokenizer": tokenizer_gpt35,
|
||||
"token_cnt": get_token_num_gpt35,
|
||||
@@ -62,7 +87,7 @@ model_info = {
|
||||
"gpt-4": {
|
||||
"fn_with_ui": chatgpt_ui,
|
||||
"fn_without_ui": chatgpt_noui,
|
||||
"endpoint": "https://api.openai.com/v1/chat/completions",
|
||||
"endpoint": openai_endpoint,
|
||||
"max_token": 8192,
|
||||
"tokenizer": tokenizer_gpt4,
|
||||
"token_cnt": get_token_num_gpt4,
|
||||
@@ -72,7 +97,7 @@ model_info = {
|
||||
"api2d-gpt-3.5-turbo": {
|
||||
"fn_with_ui": chatgpt_ui,
|
||||
"fn_without_ui": chatgpt_noui,
|
||||
"endpoint": "https://openai.api2d.net/v1/chat/completions",
|
||||
"endpoint": api2d_endpoint,
|
||||
"max_token": 4096,
|
||||
"tokenizer": tokenizer_gpt35,
|
||||
"token_cnt": get_token_num_gpt35,
|
||||
@@ -81,7 +106,7 @@ model_info = {
|
||||
"api2d-gpt-4": {
|
||||
"fn_with_ui": chatgpt_ui,
|
||||
"fn_without_ui": chatgpt_noui,
|
||||
"endpoint": "https://openai.api2d.net/v1/chat/completions",
|
||||
"endpoint": api2d_endpoint,
|
||||
"max_token": 8192,
|
||||
"tokenizer": tokenizer_gpt4,
|
||||
"token_cnt": get_token_num_gpt4,
|
||||
@@ -96,7 +121,15 @@ model_info = {
|
||||
"tokenizer": tokenizer_gpt35,
|
||||
"token_cnt": get_token_num_gpt35,
|
||||
},
|
||||
|
||||
# newbing
|
||||
"newbing": {
|
||||
"fn_with_ui": newbing_ui,
|
||||
"fn_without_ui": newbing_noui,
|
||||
"endpoint": newbing_endpoint,
|
||||
"max_token": 4096,
|
||||
"tokenizer": tokenizer_gpt35,
|
||||
"token_cnt": get_token_num_gpt35,
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
@@ -108,10 +141,7 @@ def LLM_CATCH_EXCEPTION(f):
|
||||
try:
|
||||
return f(inputs, llm_kwargs, history, sys_prompt, observe_window, console_slience)
|
||||
except Exception as e:
|
||||
from toolbox import get_conf
|
||||
import traceback
|
||||
proxies, = get_conf('proxies')
|
||||
tb_str = '\n```\n' + traceback.format_exc() + '\n```\n'
|
||||
tb_str = '\n```\n' + trimmed_format_exc() + '\n```\n'
|
||||
observe_window[0] = tb_str
|
||||
return tb_str
|
||||
return decorated
|
||||
@@ -162,7 +192,7 @@ def predict_no_ui_long_connection(inputs, llm_kwargs, history, sys_prompt, obser
|
||||
|
||||
def mutex_manager(window_mutex, observe_window):
|
||||
while True:
|
||||
time.sleep(0.5)
|
||||
time.sleep(0.25)
|
||||
if not window_mutex[-1]: break
|
||||
# 看门狗(watchdog)
|
||||
for i in range(n_model):
|
||||
@@ -190,7 +220,7 @@ def predict_no_ui_long_connection(inputs, llm_kwargs, history, sys_prompt, obser
|
||||
return_string_collect.append( f"【{str(models[i])} 说】: <font color=\"{colors[i]}\"> {future.result()} </font>" )
|
||||
|
||||
window_mutex[-1] = False # stop mutex thread
|
||||
res = '<br/>\n\n---\n\n'.join(return_string_collect)
|
||||
res = '<br/><br/>\n\n---\n\n'.join(return_string_collect)
|
||||
return res
|
||||
|
||||
|
||||
|
||||
@@ -32,6 +32,7 @@ class GetGLMHandle(Process):
|
||||
return self.chatglm_model is not None
|
||||
|
||||
def run(self):
|
||||
# 子进程执行
|
||||
# 第一次运行,加载参数
|
||||
retry = 0
|
||||
while True:
|
||||
@@ -53,17 +54,24 @@ class GetGLMHandle(Process):
|
||||
self.child.send('[Local Message] Call ChatGLM fail 不能正常加载ChatGLM的参数。')
|
||||
raise RuntimeError("不能正常加载ChatGLM的参数!")
|
||||
|
||||
# 进入任务等待状态
|
||||
while True:
|
||||
# 进入任务等待状态
|
||||
kwargs = self.child.recv()
|
||||
# 收到消息,开始请求
|
||||
try:
|
||||
for response, history in self.chatglm_model.stream_chat(self.chatglm_tokenizer, **kwargs):
|
||||
self.child.send(response)
|
||||
# # 中途接收可能的终止指令(如果有的话)
|
||||
# if self.child.poll():
|
||||
# command = self.child.recv()
|
||||
# if command == '[Terminate]': break
|
||||
except:
|
||||
self.child.send('[Local Message] Call ChatGLM fail.')
|
||||
# 请求处理结束,开始下一个循环
|
||||
self.child.send('[Finish]')
|
||||
|
||||
def stream_chat(self, **kwargs):
|
||||
# 主进程执行
|
||||
self.parent.send(kwargs)
|
||||
while True:
|
||||
res = self.parent.recv()
|
||||
@@ -92,8 +100,8 @@ def predict_no_ui_long_connection(inputs, llm_kwargs, history=[], sys_prompt="",
|
||||
|
||||
# chatglm 没有 sys_prompt 接口,因此把prompt加入 history
|
||||
history_feedin = []
|
||||
history_feedin.append(["What can I do?", sys_prompt])
|
||||
for i in range(len(history)//2):
|
||||
history_feedin.append(["What can I do?", sys_prompt] )
|
||||
history_feedin.append([history[2*i], history[2*i+1]] )
|
||||
|
||||
watch_dog_patience = 5 # 看门狗 (watchdog) 的耐心, 设置5秒即可
|
||||
@@ -130,11 +138,17 @@ def predict(inputs, llm_kwargs, plugin_kwargs, chatbot, history=[], system_promp
|
||||
if "PreProcess" in core_functional[additional_fn]: inputs = core_functional[additional_fn]["PreProcess"](inputs) # 获取预处理函数(如果有的话)
|
||||
inputs = core_functional[additional_fn]["Prefix"] + inputs + core_functional[additional_fn]["Suffix"]
|
||||
|
||||
# 处理历史信息
|
||||
history_feedin = []
|
||||
history_feedin.append(["What can I do?", system_prompt] )
|
||||
for i in range(len(history)//2):
|
||||
history_feedin.append(["What can I do?", system_prompt] )
|
||||
history_feedin.append([history[2*i], history[2*i+1]] )
|
||||
|
||||
# 开始接收chatglm的回复
|
||||
for response in glm_handle.stream_chat(query=inputs, history=history_feedin, max_length=llm_kwargs['max_length'], top_p=llm_kwargs['top_p'], temperature=llm_kwargs['temperature']):
|
||||
chatbot[-1] = (inputs, response)
|
||||
yield from update_ui(chatbot=chatbot, history=history)
|
||||
yield from update_ui(chatbot=chatbot, history=history)
|
||||
|
||||
# 总结输出
|
||||
history.extend([inputs, response])
|
||||
yield from update_ui(chatbot=chatbot, history=history)
|
||||
|
||||
@@ -21,7 +21,7 @@ import importlib
|
||||
|
||||
# config_private.py放自己的秘密如API和代理网址
|
||||
# 读取时首先看是否存在私密的config_private配置文件(不受git管控),如果有,则覆盖原config文件
|
||||
from toolbox import get_conf, update_ui, is_any_api_key, select_api_key
|
||||
from toolbox import get_conf, update_ui, is_any_api_key, select_api_key, what_keys, clip_history, trimmed_format_exc
|
||||
proxies, API_KEY, TIMEOUT_SECONDS, MAX_RETRY = \
|
||||
get_conf('proxies', 'API_KEY', 'TIMEOUT_SECONDS', 'MAX_RETRY')
|
||||
|
||||
@@ -118,7 +118,7 @@ def predict(inputs, llm_kwargs, plugin_kwargs, chatbot, history=[], system_promp
|
||||
"""
|
||||
if is_any_api_key(inputs):
|
||||
chatbot._cookies['api_key'] = inputs
|
||||
chatbot.append(("输入已识别为openai的api_key", "api_key已导入"))
|
||||
chatbot.append(("输入已识别为openai的api_key", what_keys(inputs)))
|
||||
yield from update_ui(chatbot=chatbot, history=history, msg="api_key已导入") # 刷新界面
|
||||
return
|
||||
elif not is_any_api_key(chatbot._cookies['api_key']):
|
||||
@@ -141,11 +141,11 @@ def predict(inputs, llm_kwargs, plugin_kwargs, chatbot, history=[], system_promp
|
||||
try:
|
||||
headers, payload = generate_payload(inputs, llm_kwargs, history, system_prompt, stream)
|
||||
except RuntimeError as e:
|
||||
chatbot[-1] = (inputs, f"您提供的api-key不满足要求,不包含任何可用于{llm_kwargs['llm_model']}的api-key。")
|
||||
chatbot[-1] = (inputs, f"您提供的api-key不满足要求,不包含任何可用于{llm_kwargs['llm_model']}的api-key。您可能选择了错误的模型或请求源。")
|
||||
yield from update_ui(chatbot=chatbot, history=history, msg="api-key不满足要求") # 刷新界面
|
||||
return
|
||||
|
||||
history.append(inputs); history.append(" ")
|
||||
history.append(inputs); history.append("")
|
||||
|
||||
retry = 0
|
||||
while True:
|
||||
@@ -198,19 +198,24 @@ def predict(inputs, llm_kwargs, plugin_kwargs, chatbot, history=[], system_promp
|
||||
chunk_decoded = chunk.decode()
|
||||
error_msg = chunk_decoded
|
||||
if "reduce the length" in error_msg:
|
||||
chatbot[-1] = (chatbot[-1][0], "[Local Message] Reduce the length. 本次输入过长,或历史数据过长. 历史缓存数据现已释放,您可以请再次尝试.")
|
||||
history = [] # 清除历史
|
||||
if len(history) >= 2: history[-1] = ""; history[-2] = "" # 清除当前溢出的输入:history[-2] 是本次输入, history[-1] 是本次输出
|
||||
history = clip_history(inputs=inputs, history=history, tokenizer=model_info[llm_kwargs['llm_model']]['tokenizer'],
|
||||
max_token_limit=(model_info[llm_kwargs['llm_model']]['max_token'])) # history至少释放二分之一
|
||||
chatbot[-1] = (chatbot[-1][0], "[Local Message] Reduce the length. 本次输入过长, 或历史数据过长. 历史缓存数据已部分释放, 您可以请再次尝试. (若再次失败则更可能是因为输入过长.)")
|
||||
# history = [] # 清除历史
|
||||
elif "does not exist" in error_msg:
|
||||
chatbot[-1] = (chatbot[-1][0], f"[Local Message] Model {llm_kwargs['llm_model']} does not exist. 模型不存在,或者您没有获得体验资格.")
|
||||
chatbot[-1] = (chatbot[-1][0], f"[Local Message] Model {llm_kwargs['llm_model']} does not exist. 模型不存在, 或者您没有获得体验资格.")
|
||||
elif "Incorrect API key" in error_msg:
|
||||
chatbot[-1] = (chatbot[-1][0], "[Local Message] Incorrect API key. OpenAI以提供了不正确的API_KEY为由,拒绝服务.")
|
||||
chatbot[-1] = (chatbot[-1][0], "[Local Message] Incorrect API key. OpenAI以提供了不正确的API_KEY为由, 拒绝服务.")
|
||||
elif "exceeded your current quota" in error_msg:
|
||||
chatbot[-1] = (chatbot[-1][0], "[Local Message] You exceeded your current quota. OpenAI以账户额度不足为由,拒绝服务.")
|
||||
chatbot[-1] = (chatbot[-1][0], "[Local Message] You exceeded your current quota. OpenAI以账户额度不足为由, 拒绝服务.")
|
||||
elif "bad forward key" in error_msg:
|
||||
chatbot[-1] = (chatbot[-1][0], "[Local Message] Bad forward key. API2D账户额度不足.")
|
||||
elif "Not enough point" in error_msg:
|
||||
chatbot[-1] = (chatbot[-1][0], "[Local Message] Not enough point. API2D账户点数不足.")
|
||||
else:
|
||||
from toolbox import regular_txt_to_markdown
|
||||
tb_str = '```\n' + traceback.format_exc() + '```'
|
||||
tb_str = '```\n' + trimmed_format_exc() + '```'
|
||||
chatbot[-1] = (chatbot[-1][0], f"[Local Message] 异常 \n\n{tb_str} \n\n{regular_txt_to_markdown(chunk_decoded[4:])}")
|
||||
yield from update_ui(chatbot=chatbot, history=history, msg="Json异常" + error_msg) # 刷新界面
|
||||
return
|
||||
|
||||
250
request_llm/bridge_newbing.py
Normal file
250
request_llm/bridge_newbing.py
Normal file
@@ -0,0 +1,250 @@
|
||||
"""
|
||||
========================================================================
|
||||
第一部分:来自EdgeGPT.py
|
||||
https://github.com/acheong08/EdgeGPT
|
||||
========================================================================
|
||||
"""
|
||||
from .edge_gpt import NewbingChatbot
|
||||
load_message = "等待NewBing响应。"
|
||||
|
||||
"""
|
||||
========================================================================
|
||||
第二部分:子进程Worker(调用主体)
|
||||
========================================================================
|
||||
"""
|
||||
import time
|
||||
import json
|
||||
import re
|
||||
import asyncio
|
||||
import importlib
|
||||
import threading
|
||||
from toolbox import update_ui, get_conf, trimmed_format_exc
|
||||
from multiprocessing import Process, Pipe
|
||||
|
||||
def preprocess_newbing_out(s):
|
||||
pattern = r'\^(\d+)\^' # 匹配^数字^
|
||||
sub = lambda m: '\['+m.group(1)+'\]' # 将匹配到的数字作为替换值
|
||||
result = re.sub(pattern, sub, s) # 替换操作
|
||||
if '[1]' in result:
|
||||
result += '\n\n```\n' + "\n".join([r for r in result.split('\n') if r.startswith('[')]) + '\n```\n'
|
||||
return result
|
||||
|
||||
def preprocess_newbing_out_simple(result):
|
||||
if '[1]' in result:
|
||||
result += '\n\n```\n' + "\n".join([r for r in result.split('\n') if r.startswith('[')]) + '\n```\n'
|
||||
return result
|
||||
|
||||
class NewBingHandle(Process):
|
||||
def __init__(self):
|
||||
super().__init__(daemon=True)
|
||||
self.parent, self.child = Pipe()
|
||||
self.newbing_model = None
|
||||
self.info = ""
|
||||
self.success = True
|
||||
self.local_history = []
|
||||
self.check_dependency()
|
||||
self.start()
|
||||
self.threadLock = threading.Lock()
|
||||
|
||||
def check_dependency(self):
|
||||
try:
|
||||
self.success = False
|
||||
import certifi, httpx, rich
|
||||
self.info = "依赖检测通过,等待NewBing响应。注意目前不能多人同时调用NewBing接口(有线程锁),否则将导致每个人的NewBing问询历史互相渗透。调用NewBing时,会自动使用已配置的代理。"
|
||||
self.success = True
|
||||
except:
|
||||
self.info = "缺少的依赖,如果要使用Newbing,除了基础的pip依赖以外,您还需要运行`pip install -r request_llm/requirements_newbing.txt`安装Newbing的依赖。"
|
||||
self.success = False
|
||||
|
||||
def ready(self):
|
||||
return self.newbing_model is not None
|
||||
|
||||
async def async_run(self):
|
||||
# 读取配置
|
||||
NEWBING_STYLE, = get_conf('NEWBING_STYLE')
|
||||
from request_llm.bridge_all import model_info
|
||||
endpoint = model_info['newbing']['endpoint']
|
||||
while True:
|
||||
# 等待
|
||||
kwargs = self.child.recv()
|
||||
question=kwargs['query']
|
||||
history=kwargs['history']
|
||||
system_prompt=kwargs['system_prompt']
|
||||
|
||||
# 是否重置
|
||||
if len(self.local_history) > 0 and len(history)==0:
|
||||
await self.newbing_model.reset()
|
||||
self.local_history = []
|
||||
|
||||
# 开始问问题
|
||||
prompt = ""
|
||||
if system_prompt not in self.local_history:
|
||||
self.local_history.append(system_prompt)
|
||||
prompt += system_prompt + '\n'
|
||||
|
||||
# 追加历史
|
||||
for ab in history:
|
||||
a, b = ab
|
||||
if a not in self.local_history:
|
||||
self.local_history.append(a)
|
||||
prompt += a + '\n'
|
||||
if b not in self.local_history:
|
||||
self.local_history.append(b)
|
||||
prompt += b + '\n'
|
||||
|
||||
# 问题
|
||||
prompt += question
|
||||
self.local_history.append(question)
|
||||
|
||||
# 提交
|
||||
async for final, response in self.newbing_model.ask_stream(
|
||||
prompt=question,
|
||||
conversation_style=NEWBING_STYLE, # ["creative", "balanced", "precise"]
|
||||
wss_link=endpoint, # "wss://sydney.bing.com/sydney/ChatHub"
|
||||
):
|
||||
if not final:
|
||||
print(response)
|
||||
self.child.send(str(response))
|
||||
else:
|
||||
print('-------- receive final ---------')
|
||||
self.child.send('[Finish]')
|
||||
|
||||
|
||||
def run(self):
|
||||
"""
|
||||
这个函数运行在子进程
|
||||
"""
|
||||
# 第一次运行,加载参数
|
||||
self.success = False
|
||||
self.local_history = []
|
||||
if (self.newbing_model is None) or (not self.success):
|
||||
# 代理设置
|
||||
proxies, = get_conf('proxies')
|
||||
if proxies is None:
|
||||
self.proxies_https = None
|
||||
else:
|
||||
self.proxies_https = proxies['https']
|
||||
# cookie
|
||||
NEWBING_COOKIES, = get_conf('NEWBING_COOKIES')
|
||||
try:
|
||||
cookies = json.loads(NEWBING_COOKIES)
|
||||
except:
|
||||
self.success = False
|
||||
tb_str = '\n```\n' + trimmed_format_exc() + '\n```\n'
|
||||
self.child.send(f'[Local Message] 不能加载Newbing组件。NEWBING_COOKIES未填写或有格式错误。')
|
||||
self.child.send('[Fail]')
|
||||
self.child.send('[Finish]')
|
||||
raise RuntimeError(f"不能加载Newbing组件。NEWBING_COOKIES未填写或有格式错误。")
|
||||
|
||||
try:
|
||||
self.newbing_model = NewbingChatbot(proxy=self.proxies_https, cookies=cookies)
|
||||
except:
|
||||
self.success = False
|
||||
tb_str = '\n```\n' + trimmed_format_exc() + '\n```\n'
|
||||
self.child.send(f'[Local Message] 不能加载Newbing组件。{tb_str}')
|
||||
self.child.send('[Fail]')
|
||||
self.child.send('[Finish]')
|
||||
raise RuntimeError(f"不能加载Newbing组件。")
|
||||
|
||||
self.success = True
|
||||
try:
|
||||
# 进入任务等待状态
|
||||
asyncio.run(self.async_run())
|
||||
except Exception:
|
||||
tb_str = '```\n' + trimmed_format_exc() + '```'
|
||||
self.child.send(f'[Local Message] Newbing失败 {tb_str}.')
|
||||
self.child.send('[Fail]')
|
||||
self.child.send('[Finish]')
|
||||
|
||||
def stream_chat(self, **kwargs):
|
||||
"""
|
||||
这个函数运行在主进程
|
||||
"""
|
||||
self.threadLock.acquire()
|
||||
self.parent.send(kwargs) # 发送请求到子进程
|
||||
while True:
|
||||
res = self.parent.recv() # 等待newbing回复的片段
|
||||
if res == '[Finish]':
|
||||
break # 结束
|
||||
elif res == '[Fail]':
|
||||
self.success = False
|
||||
break
|
||||
else:
|
||||
yield res # newbing回复的片段
|
||||
self.threadLock.release()
|
||||
|
||||
|
||||
"""
|
||||
========================================================================
|
||||
第三部分:主进程统一调用函数接口
|
||||
========================================================================
|
||||
"""
|
||||
global newbing_handle
|
||||
newbing_handle = None
|
||||
|
||||
def predict_no_ui_long_connection(inputs, llm_kwargs, history=[], sys_prompt="", observe_window=None, console_slience=False):
|
||||
"""
|
||||
多线程方法
|
||||
函数的说明请见 request_llm/bridge_all.py
|
||||
"""
|
||||
global newbing_handle
|
||||
if (newbing_handle is None) or (not newbing_handle.success):
|
||||
newbing_handle = NewBingHandle()
|
||||
observe_window[0] = load_message + "\n\n" + newbing_handle.info
|
||||
if not newbing_handle.success:
|
||||
error = newbing_handle.info
|
||||
newbing_handle = None
|
||||
raise RuntimeError(error)
|
||||
|
||||
# 没有 sys_prompt 接口,因此把prompt加入 history
|
||||
history_feedin = []
|
||||
for i in range(len(history)//2):
|
||||
history_feedin.append([history[2*i], history[2*i+1]] )
|
||||
|
||||
watch_dog_patience = 5 # 看门狗 (watchdog) 的耐心, 设置5秒即可
|
||||
response = ""
|
||||
observe_window[0] = "[Local Message]: 等待NewBing响应中 ..."
|
||||
for response in newbing_handle.stream_chat(query=inputs, history=history_feedin, system_prompt=sys_prompt, max_length=llm_kwargs['max_length'], top_p=llm_kwargs['top_p'], temperature=llm_kwargs['temperature']):
|
||||
observe_window[0] = preprocess_newbing_out_simple(response)
|
||||
if len(observe_window) >= 2:
|
||||
if (time.time()-observe_window[1]) > watch_dog_patience:
|
||||
raise RuntimeError("程序终止。")
|
||||
return preprocess_newbing_out_simple(response)
|
||||
|
||||
def predict(inputs, llm_kwargs, plugin_kwargs, chatbot, history=[], system_prompt='', stream = True, additional_fn=None):
|
||||
"""
|
||||
单线程方法
|
||||
函数的说明请见 request_llm/bridge_all.py
|
||||
"""
|
||||
chatbot.append((inputs, "[Local Message]: 等待NewBing响应中 ..."))
|
||||
|
||||
global newbing_handle
|
||||
if (newbing_handle is None) or (not newbing_handle.success):
|
||||
newbing_handle = NewBingHandle()
|
||||
chatbot[-1] = (inputs, load_message + "\n\n" + newbing_handle.info)
|
||||
yield from update_ui(chatbot=chatbot, history=[])
|
||||
if not newbing_handle.success:
|
||||
newbing_handle = None
|
||||
return
|
||||
|
||||
if additional_fn is not None:
|
||||
import core_functional
|
||||
importlib.reload(core_functional) # 热更新prompt
|
||||
core_functional = core_functional.get_core_functions()
|
||||
if "PreProcess" in core_functional[additional_fn]: inputs = core_functional[additional_fn]["PreProcess"](inputs) # 获取预处理函数(如果有的话)
|
||||
inputs = core_functional[additional_fn]["Prefix"] + inputs + core_functional[additional_fn]["Suffix"]
|
||||
|
||||
history_feedin = []
|
||||
for i in range(len(history)//2):
|
||||
history_feedin.append([history[2*i], history[2*i+1]] )
|
||||
|
||||
chatbot[-1] = (inputs, "[Local Message]: 等待NewBing响应中 ...")
|
||||
response = "[Local Message]: 等待NewBing响应中 ..."
|
||||
yield from update_ui(chatbot=chatbot, history=history, msg="NewBing响应缓慢,尚未完成全部响应,请耐心完成后再提交新问题。")
|
||||
for response in newbing_handle.stream_chat(query=inputs, history=history_feedin, system_prompt=system_prompt, max_length=llm_kwargs['max_length'], top_p=llm_kwargs['top_p'], temperature=llm_kwargs['temperature']):
|
||||
chatbot[-1] = (inputs, preprocess_newbing_out(response))
|
||||
yield from update_ui(chatbot=chatbot, history=history, msg="NewBing响应缓慢,尚未完成全部响应,请耐心完成后再提交新问题。")
|
||||
|
||||
history.extend([inputs, preprocess_newbing_out(response)])
|
||||
yield from update_ui(chatbot=chatbot, history=history, msg="完成全部响应,请提交新问题。")
|
||||
|
||||
409
request_llm/edge_gpt.py
Normal file
409
request_llm/edge_gpt.py
Normal file
@@ -0,0 +1,409 @@
|
||||
"""
|
||||
========================================================================
|
||||
第一部分:来自EdgeGPT.py
|
||||
https://github.com/acheong08/EdgeGPT
|
||||
========================================================================
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import asyncio
|
||||
import json
|
||||
import os
|
||||
import random
|
||||
import re
|
||||
import ssl
|
||||
import sys
|
||||
import uuid
|
||||
from enum import Enum
|
||||
from typing import Generator
|
||||
from typing import Literal
|
||||
from typing import Optional
|
||||
from typing import Union
|
||||
import websockets.client as websockets
|
||||
|
||||
DELIMITER = "\x1e"
|
||||
|
||||
|
||||
# Generate random IP between range 13.104.0.0/14
|
||||
FORWARDED_IP = (
|
||||
f"13.{random.randint(104, 107)}.{random.randint(0, 255)}.{random.randint(0, 255)}"
|
||||
)
|
||||
|
||||
HEADERS = {
|
||||
"accept": "application/json",
|
||||
"accept-language": "en-US,en;q=0.9",
|
||||
"content-type": "application/json",
|
||||
"sec-ch-ua": '"Not_A Brand";v="99", "Microsoft Edge";v="110", "Chromium";v="110"',
|
||||
"sec-ch-ua-arch": '"x86"',
|
||||
"sec-ch-ua-bitness": '"64"',
|
||||
"sec-ch-ua-full-version": '"109.0.1518.78"',
|
||||
"sec-ch-ua-full-version-list": '"Chromium";v="110.0.5481.192", "Not A(Brand";v="24.0.0.0", "Microsoft Edge";v="110.0.1587.69"',
|
||||
"sec-ch-ua-mobile": "?0",
|
||||
"sec-ch-ua-model": "",
|
||||
"sec-ch-ua-platform": '"Windows"',
|
||||
"sec-ch-ua-platform-version": '"15.0.0"',
|
||||
"sec-fetch-dest": "empty",
|
||||
"sec-fetch-mode": "cors",
|
||||
"sec-fetch-site": "same-origin",
|
||||
"x-ms-client-request-id": str(uuid.uuid4()),
|
||||
"x-ms-useragent": "azsdk-js-api-client-factory/1.0.0-beta.1 core-rest-pipeline/1.10.0 OS/Win32",
|
||||
"Referer": "https://www.bing.com/search?q=Bing+AI&showconv=1&FORM=hpcodx",
|
||||
"Referrer-Policy": "origin-when-cross-origin",
|
||||
"x-forwarded-for": FORWARDED_IP,
|
||||
}
|
||||
|
||||
HEADERS_INIT_CONVER = {
|
||||
"authority": "edgeservices.bing.com",
|
||||
"accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7",
|
||||
"accept-language": "en-US,en;q=0.9",
|
||||
"cache-control": "max-age=0",
|
||||
"sec-ch-ua": '"Chromium";v="110", "Not A(Brand";v="24", "Microsoft Edge";v="110"',
|
||||
"sec-ch-ua-arch": '"x86"',
|
||||
"sec-ch-ua-bitness": '"64"',
|
||||
"sec-ch-ua-full-version": '"110.0.1587.69"',
|
||||
"sec-ch-ua-full-version-list": '"Chromium";v="110.0.5481.192", "Not A(Brand";v="24.0.0.0", "Microsoft Edge";v="110.0.1587.69"',
|
||||
"sec-ch-ua-mobile": "?0",
|
||||
"sec-ch-ua-model": '""',
|
||||
"sec-ch-ua-platform": '"Windows"',
|
||||
"sec-ch-ua-platform-version": '"15.0.0"',
|
||||
"sec-fetch-dest": "document",
|
||||
"sec-fetch-mode": "navigate",
|
||||
"sec-fetch-site": "none",
|
||||
"sec-fetch-user": "?1",
|
||||
"upgrade-insecure-requests": "1",
|
||||
"user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/110.0.0.0 Safari/537.36 Edg/110.0.1587.69",
|
||||
"x-edge-shopping-flag": "1",
|
||||
"x-forwarded-for": FORWARDED_IP,
|
||||
}
|
||||
|
||||
def get_ssl_context():
|
||||
import certifi
|
||||
ssl_context = ssl.create_default_context()
|
||||
ssl_context.load_verify_locations(certifi.where())
|
||||
return ssl_context
|
||||
|
||||
|
||||
|
||||
class NotAllowedToAccess(Exception):
|
||||
pass
|
||||
|
||||
|
||||
class ConversationStyle(Enum):
|
||||
creative = "h3imaginative,clgalileo,gencontentv3"
|
||||
balanced = "galileo"
|
||||
precise = "h3precise,clgalileo"
|
||||
|
||||
|
||||
CONVERSATION_STYLE_TYPE = Optional[
|
||||
Union[ConversationStyle, Literal["creative", "balanced", "precise"]]
|
||||
]
|
||||
|
||||
|
||||
def _append_identifier(msg: dict) -> str:
|
||||
"""
|
||||
Appends special character to end of message to identify end of message
|
||||
"""
|
||||
# Convert dict to json string
|
||||
return json.dumps(msg) + DELIMITER
|
||||
|
||||
|
||||
def _get_ran_hex(length: int = 32) -> str:
|
||||
"""
|
||||
Returns random hex string
|
||||
"""
|
||||
return "".join(random.choice("0123456789abcdef") for _ in range(length))
|
||||
|
||||
|
||||
class _ChatHubRequest:
|
||||
"""
|
||||
Request object for ChatHub
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
conversation_signature: str,
|
||||
client_id: str,
|
||||
conversation_id: str,
|
||||
invocation_id: int = 0,
|
||||
) -> None:
|
||||
self.struct: dict = {}
|
||||
|
||||
self.client_id: str = client_id
|
||||
self.conversation_id: str = conversation_id
|
||||
self.conversation_signature: str = conversation_signature
|
||||
self.invocation_id: int = invocation_id
|
||||
|
||||
def update(
|
||||
self,
|
||||
prompt,
|
||||
conversation_style,
|
||||
options,
|
||||
) -> None:
|
||||
"""
|
||||
Updates request object
|
||||
"""
|
||||
if options is None:
|
||||
options = [
|
||||
"deepleo",
|
||||
"enable_debug_commands",
|
||||
"disable_emoji_spoken_text",
|
||||
"enablemm",
|
||||
]
|
||||
if conversation_style:
|
||||
if not isinstance(conversation_style, ConversationStyle):
|
||||
conversation_style = getattr(ConversationStyle, conversation_style)
|
||||
options = [
|
||||
"nlu_direct_response_filter",
|
||||
"deepleo",
|
||||
"disable_emoji_spoken_text",
|
||||
"responsible_ai_policy_235",
|
||||
"enablemm",
|
||||
conversation_style.value,
|
||||
"dtappid",
|
||||
"cricinfo",
|
||||
"cricinfov2",
|
||||
"dv3sugg",
|
||||
]
|
||||
self.struct = {
|
||||
"arguments": [
|
||||
{
|
||||
"source": "cib",
|
||||
"optionsSets": options,
|
||||
"sliceIds": [
|
||||
"222dtappid",
|
||||
"225cricinfo",
|
||||
"224locals0",
|
||||
],
|
||||
"traceId": _get_ran_hex(32),
|
||||
"isStartOfSession": self.invocation_id == 0,
|
||||
"message": {
|
||||
"author": "user",
|
||||
"inputMethod": "Keyboard",
|
||||
"text": prompt,
|
||||
"messageType": "Chat",
|
||||
},
|
||||
"conversationSignature": self.conversation_signature,
|
||||
"participant": {
|
||||
"id": self.client_id,
|
||||
},
|
||||
"conversationId": self.conversation_id,
|
||||
},
|
||||
],
|
||||
"invocationId": str(self.invocation_id),
|
||||
"target": "chat",
|
||||
"type": 4,
|
||||
}
|
||||
self.invocation_id += 1
|
||||
|
||||
|
||||
class _Conversation:
|
||||
"""
|
||||
Conversation API
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
cookies,
|
||||
proxy,
|
||||
) -> None:
|
||||
self.struct: dict = {
|
||||
"conversationId": None,
|
||||
"clientId": None,
|
||||
"conversationSignature": None,
|
||||
"result": {"value": "Success", "message": None},
|
||||
}
|
||||
import httpx
|
||||
self.proxy = proxy
|
||||
proxy = (
|
||||
proxy
|
||||
or os.environ.get("all_proxy")
|
||||
or os.environ.get("ALL_PROXY")
|
||||
or os.environ.get("https_proxy")
|
||||
or os.environ.get("HTTPS_PROXY")
|
||||
or None
|
||||
)
|
||||
if proxy is not None and proxy.startswith("socks5h://"):
|
||||
proxy = "socks5://" + proxy[len("socks5h://") :]
|
||||
self.session = httpx.Client(
|
||||
proxies=proxy,
|
||||
timeout=30,
|
||||
headers=HEADERS_INIT_CONVER,
|
||||
)
|
||||
for cookie in cookies:
|
||||
self.session.cookies.set(cookie["name"], cookie["value"])
|
||||
|
||||
# Send GET request
|
||||
response = self.session.get(
|
||||
url=os.environ.get("BING_PROXY_URL")
|
||||
or "https://edgeservices.bing.com/edgesvc/turing/conversation/create",
|
||||
)
|
||||
if response.status_code != 200:
|
||||
response = self.session.get(
|
||||
"https://edge.churchless.tech/edgesvc/turing/conversation/create",
|
||||
)
|
||||
if response.status_code != 200:
|
||||
print(f"Status code: {response.status_code}")
|
||||
print(response.text)
|
||||
print(response.url)
|
||||
raise Exception("Authentication failed")
|
||||
try:
|
||||
self.struct = response.json()
|
||||
except (json.decoder.JSONDecodeError, NotAllowedToAccess) as exc:
|
||||
raise Exception(
|
||||
"Authentication failed. You have not been accepted into the beta.",
|
||||
) from exc
|
||||
if self.struct["result"]["value"] == "UnauthorizedRequest":
|
||||
raise NotAllowedToAccess(self.struct["result"]["message"])
|
||||
|
||||
|
||||
class _ChatHub:
|
||||
"""
|
||||
Chat API
|
||||
"""
|
||||
|
||||
def __init__(self, conversation) -> None:
|
||||
self.wss = None
|
||||
self.request: _ChatHubRequest
|
||||
self.loop: bool
|
||||
self.task: asyncio.Task
|
||||
print(conversation.struct)
|
||||
self.request = _ChatHubRequest(
|
||||
conversation_signature=conversation.struct["conversationSignature"],
|
||||
client_id=conversation.struct["clientId"],
|
||||
conversation_id=conversation.struct["conversationId"],
|
||||
)
|
||||
|
||||
async def ask_stream(
|
||||
self,
|
||||
prompt: str,
|
||||
wss_link: str,
|
||||
conversation_style: CONVERSATION_STYLE_TYPE = None,
|
||||
raw: bool = False,
|
||||
options: dict = None,
|
||||
) -> Generator[str, None, None]:
|
||||
"""
|
||||
Ask a question to the bot
|
||||
"""
|
||||
if self.wss and not self.wss.closed:
|
||||
await self.wss.close()
|
||||
# Check if websocket is closed
|
||||
self.wss = await websockets.connect(
|
||||
wss_link,
|
||||
extra_headers=HEADERS,
|
||||
max_size=None,
|
||||
ssl=get_ssl_context()
|
||||
)
|
||||
await self._initial_handshake()
|
||||
# Construct a ChatHub request
|
||||
self.request.update(
|
||||
prompt=prompt,
|
||||
conversation_style=conversation_style,
|
||||
options=options,
|
||||
)
|
||||
# Send request
|
||||
await self.wss.send(_append_identifier(self.request.struct))
|
||||
final = False
|
||||
while not final:
|
||||
objects = str(await self.wss.recv()).split(DELIMITER)
|
||||
for obj in objects:
|
||||
if obj is None or not obj:
|
||||
continue
|
||||
response = json.loads(obj)
|
||||
if response.get("type") != 2 and raw:
|
||||
yield False, response
|
||||
elif response.get("type") == 1 and response["arguments"][0].get(
|
||||
"messages",
|
||||
):
|
||||
resp_txt = response["arguments"][0]["messages"][0]["adaptiveCards"][
|
||||
0
|
||||
]["body"][0].get("text")
|
||||
yield False, resp_txt
|
||||
elif response.get("type") == 2:
|
||||
final = True
|
||||
yield True, response
|
||||
|
||||
async def _initial_handshake(self) -> None:
|
||||
await self.wss.send(_append_identifier({"protocol": "json", "version": 1}))
|
||||
await self.wss.recv()
|
||||
|
||||
async def close(self) -> None:
|
||||
"""
|
||||
Close the connection
|
||||
"""
|
||||
if self.wss and not self.wss.closed:
|
||||
await self.wss.close()
|
||||
|
||||
|
||||
class NewbingChatbot:
|
||||
"""
|
||||
Combines everything to make it seamless
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
cookies,
|
||||
proxy
|
||||
) -> None:
|
||||
if cookies is None:
|
||||
cookies = {}
|
||||
self.cookies = cookies
|
||||
self.proxy = proxy
|
||||
self.chat_hub: _ChatHub = _ChatHub(
|
||||
_Conversation(self.cookies, self.proxy),
|
||||
)
|
||||
|
||||
async def ask(
|
||||
self,
|
||||
prompt: str,
|
||||
wss_link: str,
|
||||
conversation_style: CONVERSATION_STYLE_TYPE = None,
|
||||
options: dict = None,
|
||||
) -> dict:
|
||||
"""
|
||||
Ask a question to the bot
|
||||
"""
|
||||
async for final, response in self.chat_hub.ask_stream(
|
||||
prompt=prompt,
|
||||
conversation_style=conversation_style,
|
||||
wss_link=wss_link,
|
||||
options=options,
|
||||
):
|
||||
if final:
|
||||
return response
|
||||
await self.chat_hub.wss.close()
|
||||
return None
|
||||
|
||||
async def ask_stream(
|
||||
self,
|
||||
prompt: str,
|
||||
wss_link: str,
|
||||
conversation_style: CONVERSATION_STYLE_TYPE = None,
|
||||
raw: bool = False,
|
||||
options: dict = None,
|
||||
) -> Generator[str, None, None]:
|
||||
"""
|
||||
Ask a question to the bot
|
||||
"""
|
||||
async for response in self.chat_hub.ask_stream(
|
||||
prompt=prompt,
|
||||
conversation_style=conversation_style,
|
||||
wss_link=wss_link,
|
||||
raw=raw,
|
||||
options=options,
|
||||
):
|
||||
yield response
|
||||
|
||||
async def close(self) -> None:
|
||||
"""
|
||||
Close the connection
|
||||
"""
|
||||
await self.chat_hub.close()
|
||||
|
||||
async def reset(self) -> None:
|
||||
"""
|
||||
Reset the conversation
|
||||
"""
|
||||
await self.close()
|
||||
self.chat_hub = _ChatHub(_Conversation(self.cookies, self.proxy))
|
||||
|
||||
|
||||
8
request_llm/requirements_newbing.txt
Normal file
8
request_llm/requirements_newbing.txt
Normal file
@@ -0,0 +1,8 @@
|
||||
BingImageCreator
|
||||
certifi
|
||||
httpx
|
||||
prompt_toolkit
|
||||
requests
|
||||
rich
|
||||
websockets
|
||||
httpx[socks]
|
||||
247
theme.py
247
theme.py
@@ -137,6 +137,16 @@ advanced_css = """
|
||||
|
||||
/* 行内代码的背景设为淡灰色,设定圆角和间距. */
|
||||
.markdown-body code {
|
||||
display: inline;
|
||||
white-space: break-spaces;
|
||||
border-radius: 6px;
|
||||
margin: 0 2px 0 2px;
|
||||
padding: .2em .4em .1em .4em;
|
||||
background-color: rgba(13, 17, 23, 0.95);
|
||||
color: #c9d1d9;
|
||||
}
|
||||
|
||||
.dark .markdown-body code {
|
||||
display: inline;
|
||||
white-space: break-spaces;
|
||||
border-radius: 6px;
|
||||
@@ -144,8 +154,19 @@ advanced_css = """
|
||||
padding: .2em .4em .1em .4em;
|
||||
background-color: rgba(175,184,193,0.2);
|
||||
}
|
||||
|
||||
/* 设定代码块的样式,包括背景颜色、内、外边距、圆角。 */
|
||||
.markdown-body pre code {
|
||||
display: block;
|
||||
overflow: auto;
|
||||
white-space: pre;
|
||||
background-color: rgba(13, 17, 23, 0.95);
|
||||
border-radius: 10px;
|
||||
padding: 1em;
|
||||
margin: 1em 2em 1em 0.5em;
|
||||
}
|
||||
|
||||
.dark .markdown-body pre code {
|
||||
display: block;
|
||||
overflow: auto;
|
||||
white-space: pre;
|
||||
@@ -160,72 +181,162 @@ advanced_css = """
|
||||
if CODE_HIGHLIGHT:
|
||||
advanced_css += """
|
||||
|
||||
.hll { background-color: #ffffcc }
|
||||
.c { color: #3D7B7B; font-style: italic } /* Comment */
|
||||
.err { border: 1px solid #FF0000 } /* Error */
|
||||
.k { color: hsl(197, 94%, 51%); font-weight: bold } /* Keyword */
|
||||
.o { color: #666666 } /* Operator */
|
||||
.ch { color: #3D7B7B; font-style: italic } /* Comment.Hashbang */
|
||||
.cm { color: #3D7B7B; font-style: italic } /* Comment.Multiline */
|
||||
.cp { color: #9C6500 } /* Comment.Preproc */
|
||||
.cpf { color: #3D7B7B; font-style: italic } /* Comment.PreprocFile */
|
||||
.c1 { color: #3D7B7B; font-style: italic } /* Comment.Single */
|
||||
.cs { color: #3D7B7B; font-style: italic } /* Comment.Special */
|
||||
.gd { color: #A00000 } /* Generic.Deleted */
|
||||
.ge { font-style: italic } /* Generic.Emph */
|
||||
.gr { color: #E40000 } /* Generic.Error */
|
||||
.gh { color: #000080; font-weight: bold } /* Generic.Heading */
|
||||
.gi { color: #008400 } /* Generic.Inserted */
|
||||
.go { color: #717171 } /* Generic.Output */
|
||||
.gp { color: #000080; font-weight: bold } /* Generic.Prompt */
|
||||
.gs { font-weight: bold } /* Generic.Strong */
|
||||
.gu { color: #800080; font-weight: bold } /* Generic.Subheading */
|
||||
.gt { color: #a9dd00 } /* Generic.Traceback */
|
||||
.kc { color: #008000; font-weight: bold } /* Keyword.Constant */
|
||||
.kd { color: #008000; font-weight: bold } /* Keyword.Declaration */
|
||||
.kn { color: #008000; font-weight: bold } /* Keyword.Namespace */
|
||||
.kp { color: #008000 } /* Keyword.Pseudo */
|
||||
.kr { color: #008000; font-weight: bold } /* Keyword.Reserved */
|
||||
.kt { color: #B00040 } /* Keyword.Type */
|
||||
.m { color: #666666 } /* Literal.Number */
|
||||
.s { color: #BA2121 } /* Literal.String */
|
||||
.na { color: #687822 } /* Name.Attribute */
|
||||
.nb { color: #e5f8c3 } /* Name.Builtin */
|
||||
.nc { color: #ffad65; font-weight: bold } /* Name.Class */
|
||||
.no { color: #880000 } /* Name.Constant */
|
||||
.nd { color: #AA22FF } /* Name.Decorator */
|
||||
.ni { color: #717171; font-weight: bold } /* Name.Entity */
|
||||
.ne { color: #CB3F38; font-weight: bold } /* Name.Exception */
|
||||
.nf { color: #f9f978 } /* Name.Function */
|
||||
.nl { color: #767600 } /* Name.Label */
|
||||
.nn { color: #0000FF; font-weight: bold } /* Name.Namespace */
|
||||
.nt { color: #008000; font-weight: bold } /* Name.Tag */
|
||||
.nv { color: #19177C } /* Name.Variable */
|
||||
.ow { color: #AA22FF; font-weight: bold } /* Operator.Word */
|
||||
.w { color: #bbbbbb } /* Text.Whitespace */
|
||||
.mb { color: #666666 } /* Literal.Number.Bin */
|
||||
.mf { color: #666666 } /* Literal.Number.Float */
|
||||
.mh { color: #666666 } /* Literal.Number.Hex */
|
||||
.mi { color: #666666 } /* Literal.Number.Integer */
|
||||
.mo { color: #666666 } /* Literal.Number.Oct */
|
||||
.sa { color: #BA2121 } /* Literal.String.Affix */
|
||||
.sb { color: #BA2121 } /* Literal.String.Backtick */
|
||||
.sc { color: #BA2121 } /* Literal.String.Char */
|
||||
.dl { color: #BA2121 } /* Literal.String.Delimiter */
|
||||
.sd { color: #BA2121; font-style: italic } /* Literal.String.Doc */
|
||||
.s2 { color: #2bf840 } /* Literal.String.Double */
|
||||
.se { color: #AA5D1F; font-weight: bold } /* Literal.String.Escape */
|
||||
.sh { color: #BA2121 } /* Literal.String.Heredoc */
|
||||
.si { color: #A45A77; font-weight: bold } /* Literal.String.Interpol */
|
||||
.sx { color: #008000 } /* Literal.String.Other */
|
||||
.sr { color: #A45A77 } /* Literal.String.Regex */
|
||||
.s1 { color: #BA2121 } /* Literal.String.Single */
|
||||
.ss { color: #19177C } /* Literal.String.Symbol */
|
||||
.bp { color: #008000 } /* Name.Builtin.Pseudo */
|
||||
.fm { color: #0000FF } /* Name.Function.Magic */
|
||||
.vc { color: #19177C } /* Name.Variable.Class */
|
||||
.vg { color: #19177C } /* Name.Variable.Global */
|
||||
.vi { color: #19177C } /* Name.Variable.Instance */
|
||||
.vm { color: #19177C } /* Name.Variable.Magic */
|
||||
.il { color: #666666 } /* Literal.Number.Integer.Long */
|
||||
.codehilite .hll { background-color: #6e7681 }
|
||||
.codehilite .c { color: #8b949e; font-style: italic } /* Comment */
|
||||
.codehilite .err { color: #f85149 } /* Error */
|
||||
.codehilite .esc { color: #c9d1d9 } /* Escape */
|
||||
.codehilite .g { color: #c9d1d9 } /* Generic */
|
||||
.codehilite .k { color: #ff7b72 } /* Keyword */
|
||||
.codehilite .l { color: #a5d6ff } /* Literal */
|
||||
.codehilite .n { color: #c9d1d9 } /* Name */
|
||||
.codehilite .o { color: #ff7b72; font-weight: bold } /* Operator */
|
||||
.codehilite .x { color: #c9d1d9 } /* Other */
|
||||
.codehilite .p { color: #c9d1d9 } /* Punctuation */
|
||||
.codehilite .ch { color: #8b949e; font-style: italic } /* Comment.Hashbang */
|
||||
.codehilite .cm { color: #8b949e; font-style: italic } /* Comment.Multiline */
|
||||
.codehilite .cp { color: #8b949e; font-weight: bold; font-style: italic } /* Comment.Preproc */
|
||||
.codehilite .cpf { color: #8b949e; font-style: italic } /* Comment.PreprocFile */
|
||||
.codehilite .c1 { color: #8b949e; font-style: italic } /* Comment.Single */
|
||||
.codehilite .cs { color: #8b949e; font-weight: bold; font-style: italic } /* Comment.Special */
|
||||
.codehilite .gd { color: #ffa198; background-color: #490202 } /* Generic.Deleted */
|
||||
.codehilite .ge { color: #c9d1d9; font-style: italic } /* Generic.Emph */
|
||||
.codehilite .gr { color: #ffa198 } /* Generic.Error */
|
||||
.codehilite .gh { color: #79c0ff; font-weight: bold } /* Generic.Heading */
|
||||
.codehilite .gi { color: #56d364; background-color: #0f5323 } /* Generic.Inserted */
|
||||
.codehilite .go { color: #8b949e } /* Generic.Output */
|
||||
.codehilite .gp { color: #8b949e } /* Generic.Prompt */
|
||||
.codehilite .gs { color: #c9d1d9; font-weight: bold } /* Generic.Strong */
|
||||
.codehilite .gu { color: #79c0ff } /* Generic.Subheading */
|
||||
.codehilite .gt { color: #ff7b72 } /* Generic.Traceback */
|
||||
.codehilite .g-Underline { color: #c9d1d9; text-decoration: underline } /* Generic.Underline */
|
||||
.codehilite .kc { color: #79c0ff } /* Keyword.Constant */
|
||||
.codehilite .kd { color: #ff7b72 } /* Keyword.Declaration */
|
||||
.codehilite .kn { color: #ff7b72 } /* Keyword.Namespace */
|
||||
.codehilite .kp { color: #79c0ff } /* Keyword.Pseudo */
|
||||
.codehilite .kr { color: #ff7b72 } /* Keyword.Reserved */
|
||||
.codehilite .kt { color: #ff7b72 } /* Keyword.Type */
|
||||
.codehilite .ld { color: #79c0ff } /* Literal.Date */
|
||||
.codehilite .m { color: #a5d6ff } /* Literal.Number */
|
||||
.codehilite .s { color: #a5d6ff } /* Literal.String */
|
||||
.codehilite .na { color: #c9d1d9 } /* Name.Attribute */
|
||||
.codehilite .nb { color: #c9d1d9 } /* Name.Builtin */
|
||||
.codehilite .nc { color: #f0883e; font-weight: bold } /* Name.Class */
|
||||
.codehilite .no { color: #79c0ff; font-weight: bold } /* Name.Constant */
|
||||
.codehilite .nd { color: #d2a8ff; font-weight: bold } /* Name.Decorator */
|
||||
.codehilite .ni { color: #ffa657 } /* Name.Entity */
|
||||
.codehilite .ne { color: #f0883e; font-weight: bold } /* Name.Exception */
|
||||
.codehilite .nf { color: #d2a8ff; font-weight: bold } /* Name.Function */
|
||||
.codehilite .nl { color: #79c0ff; font-weight: bold } /* Name.Label */
|
||||
.codehilite .nn { color: #ff7b72 } /* Name.Namespace */
|
||||
.codehilite .nx { color: #c9d1d9 } /* Name.Other */
|
||||
.codehilite .py { color: #79c0ff } /* Name.Property */
|
||||
.codehilite .nt { color: #7ee787 } /* Name.Tag */
|
||||
.codehilite .nv { color: #79c0ff } /* Name.Variable */
|
||||
.codehilite .ow { color: #ff7b72; font-weight: bold } /* Operator.Word */
|
||||
.codehilite .pm { color: #c9d1d9 } /* Punctuation.Marker */
|
||||
.codehilite .w { color: #6e7681 } /* Text.Whitespace */
|
||||
.codehilite .mb { color: #a5d6ff } /* Literal.Number.Bin */
|
||||
.codehilite .mf { color: #a5d6ff } /* Literal.Number.Float */
|
||||
.codehilite .mh { color: #a5d6ff } /* Literal.Number.Hex */
|
||||
.codehilite .mi { color: #a5d6ff } /* Literal.Number.Integer */
|
||||
.codehilite .mo { color: #a5d6ff } /* Literal.Number.Oct */
|
||||
.codehilite .sa { color: #79c0ff } /* Literal.String.Affix */
|
||||
.codehilite .sb { color: #a5d6ff } /* Literal.String.Backtick */
|
||||
.codehilite .sc { color: #a5d6ff } /* Literal.String.Char */
|
||||
.codehilite .dl { color: #79c0ff } /* Literal.String.Delimiter */
|
||||
.codehilite .sd { color: #a5d6ff } /* Literal.String.Doc */
|
||||
.codehilite .s2 { color: #a5d6ff } /* Literal.String.Double */
|
||||
.codehilite .se { color: #79c0ff } /* Literal.String.Escape */
|
||||
.codehilite .sh { color: #79c0ff } /* Literal.String.Heredoc */
|
||||
.codehilite .si { color: #a5d6ff } /* Literal.String.Interpol */
|
||||
.codehilite .sx { color: #a5d6ff } /* Literal.String.Other */
|
||||
.codehilite .sr { color: #79c0ff } /* Literal.String.Regex */
|
||||
.codehilite .s1 { color: #a5d6ff } /* Literal.String.Single */
|
||||
.codehilite .ss { color: #a5d6ff } /* Literal.String.Symbol */
|
||||
.codehilite .bp { color: #c9d1d9 } /* Name.Builtin.Pseudo */
|
||||
.codehilite .fm { color: #d2a8ff; font-weight: bold } /* Name.Function.Magic */
|
||||
.codehilite .vc { color: #79c0ff } /* Name.Variable.Class */
|
||||
.codehilite .vg { color: #79c0ff } /* Name.Variable.Global */
|
||||
.codehilite .vi { color: #79c0ff } /* Name.Variable.Instance */
|
||||
.codehilite .vm { color: #79c0ff } /* Name.Variable.Magic */
|
||||
.codehilite .il { color: #a5d6ff } /* Literal.Number.Integer.Long */
|
||||
|
||||
.dark .codehilite .hll { background-color: #2C3B41 }
|
||||
.dark .codehilite .c { color: #79d618; font-style: italic } /* Comment */
|
||||
.dark .codehilite .err { color: #FF5370 } /* Error */
|
||||
.dark .codehilite .esc { color: #89DDFF } /* Escape */
|
||||
.dark .codehilite .g { color: #EEFFFF } /* Generic */
|
||||
.dark .codehilite .k { color: #BB80B3 } /* Keyword */
|
||||
.dark .codehilite .l { color: #C3E88D } /* Literal */
|
||||
.dark .codehilite .n { color: #EEFFFF } /* Name */
|
||||
.dark .codehilite .o { color: #89DDFF } /* Operator */
|
||||
.dark .codehilite .p { color: #89DDFF } /* Punctuation */
|
||||
.dark .codehilite .ch { color: #79d618; font-style: italic } /* Comment.Hashbang */
|
||||
.dark .codehilite .cm { color: #79d618; font-style: italic } /* Comment.Multiline */
|
||||
.dark .codehilite .cp { color: #79d618; font-style: italic } /* Comment.Preproc */
|
||||
.dark .codehilite .cpf { color: #79d618; font-style: italic } /* Comment.PreprocFile */
|
||||
.dark .codehilite .c1 { color: #79d618; font-style: italic } /* Comment.Single */
|
||||
.dark .codehilite .cs { color: #79d618; font-style: italic } /* Comment.Special */
|
||||
.dark .codehilite .gd { color: #FF5370 } /* Generic.Deleted */
|
||||
.dark .codehilite .ge { color: #89DDFF } /* Generic.Emph */
|
||||
.dark .codehilite .gr { color: #FF5370 } /* Generic.Error */
|
||||
.dark .codehilite .gh { color: #C3E88D } /* Generic.Heading */
|
||||
.dark .codehilite .gi { color: #C3E88D } /* Generic.Inserted */
|
||||
.dark .codehilite .go { color: #79d618 } /* Generic.Output */
|
||||
.dark .codehilite .gp { color: #FFCB6B } /* Generic.Prompt */
|
||||
.dark .codehilite .gs { color: #FF5370 } /* Generic.Strong */
|
||||
.dark .codehilite .gu { color: #89DDFF } /* Generic.Subheading */
|
||||
.dark .codehilite .gt { color: #FF5370 } /* Generic.Traceback */
|
||||
.dark .codehilite .kc { color: #89DDFF } /* Keyword.Constant */
|
||||
.dark .codehilite .kd { color: #BB80B3 } /* Keyword.Declaration */
|
||||
.dark .codehilite .kn { color: #89DDFF; font-style: italic } /* Keyword.Namespace */
|
||||
.dark .codehilite .kp { color: #89DDFF } /* Keyword.Pseudo */
|
||||
.dark .codehilite .kr { color: #BB80B3 } /* Keyword.Reserved */
|
||||
.dark .codehilite .kt { color: #BB80B3 } /* Keyword.Type */
|
||||
.dark .codehilite .ld { color: #C3E88D } /* Literal.Date */
|
||||
.dark .codehilite .m { color: #F78C6C } /* Literal.Number */
|
||||
.dark .codehilite .s { color: #C3E88D } /* Literal.String */
|
||||
.dark .codehilite .na { color: #BB80B3 } /* Name.Attribute */
|
||||
.dark .codehilite .nb { color: #82AAFF } /* Name.Builtin */
|
||||
.dark .codehilite .nc { color: #FFCB6B } /* Name.Class */
|
||||
.dark .codehilite .no { color: #EEFFFF } /* Name.Constant */
|
||||
.dark .codehilite .nd { color: #82AAFF } /* Name.Decorator */
|
||||
.dark .codehilite .ni { color: #89DDFF } /* Name.Entity */
|
||||
.dark .codehilite .ne { color: #FFCB6B } /* Name.Exception */
|
||||
.dark .codehilite .nf { color: #82AAFF } /* Name.Function */
|
||||
.dark .codehilite .nl { color: #82AAFF } /* Name.Label */
|
||||
.dark .codehilite .nn { color: #FFCB6B } /* Name.Namespace */
|
||||
.dark .codehilite .nx { color: #EEFFFF } /* Name.Other */
|
||||
.dark .codehilite .py { color: #FFCB6B } /* Name.Property */
|
||||
.dark .codehilite .nt { color: #FF5370 } /* Name.Tag */
|
||||
.dark .codehilite .nv { color: #89DDFF } /* Name.Variable */
|
||||
.dark .codehilite .ow { color: #89DDFF; font-style: italic } /* Operator.Word */
|
||||
.dark .codehilite .pm { color: #89DDFF } /* Punctuation.Marker */
|
||||
.dark .codehilite .w { color: #EEFFFF } /* Text.Whitespace */
|
||||
.dark .codehilite .mb { color: #F78C6C } /* Literal.Number.Bin */
|
||||
.dark .codehilite .mf { color: #F78C6C } /* Literal.Number.Float */
|
||||
.dark .codehilite .mh { color: #F78C6C } /* Literal.Number.Hex */
|
||||
.dark .codehilite .mi { color: #F78C6C } /* Literal.Number.Integer */
|
||||
.dark .codehilite .mo { color: #F78C6C } /* Literal.Number.Oct */
|
||||
.dark .codehilite .sa { color: #BB80B3 } /* Literal.String.Affix */
|
||||
.dark .codehilite .sb { color: #C3E88D } /* Literal.String.Backtick */
|
||||
.dark .codehilite .sc { color: #C3E88D } /* Literal.String.Char */
|
||||
.dark .codehilite .dl { color: #EEFFFF } /* Literal.String.Delimiter */
|
||||
.dark .codehilite .sd { color: #79d618; font-style: italic } /* Literal.String.Doc */
|
||||
.dark .codehilite .s2 { color: #C3E88D } /* Literal.String.Double */
|
||||
.dark .codehilite .se { color: #EEFFFF } /* Literal.String.Escape */
|
||||
.dark .codehilite .sh { color: #C3E88D } /* Literal.String.Heredoc */
|
||||
.dark .codehilite .si { color: #89DDFF } /* Literal.String.Interpol */
|
||||
.dark .codehilite .sx { color: #C3E88D } /* Literal.String.Other */
|
||||
.dark .codehilite .sr { color: #89DDFF } /* Literal.String.Regex */
|
||||
.dark .codehilite .s1 { color: #C3E88D } /* Literal.String.Single */
|
||||
.dark .codehilite .ss { color: #89DDFF } /* Literal.String.Symbol */
|
||||
.dark .codehilite .bp { color: #89DDFF } /* Name.Builtin.Pseudo */
|
||||
.dark .codehilite .fm { color: #82AAFF } /* Name.Function.Magic */
|
||||
.dark .codehilite .vc { color: #89DDFF } /* Name.Variable.Class */
|
||||
.dark .codehilite .vg { color: #89DDFF } /* Name.Variable.Global */
|
||||
.dark .codehilite .vi { color: #89DDFF } /* Name.Variable.Instance */
|
||||
.dark .codehilite .vm { color: #82AAFF } /* Name.Variable.Magic */
|
||||
.dark .codehilite .il { color: #F78C6C } /* Literal.Number.Integer.Long */
|
||||
|
||||
"""
|
||||
|
||||
172
toolbox.py
172
toolbox.py
@@ -5,7 +5,20 @@ import inspect
|
||||
import re
|
||||
from latex2mathml.converter import convert as tex2mathml
|
||||
from functools import wraps, lru_cache
|
||||
############################### 插件输入输出接驳区 #######################################
|
||||
|
||||
"""
|
||||
========================================================================
|
||||
第一部分
|
||||
函数插件输入输出接驳区
|
||||
- ChatBotWithCookies: 带Cookies的Chatbot类,为实现更多强大的功能做基础
|
||||
- ArgsGeneralWrapper: 装饰器函数,用于重组输入参数,改变输入参数的顺序与结构
|
||||
- update_ui: 刷新界面用 yield from update_ui(chatbot, history)
|
||||
- CatchException: 将插件中出的所有问题显示在界面上
|
||||
- HotReload: 实现插件的热更新
|
||||
- trimmed_format_exc: 打印traceback,为了安全而隐藏绝对地址
|
||||
========================================================================
|
||||
"""
|
||||
|
||||
class ChatBotWithCookies(list):
|
||||
def __init__(self, cookie):
|
||||
self._cookies = cookie
|
||||
@@ -20,33 +33,35 @@ class ChatBotWithCookies(list):
|
||||
def get_cookies(self):
|
||||
return self._cookies
|
||||
|
||||
|
||||
def ArgsGeneralWrapper(f):
|
||||
"""
|
||||
装饰器函数,用于重组输入参数,改变输入参数的顺序与结构。
|
||||
"""
|
||||
def decorated(cookies, max_length, llm_model, txt, txt2, top_p, temperature, chatbot, history, system_prompt, *args):
|
||||
def decorated(cookies, max_length, llm_model, txt, txt2, top_p, temperature, chatbot, history, system_prompt, plugin_advanced_arg, *args):
|
||||
txt_passon = txt
|
||||
if txt == "" and txt2 != "": txt_passon = txt2
|
||||
# 引入一个有cookie的chatbot
|
||||
cookies.update({
|
||||
'top_p':top_p,
|
||||
'top_p':top_p,
|
||||
'temperature':temperature,
|
||||
})
|
||||
llm_kwargs = {
|
||||
'api_key': cookies['api_key'],
|
||||
'llm_model': llm_model,
|
||||
'top_p':top_p,
|
||||
'top_p':top_p,
|
||||
'max_length': max_length,
|
||||
'temperature':temperature,
|
||||
}
|
||||
plugin_kwargs = {
|
||||
# 目前还没有
|
||||
"advanced_arg": plugin_advanced_arg,
|
||||
}
|
||||
chatbot_with_cookie = ChatBotWithCookies(cookies)
|
||||
chatbot_with_cookie.write_list(chatbot)
|
||||
yield from f(txt_passon, llm_kwargs, plugin_kwargs, chatbot_with_cookie, history, system_prompt, *args)
|
||||
return decorated
|
||||
|
||||
|
||||
def update_ui(chatbot, history, msg='正常', **kwargs): # 刷新界面
|
||||
"""
|
||||
刷新用户界面
|
||||
@@ -54,10 +69,18 @@ def update_ui(chatbot, history, msg='正常', **kwargs): # 刷新界面
|
||||
assert isinstance(chatbot, ChatBotWithCookies), "在传递chatbot的过程中不要将其丢弃。必要时,可用clear将其清空,然后用for+append循环重新赋值。"
|
||||
yield chatbot.get_cookies(), chatbot, history, msg
|
||||
|
||||
def trimmed_format_exc():
|
||||
import os, traceback
|
||||
str = traceback.format_exc()
|
||||
current_path = os.getcwd()
|
||||
replace_path = "."
|
||||
return str.replace(current_path, replace_path)
|
||||
|
||||
def CatchException(f):
|
||||
"""
|
||||
装饰器函数,捕捉函数f中的异常并封装到一个生成器中返回,并显示到聊天当中。
|
||||
"""
|
||||
|
||||
@wraps(f)
|
||||
def decorated(txt, top_p, temperature, chatbot, history, systemPromptTxt, WEB_PORT):
|
||||
try:
|
||||
@@ -66,7 +89,7 @@ def CatchException(f):
|
||||
from check_proxy import check_proxy
|
||||
from toolbox import get_conf
|
||||
proxies, = get_conf('proxies')
|
||||
tb_str = '```\n' + traceback.format_exc() + '```'
|
||||
tb_str = '```\n' + trimmed_format_exc() + '```'
|
||||
if chatbot is None or len(chatbot) == 0:
|
||||
chatbot = [["插件调度异常", "异常原因"]]
|
||||
chatbot[-1] = (chatbot[-1][0],
|
||||
@@ -93,7 +116,23 @@ def HotReload(f):
|
||||
return decorated
|
||||
|
||||
|
||||
####################################### 其他小工具 #####################################
|
||||
"""
|
||||
========================================================================
|
||||
第二部分
|
||||
其他小工具:
|
||||
- write_results_to_file: 将结果写入markdown文件中
|
||||
- regular_txt_to_markdown: 将普通文本转换为Markdown格式的文本。
|
||||
- report_execption: 向chatbot中添加简单的意外错误信息
|
||||
- text_divide_paragraph: 将文本按照段落分隔符分割开,生成带有段落标签的HTML代码。
|
||||
- markdown_convertion: 用多种方式组合,将markdown转化为好看的html
|
||||
- format_io: 接管gradio默认的markdown处理方式
|
||||
- on_file_uploaded: 处理文件的上传(自动解压)
|
||||
- on_report_generated: 将生成的报告自动投射到文件上传区
|
||||
- clip_history: 当历史上下文过长时,自动截断
|
||||
- get_conf: 获取设置
|
||||
- select_api_key: 根据当前的模型类别,抽取可用的api-key
|
||||
========================================================================
|
||||
"""
|
||||
|
||||
def get_reduce_token_percent(text):
|
||||
"""
|
||||
@@ -113,7 +152,6 @@ def get_reduce_token_percent(text):
|
||||
return 0.5, '不详'
|
||||
|
||||
|
||||
|
||||
def write_results_to_file(history, file_name=None):
|
||||
"""
|
||||
将对话记录history以Markdown格式写入文件中。如果没有指定文件名,则使用当前时间生成文件名。
|
||||
@@ -219,7 +257,7 @@ def markdown_convertion(txt):
|
||||
return content
|
||||
else:
|
||||
return tex2mathml_catch_exception(content)
|
||||
|
||||
|
||||
def markdown_bug_hunt(content):
|
||||
"""
|
||||
解决一个mdx_math的bug(单$包裹begin命令时多余<script>)
|
||||
@@ -227,7 +265,7 @@ def markdown_convertion(txt):
|
||||
content = content.replace('<script type="math/tex">\n<script type="math/tex; mode=display">', '<script type="math/tex; mode=display">')
|
||||
content = content.replace('</script>\n</script>', '</script>')
|
||||
return content
|
||||
|
||||
|
||||
|
||||
if ('$' in txt) and ('```' not in txt): # 有$标识的公式符号,且没有代码段```的标识
|
||||
# convert everything to html format
|
||||
@@ -248,7 +286,7 @@ def markdown_convertion(txt):
|
||||
def close_up_code_segment_during_stream(gpt_reply):
|
||||
"""
|
||||
在gpt输出代码的中途(输出了前面的```,但还没输出完后面的```),补上后面的```
|
||||
|
||||
|
||||
Args:
|
||||
gpt_reply (str): GPT模型返回的回复字符串。
|
||||
|
||||
@@ -369,6 +407,9 @@ def find_recent_files(directory):
|
||||
|
||||
|
||||
def on_file_uploaded(files, chatbot, txt, txt2, checkboxes):
|
||||
"""
|
||||
当文件被上传时的回调函数
|
||||
"""
|
||||
if len(files) == 0:
|
||||
return chatbot, txt
|
||||
import shutil
|
||||
@@ -388,8 +429,7 @@ def on_file_uploaded(files, chatbot, txt, txt2, checkboxes):
|
||||
shutil.copy(file.name, f'private_upload/{time_tag}/{file_origin_name}')
|
||||
err_msg += extract_archive(f'private_upload/{time_tag}/{file_origin_name}',
|
||||
dest_dir=f'private_upload/{time_tag}/{file_origin_name}.extract')
|
||||
moved_files = [fp for fp in glob.glob(
|
||||
'private_upload/**/*', recursive=True)]
|
||||
moved_files = [fp for fp in glob.glob('private_upload/**/*', recursive=True)]
|
||||
if "底部输入区" in checkboxes:
|
||||
txt = ""
|
||||
txt2 = f'private_upload/{time_tag}'
|
||||
@@ -432,6 +472,19 @@ def is_any_api_key(key):
|
||||
else:
|
||||
return is_openai_api_key(key) or is_api2d_key(key)
|
||||
|
||||
def what_keys(keys):
|
||||
avail_key_list = {'OpenAI Key':0, "API2D Key":0}
|
||||
key_list = keys.split(',')
|
||||
|
||||
for k in key_list:
|
||||
if is_openai_api_key(k):
|
||||
avail_key_list['OpenAI Key'] += 1
|
||||
|
||||
for k in key_list:
|
||||
if is_api2d_key(k):
|
||||
avail_key_list['API2D Key'] += 1
|
||||
|
||||
return f"检测到: OpenAI Key {avail_key_list['OpenAI Key']} 个,API2D Key {avail_key_list['API2D Key']} 个"
|
||||
|
||||
def select_api_key(keys, llm_model):
|
||||
import random
|
||||
@@ -447,20 +500,22 @@ def select_api_key(keys, llm_model):
|
||||
if is_api2d_key(k): avail_key_list.append(k)
|
||||
|
||||
if len(avail_key_list) == 0:
|
||||
raise RuntimeError(f"您提供的api-key不满足要求,不包含任何可用于{llm_model}的api-key。")
|
||||
raise RuntimeError(f"您提供的api-key不满足要求,不包含任何可用于{llm_model}的api-key。您可能选择了错误的模型或请求源。")
|
||||
|
||||
api_key = random.choice(avail_key_list) # 随机负载均衡
|
||||
return api_key
|
||||
|
||||
@lru_cache(maxsize=128)
|
||||
def read_single_conf_with_lru_cache(arg):
|
||||
from colorful import print亮红, print亮绿
|
||||
from colorful import print亮红, print亮绿, print亮蓝
|
||||
try:
|
||||
r = getattr(importlib.import_module('config_private'), arg)
|
||||
except:
|
||||
r = getattr(importlib.import_module('config'), arg)
|
||||
# 在读取API_KEY时,检查一下是不是忘了改config
|
||||
if arg == 'API_KEY':
|
||||
print亮蓝(f"[API_KEY] 本项目现已支持OpenAI和API2D的api-key。也支持同时填写多个api-key,如API_KEY=\"openai-key1,openai-key2,api2d-key3\"")
|
||||
print亮蓝(f"[API_KEY] 您既可以在config.py中修改api-key(s),也可以在问题输入区输入临时的api-key(s),然后回车键提交后即可生效。")
|
||||
if is_any_api_key(r):
|
||||
print亮绿(f"[API_KEY] 您的 API_KEY 是: {r[:15]}*** API_KEY 导入成功")
|
||||
else:
|
||||
@@ -493,10 +548,10 @@ def clear_line_break(txt):
|
||||
class DummyWith():
|
||||
"""
|
||||
这段代码定义了一个名为DummyWith的空上下文管理器,
|
||||
它的作用是……额……没用,即在代码结构不变得情况下取代其他的上下文管理器。
|
||||
它的作用是……额……就是不起作用,即在代码结构不变得情况下取代其他的上下文管理器。
|
||||
上下文管理器是一种Python对象,用于与with语句一起使用,
|
||||
以确保一些资源在代码块执行期间得到正确的初始化和清理。
|
||||
上下文管理器必须实现两个方法,分别为 __enter__()和 __exit__()。
|
||||
上下文管理器必须实现两个方法,分别为 __enter__()和 __exit__()。
|
||||
在上下文执行开始的情况下,__enter__()方法会在代码块被执行前被调用,
|
||||
而在上下文执行结束时,__exit__()方法则会被调用。
|
||||
"""
|
||||
@@ -505,3 +560,86 @@ class DummyWith():
|
||||
|
||||
def __exit__(self, exc_type, exc_value, traceback):
|
||||
return
|
||||
|
||||
def run_gradio_in_subpath(demo, auth, port, custom_path):
|
||||
"""
|
||||
把gradio的运行地址更改到指定的二次路径上
|
||||
"""
|
||||
def is_path_legal(path: str)->bool:
|
||||
'''
|
||||
check path for sub url
|
||||
path: path to check
|
||||
return value: do sub url wrap
|
||||
'''
|
||||
if path == "/": return True
|
||||
if len(path) == 0:
|
||||
print("ilegal custom path: {}\npath must not be empty\ndeploy on root url".format(path))
|
||||
return False
|
||||
if path[0] == '/':
|
||||
if path[1] != '/':
|
||||
print("deploy on sub-path {}".format(path))
|
||||
return True
|
||||
return False
|
||||
print("ilegal custom path: {}\npath should begin with \'/\'\ndeploy on root url".format(path))
|
||||
return False
|
||||
|
||||
if not is_path_legal(custom_path): raise RuntimeError('Ilegal custom path')
|
||||
import uvicorn
|
||||
import gradio as gr
|
||||
from fastapi import FastAPI
|
||||
app = FastAPI()
|
||||
if custom_path != "/":
|
||||
@app.get("/")
|
||||
def read_main():
|
||||
return {"message": f"Gradio is running at: {custom_path}"}
|
||||
app = gr.mount_gradio_app(app, demo, path=custom_path)
|
||||
uvicorn.run(app, host="0.0.0.0", port=port) # , auth=auth
|
||||
|
||||
|
||||
def clip_history(inputs, history, tokenizer, max_token_limit):
|
||||
"""
|
||||
reduce the length of history by clipping.
|
||||
this function search for the longest entries to clip, little by little,
|
||||
until the number of token of history is reduced under threshold.
|
||||
通过裁剪来缩短历史记录的长度。
|
||||
此函数逐渐地搜索最长的条目进行剪辑,
|
||||
直到历史记录的标记数量降低到阈值以下。
|
||||
"""
|
||||
import numpy as np
|
||||
from request_llm.bridge_all import model_info
|
||||
def get_token_num(txt):
|
||||
return len(tokenizer.encode(txt, disallowed_special=()))
|
||||
input_token_num = get_token_num(inputs)
|
||||
if input_token_num < max_token_limit * 3 / 4:
|
||||
# 当输入部分的token占比小于限制的3/4时,裁剪时
|
||||
# 1. 把input的余量留出来
|
||||
max_token_limit = max_token_limit - input_token_num
|
||||
# 2. 把输出用的余量留出来
|
||||
max_token_limit = max_token_limit - 128
|
||||
# 3. 如果余量太小了,直接清除历史
|
||||
if max_token_limit < 128:
|
||||
history = []
|
||||
return history
|
||||
else:
|
||||
# 当输入部分的token占比 > 限制的3/4时,直接清除历史
|
||||
history = []
|
||||
return history
|
||||
|
||||
everything = ['']
|
||||
everything.extend(history)
|
||||
n_token = get_token_num('\n'.join(everything))
|
||||
everything_token = [get_token_num(e) for e in everything]
|
||||
|
||||
# 截断时的颗粒度
|
||||
delta = max(everything_token) // 16
|
||||
|
||||
while n_token > max_token_limit:
|
||||
where = np.argmax(everything_token)
|
||||
encoded = tokenizer.encode(everything[where], disallowed_special=())
|
||||
clipped_encoded = encoded[:len(encoded)-delta]
|
||||
everything[where] = tokenizer.decode(clipped_encoded)[:-1] # -1 to remove the may-be illegal char
|
||||
everything_token[where] = get_token_num(everything[where])
|
||||
n_token = get_token_num('\n'.join(everything))
|
||||
|
||||
history = everything[1:]
|
||||
return history
|
||||
|
||||
4
version
4
version
@@ -1,5 +1,5 @@
|
||||
{
|
||||
"version": 3.1,
|
||||
"version": 3.3,
|
||||
"show_feature": true,
|
||||
"new_feature": "添加支持清华ChatGLM和GPT-4 <-> 改进架构,支持与多个LLM模型同时对话 <-> 添加支持API2D(国内,可支持gpt4)<-> 支持多API-KEY负载均衡(并列填写,逗号分割) <-> 添加输入区文本清除按键"
|
||||
"new_feature": "支持NewBing !! <-> 保存对话功能 <-> 解读任意语言代码+同时询问任意的LLM组合 <-> 添加联网(Google)回答问题插件 <-> 修复ChatGLM上下文BUG <-> 添加支持清华ChatGLM和GPT-4 <-> 改进架构,支持与多个LLM模型同时对话 <-> 添加支持API2D(国内,可支持gpt4)"
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user