{"id":1893,"date":"2024-06-10T22:33:28","date_gmt":"2024-06-10T14:33:28","guid":{"rendered":"http:\/\/www.puzi7.com\/?p=1893"},"modified":"2026-01-27T14:47:29","modified_gmt":"2026-01-27T06:47:29","slug":"langchain-chatchatllm%e6%9c%ac%e5%9c%b0%e7%9f%a5%e8%af%86%e5%ba%93","status":"publish","type":"post","link":"https:\/\/www.puzi7.net\/?p=1893","title":{"rendered":"Langchain-Chatchat+LLM+\u672c\u5730\u77e5\u8bc6\u5e93"},"content":{"rendered":"<span itemprop=\"description\">\n<p class=\"wp-block-paragraph\">\u5728LLM\u5927\u6a21\u578b\u7684\u5b9e\u8df5\u5e94\u7528\u4e2d\uff0c\u901a\u8fc7LLM\u6a21\u578b\u5bf9\u672c\u5730\u77e5\u8bc6\u5e93\u505a\u7ba1\u7406\u548c\u5185\u5bb9\u63d0\u53d6\u95ee\u7b54\u662f\u975e\u5e38\u5408\u7406\u4e14\u6709\u4ef7\u503c\u7684\u5e94\u7528\u573a\u666f\u3002\u672c\u6587\u5c06\u7ed3\u5408\u81ea\u5df1\u5728\u5b66\u4e60Langchain\u53caLangchain-Chatchat\u8fc7\u7a0b\u4e2d\u7684\u5b9e\u8df5,\u505a\u5982\u4e0b\u8bb0\u5f55\uff1a<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u4e3b\u8981\u5206\u4ee5\u4e0b\u4e09\u90e8\u5206\uff1a<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>\u73af\u5883\u51c6\u5907\n<ul class=\"wp-block-list\">\n<li>ubuntu 22.04, python 3.11, conda<\/li>\n\n\n\n<li>cuda 12.2 + GPU\uff1b\u672c\u6587\u4f7f\u7528Nvidia L40\u505a\u6d4b\u8bd5<\/li>\n\n\n\n<li>\u6a21\u578b\u4e0b\u8f7d<\/li>\n<\/ul>\n<\/li>\n\n\n\n<li>\u5b89\u88c5\u90e8\u7f72\n<ul class=\"wp-block-list\">\n<li>\u514b\u9686langchainchatchat\u4ee3\u7801<\/li>\n\n\n\n<li>\u5b89\u88c5\u4f9d\u8d56<\/li>\n\n\n\n<li>\u6570\u636e\u548c\u914d\u7f6e\u521d\u59cb\u5316<\/li>\n\n\n\n<li>\u542f\u52a8\u5e94\u7528<\/li>\n<\/ul>\n<\/li>\n\n\n\n<li>\u529f\u80fd\u9a8c\u8bc1\u6d4b\u8bd5\n<ul class=\"wp-block-list\">\n<li>\u6253\u5f00\u90e8\u7f72\u597d\u7684\u5e94\u7528\u5730\u5740 http:\/\/localhost:8501<\/li>\n\n\n\n<li>\u57fa\u4e8echatglm3-6b\u7684\u5bf9\u8bdd\u9a8c\u8bc1<\/li>\n\n\n\n<li>\u4e0a\u4f20\u6587\u6863\u7684\u95ee\u7b54<\/li>\n\n\n\n<li>\u57fa\u4e8e\u77e5\u8bc6\u5e93\u7684\u6587\u6863<\/li>\n<\/ul>\n<\/li>\n\n\n\n<li>\u5b9e\u8df5\u4e2d\u9047\u5230\u7684\u95ee\u9898<\/li>\n<\/ol>\n\n\n\n<h4><strong>\u73af\u5883\u51c6\u5907<\/strong><\/h4>\n<p>\u5b89\u88c5anaconda3 for linux<\/p>\n<p>\u4f7f\u7528conda create -n py311 python=3.11<\/p>\n<p>conda activate py311<\/p>\n<p>\u4f7f\u7528nvidia-smi\u786e\u8ba4GPU\u662f\u53ef\u4ee5\u7684<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-1897\" src=\"http:\/\/42.192.251.135:8080\/wp-content\/uploads\/2024\/06\/Screenshot-2024-06-10-220641.png\" alt=\"\" width=\"973\" height=\"287\" \/><\/p>\n<p>\u6a21\u578b\u4e0b\u8f7d\uff0c\u4f7f\u7528<a href=\"https:\/\/www.modelscope.cn\/models\/AI-ModelScope\/bge-large-zh\/files\">bge-large-zh \u00b7 \u6a21\u578b\u5e93 (modelscope.cn)<\/a><\/p>\n<blockquote>\n<p>vim model_download.py<\/p>\n<p>#\u6a21\u578b\u4e0b\u8f7d from modelscope import snapshot_download<\/p>\n<p>model_dir = snapshot_download(&#8216;AI-ModelScope\/bge-large-zh&#8217;)<\/p>\n<p>model_dir = snapshot_download(&#8216;ZhipuAI\/chatglm3-6b&#8217;)<\/p>\n<p>python model_download.py\u00a0\u00a0<\/p>\n<\/blockquote>\n<p>\u6a21\u578b\u4e0b\u8f7d\u5230\/root\/.cache\/modelscope\/hub\u76ee\u5f55<\/p>\n<p>\u9a8c\u8bc1\u6a21\u578b\u52a0\u8f7d\uff1a<\/p>\n<blockquote>\n<div>import torch<\/div>\n<div>from langchain.chains import LLMChain<\/div>\n<div>from langchain.text_splitter import CharacterTextSplitter<\/div>\n<div>from langchain.chains.mapreduce import MapReduceChain<\/div>\n<div>from langchain.prompts import PromptTemplate<\/div>\n<div>from langchain.llms.base import LLM<\/div>\n<div>from transformers import AutoTokenizer, AutoModel, AutoConfig<\/div>\n<div>from typing import Any, Dict, List, Mapping, Optional, Tuple, Union<\/div>\n<div>from torch.mps import empty_cache<\/div>\n<div>from transformers import AutoModelForCausalLM, AutoTokenizer<\/div>\n<div>from transformers.generation.utils import GenerationConfig<\/div>\n<br \/>\n<div>class GLM(LLM):<\/div>\n<div>\u00a0 \u00a0 max_token: int = 2048<\/div>\n<div>\u00a0 \u00a0 temperature: float = 0.8<\/div>\n<div>\u00a0 \u00a0 top_p = 0.9<\/div>\n<div>\u00a0 \u00a0 tokenizer: object = None<\/div>\n<div>\u00a0 \u00a0 model: object = None<\/div>\n<div>\u00a0 \u00a0 history_len: int = 1024<\/div>\n<div>\u00a0 \u00a0<\/div>\n<div>\u00a0 \u00a0 def __init__(self):<\/div>\n<div>\u00a0 \u00a0 \u00a0 \u00a0 super().__init__()<\/div>\n<div>\u00a0 \u00a0 \u00a0 \u00a0<\/div>\n<div>\u00a0 \u00a0 @property<\/div>\n<div>\u00a0 \u00a0 def _llm_type(self) -&gt; str:<\/div>\n<div>\u00a0 \u00a0 \u00a0 \u00a0 return &#8220;GLM&#8221;<\/div>\n<div>\u00a0 \u00a0 \u00a0 \u00a0\u00a0<\/div>\n<div># \u4f20\u5165\u6a21\u578b\u7684\u8def\u5f84<\/div>\n<div>\u00a0 \u00a0 def <strong>load_model(self, llm_device=&#8221;gpu&#8221;,model_name_or_path=None):<\/strong><\/div>\n<div>\u00a0 \u00a0 \u00a0 \u00a0 #tokenizer = AutoTokenizer.from_pretrained(&#8220;.\/chatglm2-6b&#8221;, trust_remote_code=True)<\/div>\n<div>\u00a0 \u00a0 \u00a0 \u00a0 #model = AutoModel.from_pretrained(&#8220;.\/chatglm2-6b&#8221;, trust_remote_code=True).cuda()<\/div>\n<div>\u00a0 \u00a0 \u00a0 \u00a0 model_config = AutoConfig.from_pretrained(model_name_or_path,trust_remote_code=True)<\/div>\n<div>\u00a0 \u00a0 \u00a0 \u00a0 self.tokenizer = AutoTokenizer.from_pretrained(model_name_or_path,trust_remote_code=True)<\/div>\n<div>\u00a0 \u00a0 \u00a0 \u00a0 self.model = AutoModel.from_pretrained(model_name_or_path, torch_dtype=torch.float16, config=model_config, trust_remote_code=True).half().cuda()<\/div>\n<div>\u00a0 \u00a0 \u00a0 \u00a0<\/div>\n<div>\u00a0 \u00a0 def _call(self,prompt:str,history:List[str] = [],stop: Optional[List[str]] = None):<\/div>\n<div>\u00a0 \u00a0 \u00a0 \u00a0 response, _ = self.model.chat(<\/div>\n<div>\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 self.tokenizer,prompt,<\/div>\n<div>\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 history=history[-self.history_len:] if self.history_len &gt; 0 else [],<\/div>\n<div>\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 max_length=self.max_token,temperature=self.temperature,<\/div>\n<div>\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 top_p=self.top_p)<\/div>\n<div>\u00a0 \u00a0 \u00a0 \u00a0 return response<\/div>\n<\/blockquote>\n<div>\u7ed3\u679c\uff1a<img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-1898\" src=\"http:\/\/42.192.251.135:8080\/wp-content\/uploads\/2024\/06\/Screenshot-2024-06-10-221842.png\" alt=\"\" width=\"2222\" height=\"430\" \/><\/div>\n<div>\u00a0<\/div>\n<div>\u00a0<\/div>\n<p>\u00a0<\/p>\n\n\n\n<h4>\u5b89\u88c5\u90e8\u7f72<\/h4>\n<ol>\n<li>git clone https:\/\/github.com\/chatchat-space\/Langchain-Chatchat.git<\/li>\n<li>\u6309\u7167READEME.md\n<ol>\n<li>pip install -r requirements.txt<\/li>\n<li>pip install -r requirements_api.txt<\/li>\n<li>pip install -r requirements_webui.txt<\/li>\n<li>pip install jq<\/li>\n<li>\n<div>&#8220;`shell<\/div>\n<div>$ python copy_config_example.py<\/div>\n<div>$ python init_database.py &#8211;recreate-vs<\/div>\n<div>\u00a0&#8220;`<\/div>\n<\/li>\n<li>\u4fee\u6539configs\/model_config.py\n<ol>\n<li>EMBEDDING_MODEL = &#8220;bge-large-zh&#8221;<\/li>\n<li>model path\/embed_model:\u00a0 \u00a0 &#8220;bge-large-zh&#8221;: &#8220;\/xxxxxx\/BAAI\/bge-large-zh&#8221;,<\/li>\n<li>model_path\/llm_model: &#8220;chatglm3-6b&#8221;: &#8220;\/xxxxxxx\/THUDM\/chatglm3-6b&#8221;,<\/li>\n<\/ol>\n<\/li>\n<li>start up<\/li>\n<li>\n<div>&#8220;`shell<\/div>\n<div>$ python startup.py -a<\/div>\n<div>&#8220;`<\/div>\n<div>\u5982\u679c\u6b63\u5e38\u542f\u52a8\u5219\u53ef\u4ee5\u8bbf\u95eehttp:\/\/localhost:8501 \u9875\u9762<\/div>\n<\/li>\n<li>\u5982\u56fe\u6240\u793a<img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-1899\" src=\"http:\/\/42.192.251.135:8080\/wp-content\/uploads\/2024\/06\/Screenshot-2024-06-10-222614.png\" alt=\"\" width=\"2013\" height=\"1297\" \/><\/li>\n<\/ol>\n<\/li>\n<\/ol>\n\n\n\n<h4>\u529f\u80fd\u9a8c\u8bc1<\/h4>\n<p>\u53ef\u4ee5\u4e0a\u4f20pdf\u6587\u4ef6\u505a\u6587\u4ef6\u6587\u6863<\/p>\n<p>\u4e5f\u53ef\u4ee5\u901a\u8fc7\u77e5\u8bc6\u5e93\u7ba1\u7406\uff0c\u521b\u5efa\u81ea\u5df1\u7684\u77e5\u8bc6\u5e93\uff0c\u7136\u540e\u57fa\u4e8e\u81ea\u5df1\u7684\u77e5\u8bc6\u5e93\u505a\u6587\u6863<\/p>\n<p>\u4e5f\u53ef\u4ee5\u57fa\u4e8echatglm3-6b\u6a21\u578b\u505a\u95ee\u7b54<\/p>\n<p>\u00a0<\/p>\n<h4>\u90e8\u7f72\u8fc7\u7a0b\u9047\u5230\u7684\u95ee\u9898\uff1a<\/h4>\n<p>\u65e0\u6cd5\u52a0\u8f7d\u6a21\u578b\uff0c\u603b\u662f\u5728model_worker\u8fd9\u91cc\u51fa\u73b0\u95ee\u9898\uff0c\u540e\u6765\u53d1\u73b0\u662fserver 127.0.0.1\u65e0\u6cd5\u542f\u52a8\uff0c\u539f\u56e0\u662f\u4f7f\u7528\u4e86\u4ee3\u7406\u5bfc\u81f4\u672c\u5730\u670d\u52a1\u65e0\u6cd5\u901a\u8fc7localhost\u6216\u8005127.0.0.1\u8c03\u7528\uff1b<\/p>\n<p>\u00a0<\/p>\n<span>","protected":false},"excerpt":{"rendered":"<p class=\"excerpt\"><span itemprop=\"description\">\u5728LLM\u5927\u6a21\u578b\u7684\u5b9e\u8df5\u5e94\u7528\u4e2d\uff0c\u901a\u8fc7LLM\u6a21\u578b\u5bf9\u672c\u5730\u77e5\u8bc6\u5e93\u505a\u7ba1\u7406\u548c\u5185\u5bb9\u63d0\u53d6\u95ee\u7b54\u662f\u975e\u5e38\u5408\u7406\u4e14\u6709\u4ef7&hellip;<\/span><\/p>\n<p class=\"more-link-p btn-align-right\"><a class=\"green zoom-btn\" href=\"https:\/\/www.puzi7.net\/?p=1893\">Read More<\/a><\/p>\n","protected":false},"author":1,"featured_media":1987,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[17,25,9],"tags":[40,83,86],"class_list":["post-1893","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai","category-llm","category-9","tag-ai","tag-langchain","tag-llm","zoom-theme-has-thumb"],"_links":{"self":[{"href":"https:\/\/www.puzi7.net\/index.php?rest_route=\/wp\/v2\/posts\/1893","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.puzi7.net\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.puzi7.net\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.puzi7.net\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.puzi7.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1893"}],"version-history":[{"count":1,"href":"https:\/\/www.puzi7.net\/index.php?rest_route=\/wp\/v2\/posts\/1893\/revisions"}],"predecessor-version":[{"id":1988,"href":"https:\/\/www.puzi7.net\/index.php?rest_route=\/wp\/v2\/posts\/1893\/revisions\/1988"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.puzi7.net\/index.php?rest_route=\/wp\/v2\/media\/1987"}],"wp:attachment":[{"href":"https:\/\/www.puzi7.net\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1893"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.puzi7.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=1893"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.puzi7.net\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=1893"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}