{"id":9765,"date":"2024-11-25T18:10:54","date_gmt":"2024-11-25T10:10:54","guid":{"rendered":"http:\/\/www.yliyun.com\/?p=9765"},"modified":"2024-11-25T18:10:54","modified_gmt":"2024-11-25T10:10:54","slug":"%e4%bd%bf%e7%94%a8ragflowitext2kg%e9%92%88%e5%af%b9%e6%96%87%e6%a1%a3%e8%bf%9b%e8%a1%8c%e6%90%9c%e7%b4%a2%e4%b8%8e%e7%9f%a5%e8%af%86%e5%9b%be%e7%89%87%e7%94%9f%e6%88%90%ef%bc%88%e4%b8%80%ef%bc%89","status":"publish","type":"post","link":"http:\/\/www.yliyun.com\/2024\/11\/25\/%e4%bd%bf%e7%94%a8ragflowitext2kg%e9%92%88%e5%af%b9%e6%96%87%e6%a1%a3%e8%bf%9b%e8%a1%8c%e6%90%9c%e7%b4%a2%e4%b8%8e%e7%9f%a5%e8%af%86%e5%9b%be%e7%89%87%e7%94%9f%e6%88%90%ef%bc%88%e4%b8%80%ef%bc%89\/","title":{"rendered":"\u4f7f\u7528RAGFlow+iText2KG\u9488\u5bf9\u6587\u6863\u8fdb\u884c\u641c\u7d22\u4e0e\u77e5\u8bc6\u56fe\u7247\u751f\u6210\uff08\u4e00\uff09"},"content":{"rendered":"\n<p class=\"has-large-font-size\">1\u3001RAGFlow \u7684\u4f7f\u7528\u6307\u5357<\/p>\n\n\n\n<p>RAGFlow \u662f\u4e00\u4e2a\u57fa\u4e8e\u5bf9\u6587\u6863\u7684\u6df1\u5165\u7406\u89e3\u7684\u5f00\u6e90 RAG\uff08\u68c0\u7d22\u589e\u5f3a\u751f\u6210\uff09\u5f15\u64ce\u3002\u5b83\u4e3a\u4efb\u4f55\u89c4\u6a21\u7684\u4f01\u4e1a\u63d0\u4f9b\u4e86\u7b80\u5316\u7684 RAG \u5de5\u4f5c\u6d41\u7a0b\uff0c\u7ed3\u5408\u4e86 LLM\uff08\u5927\u578b\u8bed\u8a00\u6a21\u578b\uff09\u4ee5\u63d0\u4f9b\u771f\u5b9e\u7684\u95ee\u7b54\u529f\u80fd\uff0c\u5e76\u4ee5\u6765\u81ea\u5404\u79cd\u590d\u6742\u683c\u5f0f\u6570\u636e\u7684\u6709\u6839\u636e\u7684\u5f15\u6587\u4e3a\u540e\u76fe\u3002<\/p>\n\n\n\n<p>demo\u94fe\u63a5\uff1aRAGFlow<\/p>\n\n\n\n<p>\u7279\u70b9\uff1a<\/p>\n\n\n\n<p>1\u3001\u6709\u4e00\u5b9a\u7684\u6570\u636e\u8d28\u91cf\u4fdd\u8bc1\uff0c\u80fd\u4ece\u590d\u6742\u7684\u975e\u7ed3\u6784\u5316\u6570\u636e\u4e2d\u63d0\u53d6\u57fa\u4e8e\u6587\u6863\u7406\u89e3\u7684\u6df1\u5ea6\u77e5\u8bc6\u3002<\/p>\n\n\n\n<p>2\u3001\u5185\u7f6e\u6a21\u677f\uff0c\u53ef\u4ee5\u57fa\u4e8e\u6a21\u677f\u5f62\u6210\u77e5\u8bc6\u5e93\uff1b\u6587\u6863\u5206\u5757\u53ef\u4ee5\u5b9e\u73b0\u4eba\u5de5\u5e72\u9884\uff0c\u63d0\u9ad8\u6587\u6863\u8d28\u91cf\uff1b<\/p>\n\n\n\n<p>3\u3001\u53ef\u4ee5\u517c\u5bb9\u5f02\u6784\u6570\u636e\u6e90\uff0c\u652f\u6301 Word\u3001\u5e7b\u706f\u7247\u3001excel\u3001txt\u3001\u56fe\u50cf\u3001\u626b\u63cf\u526f\u672c\u3001\u7ed3\u6784\u5316\u6570\u636e\u3001\u7f51\u9875\u7b49\u3002<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">4\u3001&nbsp;\u81ea\u52a8\u5316\u4e14\u8f7b\u677e\u7684 RAG \u5de5\u4f5c\u6d41\u7a0b<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>\u7b80\u5316\u7684 RAG \u7f16\u6392\u540c\u65f6\u6ee1\u8db3\u4e86\u4e2a\u4eba\u548c\u5927\u578b\u4f01\u4e1a\u7684\u9700\u6c42\u3002<\/li>\n\n\n\n<li>\u53ef\u914d\u7f6e\u7684 LLM \u4ee5\u53ca\u5d4c\u5165\u6a21\u578b\u3002<\/li>\n\n\n\n<li>\u591a\u91cd\u53ec\u56de\u4e0e\u878d\u5408\u7684\u91cd\u65b0\u6392\u540d\u914d\u5bf9\u3002<\/li>\n\n\n\n<li>\u76f4\u89c2\u7684 API\uff0c\u53ef\u4e0e\u4e1a\u52a1\u65e0\u7f1d\u96c6\u6210\u3002<br>RAGFlow\u67b6\u6784\u56fe<\/li>\n<\/ul>\n\n\n\n<p>\u90e8\u7f72\u8981\u6c42\uff1a<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>CPU >= 4 cores<\/li>\n\n\n\n<li>RAM >= 16 GB<\/li>\n\n\n\n<li>Disk >= 50 GB<\/li>\n\n\n\n<li>Docker >= 24.0.0 &amp; Docker Compose >= v2.26.1<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">\u542f\u52a8 \u670d\u52a1\u5668<\/h3>\n\n\n\n<ol class=\"wp-block-list\">\n<li>\u786e\u4fdd >= 262144\uff1a<code>vm.max_map_count<\/code>\u8981\u68c0\u67e5 \u7684\u503c \uff1a<code>vm.max_map_count<\/code>$ sysctl vm.max_map_count\u5982\u679c\u4e0d\u662f\uff0c\u5219\u91cd\u7f6e\u4e3a\u81f3\u5c11 262144 \u7684\u503c\u3002<code>vm.max_map_count<\/code><\/li>\n<\/ol>\n\n\n\n<ol class=\"wp-block-list\">\n<li># In this case, we set it to 262144:$ sudo sysctl -w vm.max_map_count=262144\u6b64\u66f4\u6539\u5c06\u5728\u7cfb\u7edf\u91cd\u542f\u540e\u91cd\u7f6e\u3002\u4e3a\u4e86\u786e\u4fdd\u60a8\u7684\u66f4\u6539\u4fdd\u6301\u6c38\u4e45\uff0c\u8bf7\u76f8\u5e94\u5730\u5728\u00a0\/etc\/sysctl.conf\u00a0\u4e2d\u6dfb\u52a0\u6216\u66f4\u65b0\u8be5\u503c\uff1a<code>vm.max_map_count<\/code><\/li>\n\n\n\n<li>vm.max_map_count=262144<\/li>\n\n\n\n<li>\u514b\u9686\u5b58\u50a8\u5e93\uff1a$ git clone https:\/\/github.com\/infiniflow\/ragflow.git\u6784\u5efa\u9884\u6784\u5efa\u7684 Docker \u955c\u50cf\u5e76\u542f\u52a8\u670d\u52a1\u5668\uff1a<\/li>\n\n\n\n<li>\u4ee5\u4e0b\u547d\u4ee4\u4e0b\u8f7d RAGFlow slim \uff08\uff09 \u7684\u5f00\u53d1\u7248\u672c Docker \u6620\u50cf\u3002\u8bf7\u6ce8\u610f\uff0cRAGFlow slim Docker \u6620\u50cf\u4e0d\u5305\u62ec\u5d4c\u5165\u6a21\u578b\u6216 Python \u5e93\uff0c\u56e0\u6b64\u5927\u5c0f\u7ea6\u4e3a 1GB\u3002<code>dev-slim<\/code>$ cd ragflow\/docker$ docker compose -f docker-compose.yml up -d\u6ce8\u610f\uff1a\u5305\u542b\u5d4c\u5165\u6a21\u578b\u548c Python \u5e93\u7684 RAGFlow Docker \u6620\u50cf\u7684\u5927\u5c0f\u7ea6\u4e3a 9GB\uff0c\u52a0\u8f7d\u65f6\u95f4\u53ef\u80fd\u8981\u957f\u5f97\u591a\u3002\n<ul class=\"wp-block-list\">\n<li>\u8981\u4e0b\u8f7d\u7279\u5b9a\u7248\u672c\u7684 RAGFlow slim Docker \u955c\u50cf\uff0c\u8bf7\u5c06\u00a0docker\/.env\u00a0\u4e2d\u7684\u53d8\u91cf\u66f4\u65b0\u4e3a\u6240\u9700\u7248\u672c\u3002\u4f8b\u5982\u3002\u8fdb\u884c\u6b64\u66f4\u6539\u540e\uff0c\u8bf7\u91cd\u65b0\u8fd0\u884c\u4e0a\u8ff0\u547d\u4ee4\u4ee5\u542f\u52a8\u4e0b\u8f7d\u3002<code>RAGFlow_IMAGE<\/code><code>RAGFLOW_IMAGE=infiniflow\/ragflow:v0.12.0-slim<\/code><\/li>\n\n\n\n<li>\u8981\u4e0b\u8f7d RAGFlow Docker \u6620\u50cf\u7684\u5f00\u53d1\u7248\u672c<em>\uff08\u5305\u62ec<\/em>\u5d4c\u5165\u6a21\u578b\u548c Python \u5e93\uff09\uff0c\u8bf7\u5c06\u00a0docker\/.env\u00a0\u4e2d\u7684\u53d8\u91cf\u66f4\u65b0\u4e3a \u3002\u8fdb\u884c\u6b64\u66f4\u6539\u540e\uff0c\u8bf7\u91cd\u65b0\u8fd0\u884c\u4e0a\u8ff0\u547d\u4ee4\u4ee5\u542f\u52a8\u4e0b\u8f7d\u3002<code>RAGFlow_IMAGE<\/code><code>RAGFLOW_IMAGE=infiniflow\/ragflow:dev<\/code><\/li>\n\n\n\n<li>\u8981\u4e0b\u8f7d\u7279\u5b9a\u7248\u672c\u7684 RAGFlow Docker \u6620\u50cf<em>\uff08\u5305\u62ec<\/em>\u5d4c\u5165\u6a21\u578b\u548c Python \u5e93\uff09\uff0c\u8bf7\u5c06\u00a0docker\/.env\u00a0\u4e2d\u7684\u53d8\u91cf\u66f4\u65b0\u4e3a\u6240\u9700\u7684\u7248\u672c\u3002\u4f8b\u5982\u3002\u8fdb\u884c\u6b64\u66f4\u6539\u540e\uff0c\u8bf7\u91cd\u65b0\u8fd0\u884c\u4e0a\u8ff0\u547d\u4ee4\u4ee5\u542f\u52a8\u4e0b\u8f7d\u3002<code>RAGFlow_IMAGE<\/code><code>RAGFLOW_IMAGE=infiniflow\/ragflow:v0.12.0<\/code><\/li>\n<\/ul>\n<\/li>\n\n\n\n<li>\u5728\u670d\u52a1\u5668\u542f\u52a8\u5e76\u8fd0\u884c\u540e\u68c0\u67e5\u670d\u52a1\u5668\u72b6\u6001\uff1a$ docker logs -f ragflow-server<em>\u4ee5\u4e0b\u8f93\u51fa\u786e\u8ba4\u7cfb\u7edf\u5df2\u6210\u529f\u542f\u52a8\uff1a<\/em><\/li>\n\n\n\n<li>____ ___ ______ ______ __<br>\/ __ \\ \/ | \/ ____\/\/ ____\/\/ \/____ _ __<br>\/ \/_\/ \/\/ \/| | \/ \/ __ \/ \/_ \/ \/\/ __ \\| | \/| \/ \/<br>\/ _, _\/\/ ___ |\/ \/_\/ \/\/ __\/ \/ \/\/ \/_\/ \/| |\/ |\/ \/<br>\/_\/ |_|\/_\/ |_|\\____\/\/_\/ \/_\/ \\____\/ |__\/|__\/<br><br>* Running on all addresses (0.0.0.0)<br>* Running on http:\/\/127.0.0.1:9380<br>* Running on http:\/\/x.x.x.x:9380 INFO:werkzeug:Press CTRL+C to quit\u5982\u679c\u60a8\u8df3\u8fc7\u6b64\u786e\u8ba4\u6b65\u9aa4\u5e76\u76f4\u63a5\u767b\u5f55 RAGFlow\uff0c\u60a8\u7684\u6d4f\u89c8\u5668\u53ef\u80fd\u4f1a\u63d0\u793a\u9519\u8bef\uff0c\u56e0\u4e3a\u6b64\u65f6\u60a8\u7684 RAGFlow \u53ef\u80fd\u6ca1\u6709\u5b8c\u5168\u521d\u59cb\u5316\u3002<code>network abnormal<\/code><\/li>\n<\/ol>\n\n\n\n<ol class=\"wp-block-list\">\n<li>\u5728\u60a8\u7684 Web \u6d4f\u89c8\u5668\u4e2d\uff0c\u8f93\u5165\u670d\u52a1\u5668\u7684 IP \u5730\u5740\u5e76\u767b\u5f55 RAGFlow\u3002\u4f7f\u7528\u9ed8\u8ba4\u8bbe\u7f6e\u65f6\uff0c\u60a8\u53ea\u9700\u8f93\u5165 \uff08sans\u00a0port number\uff09 \u4f5c\u4e3a\u4f7f\u7528\u9ed8\u8ba4\u914d\u7f6e\u65f6\u53ef\u4ee5\u7701\u7565\u9ed8\u8ba4 HTTP \u670d\u52a1\u7aef\u53e3\u3002<code>http:\/\/IP_OF_YOUR_MACHINE<\/code><code>80<\/code><\/li>\n\n\n\n<li>\u5728 service_conf.yaml \u4e2d\uff0c\u9009\u62e9\u6240\u9700\u7684 LLM \u5de5\u5382\uff0c\u5e76\u4f7f\u7528\u76f8\u5e94\u7684 API \u5bc6\u94a5\u66f4\u65b0\u5b57\u6bb5\u3002<code>user_default_llm<\/code><code>API_KEY<\/code>\u6709\u5173\u66f4\u591a\u4fe1\u606f\uff0c\u8bf7\u53c2\u9605 llm_api_key_setup\u3002<\/li>\n<\/ol>\n\n\n\n<p>\u90e8\u7f72\u5b8c\u6210\u540e\uff0c\u8fd8\u9700\u8981\u5bf9RAGFlow\u8fdb\u884c\u914d\u7f6e\uff0c\u9700\u8981\u5173\u6ce8\u4ee5\u4e0b\u51e0\u70b9\uff1a<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>.env\uff1a\u4fdd\u7559\u7cfb\u7edf\u7684\u57fa\u672c\u8bbe\u7f6e\uff0c\u4f8b\u5982<code>SVR_HTTP_PORT<\/code><code>MYSQL_PASSWORD<\/code><code>MINIO_PASSWORD<\/code><\/li>\n\n\n\n<li>service_conf.yaml\uff1a\u914d\u7f6e\u540e\u7aef\u670d\u52a1\u3002<\/li>\n\n\n\n<li>docker-compose.yml\uff1a\u7cfb\u7edf\u4f9d\u8d56 docker-compose.yml \u542f\u52a8\u3002<\/li>\n<\/ul>\n\n\n\n<p>\u60a8\u5fc5\u987b\u786e\u4fdd\u5bf9 .env \u6587\u4ef6\u7684\u66f4\u6539\u4e0e service_conf.yaml \u6587\u4ef6\u4e2d\u7684\u66f4\u6539\u4e00\u81f4\u3002<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p>.\/docker\/README \u6587\u4ef6\u63d0\u4f9b\u4e86\u73af\u5883\u8bbe\u7f6e\u548c\u670d\u52a1\u914d\u7f6e\u7684\u8be6\u7ec6\u63cf\u8ff0\uff0c\u60a8\u9700\u8981\u786e\u4fdd .\/docker\/README \u6587\u4ef6\u4e2d\u5217\u51fa\u7684\u6240\u6709\u73af\u5883\u8bbe\u7f6e\u90fd\u4e0e service_conf.yaml \u6587\u4ef6\u4e2d\u7684\u76f8\u5e94\u914d\u7f6e\u4fdd\u6301\u4e00\u81f4\u3002<\/p>\n<\/blockquote>\n\n\n\n<p>\u8981\u66f4\u65b0\u9ed8\u8ba4 HTTP \u670d\u52a1\u7aef\u53e3 \uff0880\uff09\uff0c\u8bf7\u8f6c\u5230 docker-compose.yml \u5e76\u66f4\u6539\u4e3a \u3002<code>80:80<\/code><code>&lt;YOUR_SERVING_PORT&gt;:80<\/code><\/p>\n\n\n\n<p>\u5bf9\u4e0a\u8ff0\u914d\u7f6e\u7684\u66f4\u65b0\u9700\u8981\u91cd\u542f\u6240\u6709\u5bb9\u5668\u624d\u80fd\u751f\u6548\uff1a<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<pre class=\"wp-block-preformatted\">$ docker compose -f docker\/docker-compose.yml up -d<\/pre>\n<\/blockquote>\n\n\n\n<h2 class=\"wp-block-heading\">\ud83d\udd27 \u5728\u4e0d\u5d4c\u5165\u6a21\u578b\u7684\u60c5\u51b5\u4e0b\u6784\u5efa Docker \u955c\u50cf<\/h2>\n\n\n\n<p>\u6b64\u6620\u50cf\u7684\u5927\u5c0f\u7ea6\u4e3a 1 GB\uff0c\u4f9d\u8d56\u4e8e\u5916\u90e8 LLM \u548c\u5d4c\u5165\u670d\u52a1\u3002<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">git clone https:\/\/github.com\/infiniflow\/ragflow.git<br>cd ragflow\/<br>pip3 install huggingface-hub nltk<br>python3 download_deps.py<br>docker build -f Dockerfile.slim -t infiniflow\/ragflow:dev-slim .<\/pre>\n\n\n\n<h2 class=\"wp-block-heading\">\ud83d\udd27 \u6784\u5efa\u5305\u542b\u5d4c\u5165\u6a21\u578b\u7684 Docker \u955c\u50cf<\/h2>\n\n\n\n<p>\u6b64\u6620\u50cf\u7684\u5927\u5c0f\u7ea6\u4e3a 9 GB\u3002\u7531\u4e8e\u5b83\u5305\u62ec\u5d4c\u5165\u6a21\u578b\uff0c\u56e0\u6b64\u5b83\u4ec5\u4f9d\u8d56\u4e8e\u5916\u90e8 LLM \u670d\u52a1\u3002<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">git clone https:\/\/github.com\/infiniflow\/ragflow.git<br>cd ragflow\/<br>pip3 install huggingface-hub nltk<br>python3 download_deps.py<br>docker build -f Dockerfile -t infiniflow\/ragflow:dev .<\/pre>\n\n\n\n<h2 class=\"wp-block-heading\">\ud83d\udd28 \u4ece\u6e90\u5934\u542f\u52a8\u670d\u52a1\u8fdb\u884c\u5f00\u53d1<\/h2>\n\n\n\n<ol class=\"wp-block-list\">\n<li>\u5b89\u88c5 Poetry\uff0c\u5982\u679c\u5df2\u5b89\u88c5\uff0c\u8bf7\u8df3\u8fc7\u6b64\u6b65\u9aa4\uff1acurl -sSL https:\/\/install.python-poetry.org | python3 &#8211;<\/li>\n\n\n\n<li>\u514b\u9686\u6e90\u7801\u5e76\u5b89\u88c5 Python \u4f9d\u8d56\u9879\uff1agit clone https:\/\/github.com\/infiniflow\/ragflow.git<br>cd ragflow\/<br>export POETRY_VIRTUALENVS_CREATE=true POETRY_VIRTUALENVS_IN_PROJECT=true<br>~\/.local\/bin\/poetry install &#8211;sync &#8211;no-root # install RAGFlow dependent python modules<\/li>\n\n\n\n<li>\u4f7f\u7528 Docker Compose \u542f\u52a8\u4f9d\u8d56\u670d\u52a1\uff08MinIO\u3001Elasticsearch\u3001Redis \u548c MySQL\uff09\uff1adocker compose -f docker\/docker-compose-base.yml up -d<\/li>\n\n\n\n<li>\u6dfb\u52a0\u4ee5\u4e0b\u884c\u4ee5\u5c06\u00a0docker\/service_conf.yaml\u00a0\u4e2d\u6307\u5b9a\u7684\u6240\u6709\u4e3b\u673a\u89e3\u6790\u4e3a\uff1a<code>\/etc\/hosts<\/code><code>127.0.0.1<\/code><\/li>\n<\/ol>\n\n\n\n<ol class=\"wp-block-list\">\n<li><code>127.0.0.1 es01 mysql minio redis<\/code><\/li>\n\n\n\n<li>\u5728\u00a0docker\/service_conf.yaml\u00a0\u4e2d\uff0c\u5c06 mysql \u7aef\u53e3\u66f4\u65b0\u4e3a \uff0c\u5c06 es \u7aef\u53e3\u66f4\u65b0\u4e3a\u00a0\uff0c\u5982 docker\/.env\u00a0\u4e2d\u6307\u5b9a\u3002<code>5455<\/code><code>1200<\/code><\/li>\n<\/ol>\n\n\n\n<ol class=\"wp-block-list\">\n<li>\u5982\u679c\u65e0\u6cd5\u8bbf\u95ee HuggingFace\uff0c\u8bf7\u5c06\u73af\u5883\u53d8\u91cf\u8bbe\u7f6e\u4e3a\u4f7f\u7528\u955c\u50cf\u7ad9\u70b9\uff1a<code>HF_ENDPOINT<\/code>export HF_ENDPOINT=https:\/\/hf-mirror.com\u542f\u52a8 backend service\uff1a<\/li>\n\n\n\n<li>source .venv\/bin\/activate<br>export PYTHONPATH=$(pwd)bash docker\/launch_backend_service.sh<\/li>\n\n\n\n<li>\u5b89\u88c5\u524d\u7aef\u4f9d\u8d56\u9879\uff1a<\/li>\n\n\n\n<li>cd webnpm install &#8211;force<\/li>\n\n\n\n<li>\u5c06\u524d\u7aef\u914d\u7f6e\u4e3a\u5728\u00a0.umirc.ts\u00a0\u66f4\u65b0\u4e3a\uff1a<code>proxy.target<\/code><code>http:\/\/127.0.0.1:9380<\/code><\/li>\n<\/ol>\n\n\n\n<p>\u542f\u52a8\u524d\u7aef\u670d\u52a1\uff1a<\/p>\n\n\n\n<p>npm run dev <em>\u4ee5\u4e0b\u8f93\u51fa\u786e\u8ba4\u7cfb\u7edf\u5df2\u6210\u529f\u542f\u52a8\u5b8c\u6210\u3002<\/em><\/p>\n\n\n\n<p class=\"has-large-font-size\"><br><br>1\u3001Text2KG \u7684\u4f7f\u7528<\/p>\n\n\n\n<p><br><br>Text2KG\u662f\u4e00\u4e2a\u5f00\u6e90\u9879\u76ee\uff0c\u80fd\u591f\u5229\u7528\u5927\u578b\u8bed\u8a00\u6a21\u578b\uff08zero-shot\uff09\u8de8\u9886\u57df\u4ece\u6587\u672c\u4e2d\u63d0\u53d6\u5b9e\u4f53\u548c\u5173\u7cfb\uff0c\u81ea\u52a8\u6784\u5efa\u548c\u66f4\u65b0\u77e5\u8bc6\u56fe\u8c31\uff0c\u5e76\u901a\u8fc7Neo4j\u8fdb\u884c\u53ef\u89c6\u5316\u3002<br><br>iText2KG\u7531\u56db\u4e2a\u4e3b\u8981\u6a21\u5757\u7ec4\u6210\uff1a\u6587\u6863\u63d0\u53d6\u5668\u3001\u589e\u91cf\u5b9e\u4f53\u63d0\u53d6\u5668\u3001\u589e\u91cf\u5173\u7cfb\u63d0\u53d6\u5668\u3001\u56fe\u5f62\u96c6\u6210\u5668\u548c\u53ef\u89c6\u5316\u3002\u5b83\u4eec\u534f\u540c\u5de5\u4f5c\uff0c\u4ece\u975e\u7ed3\u6784\u5316\u6587\u672c\u6784\u5efa\u548c\u53ef\u89c6\u5316\u77e5\u8bc6\u56fe\u8c31\u3002<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"737\" height=\"510\" src=\"http:\/\/www.yliyun.com\/wp-content\/uploads\/2024\/11\/image-14.png\" alt=\"\" class=\"wp-image-9766\" srcset=\"http:\/\/www.yliyun.com\/wp-content\/uploads\/2024\/11\/image-14.png 737w, http:\/\/www.yliyun.com\/wp-content\/uploads\/2024\/11\/image-14-300x208.png 300w\" sizes=\"auto, (max-width: 737px) 100vw, 737px\" \/><\/figure>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>\u6587\u6863\u63d0\u53d6\u5668\uff08Document Distiller\uff09<\/strong>\uff1a\u8be5\u6a21\u5757\u5904\u7406\u539f\u59cb\u6587\u6863\uff0c\u5e76\u6839\u636e\u7528\u6237\u5b9a\u4e49\u7684\u6a21\u5f0f\u5c06\u5176\u91cd\u65b0\u8868\u8ff0\u4e3a\u8bed\u4e49\u5757\u3002\u5b83\u901a\u8fc7\u5173\u6ce8\u76f8\u5173\u4fe1\u606f\u5e76\u4ee5\u9884\u5b9a\u4e49\u7684\u683c\u5f0f\u5bf9\u5176\u8fdb\u884c\u7ed3\u6784\u5316\u6765\u63d0\u9ad8\u4fe1\u566a\u6bd4\u3002<\/li>\n\n\n\n<li><strong>\u589e\u91cf\u5b9e\u4f53\u63d0\u53d6\u5668\uff08Incremental Entity Extractor\uff09<\/strong>\uff1a\u6b64\u6a21\u5757\u4ece\u8bed\u4e49\u5757\u4e2d\u63d0\u53d6\u552f\u4e00\u5b9e\u4f53\u5e76\u89e3\u51b3\u6b67\u4e49\u4ee5\u786e\u4fdd\u6bcf\u4e2a\u5b9e\u4f53\u90fd\u6709\u660e\u786e\u5b9a\u4e49\u3002\u5b83\u4f7f\u7528\u4f59\u5f26\u76f8\u4f3c\u5ea6\u5ea6\u91cf\u5c06\u5c40\u90e8\u5b9e\u4f53\u4e0e\u5168\u5c40\u5b9e\u4f53\u8fdb\u884c\u5339\u914d\u3002<\/li>\n\n\n\n<li><strong>\u589e\u91cf\u5173\u7cfb\u63d0\u53d6\u5668\uff08Incremental Relation Extractor\uff09<\/strong>\uff1a\u6b64\u6a21\u5757\u8bc6\u522b\u63d0\u53d6\u5b9e\u4f53\u4e4b\u95f4\u7684\u5173\u7cfb\u3002\u5b83\u53ef\u4ee5\u4ee5\u4e24\u79cd\u6a21\u5f0f\u8fd0\u884c\uff1a\u4f7f\u7528\u5168\u5c40\u5b9e\u4f53\u4e30\u5bcc\u56fe\u5f62\u4e2d\u7684\u6f5c\u5728\u4fe1\u606f\uff0c\u6216\u4f7f\u7528\u5c40\u90e8\u5b9e\u4f53\u5efa\u7acb\u66f4\u7cbe\u786e\u7684\u5173\u7cfb\u3002<\/li>\n\n\n\n<li><strong>\u56fe\u5f62\u96c6\u6210\u5668\u548c\u53ef\u89c6\u5316\uff08Graph Integrator and Visualization\uff09<\/strong>\uff1a\u6b64\u6a21\u5757\u5c06\u63d0\u53d6\u7684\u5b9e\u4f53\u548c\u5173\u7cfb\u96c6\u6210\u5230 Neo4j \u6570\u636e\u5e93\u4e2d\uff0c\u63d0\u4f9b\u77e5\u8bc6\u56fe\u8c31\u7684\u53ef\u89c6\u5316\u8868\u793a\u3002\u5b83\u5141\u8bb8\u5bf9\u7ed3\u6784\u5316\u6570\u636e\u8fdb\u884c\u4ea4\u4e92\u5f0f\u63a2\u7d22\u548c\u5206\u6790\u3002<\/li>\n<\/ul>\n\n\n\n<p>\u56db\u4e2a\u6a21\u5757\u4e2d\uff0c<strong>\u589e\u91cf\u5b9e\u4f53\u63d0\u53d6\u5668\u4e0e\u589e\u91cf\u5173\u7cfb\u63d0\u53d6\u5668<\/strong>\u6700\u4e3a\u5173\u952e\uff0c\u91c7\u7528\u5927\u6a21\u578b\u6765\u5b9e\u73b0<strong>\uff0c<\/strong>LLM\u63d0\u53d6\u4ee3\u8868\u4e00\u4e2a<strong>\u552f\u4e00\u6982\u5ff5<\/strong>\u7684\u5b9e\u4f53\uff0c\u4ee5\u907f\u514d<strong>\u8bed\u4e49\u6df7\u5408<\/strong>\u7684\u5b9e\u4f53\u3002<strong>\u663e\u793a\u4e86\u4f7f\u7528 Langchain JSON \u89e3\u6790\u5668\u7684\u5b9e\u4f53\u548c\u5173\u7cfb\u63d0\u53d6prompt\u3002<\/strong>\u5206\u7c7b\u5982\u4e0b\uff1a<strong>\u84dd\u8272<\/strong>&nbsp;&#8211; \u7531 Langchain \u81ea\u52a8\u683c\u5f0f\u5316\u7684prompt\uff1b\u5e38\u89c4 &#8211; iText2KG\u8bbe\u8ba1\u7684prompt\uff1b\u659c\u4f53 &#8211; \u4e13\u95e8\u4e3a\u5b9e\u4f53\u548c\u5173\u7cfb\u63d0\u53d6\u8bbe\u8ba1\u7684prompt\u3002\uff08a\uff09\u5173\u7cfb\u63d0\u53d6prompt\u548c\uff08b\uff09\u5b9e\u4f53\u63d0\u53d6prompt\u3002<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"739\" height=\"530\" src=\"http:\/\/www.yliyun.com\/wp-content\/uploads\/2024\/11\/image-15.png\" alt=\"\" class=\"wp-image-9767\" srcset=\"http:\/\/www.yliyun.com\/wp-content\/uploads\/2024\/11\/image-15.png 739w, http:\/\/www.yliyun.com\/wp-content\/uploads\/2024\/11\/image-15-300x215.png 300w\" sizes=\"auto, (max-width: 739px) 100vw, 739px\" \/><\/figure>\n\n\n\n<p>\u4e3a\u4e86\u8bf4\u660e\u77e5\u8bc6\u56fe\u8c31\u6784\u5efa\u7684\u7ed3\u679c\uff0c\u5728\u4e09\u79cd\u4e0d\u540c\u573a\u666f\u4e0b\uff0c\u5c06\u57fa\u7ebf\u65b9\u6cd5\u4e0eiText2KG\u8fdb\u884c\u4e86\u6bd4\u8f83\uff1a<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>\u57fa\u7ebf\u65b9\u6cd5\u5728\u6240\u6709\u4e09\u79cd\u77e5\u8bc6\u56fe\u8c31\u6784\u5efa\u573a\u666f\u4e2d\u90fd\u63ed\u793a\u4e86\u5b58\u5728\u6ca1\u6709\u5173\u7cfb\u7684\u5b64\u7acb\u8282\u70b9\u3002\u8fd9\u79cd\u73b0\u8c61\u53ef\u80fd\u5f52\u56e0\u4e8e\u5b9e\u4f53\u63d0\u53d6\u548c\u5173\u7cfb\u63d0\u53d6\u7684\u540c\u65f6\u6267\u884c\uff0c\u8fd9\u53ef\u80fd\u4f1a\u5728\u8bed\u8a00\u6a21\u578b\u4e2d\u5f15\u8d77\u5e7b\u89c9\u6548\u5e94\uff0c\u5bfc\u81f4\u201c\u9057\u5fd8\u201d\u6548\u5e94\uff0c\u5373\u5206\u79bb\u5b9e\u4f53\u548c\u5173\u7cfb\u63d0\u53d6\u7684\u8fc7\u7a0b\u53ef\u4ee5\u63d0\u9ad8\u6027\u80fd\u3002<\/li>\n\n\n\n<li>\u5728\u201c\u7f51\u7ad9\u5230\u77e5\u8bc6\u56fe\u8c31\u201d\u7684\u573a\u666f\u4e2d\uff0c\u8f93\u5165\u6587\u6863\u6570\u91cf\u7684\u589e\u52a0\u4e0e\u56fe\u4e2d\u566a\u58f0\u8282\u70b9\u7684\u51fa\u73b0\u6709\u5173\u3002\u8fd9\u5f3a\u8c03\u4e86\u5bf9\u6587\u6863\u8fdb\u884c\u6709\u6548\u7cbe\u70bc\u548c\u84b8\u998f\u7684\u6a21\u57571\u7684\u5173\u952e\u9700\u6c42\u3002<\/li>\n\n\n\n<li>iText2KG\u65b9\u6cd5\u5728\u4e09\u79cd\u77e5\u8bc6\u56fe\u8c31\u6784\u5efa\u573a\u666f\u4e2d\u5c55\u793a\u4e86\u6539\u8fdb\u7684\u5b9e\u4f53\u548c\u5173\u7cfb\u89e3\u6790\u80fd\u529b\u3002\u5f53\u8f93\u5165\u6587\u6863\u8f83\u5c11\u4e14\u7531\u7b80\u5355\u3001\u975e\u590d\u6742\u77ed\u8bed\u7ec4\u6210\u65f6\uff0c\u8bed\u8a00\u6a21\u578b\u5728\u5b9e\u4f53\u548c\u5173\u7cfb\u89e3\u6790\u65b9\u9762\u8868\u73b0\u51fa\u9ad8\u6548\u7387\uff0c\u5982\u201c\u7b80\u5386\u5230\u77e5\u8bc6\u56fe\u8c31\u201d\u8fc7\u7a0b\u4e2d\u6240\u8bc1\u660e\u7684\u3002\u76f8\u53cd\uff0c\u968f\u7740\u6570\u636e\u96c6\u53d8\u5f97\u66f4\u52a0\u590d\u6742\u548c\u5e9e\u5927\uff0c\u6311\u6218\u4e5f\u968f\u4e4b\u589e\u52a0\uff0c\u5982\u201c\u7f51\u7ad9\u5230\u77e5\u8bc6\u56fe\u8c31\u201d\u573a\u666f\u6240\u793a\u3002\u6b64\u5916\uff0c\u91cd\u8981\u7684\u662f\u8981\u5f3a\u8c03\u8f93\u5165\u6587\u6863\u7684\u5206\u5757\u5927\u5c0f\u548c\u9608\u503c\u5bf9\u77e5\u8bc6\u56fe\u8c31\u6784\u5efa\u7684\u5f71\u54cd\u3002\u6587\u6863\u5206\u998f\u5668\u7684\u8f93\u5165\u6587\u6863\u53ef\u4ee5\u662f\u72ec\u7acb\u7684\u6587\u6863\u6216\u5206\u5757\u3002\u5982\u679c\u5206\u5757\u5927\u5c0f\u8f83\u5c0f\uff0c\u5219\u8bed\u4e49\u5757\u5c06\u4ece\u6587\u6863\u4e2d\u6355\u83b7\u66f4\u5177\u4f53\u7684\u8be6\u7ec6\u4fe1\u606f\uff0c\u53cd\u4e4b\u4ea6\u7136<\/li>\n<\/ul>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"701\" height=\"792\" src=\"http:\/\/www.yliyun.com\/wp-content\/uploads\/2024\/11\/image-16.png\" alt=\"\" class=\"wp-image-9768\" srcset=\"http:\/\/www.yliyun.com\/wp-content\/uploads\/2024\/11\/image-16.png 701w, http:\/\/www.yliyun.com\/wp-content\/uploads\/2024\/11\/image-16-266x300.png 266w\" sizes=\"auto, (max-width: 701px) 100vw, 701px\" \/><\/figure>\n\n\n\n<p><\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p>\u4e00\u79cd\u7531 LLM \u9a71\u52a8\u7684\u96f6\u6837\u672c\u65b9\u6cd5\uff0c\u4f7f\u7528\u5927\u578b\u8bed\u8a00\u6a21\u578b\u6784\u5efa\u589e\u91cf\u77e5\u8bc6\u56fe\u8c31\uff08KG\uff09<\/p>\n<\/blockquote>\n\n\n\n<p><code>iText2KG<\/code>&nbsp;\u662f\u4e00\u4e2a&nbsp;<code>Python<\/code>&nbsp;\u5305\uff0c\u901a\u8fc7\u5229\u7528\u5927\u578b\u8bed\u8a00\u6a21\u578b\u4ece\u6587\u672c\u6587\u6863\u4e2d\u63d0\u53d6\u5b9e\u4f53\u548c\u5173\u7cfb\uff0c\u9010\u6b65\u6784\u5efa\u5177\u6709\u5df2\u89e3\u6790\u5b9e\u4f53\u548c\u5173\u7cfb\u7684\u4e00\u81f4\u77e5\u8bc6\u56fe\u8c31\u3002<\/p>\n\n\n\n<p>\u5b83\u5177\u6709\u96f6\u6837\u672c\u80fd\u529b\uff0c\u65e0\u9700\u4e13\u95e8\u7684\u8bad\u7ec3\u5373\u53ef\u8de8\u5404\u4e2a\u9886\u57df\u63d0\u53d6\u77e5\u8bc6\u3002<\/p>\n\n\n\n<p>\u5b83\u5305\u542b\u56db\u4e2a\u6a21\u5757\uff1a\u6587\u6863\u63d0\u70bc\u5668\u3001\u589e\u91cf\u5b9e\u4f53\u63d0\u53d6\u5668\u3001\u589e\u91cf\u5173\u7cfb\u63d0\u53d6\u5668\u548c\u56fe\u5f62\u96c6\u6210\u5668\u4e0e\u53ef\u89c6\u5316\u3002<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>\u6587\u6863\u63d0\u53d6\u5668\uff1a\u6b64\u6a21\u5757\u5c06\u539f\u59cb\u6587\u6863\u91cd\u65b0\u8868\u8ff0\u4e3a\u9884\u5b9a\u4e49\u7684\u8bed\u4e49\u5757\uff0c\u5e76\u7531\u6307\u5bfc LLM \u63d0\u53d6\u7279\u5b9a\u4fe1\u606f\u7684\u6a21\u5f0f\u5f15\u5bfc\u3002<\/li>\n\n\n\n<li>\u589e\u91cf\u5b9e\u4f53\u63d0\u53d6\u5668\uff1a\u6b64\u6a21\u5757\u8bc6\u522b\u5e76\u89e3\u6790\u8bed\u4e49\u5757\u5185\u7684\u552f\u4e00\u8bed\u4e49\u5b9e\u4f53\uff0c\u786e\u4fdd\u5b9e\u4f53\u4e4b\u95f4\u7684\u6e05\u6670\u5ea6\u548c\u533a\u522b\u3002<\/li>\n\n\n\n<li>\u589e\u91cf\u5173\u7cfb\u63d0\u53d6\u5668\uff1a\u6b64\u7ec4\u4ef6\u5904\u7406\u5df2\u89e3\u6790\u7684\u5b9e\u4f53\u4ee5\u68c0\u6d4b\u8bed\u4e49\u4e0a\u552f\u4e00\u7684\u5173\u7cfb\uff0c\u89e3\u51b3\u8bed\u4e49\u91cd\u590d\u7684\u6311\u6218\u3002<\/li>\n\n\n\n<li><code>Neo4j<\/code>\u00a0\u56fe\u5f62\u96c6\u6210\u5668\uff1a\u6700\u540e\u4e00\u4e2a\u6a21\u5757\u4ee5\u56fe\u5f62\u683c\u5f0f\u53ef\u89c6\u5316\u5173\u7cfb\u548c\u5b9e\u4f53\uff0c\u5229\u7528 Neo4j \u8fdb\u884c\u6709\u6548\u8868\u793a\u3002<\/li>\n<\/ul>\n\n\n\n<p>\u5bf9\u4e8e\u6211\u4eec\u7684&nbsp;<code>iText2KG<\/code>&nbsp;\u5b83\u5305\u542b\u4e86\u4e24\u5927\u7279\u70b9<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>\u589e\u91cf\u6784\u5efa\uff1a<code>iText2KG<\/code>\u00a0\u5141\u8bb8\u589e\u91cf\u6784\u5efa\u00a0<code>KG<\/code>\uff0c\u8fd9\u610f\u5473\u7740\u5b83\u53ef\u4ee5\u5728\u65b0\u6570\u636e\u53ef\u7528\u65f6\u4e0d\u65ad\u66f4\u65b0\u548c\u6269\u5c55\u56fe\uff0c\u800c\u65e0\u9700\u8fdb\u884c\u5927\u91cf\u91cd\u65b0\u5904\u7406\u3002<\/li>\n\n\n\n<li>\u96f6\u6837\u672c\u5b66\u4e60\uff1a\u8be5\u6846\u67b6\u5229\u7528\u00a0<code>LLM<\/code>\u00a0\u7684\u96f6\u6837\u672c\u529f\u80fd\uff0c\u4f7f\u5176\u65e0\u9700\u9884\u5b9a\u4e49\u96c6\u6216\u5916\u90e8\u672c\u4f53\u5373\u53ef\u8fd0\u884c\u3002\u8fd9\u79cd\u7075\u6d3b\u6027\u4f7f\u5176\u80fd\u591f\u9002\u5e94\u5404\u79cd\u00a0<code>KG<\/code>\u00a0\u6784\u5efa\u573a\u666f\uff0c\u800c\u65e0\u9700\u8fdb\u884c\u5927\u91cf\u8bad\u7ec3\u6216\u5fae\u8c03\u3002<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">\u4e00 \u3001\u8bbe\u7f6e\u6a21\u578b<\/h3>\n\n\n\n<p>\u5728\u8fd0\u884c&nbsp;<code>iText2KG<\/code>&nbsp;\u4e4b\u524d\uff0c\u6211\u4eec\u5148\u8bbe\u7f6e\u597d\u5927\u6a21\u578b\uff0c\u6211\u8fd9\u91cc\u9009\u62e9\u7684\u662f&nbsp;<code>OpenAi<\/code>&nbsp;\u7684\u6a21\u578b\u4ee5\u53ca&nbsp;<code>HuggingFace<\/code>&nbsp;\u7684&nbsp;<code>bge-large-zh embedding<\/code>&nbsp;\u6a21\u578b\u3002\u8fd9\u4e48\u9009\u62e9\u4e5f\u662f\u8003\u8651\u5230\u6784\u5efa&nbsp;<code>KG<\/code>&nbsp;\u7684\u51c6\u786e\u5ea6\u3002<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>from langchain_openai import ChatOpenAI, OpenAIEmbeddings<br>import os<br>os.environ&#91;\"OPENAI_API_KEY\"] = \"*****\"<br>openai_api_key = os.environ&#91;\"OPENAI_API_KEY\"]<br>openai_llm_model = llm = ChatOpenAI(<br>    model=\"gpt-4o-mini\",<br>    temperature=0,<br>    max_tokens=None,<br>    timeout=None,<br>    max_retries=2,<br>)<br>messages = &#91;<br>    (<br>        \"system\",<br>        \"You are a helpful assistant that translates English to French. Translate the user sentence.\",<br>    ),<br>    (\"human\", \"I love programming.\"),<br>]<br><br>ai_msg=openai_llm_model.invoke(messages)<\/code><\/pre>\n\n\n\n<p>\u5f00\u59cb\u90e8\u7f72\u6211\u4eec\u7684&nbsp;<code>Embedding<\/code>&nbsp;\u6a21\u578b\uff1a<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>from langchain_huggingface.embeddings import HuggingFaceEmbeddings<br>openai_embeddings_model = HuggingFaceEmbeddings(model_name=\"BAAI\/bge-large-zh-v1.5\")<br>text = \"This is a test document.\"<br>query_result = openai_embeddings_model.embed_query(text)<br>query_result&#91;:3]<br>doc_result = openai_embeddings_model.embed_documents(&#91;text])<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\">\u4e8c \u3001\u4f7f\u7528&nbsp;<code>iText2KG<\/code>&nbsp;\u6784\u5efa&nbsp;<code>KG<\/code><\/h3>\n\n\n\n<p>\u6211\u4eec\u8fd9\u91cc\u7684\u573a\u666f\u662f\uff0c\u7ed9\u51fa\u4e00\u7bc7\u7b80\u5386\uff0c\u4f7f\u7528\u77e5\u8bc6\u56fe\u8c31\u5c06\u5728\u7ebf\u804c\u4f4d\u63cf\u8ff0\u4e0e\u751f\u6210\u7684\u7b80\u5386\u8054\u7cfb\u8d77\u6765\u3002<\/p>\n\n\n\n<p>\u8bbe\u5b9a\u76ee\u6807\u662f\u8bc4\u4f30\u5019\u9009\u4eba\u662f\u5426\u9002\u5408\u8fd9\u4efd\u5de5\u4f5c\u3002<\/p>\n\n\n\n<p>\u6211\u4eec\u53ef\u4ee5\u4e3a&nbsp;<code>iText2KG<\/code>&nbsp;\u7684\u6bcf\u4e2a\u6a21\u5757\u4f7f\u7528\u4e0d\u540c\u7684&nbsp;<code>LLM<\/code>&nbsp;\u6216\u5d4c\u5165\u6a21\u578b\u3002\u4f46\u662f\uff0c\u91cd\u8981\u7684\u662f\u786e\u4fdd\u8282\u70b9\u548c\u5173\u7cfb\u5d4c\u5165\u7684\u7ef4\u5ea6\u5728\u5404\u4e2a\u6a21\u578b\u4e4b\u95f4\u4fdd\u6301\u4e00\u81f4\u3002<\/p>\n\n\n\n<p>\u5982\u679c\u5d4c\u5165\u7ef4\u5ea6\u4e0d\u540c\uff0c\u4f59\u5f26\u76f8\u4f3c\u5ea6\u53ef\u80fd\u96be\u4ee5\u51c6\u786e\u6d4b\u91cf\u5411\u91cf\u8ddd\u79bb\u4ee5\u8fdb\u884c\u8fdb\u4e00\u6b65\u5339\u914d\u3002<\/p>\n\n\n\n<p>\u6211\u4eec\u7684\u7b80\u5386\u653e\u5230\u6839\u76ee\u5f55\uff0c\u52a0\u8f7d\u7b80\u5386\uff1a<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>from langchain.document_loaders import PyPDFLoader<br>loader = PyPDFLoader(f\".\/CV_Emily_Davis.pdf\")<br>pages = loader.load_and_split()<\/code><\/pre>\n\n\n\n<p>\u521d\u59cb\u5316&nbsp;<code>DocumentDistiller<\/code>&nbsp;\u5f15\u5165&nbsp;<code>llm<\/code>&nbsp;\uff1a<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>from itext2kg.documents_distiller import DocumentsDisiller, CV<br>document_distiller = DocumentsDisiller(llm_model = openai_llm_model)<\/code><\/pre>\n\n\n\n<p>\u4fe1\u606f\u63d0\u70bc\uff1a<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>IE_query = '''<br># DIRECTIVES :<br>- Act like an experienced information extractor.<br>- You have a chunk of a CV.<br>- If you do not find the right information, keep its place empty.<br>'''<br># \u4f7f\u7528\u5b9a\u4e49\u597d\u7684\u67e5\u8be2\u548c\u8f93\u51fa\u6570\u636e\u7ed3\u6784\u63d0\u70bc\u6587\u6863\u3002<br>distilled_cv = document_distiller.distill(documents=&#91;page.page_content.replace(\"{\", '&#91;').replace(\"}\", \"]\") for page in pages], IE_query=IE_query, output_data_structure=CV)<\/code><\/pre>\n\n\n\n<p>\u5c06\u63d0\u70bc\u540e\u7684\u6587\u6863\u683c\u5f0f\u5316\u4e3a\u8bed\u4e49\u90e8\u5206\u3002<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>semantic_blocks_cv = &#91;f\"{key} - {value}\".replace(\"{\", \"&#91;\").replace(\"}\", \"]\") for key, value in distilled_cv.items() if value !=&#91;] and value != \"\"  and value != None]<\/code><\/pre>\n\n\n\n<p>\u6211\u4eec\u53ef\u4ee5\u81ea\u5b9a\u4e49\u8f93\u51fa\u6570\u636e\u7ed3\u6784\uff0c\u6211\u4eec\u8fd9\u91cc\u5b9a\u4e49\u4e864\u79cd\uff0c\u5de5\u4f5c\u7ecf\u5386\u6a21\u578b\uff0c\u5c97\u4f4d\uff0c\u6280\u80fd\uff0c\u8bc1\u4e66\u3002<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>from pydantic import BaseModel, Field<br>from typing import List, Optional<br><br>class JobResponsibility(BaseModel):<br>    description: str = Field(..., description=\"A specific responsibility in the job role\")<br><br>class JobQualification(BaseModel):<br>    skill: str = Field(..., description=\"A required or preferred skill for the job\")<br><br>class JobCertification(BaseModel):<br>    certification: str = Field(..., description=\"Required or preferred certifications for the job\")<br><br>class JobOffer(BaseModel):<br>    job_offer_title: str = Field(..., description=\"The job title\")<br>    company: str = Field(..., description=\"The name of the company offering the job\")<br>    location: str = Field(..., description=\"The job location (can specify if remote\/hybrid)\")<br>    job_type: str = Field(..., description=\"Type of job (e.g., full-time, part-time, contract)\")<br>    responsibilities: List&#91;JobResponsibility] = Field(..., description=\"List of key responsibilities\")<br>    qualifications: List&#91;JobQualification] = Field(..., description=\"List of required or preferred qualifications\")<br>    certifications: Optional&#91;List&#91;JobCertification]] = Field(None, description=\"Required or preferred certifications\")<br>    benefits: Optional&#91;List&#91;str]] = Field(None, description=\"List of job benefits\")<br>    experience_required: str = Field(..., description=\"Required years of experience\")<br>    salary_range: Optional&#91;str] = Field(None, description=\"Salary range for the position\")<br>    apply_url: Optional&#91;str] = Field(None, description=\"URL to apply for the job\")<\/code><\/pre>\n\n\n\n<p>\u5b9a\u4e49\u4e00\u4e2a\u62db\u8058\u5de5\u4f5c\u9700\u6c42\u7684\u63cf\u8ff0\uff1a<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>job_offer = \"\"\"<br>About the Job Offer<br>THE FICTITIOUS COMPANY<br><br>FICTITIOUS COMPANY is a high-end French fashion brand known for its graphic and poetic style, driven by the values of authenticity and transparency upheld by its creator Simon Porte Jacquemus.<br><br>Your Role<br><br>Craft visual stories that captivate, inform, and inspire. Transform concepts and ideas into visual representations. As a member of the studio, in collaboration with the designers and under the direction of the Creative Designer, you should be able to take written or spoken ideas and convert them into designs that resonate. You need to have a deep understanding of the brand image and DNA, being able to find the style and layout suited to each project.<br><br>Your Missions<br><br>Translate creative direction into high-quality silhouettes using Photoshop<br>Work on a wide range of projects to visualize and develop graphic designs that meet each brief<br>Work independently as well as in collaboration with the studio team to meet deadlines, potentially handling five or more projects simultaneously<br>Develop color schemes and renderings in Photoshop, categorized by themes, subjects, etc.<br>Your Profile<br><br>Bachelor\u2019s degree (Bac+3\/5) in Graphic Design or Art<br>3 years of experience in similar roles within a luxury brand's studio<br>Proficiency in Adobe Suite, including Illustrator, InDesign, Photoshop<br>Excellent communication and presentation skills<br>Strong organizational and time management skills to meet deadlines in a fast-paced environment<br>Good understanding of the design process<br>Freelance cont<\/code><\/pre>\n\n\n\n<p>\u7ee7\u7eed\u4f7f\u7528\u4e0a\u9762\u65b9\u6cd5\u505a\u4fe1\u606f\u63d0\u70bc\uff1a<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>IE_query = '''<br># DIRECTIVES :<br>- Act like an experienced information extractor.<br>- You have a chunk of a job offer description.<br>- If you do not find the right information, keep its place empty.<br>'''<br>distilled_Job_Offer = document_distiller.distill(documents=&#91;job_offer], IE_query=IE_query, output_data_structure=JobOffer)<br>print(distilled_Job_Offer)<br>semantic_blocks_job_offer = &#91;f\"{key} - {value}\".replace(\"{\", \"&#91;\").replace(\"}\", \"]\") for key, value in distilled_Job_Offer.items() if value !=&#91;] and value != \"\"  and value != None]<\/code><\/pre>\n\n\n\n<p>\u5230\u8fd9\u91cc\u51c6\u5907\u5de5\u4f5c\u5b8c\u6210\uff0c\u7b80\u5386\u548c\u5de5\u4f5c\u9700\u6c42\u90fd\u5df2\u7ecf\u63d0\u70bc\u5b8c\u6bd5\uff0c\u7136\u540e\u6b63\u5f0f\u5f00\u59cb\u6784\u5efa&nbsp;<code>graph<\/code>\uff0c\u6211\u4eec\u5c06\u7b80\u5386\u7684\u6240\u6709\u8bed\u4e49\u5757\u4f5c\u4e3a\u4e00\u4e2a\u5757\u4f20\u9012\u7ed9\u4e86&nbsp;<code>LLM<\/code>\u3002<\/p>\n\n\n\n<p>\u4e5f\u5c06\u5de5\u4f5c\u9700\u6c42\u4f5c\u4e3a\u53e6\u4e00\u4e2a\u8bed\u4e49\u5757\u4f20\u9012\uff0c\u4e5f\u53ef\u4ee5\u5728\u6784\u5efa\u56fe\u65f6\u5c06\u8bed\u4e49\u5757\u5206\u5f00\u3002<\/p>\n\n\n\n<p>\u6211\u4eec\u9700\u8981\u6ce8\u610f\u6bcf\u4e2a\u5757\u4e2d\u5305\u542b\u591a\u5c11\u4fe1\u606f\uff0c\u7136\u540e\u597d\u5c06\u5b83\u4e0e\u5176\u4ed6\u5757\u8fde\u63a5\u8d77\u6765\uff0c\u6211\u4eec\u5728\u8fd9\u91cc\u505a\u7684\u5c31\u662f\u4e00\u6b21\u6027\u4f20\u9012\u6240\u6709\u8bed\u4e49\u5757\u3002<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>from itext2kg import iText2KG<br>itext2kg = iText2KG(llm_model = openai_llm_model, embeddings_model = openai_embeddings_model)<br><br>global_ent, global_rel = itext2kg.build_graph(sections=&#91;semantic_blocks_cv], ent_threshold=0.6, rel_threshold=0.6)<br><br>global_ent_, global_rel_ = itext2kg.build_graph(sections=&#91;semantic_blocks_job_offer], existing_global_entities = global_ent, existing_global_relationships = global_rel,  ent_threshold=0.6, rel_threshold=0.6)<\/code><\/pre>\n\n\n\n<p><code>iText2KG<\/code>&nbsp;\u6784\u5efa&nbsp;<code>KG<\/code>&nbsp;\u7684\u8fc7\u7a0b\u6211\u4eec\u770b\u5230\u6709\u5f88\u591a\u53c2\u6570\uff0c\u4e0b\u9762\u5206\u8d1d\u662f\u5bf9\u6bcf\u4e2a\u53c2\u6570\u7684\u8868\u793a\u505a\u4e00\u4e9b\u89e3\u91ca\uff1a<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><code>llm_model<\/code>\uff1a\u7528\u4e8e\u4ece\u6587\u672c\u4e2d\u63d0\u53d6\u5b9e\u4f53\u548c\u5173\u7cfb\u7684\u8bed\u8a00\u6a21\u578b\u5b9e\u4f8b\u3002<\/li>\n\n\n\n<li><code>embeddings_model<\/code>\uff1a\u7528\u4e8e\u521b\u5efa\u63d0\u53d6\u5b9e\u4f53\u7684\u5411\u91cf\u8868\u793a\u7684\u5d4c\u5165\u6a21\u578b\u5b9e\u4f8b\u3002<\/li>\n\n\n\n<li><code>sleep_time (int)<\/code>\uff1a\u9047\u5230\u901f\u7387\u9650\u5236\u6216\u9519\u8bef\u65f6\u7b49\u5f85\u7684\u65f6\u95f4\uff08\u4ee5\u79d2\u4e3a\u5355\u4f4d\uff09\uff08\u4ec5\u9002\u7528\u4e8e\u00a0<code>OpenAI<\/code>\uff09\u3002\u9ed8\u8ba4\u4e3a 5 \u79d2\u3002<\/li>\n<\/ul>\n\n\n\n<p><code>iText2KG<\/code>&nbsp;\u7684&nbsp;<code>build_graph<\/code>&nbsp;\u53c2\u6570\uff1a<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><code>sections<\/code>\u00a0<code>(List[str])<\/code>\uff1a\u5b57\u7b26\u4e32\uff08\u8bed\u4e49\u5757\uff09\u5217\u8868\uff0c\u5176\u4e2d\u6bcf\u4e2a\u5b57\u7b26\u4e32\u4ee3\u8868\u6587\u6863\u7684\u4e00\u90e8\u5206\uff0c\u5c06\u4ece\u4e2d\u63d0\u53d6\u5b9e\u4f53\u548c\u5173\u7cfb\u3002<\/li>\n\n\n\n<li><code>existing_global_entities<\/code>\u00a0<code>(List[dict], optional)<\/code>\uff1a\u4e0e\u65b0\u63d0\u53d6\u7684\u5b9e\u4f53\u8fdb\u884c\u5339\u914d\u7684\u73b0\u6709\u5168\u5c40\u5b9e\u4f53\u5217\u8868\u3002\u6bcf\u4e2a\u5b9e\u4f53\u90fd\u8868\u793a\u4e3a\u4e00\u4e2a\u5b57\u5178\u3002<\/li>\n\n\n\n<li><code>existing_global_relationships (List[dict], optional)<\/code>\uff1a\u4e0e\u65b0\u63d0\u53d6\u7684\u5173\u7cfb\u5339\u914d\u7684\u73b0\u6709\u5168\u5c40\u5173\u7cfb\u5217\u8868\u3002\u6bcf\u4e2a\u5173\u7cfb\u90fd\u8868\u793a\u4e3a\u4e00\u4e2a\u5b57\u5178\u3002<\/li>\n\n\n\n<li><code>ent_threshold (float, optional)<\/code>\uff1a\u5b9e\u4f53\u5339\u914d\u7684\u9608\u503c\uff0c\u7528\u4e8e\u5408\u5e76\u4e0d\u540c\u90e8\u5206\u7684\u5b9e\u4f53\u3002\u9ed8\u8ba4\u503c\u4e3a 0.7\u3002<\/li>\n\n\n\n<li><code>rel_threshold (float, optional)<\/code>\uff1a\u5173\u7cfb\u5339\u914d\u7684\u9608\u503c\uff0c\u7528\u4e8e\u5408\u5e76\u4e0d\u540c\u90e8\u5206\u7684\u5173\u7cfb\u3002\u9ed8\u8ba4\u503c\u4e3a 0.7\u3002<\/li>\n<\/ul>\n\n\n\n<p>\u4ece\u56fe\u4e2d\u7ed3\u679c\u770b\u5230\u6211\u4eec\u6784\u5efa\u8fc7\u7a0b\u4e2d\u7684\u5b9e\u4f53\uff0c\u548c\u5173\u8054\u5173\u7cfb\u3002<\/p>\n\n\n\n<p>\u6700\u540e\u4f7f\u7528&nbsp;<code>GraphIntegrator<\/code>&nbsp;\u5bf9\u6784\u5efa\u7684\u77e5\u8bc6\u56fe\u8c31\u8fdb\u884c\u53ef\u89c6\u5316\u3002<\/p>\n\n\n\n<p>\u4f7f\u7528\u6307\u5b9a\u7684\u51ed\u636e\u8bbf\u95ee\u56fe\u5f62\u6570\u636e\u5e93&nbsp;<code>Neo4j<\/code>\uff0c\u5e76\u5bf9\u751f\u6210\u7684\u56fe\u5f62\u8fdb\u884c\u53ef\u89c6\u5316\uff0c\u4ee5\u63d0\u4f9b\u4ece\u6587\u6863\u4e2d\u63d0\u53d6\u7684\u5173\u7cfb\u548c\u5b9e\u4f53\u7684\u89c6\u89c9\u8868\u793a\u3002<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>from itext2kg.graph_integration import GraphIntegrator<br>URI = \"bolt:\/\/3.216.93.32:7687\"<br>USERNAME = \"neo4j\"<br>PASSWORD = \"selection-cosal-cubes\"<br>new_graph = {}<br>new_graph&#91;\"nodes\"] = global_ent_<br>new_graph&#91;\"relationships\"] = global_rel_<br>GraphIntegrator(uri=URI, username=USERNAME, password=PASSWORD).visualize_graph(json_graph=new_graph)<\/code><\/pre>\n\n\n\n<p>\u6253\u5f00\u6211\u4eec\u7684&nbsp;<code>Neo4j<\/code>&nbsp;\u56fe\u5f62\u6570\u636e\u5e93\uff1a<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"666\" height=\"345\" src=\"http:\/\/www.yliyun.com\/wp-content\/uploads\/2024\/11\/image-17.png\" alt=\"\" class=\"wp-image-9769\" srcset=\"http:\/\/www.yliyun.com\/wp-content\/uploads\/2024\/11\/image-17.png 666w, http:\/\/www.yliyun.com\/wp-content\/uploads\/2024\/11\/image-17-300x155.png 300w\" sizes=\"auto, (max-width: 666px) 100vw, 666px\" \/><\/figure>\n","protected":false},"excerpt":{"rendered":"<p>1\u3001RAGFlow \u7684\u4f7f\u7528\u6307\u5357 RAGFlow \u662f [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":9769,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[83,84,86,85],"tags":[79,87],"class_list":["post-9765","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai-rag","category-rag","category-86","category-85","tag-rag","tag-87"],"_links":{"self":[{"href":"http:\/\/www.yliyun.com\/wp-json\/wp\/v2\/posts\/9765","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/www.yliyun.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/www.yliyun.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/www.yliyun.com\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/www.yliyun.com\/wp-json\/wp\/v2\/comments?post=9765"}],"version-history":[{"count":1,"href":"http:\/\/www.yliyun.com\/wp-json\/wp\/v2\/posts\/9765\/revisions"}],"predecessor-version":[{"id":9770,"href":"http:\/\/www.yliyun.com\/wp-json\/wp\/v2\/posts\/9765\/revisions\/9770"}],"wp:featuredmedia":[{"embeddable":true,"href":"http:\/\/www.yliyun.com\/wp-json\/wp\/v2\/media\/9769"}],"wp:attachment":[{"href":"http:\/\/www.yliyun.com\/wp-json\/wp\/v2\/media?parent=9765"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/www.yliyun.com\/wp-json\/wp\/v2\/categories?post=9765"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/www.yliyun.com\/wp-json\/wp\/v2\/tags?post=9765"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}