基于ModelScope镜像,进行简单模型应用

2022-12-26 18 0

基于ModelScope官方镜像直接使用

下载docker镜像

docker pull registry.cn-hangzhou.aliyuncs.com/modelscope-repo/modelscope:ubuntu20.04-py37-torch1.11.0-tf1.15.5-1.1.2

查看镜像

docker images

root@Ubuntu:/etc/docker# docker images
REPOSITORY                                                     TAG                                           IMAGE ID       CREATED         SIZE
registry.cn-hangzhou.aliyuncs.com/modelscope-repo/modelscope   ubuntu20.04-py37-torch1.11.0-tf1.15.5-1.1.2   e9067a5c9537   2 weeks ago     5.78GB

从镜像创建实例(根据需要配置端口和文件夹映射,不需要可移除相关参数)

docker run -d -p 8080:8080 -v /data/modelscope-data:/data/modelscope -it e9067a5c9537 /bin/bash

查看实例

docker ps

root@Ubuntu:/etc/docker# docker ps
CONTAINER ID   IMAGE          COMMAND            CREATED        STATUS       PORTS                                                                          NAMES
35b9884bc46c   e9067a5c9537   "/bin/bash"        5 hours ago    Up 3 hours   0.0.0.0:8080->8080/tcp, :::8080->8080/tcp                                      fervent_mirzakhani

进入容器

docker exec -it 35b9884bc46c /bin/bash

安装验证

安装成功后,即可使用对应领域模型进行推理,训练等操作。这里我们以NLP领域为例。安装后,可执行如下命令,运行中文分词任务,来验证安装是否正确:

python -c "from modelscope.pipelines import pipeline;print(pipeline('word-segmentation')('今天天气不错,适合 出去游玩'))"

root@35b9884bc46c:/# python -c "from modelscope.pipelines import pipeline;print(pipeline('word-segmentation')('今天天气不错,适合 出去游玩'))"
2022-12-26 16:21:52,321 - modelscope - INFO - PyTorch version 1.11.0+cpu Found.
2022-12-26 16:21:52,323 - modelscope - INFO - Loading ast index from /mnt/workspace/.cache/modelscope/ast_indexer
2022-12-26 16:21:52,323 - modelscope - INFO - No valid ast index found from /mnt/workspace/.cache/modelscope/ast_indexer, rebuilding ast index!
2022-12-26 16:21:52,330 - modelscope - INFO - AST-Scaning the path "/opt/conda/lib/python3.7/site-packages/modelscope" with the following sub folders ['models', 'metrics', 'pipelines', 'preprocessors', 'trainers', 'msdatasets']
2022-12-26 16:22:21,131 - modelscope - INFO - Scaning done! A number of 474 components indexed or updated! Time consumed 28.808072328567505s
2022-12-26 16:22:21,152 - modelscope - INFO - Loading done! Current index file version is 1.1.2, with md5 45729300fbcccc731a81cca0f5b3fc63 and a total number of 474 components indexed
2022-12-26 16:22:22,219 - modelscope - INFO - Model revision not specified, use the latest revision: v1.0.2
2022-12-26 16:22:22,352 - modelscope - INFO - downloading http://www.modelscope.cn/api/v1/models/damo/nlp_structbert_word-segmentation_chinese-base/repo?Revision=v1.0.2&FilePath=resources/bastructbert.png to /mnt/workspace/.cache/modelscope/temp/tmp6ug0axvj/tmp75r66yl1
Downloading: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 243k/243k [00:00<00:00, 7.55MB/s]
2022-12-26 16:22:22,480 - modelscope - INFO - storing http://www.modelscope.cn/api/v1/models/damo/nlp_structbert_word-segmentation_chinese-base/repo?Revision=v1.0.2&FilePath=resources/bastructbert.png in cache at /mnt/workspace/.cache/modelscope/temp/tmp6ug0axvj
2022-12-26 16:22:22,482 - modelscope - INFO - downloading http://www.modelscope.cn/api/v1/models/damo/nlp_structbert_word-segmentation_chinese-base/repo?Revision=v1.0.2&FilePath=config.json to /mnt/workspace/.cache/modelscope/temp/tmp6ug0axvj/tmp_ybwbodr
Downloading: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 977/977 [00:00<00:00, 3.40MB/s]
2022-12-26 16:22:22,581 - modelscope - INFO - storing http://www.modelscope.cn/api/v1/models/damo/nlp_structbert_word-segmentation_chinese-base/repo?Revision=v1.0.2&FilePath=config.json in cache at /mnt/workspace/.cache/modelscope/temp/tmp6ug0axvj
2022-12-26 16:22:22,582 - modelscope - INFO - downloading http://www.modelscope.cn/api/v1/models/damo/nlp_structbert_word-segmentation_chinese-base/repo?Revision=v1.0.2&FilePath=configuration.json to /mnt/workspace/.cache/modelscope/temp/tmp6ug0axvj/tmp_vl5zcko
Downloading: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 161/161 [00:00<00:00, 609kB/s]
2022-12-26 16:22:22,685 - modelscope - INFO - storing http://www.modelscope.cn/api/v1/models/damo/nlp_structbert_word-segmentation_chinese-base/repo?Revision=v1.0.2&FilePath=configuration.json in cache at /mnt/workspace/.cache/modelscope/temp/tmp6ug0axvj
2022-12-26 16:22:22,686 - modelscope - INFO - downloading http://www.modelscope.cn/api/v1/models/damo/nlp_structbert_word-segmentation_chinese-base/repo?Revision=v1.0.2&FilePath=resources/cws_model.png to /mnt/workspace/.cache/modelscope/temp/tmp6ug0axvj/tmp3272_all
Downloading: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 46.7k/46.7k [00:00<00:00, 3.37MB/s]
2022-12-26 16:22:22,793 - modelscope - INFO - storing http://www.modelscope.cn/api/v1/models/damo/nlp_structbert_word-segmentation_chinese-base/repo?Revision=v1.0.2&FilePath=resources/cws_model.png in cache at /mnt/workspace/.cache/modelscope/temp/tmp6ug0axvj
2022-12-26 16:22:22,795 - modelscope - INFO - downloading http://www.modelscope.cn/api/v1/models/damo/nlp_structbert_word-segmentation_chinese-base/repo?Revision=v1.0.2&FilePath=pytorch_model.bin to /mnt/workspace/.cache/modelscope/temp/tmp6ug0axvj/tmpmmewsdme
Downloading: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 388M/388M [01:54<00:00, 3.54MB/s]
2022-12-26 16:24:17,879 - modelscope - INFO - storing http://www.modelscope.cn/api/v1/models/damo/nlp_structbert_word-segmentation_chinese-base/repo?Revision=v1.0.2&FilePath=pytorch_model.bin in cache at /mnt/workspace/.cache/modelscope/temp/tmp6ug0axvj
2022-12-26 16:24:19,097 - modelscope - INFO - downloading http://www.modelscope.cn/api/v1/models/damo/nlp_structbert_word-segmentation_chinese-base/repo?Revision=v1.0.2&FilePath=README.md to /mnt/workspace/.cache/modelscope/temp/tmp6ug0axvj/tmpsx49pzai
Downloading: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 5.67k/5.67k [00:00<00:00, 1.02MB/s]
2022-12-26 16:24:19,191 - modelscope - INFO - storing http://www.modelscope.cn/api/v1/models/damo/nlp_structbert_word-segmentation_chinese-base/repo?Revision=v1.0.2&FilePath=README.md in cache at /mnt/workspace/.cache/modelscope/temp/tmp6ug0axvj
2022-12-26 16:24:19,192 - modelscope - INFO - downloading http://www.modelscope.cn/api/v1/models/damo/nlp_structbert_word-segmentation_chinese-base/repo?Revision=v1.0.2&FilePath=tokenizer_config.json to /mnt/workspace/.cache/modelscope/temp/tmp6ug0axvj/tmpbk34idb_
Downloading: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 30.0/30.0 [00:00<00:00, 96.4kB/s]
2022-12-26 16:24:19,291 - modelscope - INFO - storing http://www.modelscope.cn/api/v1/models/damo/nlp_structbert_word-segmentation_chinese-base/repo?Revision=v1.0.2&FilePath=tokenizer_config.json in cache at /mnt/workspace/.cache/modelscope/temp/tmp6ug0axvj
2022-12-26 16:24:19,292 - modelscope - INFO - downloading http://www.modelscope.cn/api/v1/models/damo/nlp_structbert_word-segmentation_chinese-base/repo?Revision=v1.0.2&FilePath=vocab.txt to /mnt/workspace/.cache/modelscope/temp/tmp6ug0axvj/tmp13q8y9o_
Downloading: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 107k/107k [00:00<00:00, 5.28MB/s]
2022-12-26 16:24:19,400 - modelscope - INFO - storing http://www.modelscope.cn/api/v1/models/damo/nlp_structbert_word-segmentation_chinese-base/repo?Revision=v1.0.2&FilePath=vocab.txt in cache at /mnt/workspace/.cache/modelscope/temp/tmp6ug0axvj
2022-12-26 16:24:20,113 - modelscope - INFO - initiate model from /mnt/workspace/.cache/modelscope/damo/nlp_structbert_word-segmentation_chinese-base
2022-12-26 16:24:20,113 - modelscope - INFO - initiate model from location /mnt/workspace/.cache/modelscope/damo/nlp_structbert_word-segmentation_chinese-base.
2022-12-26 16:24:20,117 - modelscope - INFO - initialize model from /mnt/workspace/.cache/modelscope/damo/nlp_structbert_word-segmentation_chinese-base
You are using a model of type sbert to instantiate a model of type structbert. This is not supported for all configurations of models and can yield errors.
2022-12-26 16:24:24,645 - modelscope - INFO - cuda is not available, using cpu instead.
2022-12-26 16:24:24,646 - modelscope - WARNING - No preprocessor field found in cfg.
2022-12-26 16:24:24,647 - modelscope - WARNING - No val key and type key found in preprocessor domain of configuration.json file.
2022-12-26 16:24:24,647 - modelscope - WARNING - Cannot find available config to build preprocessor at mode inference, current config: {'model_dir': '/mnt/workspace/.cache/modelscope/damo/nlp_structbert_word-segmentation_chinese-base', 'sequence_length': 128}. trying to build by task and model information.
/opt/conda/lib/python3.7/site-packages/transformers/modeling_utils.py:769: FutureWarning: The `device` argument is deprecated and will be removed in v5 of Transformers.
  "The `device` argument is deprecated and will be removed in v5 of Transformers.", FutureWarning
{'output': ['今天', '天气', '不错', ',', '适合', '出去', '游玩']}
root@35b9884bc46c:/# 
1dd9

相关文章

借助《文心一言》,我迅速学会Flutter并完成了一个项目

0