Skip to content

GitLab

M
mt-model-deploy-dhruva
Commit 470398da authored by Nikhilesh Bhatnagar's avatar Nikhilesh Bhatnagar
Browse files
Options

Fix build errors on azure

parent a80be722
Show whitespace changes
Inline Side-by-side
Showing with 5 additions and 4 deletions
README.md
View file @ 470398da
...@@ -5,11 +5,11 @@ ...@@ -5,11 +5,11 @@
## TL;DR ## TL;DR
This repo contains code for python backend CTranslate2 based triton models for the SSMT project. This repo contains code for python backend CTranslate2 based triton models for the SSMT project.
Prerequisites: `python3.xx-venv`, `nvidia-docker` Prerequisites: `python3.xx-venv`, `nvidia-docker`, `bash`
```bash ```bash
git clone https://ssmt.iiit.ac.in/meitygit/ssmt/mt-model-deploy-dhruva.git git clone https://ssmt.iiit.ac.in/meitygit/ssmt/mt-model-deploy-dhruva.git
cd mt-model-deploy-dhruva cd mt-model-deploy-dhruva
sh make_triton_model_repo.sh "https://ssmt.iiit.ac.in/uploads/data_mining/models.zip" "float16" bash make_triton_model_repo.sh "https://ssmt.iiit.ac.in/uploads/data_mining/models.zip" "float16"
docker build -t dhruva/ssmt-model-server:1 . docker build -t dhruva/ssmt-model-server:1 .
nvidia-docker run --gpus=all --rm --shm-size 5g --network=host --name dhruva-ssmt-triton-server -v./ssmt_triton_repo:/models dhruva/ssmt-model-server:1 nvidia-docker run --gpus=all --rm --shm-size 5g --network=host --name dhruva-ssmt-triton-server -v./ssmt_triton_repo:/models dhruva/ssmt-model-server:1
``` ```
...@@ -37,7 +37,7 @@ One can construct the triton repo like so: ...@@ -37,7 +37,7 @@ One can construct the triton repo like so:
```bash ```bash
git clone https://ssmt.iiit.ac.in/meitygit/ssmt/mt-model-deploy-dhruva.git git clone https://ssmt.iiit.ac.in/meitygit/ssmt/mt-model-deploy-dhruva.git
cd mt-model-deploy-dhruva cd mt-model-deploy-dhruva
sh make_triton_model_repo.sh "https://ssmt.iiit.ac.in/uploads/data_mining/models.zip" "float16" bash make_triton_model_repo.sh "https://ssmt.iiit.ac.in/uploads/data_mining/models.zip" "float16"
``` ```
## Starting the triton server ## Starting the triton server
......
make_triton_model_repo.sh
View file @ 470398da
#!/bin/bash
MODELS_URL=$1 MODELS_URL=$1
QUANTIZATION=$2 QUANTIZATION=$2
wget -O models.zip $MODELS_URL --no-check-certificate wget -O models.zip $MODELS_URL --no-check-certificate
...@@ -46,7 +47,7 @@ cp -r ../triton_models/ssmt_template_model_repo ssmt_9_ct2 ...@@ -46,7 +47,7 @@ cp -r ../triton_models/ssmt_template_model_repo ssmt_9_ct2
cp -r ../models/9_ct2 ssmt_9_ct2/1/translator cp -r ../models/9_ct2 ssmt_9_ct2/1/translator
sed -i 's/model_name/ssmt_9_ct2/' ssmt_9_ct2/config.pbtxt sed -i 's/model_name/ssmt_9_ct2/' ssmt_9_ct2/config.pbtxt
cd .. cd ..
source deactivate source ./ssmt_ct2/bin/activate
rm -rf ssmt_ct2 rm -rf ssmt_ct2
rm -f models.zip rm -f models.zip
rm -rf models rm -rf models
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment