C3-BenchMark数据集链接：

https://modelscope.cn/datasets/Tencent-Hunyuan/C3-BenchMark

from modelscope import AutoModelForCausalLM, AutoTokenizer
import os
import re
model_name_or_path = "Tencent-Hunyuan/Hunyuan-A13B-Instruct"
tokenizer = AutoTokenizer.from_pretrained(model_name_or_path, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(model_name_or_path, device_map="auto",trust_remote_code=True)  # You may want to use bfloat16 and/or move to GPU here
messages = [
    {"role": "user", "content": "Write a short summary of the benefits of regular exercise"},
]
tokenized_chat = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True,return_tensors="pt",
                                                enable_thinking=True # Toggle thinking mode (default: True)
                                                )
outputs = model.generate(tokenized_chat.to(model.device), max_new_tokens=4096)
output_text = tokenizer.decode(outputs[0])
think_pattern = r'<think>(.*?)</think>'
think_matches = re.findall(think_pattern, output_text, re.DOTALL)
answer_pattern = r'<answer>(.*?)</answer>'
answer_matches = re.findall(answer_pattern, output_text, re.DOTALL)
think_content = [match.strip() for match in think_matches][0]
answer_content = [match.strip() for match in answer_matches][0]
print(f"thinking_content:{think_content}\n\n")
print(f"answer_content:{answer_content}\n\n")

git clone https://github.com/modelscope/ms-swift.git
cd ms-swift
pip install -e .
pip install liger-kernel transformers -U

{"messages": [{"role": "user", "content": "浙江的省会在哪？"}, {"role": "assistant", "content": "<think>\nxxx\n</think>\n浙江的省会在杭州。"}]}

{"messages": [{"role": "user", "content": "浙江的省会在哪？"}, {"role": "assistant", "content": "<think>\n\n</think>\n浙江的省会在杭州。"}]}

# 训练显存：4 * 47GiB
CUDA_VISIBLE_DEVICES=0,1,2,3 \
swift sft \
    --model Tencent-Hunyuan/Hunyuan-A13B-Instruct \
    --train_type lora \
    --dataset 'liucong/Chinese-DeepSeek-R1-Distill-data-110k-SFT#1500' \
              'swift/self-cognition:empty_think#600' \
    --loss_scale ignore_empty_think \
    --torch_dtype bfloat16 \
    --num_train_epochs 1 \
    --per_device_train_batch_size 1 \
    --per_device_eval_batch_size 1 \
    --learning_rate 1e-4 \
    --lora_rank 8 \
    --lora_alpha 32 \
    --target_modules all-linear \
    --gradient_accumulation_steps 8 \
    --load_from_cache_file false \
    --eval_steps 50 \
    --save_steps 50 \
    --save_total_limit 2 \
    --logging_steps 5 \
    --max_length 2048 \
    --output_dir output \
    --warmup_ratio 0.05 \
    --packing true \
    --attn_impl flash_attn \
    --dataloader_num_workers 4 \
    --model_author swift \
    --model_name swift-robot

训练显存占用：

CUDA_VISIBLE_DEVICES=0,1,2,3 \
swift infer \
    --adapters output/vx-xxx/checkpoint-xxx \
    --stream true \
    --temperature 0 \
    --max_new_tokens 2048

swift export \
    --adapters output/vx-xxx/checkpoint-xxx \
    --push_to_hub true \
    --hub_model_id '<your-model-id>' \
    --hub_token '<your-sdk-token>'

第一步：拉取镜像：

docker pull hunyuaninfer/hunyuan-a13b:hunyuan-moe-A13B-trtllm

启动 API 服务器：

docker run --name hunyuanLLM_infer --rm -it --ipc=host --ulimit memlock=-1 --ulimit stack=67108864 --gpus=all hunyuaninfer/hunyuan-a13b:hunyuan-moe-A13B-trtllm

trtllm-serve \
  /path/to/HunYuan-moe-A13B \
  --host localhost \
  --port 8000 \
  --backend pytorch \
  --max_batch_size 128 \
  --max_num_tokens 16384 \
  --tp_size 2 \
  --kv_cache_free_gpu_memory_fraction 0.95 \
  --extra_llm_api_options /path/to/extra-llm-api-config.yml

vLLM

开始：

docker pull docker.cnb.cool/tencent/hunyuan/hunyuan-a13b:hunyuan-moe-A13B-vllm 
#docker pull hunyuaninfer/hunyuan-a13b:hunyuan-moe-A13B-vllm

下载模型文件：
Huggingface：将由 vllm 自动下载。
ModelScope：

modelscope download --model Tencent-Hunyuan/Hunyuan-A13B-Instruct

或者设置环境变量

export VLLM_USE_MODELSCOPE=True

👇点击关注ModelScope公众号获取
更多技术信息~