LLM_deploy

• Usage about deploy LLM in Computing Power Platform

In Terminal 1

• This process is responsible for listening to the memory consumption ratio of CUDA

nvidia-smi -l 1

In Terminal 2

• Run Ollama serve

ollama serve

In Terminal 3

• This process is responsible for running the Open-WebUI

open-webui serve

In Terminal 4

• This process is responsible for running Ngrok for Intranet penetration

ngrok http 8080

Be sure to turn off Academic acceleration(VPN)