SpringBoot集成Xinference实战:手把手教你搭建企业级AI服务
现在很多开发团队都想给自己的SpringBoot应用加上AI能力,比如智能客服、内容生成、数据分析这些功能。但真动手做的时候,问题就来了:模型怎么选?服务怎么部署?性能怎么保证?特别是微服务架构下,怎么让AI能力既好用又好管?
最近我帮几个项目集成了Xinference-v1.17.1,发现它跟SpringBoot简直是绝配。Xinference不只是个模型推理工具,更像是个AI服务编排平台,能让你用一行代码就把各种开源大模型集成到自己的系统里。今天我就来手把手教你,怎么从零开始搭建一套企业级的AI服务。
1. 为什么选择Xinference?
先说说为什么推荐Xinference。市面上AI推理框架不少,但Xinference有几个特别适合企业场景的优点。
1.1 开箱即用,部署简单
Xinference最大的好处就是简单。你不用自己折腾模型转换、环境配置这些麻烦事,它都帮你搞定了。比如你想用Qwen2.5做聊天,用BGE做文本嵌入,用Stable Diffusion做图片生成,Xinference都能一键部署。
几个关键特点:
- 统一API接口:不管什么模型,都用同样的方式调用,开发特别省心
- 多模型支持:文本、图片、语音、多模态模型都能跑,不用来回切换工具
- 资源智能调度:能自动利用GPU和CPU,让硬件发挥最大价值
- 生产就绪:自带监控、日志、健康检查,直接就能上生产环境
1.2 跟SpringBoot天生一对
Xinference提供的是标准的RESTful API,跟SpringBoot的HTTP客户端完美匹配。你不用写复杂的适配代码,就像调用普通Web服务一样简单。
更重要的是,Xinference支持分布式部署。你可以把模型服务单独部署,SpringBoot应用通过负载均衡调用,这样既保证了性能,又实现了服务解耦。
2. 整体架构设计
在开始写代码之前,我们先看看整体架构怎么设计。好的架构能让后续开发少踩很多坑。
用户请求 → SpringBoot应用 → AI服务层 → Xinference集群 → 返回结果
具体来说,我们建议这样分层:
- 业务层:你的SpringBoot应用,处理具体的业务逻辑
- AI服务层:封装所有AI调用,提供统一的接口给业务层
- Xinference集群:专门跑模型推理,可以独立部署和扩展
- 基础设施:数据库、缓存、监控这些支撑服务
这种设计有几个好处:
- 业务代码干净:AI相关的代码都集中在AI服务层,业务层只管调用
- 易于维护:模型升级、参数调整都在Xinference那边,不影响业务代码
- 弹性扩展:AI服务压力大了,单独给Xinference加机器就行
- 故障隔离:一个模型出问题,不会影响其他服务
3. 环境准备与部署
3.1 部署Xinference服务
我们先来部署Xinference。推荐用Docker,最简单省事。
如果你有GPU:
# 拉取带CUDA的镜像
docker pull xprobe/xinference:v1.17.1-cu129
# 运行容器
docker run -d \
--name xinference-server \
--gpus all \
-p 9997:9997 \
-e XINFERENCE_MODEL_SRC=modelscope \
-v /data/xinference/models:/root/.xinference/models \
xprobe/xinference:v1.17.1-cu129 \
xinference-local -H 0.0.0.0
如果只有CPU:
# 拉取CPU版本镜像
docker pull xprobe/xinference:v1.17.1
# 运行容器
docker run -d \
--name xinference-server \
-p 9997:9997 \
-e XINFERENCE_MODEL_SRC=modelscope \
-v /data/xinference/models:/root/.xinference/models \
xprobe/xinference:v1.17.1 \
xinference-local -H 0.0.0.0
参数说明:
-p 9997:9997:把容器的9997端口映射到主机,这样就能通过http://localhost:9997访问了-e XINFERENCE_MODEL_SRC=modelscope:指定从ModelScope下载模型,国内访问速度快-v /data/xinference/models:/root/.xinference/models:把模型目录挂载出来,下次启动不用重新下载
部署完成后,打开浏览器访问 http://localhost:9997,应该能看到Xinference的管理界面。
3.2 启动第一个模型
现在我们来启动一个聊天模型试试。打开终端,执行:
# 连接到Xinference服务
xinference launch --model-name qwen2.5-instruct --model-format pytorch --size-in-billions 7 --replica 1
这个命令会下载并启动Qwen2.5-7B模型。第一次下载可能需要一些时间,取决于你的网络速度。
启动成功后,你会看到类似这样的输出:
Model uid: 731edc3a-2e5f-4b5f-8c5a-9e3b7c8d2a1f
Endpoint: http://localhost:9997/v1
记下这个Model UID,后面调用的时候要用。
3.3 测试模型是否正常
用curl简单测试一下:
curl http://localhost:9997/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "qwen2.5-instruct",
"messages": [
{"role": "user", "content": "你好,介绍一下你自己"}
],
"max_tokens": 100
}'
如果返回正常的JSON响应,说明模型启动成功了。
4. SpringBoot项目集成
4.1 创建SpringBoot项目
如果你还没有SpringBoot项目,可以用Spring Initializr快速创建一个:
# 使用curl创建项目
curl https://start.spring.io/starter.zip \
-d type=maven-project \
-d language=java \
-d bootVersion=3.2.0 \
-d baseDir=xinference-demo \
-d groupId=com.example \
-d artifactId=xinference-demo \
-d name=xinference-demo \
-d description="Xinference集成示例" \
-d packageName=com.example.xinference \
-d packaging=jar \
-d javaVersion=17 \
-d dependencies=web,configuration-processor \
-o xinference-demo.zip
# 解压
unzip xinference-demo.zip
cd xinference-demo
4.2 添加依赖
在pom.xml里添加必要的依赖:
<dependencies>
<!-- SpringBoot基础依赖 -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<!-- HTTP客户端 -->
<dependency>
<groupId>org.apache.httpcomponents.client5</groupId>
<artifactId>httpclient5</artifactId>
<version>5.3.1</version>
</dependency>
<!-- JSON处理 -->
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
</dependency>
<!-- 配置处理 -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-configuration-processor</artifactId>
<optional>true</optional>
</dependency>
<!-- 连接池 -->
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-pool2</artifactId>
</dependency>
<!-- 参数校验 -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-validation</artifactId>
</dependency>
</dependencies>
4.3 配置Xinference连接
在application.yml里配置Xinference服务地址:
server:
port: 8080
spring:
application:
name: xinference-demo
xinference:
# Xinference服务地址
endpoint: http://localhost:9997
# 连接超时(毫秒)
connect-timeout: 5000
# 读取超时(毫秒)
read-timeout: 30000
# 最大连接数
max-connections: 100
# 默认模型配置
default-models:
chat: qwen2.5-instruct # 默认聊天模型
embedding: bge-large-zh-v1.5 # 默认嵌入模型
创建配置类读取这些配置:
@Configuration
@ConfigurationProperties(prefix = "xinference")
@Data
public class XinferenceConfig {
private String endpoint;
private int connectTimeout = 5000;
private int readTimeout = 30000;
private int maxConnections = 100;
private Map<String, String> defaultModels = new HashMap<>();
@Bean
public CloseableHttpClient xinferenceHttpClient() {
PoolingHttpClientConnectionManager connectionManager =
new PoolingHttpClientConnectionManager();
connectionManager.setMaxTotal(maxConnections);
connectionManager.setDefaultMaxPerRoute(20);
RequestConfig requestConfig = RequestConfig.custom()
.setConnectTimeout(connectTimeout)
.setSocketTimeout(readTimeout)
.build();
return HttpClients.custom()
.setConnectionManager(connectionManager)
.setDefaultRequestConfig(requestConfig)
.build();
}
}
5. 核心服务层实现
5.1 定义数据模型
先定义请求和响应的数据模型:
@Data
public class ChatRequest {
@NotBlank(message = "消息不能为空")
private List<Map<String, Object>> messages;
private String modelUid; // 如果不指定,使用默认模型
private Integer maxTokens = 500;
private Double temperature = 0.7;
private Boolean stream = false;
}
@Data
public class ChatResponse {
private String content;
private String model;
private Long usageTokens;
private Long totalTokens;
private Long completionTokens;
private Long promptTokens;
}
@Data
public class EmbeddingRequest {
@NotBlank(message = "输入文本不能为空")
private String input;
private String modelUid;
}
@Data
public class EmbeddingResponse {
private List<Float> embedding;
private String model;
private Long totalTokens;
}
5.2 实现AI服务接口
创建AI服务接口:
public interface AIModelService {
/**
* 文本对话
*/
ChatResponse chat(ChatRequest request);
/**
* 生成文本嵌入
*/
EmbeddingResponse createEmbedding(EmbeddingRequest request);
/**
* 获取模型列表
*/
List<Map<String, Object>> listModels();
/**
* 启动新模型
*/
String launchModel(String modelName, String modelFormat,
Integer sizeInBillions, Integer replica);
/**
* 终止模型
*/
boolean terminateModel(String modelUid);
}
实现Xinference服务:
@Service
@Slf4j
public class XinferenceServiceImpl implements AIModelService {
@Autowired
private CloseableHttpClient httpClient;
@Value("${xinference.endpoint}")
private String endpoint;
@Value("${xinference.default-models.chat}")
private String defaultChatModel;
@Value("${xinference.default-models.embedding}")
private String defaultEmbeddingModel;
private final ObjectMapper objectMapper = new ObjectMapper();
@Override
public ChatResponse chat(ChatRequest request) {
try {
// 构建请求体
Map<String, Object> requestBody = new HashMap<>();
requestBody.put("model", request.getModelUid() != null ?
request.getModelUid() : defaultChatModel);
requestBody.put("messages", request.getMessages());
requestBody.put("stream", request.getStream());
if (request.getMaxTokens() != null) {
requestBody.put("max_tokens", request.getMaxTokens());
}
if (request.getTemperature() != null) {
requestBody.put("temperature", request.getTemperature());
}
// 调用Xinference API
String response = post("/v1/chat/completions", requestBody);
Map<String, Object> result = objectMapper.readValue(response, Map.class);
// 解析响应
ChatResponse chatResponse = new ChatResponse();
chatResponse.setModel((String) result.get("model"));
// 解析使用量
Map<String, Object> usage = (Map<String, Object>) result.get("usage");
if (usage != null) {
chatResponse.setPromptTokens(((Number) usage.get("prompt_tokens")).longValue());
chatResponse.setCompletionTokens(((Number) usage.get("completion_tokens")).longValue());
chatResponse.setTotalTokens(((Number) usage.get("total_tokens")).longValue());
}
// 解析回复内容
List<Map<String, Object>> choices = (List<Map<String, Object>>) result.get("choices");
if (!choices.isEmpty()) {
Map<String, Object> choice = choices.get(0);
Map<String, Object> message = (Map<String, Object>) choice.get("message");
chatResponse.setContent((String) message.get("content"));
}
return chatResponse;
} catch (Exception e) {
log.error("调用Xinference聊天接口失败", e);
throw new RuntimeException("AI服务调用失败: " + e.getMessage(), e);
}
}
@Override
public EmbeddingResponse createEmbedding(EmbeddingRequest request) {
try {
Map<String, Object> requestBody = new HashMap<>();
requestBody.put("input", request.getInput());
requestBody.put("model", request.getModelUid() != null ?
request.getModelUid() : defaultEmbeddingModel);
String response = post("/v1/embeddings", requestBody);
Map<String, Object> result = objectMapper.readValue(response, Map.class);
EmbeddingResponse embeddingResponse = new EmbeddingResponse();
embeddingResponse.setModel((String) result.get("model"));
// 解析使用量
Map<String, Object> usage = (Map<String, Object>) result.get("usage");
if (usage != null) {
embeddingResponse.setTotalTokens(((Number) usage.get("total_tokens")).longValue());
}
// 解析嵌入向量
List<Map<String, Object>> data = (List<Map<String, Object>>) result.get("data");
if (!data.isEmpty()) {
List<Double> embeddingList = (List<Double>) data.get(0).get("embedding");
// 转换为Float列表
List<Float> floatList = embeddingList.stream()
.map(Double::floatValue)
.collect(Collectors.toList());
embeddingResponse.setEmbedding(floatList);
}
return embeddingResponse;
} catch (Exception e) {
log.error("生成文本嵌入失败", e);
throw new RuntimeException("嵌入生成失败: " + e.getMessage(), e);
}
}
@Override
public List<Map<String, Object>> listModels() {
try {
String response = get("/v1/models");
Map<String, Object> result = objectMapper.readValue(response, Map.class);
return (List<Map<String, Object>>) result.get("data");
} catch (Exception e) {
log.error("获取模型列表失败", e);
throw new RuntimeException("获取模型列表失败", e);
}
}
@Override
public String launchModel(String modelName, String modelFormat,
Integer sizeInBillions, Integer replica) {
try {
Map<String, Object> requestBody = new HashMap<>();
requestBody.put("model_name", modelName);
requestBody.put("model_format", modelFormat);
if (sizeInBillions != null) {
requestBody.put("size_in_billions", sizeInBillions);
}
if (replica != null) {
requestBody.put("replica", replica);
}
String response = post("/v1/models", requestBody);
Map<String, Object> result = objectMapper.readValue(response, Map.class);
return (String) result.get("model_uid");
} catch (Exception e) {
log.error("启动模型失败", e);
throw new RuntimeException("启动模型失败: " + e.getMessage(), e);
}
}
@Override
public boolean terminateModel(String modelUid) {
try {
delete("/v1/models/" + modelUid);
return true;
} catch (Exception e) {
log.error("终止模型失败: {}", modelUid, e);
return false;
}
}
private String get(String path) throws IOException {
HttpGet httpGet = new HttpGet(endpoint + path);
return executeRequest(httpGet);
}
private String post(String path, Object body) throws IOException {
HttpPost httpPost = new HttpPost(endpoint + path);
httpPost.setHeader("Content-Type", "application/json");
String jsonBody = objectMapper.writeValueAsString(body);
httpPost.setEntity(new StringEntity(jsonBody, StandardCharsets.UTF_8));
return executeRequest(httpPost);
}
private String delete(String path) throws IOException {
HttpDelete httpDelete = new HttpDelete(endpoint + path);
return executeRequest(httpDelete);
}
private String executeRequest(HttpUriRequest request) throws IOException {
try (CloseableHttpResponse response = httpClient.execute(request)) {
int statusCode = response.getStatusLine().getStatusCode();
String responseBody = EntityUtils.toString(response.getEntity(), StandardCharsets.UTF_8);
if (statusCode >= 200 && statusCode < 300) {
return responseBody;
} else {
log.error("Xinference API调用失败,状态码:{},响应:{}", statusCode, responseBody);
throw new IOException("API调用失败,状态码:" + statusCode + ",响应:" + responseBody);
}
}
}
}
6. 业务层调用示例
6.1 智能客服场景
假设我们要做一个智能客服系统:
@RestController
@RequestMapping("/api/customer-service")
@Slf4j
public class CustomerServiceController {
@Autowired
private AIModelService aiModelService;
@PostMapping("/chat")
public ResponseEntity<Map<String, Object>> handleCustomerQuery(
@Valid @RequestBody CustomerQueryRequest request) {
try {
// 构建系统提示词
String systemPrompt = "你是一个专业的客服助手,请用友好、专业的态度回答用户问题。" +
"如果用户的问题涉及具体业务,请根据以下信息回答:\n" +
"1. 工作时间:周一至周五 9:00-18:00\n" +
"2. 联系方式:客服电话 400-123-4567\n" +
"3. 退货政策:7天无理由退货\n" +
"如果不知道答案,请如实告知并建议用户联系人工客服。";
// 构建对话消息
List<Map<String, Object>> messages = new ArrayList<>();
messages.add(Map.of("role", "system", "content", systemPrompt));
messages.add(Map.of("role", "user", "content", request.getQuestion()));
// 如果有历史对话,添加到上下文
if (request.getHistory() != null && !request.getHistory().isEmpty()) {
for (Map<String, String> history : request.getHistory()) {
messages.add(Map.of(
"role", history.get("role"),
"content", history.get("content")
));
}
}
// 调用AI服务
ChatRequest chatRequest = new ChatRequest();
chatRequest.setMessages(messages);
chatRequest.setMaxTokens(500);
chatRequest.setTemperature(0.7);
ChatResponse response = aiModelService.chat(chatRequest);
// 记录对话历史
saveConversationHistory(
request.getUserId(),
request.getQuestion(),
response.getContent()
);
// 返回响应
Map<String, Object> result = new HashMap<>();
result.put("success", true);
result.put("answer", response.getContent());
result.put("model", response.getModel());
result.put("timestamp", System.currentTimeMillis());
return ResponseEntity.ok(result);
} catch (Exception e) {
log.error("处理客户查询失败", e);
Map<String, Object> errorResult = new HashMap<>();
errorResult.put("success", false);
errorResult.put("error", "系统繁忙,请稍后再试");
errorResult.put("timestamp", System.currentTimeMillis());
return ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR)
.body(errorResult);
}
}
private void saveConversationHistory(String userId, String question, String answer) {
// 这里可以保存到数据库
log.info("保存对话历史 - 用户: {}, 问题: {}, 回答长度: {}",
userId, question, answer.length());
}
}
@Data
class CustomerQueryRequest {
@NotBlank(message = "用户ID不能为空")
private String userId;
@NotBlank(message = "问题不能为空")
private String question;
private List<Map<String, String>> history;
}
6.2 内容生成场景
电商场景下,我们需要自动生成商品描述:
@Service
@Slf4j
public class ContentGenerationService {
@Autowired
private AIModelService aiModelService;
/**
* 生成商品描述
*/
public String generateProductDescription(ProductInfo product) {
String prompt = String.format(
"请为以下商品生成一段吸引人的电商描述:\n\n" +
"商品名称:%s\n" +
"商品类别:%s\n" +
"主要特点:%s\n" +
"适用人群:%s\n" +
"价格区间:%s\n\n" +
"要求:\n" +
"1. 突出商品的核心卖点\n" +
"2. 语言生动有趣,吸引目标客户\n" +
"3. 适合在电商平台展示\n" +
"4. 长度在200-300字之间",
product.getName(),
product.getCategory(),
String.join("、", product.getFeatures()),
product.getTargetAudience(),
product.getPriceRange()
);
List<Map<String, Object>> messages = new ArrayList<>();
messages.add(Map.of("role", "user", "content", prompt));
ChatRequest request = new ChatRequest();
request.setMessages(messages);
request.setMaxTokens(400);
request.setTemperature(0.8); // 温度稍高,让生成更有创意
try {
ChatResponse response = aiModelService.chat(request);
return response.getContent();
} catch (Exception e) {
log.error("生成商品描述失败", e);
return getDefaultDescription(product);
}
}
/**
* 批量生成营销文案
*/
public List<MarketingCopy> generateMarketingCopies(ProductInfo product, int count) {
List<MarketingCopy> copies = new ArrayList<>();
String[] tones = {"专业正式", "活泼有趣", "简洁直接", "温馨亲切", "科技感强"};
String[] platforms = {"电商详情页", "社交媒体", "广告文案", "邮件营销", "宣传册"};
Random random = new Random();
for (int i = 0; i < count; i++) {
String tone = tones[random.nextInt(tones.length)];
String platform = platforms[random.nextInt(platforms.length)];
String prompt = String.format(
"请以%s的风格,为商品'%s'写一段%s文案。\n\n" +
"商品信息:\n" +
"- 类别:%s\n" +
"- 特点:%s\n" +
"- 目标人群:%s\n\n" +
"文案要求:\n" +
"1. 符合%s平台的调性\n" +
"2. 突出%s这个核心卖点\n" +
"3. 包含明确的行动号召\n" +
"4. 长度在100-150字",
tone, product.getName(), platform,
product.getCategory(),
String.join("、", product.getFeatures()),
product.getTargetAudience(),
platform,
product.getFeatures().get(0) // 取第一个特点作为核心卖点
);
List<Map<String, Object>> messages = new ArrayList<>();
messages.add(Map.of("role", "user", "content", prompt));
ChatRequest request = new ChatRequest();
request.setMessages(messages);
request.setMaxTokens(200);
request.setTemperature(0.9); // 高温度,生成更多样化的文案
try {
ChatResponse response = aiModelService.chat(request);
MarketingCopy copy = new MarketingCopy();
copy.setContent(response.getContent());
copy.setTone(tone);
copy.setPlatform(platform);
copy.setGeneratedAt(new Date());
copies.add(copy);
} catch (Exception e) {
log.warn("生成营销文案失败,跳过第{}个", i + 1, e);
}
// 避免请求过快
try {
Thread.sleep(100);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
}
return copies;
}
/**
* 生成搜索关键词
*/
public List<String> generateSearchKeywords(ProductInfo product) {
String prompt = String.format(
"请为商品'%s'生成20个相关的搜索关键词。\n" +
"商品信息:%s\n" +
"要求:关键词要覆盖不同搜索意图,包括品牌词、产品词、场景词、长尾词等。",
product.getName(),
String.join(",", product.getFeatures())
);
List<Map<String, Object>> messages = new ArrayList<>();
messages.add(Map.of("role", "user", "content", prompt));
ChatRequest request = new ChatRequest();
request.setMessages(messages);
request.setMaxTokens(300);
request.setTemperature(0.5); // 低温度,让生成更准确
try {
ChatResponse response = aiModelService.chat(request);
return parseKeywords(response.getContent());
} catch (Exception e) {
log.error("生成搜索关键词失败", e);
return Collections.emptyList();
}
}
private List<String> parseKeywords(String content) {
// 简单解析关键词,实际可以根据需要调整
return Arrays.stream(content.split("[,\n、;]"))
.map(String::trim)
.filter(keyword -> !keyword.isEmpty())
.collect(Collectors.toList());
}
private String getDefaultDescription(ProductInfo product) {
return String.format(
"%s是一款%s产品,具有%s等特点,适合%s人群使用。",
product.getName(),
product.getCategory(),
String.join("、", product.getFeatures()),
product.getTargetAudience()
);
}
}
@Data
class ProductInfo {
private String name;
private String category;
private List<String> features;
private String targetAudience;
private String priceRange;
}
@Data
class MarketingCopy {
private String content;
private String tone;
private String platform;
private Date generatedAt;
}
6.3 文本嵌入与相似度搜索
很多业务场景需要做相似度搜索,比如推荐相关商品、查找相似问题:
@Service
@Slf4j
public class SimilaritySearchService {
@Autowired
private AIModelService aiModelService;
/**
* 计算文本相似度
*/
public double calculateSimilarity(String text1, String text2) {
try {
// 生成两个文本的嵌入向量
EmbeddingRequest request1 = new EmbeddingRequest();
request1.setInput(text1);
EmbeddingResponse embedding1 = aiModelService.createEmbedding(request1);
EmbeddingRequest request2 = new EmbeddingRequest();
request2.setInput(text2);
EmbeddingResponse embedding2 = aiModelService.createEmbedding(request2);
// 计算余弦相似度
return cosineSimilarity(embedding1.getEmbedding(), embedding2.getEmbedding());
} catch (Exception e) {
log.error("计算文本相似度失败", e);
return 0.0;
}
}
/**
* 查找最相似的商品
*/
public List<ProductMatch> findSimilarProducts(String query, List<Product> products, int topK) {
try {
// 生成查询文本的嵌入向量
EmbeddingRequest queryRequest = new EmbeddingRequest();
queryRequest.setInput(query);
EmbeddingResponse queryEmbedding = aiModelService.createEmbedding(queryRequest);
// 批量生成商品描述的嵌入向量
List<CompletableFuture<ProductEmbedding>> futures = products.stream()
.map(product -> CompletableFuture.supplyAsync(() -> {
try {
EmbeddingRequest productRequest = new EmbeddingRequest();
productRequest.setInput(product.getDescription());
EmbeddingResponse embedding = aiModelService.createEmbedding(productRequest);
ProductEmbedding result = new ProductEmbedding();
result.setProduct(product);
result.setEmbedding(embedding.getEmbedding());
result.setSimilarity(cosineSimilarity(
queryEmbedding.getEmbedding(),
embedding.getEmbedding()
));
return result;
} catch (Exception e) {
log.warn("生成商品嵌入失败: {}", product.getId(), e);
return null;
}
}))
.collect(Collectors.toList());
// 等待所有任务完成
CompletableFuture.allOf(futures.toArray(new CompletableFuture[0])).join();
// 收集结果并排序
return futures.stream()
.map(CompletableFuture::join)
.filter(Objects::nonNull)
.sorted((a, b) -> Double.compare(b.getSimilarity(), a.getSimilarity()))
.limit(topK)
.map(pe -> {
ProductMatch match = new ProductMatch();
match.setProduct(pe.getProduct());
match.setSimilarity(pe.getSimilarity());
return match;
})
.collect(Collectors.toList());
} catch (Exception e) {
log.error("查找相似商品失败", e);
return Collections.emptyList();
}
}
/**
* 计算余弦相似度
*/
private double cosineSimilarity(List<Float> vectorA, List<Float> vectorB) {
if (vectorA == null || vectorB == null ||
vectorA.size() != vectorB.size() ||
vectorA.isEmpty()) {
return 0.0;
}
double dotProduct = 0.0;
double normA = 0.0;
double normB = 0.0;
for (int i = 0; i < vectorA.size(); i++) {
float a = vectorA.get(i);
float b = vectorB.get(i);
dotProduct += a * b;
normA += a * a;
normB += b * b;
}
if (normA == 0.0 || normB == 0.0) {
return 0.0;
}
return dotProduct / (Math.sqrt(normA) * Math.sqrt(normB));
}
}
@Data
class Product {
private String id;
private String name;
private String description;
private String category;
}
@Data
class ProductEmbedding {
private Product product;
private List<Float> embedding;
private double similarity;
}
@Data
class ProductMatch {
private Product product;
private double similarity;
}
7. 高级特性与优化
7.1 连接池优化
微服务环境下,网络调用性能很重要。我们来优化一下HTTP客户端:
@Configuration
public class HttpClientConfig {
@Bean
public CloseableHttpClient xinferenceHttpClient(XinferenceConfig config) {
// 连接池配置
PoolingHttpClientConnectionManager connectionManager =
new PoolingHttpClientConnectionManager();
// 最大连接数
connectionManager.setMaxTotal(config.getMaxConnections());
// 每个路由的最大连接数
connectionManager.setDefaultMaxPerRoute(20);
// 空闲连接存活时间(秒)
connectionManager.setValidateAfterInactivity(5000);
// 请求配置
RequestConfig requestConfig = RequestConfig.custom()
.setConnectTimeout(config.getConnectTimeout())
.setSocketTimeout(config.getReadTimeout())
.setConnectionRequestTimeout(3000) // 从连接池获取连接的超时
.build();
// 重试策略
HttpRequestRetryStrategy retryStrategy = new DefaultHttpRequestRetryStrategy(
3, // 最大重试次数
TimeValue.ofSeconds(1) // 重试间隔
);
// 连接保活策略
ConnectionKeepAliveStrategy keepAliveStrategy = (response, context) -> {
// 服务器建议的保活时间,默认30秒
HeaderElementIterator it = new BasicHeaderElementIterator(
response.headerIterator(HTTP.CONN_KEEP_ALIVE));
while (it.hasNext()) {
HeaderElement he = it.nextElement();
String param = he.getName();
String value = he.getValue();
if (value != null && param.equalsIgnoreCase("timeout")) {
try {
return Long.parseLong(value) * 1000;
} catch (NumberFormatException ignore) {
}
}
}
// 默认保活30秒
return 30 * 1000;
};
return HttpClients.custom()
.setConnectionManager(connectionManager)
.setDefaultRequestConfig(requestConfig)
.setRetryStrategy(retryStrategy)
.setKeepAliveStrategy(keepAliveStrategy)
// 连接复用策略
.setConnectionReuseStrategy(DefaultConnectionReuseStrategy.INSTANCE)
// 连接存活时间监控
.evictExpiredConnections()
.evictIdleConnections(TimeValue.ofSeconds(30))
.build();
}
}
7.2 异步调用提升性能
对于批量处理场景,可以用异步调用提升吞吐量:
@Service
public class AsyncAIService {
@Autowired
private AIModelService aiModelService;
@Autowired
private ThreadPoolTaskExecutor taskExecutor;
/**
* 异步文本生成
*/
public CompletableFuture<ChatResponse> asyncChat(ChatRequest request) {
return CompletableFuture.supplyAsync(
() -> aiModelService.chat(request),
taskExecutor
);
}
/**
* 批量生成文本嵌入
*/
public CompletableFuture<List<EmbeddingResponse>> batchCreateEmbeddings(List<String> texts) {
List<CompletableFuture<EmbeddingResponse>> futures = texts.stream()
.map(text -> CompletableFuture.supplyAsync(() -> {
EmbeddingRequest request = new EmbeddingRequest();
request.setInput(text);
return aiModelService.createEmbedding(request);
}, taskExecutor))
.collect(Collectors.toList());
return CompletableFuture.allOf(futures.toArray(new CompletableFuture[0]))
.thenApply(v -> futures.stream()
.map(CompletableFuture::join)
.collect(Collectors.toList()));
}
/**
* 并行处理多个对话请求
*/
public Map<String, ChatResponse> parallelChat(List<ChatRequest> requests) {
Map<String, CompletableFuture<ChatResponse>> futures = new ConcurrentHashMap<>();
for (int i = 0; i < requests.size(); i++) {
ChatRequest request = requests.get(i);
String requestId = "req_" + i + "_" + System.currentTimeMillis();
futures.put(requestId, CompletableFuture.supplyAsync(
() -> {
try {
return aiModelService.chat(request);
} catch (Exception e) {
log.error("处理请求失败: {}", requestId, e);
// 返回降级响应
ChatResponse fallback = new ChatResponse();
fallback.setContent("抱歉,AI服务暂时不可用");
return fallback;
}
},
taskExecutor
));
}
// 设置超时
CompletableFuture<Void> allFutures = CompletableFuture.allOf(
futures.values().toArray(new CompletableFuture[0])
);
try {
allFutures.get(30, TimeUnit.SECONDS);
} catch (TimeoutException e) {
log.warn("批量处理超时,取消未完成的任务");
futures.values().forEach(future -> future.cancel(true));
} catch (Exception e) {
log.error("批量处理失败", e);
}
// 收集结果
Map<String, ChatResponse> results = new HashMap<>();
futures.forEach((requestId, future) -> {
try {
if (!future.isCancelled() && future.isDone()) {
ChatResponse response = future.get();
results.put(requestId, response);
}
} catch (Exception e) {
log.warn("获取结果失败: {}", requestId, e);
}
});
return results;
}
@Bean
public ThreadPoolTaskExecutor aiTaskExecutor() {
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
// 核心线程数 = CPU核心数
int corePoolSize = Runtime.getRuntime().availableProcessors();
executor.setCorePoolSize(corePoolSize);
// 最大线程数
executor.setMaxPoolSize(corePoolSize * 4);
// 队列容量
executor.setQueueCapacity(1000);
// 线程空闲时间
executor.setKeepAliveSeconds(60);
// 线程名前缀
executor.setThreadNamePrefix("ai-async-");
// 拒绝策略:调用者运行
executor.setRejectedExecutionHandler(new ThreadPoolExecutor.CallerRunsPolicy());
// 等待所有任务完成后关闭
executor.setWaitForTasksToCompleteOnShutdown(true);
executor.setAwaitTerminationSeconds(60);
executor.initialize();
return executor;
}
}
7.3 缓存优化
对于频繁查询的相同内容,可以加缓存提升性能:
@Service
@Slf4j
public class CachedAIService implements AIModelService {
@Autowired
private AIModelService delegate;
@Autowired
private CacheManager cacheManager;
private static final String EMBEDDING_CACHE = "embeddingCache";
private static final String CHAT_CACHE = "chatCache";
@Override
@Cacheable(value = EMBEDDING_CACHE, key = "#request.input + '|' + #request.modelUid")
public EmbeddingResponse createEmbedding(EmbeddingRequest request) {
log.debug("缓存未命中,调用实际服务生成嵌入: {}", request.getInput());
return delegate.createEmbedding(request);
}
@Override
@Cacheable(value = CHAT_CACHE, key = {
"#request.messages.hashCode()",
"#request.modelUid",
"#request.maxTokens",
"#request.temperature"
})
public ChatResponse chat(ChatRequest request) {
log.debug("缓存未命中,调用实际服务生成对话");
return delegate.chat(request);
}
@Override
public List<Map<String, Object>> listModels() {
// 模型列表变化不频繁,可以缓存较长时间
Cache cache = cacheManager.getCache("modelListCache");
if (cache != null) {
List<Map<String, Object>> cached = cache.get("modelList", List.class);
if (cached != null) {
return cached;
}
}
List<Map<String, Object>> models = delegate.listModels();
if (cache != null) {
cache.put("modelList", models);
}
return models;
}
/**
* 清除缓存
*/
public void clearCache() {
Cache embeddingCache = cacheManager.getCache(EMBEDDING_CACHE);
Cache chatCache = cacheManager.getCache(CHAT_CACHE);
Cache modelCache = cacheManager.getCache("modelListCache");
if (embeddingCache != null) {
embeddingCache.clear();
}
if (chatCache != null) {
chatCache.clear();
}
if (modelCache != null) {
modelCache.clear();
}
log.info("AI服务缓存已清除");
}
@Configuration
@EnableCaching
public static class CacheConfig {
@Bean
public CacheManager cacheManager() {
SimpleCacheManager cacheManager = new SimpleCacheManager();
List<CaffeineCache> caches = new ArrayList<>();
// 嵌入缓存:最大1000条,过期时间1小时
caches.add(new CaffeineCache(EMBEDDING_CACHE,
Caffeine.newBuilder()
.maximumSize(1000)
.expireAfterWrite(1, TimeUnit.HOURS)
.recordStats()
.build()));
// 对话缓存:最大500条,过期时间30分钟
caches.add(new CaffeineCache(CHAT_CACHE,
Caffeine.newBuilder()
.maximumSize(500)
.expireAfterWrite(30, TimeUnit.MINUTES)
.recordStats()
.build()));
// 模型列表缓存:最大10条,过期时间5分钟
caches.add(new CaffeineCache("modelListCache",
Caffeine.newBuilder()
.maximumSize(10)
.expireAfterWrite(5, TimeUnit.MINUTES)
.recordStats()
.build()));
cacheManager.setCaches(caches);
return cacheManager;
}
}
}
7.4 监控与日志
生产环境必须做好监控:
@Aspect
@Component
@Slf4j
public class AIMonitoringAspect {
private final MeterRegistry meterRegistry;
private final Counter successCounter;
private final Counter errorCounter;
private final Timer timer;
public AIMonitoringAspect(MeterRegistry meterRegistry) {
this.meterRegistry = meterRegistry;
// 初始化指标
this.successCounter = Counter.builder("ai.service.calls")
.tag("status", "success")
.description("AI服务成功调用次数")
.register(meterRegistry);
this.errorCounter = Counter.builder("ai.service.calls")
.tag("status", "error")
.description("AI服务失败调用次数")
.register(meterRegistry);
this.timer = Timer.builder("ai.service.duration")
.description("AI服务调用耗时")
.publishPercentiles(0.5, 0.95, 0.99) // 50%, 95%, 99%分位
.register(meterRegistry);
}
@Around("execution(* com.example.service.AIModelService.*(..))")
public Object monitorAICalls(ProceedingJoinPoint joinPoint) throws Throwable {
String methodName = joinPoint.getSignature().getName();
String className = joinPoint.getTarget().getClass().getSimpleName();
long startTime = System.currentTimeMillis();
Timer.Sample sample = Timer.start(meterRegistry);
try {
Object result = joinPoint.proceed();
long duration = System.currentTimeMillis() - startTime;
// 记录成功指标
successCounter.increment();
sample.stop(timer);
// 记录详细日志
if (log.isDebugEnabled()) {
log.debug("AI服务调用成功 - 类: {}, 方法: {}, 耗时: {}ms",
className, methodName, duration);
}
// 记录慢查询
if (duration > 5000) { // 超过5秒算慢查询
log.warn("AI服务慢查询 - 类: {}, 方法: {}, 耗时: {}ms",
className, methodName, duration);
}
return result;
} catch (Exception e) {
long duration = System.currentTimeMillis() - startTime;
// 记录失败指标
errorCounter.increment();
sample.stop(timer);
// 记录错误日志
log.error("AI服务调用失败 - 类: {}, 方法: {}, 耗时: {}ms, 错误: {}",
className, methodName, duration, e.getMessage(), e);
throw e;
}
}
/**
* 获取服务状态
*/
public Map<String, Object> getServiceStatus() {
Map<String, Object> status = new HashMap<>();
// 获取调用统计
double successRate = successCounter.count() /
(successCounter.count() + errorCounter.count() + 0.0001);
status.put("totalCalls", successCounter.count() + errorCounter.count());
status.put("successCalls", successCounter.count());
status.put("errorCalls", errorCounter.count());
status.put("successRate", String.format("%.2f%%", successRate * 100));
// 获取耗时统计
TimerSnapshot snapshot = timer.takeSnapshot();
status.put("avgDuration", snapshot.mean() / 1_000_000); // 转换为毫秒
status.put("p95Duration", snapshot.percentile(0.95) / 1_000_000);
status.put("p99Duration", snapshot.percentile(0.99) / 1_000_000);
status.put("maxDuration", snapshot.max() / 1_000_000);
return status;
}
}
@RestController
@RequestMapping("/api/monitor")
@Slf4j
public class MonitorController {
@Autowired
private AIMonitoringAspect monitoringAspect;
@GetMapping("/ai-status")
public ResponseEntity<Map<String, Object>> getAIStatus() {
try {
Map<String, Object> status = monitoringAspect.getServiceStatus();
status.put("timestamp", System.currentTimeMillis());
status.put("status", "healthy");
return ResponseEntity.ok(status);
} catch (Exception e) {
log.error("获取AI服务状态失败", e);
Map<String, Object> errorStatus = new HashMap<>();
errorStatus.put("timestamp", System.currentTimeMillis());
errorStatus.put("status", "error");
errorStatus.put("error", e.getMessage());
return ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR)
.body(errorStatus);
}
}
@GetMapping("/health")
public ResponseEntity<Map<String, Object>> healthCheck() {
Map<String, Object> health = new HashMap<>();
health.put("status", "UP");
health.put("timestamp", System.currentTimeMillis());
health.put("service", "xinference-integration");
// 可以添加更多健康检查项
health.put("database", "UP");
health.put("cache", "UP");
health.put("xinference", checkXinferenceHealth());
return ResponseEntity.ok(health);
}
private String checkXinferenceHealth() {
// 这里可以添加实际的健康检查逻辑
// 比如调用Xinference的health接口
return "UP";
}
}
8. 生产环境部署建议
8.1 Docker Compose部署
对于生产环境,建议用Docker Compose来管理:
# docker-compose.prod.yml
version: '3.8'
services:
# Xinference主节点
xinference-master:
image: xprobe/xinference:v1.17.1-cu129
container_name: xinference-master
command: xinference-local -H 0.0.0.0 --log-level INFO
ports:
- "9997:9997"
environment:
- XINFERENCE_MODEL_SRC=modelscope
- XINFERENCE_HEALTH_CHECK_INTERVAL=30
- XINFERENCE_LOG_LEVEL=INFO
volumes:
- xinference-models:/root/.xinference/models
- xinference-logs:/root/.xinference/logs
- ./config/models.yaml:/root/.xinference/config/models.yaml
networks:
- ai-network
restart: unless-stopped
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:9997/v1/models"]
interval: 30s
timeout: 10s
retries: 3
# Xinference工作节点(可以根据需要扩展)
xinference-worker-1:
image: xprobe/xinference:v1.17.1-cu129
container_name: xinference-worker-1
command: xinference-worker --endpoint http://xinference-master:9997
environment:
- CUDA_VISIBLE_DEVICES=0
- XINFERENCE_WORKER_NAME=worker-1
volumes:
- xinference-models:/root/.xinference/models
networks:
- ai-network
depends_on:
xinference-master:
condition: service_healthy
restart: unless-stopped
# SpringBoot应用
springboot-app:
build:
context: .
dockerfile: Dockerfile
container_name: springboot-app
ports:
- "8080:8080"
environment:
- XINFERENCE_ENDPOINT=http://xinference-master:9997
- SPRING_PROFILES_ACTIVE=prod
- JAVA_OPTS=-Xmx2g -Xms1g -XX:+UseG1GC
volumes:
- ./logs:/app/logs
- ./config/application-prod.yml:/app/config/application.yml
networks:
- ai-network
depends_on:
xinference-master:
condition: service_healthy
restart: unless-stopped
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:8080/actuator/health"]
interval: 30s
timeout: 10s
retries: 3
# Nginx负载均衡(可选)
nginx:
image: nginx:alpine
container_name: nginx-proxy
ports:
- "80:80"
- "443:443"
volumes:
- ./nginx/nginx.conf:/etc/nginx/nginx.conf
- ./nginx/ssl:/etc/nginx/ssl
networks:
- ai-network
depends_on:
- springboot-app
restart: unless-stopped
# Redis缓存
redis:
image: redis:alpine
container_name: redis-cache
ports:
- "6379:6379"
command: redis-server --appendonly yes --requirepass ${REDIS_PASSWORD}
volumes:
- redis-data:/data
networks:
- ai-network
restart: unless-stopped
# 监控:Prometheus
prometheus:
image: prom/prometheus:latest
container_name: prometheus
ports:
- "9090:9090"
volumes:
- ./monitoring/prometheus.yml:/etc/prometheus/prometheus.yml
- prometheus-data:/prometheus
networks:
- ai-network
restart: unless-stopped
# 监控:Grafana
grafana:
image: grafana/grafana:latest
container_name: grafana
ports:
- "3000:3000"
environment:
- GF_SECURITY_ADMIN_PASSWORD=${GRAFANA_PASSWORD}
- GF_INSTALL_PLUGINS=grafana-piechart-panel
volumes:
- grafana-data:/var/lib/grafana
- ./monitoring/dashboards:/etc/grafana/provisioning/dashboards
networks:
- ai-network
restart: unless-stopped
networks:
ai-network:
driver: bridge
volumes:
xinference-models:
xinference-logs:
redis-data:
prometheus-data:
grafana-data:
8.2 配置文件示例
生产环境配置文件:
# application-prod.yml
server:
port: 8080
compression:
enabled: true
mime-types: text/html,text/xml,text/plain,text/css,text/javascript,application/javascript,application/json
min-response-size: 1024
spring:
application:
name: xinference-integration
profiles:
active: prod
datasource:
url: jdbc:mysql://${DB_HOST:localhost}:3306/ai_service?useSSL=false&serverTimezone=UTC
username: ${DB_USERNAME}
password: ${DB_PASSWORD}
hikari:
maximum-pool-size: 20
minimum-idle: 5
connection-timeout: 30000
idle-timeout: 600000
max-lifetime: 1800000
redis:
host: ${REDIS_HOST:localhost}
port: ${REDIS_PORT:6379}
password: ${REDIS_PASSWORD}
lettuce:
pool:
max-active: 20
max-idle: 10
min-idle: 5
max-wait: 10000
cache:
type: redis
redis:
time-to-live: 3600000 # 1小时
cache-null-values: false
xinference:
endpoint: ${XINFERENCE_ENDPOINT:http://xinference-master:9997}
connect-timeout: 10000 # 生产环境可以设置长一些
read-timeout: 60000 # 大模型推理可能需要更长时间
max-connections: 200 # 根据实际并发调整
default-models:
chat: qwen2.5-instruct
embedding: bge-large-zh-v1.5
image: qwen-vl-chat
management:
endpoints:
web:
exposure:
include: health,info,metrics,prometheus
metrics:
export:
prometheus:
enabled: true
endpoint:
health:
show-details: always
logging:
level:
com.example: INFO
org.springframework: WARN
org.apache.http: WARN
file:
name: /app/logs/application.log
max-size: 10MB
max-history: 30
pattern:
file: "%d{yyyy-MM-dd HH:mm:ss} [%thread] %-5level %logger{36} - %msg%n"
console: "%d{yyyy-MM-dd HH:mm:ss} [%thread] %-5level %logger{36} - %msg%n"
8.3 安全配置
@Configuration
@EnableWebSecurity
public class SecurityConfig extends WebSecurityConfigurerAdapter {
@Value("${api.key.header:X-API-Key}")
private String apiKeyHeader;
@Value("${api.keys:}")
private List<String> validApiKeys;
@Override
protected void configure(HttpSecurity http) throws Exception {
http
.csrf().disable()
.authorizeRequests()
// 公开接口
.antMatchers("/api/public/**", "/actuator/health").permitAll()
// AI接口需要API密钥
.antMatchers("/api/ai/**").authenticated()
.anyRequest().authenticated()
.and()
.addFilterBefore(apiKeyFilter(), UsernamePasswordAuthenticationFilter.class)
.sessionManagement().sessionCreationPolicy(SessionCreationPolicy.STATELESS);
}
@Bean
public ApiKeyFilter apiKeyFilter() {
return new ApiKeyFilter(apiKeyHeader, validApiKeys);
}
@Bean
public FilterRegistrationBean<RateLimitFilter> rateLimitFilter() {
FilterRegistrationBean<RateLimitFilter> registration = new FilterRegistrationBean<>();
registration.setFilter(new RateLimitFilter());
registration.addUrlPatterns("/api/ai/*");
registration.setOrder(2);
return registration;
}
}
@Component
@Slf4j
public class ApiKeyFilter extends OncePerRequestFilter {
private final String apiKeyHeader;
private final Set<String> validApiKeys;
public ApiKeyFilter(String apiKeyHeader, List<String> validApiKeys) {
this.apiKeyHeader = apiKeyHeader;
this.validApiKeys = new HashSet<>(validApiKeys);
}
@Override
protected void doFilterInternal(HttpServletRequest request,
HttpServletResponse response,
FilterChain filterChain) throws ServletException, IOException {
String apiKey = request.getHeader(apiKeyHeader);
if (apiKey == null || apiKey.trim().isEmpty()) {
sendError(response, HttpServletResponse.SC_UNAUTHORIZED, "Missing API key");
return;
}
if (!validApiKeys.contains(apiKey.trim())) {
log.warn("无效的API密钥访问:{},路径:{}", apiKey, request.getRequestURI());
sendError(response, HttpServletResponse.SC_UNAUTHORIZED, "Invalid API key");
return;
}
// 设置认证信息
UsernamePasswordAuthenticationToken authentication =
new UsernamePasswordAuthenticationToken(apiKey, null, Collections.emptyList());
SecurityContextHolder.getContext().setAuthentication(authentication);
filterChain.doFilter(request, response);
}
private void sendError(HttpServletResponse response, int status, String message) throws IOException {
response.setStatus(status);
response.setContentType("application/json");
response.setCharacterEncoding("UTF-8");
Map<String, Object> error = new HashMap<>();
error.put("timestamp", System.currentTimeMillis());
error.put("status", status);
error.put("error", HttpStatus.valueOf(status).getReasonPhrase());
error.put("message", message);
error.put("path", ((HttpServletRequest) response).getRequestURI());
ObjectMapper mapper = new ObjectMapper();
response.getWriter().write(mapper.writeValueAsString(error));
}
}
@Component
@Slf4j
public class RateLimitFilter extends OncePerRequestFilter {
@Autowired
private RedisTemplate<String, Object> redisTemplate;
// 限流配置:每分钟最多100次调用
private static final int RATE_LIMIT = 100;
private static final long TIME_WINDOW = 60; // 秒
@Override
protected void doFilterInternal(HttpServletRequest request,
HttpServletResponse response,
FilterChain filterChain) throws ServletException, IOException {
String clientIp = getClientIp(request);
String apiKey = request.getHeader("X-API-Key");
String key = "rate_limit:" + (apiKey != null ? apiKey : clientIp);
Long current = redisTemplate.opsForValue().increment(key, 1);
if (current == 1) {
// 第一次设置,设置过期时间
redisTemplate.expire(key, TIME_WINDOW, TimeUnit.SECONDS);
}
if (current > RATE_LIMIT) {
log.warn("访问频率超限:{},路径:{}", key, request.getRequestURI());
sendError(response, HttpServletResponse.SC_TOO_MANY_REQUESTS, "Rate limit exceeded");
return;
}
// 添加限流头信息
response.setHeader("X-RateLimit-Limit", String.valueOf(RATE_LIMIT));
response.setHeader("X-RateLimit-Remaining", String.valueOf(Math.max(0, RATE_LIMIT - current)));
response.setHeader("X-RateLimit-Reset", String.valueOf(
redisTemplate.getExpire(key, TimeUnit.SECONDS)
));
filterChain.doFilter(request, response);
}
private String getClientIp(HttpServletRequest request) {
String ip = request.getHeader("X-Forwarded-For");
if (ip == null || ip.isEmpty() || "unknown".equalsIgnoreCase(ip)) {
ip = request.getHeader("Proxy-Client-IP");
}
if (ip == null || ip.isEmpty() || "unknown".equalsIgnoreCase(ip)) {
ip = request.getHeader("WL-Proxy-Client-IP");
}
if (ip == null || ip.isEmpty() || "unknown".equalsIgnoreCase(ip)) {
ip = request.getRemoteAddr();
}
return ip;
}
private void sendError(HttpServletResponse response, int status, String message) throws IOException {
response.setStatus(status);
response.setContentType("application/json");
response.setCharacterEncoding("UTF-8");
Map<String, Object> error = new HashMap<>();
error.put("timestamp", System.currentTimeMillis());
error.put("status", status);
error.put("error", HttpStatus.valueOf(status).getReasonPhrase());
error.put("message", message);
ObjectMapper mapper = new ObjectMapper();
response.getWriter().write(mapper.writeValueAsString(error));
}
}
9. 总结
把Xinference集成到SpringBoot项目里,技术上其实不复杂,关键是要理解微服务架构的特点,设计出合理的服务边界和调用方式。从我实际项目的经验来看,这种方案有几个明显的优势:
开发效率高:业务团队不用关心AI底层的实现细节,就像调用普通Web服务一样简单。所有的模型管理、版本控制、资源调度都交给Xinference处理,开发人员可以专注于业务逻辑。
运维方便:Xinference提供了完整的管理界面和API,模型的上线、下线、监控都很方便。而且支持分布式部署,可以根据业务压力灵活扩缩容。
性能可控:通过连接池、缓存、异步调用这些优化手段,可以很好地控制响应时间和吞吐量。监控系统也能及时发现问题,保证服务稳定性。
成本优化:Xinference支持多种推理引擎,可以根据模型特点选择最合适的引擎。还能智能调度GPU和CPU资源,让硬件利用率最大化。
当然,每个项目的需求都不一样,具体实现的时候还需要根据实际情况调整。比如有的场景对实时性要求高,可能需要更短的超时设置;有的场景数据量大,可能需要优化批量处理逻辑;有的场景对安全性要求高,可能需要更严格的访问控制。
但整体来说,Xinference-v1.17.1的稳定性和功能完整性都值得信赖,跟SpringBoot的集成也很顺畅。如果你正在考虑在微服务架构里加入AI能力,这个方案值得一试。
实际落地的时候,建议先从简单的场景开始,比如智能客服或者内容生成,等跑通了再扩展到更复杂的场景。过程中多关注监控指标,及时调整配置,确保服务稳定可靠。
获取更多AI镜像
想探索更多AI镜像和应用场景?访问 CSDN星图镜像广场,提供丰富的预置镜像,覆盖大模型推理、图像生成、视频生成、模型微调等多个领域,支持一键部署。



被折叠的 条评论
为什么被折叠?



