我正试图通过使用Flask的蓝图的API为机器学习模型提供服务,这是我的Flask\uuuu init\uuuu。py
文件
from flask import Flask
def create_app(test_config=None):
app = Flask(__name__)
@app.route("/healthcheck")
def healthcheck() -> str:
return "OK"
# Registers the machine learning blueprint
from . import ml
app.register_blueprint(ml.bp)
return app
包含/ml
endpoint蓝图的ml.py
文件
import numpy as np
from . import configuration as cfg
import tensorflow as tf
from flask import (
Blueprint, flash, request, url_for
)
bp = Blueprint("ml", __name__, url_prefix="/ml")
keras_model = None
graph = None
@bp.before_app_first_request
def load_model():
print("Loading keras model")
global keras_model
global graph
with open(cfg.config["model"]["path"], 'r') as model_file:
yaml_model = model_file.read()
keras_model = tf.keras.models.model_from_yaml(yaml_model)
graph = tf.get_default_graph()
keras_model.load_weights(cfg.config["model"]["weights"])
@bp.route('/predict', methods=['POST'])
def predict() -> str:
global graph
features = np.array([request.get_json()['features']])
print(features, len(features), features.shape)
with graph.as_default():
prediction = keras_model.predict(features)
print(prediction)
return "%.2f" % prediction
我使用命令行脚本运行服务器
#!/bin/bash
export FLASK_APP=src
export FLASK_ENV=development
flask run
如果我转到localhost:5000/healthcheck
我会得到OK
响应
curl -X POST \
http://localhost:5000/ml/predict \
-H 'Cache-Control: no-cache' \
-H 'Content-Type: application/json' \
-d '{
"features" : [17.0, 0, 0, 12.0, 1, 0, 0]
}'
第一次,我得到响应[[1.00]
,如果我再次运行它,我得到以下错误
tensorflow.python.framework.errors_impl.FailedPreconditionError:
Error while reading resource variable dense/kernel from
Container: localhost. This could mean that the variable was uninitialized.
Not found: Container localhost does not exist. (Could not find resource: localhost/dense/kernel)
[[{{node dense/MatMul/ReadVariableOp}}]]
如果我修改Blueprint文件,服务器将检测到更改并刷新它,我可以再次调用API,它将为第一次调用返回正确的结果,我将再次返回错误。为什么会发生这种情况?为什么只为第一次之后的电话?
您可以尝试创建对用于加载模型的会话的引用,然后将其设置为keras在每个请求中使用。i、 e.执行以下操作:
from tensorflow.python.keras.backend import set_session
from tensorflow.python.keras.models import load_model
tf_config = some_custom_config
sess = tf.Session(config=tf_config)
graph = tf.get_default_graph()
# IMPORTANT: models have to be loaded AFTER SETTING THE SESSION for keras!
# Otherwise, their weights will be unavailable in the threads after the session there has been set
set_session(sess)
model = load_model(...)
然后在每个请求中:
global sess
global graph
with graph.as_default():
set_session(sess)
model.predict(...)