> ## Documentation Index > Fetch the complete documentation index at: https://rasa.com/docs/llms.txt > Use this file to discover all available pages before exploring further. # Perform an intent evaluation > Evaluates NLU model against a model or using cross-validation. ## OpenAPI ````yaml /openapi/pro.yaml post /model/test/intents openapi: 3.0.1 info: title: Rasa - Server Endpoints version: 1.0.0 description: >- The Rasa server provides endpoints to retrieve trackers of conversations as well as endpoints to modify them. Additionally, endpoints for training and testing models are provided. servers: - url: http://localhost:5005 description: Local development server security: [] paths: /model/test/intents: post: tags: - Model summary: Perform an intent evaluation description: Evaluates NLU model against a model or using cross-validation. operationId: testModelIntent parameters: - $ref: '#/components/parameters/model' - $ref: '#/components/parameters/callback_url' - in: query name: cross_validation_folds schema: type: integer default: null description: >- Number of cross validation folds. If this parameter is specified the given training data will be used for a cross-validation instead of using it as test set for the specified model. Note that this is only supported for YAML data. requestBody: required: true content: application/x-yaml: schema: type: string description: >- NLU training data and model configuration. The model configuration is only required if cross-validation is used. example: |- nlu: - intent: greet examples: | - hey - hello - hi - intent: bye examples: | - goodbye - bye - cheers pipeline: - name: KeywordIntentClassifier application/json: schema: $ref: '#/components/schemas/RasaNLUData' responses: '200': description: NLU evaluation result content: application/json: schema: $ref: '#/components/schemas/NLUEvaluationResult' '204': $ref: '#/components/responses/204Callback' '400': $ref: '#/components/responses/400BadRequest' '401': $ref: '#/components/responses/401NotAuthenticated' '403': $ref: '#/components/responses/403NotAuthorized' '409': $ref: '#/components/responses/409Conflict' '500': $ref: '#/components/responses/500ServerError' security: - TokenAuth: [] - JWT: [] components: parameters: model: in: query name: model description: >- Model that should be used for evaluation. If the parameter is set, the model will be fetched with the currently loaded configuration setup. However, the currently loaded model will not be updated. The state of the server will not change. If the parameter is not set, the currently loaded model will be used for the evaluation. example: rasa-model.tar.gz schema: type: string required: false callback_url: in: query name: callback_url description: >- If specified the call will return immediately with an empty response and status code 204. The actual result or any errors will be sent to the given callback URL as the body of a post request. example: https://example.com/rasa_evaluations schema: type: string default: None required: false schemas: RasaNLUData: type: object properties: common_examples: type: object items: type: array items: $ref: '#/components/schemas/CommonExample' example: rasa_nlu_data: common_examples: - text: hey intent: greet entities: [] - text: dear sir intent: greet entities: [] - text: i'm looking for a place to eat intent: restaurant_search entities: [] - text: i'm looking for a place in the north of town intent: restaurant_search entities: - start: 31 end: 36 value: north entity: location - text: show me a mexican place in the centre intent: restaurant_search entities: - start: 31 end: 37 value: centre entity: location - start: 10 end: 17 value: mexican entity: cuisine NLUEvaluationResult: type: object properties: intent_evaluation: $ref: '#/components/schemas/EvaluationItem' description: Rasa NLU intent evaluation response_selection_evaluation: $ref: '#/components/schemas/EvaluationItem' description: Evaluation for the retrieval intents entity_evaluation: description: Rasa NLU entity evaluation. type: object additionalProperties: $ref: '#/components/schemas/EvaluationItem' type: object description: Evaluation for a specific extractor CommonExample: type: object properties: entities: description: Expected entities type: array items: $ref: '#/components/schemas/Entity' intent: type: string description: Intent name text: type: string description: Text of the message example: Hello! EvaluationItem: type: object description: Evaluation Result properties: report: $ref: '#/components/schemas/EvaluationReport' accuracy: type: number example: 0.19047619047619047 f1_score: type: number example: 0.06095238095238095 precision: type: number example: 0.036281179138321996 predictions: type: array description: The predictions for each item in the test set items: type: object properties: intent: type: string example: greet predicted: type: string example: greet text: type: string example: hey confidence: type: number example: 0.9973567 errors: description: The errors which were made during the testing. type: array items: oneOf: - $ref: '#/components/schemas/IntentTestError' - $ref: '#/components/schemas/EntityTestError' - $ref: '#/components/schemas/ResponseSelectorTestError' Error: type: object properties: version: type: string description: Rasa version status: type: string enum: - failure description: Status of the requested action message: type: string description: Error message reason: type: string description: Error category details: type: object description: Additional error information help: type: string description: Optional URL to additonal material code: type: number description: HTTP status code Entity: type: object description: Entities within a message properties: start: type: integer description: Char offset of the start end: type: integer description: Char offset of the end value: type: string description: Found value for entity entity: type: string description: Type of the entity confidence: type: number required: - start - end - value - entity EvaluationReport: type: object description: >- Sklearn classification report, see http://scikit-learn.org/stable/modules/generated/sklearn.metrics.classification_report.html example: greet: precision: 0.123 recall: 0.456 f1-score: 0.12 support: 100 confused_with: chitchat: 3 nlu_fallback: 5 micro avg: precision: 0.123 recall: 0.456 f1-score: 0.12 support: 100 macro avg: precision: 0.123 recall: 0.456 f1-score: 0.12 support: 100 weightedq avg: precision: 0.123 recall: 0.456 f1-score: 0.12 support: 100 IntentTestError: description: Intent prediction errors which was made during testing type: object properties: text: type: string description: Test message example: are you alright? intent_response_key_target: description: Expected intent type: string intent_response_key_prediction: $ref: '#/components/schemas/Intent' description: Predicted intent EntityTestError: description: Entity prediction errors which was made during testing type: object properties: text: type: string description: Test message example: what is the weather in zurich? entities: description: Expected entities type: array items: $ref: '#/components/schemas/Entity' predicted_entities: description: Predicted entities type: array items: $ref: '#/components/schemas/Entity' ResponseSelectorTestError: description: Error during response prediction which was made during testing type: object properties: text: type: string description: Test message example: are you alright? intent_response_key_target: description: Expected retrieval intent type: string intent_response_key_prediction: $ref: '#/components/schemas/Intent' description: Predicted retrieval intent Intent: type: object description: Intent of the text properties: confidence: type: number description: Confidence of the intent example: 0.6323 name: type: string description: Intent name example: greet required: - confidence - name responses: 204Callback: description: >- The incoming request specified a `callback_url` and hence the request will return immediately with an empty response. The actual response will be sent to the provided `callback_url` via POST request. 400BadRequest: description: Bad Request content: application/json: schema: $ref: '#/components/schemas/Error' example: version: 1.0.0 status: failure reason: BadRequest code: 400 401NotAuthenticated: description: User is not authenticated. content: application/json: schema: $ref: '#/components/schemas/Error' example: version: 1.0.0 status: failure reason: NotAuthenticated message: User is not authenticated to access resource. code: 401 403NotAuthorized: description: User has insufficient permission. content: application/json: schema: $ref: '#/components/schemas/Error' example: version: 1.0.0 status: failure reason: NotAuthorized message: User has insufficient permission to access resource. code: 403 409Conflict: description: The request conflicts with the currently loaded model. content: application/json: schema: $ref: '#/components/schemas/Error' example: version: 1.0.0 status: failure reason: Conflict message: The request conflicts with the currently loaded model. code: 409 500ServerError: description: An unexpected error occurred. content: application/json: schema: $ref: '#/components/schemas/Error' example: version: 1.0.0 status: ServerError message: An unexpected error occurred. code: 500 securitySchemes: TokenAuth: type: apiKey in: query name: token description: > A plaintext token to secure your server, specified at startup in the argument `--auth-token thisismysecret` JWT: type: http scheme: bearer bearerFormat: JWT description: > A JWT token that is signed using the JWT secret specified at startup in the argument `--jwt-secret thisismysecret`, using the `HS256` algorithm. The token's payload must contain an object under the `user` key, which in turn must contain the `username` and `role` attributes. The following is an example payload for a JWT token: ```json { "user": { "username": "", "role": "user" } } ``` If the `role` is `admin`, all endpoints are accessible. If the `role` is `user`, endpoints with a `sender_id` parameter are only accessible if the `sender_id` matches the payload's `username` property. For the user trackers endpoint (`GET /users/{user_id}/trackers`), you should use `user_id` instead of `username` in the payload. In this case, the path `user_id` parameter is matched against the payload's `username` property. For example: ```json { "user": { "username": "", "role": "user" } } ``` ````