Knowledge Base Academy Community My Tickets
Fabric 8.5
Fabric 8.5

Collecting Feedback

Agents improve fastest when the people who use them say which answers were good and which were not. AI Fusion lets users rate each agent reply with a thumbs-up or thumbs-down and attach a free-text note that explains the rating. The feedback is stored against the exact turn of the conversation, so a supervisor or QA engineer can later open the conversation, see the rated reply in context, and decide what to fix.

A thumbs-down alone tells you that something went wrong. A note such as "quoted the wrong account balance" tells you what went wrong, and saves the reviewer from re-reading the whole conversation. Encourage users to add notes, especially on negative ratings.

Where Feedback Is Collected and Shown

Surface Feedback
Preview in AI Fusion Studio Collected. Good response, Bad response and Note buttons under each reply. See Preview and Trace.
Your own front end Collected through the feedback web service. See The Feedback Web Service.
Observation Shown, read-only. The conversation review shows the rating and note of each turn, and the conversation list can be filtered by feedback.

All surfaces read and write the same records, so feedback given in Studio Preview or through your own front end appears in Observation.

Giving Feedback in Studio Preview

When you test an agent in the Preview of AI Fusion Studio, each reply has Good response, Bad response and Note buttons. The two ratings exclude each other, clicking an active rating removes it, and Note opens a box where you can save or remove a note. Developers can use this while testing to mark answers to come back to. See Preview and Trace.

Reviewing Feedback in Observation

Observation is where supervisors and QA engineers act on feedback:

  • The conversation list shows positive and negative feedback counts per conversation.
  • The Feedback filter narrows the list to Any feedback, No feedback, Has positive, Has negative, Has notes, Positive only, Negative only, or Mixed (positive + negative).
  • The conversation review shows each turn's rating, and its note behind the note icon. Feedback cannot be changed there.

See Observation Dashboard and Conversation Review.

From Feedback to a Better Agent

A practical loop for turning negative feedback into durable improvements:

  1. Find the bad answers. In Observation, filter conversations by Has negative or Has notes, and start with the ones whose notes point to a concrete error.
  2. Review in context. Open the conversation and read the turns before the rated reply. Decide whether the cause is missing data, a wrong tool result, a prompt or reasoning issue, a question outside the agent's scope, or a reasonable answer that did not match the user's expectation.
  3. Capture it as a test. Turn the conversation into an evaluation test case so that the failure is reproducible. See Snapshots and Production Import and Authoring Test Cases.
  4. Fix and verify. Change the agent, rerun the test case to confirm the fix, and run the agent's suites to check for regressions. See Running Evaluations and Results.

Because the case now lives in a suite, a later change that brings the problem back is caught by the evaluation, not by the next unhappy user.

The Feedback Web Service

Use the feedback web service of the Web Services LU to collect feedback from your own front end, for example a rating widget in a mobile app that talks to an agent through the agent web service (see Invoking Agents). The service uses the same records as Studio Preview, so feedback you submit appears in Observation like any other.

Identifying the Reply

Feedback is attached to one turn of one session:

Parameter Description
session_id The session ID you passed as sessionId to the agent web service.
message_id The turn number within the session, starting at 1: the reply to the first user message is 1, the reply to the second is 2, and so on.

Feedback Values

Parameter Values
reaction "1" for positive, "-1" for negative, "" (empty) for no rating, for example a note without a rating.
note Free text. Optional.

The user who submitted the feedback is not a parameter. The service records the user of the calling session.

Operations

Verb Parameters Use it to
POST session_id, message_id, reaction, note Create the feedback of a turn. Fails if that turn already has feedback.
PUT session_id, message_id, reaction, note Replace the rating and note of a turn that already has feedback.
DELETE session_id, message_id Remove the feedback of a turn.
GET chat_id (the session ID) Return all feedback records of the session. Returns an empty list for a session that has not produced its first turn yet.

Each turn has at most one feedback record. Studio Preview follows this pattern, which you can reuse:

  • POST when the user gives the first feedback on a turn (a rating or a note).
  • PUT when the user changes it, and still has a rating or a note.
  • DELETE when the user clears both the rating and the note.

Call the service like any other Fabric web service, with the same authentication your front end uses for the agent web service (see Web Services). For example, to rate the second reply of a session as negative with a note:

curl -X POST "https://<server>/api/feedback?format=json" \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{"session_id": "6f1c2a7e-…", "message_id": 2, "reaction": "-1", "note": "Quoted the wrong account balance"}'

A successful POST or PUT returns:

{ "STATUS": "OK" }

Invalid requests return HTTP 400, for example when the session ID does not exist, or on a POST for a turn that already has feedback.

What Is Stored

Feedback is stored in the FEEDBACK table of the session's instance of the aifusion LU, one row per rated turn:

Column Content
id A generated record ID.
step_id The turn (agent step) the feedback refers to.
reaction 1, -1 or empty.
note The note text.
creation_date, change_date When the feedback was created and last changed.
submitted_by The user who submitted or last changed it.

The GET operation returns these columns.

Collecting Feedback

Agents improve fastest when the people who use them say which answers were good and which were not. AI Fusion lets users rate each agent reply with a thumbs-up or thumbs-down and attach a free-text note that explains the rating. The feedback is stored against the exact turn of the conversation, so a supervisor or QA engineer can later open the conversation, see the rated reply in context, and decide what to fix.

A thumbs-down alone tells you that something went wrong. A note such as "quoted the wrong account balance" tells you what went wrong, and saves the reviewer from re-reading the whole conversation. Encourage users to add notes, especially on negative ratings.

Where Feedback Is Collected and Shown

Surface Feedback
Preview in AI Fusion Studio Collected. Good response, Bad response and Note buttons under each reply. See Preview and Trace.
Your own front end Collected through the feedback web service. See The Feedback Web Service.
Observation Shown, read-only. The conversation review shows the rating and note of each turn, and the conversation list can be filtered by feedback.

All surfaces read and write the same records, so feedback given in Studio Preview or through your own front end appears in Observation.

Giving Feedback in Studio Preview

When you test an agent in the Preview of AI Fusion Studio, each reply has Good response, Bad response and Note buttons. The two ratings exclude each other, clicking an active rating removes it, and Note opens a box where you can save or remove a note. Developers can use this while testing to mark answers to come back to. See Preview and Trace.

Reviewing Feedback in Observation

Observation is where supervisors and QA engineers act on feedback:

  • The conversation list shows positive and negative feedback counts per conversation.
  • The Feedback filter narrows the list to Any feedback, No feedback, Has positive, Has negative, Has notes, Positive only, Negative only, or Mixed (positive + negative).
  • The conversation review shows each turn's rating, and its note behind the note icon. Feedback cannot be changed there.

See Observation Dashboard and Conversation Review.

From Feedback to a Better Agent

A practical loop for turning negative feedback into durable improvements:

  1. Find the bad answers. In Observation, filter conversations by Has negative or Has notes, and start with the ones whose notes point to a concrete error.
  2. Review in context. Open the conversation and read the turns before the rated reply. Decide whether the cause is missing data, a wrong tool result, a prompt or reasoning issue, a question outside the agent's scope, or a reasonable answer that did not match the user's expectation.
  3. Capture it as a test. Turn the conversation into an evaluation test case so that the failure is reproducible. See Snapshots and Production Import and Authoring Test Cases.
  4. Fix and verify. Change the agent, rerun the test case to confirm the fix, and run the agent's suites to check for regressions. See Running Evaluations and Results.

Because the case now lives in a suite, a later change that brings the problem back is caught by the evaluation, not by the next unhappy user.

The Feedback Web Service

Use the feedback web service of the Web Services LU to collect feedback from your own front end, for example a rating widget in a mobile app that talks to an agent through the agent web service (see Invoking Agents). The service uses the same records as Studio Preview, so feedback you submit appears in Observation like any other.

Identifying the Reply

Feedback is attached to one turn of one session:

Parameter Description
session_id The session ID you passed as sessionId to the agent web service.
message_id The turn number within the session, starting at 1: the reply to the first user message is 1, the reply to the second is 2, and so on.

Feedback Values

Parameter Values
reaction "1" for positive, "-1" for negative, "" (empty) for no rating, for example a note without a rating.
note Free text. Optional.

The user who submitted the feedback is not a parameter. The service records the user of the calling session.

Operations

Verb Parameters Use it to
POST session_id, message_id, reaction, note Create the feedback of a turn. Fails if that turn already has feedback.
PUT session_id, message_id, reaction, note Replace the rating and note of a turn that already has feedback.
DELETE session_id, message_id Remove the feedback of a turn.
GET chat_id (the session ID) Return all feedback records of the session. Returns an empty list for a session that has not produced its first turn yet.

Each turn has at most one feedback record. Studio Preview follows this pattern, which you can reuse:

  • POST when the user gives the first feedback on a turn (a rating or a note).
  • PUT when the user changes it, and still has a rating or a note.
  • DELETE when the user clears both the rating and the note.

Call the service like any other Fabric web service, with the same authentication your front end uses for the agent web service (see Web Services). For example, to rate the second reply of a session as negative with a note:

curl -X POST "https://<server>/api/feedback?format=json" \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{"session_id": "6f1c2a7e-…", "message_id": 2, "reaction": "-1", "note": "Quoted the wrong account balance"}'

A successful POST or PUT returns:

{ "STATUS": "OK" }

Invalid requests return HTTP 400, for example when the session ID does not exist, or on a POST for a turn that already has feedback.

What Is Stored

Feedback is stored in the FEEDBACK table of the session's instance of the aifusion LU, one row per rated turn:

Column Content
id A generated record ID.
step_id The turn (agent step) the feedback refers to.
reaction 1, -1 or empty.
note The note text.
creation_date, change_date When the feedback was created and last changed.
submitted_by The user who submitted or last changed it.

The GET operation returns these columns.

© Copyright 2026 K2VIEW