LLM Applications
In recent years, LLMs (Large Language Models) have been leading a technological revolution with their remarkable capabilities in language generation and understanding, prompting many enterprises to integrate LLM applications into their businesses. While LLMs handle complex natural language processing tasks, their operations often resemble a black box, making it difficult for external understanding and validation, and challenging for enterprises to effectively monitor and optimize LLM applications.
For LLMs, observability represents understanding the model's decision-making process from an external perspective. Through Prompt input/output interaction information, Token consumption, model response time, error exceptions, Agent workflow, and user feedback, it provides deep insights into the model's internal working mechanisms.
LLM observability not only improves model transparency but also helps identify and correct potential issues promptly, ensuring model safety and accuracy, reducing errors, increasing user trust, and thus promoting the safe and sustainable development of artificial intelligence technology.
LLM Application List
The LLM application list page displays information for all created LLM applications, including application name, average latency, user count, Session count, Traces count, and total Token usage. Members with permissions can perform edit and delete operations on applications.
Overview
Clicking on an application in the "Application List" takes you to the "Overview" page by default.
The four charts at the top show trend graphs for the current application's Session count, Trace count, average Trace response time, and Token usage.
The two tables below show performance data at the Trace level: the top 10 Traces by average response time and the top 10 Traces by total Token usage. Clicking on a Trace in the table opens a new page directing to the "Trace Details" page.
Session Analysis
Sessions
Sessions List
The search box at the top allows quick search for corresponding Session data using Session ID.
The data list below shows the last access time, initial start time, duration, average response time, Traces count, and total Token usage for each Session within the selected time period. The data list is sorted by last access time in descending order by default.
Session Details
Clicking on a record in the Sessions list opens a drawer page showing Session details.
The top section displays current Session information, such as start time, duration, Session ID, Traces count, Token usage, etc.
Different tabs below show different detailed information:
- Traces: Shows the Trace data information contained in the current Session, including start time, type, response time, Token input count, Token return count, total Token usage, input content, and return content for each Trace.
- Properties: View the associated field information collected for the current Session.
Traces
Traces List
The search box at the top allows quick search for corresponding Trace data using Trace ID.
The data list below shows start time, type, response time, Token input count, Token return count, total Token usage, input content, and return content for each Trace within the selected time period. The data list is sorted by time in descending order by default.
Trace Details
Clicking on a record in the Traces list opens a drawer page showing Trace details.
The top section displays current Trace information, such as start time, response time, Trace ID, type, Token input count, Token return count, total Token usage, Session source, etc.
Different tabs below show different detailed information:
- Event Content: The left side shows input, output content, and source data contained in the current Trace; the right side shows a tree diagram of different observation items in the current Trace and their response times and Token usage.
- Waterfall: Shows the execution distribution of different observation items contained in the current Trace.
- Properties: View the associated field information collected for the current Trace.