Details
-
Suggestion
-
Status: New
-
Resolution: Unresolved
-
None
-
None
-
None
-
-
21
Description
Description
Currently, Jira Administrators do not have visibility into how users are leveraging AI powered Test Case generation features. Admins need the ability to track AI usage at a user level, including:
- Which user triggered AI-assisted test generation
- The prompt used
- Number of test cases generated per prompt
- Date/time of usage
This capability is critical for governance, auditing, usage optimization, and evaluating the value of AI components.
User friction
Administrators and managers have no way to understand:
- How widely AI features are being used
- Who is relying on AI to generate test content
- Whether prompts are appropriate and aligned with internal guidelines
- How much AI-generated content is being created over time
This leads to challenges in both accountability and reporting.
Steps to reproduce (right now) / Actual Result:
- User opens the Xray AI Test Case creation feature.
- User enters a prompt.
- AI generates test cases.
- No logs or usage history is available to Jira Admins.
Current system behavior: No audit trail, no visibility, no reporting.
IMPACT
What would improve if solved:
- Better monitoring of AI adoption and ROI
- Clear audit trail for compliance and internal reviews
- Ability to identify improper or low-quality prompts
- Improved team performance insights
- Supports governance, especially in regulated industries
Impact on stakeholders: - Admins: Gain full visibility and control over AI feature usage
- QA Leads / Managers: Can assess contribution and efficiency
- Compliance Teams: Ensure responsible usage of AI
- Developers / Product Teams: Receive insights for improving AI features
Current workaround: - Relying on self-reporting
- No automated or reliable method
- Cannot verify prompt quality or usage volumes
CONTEXT & EXAMPLES:
Concrete example:
A QA engineer generates 50 test cases using AI from a single vague prompt.
Another engineer uses multiple structured prompts to generate smaller, more accurate sets.
Admins currently cannot compare these usage patterns, identify issues, or optimize workflow.
Workaround risk:
- No audit trail → compliance and quality risk
- Over-reliance on AI without monitoring → potential content inaccuracies
- Difficulty managing usage
- Reduced transparency for team performance